From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6C7737EFF7 for ; Mon, 23 Mar 2026 23:58:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310321; cv=none; b=Vez1c4XR8mgWNNlG0+s1Pj3NLiCveQ+R12KXmJ3mI/+4C4CDThUXqVnrQeBQX4/jR2EMpKVr7TTAA70VgZptl+9CYAkB8MGpUKdOikAUC2SPKxtFrmL8zMwv4x582qApYz5HwE/PiwN77glY/X09SIvgYHSNwhq0oD1/BzIrtE8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310321; c=relaxed/simple; bh=sFYnTC/23rpMheriSt7XZpSxHX7Zbfc4cUo4vPsisIE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WNgyHglNnUy55JN4ZTVmU/M+M7K26OAYabfKJZwVLLkQPkbEKw9fUDQod/afz75hXFA7KeOr6/pIWaSvBwZ/eXB+VbYnUV4ta4mjcY2dVgA3QuYvzEew9eq5TxDF2el2fL0jiH6/vXKB0iyJxw2xlRDqpSLUmNNOpZXVR3fN9gE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oz480YzC; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oz480YzC" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2b064884a7cso335538755ad.3 for ; Mon, 23 Mar 2026 16:58:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310319; x=1774915119; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2Zq8TWEDbszxKidgDH4WcoGu6pRsZKEzAg4VfsbVeec=; b=oz480YzCt+c6CJEXIEK52868Ffi03PTUXOWjT25cqLhAJ7qsgq8ChKLTTPtI3aOdz2 aSrWiizprrVIp3s0kjIxDHXdYPUAxA0/FquUC/i/9Yu1arfKnXjJNeEX46j2LE9lZeX9 03q5jQHn7pkd3okd0dEBDhpiGllQFKMaFKkSwHd6p6oGdbgJmD2GZqhEFH6ZS8KIbXb9 13uTFfCCP+yQjN3iVO+4GYSrXswtwnl3txuHbMv8LLLyS0H3L5ZeYmKNSCmBqU+X0tQY 4socm9pL+L32mm4ab37j54UlWM6QMB6WuFlZ9fs0ii9mpnJ9FQ99WlnSWvBh9ql/cbSi Ef5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310319; x=1774915119; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2Zq8TWEDbszxKidgDH4WcoGu6pRsZKEzAg4VfsbVeec=; b=rtadenAFkH7hT9AVUWbNcQ506RN29TqMkvulOlvIPXJsxVcr/IVuOJh2VJddWFi9/b yi7LmBVmv9jg88GNpj95PYjUCORxXR6IVLVX8hrwzxt100elW9mDu4OGYXBz1xqYkoAi PilJYeXIib6Yq4ChzzjfmfyLX75YlhWzMZOborIrR/uWGy/Ihcs1jEh2cvSUt9CAfJMD Q7tRnUwCA9BE0pug0zUWPz/zd05hpo8qoBJPwtR7zuCONG70C2LcfPaub5ClViL57YM8 WNfH07pI+lY4WpayiLc9SsR/J2IpLbioDel+xQN6zTYT6grHeSQ/R0VFqgHgeGCEN21J sRJg== X-Forwarded-Encrypted: i=1; AJvYcCU00/VVu3wrUBYGH5kZBGNdBf2srVTSocqqjNsnk3MF9wTMSQAlu7U4a7lZ9aMVHHB4ixJPMsFctmfTLZ8=@vger.kernel.org X-Gm-Message-State: AOJu0YxFutPTYE4P+/WppwfcdZYSYqCgqoNdkEAUaEI9assD2kcKCTsI W2SxtVJpPUO81MTVIYPj/2KsZzSWKaWLQvwTaVw6+Em8vHvbrqOo1PKQTimXChzOEJ6fXYINMoG mZPFT9ascs20xvA== X-Received: from plbjz7.prod.google.com ([2002:a17:903:4307:b0:2b0:5cee:2504]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:dace:b0:2b0:62dd:3a80 with SMTP id d9443c01a7336-2b08271a32emr137995625ad.17.1774310319196; Mon, 23 Mar 2026 16:58:39 -0700 (PDT) Date: Mon, 23 Mar 2026 23:57:53 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-2-dmatlack@google.com> Subject: [PATCH v3 01/24] liveupdate: Export symbols needed by modules From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Export liveupdate_enabled(), liveupdate_register_file_handler(), and liveupdate_unregister_file_handler(). All of these will be used by vfio-pci in a subsequent commit, which can be built as a module. Reviewed-by: Samiullah Khawaja Reviewed-by: Pranjal Shrivastava Signed-off-by: David Matlack --- kernel/liveupdate/luo_core.c | 1 + kernel/liveupdate/luo_file.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/kernel/liveupdate/luo_core.c b/kernel/liveupdate/luo_core.c index dda7bb57d421..59d7793d9444 100644 --- a/kernel/liveupdate/luo_core.c +++ b/kernel/liveupdate/luo_core.c @@ -255,6 +255,7 @@ bool liveupdate_enabled(void) { return luo_global.enabled; } +EXPORT_SYMBOL_GPL(liveupdate_enabled); =20 /** * DOC: LUO ioctl Interface diff --git a/kernel/liveupdate/luo_file.c b/kernel/liveupdate/luo_file.c index a38ea4975824..cdc48d49e5e5 100644 --- a/kernel/liveupdate/luo_file.c +++ b/kernel/liveupdate/luo_file.c @@ -866,6 +866,7 @@ int liveupdate_register_file_handler(struct liveupdate_= file_handler *fh) =20 return 0; } +EXPORT_SYMBOL_GPL(liveupdate_register_file_handler); =20 /** * liveupdate_unregister_file_handler - Unregister a liveupdate file handl= er @@ -884,3 +885,4 @@ void liveupdate_unregister_file_handler(struct liveupda= te_file_handler *fh) list_del(&ACCESS_PRIVATE(fh, list)); } } +EXPORT_SYMBOL_GPL(liveupdate_unregister_file_handler); --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B922386438 for ; Mon, 23 Mar 2026 23:58:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310324; cv=none; b=CH84/TbQyyCFS+vMliH83rZj3r7281yO2/TdqvGdPRkWLwaDKJcd0TAANYRzHhU9GC8woIg0edzzWil+YSqy75a2wTqvl7czVhjFRlO2FJnt/esqye3wvUgXPj8Adwkl1njy1PeF8tmslN70ipkGAJb3PT+ENyapiRe+b30f9Dk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310324; c=relaxed/simple; bh=KBQSOWPxNbDwVbOoVizZk7AEJs1MvMPQeFpoUxsnCy0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YMCuPFBKEdv3TBR5yZ9tJJEjMbxWAz9TAYSEbdKMgXqjoiuW+Ks0TTcaglUKMHBipyO6lPpHRNcRD3RUncJdm/ptSVDgNgVAyJCJO8zDghZm34QRw+RXgsR/OkQpfcbIjmIQ0HeMuDH2BeD+CUPKoVBrnCO+7SfzEj1VXL+swmM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=je/N5G5a; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="je/N5G5a" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2aebfa0af7dso77255675ad.1 for ; Mon, 23 Mar 2026 16:58:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310321; x=1774915121; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uXuqbjp18TTblR4dtTQV0wcbuwX6U0fGWW/v6G8ddek=; b=je/N5G5a4ir2IjUfCfNxBT93KnI6Van7kJVF+st6bpJznnnqoTNxGUlJ27qW4PEZxB WNlMlRFp7iN77vYmBuwf7J8NB9Ikc4w8qD6YdYplEZDNqGPN8utUwhbFPpdE61O4BCeu +q+xEqoP0ghJ/zUaX6zWS6KREOnGpZ/d6yvpOoPtGzncw1U3feG7Nk7QlDdx6QxTDOL7 x8Cpg5CPN5cZl9gaw/tbunpuwUsWm16GN+kQPM4X+u/zfGi+ZgCNmLdSkdBgbLfUpJik tdCP/CzkvjGMkbneA8CLgA/qBe7gdrYeDWNUHAOSq1xFq1AKV6xpQ3ke6AObRN8WhEe1 dd2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310321; x=1774915121; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uXuqbjp18TTblR4dtTQV0wcbuwX6U0fGWW/v6G8ddek=; b=HRKL8tMOG6TpQ6droS4WiltcaHPIqhz6Y+HBJe+mI9U44yotU2gws1klwiBnY29jY0 0gq5VaHe+13odC/yFqFZebxzfUen4OyPRaHDbz87OVQLpH/1nmnRmFRMMJqKfV/fNDLC QB+x97WE9m4bVESOp8oDKYZBbuV/JqiAdEhcDJuNvqCW3SYSdmhiO+VgSdnYXB4OzI26 g+StAK8twICQDcloBzMcsy24w9JhfDsrOudF7WydtA2dCMYL/ocw6MZxdScQTmLuMkEx Wp6DtkC4R9XvC+/64G3uAQR59RjPScACqc3euBIn0KnwO9x2dILaHiu8zEmk47y3sI29 Sppw== X-Forwarded-Encrypted: i=1; AJvYcCWq4fOopQSKqcwKIcWccI1K0mDXSPvDC562r/p8e+i0E1GCGYdjgvxWMVzmNqXlo5+d7A2LzzFWdqzXJFU=@vger.kernel.org X-Gm-Message-State: AOJu0Ywped0yGycH2IqmB6Qtxo8ph3qLb2bEraKBFCMm9aVFhVHO/KJW 0h4x5OqArdTgrH9eUo8Ei83oDH5xv8FPAy6LOd+eGEKKZymI9BwAbZ7ppDHg1W/AXAsQEZpm4QA ilR34g7Ey7kTGCw== X-Received: from pleo1.prod.google.com ([2002:a17:903:2101:b0:2b0:5a4b:5d45]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:3890:b0:2b0:6638:b357 with SMTP id d9443c01a7336-2b0826c751dmr140143485ad.11.1774310320820; Mon, 23 Mar 2026 16:58:40 -0700 (PDT) Date: Mon, 23 Mar 2026 23:57:54 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-3-dmatlack@google.com> Subject: [PATCH v3 02/24] PCI: Add API to track PCI devices preserved across Live Update From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an API to enable the PCI subsystem to participate in a Live Update and track all devices that are being preserved by drivers. Since this support is still under development, hide it behind a new Kconfig PCI_LIVEUPDATE that is marked experimental. This API will be used in subsequent commits by the vfio-pci driver to preserve VFIO devices across Live Update. Signed-off-by: David Matlack --- drivers/pci/Kconfig | 11 ++ drivers/pci/Makefile | 1 + drivers/pci/liveupdate.c | 380 ++++++++++++++++++++++++++++++++++++ drivers/pci/pci.h | 14 ++ drivers/pci/probe.c | 2 + include/linux/kho/abi/pci.h | 62 ++++++ include/linux/pci.h | 41 ++++ 7 files changed, 511 insertions(+) create mode 100644 drivers/pci/liveupdate.c create mode 100644 include/linux/kho/abi/pci.h diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index e3f848ffb52a..05307d89c3f4 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -334,6 +334,17 @@ config VGA_ARB_MAX_GPUS Reserves space in the kernel to maintain resource locking for multiple GPUS. The overhead for each GPU is very small. =20 +config PCI_LIVEUPDATE + bool "PCI Live Update Support (EXPERIMENTAL)" + depends on PCI && LIVEUPDATE + help + Support for preserving PCI devices across a Live Update. This option + should only be enabled by developers working on implementing this + support. Once enough support as landed in the kernel, this option + will no longer be marked EXPERIMENTAL. + + If unsure, say N. + source "drivers/pci/hotplug/Kconfig" source "drivers/pci/controller/Kconfig" source "drivers/pci/endpoint/Kconfig" diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 41ebc3b9a518..e8d003cb6757 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -16,6 +16,7 @@ obj-$(CONFIG_PROC_FS) +=3D proc.o obj-$(CONFIG_SYSFS) +=3D pci-sysfs.o slot.o obj-$(CONFIG_ACPI) +=3D pci-acpi.o obj-$(CONFIG_GENERIC_PCI_IOMAP) +=3D iomap.o +obj-$(CONFIG_PCI_LIVEUPDATE) +=3D liveupdate.o endif =20 obj-$(CONFIG_OF) +=3D of.o diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c new file mode 100644 index 000000000000..bec7b3500057 --- /dev/null +++ b/drivers/pci/liveupdate.c @@ -0,0 +1,380 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright (c) 2026, Google LLC. + * David Matlack + */ + +/** + * DOC: PCI Live Update + * + * The PCI subsystem participates in the Live Update process to enable dri= vers + * to preserve their PCI devices across kexec. + * + * Device preservation across Live Update is built on top of the Live Upda= te + * Orchestrator (LUO) support for file preservation across kexec. Userspace + * indicates that a device should be preserved by preserving the file asso= ciated + * with the device with ``ioctl(LIVEUPDATE_SESSION_PRESERVE_FD)``. + * + * .. note:: + * The support for preserving PCI devices across Live Update is current= ly + * *partial* and should be considered *experimental*. It should only be + * used by developers working on the implementation for the time being. + * + * To enable the support, enable ``CONFIG_PCI_LIVEUPDATE``. + * + * Driver API + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * Drivers that support file-based device preservation must register their + * ``liveupdate_file_handler`` with the PCI subsystem by calling + * ``pci_liveupdate_register_flb()``. This ensures the PCI subsystem will = be + * notified whenever a device file is preserved so that ``struct pci_ser`` + * can be allocated to track all preserved devices. This struct is an ABI + * and is eventually handed off to the next kernel via Kexec-Handover (KHO= ). + * + * In the "outgoing" kernel (before kexec), drivers should then notify the= PCI + * subsystem directly whenever the preservation status for a device change= s: + * + * * ``pci_liveupdate_preserve(pci_dev)``: The device is being preserved. + * + * * ``pci_liveupdate_unpreserve(pci_dev)``: The device is no longer being + * preserved (preservation is cancelled). + * + * In the "incoming" kernel (after kexec), drivers should notify the PCI + * subsystem with the following calls: + * + * * ``pci_liveupdate_retrieve(pci_dev)``: The device file is being retri= eved + * by userspace. + * + * * ``pci_liveupdate_finish(pci_dev)``: The device is done participating= in + * Live Update. After this point the device may no longer be even assoc= iated + * with the same driver. + * + * Incoming/Outgoing + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * The state of each device's participation in Live Update is stored in + * ``struct pci_dev``: + * + * * ``liveupdate_outgoing``: True if the device is being preserved in the + * outgoing kernel. Set in ``pci_liveupdate_preserve()`` and cleared in + * ``pci_liveupdate_unpreserve()``. + * + * * ``liveupdate_incoming``: True if the device is preserved in the inco= ming + * kernel. Set during probing when the device is first created and clea= red + * in ``pci_liveupdate_finish()``. + * + * Restrictions + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * Preserved devices currently have the following restrictions. Each of th= ese + * may be relaxed in the future. + * + * * The device must not be a Virtual Function (VF). + * + * * The device must not be a Physical Function (PF). + * + * Preservation Behavior + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * The kernel preserves the following state for devices preserved across a= Live + * Update: + * + * * The PCI Segment, Bus, Device, and Function numbers assigned to the d= evice + * are guaranteed to remain the same across Live Update. + * + * This list will be extended in the future as new support is added. + * + * Driver Binding + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * It is the driver's responsibility for ensuring that preserved devices a= re not + * released or bound to a different driver for as long as they are preserv= ed. In + * practice, this is enforced by LUO taking an extra referenced to the pre= served + * device file for as long as it is preserved. + * + * However, there is a window of time in the incoming kernel when a device= is + * first probed and when userspace retrieves the device file with + * ``LIVEUPDATE_SESSION_RETRIEVE_FD`` when the device could be bound to any + * driver. + * + * It is currently userspace's responsibility to ensure that the device is= bound + * to the correct driver in this window. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "pci.h" + +static DEFINE_MUTEX(pci_flb_outgoing_lock); + +static int pci_flb_preserve(struct liveupdate_flb_op_args *args) +{ + struct pci_dev *dev =3D NULL; + int max_nr_devices =3D 0; + struct pci_ser *ser; + unsigned long size; + + /* + * Don't both accounting for VFs that could be created after this + * since preserving VFs is not supported yet. Also don't account + * for devices that could be hot-plugged after this since preserving + * hot-plugged devices across Live Update is not yet an expected + * use-case. + */ + for_each_pci_dev(dev) + max_nr_devices++; + + size =3D struct_size_t(struct pci_ser, devices, max_nr_devices); + + ser =3D kho_alloc_preserve(size); + if (IS_ERR(ser)) + return PTR_ERR(ser); + + ser->max_nr_devices =3D max_nr_devices; + + args->obj =3D ser; + args->data =3D virt_to_phys(ser); + return 0; +} + +static void pci_flb_unpreserve(struct liveupdate_flb_op_args *args) +{ + struct pci_ser *ser =3D args->obj; + + WARN_ON_ONCE(ser->nr_devices); + kho_unpreserve_free(ser); +} + +static int pci_flb_retrieve(struct liveupdate_flb_op_args *args) +{ + args->obj =3D phys_to_virt(args->data); + return 0; +} + +static void pci_flb_finish(struct liveupdate_flb_op_args *args) +{ + kho_restore_free(args->obj); +} + +static struct liveupdate_flb_ops pci_liveupdate_flb_ops =3D { + .preserve =3D pci_flb_preserve, + .unpreserve =3D pci_flb_unpreserve, + .retrieve =3D pci_flb_retrieve, + .finish =3D pci_flb_finish, + .owner =3D THIS_MODULE, +}; + +static struct liveupdate_flb pci_liveupdate_flb =3D { + .ops =3D &pci_liveupdate_flb_ops, + .compatible =3D PCI_LUO_FLB_COMPATIBLE, +}; + +#define INIT_PCI_DEV_SER(_dev) { \ + .domain =3D pci_domain_nr((_dev)->bus), \ + .bdf =3D pci_dev_id(_dev), \ +} + +static int pci_dev_ser_cmp(const void *__a, const void *__b) +{ + const struct pci_dev_ser *a =3D __a, *b =3D __b; + + return cmp_int((u64)a->domain << 16 | a->bdf, + (u64)b->domain << 16 | b->bdf); +} + +static struct pci_dev_ser *pci_ser_find(struct pci_ser *ser, + struct pci_dev *dev) +{ + const struct pci_dev_ser key =3D INIT_PCI_DEV_SER(dev); + + return bsearch(&key, ser->devices, ser->nr_devices, + sizeof(key), pci_dev_ser_cmp); +} + +static void pci_ser_delete(struct pci_ser *ser, struct pci_dev *dev) +{ + struct pci_dev_ser *dev_ser; + int i; + + dev_ser =3D pci_ser_find(ser, dev); + + /* + * This should never happen unless there is a kernel bug or + * corruption that causes the state in struct pci_ser to get + * out of sync with struct pci_dev. + */ + if (pci_WARN_ONCE(dev, !dev_ser, "Cannot find preserved device!")) + return; + + for (i =3D dev_ser - ser->devices; i < ser->nr_devices - 1; i++) + ser->devices[i] =3D ser->devices[i + 1]; + + ser->nr_devices--; +} + +int pci_liveupdate_preserve(struct pci_dev *dev) +{ + struct pci_dev_ser new =3D INIT_PCI_DEV_SER(dev); + struct pci_ser *ser; + int i, ret; + + /* SR-IOV is not supported yet. */ + if (dev->is_virtfn || dev->is_physfn) + return -EINVAL; + + guard(mutex)(&pci_flb_outgoing_lock); + + if (dev->liveupdate_outgoing) + return -EBUSY; + + ret =3D liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser); + if (ret) + return ret; + + if (ser->nr_devices =3D=3D ser->max_nr_devices) + return -E2BIG; + + for (i =3D ser->nr_devices; i > 0; i--) { + struct pci_dev_ser *prev =3D &ser->devices[i - 1]; + int cmp =3D pci_dev_ser_cmp(&new, prev); + + /* + * This should never happen unless there is a kernel bug or + * corruption that causes the state in struct pci_ser to get out + * of sync with struct pci_dev. + */ + if (WARN_ON_ONCE(!cmp)) + return -EBUSY; + + if (cmp > 0) + break; + + ser->devices[i] =3D *prev; + } + + ser->devices[i] =3D new; + ser->nr_devices++; + dev->liveupdate_outgoing =3D true; + return 0; +} +EXPORT_SYMBOL_GPL(pci_liveupdate_preserve); + +void pci_liveupdate_unpreserve(struct pci_dev *dev) +{ + struct pci_ser *ser; + int ret; + + /* This should never happen unless the caller (driver) is buggy */ + if (WARN_ON_ONCE(!dev->liveupdate_outgoing)) + return; + + guard(mutex)(&pci_flb_outgoing_lock); + + ret =3D liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser); + + /* This should never happen unless there is a bug in LUO */ + if (WARN_ON_ONCE(ret)) + return; + + pci_ser_delete(ser, dev); + dev->liveupdate_outgoing =3D false; +} +EXPORT_SYMBOL_GPL(pci_liveupdate_unpreserve); + +static int pci_liveupdate_flb_get_incoming(struct pci_ser **serp) +{ + int ret; + + ret =3D liveupdate_flb_get_incoming(&pci_liveupdate_flb, (void **)serp); + + /* Live Update is not enabled. */ + if (ret =3D=3D -EOPNOTSUPP) + return ret; + + /* Live Update is enabled, but there is no incoming FLB data. */ + if (ret =3D=3D -ENODATA) + return ret; + + /* + * Live Update is enabled and there is incoming FLB data, but none of it + * matches pci_liveupdate_flb.compatible. + * + * This could mean that no PCI FLB data was passed by the previous + * kernel, but it could also mean the previous kernel used a different + * compatibility string (i.e.a different ABI). The latter deserves at + * least a WARN_ON_ONCE() but it cannot be distinguished from the + * former. + */ + if (ret =3D=3D -ENOENT) { + pr_info_once("PCI: No incoming FLB data detected during Live Update"); + return ret; + } + + /* + * There is incoming FLB data that matches pci_liveupdate_flb.compatible + * but it cannot be retrieved. Proceed with standard initialization as + * if there was not incoming PCI FLB data. + */ + WARN_ONCE(ret, "PCI: Failed to retrieve incoming FLB data during Live Upd= ate"); + return ret; +} + +u32 pci_liveupdate_incoming_nr_devices(void) +{ + struct pci_ser *ser; + + if (pci_liveupdate_flb_get_incoming(&ser)) + return 0; + + return ser->nr_devices; +} + +void pci_liveupdate_setup_device(struct pci_dev *dev) +{ + struct pci_ser *ser; + + if (pci_liveupdate_flb_get_incoming(&ser)) + return; + + if (!pci_ser_find(ser, dev)) + return; + + dev->liveupdate_incoming =3D true; +} + +int pci_liveupdate_retrieve(struct pci_dev *dev) +{ + if (!dev->liveupdate_incoming) + return -EINVAL; + + return 0; +} +EXPORT_SYMBOL_GPL(pci_liveupdate_retrieve); + +void pci_liveupdate_finish(struct pci_dev *dev) +{ + dev->liveupdate_incoming =3D false; +} +EXPORT_SYMBOL_GPL(pci_liveupdate_finish); + +int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh) +{ + return liveupdate_register_flb(fh, &pci_liveupdate_flb); +} +EXPORT_SYMBOL_GPL(pci_liveupdate_register_flb); + +void pci_liveupdate_unregister_flb(struct liveupdate_file_handler *fh) +{ + liveupdate_unregister_flb(fh, &pci_liveupdate_flb); +} +EXPORT_SYMBOL_GPL(pci_liveupdate_unregister_flb); diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 13d998fbacce..979cb9921340 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -1434,4 +1434,18 @@ static inline int pci_msix_write_tph_tag(struct pci_= dev *pdev, unsigned int inde (PCI_CONF1_ADDRESS(bus, dev, func, reg) | \ PCI_CONF1_EXT_REG(reg)) =20 +#ifdef CONFIG_PCI_LIVEUPDATE +void pci_liveupdate_setup_device(struct pci_dev *dev); +u32 pci_liveupdate_incoming_nr_devices(void); +#else +static inline void pci_liveupdate_setup_device(struct pci_dev *dev) +{ +} + +static inline u32 pci_liveupdate_incoming_nr_devices(void) +{ + return 0; +} +#endif + #endif /* DRIVERS_PCI_H */ diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index bccc7a4bdd79..c60222d45659 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2064,6 +2064,8 @@ int pci_setup_device(struct pci_dev *dev) if (pci_early_dump) early_dump_pci_device(dev); =20 + pci_liveupdate_setup_device(dev); + /* Need to have dev->class ready */ dev->cfg_size =3D pci_cfg_space_size(dev); =20 diff --git a/include/linux/kho/abi/pci.h b/include/linux/kho/abi/pci.h new file mode 100644 index 000000000000..7764795f6818 --- /dev/null +++ b/include/linux/kho/abi/pci.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Copyright (c) 2025, Google LLC. + * David Matlack + */ + +#ifndef _LINUX_KHO_ABI_PCI_H +#define _LINUX_KHO_ABI_PCI_H + +#include +#include +#include + +/** + * DOC: PCI File-Lifecycle Bound (FLB) Live Update ABI + * + * This header defines the ABI for preserving core PCI state across kexec = using + * Live Update File-Lifecycle Bound (FLB) data. + * + * This interface is a contract. Any modification to any of the serializat= ion + * structs defined here constitutes a breaking change. Such changes require + * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE string. + */ + +#define PCI_LUO_FLB_COMPATIBLE "pci-v1" + +/** + * struct pci_dev_ser - Serialized state about a single PCI device. + * + * @domain: The device's PCI domain number (segment). + * @bdf: The device's PCI bus, device, and function number. + * @reserved: Reserved (to naturally align struct pci_dev_ser). + */ +struct pci_dev_ser { + u32 domain; + u16 bdf; + u16 reserved; +} __packed; + +/** + * struct pci_ser - PCI Subsystem Live Update State + * + * This struct tracks state about all devices that are being preserved acr= oss + * a Live Update for the next kernel. + * + * @max_nr_devices: The length of the devices[] flexible array. + * @nr_devices: The number of devices that were preserved. + * @devices: Flexible array of pci_dev_ser structs for each device. Guaran= teed + * to be sorted ascending by domain and bdf. + */ +struct pci_ser { + u64 max_nr_devices; + u64 nr_devices; + struct pci_dev_ser devices[]; +} __packed; + +/* Ensure all elements of devices[] are naturally aligned. */ +static_assert(offsetof(struct pci_ser, devices) % sizeof(unsigned long) = =3D=3D 0); +static_assert(sizeof(struct pci_dev_ser) % sizeof(unsigned long) =3D=3D 0); + +#endif /* _LINUX_KHO_ABI_PCI_H */ diff --git a/include/linux/pci.h b/include/linux/pci.h index 1c270f1d5123..27ee9846a2fd 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -40,6 +40,7 @@ #include #include #include +#include =20 #include =20 @@ -591,6 +592,10 @@ struct pci_dev { u8 tph_mode; /* TPH mode */ u8 tph_req_type; /* TPH requester type */ #endif +#ifdef CONFIG_PCI_LIVEUPDATE + unsigned int liveupdate_incoming:1; /* Preserved by previous kernel */ + unsigned int liveupdate_outgoing:1; /* Preserved for next kernel */ +#endif }; =20 static inline struct pci_dev *pci_physfn(struct pci_dev *dev) @@ -2871,4 +2876,40 @@ void pci_uevent_ers(struct pci_dev *pdev, enum pci_= ers_result err_type); WARN_ONCE(condition, "%s %s: " fmt, \ dev_driver_string(&(pdev)->dev), pci_name(pdev), ##arg) =20 +#ifdef CONFIG_PCI_LIVEUPDATE +int pci_liveupdate_preserve(struct pci_dev *dev); +void pci_liveupdate_unpreserve(struct pci_dev *dev); +int pci_liveupdate_retrieve(struct pci_dev *dev); +void pci_liveupdate_finish(struct pci_dev *dev); +int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh); +void pci_liveupdate_unregister_flb(struct liveupdate_file_handler *fh); +#else +static inline int pci_liveupdate_preserve(struct pci_dev *dev) +{ + return -EOPNOTSUPP; +} + +static inline void pci_liveupdate_unpreserve(struct pci_dev *dev) +{ +} + +static inline int pci_liveupdate_retrieve(struct pci_dev *dev) +{ + return -EOPNOTSUPP; +} + +static inline void pci_liveupdate_finish(struct pci_dev *dev) +{ +} + +static inline int pci_liveupdate_register_flb(struct liveupdate_file_handl= er *fh) +{ + return -EOPNOTSUPP; +} + +static inline void pci_liveupdate_unregister_flb(struct liveupdate_file_ha= ndler *fh) +{ +} +#endif + #endif /* LINUX_PCI_H */ --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 549E53859CE for ; Mon, 23 Mar 2026 23:58:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310325; cv=none; b=uK52YJy+5NtmXLTjH40uOpnnj9KDR4Qj2euwK15LfnDPHGaQxujyVvcWqDaVReQ+PNtsWDPHwPpe7CTXn4U26fiSTJxdFbcqI1bfXL3yKrbHNr36cDOq/bZWSiHXkTvKXjSMChqSq7hCMOt6P55xN68GtbAOX0LdoupXS+roOC0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310325; c=relaxed/simple; bh=vLDgiBBZd58h0T0ReKqdykl5DlgchHSnpVYqQ6ykqu8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ihzvpoow4fIXQPGWSBxFcTjy1LhnizQgG3ITht5gYlD6t0XkGXKhF2P4l4rxIOgmmS2+IzLAzVTScJRpDgP3qvsRWSZKffVwClugJyVB9wVs6KI25D5IAp8tcZ9HZ9EhJRQ+s/gYCuXvTWk+fFuzSmF3UphKkdnuj1mBsPTmPuI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=F1gkYkqP; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="F1gkYkqP" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2b056b2f0cfso8989565ad.0 for ; Mon, 23 Mar 2026 16:58:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310323; x=1774915123; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Gw5NKF4E66vyohb8n/pa3diZrC0/+CJM8Skon/TvWYk=; b=F1gkYkqPc+XdbjhbEZ9eztV6hD5/Rd6mu4MK2/mGOdF+e97YHQIKpaKbenq82XSibi RxDRMPgvRENeXLq8GmvoRQPUY73un6Q/Fcb/sJsjhgH/imGRygfZsL60y0maJPtIeF9u iS1QzMZ7xzBL/tHsROAf+j7U9GqftIp/slvyx/m+U1Y1cfIg+CDuK+N5P4oseShqs65J uvSFfTzxUi0u8H3qaymMi6nT8dOdWM43zYNyCHhlKvPimNxhJ85Vbg97BjS8V/7BHz0s U49yfXPQO4CN5glC9UZFv+4lb3mMMX3RdXiFliIpkWXKJ4aU5ejVeZlOyQdobWqwEM4F wwPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310323; x=1774915123; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Gw5NKF4E66vyohb8n/pa3diZrC0/+CJM8Skon/TvWYk=; b=n5gzdtAoIFbGtgYxs2+lmJyz57BObKUTgfDPUdZQrRbt7o1tyGR9mVyVSd8w/A+iPZ AJIuGue1r1h62j+luD9LSgFpCN9/RsdJK5Tdger134/ldJUD21j73r/4ZFc0ySolz0hp SSDjT21lZnoRbFD1pWJo/7iYv/JyvJXTZk5L5zBJ8Sou6pYOgyScmTCAfW6rFdV9aWkh f7Dw7msjxPZZKZvWQ23vJS5Lh4WzSzox9P/EWdHiMzNWH/+GhYa14Z6c/vK571NpaVJL VxnmB4+utUZGj8sqWXA/5UXcG1+HPqHVue4MWD0whfCID6vv4GXkMx4eBRBmtZvL7znP 9TuA== X-Forwarded-Encrypted: i=1; AJvYcCUcsJAuv/yQCLhv9iEbhjOqBiMxAZsaItnx7Q71aVWklzog0BI8Y0dexpAd1VoAQyGkJ99o/gptCUYVXKw=@vger.kernel.org X-Gm-Message-State: AOJu0YyW5uiev8ufbHWOCz5hA86VYGf4rb2kXIcIenXY5ODQaOyN0Kn8 zxtq7pHo61mesDgqJeHn2OxAYhDBbv4Z120EqYpiPBqtCJ96SIQJw+Vs4MNs8joIOes5ZH++wjU HYKP9V3oErpdwmg== X-Received: from plar5.prod.google.com ([2002:a17:902:c7c5:b0:2b0:5a1f:4fdf]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:d507:b0:2b0:5795:9ead with SMTP id d9443c01a7336-2b0825e35e3mr130027435ad.0.1774310322445; Mon, 23 Mar 2026 16:58:42 -0700 (PDT) Date: Mon, 23 Mar 2026 23:57:55 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-4-dmatlack@google.com> Subject: [PATCH v3 03/24] PCI: Require Live Update preserved devices are in singleton iommu_groups From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Require that Live Update preserved devices are in singleton iommu_groups during preservation (outgoing kernel) and retrieval (incoming kernel). PCI devices preserved across Live Update will be allowed to perform memory transactions throughout the Live Update. Thus IOMMU groups for preserved devices must remain fixed. Since all current use cases for Live Update are for PCI devices in singleton iommu_groups, require that as a starting point. This avoids the complexity of needing to enforce arbitrary iommu_group topologies while still allowing all current use cases. Suggested-by: Jason Gunthorpe Signed-off-by: David Matlack --- drivers/pci/liveupdate.c | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c index bec7b3500057..a3dbe06650ff 100644 --- a/drivers/pci/liveupdate.c +++ b/drivers/pci/liveupdate.c @@ -75,6 +75,8 @@ * * * The device must not be a Physical Function (PF). * + * * The device must be the only device in its IOMMU group. + * * Preservation Behavior * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * @@ -105,6 +107,7 @@ =20 #include #include +#include #include #include #include @@ -222,6 +225,31 @@ static void pci_ser_delete(struct pci_ser *ser, struct= pci_dev *dev) ser->nr_devices--; } =20 +static int count_devices(struct device *dev, void *__nr_devices) +{ + (*(int *)__nr_devices)++; + return 0; +} + +static int pci_liveupdate_validate_iommu_group(struct pci_dev *dev) +{ + struct iommu_group *group; + int nr_devices =3D 0; + + group =3D iommu_group_get(&dev->dev); + if (group) { + iommu_group_for_each_dev(group, &nr_devices, count_devices); + iommu_group_put(group); + } + + if (nr_devices !=3D 1) { + pci_warn(dev, "Live Update preserved devices must be in singleton iommu = groups!"); + return -EINVAL; + } + + return 0; +} + int pci_liveupdate_preserve(struct pci_dev *dev) { struct pci_dev_ser new =3D INIT_PCI_DEV_SER(dev); @@ -232,6 +260,10 @@ int pci_liveupdate_preserve(struct pci_dev *dev) if (dev->is_virtfn || dev->is_physfn) return -EINVAL; =20 + ret =3D pci_liveupdate_validate_iommu_group(dev); + if (ret) + return ret; + guard(mutex)(&pci_flb_outgoing_lock); =20 if (dev->liveupdate_outgoing) @@ -357,7 +389,7 @@ int pci_liveupdate_retrieve(struct pci_dev *dev) if (!dev->liveupdate_incoming) return -EINVAL; =20 - return 0; + return pci_liveupdate_validate_iommu_group(dev); } EXPORT_SYMBOL_GPL(pci_liveupdate_retrieve); =20 --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0AF403876CE for ; Mon, 23 Mar 2026 23:58:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310329; cv=none; b=qVBsfE+4s4M73QtsZYQMvwcMwBmoz5+bC65jgr2PU3SStASGJ0GhBFJPA9l9xAFtaGNvLBFT44lm/ustf99P7bkEQaW5DZWJLo2x4TRwGL4e77xT89nAqDlfXZ4lo7YFpY3w8tBHFB8NUX7iACqwchxg1mR7eaZsi5XZfht+LjM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310329; c=relaxed/simple; bh=Xh0VD6c2DOiBYyNXtlX2CiZhzQ4nziyD7BKX5GcXGCo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Bo5jAGGP9ZNyfoF6UKZWknIng2s3OiKlOXYFM57mKTyaHLm882LhETpml1XMqkO8Mujk3osmbpcMn+xeQCPr4wqa7rBl2Yr9ou2Sn/G8gYBK/8ASdc8RFzxQUiypNfjZeGREJ1B2Pu5gPcpL/8JBAaENzyYGfWYmkC+XHaXq+HI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=o5GR4Kwg; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="o5GR4Kwg" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2b061868724so3401175ad.1 for ; Mon, 23 Mar 2026 16:58:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310324; x=1774915124; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=g7UW0RmcLc1SB8VXNrTH33hzd4ZkDSbd/G4mL4V5lhA=; b=o5GR4KwglfgJGcj5bp7gGsk36lVS0yOcgt+DzXm3gJOZxuYR7i9lYKXiw1RZjz3kIX qAL0rQkLoIc6YXKRWBa3MG6X+5Ayd8QUsy86K/SeHd++O95BxLnU0G9nwgdU8vY3lqEk 6/2gzybg5r8tm/V8LDDpMjy4awdshqFuuTB40RM23Jknh5XRhZO8rAz5jM68ClgO+D7v jC1lDzWsMhuczOeLAmY15eLgaHzL4U/02phpopjJKZtBLEClstBkpIWwmRuR41obhDRG YcuKmHKdWv03ogegnVpR6ud+js3rIjRtCSiSI1PhGV3GPB9ghf4SgbMNgRp5hP4k3wci raTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310324; x=1774915124; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=g7UW0RmcLc1SB8VXNrTH33hzd4ZkDSbd/G4mL4V5lhA=; b=lzukBlVHqHdF13QpaDjudAI8+eb0/i+54v5u7s2g3mSWMPWxewgaNGOA2TRCSi1hTd gkenFjzdwtH6HJHF8p/kpRlrjtMIlef8V7bp9iyQE073wwUyk7rpKOrw5SsA2gA5wa86 Rd+q2WoI40iDSp3DWonQSnwL8RIIND+ttbFrJrQCnyOunIbMFx+SyXdh82gXop/MdfyI /Q/rlbNSNUCDDMntWpqKvdCp1VYzZVyMS94mEQAU4FthlQ90vKrKpSg7fH9MsNw63Yyk 1GzgoUZhiK+Ss9A4elSGtfWkgAp1GtK/C0wiFVoGlDTjEMmXYXb4Hj/x5fR5MsKGLVUp Oq2A== X-Forwarded-Encrypted: i=1; AJvYcCVkgrwFxXzbBltHudNX2u4cGtwYnLYmsjPIgToc5FARENyYH7HRxDwI3rOIQfwnqFhnGPiucqCIZSeRJKo=@vger.kernel.org X-Gm-Message-State: AOJu0YzQD6alFOMq36XAd64CZ7ruWwJNVMBBayfARPO4mWN8Ij9RC+eJ dVgs29/Z7rD3z/lxoPEQ15IfTu3IBmIIqW1IpRtltQKcYDuxBfS8cJef7VaqeWvmbnsTpqI+yCU rRM8XgTGJ5RkxFA== X-Received: from plbjx15.prod.google.com ([2002:a17:903:138f:b0:2b0:5b78:8e47]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e54a:b0:2b0:9183:ce1e with SMTP id d9443c01a7336-2b09183d6ddmr75835585ad.30.1774310324185; Mon, 23 Mar 2026 16:58:44 -0700 (PDT) Date: Mon, 23 Mar 2026 23:57:56 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-5-dmatlack@google.com> Subject: [PATCH v3 04/24] PCI: Inherit bus numbers from previous kernel during Live Update From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Inherit bus numbers from the previous kernel during a Live Update when one or more PCI devices are being preserved, even if pci=3Dassign-busses is enabled. During a Live Update, preserved devices will be allowed to continue performing memory transactions. Thus the kernel cannot change the fabric topology, including changing bus numbers, since that would requiring disabling and flushing any memory transactions first. So if pci=3Dassign-busses is enabled, ignore it during the Live Update and inherit all bus numbers assigned by the previous kernel. This will not break users that rely on pci=3Dassign-busses for their system to function correctly since the system can be assumed to be in a functional state already if a Live Update is underway. In other words, pci=3Dassign-busses would establish a functional topology during the initial cold boot, and then that topology would remain fixed across any subsequent Live Updates. Signed-off-by: David Matlack --- .../admin-guide/kernel-parameters.txt | 6 +++- drivers/pci/liveupdate.c | 5 ++- drivers/pci/probe.c | 35 ++++++++++++++++--- 3 files changed, 40 insertions(+), 6 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index 03a550630644..beff9f3f8e3b 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5156,7 +5156,11 @@ Kernel parameters explicitly which ones they are. assign-busses [X86] Always assign all PCI bus numbers ourselves, overriding - whatever the firmware may have done. + whatever the firmware may have done. Ignored + during a Live Update, where the kernel must + inherit the PCI topology (including bus numbers) + to avoid interrupting ongoing memory + transactions of preserved devices. usepirqmask [X86] Honor the possible IRQ mask stored in the BIOS $PIR table. This is needed on some systems with broken BIOSes, notably diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c index a3dbe06650ff..c1251f4f8438 100644 --- a/drivers/pci/liveupdate.c +++ b/drivers/pci/liveupdate.c @@ -84,7 +84,10 @@ * Update: * * * The PCI Segment, Bus, Device, and Function numbers assigned to the d= evice - * are guaranteed to remain the same across Live Update. + * are guaranteed to remain the same across Live Update. Note that this= is + * true even if pci=3Dassign-busses is set on the command line. The ker= nel will + * always inherit bus numbers assigned by the previous kernel during a = Live + * Update. * * This list will be extended in the future as new support is added. * diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index c60222d45659..165056d71e66 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1369,6 +1369,34 @@ bool pci_ea_fixed_busnrs(struct pci_dev *dev, u8 *se= c, u8 *sub) return true; } =20 +static bool pci_assign_all_busses(void) +{ + if (!pcibios_assign_all_busses()) + return false; + + /* + * During a Live Update, preserved devices are are allowed to continue + * performing memory transactions. Thus the kernel cannot change the + * fabric topology, including changing bus numbers, since that would + * requiring disabling and flushing any memory transactions first. + * + * So if pci=3Dassign-busses is enabled, ignore it during the Live Update + * and inherit all bus numbers assigned by the previous kernel. This + * will not break users that rely on pci=3Dassign-busses for their system + * to function correctly since the system can be assumed to be in a + * functional state already if a Live Update is underway. In other + * words, pci=3Dassign-busses should be used to establish working bus + * numbers during the initial cold boot, and then that topology would + * then remain fixed across any subsequent Live Updates. + */ + if (pci_liveupdate_incoming_nr_devices()) { + pr_info_once("Ignoring pci=3Dassign-busses and inheriting bus numbers du= ring Live Update\n"); + return false; + } + + return true; +} + /* * pci_scan_bridge_extend() - Scan buses behind a bridge * @bus: Parent bus the bridge is on @@ -1396,6 +1424,7 @@ static int pci_scan_bridge_extend(struct pci_bus *bus= , struct pci_dev *dev, int max, unsigned int available_buses, int pass) { + const bool assign_all_busses =3D pci_assign_all_busses(); struct pci_bus *child; u32 buses; u16 bctl; @@ -1448,8 +1477,7 @@ static int pci_scan_bridge_extend(struct pci_bus *bus= , struct pci_dev *dev, goto out; } =20 - if ((secondary || subordinate) && - !pcibios_assign_all_busses() && !broken) { + if ((secondary || subordinate) && !assign_all_busses && !broken) { unsigned int cmax, buses; =20 /* @@ -1491,8 +1519,7 @@ static int pci_scan_bridge_extend(struct pci_bus *bus= , struct pci_dev *dev, * do in the second pass. */ if (!pass) { - if (pcibios_assign_all_busses() || broken) - + if (assign_all_busses || broken) /* * Temporarily disable forwarding of the * configuration cycles on all bridges in --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B00FB385508 for ; Mon, 23 Mar 2026 23:58:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310328; cv=none; b=r1KWUakygNBLJn1buu9j4uJ9bNj+p1uOZqaACAXYVlDYXXKCfnrx0j7chcsOb7ZTSctzG4Y0jaYTY6G6jrlhoEECbn4VGO/m+5FT+/3lPY+AuA6gIHQniVd09t8OyFnAoLzgTfgv2Q3oP5xtgRWzg0A/tFd5dZoALBvD3CmC8I0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310328; c=relaxed/simple; bh=nufixlrxKDA9RxLJCcKgJiD/jb0wnKOqEaMV67UGvLw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RlWYdQLg8OjAdtMXMpslTPJuSw4aL/5bYXaECRacOZJEOBtonMTVb1hTsjrdGVpr7vCwLlZSBhwARvYSW1qZYkdPeuVjDfG4nGCDQ+qBf0fLLOsYkh21D6Qve04iiLvp+N8u8D1cP2iGKQ/4kKtyrsN9Q+h3IXm5uUTX0iWh/jc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cmziZylm; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cmziZylm" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-358df8fbd1cso1042959a91.0 for ; Mon, 23 Mar 2026 16:58:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310326; x=1774915126; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Z81MLxHKunKSpfd9G8Gazc1UStxzf4ZSRozQrfTx8r0=; b=cmziZylmbR22KBlTJw3YBAD0h4KEiYTt9tBnAaZDYK37JZj0W6BZyiNKKc9M3n7I5r /WHbloHMOkUlcII14sI7xSLS05UQBchd6XAG+EvifXbxp4mUSH6FGvpCeef4YkYJUtjT lgJx1Vm593wZZaOxauARkMjMgMWvaAwjiEiPfRBD8RtAmPrmSLv3TYRfkzFuLtyEPyIq FgAF0zDle4ohJPldBYPAibP7B1s1tv20mQ4QDNQPL/nazzyUHuF2urlQMN0iH7TbNIzb FDO9Pze8JlqjHWUpv2Xt+M5cVeJS7OQjjr+WZP4WOPcXhsYY1VNek09kjKM6dTUxKvdq ZSTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310326; x=1774915126; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Z81MLxHKunKSpfd9G8Gazc1UStxzf4ZSRozQrfTx8r0=; b=avFu0h0xEcgDkKrC55cyWaCnymoL50qGMdoyhULn8v+YxL0BkVxYQv4yTRGMgR4Ole G6DuWdSAwzzuwHt8lD1i7epWd3xEIPPlF6Yle0KpOXSTJdiq9vVsmLh3UqunvyGQibEE h/2auS3biIO/vJGrHMtZGh9pz6YFR6ptm4Rj0dlzc0JkOXvcW8L4ApRx2yzs18T00X38 cVaE02Jt1rj5FTrMSPMdHHjl/YeFX8m4yKGh4VbcrkPB+EYSoTGChNuNPWggqFwCFlvE 0GW0tmtrENdE/WXuKcodrZl8UEFwWHZbxo+dIBxtFe8MwyeRYz9hlF8NToq97HWtycuX YheA== X-Forwarded-Encrypted: i=1; AJvYcCVrGC5XLGuy0CYRw2oHQKouK9Bvdyj3e32qUH3R46iRKg3PEXgsiiPdMbzc11+/GF+YMGq+w9wsagFQ790=@vger.kernel.org X-Gm-Message-State: AOJu0YxvXpMOdkdtC8fXt2WmAo4sBVkhJcHO6rUzEpzmH/dfxNJK+kRi qNCOpry4kCdFVTdAYUilD0EQ+VXrGWNgHovfF3GcOgBKvtVhTgfTa67dh69Wc2A5W6mdJW88/qo cdiu6L7DmhpWpAw== X-Received: from pjbfs13.prod.google.com ([2002:a17:90a:f28d:b0:359:979d:cee5]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5390:b0:35a:275c:4c27 with SMTP id 98e67ed59e1d1-35bd2bba27cmr10391028a91.2.1774310325767; Mon, 23 Mar 2026 16:58:45 -0700 (PDT) Date: Mon, 23 Mar 2026 23:57:57 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-6-dmatlack@google.com> Subject: [PATCH v3 05/24] docs: liveupdate: Add documentation for PCI From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add documentation files for the PCI subsystem's participation in Live Update, generated from the kernel-doc comments the code. Signed-off-by: David Matlack --- Documentation/PCI/liveupdate.rst | 23 +++++++++++++++++++++++ Documentation/core-api/liveupdate.rst | 1 + 2 files changed, 24 insertions(+) create mode 100644 Documentation/PCI/liveupdate.rst diff --git a/Documentation/PCI/liveupdate.rst b/Documentation/PCI/liveupdat= e.rst new file mode 100644 index 000000000000..04c9b675e8df --- /dev/null +++ b/Documentation/PCI/liveupdate.rst @@ -0,0 +1,23 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D +PCI Support for Live Update +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D + +.. kernel-doc:: drivers/pci/liveupdate.c + :doc: PCI Live Update + +PCI Preservation ABI +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. kernel-doc:: include/linux/kho/abi/pci.h + :doc: PCI File-Lifecycle Bound (FLB) Live Update ABI + +.. kernel-doc:: include/linux/kho/abi/pci.h + :internal: + +See Also +=3D=3D=3D=3D=3D=3D=3D=3D + + * :doc:`/core-api/liveupdate` + * :doc:`/core-api/kho/index` diff --git a/Documentation/core-api/liveupdate.rst b/Documentation/core-api= /liveupdate.rst index 5a292d0f3706..d56a7760978a 100644 --- a/Documentation/core-api/liveupdate.rst +++ b/Documentation/core-api/liveupdate.rst @@ -70,3 +70,4 @@ See Also =20 - :doc:`Live Update uAPI ` - :doc:`/core-api/kho/index` +- :doc:`PCI ` --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87B9E38838C for ; Mon, 23 Mar 2026 23:58:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310330; cv=none; b=eUDCdPag1L9EmkAtldE+qCLFKqlaVkCAPlxITD2Xoj8dCBf+rTHvy6mHc7PUqv0qp8Su+2MfNoru+/uTXTD/hdT11I6HYf6USwgm6hy9ysnPWiF1pT983lzYAYQecfJHjEklzt5gGz2iOx5pH2YmUhhriWefd40SUYLiovvSSNE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310330; c=relaxed/simple; bh=l00LIHmX2FmWkYpSjP6Q+peOSbjX0cWvYczRtB6/PPc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=p/XAz6qAIHX72Gm7dcraZSsozKKk7wJe4mmXEsbVAiT4OzqnG/xAy8xXL0WVK5ODHFm8JYsh+cg1+k+gCUZKojGDa6oSVLnDyMjOfP9yhH9DPXK7wKfBPg1VQ87vaG77tLp7RzzdpYy9rjyaIubdNegsJA1I5ZxjfI10Vkbnt/U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hIg1nEzq; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hIg1nEzq" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-35a032cdd78so2910332a91.1 for ; Mon, 23 Mar 2026 16:58:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310328; x=1774915128; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=YCGcH5x5HW+NRzEKEkZIp2gqKlZo/LGIoKmQ3sM/JLo=; b=hIg1nEzq82bKlCD8cGLW1/bMAov6oOdwDux59+GdjIx8RqTysB6T+jrl0B95VUknVV 4lYrJ2YnYqK+txBL8gaIXpBZ+GN1C0LMFd54OJiE36GQn8znAQrOmAkaAM8Jj4lir0Gs NG7BfovrRadhImYowOKg6Sfi2D8SmIDBXG7EPP7dCJZnc004J4AUvN2HlwEqADgGOIwd 3BuXaIAuVhRb8/8MWWQp9m827fs9UCVweyAyErS616OsWAJ5niWsSWnJM5cNhSHh8Kup DS6oSwNEhxNV+X/9swNSqWqJ+Dm+VjRDnjKUj7BZ/7o3VpR+hfDzpAgin6edngvqGz32 lX/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310328; x=1774915128; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=YCGcH5x5HW+NRzEKEkZIp2gqKlZo/LGIoKmQ3sM/JLo=; b=Q+eS/bzv3o3DVeqyZApE9r/9aCBvLsqxxyy82+PUxQECQWJSWJ/qN4ZPNhN9UaLW7O DycoXBZm33rrOn+UfLE1VeIrzBOuUaakxwfUA2XnufYgomFZHco8CnPdVfzvSviK8D9W IkVNE3YnXUonTaEBzXKJGKuy5xUXcT6spJTNqvgznMudlNuCvBEG7+/2LQuvQ5cedngn NzZaVNcN7dr/djWz3jIv//DvNYwMwy2rAzk+Ry384bdFPoaOYs7KUgLPoBx1oto3TKde qux52JoyOEiozmBJhzPcudQXT4NgxiVtQiwaEMoiNUX+xG5mEc2XlyI/jY0ll7ATzzTt eJwg== X-Forwarded-Encrypted: i=1; AJvYcCXCTz+Zg18LQ5TNO2Nn3qlcqLljQ/9gw3vO036cXyiLRGwH3Gy4cW8qKooKuUS1QaL/KzAHfydj4qbh1UM=@vger.kernel.org X-Gm-Message-State: AOJu0YxywIySw6KIutjZDTuNNYVGKvAWQpjpPIaENDaHJFttHAc3JZwV oNRlvaUlY3bt9eFsZbxVCrpZ6QwfwYmis9IkCgBwpr1aiTc/OTLtff79rJ4wsS6fmfRwkot8Dux EaJQm1jckcDaYng== X-Received: from pjvp18.prod.google.com ([2002:a17:90a:df92:b0:35b:9822:9f79]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1a8a:b0:359:f43d:4a6e with SMTP id 98e67ed59e1d1-35bd2996301mr12903584a91.0.1774310327594; Mon, 23 Mar 2026 16:58:47 -0700 (PDT) Date: Mon, 23 Mar 2026 23:57:58 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-7-dmatlack@google.com> Subject: [PATCH v3 06/24] vfio/pci: Register a file handler with Live Update Orchestrator From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Register a live update file handler for vfio-pci device files. Add stub implementations of all required callbacks so that registration does not fail (i.e. to avoid breaking git-bisect). This file handler will be extended in subsequent commits to enable a device bound to vfio-pci to run without interruption while the host is going through a kexec Live Update. Put this support behind a new Kconfig VFIO_PCI_LIVEUPDATE that is marked experimental and default-disabled until more of the device preservation support has landed in the kernel. Signed-off-by: Vipin Sharma Co-developed-by: David Matlack Signed-off-by: David Matlack --- MAINTAINERS | 1 + drivers/vfio/pci/Kconfig | 11 ++++ drivers/vfio/pci/Makefile | 1 + drivers/vfio/pci/vfio_pci.c | 12 ++++- drivers/vfio/pci/vfio_pci_liveupdate.c | 69 ++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 14 ++++++ include/linux/kho/abi/vfio_pci.h | 28 +++++++++++ 7 files changed, 135 insertions(+), 1 deletion(-) create mode 100644 drivers/vfio/pci/vfio_pci_liveupdate.c create mode 100644 include/linux/kho/abi/vfio_pci.h diff --git a/MAINTAINERS b/MAINTAINERS index 96ea84948d76..a16a7ecc67a4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -27685,6 +27685,7 @@ F: Documentation/ABI/testing/debugfs-vfio F: Documentation/ABI/testing/sysfs-devices-vfio-dev F: Documentation/driver-api/vfio.rst F: drivers/vfio/ +F: include/linux/kho/abi/vfio_pci.h F: include/linux/vfio.h F: include/linux/vfio_pci_core.h F: include/uapi/linux/vfio.h diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index 1e82b44bda1a..8f087f7b58c3 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -58,6 +58,17 @@ config VFIO_PCI_ZDEV_KVM config VFIO_PCI_DMABUF def_bool y if VFIO_PCI_CORE && PCI_P2PDMA && DMA_SHARED_BUFFER =20 +config VFIO_PCI_LIVEUPDATE + bool "VFIO PCI support for Live Update (EXPERIMENTAL)" + depends on VFIO_PCI && PCI_LIVEUPDATE + help + Support for preserving devices bound to vfio-pci across a Live + Update. This option should only be enabled by developers working on + implementing this support. Once enough support has landed in the + kernel, this option will no longer be marked EXPERIMENTAL. + + If you don't know what to do here, say N. + source "drivers/vfio/pci/mlx5/Kconfig" =20 source "drivers/vfio/pci/hisilicon/Kconfig" diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index e0a0757dd1d2..f462df61edb9 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_VFIO_PCI_CORE) +=3D vfio-pci-core.o =20 vfio-pci-y :=3D vfio_pci.o vfio-pci-$(CONFIG_VFIO_PCI_IGD) +=3D vfio_pci_igd.o +vfio-pci-$(CONFIG_VFIO_PCI_LIVEUPDATE) +=3D vfio_pci_liveupdate.o obj-$(CONFIG_VFIO_PCI) +=3D vfio-pci.o =20 obj-$(CONFIG_MLX5_VFIO_PCI) +=3D mlx5/ diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 0c771064c0b8..41dcbe4ace67 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -170,6 +170,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const s= truct pci_device_id *id) ret =3D vfio_pci_core_register_device(vdev); if (ret) goto out_put_vdev; + return 0; =20 out_put_vdev: @@ -264,10 +265,14 @@ static int __init vfio_pci_init(void) =20 vfio_pci_core_set_params(nointxmask, is_disable_vga, disable_idle_d3); =20 + ret =3D vfio_pci_liveupdate_init(); + if (ret) + return ret; + /* Register and scan for devices */ ret =3D pci_register_driver(&vfio_pci_driver); if (ret) - return ret; + goto err_liveupdate_cleanup; =20 vfio_pci_fill_ids(); =20 @@ -275,12 +280,17 @@ static int __init vfio_pci_init(void) pr_warn("device denylist disabled.\n"); =20 return 0; + +err_liveupdate_cleanup: + vfio_pci_liveupdate_cleanup(); + return ret; } module_init(vfio_pci_init); =20 static void __exit vfio_pci_cleanup(void) { pci_unregister_driver(&vfio_pci_driver); + vfio_pci_liveupdate_cleanup(); } module_exit(vfio_pci_cleanup); =20 diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c new file mode 100644 index 000000000000..5ea5af46b159 --- /dev/null +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright (c) 2026, Google LLC. + * Vipin Sharma + * David Matlack + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include + +#include "vfio_pci_priv.h" + +static bool vfio_pci_liveupdate_can_preserve(struct liveupdate_file_handle= r *handler, + struct file *file) +{ + return false; +} + +static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *ar= gs) +{ + return -EOPNOTSUPP; +} + +static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args = *args) +{ +} + +static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) +{ + return -EOPNOTSUPP; +} + +static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) +{ +} + +static const struct liveupdate_file_ops vfio_pci_liveupdate_file_ops =3D { + .can_preserve =3D vfio_pci_liveupdate_can_preserve, + .preserve =3D vfio_pci_liveupdate_preserve, + .unpreserve =3D vfio_pci_liveupdate_unpreserve, + .retrieve =3D vfio_pci_liveupdate_retrieve, + .finish =3D vfio_pci_liveupdate_finish, + .owner =3D THIS_MODULE, +}; + +static struct liveupdate_file_handler vfio_pci_liveupdate_fh =3D { + .ops =3D &vfio_pci_liveupdate_file_ops, + .compatible =3D VFIO_PCI_LUO_FH_COMPATIBLE, +}; + +int __init vfio_pci_liveupdate_init(void) +{ + int ret; + + ret =3D liveupdate_register_file_handler(&vfio_pci_liveupdate_fh); + if (ret && ret !=3D -EOPNOTSUPP) + return ret; + + return 0; +} + +void vfio_pci_liveupdate_cleanup(void) +{ + liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); +} diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index 27ac280f00b9..cbf46e09da30 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -133,4 +133,18 @@ static inline void vfio_pci_dma_buf_move(struct vfio_p= ci_core_device *vdev, } #endif =20 +#ifdef CONFIG_VFIO_PCI_LIVEUPDATE +int __init vfio_pci_liveupdate_init(void); +void vfio_pci_liveupdate_cleanup(void); +#else +static inline int vfio_pci_liveupdate_init(void) +{ + return 0; +} + +static inline void vfio_pci_liveupdate_cleanup(void) +{ +} +#endif /* CONFIG_VFIO_PCI_LIVEUPDATE */ + #endif diff --git a/include/linux/kho/abi/vfio_pci.h b/include/linux/kho/abi/vfio_= pci.h new file mode 100644 index 000000000000..e2412b455e61 --- /dev/null +++ b/include/linux/kho/abi/vfio_pci.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Copyright (c) 2025, Google LLC. + * Vipin Sharma + * David Matlack + */ + +#ifndef _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H +#define _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H + +/** + * DOC: VFIO PCI Live Update ABI + * + * VFIO uses the ABI defined below for preserving device files across a ke= xec + * reboot using LUO. + * + * Device metadata is serialized into memory which is then handed to the n= ext + * kernel via KHO. + * + * This interface is a contract. Any modification to any of the serializat= ion + * structs defined here constitutes a breaking change. Such changes require + * incrementing the version number in the VFIO_PCI_LUO_FH_COMPATIBLE strin= g. + */ + +#define VFIO_PCI_LUO_FH_COMPATIBLE "vfio-pci-v1" + +#endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */ --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7CAA83890F6 for ; Mon, 23 Mar 2026 23:58:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310333; cv=none; b=U3l0noJfr84dChYIulYoXoOjnJP68BnfL63MPpa512oeKtBgeoeDZmt5Mj7/i98j7U1fi1xMswvNPDwwI/9+8XmPe4nlSoOWAp3kHeh2sWI0TmNzlpMPJEh2BfBBt+D/EX40pwknfOIqGgXXJydyXgpvj0CvC8lvtfKXxieqELU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310333; c=relaxed/simple; bh=jQ2xJOyJci43vfVGmUKEJiEGXMgXbDfzFbOvAg1Duyc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZR/nJZHfbXsgs16GqS/LA+655fDuK74zdFvH/xTgtYZqNx/f28pi63vjeT82yK4FlzCPyqdmWJRN54DVzPWeBtEObeQqxg6UK1+6+AxNJu+RIYN6Buzn+5qCGQrd0gLL/25L5gZejSBgbGbcrfiaMdCBFaqUidMvuD69/n6lTdc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PgMKiSMI; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PgMKiSMI" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-35b977ffbcdso744352a91.3 for ; Mon, 23 Mar 2026 16:58:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310330; x=1774915130; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=g7yaJ9779yd/8dNUYbHNz1geAYfqq+6PqCMDWCMvl5g=; b=PgMKiSMIy8wa4S+6q7F7P4qXEPA1DCm61+ffUc0h+mhYB9V7FT8uzjqtm0uhXdZYNh MeZ5a6OkOh0V+Mzbti8ZnBJCT8JBQR9rcvDgcvHteapwe2+F0AbRRH0Lt9mK8Qo21kpJ lu58Z8IXvwPWv2fAST/0ASFikDmhuzYJRfu5RVDBEsiR0wMdI6Y9uY1e+MZtw4JhYrbA UOSf27M5zpWhx7JnipvAKgWbCyjCSLRRoytw1CFyA6tDqrrFHKcLjMWBuePLpQevygHh 7HTSNN2MTuzbIhdi9psNIyO6l2DTL7D5b/PD9e5HxBz7bp45myfq+AK9SDdzmaZVYmAN 1J6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310330; x=1774915130; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=g7yaJ9779yd/8dNUYbHNz1geAYfqq+6PqCMDWCMvl5g=; b=UYz9Cbvw4K/lJH+t1cyEm2y+IgtLCfwmrR+Od7eEEIwQrCJdxYUYbY+73cp7OZ3eqc N022lWK19EmsFmfUob/ofB9r50+NwvhBD3mVlwgAVcnQ8tvZtZWE/+HDNKwiIiWG9FQ3 tAhH082UT2o3/mQdzuxsX4Q/0E42D8Hz7Y3XXOdAtwHV/1YKeZ3a4630lhZMxWkUCZDM BHQfPc5PVcKcJPx7omH0FknRImEeoOT5fD/sQdyfRey9eKaiBwsFhc4nxTrn4aef2N70 Gn8AkqzxFkx9mEbBrhXAHIpeG/1WQLshJzYiQFgkiQ3GKN08kALICL/bCLdtDMg7UDu8 v2hQ== X-Forwarded-Encrypted: i=1; AJvYcCWizGcldXu5o98ZsN4Wk/mvyi5ad18+YYhq6YABfXHEkwIkZxKXDYaeRyOTL5HDHdsmIt+6pKc1MKo9qKg=@vger.kernel.org X-Gm-Message-State: AOJu0YwKO/1bTHNy/BD0QdzpqzBRMekixWLVg1b+B9S0pjHSJypOb2Kj sRd7hXWUPrp4PaK8r8nLmSbtcw55s/r/8ysPAhUIEoZcRxnN5pl4Ss20pEkPj0pMo5NPpnHe4xO iChepwE5wTK+O6w== X-Received: from pjbiq1.prod.google.com ([2002:a17:90a:fb41:b0:359:f249:13d5]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4acb:b0:35b:a30f:8bf9 with SMTP id 98e67ed59e1d1-35bd2bf4cccmr12732243a91.13.1774310329456; Mon, 23 Mar 2026 16:58:49 -0700 (PDT) Date: Mon, 23 Mar 2026 23:57:59 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-8-dmatlack@google.com> Subject: [PATCH v3 07/24] vfio/pci: Preserve vfio-pci device files across Live Update From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Implement the live update file handler callbacks to preserve a vfio-pci device across a Live Update. Subsequent commits will enable userspace to then retrieve this file after the Live Update. Live Update support is scoped only to cdev files (i.e. not VFIO_GROUP_GET_DEVICE_FD files). State about each device is serialized into a new ABI struct vfio_pci_core_device_ser. The contents of this struct are preserved across the Live Update to the next kernel using a combination of Kexec-Handover (KHO) to preserve the page(s) holding the struct and the Live Update Orchestrator (LUO) to preserve the physical address of the struct. For now the only contents of struct vfio_pci_core_device_ser the device's PCI segment number and BDF, so that the device can be uniquely identified after the Live Update. Require that userspace disables interrupts on the device prior to freeze() so that the device does not send any interrupts until new interrupt handlers have been set up by the next kernel. Reset the device and restore its state in the freeze() callback. This ensures the device can be received by the next kernel in a consistent state. Eventually this will be dropped and the device can be preserved across in a running state, but that requires further work in VFIO and the core PCI layer. Note that LUO holds a reference to this file when it is preserved. So VFIO is guaranteed that vfio_df_device_last_close() will not be called on this device no matter what userspace does. Signed-off-by: Vipin Sharma Co-developed-by: David Matlack Signed-off-by: David Matlack --- drivers/vfio/pci/vfio_pci.c | 2 +- drivers/vfio/pci/vfio_pci_core.c | 57 +++++---- drivers/vfio/pci/vfio_pci_liveupdate.c | 156 ++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_priv.h | 4 + drivers/vfio/vfio_main.c | 3 +- include/linux/kho/abi/vfio_pci.h | 15 +++ include/linux/vfio.h | 2 + 7 files changed, 213 insertions(+), 26 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 41dcbe4ace67..351480d13f6e 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -125,7 +125,7 @@ static int vfio_pci_open_device(struct vfio_device *cor= e_vdev) return 0; } =20 -static const struct vfio_device_ops vfio_pci_ops =3D { +const struct vfio_device_ops vfio_pci_ops =3D { .name =3D "vfio-pci", .init =3D vfio_pci_core_init_dev, .release =3D vfio_pci_core_release_dev, diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index d43745fe4c84..81f941323641 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -585,9 +585,42 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *= vdev) } EXPORT_SYMBOL_GPL(vfio_pci_core_enable); =20 +void vfio_pci_core_try_reset(struct vfio_pci_core_device *vdev) +{ + struct pci_dev *pdev =3D vdev->pdev; + struct pci_dev *bridge =3D pci_upstream_bridge(pdev); + + lockdep_assert_held(&vdev->vdev.dev_set->lock); + + if (!vdev->reset_works) + return; + + /* + * Try to get the locks ourselves to prevent a deadlock. The + * success of this is dependent on being able to lock the device, + * which is not always possible. + * + * We cannot use the "try" reset interface here, since that will + * overwrite the previously restored configuration information. + */ + if (bridge && !pci_dev_trylock(bridge)) + return; + + if (!pci_dev_trylock(pdev)) + goto out; + + if (!__pci_reset_function_locked(pdev)) + vdev->needs_reset =3D false; + + pci_dev_unlock(pdev); +out: + if (bridge) + pci_dev_unlock(bridge); +} +EXPORT_SYMBOL_GPL(vfio_pci_core_try_reset); + void vfio_pci_core_disable(struct vfio_pci_core_device *vdev) { - struct pci_dev *bridge; struct pci_dev *pdev =3D vdev->pdev; struct vfio_pci_dummy_resource *dummy_res, *tmp; struct vfio_pci_ioeventfd *ioeventfd, *ioeventfd_tmp; @@ -687,27 +720,7 @@ void vfio_pci_core_disable(struct vfio_pci_core_device= *vdev) */ pci_write_config_word(pdev, PCI_COMMAND, PCI_COMMAND_INTX_DISABLE); =20 - /* - * Try to get the locks ourselves to prevent a deadlock. The - * success of this is dependent on being able to lock the device, - * which is not always possible. - * We can not use the "try" reset interface here, which will - * overwrite the previously restored configuration information. - */ - if (vdev->reset_works) { - bridge =3D pci_upstream_bridge(pdev); - if (bridge && !pci_dev_trylock(bridge)) - goto out_restore_state; - if (pci_dev_trylock(pdev)) { - if (!__pci_reset_function_locked(pdev)) - vdev->needs_reset =3D false; - pci_dev_unlock(pdev); - } - if (bridge) - pci_dev_unlock(bridge); - } - -out_restore_state: + vfio_pci_core_try_reset(vdev); pci_restore_state(pdev); out: pci_disable_device(pdev); diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 5ea5af46b159..c4ebc7c486e5 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -6,27 +6,178 @@ * David Matlack */ =20 +/** + * DOC: VFIO PCI Preservation via LUO + * + * VFIO PCI devices can be preserved over a kexec using the Live Update + * Orchestrator (LUO) file preservation. This allows userspace (such as a = VMM) + * to transfer an in-use device to the next kernel. + * + * .. note:: + * The support for preserving VFIO PCI devices is currently *partial* a= nd + * should be considered *experimental*. It should only be used by devel= opers + * working on expanding the support for the time being. + * + * To avoid accidental usage while the support is still experimental, t= his + * support is hidden behind a default-disable config option + * ``CONFIG_VFIO_PCI_LIVEUPDATE``. Once the kernel support has stabiliz= ed and + * become complete, this option will be enabled by default when + * ``CONFIG_VFIO_PCI`` and ``CONFIG_LIVEUPDATE`` are enabled. + * + * Usage Example + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * VFIO PCI devices can be preserved across a kexec by preserving the file + * associated with the device in a LUO session:: + * + * device_fd =3D open("/dev/vfio/devices/X"); + * ... + * ioctl(session_fd, LIVEUPDATE_SESSION_PRESERVE_FD, { ..., device_fd, .= ..}); + * + * .. note:: + * LUO will hold an extra reference to the device file for as long as i= t is + * preserved, so there is no way for the file to be destroyed or the de= vice + * to be unbound from the vfio-pci driver while it is preserved. + * + * Retrieving the file after kexec is not yet supported. + * + * Restrictions + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * The kernel imposes the following restrictions when preserving VFIO devi= ces: + * + * * The device must be bound to the ``vfio-pci`` driver. + * + * * ``CONFIG_VFIO_PCI_ZDEV_KVM`` must not be enabled. This may be relaxe= d in + * the future. + * + * * The device not be an Intel display device. This may be relaxed in the + * future. + * + * * The device file must have been acquired from the VFIO character devi= ce, + * not ``VFIO_GROUP_GET_DEVICE_FD``. + * + * * The device must have interrupt disable prior to kexec. Failure to di= sable + * interrupts on the device will cause the ``reboot(LINUX_REBOOT_CMD_KE= XEC)`` + * syscall (to initiate the kexec) to fail. + * + * Preservation Behavior + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * The eventual goal of this support is to avoid disrupting the workload, = state, + * or configuration of each preserved device during a Live Update. This wo= uld + * include allowing the device to perform DMA to preserved memory buffers = and + * perform P2P DMA to other preserved devices. However, there are many pie= ces + * that still need to land in the kernel. + * + * For now, VFIO only preserves the following state for for devices: + * + * * The PCI Segment, Bus, Device, and Function numbers of the device. The + * kernel guarantees the these will not change across a kexec when a de= vice + * is preserved. + * + * Since the kernel is not yet prepared to preserve all parts of the devic= e and + * its dependencies (such as DMA mappings), VFIO currently resets and rest= ores + * preserved devices back into an idle state during kexec, before handing = off + * control to the next kernel. This will be relaxed in future versions of = the + * kernel once it is safe to allow the device to keep running across kexec. + */ + #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt =20 +#include #include #include #include +#include =20 #include "vfio_pci_priv.h" =20 static bool vfio_pci_liveupdate_can_preserve(struct liveupdate_file_handle= r *handler, struct file *file) { - return false; + struct vfio_device *device =3D vfio_device_from_file(file); + struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + + if (!device) + return false; + + /* Live Update support is limited to cdev files. */ + if (!vfio_device_cdev_opened(device)) + return false; + + if (device->ops !=3D &vfio_pci_ops) + return false; + + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + pdev =3D vdev->pdev; + + /* + * Don't support specialized vfio-pci devices for now since they haven't + * been tested. + */ + if (IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM) || vfio_pci_is_intel_display(pde= v)) + return false; + + return true; } =20 static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *ar= gs) { - return -EOPNOTSUPP; + struct vfio_device *device =3D vfio_device_from_file(args->file); + struct vfio_pci_core_device_ser *ser; + struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + pdev =3D vdev->pdev; + + ser =3D kho_alloc_preserve(sizeof(*ser)); + if (IS_ERR(ser)) + return PTR_ERR(ser); + + ser->bdf =3D pci_dev_id(pdev); + ser->domain =3D pci_domain_nr(pdev->bus); + + args->serialized_data =3D virt_to_phys(ser); + return 0; } =20 static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args = *args) { + kho_unpreserve_free(phys_to_virt(args->serialized_data)); +} + +static int vfio_pci_liveupdate_freeze(struct liveupdate_file_op_args *args) +{ + struct vfio_device *device =3D vfio_device_from_file(args->file); + struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + int ret; + + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + pdev =3D vdev->pdev; + + guard(mutex)(&device->dev_set->lock); + + /* + * Userspace must disable interrupts on the device prior to freeze so + * that the device does not send any interrupts until new interrupt + * handlers have been established by the next kernel. + */ + if (vdev->irq_type !=3D VFIO_PCI_NUM_IRQS) { + pci_err(pdev, "Freeze failed! Interrupts are still enabled.\n"); + return -EINVAL; + } + + ret =3D pci_load_saved_state(pdev, vdev->pci_saved_state); + if (ret) + return ret; + + vfio_pci_core_try_reset(vdev); + pci_restore_state(pdev); + return 0; } =20 static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) @@ -42,6 +193,7 @@ static const struct liveupdate_file_ops vfio_pci_liveupd= ate_file_ops =3D { .can_preserve =3D vfio_pci_liveupdate_can_preserve, .preserve =3D vfio_pci_liveupdate_preserve, .unpreserve =3D vfio_pci_liveupdate_unpreserve, + .freeze =3D vfio_pci_liveupdate_freeze, .retrieve =3D vfio_pci_liveupdate_retrieve, .finish =3D vfio_pci_liveupdate_finish, .owner =3D THIS_MODULE, diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index cbf46e09da30..fa5c7f544f8a 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -11,6 +11,10 @@ /* Cap maximum number of ioeventfds per device (arbitrary) */ #define VFIO_PCI_IOEVENTFD_MAX 1000 =20 +extern const struct vfio_device_ops vfio_pci_ops; + +void vfio_pci_core_try_reset(struct vfio_pci_core_device *vdev); + struct vfio_pci_ioeventfd { struct list_head next; struct vfio_pci_core_device *vdev; diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c index 742477546b15..8b222f71bbab 100644 --- a/drivers/vfio/vfio_main.c +++ b/drivers/vfio/vfio_main.c @@ -1436,7 +1436,7 @@ const struct file_operations vfio_device_fops =3D { #endif }; =20 -static struct vfio_device *vfio_device_from_file(struct file *file) +struct vfio_device *vfio_device_from_file(struct file *file) { struct vfio_device_file *df =3D file->private_data; =20 @@ -1444,6 +1444,7 @@ static struct vfio_device *vfio_device_from_file(stru= ct file *file) return NULL; return df->device; } +EXPORT_SYMBOL_GPL(vfio_device_from_file); =20 /** * vfio_file_is_valid - True if the file is valid vfio file diff --git a/include/linux/kho/abi/vfio_pci.h b/include/linux/kho/abi/vfio_= pci.h index e2412b455e61..876aaf81dd92 100644 --- a/include/linux/kho/abi/vfio_pci.h +++ b/include/linux/kho/abi/vfio_pci.h @@ -9,6 +9,9 @@ #ifndef _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H #define _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H =20 +#include +#include + /** * DOC: VFIO PCI Live Update ABI * @@ -25,4 +28,16 @@ =20 #define VFIO_PCI_LUO_FH_COMPATIBLE "vfio-pci-v1" =20 +/** + * struct vfio_pci_core_device_ser - Serialized state of a single VFIO PCI + * device. + * + * @domain: The device's PCI domain number (segment). + * @bdf: The device's PCI bus, device, and function number. + */ +struct vfio_pci_core_device_ser { + u32 domain; + u16 bdf; +} __packed; + #endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index e90859956514..e9d3ddb715c5 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -81,6 +81,8 @@ struct vfio_device { #endif }; =20 +struct vfio_device *vfio_device_from_file(struct file *file); + /** * struct vfio_device_ops - VFIO bus driver device callbacks * --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 279F6389119 for ; Mon, 23 Mar 2026 23:58:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310334; cv=none; b=tmdDdF5XMlXi5wqlg1XcUyPCPMTrAGYH21ANfYFjbopFhkFLyxG5xaPk2Ow3Mlv7IXSvW0+oTDsxEGTfsVmQsfc5t6AfePUqtvhLjoylIn1s/gAswRqW2B2xefJQ8pyYn0PqUiTFZtiozWr8NLvS0pj2RgRH1GjoEwkMI93c4bA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310334; c=relaxed/simple; bh=hpHWwpCyWNYJavxcdBciCVs72hFwampWkekPO66znUs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=m2v3o+VtUktAUSOEvo+uLZ35OW4Vsy/YK9fWQg8xLxvEo/Rb3j1dmJKwRWZgzj0eNwp/Xm4TIp5PyQvc6ykAIuuPCHPKPqbIEQsYIqC5N54w8CA1UQRO/J3162hat7674APjDhIqbSP1ZrzkgkxicTLP+U26zukLSxJr9IkNOHo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Aqd6XXCz; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Aqd6XXCz" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-358f058973fso1103385a91.1 for ; Mon, 23 Mar 2026 16:58:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310331; x=1774915131; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kmzYNzW0+Ct+eXaBxLnJaomUJdCksoGs2CQSxI1y3bk=; b=Aqd6XXCzQEqQ+JMYsEMeKF8bxKs7kJa+7EfSfX6pMR6ef3N1JzIbSLGACB9zgw42vu 9Rw4mPWDe4Sd8iNutrPnb1etpj+vhMBuxtI5u+gokIIDaiVJqk1sq6rbMaZumvDxoyyy MxRfXEQK1HHinl+z0vI8Or2n4NX1Bowkjm3pTNO0uWtTMzrabD4wQreGP0QvhL1r8zFn WVAVF6bSgDs9HjFnYyJxTnsyKx0hA68CMCiEFaxD7G3qrsxtw/WQGpmIfwCcHSkGjXpt hEvon4owwTAI+TztBdDpBsco+D6PMUQoFxaJ5t+5pR9KGaoy8afelk0xYDOsBxMWw2Ic m5UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310331; x=1774915131; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kmzYNzW0+Ct+eXaBxLnJaomUJdCksoGs2CQSxI1y3bk=; b=cXw142qbtiHU74DT+V8pxRHiGxn+MUNNGAt9IVyIcZBvIe9Hb06pFuQ6W0Cplai36D 8inqwWYNU9SV1UljjNgomzEDvkigKOXBnWQBdRBUBLb96enDsXNzBhiAoeyNxc0I7UQB IrgKvbE2s32Fxx/iFxH3XdfEpvoCu37WjQ5NdAoZXqnNjwAO2QPvOlSZnf2zmQjn1UmC bfj4GAcpUHAGwGWay8hze3/S1EHItZK9DwukVDt8bFvuBjMp+yaOvrirusloINxlCaLl zA1AS8/S+Da3IBhBD48pex3BujwLTSeGl2k/smMmg8MfV1jURExFEVi218tBrxVYNHob CCyQ== X-Forwarded-Encrypted: i=1; AJvYcCV0m1xjK1F8durA2TwO07qFClZNhT0X0pHKwzNrtd9pFNsEzL5FcxYm2WxvcrvcNOMIUAEpDSvwVlbjFe4=@vger.kernel.org X-Gm-Message-State: AOJu0YyEJylKfuKul4b1D2wTTaQOoZki0tGo5JQ5FHMmkOdKFlAeMqX7 46GCjnlt9OdDtunhbXZ4mtk00PJkCNpM5C0zzIq4okqf7VcrAz41sBg5/CwVkR7U+2v8IXoi/tQ ohglAbH5E2xSOkg== X-Received: from pjbfv10.prod.google.com ([2002:a17:90b:e8a:b0:35b:939b:1e33]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:a82:b0:35b:e4d6:73cd with SMTP id 98e67ed59e1d1-35be4d6748bmr4279808a91.31.1774310331243; Mon, 23 Mar 2026 16:58:51 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:00 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-9-dmatlack@google.com> Subject: [PATCH v3 08/24] vfio/pci: Retrieve preserved device files after Live Update From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Enable userspace to retrieve preserved VFIO device files from VFIO after a Live Update by implementing the retrieve() and finish() file handler callbacks. Use an anonymous inode when creating the file, since the retrieved device file is not opened through any particular cdev inode, and the cdev inode does not matter in practice. For now the retrieved file is functionally equivalent a opening the corresponding VFIO cdev file. Subsequent commits will leverage the preserved state associated with the retrieved file to preserve bits of the device across Live Update. Signed-off-by: Vipin Sharma Co-developed-by: David Matlack Signed-off-by: David Matlack --- drivers/vfio/device_cdev.c | 59 ++++++++++++++++++++++---- drivers/vfio/pci/vfio_pci_liveupdate.c | 52 ++++++++++++++++++++++- drivers/vfio/vfio_main.c | 13 ++++++ include/linux/vfio.h | 11 +++++ 4 files changed, 124 insertions(+), 11 deletions(-) diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c index 8ceca24ac136..edf322315a41 100644 --- a/drivers/vfio/device_cdev.c +++ b/drivers/vfio/device_cdev.c @@ -2,6 +2,7 @@ /* * Copyright (c) 2023 Intel Corporation. */ +#include #include #include =20 @@ -16,15 +17,10 @@ void vfio_init_device_cdev(struct vfio_device *device) device->cdev.owner =3D THIS_MODULE; } =20 -/* - * device access via the fd opened by this function is blocked until - * .open_device() is called successfully during BIND_IOMMUFD. - */ -int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep) +static int vfio_device_cdev_open(struct vfio_device *device, struct file *= *filep) { - struct vfio_device *device =3D container_of(inode->i_cdev, - struct vfio_device, cdev); struct vfio_device_file *df; + struct file *file =3D *filep; int ret; =20 /* Paired with the put in vfio_device_fops_release() */ @@ -37,22 +33,67 @@ int vfio_device_fops_cdev_open(struct inode *inode, str= uct file *filep) goto err_put_registration; } =20 - filep->private_data =3D df; + /* + * Simulate opening the character device using an anonymous inode. The + * returned file has the same properties as a cdev file (e.g. operations + * are blocked until BIND_IOMMUFD is called). + */ + if (!file) { + file =3D anon_inode_getfile_fmode("[vfio-device-liveupdate]", + &vfio_device_fops, NULL, + O_RDWR, FMODE_PREAD | FMODE_PWRITE); + + if (IS_ERR(file)) { + ret =3D PTR_ERR(file); + goto err_free_device_file; + } + + *filep =3D file; + } + + file->private_data =3D df; =20 /* * Use the pseudo fs inode on the device to link all mmaps * to the same address space, allowing us to unmap all vmas * associated to this device using unmap_mapping_range(). */ - filep->f_mapping =3D device->inode->i_mapping; + file->f_mapping =3D device->inode->i_mapping; =20 return 0; =20 +err_free_device_file: + kvfree(df); err_put_registration: vfio_device_put_registration(device); return ret; } =20 +struct file *vfio_device_liveupdate_cdev_open(struct vfio_device *device) +{ + struct file *file =3D NULL; + int ret; + + ret =3D vfio_device_cdev_open(device, &file); + if (ret) + return ERR_PTR(ret); + + return file; +} +EXPORT_SYMBOL_GPL(vfio_device_liveupdate_cdev_open); + +/* + * device access via the fd opened by this function is blocked until + * .open_device() is called successfully during BIND_IOMMUFD. + */ +int vfio_device_fops_cdev_open(struct inode *inode, struct file *file) +{ + struct vfio_device *device =3D container_of(inode->i_cdev, + struct vfio_device, cdev); + + return vfio_device_cdev_open(device, &file); +} + static void vfio_df_get_kvm_safe(struct vfio_device_file *df) { spin_lock(&df->kvm_ref_lock); diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index c4ebc7c486e5..4b83a02401aa 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -39,7 +39,13 @@ * preserved, so there is no way for the file to be destroyed or the de= vice * to be unbound from the vfio-pci driver while it is preserved. * - * Retrieving the file after kexec is not yet supported. + * After kexec, the preserved VFIO device file can be retrieved from the s= ession + * just like any other preserved file:: + * + * ioctl(session_fd, LIVEUPDATE_SESSION_RETRIEVE_FD, &arg); + * device_fd =3D arg.fd; + * ... + * ioctl(session_fd, LIVEUPDATE_SESSION_FINISH, ...); * * Restrictions * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D @@ -85,6 +91,7 @@ =20 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt =20 +#include #include #include #include @@ -180,13 +187,53 @@ static int vfio_pci_liveupdate_freeze(struct liveupda= te_file_op_args *args) return 0; } =20 +static int match_device(struct device *dev, const void *arg) +{ + struct vfio_device *device =3D container_of(dev, struct vfio_device, devi= ce); + const struct vfio_pci_core_device_ser *ser =3D arg; + struct pci_dev *pdev; + + pdev =3D dev_is_pci(device->dev) ? to_pci_dev(device->dev) : NULL; + if (!pdev) + return false; + + return ser->bdf =3D=3D pci_dev_id(pdev) && ser->domain =3D=3D pci_domain_= nr(pdev->bus); +} + static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) { - return -EOPNOTSUPP; + struct vfio_pci_core_device_ser *ser; + struct vfio_device *device; + struct file *file; + int ret =3D 0; + + ser =3D phys_to_virt(args->serialized_data); + + device =3D vfio_find_device(ser, match_device); + if (!device) + return -ENODEV; + + file =3D vfio_device_liveupdate_cdev_open(device); + if (IS_ERR(file)) { + ret =3D PTR_ERR(file); + goto out; + } + + args->file =3D file; +out: + /* Drop the reference from vfio_find_device() */ + put_device(&device->device); + return ret; +} + +static bool vfio_pci_liveupdate_can_finish(struct liveupdate_file_op_args = *args) +{ + return args->retrieve_status > 0; } =20 static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) { + kho_restore_free(phys_to_virt(args->serialized_data)); } =20 static const struct liveupdate_file_ops vfio_pci_liveupdate_file_ops =3D { @@ -195,6 +242,7 @@ static const struct liveupdate_file_ops vfio_pci_liveup= date_file_ops =3D { .unpreserve =3D vfio_pci_liveupdate_unpreserve, .freeze =3D vfio_pci_liveupdate_freeze, .retrieve =3D vfio_pci_liveupdate_retrieve, + .can_finish =3D vfio_pci_liveupdate_can_finish, .finish =3D vfio_pci_liveupdate_finish, .owner =3D THIS_MODULE, }; diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c index 8b222f71bbab..e5886235cad4 100644 --- a/drivers/vfio/vfio_main.c +++ b/drivers/vfio/vfio_main.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -1766,6 +1767,18 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr= _t iova, void *data, } EXPORT_SYMBOL(vfio_dma_rw); =20 +struct vfio_device *vfio_find_device(const void *data, device_match_t matc= h) +{ + struct device *device; + + device =3D class_find_device(vfio.device_class, NULL, data, match); + if (!device) + return NULL; + + return container_of(device, struct vfio_device, device); +} +EXPORT_SYMBOL_GPL(vfio_find_device); + /* * Module/class support */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index e9d3ddb715c5..7384965d15d7 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -393,4 +393,15 @@ int vfio_virqfd_enable(void *opaque, int (*handler)(vo= id *, void *), void vfio_virqfd_disable(struct virqfd **pvirqfd); void vfio_virqfd_flush_thread(struct virqfd **pvirqfd); =20 +#if IS_ENABLED(CONFIG_VFIO_DEVICE_CDEV) +struct file *vfio_device_liveupdate_cdev_open(struct vfio_device *device); +#else +static inline struct file *vfio_device_liveupdate_cdev_open(struct vfio_de= vice *device) +{ + return ERR_PTR(-EOPNOTSUPP); +} +#endif /* IS_ENABLED(CONFIG_VFIO_DEVICE_CDEV) */ + +struct vfio_device *vfio_find_device(const void *data, device_match_t matc= h); + #endif /* VFIO_H */ --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D4A59386545 for ; Mon, 23 Mar 2026 23:58:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310335; cv=none; b=AxiX0q9VXfPKahDHWqOoQyfT2yDbgAaAKgemjkGLfWZYus6oU6lxiQxIEr3xDZAgS4xUvfgHnHxNCTy0Gnk8/97T6gHk2s+YyPlWAnY3+unG5v/YaaAy6mWkGcLAiYwBSnd+NAQh6wYfxFHfD8vi91hRSZuPjT212hWuK2TQiIA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310335; c=relaxed/simple; bh=PBlaixBFERkwnraELbaSfRJMLkjw7p/DT3tIx1ijTEg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GNf0AG+pJvP1QQr4gJfd1on6G4h6c5p+DBxLHlDQukpibSQw98JzGdQUKXZhDdX247ETH58Hqooi2VLqSvdQVdEQY1coCz8xnQYaX1gnuNS566QRstkbJ7d9k1kNa/6u85GUBx88AFFoXEF4HT998zeLKMrNdOOxTgPf/fkwXK4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZiF04+ZA; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZiF04+ZA" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-35a1e725a8fso24244137a91.3 for ; Mon, 23 Mar 2026 16:58:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310333; x=1774915133; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FxGJfmbE6y6D4fg7v0R8cmHFXmMbzFJlr4dbGGaFVKE=; b=ZiF04+ZAJ6wsl9QhyvqABMNwvUHXKldkY0AemS7SUo0vutKxPbtIUY2QXwEeryz+Gb v4JRthi91fBxgKcIPG3HRL7Mp42u5/rDOJkAmuQLgM9/kFUXhouDzfi8oMZ/CjeDI8fv qdIomlJSLGZhGlJ7UE701ik8TSSxShE9DBnXlmp2ZA8SjEom5ceT6TT6vJ4rhlf0h5uQ iaPn9tj+1O7/yNQpLIr/TE42kIhmbdqSonxutB7hX31VoDnWRfThCHcCGgFi/lty1t7h A68TLaKqPEC1Jb2qCYhjMWdo79+Jir99UucMpIHBvpJaAGo/HUWzl7mUL8cUGIjiKaA1 /oNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310333; x=1774915133; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FxGJfmbE6y6D4fg7v0R8cmHFXmMbzFJlr4dbGGaFVKE=; b=K0ir7TkEz+OXMBzkbS9V9KVRsUi6F969S0PfpPuy4gssImNvO7uDP9HtJf+A8ipboD RVsVWvrtE+N0eKhUSieh9OL8YZeLszqoOVhP5PptoJleEbXFzzrXIymbyIl6T/EBRZOs EdonRC1k96Iz0r1F69wTpB6KvVFd06S/TGLkaH69TglkU2SDapgFP3m+fj2EfG0J03jE D/1sCJwmAoUIguWzyL6EAltFdyyh3kp6csGRbk+cIQFrV4gK9NIloGvPFlryrduZQqI/ VfDiIkF7UarWx49CNdlcKIrpqAx8iBSd68fhglsuUNaxSa1tKhjFh6FA1GOfK5P9KRDe 2+Tg== X-Forwarded-Encrypted: i=1; AJvYcCVoB6J+AyESyfjGAC3dopUvun69/BEBd3Fe6QX64SxxWoBwOPL1xpDrqDCAMYLQMaDUuHdNpIMhCkYD03M=@vger.kernel.org X-Gm-Message-State: AOJu0YyqJDr7oXNxoa2Lr+26dBNsmPwR4dcimhJKQkaai6RbbIyoPZik ro5Ms3bqeKqs8K+qHVWBzPTII6YxLU7L5A+3aafu+vazXME+smpu4sjYqwc5Xmmy/8f6Iwp0itV EvLXFmWCJzUcQVg== X-Received: from pjbca11.prod.google.com ([2002:a17:90a:f30b:b0:359:84f3:a9d]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:384b:b0:35b:9ae3:9080 with SMTP id 98e67ed59e1d1-35bd2c62dc6mr13468908a91.14.1774310333147; Mon, 23 Mar 2026 16:58:53 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:01 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-10-dmatlack@google.com> Subject: [PATCH v3 09/24] vfio/pci: Notify PCI subsystem about devices preserved across Live Update From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Notify the PCI subsystem about devices vfio-pci is preserving across Live Update by registering the vfio-pci liveupdate file handler with the PCI subsystem's FLB handler. Notably this will ensure that devices preserved through vfio-pci will have their PCI bus numbers preserved across Live Update, allowing VFIO to use BDF as a key to identify the device across the Live Update and (in the future) allow the device to continue DMA operations across the Live Update. This also enables VFIO to detect that a device was preserved before userspace first retrieves the file from it, which will be used in subsequent commits. Signed-off-by: David Matlack --- drivers/vfio/pci/vfio_pci_liveupdate.c | 44 +++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 5 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 4b83a02401aa..b960ec3ffbf2 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -67,6 +67,9 @@ * interrupts on the device will cause the ``reboot(LINUX_REBOOT_CMD_KE= XEC)`` * syscall (to initiate the kexec) to fail. * + * In addition, the device must meet all of the restrictions imposed by the + * core PCI layer documented at :doc:`/PCI/liveupdate`. + * * Preservation Behavior * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * @@ -136,23 +139,37 @@ static int vfio_pci_liveupdate_preserve(struct liveup= date_file_op_args *args) struct vfio_pci_core_device_ser *ser; struct vfio_pci_core_device *vdev; struct pci_dev *pdev; + int ret; =20 vdev =3D container_of(device, struct vfio_pci_core_device, vdev); pdev =3D vdev->pdev; =20 + ret =3D pci_liveupdate_preserve(pdev); + if (ret) + return ret; + ser =3D kho_alloc_preserve(sizeof(*ser)); - if (IS_ERR(ser)) - return PTR_ERR(ser); + if (IS_ERR(ser)) { + ret =3D PTR_ERR(ser); + goto err_unpreserve; + } =20 ser->bdf =3D pci_dev_id(pdev); ser->domain =3D pci_domain_nr(pdev->bus); =20 args->serialized_data =3D virt_to_phys(ser); return 0; + +err_unpreserve: + pci_liveupdate_unpreserve(pdev); + return ret; } =20 static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args = *args) { + struct vfio_device *device =3D vfio_device_from_file(args->file); + + pci_liveupdate_unpreserve(to_pci_dev(device->dev)); kho_unpreserve_free(phys_to_virt(args->serialized_data)); } =20 @@ -213,6 +230,10 @@ static int vfio_pci_liveupdate_retrieve(struct liveupd= ate_file_op_args *args) if (!device) return -ENODEV; =20 + ret =3D pci_liveupdate_retrieve(to_pci_dev(device->dev)); + if (ret) + goto out; + file =3D vfio_device_liveupdate_cdev_open(device); if (IS_ERR(file)) { ret =3D PTR_ERR(file); @@ -233,6 +254,9 @@ static bool vfio_pci_liveupdate_can_finish(struct liveu= pdate_file_op_args *args) =20 static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) { + struct vfio_device *device =3D vfio_device_from_file(args->file); + + pci_liveupdate_finish(to_pci_dev(device->dev)); kho_restore_free(phys_to_virt(args->serialized_data)); } =20 @@ -257,13 +281,23 @@ int __init vfio_pci_liveupdate_init(void) int ret; =20 ret =3D liveupdate_register_file_handler(&vfio_pci_liveupdate_fh); - if (ret && ret !=3D -EOPNOTSUPP) - return ret; + if (ret) + goto err_return; + + ret =3D pci_liveupdate_register_flb(&vfio_pci_liveupdate_fh); + if (ret) + goto err_unregister; =20 return 0; + +err_unregister: + liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); +err_return: + return (ret =3D=3D -EOPNOTSUPP) ? 0 : ret; } =20 void vfio_pci_liveupdate_cleanup(void) { - liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); + pci_liveupdate_unregister_flb(&vfio_pci_liveupdate_fh); + liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); } --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA4D438B7D9 for ; Mon, 23 Mar 2026 23:58:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310337; cv=none; b=o1hsbxVePzxrYQg4AQLEEx6jjgO95epLT8OOoGQ/V16pQbQGy9yXyMY1g7lKQMNlHlEA1HQh/leBI2xdCCR1o8Qhb1AaFIpGz1QDhUbr+ys+Vj53kNIHkhs3wuI6QgwsYp4JSXHzSp7I7jsZ89Ud9loSwO1GwXahI+MI+uXgZtE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310337; c=relaxed/simple; bh=JIeulFoC8MyqRgbySpHafWJZwGtuNWg8OY+XTEFKwh4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=h2kLEZzy4gXXaVvRnkrnDVpnrlBp1wNU52qjgScjnf4Re/J+G9HAjz8pAqbhZV3ZaKoDvHCqB7cxN6jH4VN/+R/mycwHbLlj2pD/CSAz6NgQyKNexFhQd/mnfOZg7+SRurQv1ag5qeMB4+JWbp3Kzd+SwEf7Rlhrq1OE0seuH2A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GSpmzAY4; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GSpmzAY4" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-358df8fbd1cso1043111a91.0 for ; Mon, 23 Mar 2026 16:58:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310335; x=1774915135; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UG6U+6kAmFzvMZjga5OgV1LR5RVhGZN2WyiuuoZC9eg=; b=GSpmzAY4CKU4FJrD5pZnlzEgXy51vaw4RRfSxB09bx/fi18h5PgKCqINx/aFl25civ uu5siRoewCHYyqM+XWVkgAm7w/8TwFBmnQ2jSVJ3Sn4V2vVV0TbTx6dbSRgJvshQhcoF f4RhTY/l4ewQtMYJdRWETnju2Gquj4CdPSV1UyIaXY8Z+TRLG2DsWrOMylaRzPtN/Onf twmGORsCtKS4mZMMj8QMgHjxV/E26F9dJdwL97crYwjhl6SSPEFx6gOvp8lnhUJ8ZJns k9B10wEggsJLLH8grZc7OnYFVvis/BT/g17qyNZGD5gfvV27HjxCL3MPSW3aY+p2xL84 HRMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310335; x=1774915135; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UG6U+6kAmFzvMZjga5OgV1LR5RVhGZN2WyiuuoZC9eg=; b=Z7La5JSGXXKIh/0EoRD//c32u1DE3dDqqxLBDQeVT2PyaeVhtK4RZ7mF2tHoQ7LIk3 q3pROV/5lPgYS4QjHAZT/2lEdRnQTgwDUeWpZ8Wc7FKaZL+E+4LG9QwKa49PCZhukVzb XeJo3vKDGQYLs/pMavfClki9aTBpfHZPxdm29EtTcBl8vCHPriS4ZS6BgGzURUQGO8lm /T2j+8XWb6/ZQyzRyxwpp9udzveZU5IMANa/xWfxuqwagexW1QguXPw/lrDQt06/64am e/lNNO2w3Z30cjNS2LtrqOsnyRmEWslHFnLa5bqUQC5PfKNn7vk0GUFEBpuRBtJBRsxd utBQ== X-Forwarded-Encrypted: i=1; AJvYcCX1d/ZGiXnLJlMEKNjzI5hMoOWkFpwcXZOtgRmUGbUMLY5AU28MO3m3wNvIxxyzyWF9cO/VgsInsSVYkuw=@vger.kernel.org X-Gm-Message-State: AOJu0YxZILgoKMWjCkxPgpzjXIm0yFwFRuiktQgTEkrPe3xgWBn6xOSu qtb7Er6czRPOj4IqWiiTOLonsHe5zT5q4ZXQTRZiNN6s6HcHq8TQ9RJaSjH71/I58G7p7ZikeMb UkSjHqZERyK9g3w== X-Received: from pgcp23.prod.google.com ([2002:a63:7417:0:b0:c74:1130:c2ea]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:5493:b0:39b:f026:6f7c with SMTP id adf61e73a8af0-39bf0267ba0mr8201078637.49.1774310334903; Mon, 23 Mar 2026 16:58:54 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:02 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-11-dmatlack@google.com> Subject: [PATCH v3 10/24] vfio: Enforce preserved devices are retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Enforce that files for incoming (preserved by previous kernel) VFIO devices are retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD rather than by opening the corresponding VFIO character device or via VFIO_GROUP_GET_DEVICE_FD. Both of these methods would result in VFIO initializing the device without access to the preserved state of the device passed by the previous kernel. Reviewed-by: Pranjal Shrivastava Signed-off-by: David Matlack --- drivers/vfio/device_cdev.c | 4 ++++ drivers/vfio/group.c | 9 +++++++++ drivers/vfio/pci/vfio_pci_liveupdate.c | 6 ++++++ drivers/vfio/vfio.h | 18 ++++++++++++++++++ 4 files changed, 37 insertions(+) diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c index edf322315a41..6844684a3d8e 100644 --- a/drivers/vfio/device_cdev.c +++ b/drivers/vfio/device_cdev.c @@ -91,6 +91,10 @@ int vfio_device_fops_cdev_open(struct inode *inode, stru= ct file *file) struct vfio_device *device =3D container_of(inode->i_cdev, struct vfio_device, cdev); =20 + /* Device file must be retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD */ + if (vfio_liveupdate_incoming_is_preserved(device)) + return -EBUSY; + return vfio_device_cdev_open(device, &file); } =20 diff --git a/drivers/vfio/group.c b/drivers/vfio/group.c index 4f15016d2a5f..0fa9761b13d3 100644 --- a/drivers/vfio/group.c +++ b/drivers/vfio/group.c @@ -311,6 +311,15 @@ static int vfio_group_ioctl_get_device_fd(struct vfio_= group *group, if (IS_ERR(device)) return PTR_ERR(device); =20 + /* + * This device was preserved across a Live Update. Accessing it via + * VFIO_GROUP_GET_DEVICE_FD is not allowed. + */ + if (vfio_liveupdate_incoming_is_preserved(device)) { + vfio_device_put_registration(device); + return -EBUSY; + } + fd =3D FD_ADD(O_CLOEXEC, vfio_device_open_file(device)); if (fd < 0) vfio_device_put_registration(device); diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index b960ec3ffbf2..6f760ace7065 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -47,6 +47,12 @@ * ... * ioctl(session_fd, LIVEUPDATE_SESSION_FINISH, ...); * + * .. note:: + * After kexec, if a device was preserved by the previous kernel, attem= pting + * to open a new file for the device via its character device + * (``/dev/vfio/devices/X``) or via ``VFIO_GROUP_GET_DEVICE_FD`` will f= ail + * with ``-EBUSY``. + * * Restrictions * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h index 50128da18bca..8fcc98cf9577 100644 --- a/drivers/vfio/vfio.h +++ b/drivers/vfio/vfio.h @@ -11,6 +11,7 @@ #include #include #include +#include =20 struct iommufd_ctx; struct iommu_group; @@ -462,4 +463,21 @@ static inline void vfio_device_debugfs_init(struct vfi= o_device *vdev) { } static inline void vfio_device_debugfs_exit(struct vfio_device *vdev) { } #endif /* CONFIG_VFIO_DEBUGFS */ =20 +#ifdef CONFIG_PCI_LIVEUPDATE +static inline bool vfio_liveupdate_incoming_is_preserved(struct vfio_devic= e *device) +{ + struct device *d =3D device->dev; + + if (dev_is_pci(d)) + return to_pci_dev(d)->liveupdate_incoming; + + return false; +} +#else +static inline bool vfio_liveupdate_incoming_is_preserved(struct vfio_devic= e *device) +{ + return false; +} +#endif /* CONFIG_PCI_LIVEUPDATE */ + #endif --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A152838C43E for ; Mon, 23 Mar 2026 23:58:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310339; cv=none; b=k7Ev+IJfvIoaVxGJlDdMf+1vrUuScFQvKVeZveW4JFbcWCoKwkYxHWmHQC2g0dkTqnEJUNq3prpnotmXzUUNDK3QPRF8dxBwi6/QGDzFwvOK82gN7zLidzTnv8biBEYID1hYRSJIzsl6VrDa/sf+f7PlVLyjtkiBKDJPv8dL0Mw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310339; c=relaxed/simple; bh=+0b5yfHhqKc5WRkrCQm2Jd3F6UU1rPdRTjAvF2KhK0Y=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MwMWqPl70lGvqKlGO0BO9JsLNDECEXEVtdNZLYONaONcvC5o2rvQyxEXZk3+A4OTBV9qY8RNMx0puu7iAS0rKdBXPp57pSe/OJKyN371JT+Zf+qpRD1Na1bXBGI5Xl6Q7fmZ6Vr8RV5knrlD5L3Zu2b+c9T2mpm29JzSywITNns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=o2ibxz68; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="o2ibxz68" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2b079b4a8c3so3148805ad.3 for ; Mon, 23 Mar 2026 16:58:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310337; x=1774915137; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=i6NpvLJAkS4c52+WboP6wAkf2PbPQKSMM3U4nd1tgoo=; b=o2ibxz68IXVTcIRdHzzqBa25/gq1A2igO71wTFgztZP/Ct2yhr0W7TM4nPTlvb6uXT a675CAbJ7xrdXRDTnpLslUarYiYWAPoniZYniKrCjcb5eDKN+6Oz33cjQnjtw9dWRwPm eta35Qgbx0E9/UkKofRss8tJe+TDO7xWZwyP71mNKZ4r+TLqWDvyLOxv51W438S9q/Hn oHd36fHDLbH3zIDV4WEob31theMxppbT69D90N+TEe6+R0dalhZQqrpZ8m/HcFq5shpK CytSn3es3Nw274l75XtO66yDCC7EaIJXZg5627sC6FArk8CIEPQDdskqL2aJYvoZcVGu 0unA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310337; x=1774915137; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=i6NpvLJAkS4c52+WboP6wAkf2PbPQKSMM3U4nd1tgoo=; b=O+pWC0ZeufaANHyfwJRC/gjOgNO0qPogD91xxiTHt/t2Driq9ImEkWmo9TviCtnZXv ebq4lXpeNigInNycKoJn8K64iH+9qy/7n669NixF7n0V8C5SXrE3mh31BdcqA5KU+VJT JVRCWDNkvhF6f9D8sndzkNKe4uEnLbd0VAyENzijhpywaUo0+xV5Lchs/i5me2m0tjY+ jjHOxPTQKXJHBO/AwzhIWlrJVmT9yI/P+iqh0A3xdD79eVYNj41AkMy4XNTsF3B7AvfI /HQ8TnW1JZblxt6wWKq20xU9ItGHsqbdYPNNkLNA7wkuDzvnBIIaSwcAJKe6E1pU1O+U 7GZg== X-Forwarded-Encrypted: i=1; AJvYcCWqQjuBFbs0qrXk/zKhdBRxOFjVogOguRUaYnMnx76JAqyKhOpchzF09sXtuAtgp+tZ9NCZANT3H9sAK0g=@vger.kernel.org X-Gm-Message-State: AOJu0YwslLzd7njFO7k119hvfekbpWPkNEg5HpDad6tjsHW+54wiG+2/ Dl6J6OpNoZVz+5uf85GKQ9sCl70LXFBTuDcGxnAGepG8adVwM+4jt7H1svQIZwpHtTVK6cz4fAA rrNynv1ms8h96eQ== X-Received: from plgd12.prod.google.com ([2002:a17:902:cecc:b0:2a8:71ec:6799]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:f984:b0:2ae:c795:6b4a with SMTP id d9443c01a7336-2b0826d73f3mr94191885ad.11.1774310336851; Mon, 23 Mar 2026 16:58:56 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:03 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-12-dmatlack@google.com> Subject: [PATCH v3 11/24] vfio/pci: Store incoming Live Update state in struct vfio_pci_core_device From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Stash a pointer to a device's incoming Live Updated state in struct vfio_pci_core_device. This will enable subsequent commits to use the preserved state when initializing the device. To enable VFIO to safely access this pointer during device enablement, require that the device is fully enabled before returning true from can_finish(). This is synchronized by vfio_pci_core.c setting vdev->liveupdate_incoming_state to NULL under dev_set lock once it's done using it. Signed-off-by: David Matlack --- drivers/vfio/pci/vfio_pci_core.c | 2 +- drivers/vfio/pci/vfio_pci_liveupdate.c | 17 ++++++++++++++++- include/linux/vfio_pci_core.h | 2 ++ 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 81f941323641..d7c472cf4729 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -568,7 +568,7 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *v= dev) if (!vfio_vga_disabled() && vfio_pci_is_vga(pdev)) vdev->has_vga =3D true; =20 - + vdev->liveupdate_incoming_state =3D NULL; return 0; =20 out_free_zdev: diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 6f760ace7065..8d6681e1d328 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -226,6 +226,7 @@ static int match_device(struct device *dev, const void = *arg) static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) { struct vfio_pci_core_device_ser *ser; + struct vfio_pci_core_device *vdev; struct vfio_device *device; struct file *file; int ret =3D 0; @@ -246,6 +247,9 @@ static int vfio_pci_liveupdate_retrieve(struct liveupda= te_file_op_args *args) goto out; } =20 + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + vdev->liveupdate_incoming_state =3D ser; + args->file =3D file; out: /* Drop the reference from vfio_find_device() */ @@ -255,7 +259,18 @@ static int vfio_pci_liveupdate_retrieve(struct liveupd= ate_file_op_args *args) =20 static bool vfio_pci_liveupdate_can_finish(struct liveupdate_file_op_args = *args) { - return args->retrieve_status > 0; + struct vfio_pci_core_device *vdev; + struct vfio_device *device; + + if (args->retrieve_status <=3D 0) + return false; + + device =3D vfio_device_from_file(args->file); + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + + /* Check that vdev->liveupdate_incoming_state is no longer in use. */ + guard(mutex)(&device->dev_set->lock); + return !vdev->liveupdate_incoming_state; } =20 static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 2ebba746c18f..0c508dd8d1ac 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -26,6 +26,7 @@ #define VFIO_PCI_OFFSET_MASK (((u64)(1) << VFIO_PCI_OFFSET_SHIFT) - 1) =20 struct vfio_pci_core_device; +struct vfio_pci_core_device_ser; struct vfio_pci_region; struct p2pdma_provider; struct dma_buf_attachment; @@ -142,6 +143,7 @@ struct vfio_pci_core_device { struct notifier_block nb; struct rw_semaphore memory_lock; struct list_head dmabufs; + struct vfio_pci_core_device_ser *liveupdate_incoming_state; }; =20 enum vfio_pci_io_width { --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D4173876BD for ; Mon, 23 Mar 2026 23:58:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310341; cv=none; b=VdvyfWQY7QdkDltjfWhuwC5fyQTTv5dADxxI6j9aHa9hXr23CxaJ734n4X0DIR7R/JbNlBBkHExoZzQ8yDk0BpWMYcWyXLCXWPeXpxh7PRZhD3PEIjQ4hPV+NAiCae4Cwv+VHsQNhEBUzGP0UPpWGNS1xuppGvefXobaUZhGjto= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310341; c=relaxed/simple; bh=YnyFu7Hq5DfkB2TP1cJWlWPeBT9cbNudoys76NodHZs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=d/S6dfV8KGJw0sPQ2vh7JonozvfE++7zUISQ9fXgIUZft6RAiKdXorGf8a5xsPWmnJGm+7jkDFDiX4k6X/yKtT7fuALV4MWVCZmRrWNC7Spjxg6/Ul2g9tWl6WkX3wIvqf2qSn/ybT8lZMShwKoNJsMQPpUI7ztyAI51XDc5qMI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qHoH3/1q; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qHoH3/1q" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2b069bfa817so8365915ad.3 for ; Mon, 23 Mar 2026 16:58:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310339; x=1774915139; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WDvzsZ/wW7otbfhEhu+LWvOHypwI+AXZcNsT2i6QVdU=; b=qHoH3/1qrDc4Llk2/CjdmQ5F+FP364nEtKTdWQnSA+Aq0mjTS3KiiMBqQO0aD2IxfT 30/YPj2MJMudFABAIgGABWxSqdZb5XGCiqA8tfS1fiIizdebEV3Y0m7+5ekOMFvObXAK JhxaUbgiscB1Y/CDHuSYXdlq4HcKBvhIObzPEI3KqKJ/T/58WPh+GbKtzqjqpZQ/Wbi+ C98iu1eBMhTS+rv0QwtpnBtghxO0T96RWnpX1jdzdIKBySG7n7RUj2p5E98wLt9HVuFP B659dpJf4RXvE8TL6eiUFzkUGtX/K8Covb94i0raIgP8Zh1jxaIXDYHGtrw1SR0795ON cgXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310339; x=1774915139; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WDvzsZ/wW7otbfhEhu+LWvOHypwI+AXZcNsT2i6QVdU=; b=TKtlTaHGUup/1r7ZoiYU2OAZHs8fMLErWt+5Hd6Q57UFii2kLhBEaOBaVCNBn8WTcP D22qFqDbaV+ur/tm/BUYJxUPi/x8CziCq7jsawAgaVc/SayI1d7BKa03PuYebjcVsdkN djTVPl+krNIwEG5G9lBY1f3jUWMe5q0ajnB3TAS//g3tnzWCZ/CrqK3b5Qvttkei2FDC GvoRtH19YLa3dyzsabgkQsR0QOlxIQqv6l2IX/Byj83HoOuY+OWD8Aq5NiA9KZ8VLDvR r+xEG4y1n7mQdx5pYKCpOmhbhlHr2ZjNzmV6zoq9Z0XNRORnGbF6DjF8yrhIvKd/+kU/ V7Zw== X-Forwarded-Encrypted: i=1; AJvYcCUNYAldfC6ICB+DYt1ppfj8zuY8vHFNMwnNBFUMHLUnov/Xj8mTkd5+E2LVcz6/E0wHGrMv6x+nR7NE0vc=@vger.kernel.org X-Gm-Message-State: AOJu0YwYJVGiWLycomdMHU9UCs8iii3S9jaSc/u01/z3hXa6xiQF8+/i Z5fUV3jSzbpinUidcjxxUYkuLaNA33AhKY8rr89T5ynFKl9BR+wuS0mFCcS6kBfpHz240VUMF0K eE0d0t6gXrjcCyQ== X-Received: from pldt15.prod.google.com ([2002:a17:903:40cf:b0:2b0:59ce:931b]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:1c7:b0:2b0:5fa5:a68c with SMTP id d9443c01a7336-2b0826b9030mr140489925ad.9.1774310338569; Mon, 23 Mar 2026 16:58:58 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:04 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-13-dmatlack@google.com> Subject: [PATCH v3 12/24] vfio/pci: Skip reset of preserved device after Live Update From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Do not reset the device when a Live Update preserved vfio-pci device is retrieved and first enabled. vfio_pci_liveupdate_freeze() guarantees the device is reset prior to Live Update, so there's no reason to reset it again after Live Update. Since VFIO normally uses the initial reset to detect if the device supports function resets, pass that from the previous kernel via struct vfio_pci_core_dev_ser. Signed-off-by: Vipin Sharma Signed-off-by: David Matlack --- drivers/vfio/pci/vfio_pci_core.c | 31 ++++++++++++++++++++++---- drivers/vfio/pci/vfio_pci_liveupdate.c | 4 ++++ include/linux/kho/abi/vfio_pci.h | 4 +++- 3 files changed, 34 insertions(+), 5 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index d7c472cf4729..849a3b57d56b 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -494,6 +495,30 @@ static const struct dev_pm_ops vfio_pci_core_pm_ops = =3D { NULL) }; =20 +static int vfio_pci_core_probe_reset(struct vfio_pci_core_device *vdev) +{ + int ret; + + /* + * This device was preserved by the previous kernel across a Live + * Update, so it does not need to be reset and reset_works can be + * inherited from the previous kernel. + */ + if (vdev->liveupdate_incoming_state) { + vdev->reset_works =3D vdev->liveupdate_incoming_state->reset_works; + return 0; + } + + ret =3D pci_try_reset_function(vdev->pdev); + + /* Bail if the device lock cannot be acquired. */ + if (ret =3D=3D -EAGAIN) + return ret; + + vdev->reset_works =3D !ret; + return 0; +} + int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev =3D vdev->pdev; @@ -514,12 +539,10 @@ int vfio_pci_core_enable(struct vfio_pci_core_device = *vdev) if (ret) goto out_power; =20 - /* If reset fails because of the device lock, fail this path entirely */ - ret =3D pci_try_reset_function(pdev); - if (ret =3D=3D -EAGAIN) + ret =3D vfio_pci_core_probe_reset(vdev); + if (ret) goto out_disable_device; =20 - vdev->reset_works =3D !ret; pci_save_state(pdev); vdev->pci_saved_state =3D pci_store_saved_state(pdev); if (!vdev->pci_saved_state) diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 8d6681e1d328..874c821bf6eb 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -91,6 +91,9 @@ * kernel guarantees the these will not change across a kexec when a de= vice * is preserved. * + * * Whether or not the device supports function resets. This is necessar= y to + * avoid resetting the device after kexec to probe for reset support. + * * Since the kernel is not yet prepared to preserve all parts of the devic= e and * its dependencies (such as DMA mappings), VFIO currently resets and rest= ores * preserved devices back into an idle state during kexec, before handing = off @@ -162,6 +165,7 @@ static int vfio_pci_liveupdate_preserve(struct liveupda= te_file_op_args *args) =20 ser->bdf =3D pci_dev_id(pdev); ser->domain =3D pci_domain_nr(pdev->bus); + ser->reset_works =3D vdev->reset_works; =20 args->serialized_data =3D virt_to_phys(ser); return 0; diff --git a/include/linux/kho/abi/vfio_pci.h b/include/linux/kho/abi/vfio_= pci.h index 876aaf81dd92..c057794a044f 100644 --- a/include/linux/kho/abi/vfio_pci.h +++ b/include/linux/kho/abi/vfio_pci.h @@ -26,7 +26,7 @@ * incrementing the version number in the VFIO_PCI_LUO_FH_COMPATIBLE strin= g. */ =20 -#define VFIO_PCI_LUO_FH_COMPATIBLE "vfio-pci-v1" +#define VFIO_PCI_LUO_FH_COMPATIBLE "vfio-pci-v2" =20 /** * struct vfio_pci_core_device_ser - Serialized state of a single VFIO PCI @@ -34,10 +34,12 @@ * * @domain: The device's PCI domain number (segment). * @bdf: The device's PCI bus, device, and function number. + * @reset_works: Non-zero if the device supports function resets. */ struct vfio_pci_core_device_ser { u32 domain; u16 bdf; + u8 reset_works; } __packed; =20 #endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */ --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0238138F62E for ; Mon, 23 Mar 2026 23:59:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310342; cv=none; b=iIg6kRKctQPw/VLd0mk+HorFx7lKJzZcbrw9hlWfSBIVTipfXlOFmfEPesWQpqGw2oibzpqpcYwMsEbHXdy6EohUbKaqd1E4iPTQq3hLYKqpV6gR0Rq8pteh0n/DsnQUXZTGYI/6NWkhG6AeB7iBlNPLXKSvpX4E01Ap2Z8GftY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310342; c=relaxed/simple; bh=+70kIpVyn2x8kiDDC1PyFViZ/JQI+XUMMGbOcdpeoO4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PjpcY/yJ9SHFSVcrpRas574mxpvglf/5ytc4EyEwTjxIheNaXNTtJdzo0Yhr6Q3bY5SkPZtsL2SIC8HQP1fpc3XYqsOWuKTeXR9u99j5TlEK/flbnINAfcb9N4B4YOwcbMqSHnjPsl5mDyTv5EmUpGkCkFkTLqey72aSHx6I7mc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=KonaVZOw; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KonaVZOw" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2aec07e8aafso12272685ad.1 for ; Mon, 23 Mar 2026 16:59:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310340; x=1774915140; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=aeeFxLRNeEzMbfk8m3pX/GNMnylfMV1s3l1hGn54HBA=; b=KonaVZOwrYOL3fHMUF+ukavBeIxBBYT0e1WJa4WvMJ1RSBuLqVV3Mk00Rrky5dagLx fSp009YFXfhs6YpR64ZSDt9o0Gx5tFeW2h8hSdMd5p07DR2H5BzGHf1Rh0VlayyCtHuo cS518Ky3jhHGrVUdoZ4eViVu5rsz+zFMmEtG+ZA4p0bDqxs1LHfaxv7Y8d9A5+gvkxE8 /xZml+WaebaztMPMViWBzVXwvU3k0br3vM8z0hkLDstZ0y1LUjnOQLE1ICpDqg8Gq0zJ f+v5z6k6SeaDg7+6dKEkku3vtwTz71tCrVxXoc9R0K0tf2MJlFH/InQhQtIhzviMKipW ukfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310340; x=1774915140; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=aeeFxLRNeEzMbfk8m3pX/GNMnylfMV1s3l1hGn54HBA=; b=BYBicvY9X/dJWdNtGEonqORxSW+2RyN82XMKYRZbW6v9D/qkiCROk6Tu8ltliVQdDR MjWu6M1neTVsIG0jFPaS+TVKJXvmggPpV0egrXpcidwA1AYxnkhhdqv9MEaoDywswjtM AoBjXiOlcg7IEiKRhIji7l1Eunl9f/3o0UgSIkFT3iSNhswBCs24mJBRJuo6eC+9NFaE 8S6TDPf9/5IXs4z6MnL76hapN5yIBCIOHYH2xgiUVMlS5j1Lg6mwzNGXDQxWalDRYGJ+ oEGn/dDZqeevC0bTsibWKVKYTZaHYXX/sWkM8JFpUPLzsjfLaeP7KnxYLEeSgKncVraT uCPg== X-Forwarded-Encrypted: i=1; AJvYcCVRlREsTlix3jfRGT7+tccvWZ++Qkz9HK7c6gWU2oBMh1LrK/Kl+XiroPM8ErLQ8S0CVq1nhTrdMb3xVX4=@vger.kernel.org X-Gm-Message-State: AOJu0YyhQjvdB9rHOIApFTuaxouyb0Cjmk41wQZmPvi1CsGCMn351dIG fcajXlRf5r92TPJ4GE/XqRTuT5s0GPcRelwZhVZtMpxMJR5sNsXisL79xRKo2A+W5i+ejGGILDq 6EwWqRJgPbuhevw== X-Received: from pgbcl21.prod.google.com ([2002:a05:6a02:995:b0:c73:90cf:9638]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:3286:b0:398:7ca0:43a9 with SMTP id adf61e73a8af0-39bceb46904mr12137508637.36.1774310340303; Mon, 23 Mar 2026 16:59:00 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:05 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-14-dmatlack@google.com> Subject: [PATCH v3 13/24] docs: liveupdate: Add documentation for VFIO PCI From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add documentation for preserving VFIO device files across a Live Update, generated from the kernel-doc comments in the code. Signed-off-by: David Matlack --- Documentation/core-api/liveupdate.rst | 1 + .../driver-api/vfio_pci_liveupdate.rst | 23 +++++++++++++++++++ MAINTAINERS | 1 + 3 files changed, 25 insertions(+) create mode 100644 Documentation/driver-api/vfio_pci_liveupdate.rst diff --git a/Documentation/core-api/liveupdate.rst b/Documentation/core-api= /liveupdate.rst index d56a7760978a..c55d0d9d1d3b 100644 --- a/Documentation/core-api/liveupdate.rst +++ b/Documentation/core-api/liveupdate.rst @@ -34,6 +34,7 @@ The following types of file descriptors can be preserved :maxdepth: 1 =20 ../mm/memfd_preservation + ../driver-api/vfio_pci_liveupdate =20 Public API =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/Documentation/driver-api/vfio_pci_liveupdate.rst b/Documentati= on/driver-api/vfio_pci_liveupdate.rst new file mode 100644 index 000000000000..1098b84e5ecd --- /dev/null +++ b/Documentation/driver-api/vfio_pci_liveupdate.rst @@ -0,0 +1,23 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +VFIO PCI Device Preservation via LUO +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. kernel-doc:: drivers/vfio/pci/vfio_pci_liveupdate.c + :doc: VFIO PCI Preservation via LUO + +VFIO PCI Preservation ABI +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D + +.. kernel-doc:: include/linux/kho/abi/vfio_pci.h + :doc: VFIO PCI Live Update ABI + +.. kernel-doc:: include/linux/kho/abi/vfio_pci.h + :internal: + +See Also +=3D=3D=3D=3D=3D=3D=3D=3D + +- :doc:`/core-api/liveupdate` +- :doc:`/core-api/kho/index` diff --git a/MAINTAINERS b/MAINTAINERS index a16a7ecc67a4..a6a31b94a4e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -27684,6 +27684,7 @@ T: git https://github.com/awilliam/linux-vfio.git F: Documentation/ABI/testing/debugfs-vfio F: Documentation/ABI/testing/sysfs-devices-vfio-dev F: Documentation/driver-api/vfio.rst +F: Documentation/driver-api/vfio_pci_liveupdate.rst F: drivers/vfio/ F: include/linux/kho/abi/vfio_pci.h F: include/linux/vfio.h --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8583388E69 for ; Mon, 23 Mar 2026 23:59:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310346; cv=none; b=HM99Pze47M4UnVmpbMcICTCkDYT9gtIeI2Os26c8wnELwGvSrO6F3/3IWsm7XouZC2vJN5BpLJvCYnR5ag1BLC3reK5+nZJpeOW4+4RXqvmKTREF/Vcbv9302pGcRKmrmsvNfasJJ1grQtaIKbhsHPTzF9/bIGyfnJyMPytN6S0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310346; c=relaxed/simple; bh=CB98rKqSSxQ2+oJREog3POAWNqPld4A6srfHRMe7lyw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=EZrIDVhU2piUUjc2mijwu+5asWvUkekQHAswnPJEyAZ7/KEnNaOXYfOb6szsd0udr0ABalI36/rJXQni0tZC/2b6CWUTJCjT0zzLsh+xZDgP4S5aNIgR/kkwP40E1NC1siJivD7wGVgr8cbQuAaDrHg2do+XDIcAro9jqaHllgY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lBmi4YML; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lBmi4YML" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c74262a2988so13982586a12.1 for ; Mon, 23 Mar 2026 16:59:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310342; x=1774915142; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=omO2IOfE0pDlFemBHvNO/nr0zzMPm3bDzAx2yMbkgWA=; b=lBmi4YMLKGYaM4jQRGbiviDF7S/K5qmjnPT0odqWbfhy0TvJ4zmCEbW7VVpA1AlmZK 5B+iE3vPJh+tHJ1hh39UXRl9sC8I+4jIz/U534xir8M2JcgUTWl4ONorjZA3iQ5r5cNy BsBA5LEDxeDIYlsqyb+8S613rapdS5GGQdCO6DediVVp8PyN750+FPch9FeWEVM6rGwg w2MBepqytGuzagTfKgVQmofH6TFDM8FMvZ9CKdn7p5B5UgcZSh6cUtKXKthgbfHqeSJd fHROWWQ2nXUYRqeYa3AiGTMtOlksz1I/Qnw91dHugNK3JbBdgmcvKjMzRwlQOml6ILi6 CmRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310342; x=1774915142; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=omO2IOfE0pDlFemBHvNO/nr0zzMPm3bDzAx2yMbkgWA=; b=ia4r6JKn9dfXXyo6RihO+eyA1OkhDL1DJxa+RgPzl8twDKo/3yPzYprcHrDtSbs5Xr i4+7Qn863MnETXQ4Hkx7NIaTjSdhD4rRu6sgoatMsNUByQIm1tNZ8DJ2QH0hxnQzupJ3 398XD1UugPYGYIg1LsratZYlYf5Ij3KFce8W3vTupMO7oG7hdhayELb2MojjIb4rV9m2 yzC3E8hdI94QSu13vPNoBAsn4s5ievQiXJQJl4favzHc9rtIZR/DmYcmL9Imv+mEI0JY T9LvS5Un1b4D3RckdGqcVMUwSsve3HpOmTwUwf0jkxeyv3uLGm6lBUa98ejQ8ycASYj5 qMVA== X-Forwarded-Encrypted: i=1; AJvYcCW6tj7BxcFENVDpuxM96SxfJmh0J5XYwtr7HPESfAj17e7ryACY4uXzxWYPsg5go4TPtOOohzocaGpz1II=@vger.kernel.org X-Gm-Message-State: AOJu0YxbS19YNiwJpISGSFFRqor+4Vj0NltJIbtC0CYe9ohBxVA/6cgq Cj1up5l5QO3c7muqrMc14dPdhDMG6uXRyUy2pbZnbaEYkGlq8hO47xFA/az1ocKF2LbkkUQB5Me 2IWHfz6l6/t/ZZA== X-Received: from pgbci10.prod.google.com ([2002:a05:6a02:200a:b0:c74:12b9:ee05]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:33a3:b0:39b:e710:e2ec with SMTP id adf61e73a8af0-39be7111013mr9375027637.43.1774310342018; Mon, 23 Mar 2026 16:59:02 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:06 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-15-dmatlack@google.com> Subject: [PATCH v3 14/24] selftests/liveupdate: Move luo_test_utils.* into a reusable library From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Move luo_test_utils.[ch] into a lib/ directory and pull the rules to build them out into a separate make script. This will enable these utilities to be also built by and used within other selftests (such as VFIO) in subsequent commits. No functional change intended. Signed-off-by: Vipin Sharma Co-developed-by: David Matlack Signed-off-by: David Matlack --- tools/testing/selftests/liveupdate/.gitignore | 1 + tools/testing/selftests/liveupdate/Makefile | 14 ++++--------- .../include/libliveupdate.h} | 8 ++++---- .../selftests/liveupdate/lib/libliveupdate.mk | 20 +++++++++++++++++++ .../{luo_test_utils.c =3D> lib/liveupdate.c} | 2 +- .../selftests/liveupdate/luo_kexec_simple.c | 2 +- .../selftests/liveupdate/luo_multi_session.c | 2 +- 7 files changed, 32 insertions(+), 17 deletions(-) rename tools/testing/selftests/liveupdate/{luo_test_utils.h =3D> lib/inclu= de/libliveupdate.h} (87%) create mode 100644 tools/testing/selftests/liveupdate/lib/libliveupdate.mk rename tools/testing/selftests/liveupdate/{luo_test_utils.c =3D> lib/liveu= pdate.c} (99%) diff --git a/tools/testing/selftests/liveupdate/.gitignore b/tools/testing/= selftests/liveupdate/.gitignore index 661827083ab6..18a0c7036cf3 100644 --- a/tools/testing/selftests/liveupdate/.gitignore +++ b/tools/testing/selftests/liveupdate/.gitignore @@ -3,6 +3,7 @@ !/**/ !*.c !*.h +!*.mk !*.sh !.gitignore !config diff --git a/tools/testing/selftests/liveupdate/Makefile b/tools/testing/se= lftests/liveupdate/Makefile index 080754787ede..a060cc21f27f 100644 --- a/tools/testing/selftests/liveupdate/Makefile +++ b/tools/testing/selftests/liveupdate/Makefile @@ -1,7 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only =20 -LIB_C +=3D luo_test_utils.c - TEST_GEN_PROGS +=3D liveupdate =20 TEST_GEN_PROGS_EXTENDED +=3D luo_kexec_simple @@ -10,25 +8,21 @@ TEST_GEN_PROGS_EXTENDED +=3D luo_multi_session TEST_FILES +=3D do_kexec.sh =20 include ../lib.mk +include lib/libliveupdate.mk =20 CFLAGS +=3D $(KHDR_INCLUDES) CFLAGS +=3D -Wall -O2 -Wno-unused-function CFLAGS +=3D -MD =20 -LIB_O :=3D $(patsubst %.c, $(OUTPUT)/%.o, $(LIB_C)) TEST_O :=3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) TEST_O +=3D $(patsubst %, %.o, $(TEST_GEN_PROGS_EXTENDED)) =20 -TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(LIB_O)) +TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(LIBLIVEUPDATE_O)) TEST_DEP_FILES +=3D $(patsubst %.o, %.d, $(TEST_O)) -include $(TEST_DEP_FILES) =20 -$(LIB_O): $(OUTPUT)/%.o: %.c - $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@ - -$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/%: %.o $(LIB_O) - $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIB_O) $(LDLIB= S) -o $@ +$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/%: %.o $(LIBLIVEUP= DATE_O) + $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIBLIVEUPDATE_= O) $(LDLIBS) -o $@ =20 -EXTRA_CLEAN +=3D $(LIB_O) EXTRA_CLEAN +=3D $(TEST_O) EXTRA_CLEAN +=3D $(TEST_DEP_FILES) diff --git a/tools/testing/selftests/liveupdate/luo_test_utils.h b/tools/te= sting/selftests/liveupdate/lib/include/libliveupdate.h similarity index 87% rename from tools/testing/selftests/liveupdate/luo_test_utils.h rename to tools/testing/selftests/liveupdate/lib/include/libliveupdate.h index 90099bf49577..4390a2737930 100644 --- a/tools/testing/selftests/liveupdate/luo_test_utils.h +++ b/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h @@ -7,13 +7,13 @@ * Utility functions for LUO kselftests. */ =20 -#ifndef LUO_TEST_UTILS_H -#define LUO_TEST_UTILS_H +#ifndef SELFTESTS_LIVEUPDATE_LIB_LIVEUPDATE_H +#define SELFTESTS_LIVEUPDATE_LIB_LIVEUPDATE_H =20 #include #include #include -#include "../kselftest.h" +#include "../../../kselftest.h" =20 #define LUO_DEVICE "/dev/liveupdate" =20 @@ -41,4 +41,4 @@ typedef void (*luo_test_stage2_fn)(int luo_fd, int state_= session_fd); int luo_test(int argc, char *argv[], const char *state_session_name, luo_test_stage1_fn stage1, luo_test_stage2_fn stage2); =20 -#endif /* LUO_TEST_UTILS_H */ +#endif /* SELFTESTS_LIVEUPDATE_LIB_LIVEUPDATE_H */ diff --git a/tools/testing/selftests/liveupdate/lib/libliveupdate.mk b/tool= s/testing/selftests/liveupdate/lib/libliveupdate.mk new file mode 100644 index 000000000000..fffd95b085b6 --- /dev/null +++ b/tools/testing/selftests/liveupdate/lib/libliveupdate.mk @@ -0,0 +1,20 @@ +include $(top_srcdir)/scripts/subarch.include +ARCH ?=3D $(SUBARCH) + +LIBLIVEUPDATE_SRCDIR :=3D $(selfdir)/liveupdate/lib + +LIBLIVEUPDATE_C :=3D liveupdate.c + +LIBLIVEUPDATE_OUTPUT :=3D $(OUTPUT)/libliveupdate + +LIBLIVEUPDATE_O :=3D $(patsubst %.c, $(LIBLIVEUPDATE_OUTPUT)/%.o, $(LIBLIV= EUPDATE_C)) + +LIBLIVEUPDATE_O_DIRS :=3D $(shell dirname $(LIBLIVEUPDATE_O) | uniq) +$(shell mkdir -p $(LIBLIVEUPDATE_O_DIRS)) + +CFLAGS +=3D -I$(LIBLIVEUPDATE_SRCDIR)/include + +$(LIBLIVEUPDATE_O): $(LIBLIVEUPDATE_OUTPUT)/%.o : $(LIBLIVEUPDATE_SRCDIR)/= %.c + $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@ + +EXTRA_CLEAN +=3D $(LIBLIVEUPDATE_OUTPUT) diff --git a/tools/testing/selftests/liveupdate/luo_test_utils.c b/tools/te= sting/selftests/liveupdate/lib/liveupdate.c similarity index 99% rename from tools/testing/selftests/liveupdate/luo_test_utils.c rename to tools/testing/selftests/liveupdate/lib/liveupdate.c index 3c8721c505df..60121873f685 100644 --- a/tools/testing/selftests/liveupdate/luo_test_utils.c +++ b/tools/testing/selftests/liveupdate/lib/liveupdate.c @@ -21,7 +21,7 @@ #include #include =20 -#include "luo_test_utils.h" +#include =20 int luo_open_device(void) { diff --git a/tools/testing/selftests/liveupdate/luo_kexec_simple.c b/tools/= testing/selftests/liveupdate/luo_kexec_simple.c index d7ac1f3dc4cb..786ac93b9ae3 100644 --- a/tools/testing/selftests/liveupdate/luo_kexec_simple.c +++ b/tools/testing/selftests/liveupdate/luo_kexec_simple.c @@ -8,7 +8,7 @@ * across a single kexec reboot. */ =20 -#include "luo_test_utils.h" +#include =20 #define TEST_SESSION_NAME "test-session" #define TEST_MEMFD_TOKEN 0x1A diff --git a/tools/testing/selftests/liveupdate/luo_multi_session.c b/tools= /testing/selftests/liveupdate/luo_multi_session.c index 0ee2d795beef..aac24a5f5ce3 100644 --- a/tools/testing/selftests/liveupdate/luo_multi_session.c +++ b/tools/testing/selftests/liveupdate/luo_multi_session.c @@ -9,7 +9,7 @@ * files. */ =20 -#include "luo_test_utils.h" +#include =20 #define SESSION_EMPTY_1 "multi-test-empty-1" #define SESSION_EMPTY_2 "multi-test-empty-2" --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D01F1390C86 for ; Mon, 23 Mar 2026 23:59:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310351; cv=none; b=b+pq4AZfdgv1D2/eOb1qqMdtGeysywICZbRTLBecSo+wEoODbRtRetyahnF43gJ9J0yPLAFl7EvLrL1lwrf5nN+jxtDWc+xCyWvZsEuwEYfCuQvJMRA7uJr0pVv1oL7TjnuJD9EiBGoltStCPB13b1O7MdrFCqLbAjvL7XjX7HA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310351; c=relaxed/simple; bh=Pal8NLhSzj5NFCdkK3kSKC8PcLcfaBEm4Hex4yzxfHQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LQ2MfDDR0SsnSKZmw94NadP5Lxar/URY1bCOnkplxSdwppTYvOVVBGDJf2E14JlW6WU05cPWg7FREYqau+1LIKsPLzAwyTx+nEEOL2GYphTkTZFlWRQdpbKMcJTiW2Td65FPk1pI6NGH9rdol7GvdDbfLhpZSpJ/VfDOjFM+uBs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Z2t4gcwi; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Z2t4gcwi" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2b05a3c2421so8471915ad.1 for ; Mon, 23 Mar 2026 16:59:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310344; x=1774915144; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=d5dX1H4XkIVUdogReT0P+xEIybXWqypp015VM5/RpRc=; b=Z2t4gcwiYdiRpMKelEM5gq77MCDju8RQIdsyXTs2OzZCyqLxRg7gT1yeIU5DlOKpR4 yY+RKvGB+tdPDY33SuRwansSOKKKNrcZmF1pXDfKjK62tuwUFitbvm7sKACE+qXmkcAU AmMhWNFv4ZL2ONeOg7ceMa59EGtNu23uTZXelEqdsDPr3XM/Lvdk+glOkq7wUxTbON5C m1zVxMqHK8zZEz8Vp80w5wDmZ5Zmx9HHdtrsT3C1SoaXgAJIgbnJoyID0UkCgGgXCcHz bZ2lob6kHuPGGzUQkdZbXgGiijYLOyet3PpHGUEQwue4/F3+Y2V2ccc8DOf2mqfM9Q4n jCjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310344; x=1774915144; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=d5dX1H4XkIVUdogReT0P+xEIybXWqypp015VM5/RpRc=; b=GaIIpldVAtFlhpKNIij53LsVTTCoWxQbI6M50++VpS4im4VzI8PqMSX0FswxnEZlPQ 2t00hTE38/6pkSgc4erGVc0TAKlmIXZI7p074T0i+iNoetA0NyyxncPlfCyoyTO5PiG3 N9yF7Tj+kUdHU/XBv7Gt7WXAkdG7jwmTWLIFMp/0DnLAraLSttk9WkInl9aOiO6+qr/8 vBnumWOLot7Lr4V0iQVJVuytJrgDRrjZJphVkeJr7ktHU7hPtAIQXaqXxiTiUfa3+hYR YhXzuOEdeKD8BHCIT1pEtvNEUNRpjlCXP6DyLeuZ+Wo/JvCbIagTwJHO/XQTKZ6vAMLn +FRQ== X-Forwarded-Encrypted: i=1; AJvYcCW1tcp+oMi7tj2/HXDuEgNuNWB1RJLJGFegeH4NO9nSsZRuy95nKEH5eO4p9JOamjNGXJI0vRWoq++k/9E=@vger.kernel.org X-Gm-Message-State: AOJu0Yz7/AylUCdENhsgGjGZlq5jHA9O0uiamL1/vbXWONUWIbgjM/xn yEMnO4GpQq8Z2Vfnbynlv4F+OkQId0HI3+w+tZG+Vsb4fZYmr7pUUYOZG7Mks5+IFYNxDZ7Y02i DRZyb/WqGaA4e5A== X-Received: from plpn15.prod.google.com ([2002:a17:902:968f:b0:2ae:c5aa:fcd7]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:124f:b0:2b0:54dc:62c with SMTP id d9443c01a7336-2b0827c9a7cmr151447295ad.48.1774310343811; Mon, 23 Mar 2026 16:59:03 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:07 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-16-dmatlack@google.com> Subject: [PATCH v3 15/24] selftests/liveupdate: Add helpers to preserve/retrieve FDs From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Add helper functions to preserve and retrieve file descriptors from an LUO session. These will be used be used in subsequent commits to preserve FDs other than memfd. No functional change intended. Signed-off-by: Vipin Sharma Co-developed-by: David Matlack Signed-off-by: David Matlack --- .../liveupdate/lib/include/libliveupdate.h | 3 ++ .../selftests/liveupdate/lib/liveupdate.c | 41 +++++++++++++++---- 2 files changed, 35 insertions(+), 9 deletions(-) diff --git a/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h= b/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h index 4390a2737930..2b04b3256382 100644 --- a/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h +++ b/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h @@ -26,6 +26,9 @@ int luo_create_session(int luo_fd, const char *name); int luo_retrieve_session(int luo_fd, const char *name); int luo_session_finish(int session_fd); =20 +int luo_session_preserve_fd(int session_fd, int fd, __u64 token); +int luo_session_retrieve_fd(int session_fd, __u64 token); + int create_and_preserve_memfd(int session_fd, int token, const char *data); int restore_and_verify_memfd(int session_fd, int token, const char *expect= ed_data); =20 diff --git a/tools/testing/selftests/liveupdate/lib/liveupdate.c b/tools/te= sting/selftests/liveupdate/lib/liveupdate.c index 60121873f685..3e070975a3ec 100644 --- a/tools/testing/selftests/liveupdate/lib/liveupdate.c +++ b/tools/testing/selftests/liveupdate/lib/liveupdate.c @@ -54,9 +54,35 @@ int luo_retrieve_session(int luo_fd, const char *name) return arg.fd; } =20 +int luo_session_preserve_fd(int session_fd, int fd, __u64 token) +{ + struct liveupdate_session_preserve_fd arg =3D { + .size =3D sizeof(arg), + .fd =3D fd, + .token =3D token, + }; + + if (ioctl(session_fd, LIVEUPDATE_SESSION_PRESERVE_FD, &arg)) + return -errno; + + return 0; +} + +int luo_session_retrieve_fd(int session_fd, __u64 token) +{ + struct liveupdate_session_retrieve_fd arg =3D { + .size =3D sizeof(arg), + .token =3D token, + }; + + if (ioctl(session_fd, LIVEUPDATE_SESSION_RETRIEVE_FD, &arg)) + return -errno; + + return arg.fd; +} + int create_and_preserve_memfd(int session_fd, int token, const char *data) { - struct liveupdate_session_preserve_fd arg =3D { .size =3D sizeof(arg) }; long page_size =3D sysconf(_SC_PAGE_SIZE); void *map =3D MAP_FAILED; int mfd =3D -1, ret =3D -1; @@ -75,9 +101,8 @@ int create_and_preserve_memfd(int session_fd, int token,= const char *data) snprintf(map, page_size, "%s", data); munmap(map, page_size); =20 - arg.fd =3D mfd; - arg.token =3D token; - if (ioctl(session_fd, LIVEUPDATE_SESSION_PRESERVE_FD, &arg) < 0) + ret =3D luo_session_preserve_fd(session_fd, mfd, token); + if (ret) goto out; =20 ret =3D 0; @@ -92,15 +117,13 @@ int create_and_preserve_memfd(int session_fd, int toke= n, const char *data) int restore_and_verify_memfd(int session_fd, int token, const char *expected_data) { - struct liveupdate_session_retrieve_fd arg =3D { .size =3D sizeof(arg) }; long page_size =3D sysconf(_SC_PAGE_SIZE); void *map =3D MAP_FAILED; int mfd =3D -1, ret =3D -1; =20 - arg.token =3D token; - if (ioctl(session_fd, LIVEUPDATE_SESSION_RETRIEVE_FD, &arg) < 0) - return -errno; - mfd =3D arg.fd; + mfd =3D luo_session_retrieve_fd(session_fd, token); + if (mfd < 0) + return mfd; =20 map =3D mmap(NULL, page_size, PROT_READ, MAP_SHARED, mfd, 0); if (map =3D=3D MAP_FAILED) --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9F1F390C84 for ; Mon, 23 Mar 2026 23:59:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310354; cv=none; b=QSlJPk1UEzhvgCYY5mUv3PEoqtBNBAlVmEy3CAA5VIr2cwYFDcSJc6MCrXNG9lzaQBVWjoHS3MQVNH9294M6Jjo+dy8hHJAnASQxO68rYTPi8OWXnmuY+iYE5VMn1n9DbaTzlLcNsHaaXjP8jrIazlqax8zsNEA3L8HeMHU4aow= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310354; c=relaxed/simple; bh=XYunvBN/JOFiTVeFj3WUZep3oT3AVgN42lf66K2Tnww=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RmIos0nvIq3/bK5NktvGBglhiUGudx6RfoofWo/DECeQpZEi0HPYyWRmZYK64mYfrGUY38tteopsKNriITCjDVXkkb0bsN+CRArdoZk9A5Pw2NAYWb4YCHTej9iHO62ArDa3EUAjPJfWmwdsd0m5l5WOFuzHjiqXS2GrwJb7/bU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=M3osV84X; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="M3osV84X" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-35a0998a441so29769270a91.2 for ; Mon, 23 Mar 2026 16:59:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310346; x=1774915146; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=bCulrz/+ZC3yY//mGMCNHFkV5P7dfGgY4CJSUX7daUA=; b=M3osV84Xk+p/MMKFxwwAMN5wu3Crzw6Z1XVO86jMGaE7SQreQW6vHZXBv4bbbkv6YF 8VTH5vWCCSYgICG7ZUB9hkKD0e8VGw5WRdw0/3wf9BgWcEeVA/LvlHCBWaZEzr+6z70F GC4JcmbZ3hTYr760jIYOqkFvCrHPfA+8ciQRYqL02OXzXb66zFAiYDOF7a6I2XaKJti6 zzrxbid2UmCaxEIDZ3Wzsp93wyJ3v8/f2gEYUFe/VQ171Dt1rHlsSQqdc/PyvJOkZEa2 8WOI22OTSTSnKVfQh0wBGEIUJn2l6l+lwYaisiy3BjYAreQQ3pFFFEljZSmgG8xHGyej RgRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310346; x=1774915146; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bCulrz/+ZC3yY//mGMCNHFkV5P7dfGgY4CJSUX7daUA=; b=F6LWjGQLeAqu9jP2HAX8B6xoH2ZHI0y/1FRU2/dswQHJrHKm+7QkI2WAxK4Cswk+6v 3KojudWDSMoDEhC1YSpI0srGowUExvaLuMfpJdl1DSFqrCAUo/JZ1Y6CLDYM+nGzCt+b BmcQvN4TXxs/JlgzaP1MFEnP2CEYS1OdqlrGtDy3tJBsEwd88bMeDssEGu9/Jfdczfm/ FRB/fMzSzgrXpfW/myYSNxYS1BDipBeQcnwVe5MfVsnc5pz5NMrEKIjPzmdO7zXHgk8I LTvQuy6D6cCFJh2jEeRi1PiSsryafSIiOF2gmSKAkT/0XR13AIaS9XxVZXdKeF4XwyXQ Cg/g== X-Forwarded-Encrypted: i=1; AJvYcCVRGbioQrFpemjYHj0OuqbflsQ0b7sIcEsoPdar0b9CvcqSqENEy4LSjDplS0A5Y6uZrullamOEIGUuIZI=@vger.kernel.org X-Gm-Message-State: AOJu0YylbrQiDQNBdpIMFdUoj1kgWW4sD0Gfrs6zRpjdUWPpFFNF4kpR HuGgPOt5gEnIBR3UHtnFmIhyzyVc2bl+6TRFYI36uW8RaxUOTeH3KJpVtXPG0F6quAd5wQLJ5NJ /sipmGjvzmaAhoQ== X-Received: from pjvd16.prod.google.com ([2002:a17:90a:d990:b0:35b:a305:76f5]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:15c2:b0:35b:e4f8:78e1 with SMTP id 98e67ed59e1d1-35be4f87c2bmr4832204a91.11.1774310345696; Mon, 23 Mar 2026 16:59:05 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:08 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-17-dmatlack@google.com> Subject: [PATCH v3 16/24] vfio: selftests: Build liveupdate library in VFIO selftests From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Import and build liveupdate selftest library in VFIO selftests. It allows to use liveupdate ioctls in VFIO selftests Signed-off-by: Vipin Sharma Signed-off-by: David Matlack --- tools/testing/selftests/vfio/Makefile | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 8e90e409e91d..7f3c94da289d 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -20,6 +20,7 @@ TEST_FILES +=3D scripts/setup.sh =20 include ../lib.mk include lib/libvfio.mk +include ../liveupdate/lib/libliveupdate.mk =20 CFLAGS +=3D -I$(top_srcdir)/tools/include CFLAGS +=3D -MD @@ -27,11 +28,15 @@ CFLAGS +=3D $(EXTRA_CFLAGS) =20 LDFLAGS +=3D -pthread =20 -$(TEST_GEN_PROGS): %: %.o $(LIBVFIO_O) - $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $< $(LIBVFIO_O) $(LDLIBS) -o $@ +LIBS_O :=3D $(LIBVFIO_O) +LIBS_O +=3D $(LIBLIVEUPDATE_O) + +$(TEST_GEN_PROGS): %: %.o $(LIBS_O) + $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIBS_O) $(LDLI= BS) -o $@ =20 TEST_GEN_PROGS_O =3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) -TEST_DEP_FILES =3D $(patsubst %.o, %.d, $(TEST_GEN_PROGS_O) $(LIBVFIO_O)) +TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(TEST_GEN_PROGS_O)) +TEST_DEP_FILES +=3D $(patsubst %.o, %.d, $(LIBS_O)) -include $(TEST_DEP_FILES) =20 EXTRA_CLEAN +=3D $(TEST_GEN_PROGS_O) $(TEST_DEP_FILES) --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 360FB3914EC for ; Mon, 23 Mar 2026 23:59:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310353; cv=none; b=oIxBIHMvQ6pOiYvQayxSKnLCxEXMY7AKkNyer7uEVNlA9TdaczDEVslnHQNraXJbpYNYOSmHr3uJ22se3Bxlb0cgbcZfqQC8jReJJ2RmJD8SWXYbm5QcV1xBgc1xkk6lwykXze3vLZQ61/NatlVA+jNk2N1PlayqC8fyMerGHqg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310353; c=relaxed/simple; bh=W7eZwT9DvKJLgaaduhhOROtuKExMtdW5rdefHIwBCqg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jtiEAnaloa5ZfOPBitaHFta+ik+7urDG93dWZxksRN9oI7pb6Ogysy4McePBtLoN3LO9iCVkqktX86gFBPMa9uyf02n8MNQBcjj1w8eZT3BnTnAR21Ebvevf0zposqy2apzYk7iFzFUp7xcvMzvnaxPxs1FrUE3UkpPzhCL4d10= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ACki2pio; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ACki2pio" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-35449510446so773000a91.0 for ; Mon, 23 Mar 2026 16:59:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310348; x=1774915148; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7enZU/vxLy/6ohUczG3HchuR4oREBV6X1oYMwMHYNIE=; b=ACki2pioHscddV/iJdsmploffmaE3aze8hj1+YgUKEGvf6eB2GnMzQlOUvvRWQHxMc +4bbK1vGmBiC7mKl6GgWk8B7kcvVI175RPyClQ9hOf9nS63znMFtB4jloSwEo/h1WgNl X2HAQRVgjuJMPkKMUt5InIcuyfPalXMljz6NpD+zy/DqFWzTQzkVk8caICEyY7QIVMvX pn3kPWZCLgOMaEhCZKkRdiB7xTBIoP1MXTSMR3iJD36iRpXYTtBiRNhVuVmArPiOAphy 4VkQyL+uUrJKBFGSM+2NpVHasdGAj4qQdJzS8M6FpAOvPcn5oHU39kiP6hWpcUzE8sUh tqsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310348; x=1774915148; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7enZU/vxLy/6ohUczG3HchuR4oREBV6X1oYMwMHYNIE=; b=oxK07W3u6fqDd+qE8OCVL2nzb8jw3p9UfQKbkWRuQDwiBJr6kub9gMqsDh/u20BwKO 4BWtEj0tnjGeUHuPC8nYMoelZq2hjuPmsHU6904oYy6EDcPLFaSg6s452ZzvOGXXNBgR pfydCo7cDVDZM/RTgnqVKnLHKFHnjsE9xr0qTD8Yk5kXdy9QaQcXhZzWIAlCJ4MWMNhw FvNUhZY06IxtwG1a00xvX4PHKOXoCLJnlqMwwBhfyLb6Jm/U5Jy87b4tqBX+qbCCJ4Gl FYHtFYQ2/LHMy3PD6GjgxurkYkdrv55YylKAK9Ud+Axz5bUyrIF8C9v4ciAmySn5oCu1 PEhg== X-Forwarded-Encrypted: i=1; AJvYcCWKzHYu4sKeLtDBKufNLVD667duYAOaLm3FJdIU1ZgRBe1mPwgttMWhEnM/kzpvbiSNJ3Q+9Pb6Onbb628=@vger.kernel.org X-Gm-Message-State: AOJu0YxRiDLmDfz+rYLdOmZeb69H83mZutU5TAcVDa14R8f+ufzWKOtz EOG6rELoAusY/kek2n9uSfQs6ALc+x8EKjIgciTogo7DfXPtIT0Uluo/6XHSUue5V41pQjbWbIM SuLFL2Swj/WENIA== X-Received: from pgbdo13.prod.google.com ([2002:a05:6a02:e8d:b0:bac:6acd:8182]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:6a15:b0:398:7855:1596 with SMTP id adf61e73a8af0-39bcea6ed94mr12817264637.10.1774310347510; Mon, 23 Mar 2026 16:59:07 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:09 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-18-dmatlack@google.com> Subject: [PATCH v3 17/24] vfio: selftests: Add Makefile support for TEST_GEN_PROGS_EXTENDED From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add Makefile support for TEST_GEN_PROGS_EXTENDED targets. These tests are not run by default. TEST_GEN_PROGS_EXTENDED will be used for Live Update selftests in subsequent commits. These selftests must be run manually because they require the user/runner to perform additional actions, such as kexec, during the test. Signed-off-by: David Matlack --- tools/testing/selftests/vfio/Makefile | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 7f3c94da289d..9d5e390a61b7 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -31,14 +31,17 @@ LDFLAGS +=3D -pthread LIBS_O :=3D $(LIBVFIO_O) LIBS_O +=3D $(LIBLIVEUPDATE_O) =20 -$(TEST_GEN_PROGS): %: %.o $(LIBS_O) +$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): %: %.o $(LIBS_O) $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIBS_O) $(LDLI= BS) -o $@ =20 -TEST_GEN_PROGS_O =3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) -TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(TEST_GEN_PROGS_O)) +TESTS_O :=3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) +TESTS_O +=3D $(patsubst %, %.o, $(TEST_GEN_PROGS_EXTENDED)) + +TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(TESTS_O)) TEST_DEP_FILES +=3D $(patsubst %.o, %.d, $(LIBS_O)) -include $(TEST_DEP_FILES) =20 -EXTRA_CLEAN +=3D $(TEST_GEN_PROGS_O) $(TEST_DEP_FILES) +EXTRA_CLEAN +=3D $(TESTS_O) +EXTRA_CLEAN +=3D $(TEST_DEP_FILES) =20 endif --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 655FC39184E for ; Mon, 23 Mar 2026 23:59:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310354; cv=none; b=mA7UPbd3njsQFx4GZCujA1JmVb9zk4xVqFzQTv+ivKuVgJ1a9dfl6k1h/REfKCia3Dy0XbSFfQdmvuLW462lUmVqNwSztSw5RTFO9pL7KTuDwp4PyGXFAMxqN7MoJ7Va7Lx+M5Z3h2nFuFzjUjoYOdTHtS+nrt7LjWwWJFiX1s4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310354; c=relaxed/simple; bh=T10nGtc1MQS8A8zTtf9tLNZQEznWmOM6Cyw8cPMZCmA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lD9PPDD7xk2+OtTfu8jeTtknVnyNmyo0wAjToMsNKXtVnZASCd8v3jR9EatJV04zix5WAoNDELF317Nxrs6gNgIk0aTSk4O2FxnMkQW9XaFYqck864JmGHthfPNlClclu5Dx8mC+LQtmi6bXnt3OGpRndYAl7nmd3kfZi1qPiiQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cjlBS0z9; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cjlBS0z9" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-35a1e725a8fso24245747a91.3 for ; Mon, 23 Mar 2026 16:59:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310350; x=1774915150; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=JUFryWFBPXxe6T+DTeCmqMtrNHQ62SSe5LgDU16cd+0=; b=cjlBS0z9u8LTaguUFDgZppTU+iOwkd5MFUWJGKcEUYmEgnpsVnxTZj7/fDl0JluwUd fepN76z/2z2L9cs42K0nzR4cjpQoK/4DWH00ko27wfIbR7l+vv0axGkwtBn9I9I8ffWl NfT+7G2BaDIPsLfidlx/J7J8KZEbdyh1TlYHi00ZnpU608kqX28EyxDCom7CFEiqE4LA CcsltM16jJKJnp3DU/6yhIb1ChZNlXcb8IEFUOMktSk9xOTmSgzPxwzAGAlYizXUg6sd Dh+HZvTInnJ7QlTQvo5vwKCjQpxVvSBCJFU065SaHn9Tdir1gJrFKG4AKdrW+0kxupVb GLsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310350; x=1774915150; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JUFryWFBPXxe6T+DTeCmqMtrNHQ62SSe5LgDU16cd+0=; b=CJQ2afFeBz2Os9btoPrGed4ftG0Ze0gUDqTicmX/CmDHw5ptnLlILRIU9cfsqj6vSt 10xqGH7dOLHazpbLY6ECJYMILhs5OQx7tuWb5/+L8EfQsEXXx2QKPYrdwkCaDXNZrzov qyC5xvTzbiADkaUeJ1eeb2IvCPWEUq8fY166GpVlOwX2Znu6r/h5NsHvnOXWygOQz34q ekcJaNzhkW2yzcymwXCdsUGXGthQFMRuzpGXo4DOyFSzw4PmmsO9HS4oMp6/pT1MbyL3 Qw43Ucw7N5Rbrqi3Vtyz9hVxsAjqpHh6ckxngf6E2J2SQ53pmwbNKsL/73XKQM6Iimn8 ip3w== X-Forwarded-Encrypted: i=1; AJvYcCVAFUf0K9cbI3kNuHDmF2uBsWSJ+c67UjaTOG+0iQNndeP41s7Zx15cRfnhsADkOemSm0xngi3Q0aNBhKo=@vger.kernel.org X-Gm-Message-State: AOJu0Yxy+VuS+/4DuqOtljosRLU3Ou3a5oE/5U3qPSyZOGgso3RSYQNS gtE63scgWQKTSfSHWr0eL0W0oKwgc1xrkKGqAA3hnBhJexsi63tWa96pMCwbc55FM0EfXub6C/N 3jLkMJl6lGXdq+Q== X-Received: from pjub5.prod.google.com ([2002:a17:90a:cc05:b0:359:f1f1:7bcb]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:1c08:b0:35b:e519:213b with SMTP id 98e67ed59e1d1-35be5192186mr4242377a91.30.1774310349283; Mon, 23 Mar 2026 16:59:09 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:10 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-19-dmatlack@google.com> Subject: [PATCH v3 18/24] vfio: selftests: Add vfio_pci_liveupdate_uapi_test From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a selftest to exercise preserving a various VFIO files through /dev/liveupdate. Ensure that VFIO cdev device files can be preserved and everything else (group-based device files, group files, and container files) all fail. Signed-off-by: David Matlack --- tools/testing/selftests/vfio/Makefile | 1 + .../vfio/vfio_pci_liveupdate_uapi_test.c | 93 +++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_t= est.c diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 9d5e390a61b7..5b6e79593555 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -12,6 +12,7 @@ TEST_GEN_PROGS +=3D vfio_iommufd_setup_test TEST_GEN_PROGS +=3D vfio_pci_device_test TEST_GEN_PROGS +=3D vfio_pci_device_init_perf_test TEST_GEN_PROGS +=3D vfio_pci_driver_test +TEST_GEN_PROGS +=3D vfio_pci_liveupdate_uapi_test =20 TEST_FILES +=3D scripts/cleanup.sh TEST_FILES +=3D scripts/lib.sh diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c b= /tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c new file mode 100644 index 000000000000..1d89b08ab0a4 --- /dev/null +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include +#include + +static const char *device_bdf; + +FIXTURE(vfio_pci_liveupdate_uapi_test) { + int luo_fd; + int session_fd; + struct iommu *iommu; + struct vfio_pci_device *device; +}; + +FIXTURE_VARIANT(vfio_pci_liveupdate_uapi_test) { + const char *iommu_mode; +}; + +#define FIXTURE_VARIANT_ADD_IOMMU_MODE(_iommu_mode) \ +FIXTURE_VARIANT_ADD(vfio_pci_liveupdate_uapi_test, _iommu_mode) { \ + .iommu_mode =3D #_iommu_mode, \ +} + +FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(); +#undef FIXTURE_VARIANT_ADD_IOMMU_MODE + +FIXTURE_SETUP(vfio_pci_liveupdate_uapi_test) +{ + self->luo_fd =3D luo_open_device(); + ASSERT_GE(self->luo_fd, 0); + + self->session_fd =3D luo_create_session(self->luo_fd, "session"); + ASSERT_GE(self->session_fd, 0); + + self->iommu =3D iommu_init(variant->iommu_mode); + self->device =3D vfio_pci_device_init(device_bdf, self->iommu); +} + +FIXTURE_TEARDOWN(vfio_pci_liveupdate_uapi_test) +{ + vfio_pci_device_cleanup(self->device); + iommu_cleanup(self->iommu); + close(self->session_fd); + close(self->luo_fd); +} + +TEST_F(vfio_pci_liveupdate_uapi_test, preserve_device) +{ + int ret; + + ret =3D luo_session_preserve_fd(self->session_fd, self->device->fd, 0); + + /* Preservation should only be supported for VFIO cdev files. */ + ASSERT_EQ(ret, self->iommu->iommufd ? 0 : -ENOENT); +} + +TEST_F(vfio_pci_liveupdate_uapi_test, preserve_group_fails) +{ + int ret; + + if (self->iommu->iommufd) + SKIP(return, "iommufd-mode does not have group files"); + + ret =3D luo_session_preserve_fd(self->session_fd, self->device->group_fd,= 0); + ASSERT_EQ(ret, -ENOENT); +} + +TEST_F(vfio_pci_liveupdate_uapi_test, preserve_container_fails) +{ + int ret; + + if (self->iommu->iommufd) + SKIP(return, "iommufd-mode does not have container files"); + + ret =3D luo_session_preserve_fd(self->session_fd, self->iommu->container_= fd, 0); + ASSERT_EQ(ret, -ENOENT); +} + +int main(int argc, char *argv[]) +{ + int fd; + + fd =3D luo_open_device(); + if (fd < 0) + ksft_exit_skip("open(%s) failed: %s, skipping\n", + LUO_DEVICE, strerror(errno)); + + close(fd); + + device_bdf =3D vfio_selftests_get_bdf(&argc, argv); + return test_harness_run(argc, argv); +} --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A3C0391826 for ; Mon, 23 Mar 2026 23:59:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310354; cv=none; b=pfRsa4Y9uXwaP3L5Qom126IQu8oh5goAMnmXBRwehIkzvO8eqwwO6JEQEYdVDbFSoplEESK2kBlzbeI4kHksshNUs1UKi+oUU8x9jUqNEpEB+7wkEfA3BN/LgQGMLo824kTFtAPCRgJQnb0LRfbo0Mep9u068huYzWtEZmL0bsI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310354; c=relaxed/simple; bh=yR3Ih0bIEON9F49BtGTf8FaVcyrweJUGaTqAJ2uoch0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=g0xhoXJVw4kNzpW62KEOPMzg84a9+uMd9Y4IBy/oIiYH+GNzMo2Y8soirqjnaSXbjxjPnLq9+lnWURuoeuugfXIQ/azu0PH+IL0WiBu+qkVbB2qIiSDZoguicv1AJAKlxmCIhObu/omfYqhXNEIB7Juc8b4ClaX35/bztaaxNbg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RIfbp0rT; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RIfbp0rT" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2aed1beaa73so62839335ad.2 for ; Mon, 23 Mar 2026 16:59:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310351; x=1774915151; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uoPb6rPvbQAZiMMCvtGaLSKACFZ3kOlM4xYp0I2xeaE=; b=RIfbp0rTKJcMcrhasbVrzfep0+zVqyFCNmoWpzqELIgZvEmTe+9JjWuuAxRchjd2cK TYAj3ajNFhaUwssfX7fC7bmqAXoAUJL/oD82uIc8nm0otWUpge1E92lubjtpL7/ipEaC lKrFX4pZY5JjBxh/ZIzGXPbmKxNx7a3HyIs6zW9AKHM0GYQc0Xi0sWnjQxXzttMiU6Ms mZTXF+vyKYfbyDdTj5MVF6tuQL1Zeo1LpCqxLNZncZR3ttnMDEnNCrdq2UYFkOgdKZlW F2UYapKnb0c9oG35YiifRvODTAp3+sD6nN1ryHX5Ndjfx+vqSdJRB/HZw1FmHiXtqurt wRAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310351; x=1774915151; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uoPb6rPvbQAZiMMCvtGaLSKACFZ3kOlM4xYp0I2xeaE=; b=MESwxMA2MPvk7eIpflBoEJUdqVpbKXoR6DcN9IJXy4X/uuAeE1TkuK4BIRvp4RZXQ7 EX7P0tnj2LMxo7NZGUB+DC5PsH+QAZgO65K18s6xWvu9EkfrD5ZBxfvls7qPjfT+nilm fiIrESaPvDmMi8MqBsq2l1Fwc6Dwk5YASmo1i3dvHmg46aj32mOZ0NmExPtbNsEX8aqo LC8tacbz2MZvRgZh4FIitU4PCTC42RnSRiB14+elaXtcERPJ6O3nwYHsCOfogow1L5hl VuBVNvkFpTkVo975IEBZ1KfJYXfIXX8McKHUWmz9bHyiuiokfKnU7cd5BWyHk6y+S5IE fYqQ== X-Forwarded-Encrypted: i=1; AJvYcCWy66xvrLdrvgrdvsGXLVoSfYy5L3o5Ayc9iTI3+wVhMVC2U51YgEJv/ljOEqZTaKWtB/S1rKYCHU1e1Ag=@vger.kernel.org X-Gm-Message-State: AOJu0YzALz4x3N/IYQk8JK6dxl+VXhdmTL10Fe/YnEBzLHo4LPhALIuX Kc+M1SYE758n8ZUDUE8hKfMfxXCeJVnjXkaObaxfrJjNz1yYJrpexshQJ+SkukCWyu/9/FpUNJl NGATJJQyZak1ZLQ== X-Received: from pga26.prod.google.com ([2002:a05:6a02:4f9a:b0:c6e:28c3:dd5c]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6300:218a:b0:39b:81bf:15e3 with SMTP id adf61e73a8af0-39bcec29166mr12126845637.66.1774310351170; Mon, 23 Mar 2026 16:59:11 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:11 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-20-dmatlack@google.com> Subject: [PATCH v3 19/24] vfio: selftests: Initialize vfio_pci_device using a VFIO cdev FD From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Use the given VFIO cdev FD to initialize vfio_pci_device in VFIO selftests. Add the assertion to make sure that passed cdev FD is not used with legacy VFIO APIs. If VFIO cdev FD is provided then do not open the device instead use the FD for any interaction with the device. This API will allow to write selftests where VFIO device FD is preserved using liveupdate and retrieved later using liveupdate ioctl after kexec. Signed-off-by: Vipin Sharma Co-developed-by: David Matlack Signed-off-by: David Matlack --- .../lib/include/libvfio/vfio_pci_device.h | 3 ++ .../selftests/vfio/lib/vfio_pci_device.c | 33 ++++++++++++++----- 2 files changed, 27 insertions(+), 9 deletions(-) diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_devi= ce.h b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h index 2858885a89bb..896dfde88118 100644 --- a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +++ b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h @@ -38,6 +38,9 @@ struct vfio_pci_device { #define dev_info(_dev, _fmt, ...) printf("%s: " _fmt, (_dev)->bdf, ##__VA_= ARGS__) #define dev_err(_dev, _fmt, ...) fprintf(stderr, "%s: " _fmt, (_dev)->bdf,= ##__VA_ARGS__) =20 +struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, + struct iommu *iommu, + int device_fd); struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu= *iommu); void vfio_pci_device_cleanup(struct vfio_pci_device *device); =20 diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/tes= ting/selftests/vfio/lib/vfio_pci_device.c index 4e5871f1ebc3..e9215c712cda 100644 --- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c +++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c @@ -340,19 +340,27 @@ static void vfio_device_attach_iommufd_pt(int device_= fd, u32 pt_id) ioctl_assert(device_fd, VFIO_DEVICE_ATTACH_IOMMUFD_PT, &args); } =20 -static void vfio_pci_iommufd_setup(struct vfio_pci_device *device, const c= har *bdf) +static void vfio_pci_iommufd_setup(struct vfio_pci_device *device, + const char *bdf, int device_fd) { - const char *cdev_path =3D vfio_pci_get_cdev_path(bdf); + const char *cdev_path; =20 - device->fd =3D open(cdev_path, O_RDWR); - VFIO_ASSERT_GE(device->fd, 0); - free((void *)cdev_path); + if (device_fd >=3D 0) { + device->fd =3D device_fd; + } else { + cdev_path =3D vfio_pci_get_cdev_path(bdf); + device->fd =3D open(cdev_path, O_RDWR); + VFIO_ASSERT_GE(device->fd, 0); + free((void *)cdev_path); + } =20 vfio_device_bind_iommufd(device->fd, device->iommu->iommufd); vfio_device_attach_iommufd_pt(device->fd, device->iommu->ioas_id); } =20 -struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu= *iommu) +struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, + struct iommu *iommu, + int device_fd) { struct vfio_pci_device *device; =20 @@ -363,10 +371,12 @@ struct vfio_pci_device *vfio_pci_device_init(const ch= ar *bdf, struct iommu *iomm device->iommu =3D iommu; device->bdf =3D bdf; =20 - if (iommu->mode->container_path) + if (iommu->mode->container_path) { + VFIO_ASSERT_EQ(device_fd, -1); vfio_pci_container_setup(device, bdf); - else - vfio_pci_iommufd_setup(device, bdf); + } else { + vfio_pci_iommufd_setup(device, bdf, device_fd); + } =20 vfio_pci_device_setup(device); vfio_pci_driver_probe(device); @@ -374,6 +384,11 @@ struct vfio_pci_device *vfio_pci_device_init(const cha= r *bdf, struct iommu *iomm return device; } =20 +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu= *iommu) +{ + return __vfio_pci_device_init(bdf, iommu, /*device_fd=3D*/-1); +} + void vfio_pci_device_cleanup(struct vfio_pci_device *device) { int i; --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB40B3932D0 for ; Mon, 23 Mar 2026 23:59:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310356; cv=none; b=DKlDSYvEW/szzVSmPq/HMm5iyAInM1VAQvPyZMZDp9ckNv1ZPO4tMjWk9rhyVWhUNTQDS3V15s7O548/D02KAneRACpo/MtLAwXhBGeJ+NBCrKmJ6acB8IWzpguAqTZdglJFMLJBn0XU9Nj+fluZ3kGEpynlACNfavGVok2jbH0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310356; c=relaxed/simple; bh=0srt54TVUaweZ/O58H7olAoGMtQIEEkzyNB+t5cltrc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=V9by2NumQEaOryv5TC1xtr8bdJ1TPyOfo1QCJrT0lTSl9Xix4vNrXrXy4wvBVKFTgEi4k2RGo0NMuvmi2DfG4tKLryOL3tdTskVyL89NmwbN6nq71rztoqFwm4R5wStOvmffwJDiK71Sr2huv1xHCvt/gBMSBnf47q1GZQe6HN0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Ydxndrtg; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Ydxndrtg" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-35a1e725a8fso24246147a91.3 for ; Mon, 23 Mar 2026 16:59:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310353; x=1774915153; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=IXP5xRG4iZUBSse1Xd5a8GnmNvg4u/BwNl26bCYdYz4=; b=YdxndrtguqXxAoH0MXC+eUfZh0H8+uraaV2+6fllN9EgSot7lfkQoAHcT3s8rtD088 6jZCWy/K1DQXDID45pjWrAOIBe7JzEI+Cuk+Eqd75EfReeXBVZTbfP1d/mwho3ZxBRP9 7rMNKR4BhdPRHOkIJ1TWqs+SPuu7xPpGp7EOwxexPHeaED+SY7ePoE5i1RvslAaOOAna Ksio76e+eiQ62JPyCbme4bqzYYDQPRlMatXZeQ3xlT+9dS3fwywTAhIte3LyAaoWGs6/ 2PKbhWfkdhla42KV8DUoNVHdvhfXsz9YCc+Pgz1b1vDir6vBDs10QeT+1s4IOII3QZJL 56bQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310353; x=1774915153; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IXP5xRG4iZUBSse1Xd5a8GnmNvg4u/BwNl26bCYdYz4=; b=ZTP7G+EiPXsH83Ut4WSFdAP2huCsneuQhb5Ba4fMulWUT65SurHqMyrxDGok6/HHUJ JwuKiedt0ivkg3zXUtMqb1tkIlg1mRCJdyHWLZUD7Ix0QttsjdlowIIB5XYZSx+OTAhp Qgi29gHtyagM+9nlgqVdMq5z3OI73FXjHthA7rr6XuTvjuOvGPBTyGZ16o79lKOy/je2 BQDWK5zhmLQgRVjVqfis3DLCljiQrUQfJEvbJtqa+cpWLwur9S9JxUaNsuUlNtI/bPW9 UDVxTVkT4c3yW9mpIbnxtmgnj00BcayUjXP3G+2InwupK5RAPCYvwYtLaStHuhE9MTfM QOpg== X-Forwarded-Encrypted: i=1; AJvYcCXhklKgO+RTsJZ+9r+63l6auaNV+gYnWcsUwEeY4NYooHz601woy33+eq+PJzk7phEjIlkw3MpFRdrApFk=@vger.kernel.org X-Gm-Message-State: AOJu0YzlDDt9g3D7QafuKkl1kKXlWKL1KoKM8gWnhJGB77O6S7x+yUgQ 3dBGW/Gde4/W2Yvbyzk1KtuO//YWZf65tAV68YXsNCBLuaTfnWQhK9O7ugS9RK2gdVig3JEt7AC 6tywD5f8PaGtEQw== X-Received: from pjbil12.prod.google.com ([2002:a17:90b:164c:b0:35b:9d0e:7821]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:d406:b0:35b:9c13:30cc with SMTP id 98e67ed59e1d1-35bd2d3fbffmr11092911a91.23.1774310352922; Mon, 23 Mar 2026 16:59:12 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:12 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-21-dmatlack@google.com> Subject: [PATCH v3 20/24] vfio: selftests: Add vfio_pci_liveupdate_kexec_test From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Add a selftest to exercise preserving a vfio-pci device across a Live Update. For now the test is extremely simple and just verifies that the device file can be preserved and retrieved. In the future this test will be extended to verify more parts about device preservation as they are implemented. This test is added to TEST_GEN_PROGS_EXTENDED since it must be run manually along with a kexec. To run this test manually: $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 1 00= 00:00:04.0 $ kexec ... # NOTE: Exact method will be distro-dependent $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 2 00= 00:00:04.0 The second call to setup.sh is necessary because preserved devices are not bound to a driver after Live Update. Such devices must be manually bound by userspace after Live Update via driver_override. This test is considered passing if all commands exit with 0. Signed-off-by: Vipin Sharma Co-developed-by: David Matlack Signed-off-by: David Matlack --- tools/testing/selftests/vfio/Makefile | 4 + .../vfio/vfio_pci_liveupdate_kexec_test.c | 89 +++++++++++++++++++ 2 files changed, 93 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_= test.c diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 5b6e79593555..792c4245d4f7 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -14,6 +14,10 @@ TEST_GEN_PROGS +=3D vfio_pci_device_init_perf_test TEST_GEN_PROGS +=3D vfio_pci_driver_test TEST_GEN_PROGS +=3D vfio_pci_liveupdate_uapi_test =20 +# This test must be run manually since it requires the user/automation to +# perform a kexec during the test. +TEST_GEN_PROGS_EXTENDED +=3D vfio_pci_liveupdate_kexec_test + TEST_FILES +=3D scripts/cleanup.sh TEST_FILES +=3D scripts/lib.sh TEST_FILES +=3D scripts/run.sh diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c = b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c new file mode 100644 index 000000000000..15b3e3af91d1 --- /dev/null +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include + +static const char *device_bdf; + +static char state_session[LIVEUPDATE_SESSION_NAME_LENGTH]; +static char device_session[LIVEUPDATE_SESSION_NAME_LENGTH]; + +enum { + STATE_TOKEN, + DEVICE_TOKEN, +}; + +static void before_kexec(int luo_fd) +{ + struct vfio_pci_device *device; + struct iommu *iommu; + int session_fd; + int ret; + + iommu =3D iommu_init("iommufd"); + device =3D vfio_pci_device_init(device_bdf, iommu); + + create_state_file(luo_fd, state_session, STATE_TOKEN, /*next_stage=3D*/2); + + session_fd =3D luo_create_session(luo_fd, device_session); + VFIO_ASSERT_GE(session_fd, 0); + + printf("Preserving device in session\n"); + ret =3D luo_session_preserve_fd(session_fd, device->fd, DEVICE_TOKEN); + VFIO_ASSERT_EQ(ret, 0); + + close(luo_fd); + daemonize_and_wait(); +} + +static void after_kexec(int luo_fd, int state_session_fd) +{ + struct vfio_pci_device *device; + struct iommu *iommu; + int session_fd; + int device_fd; + int stage; + + restore_and_read_stage(state_session_fd, STATE_TOKEN, &stage); + VFIO_ASSERT_EQ(stage, 2); + + session_fd =3D luo_retrieve_session(luo_fd, device_session); + VFIO_ASSERT_GE(session_fd, 0); + + printf("Finishing the session before retrieving the device (should fail)\= n"); + VFIO_ASSERT_NE(luo_session_finish(session_fd), 0); + + printf("Retrieving the device FD from LUO\n"); + device_fd =3D luo_session_retrieve_fd(session_fd, DEVICE_TOKEN); + VFIO_ASSERT_GE(device_fd, 0); + + printf("Finishing the session before binding to iommufd (should fail)\n"); + VFIO_ASSERT_NE(luo_session_finish(session_fd), 0); + + printf("Binding the device to an iommufd and setting it up\n"); + iommu =3D iommu_init("iommufd"); + + /* + * This will invoke various ioctls on device_fd such as + * VFIO_DEVICE_GET_INFO. So this is a decent sanity test + * that LUO actually handed us back a valid VFIO device + * file and not something else. + */ + device =3D __vfio_pci_device_init(device_bdf, iommu, device_fd); + + printf("Finishing the session\n"); + VFIO_ASSERT_EQ(luo_session_finish(session_fd), 0); + + vfio_pci_device_cleanup(device); + iommu_cleanup(iommu); +} + +int main(int argc, char *argv[]) +{ + device_bdf =3D vfio_selftests_get_bdf(&argc, argv); + + sprintf(device_session, "device-%s", device_bdf); + sprintf(state_session, "state-%s", device_bdf); + + return luo_test(argc, argv, state_session, before_kexec, after_kexec); +} --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A339A3921DC for ; Mon, 23 Mar 2026 23:59:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310357; cv=none; b=rmiU93EeQcZD/AwLQCcoGbfxSb/TaMHwJeHnIl5wOqzCY0pJRlx4T+meBXv6/RnrynzUz1F+hMhd2fwNwmtb+aXYoev4GGJqq1/Dmxlx4+b5K2Dmy8FilQjlGqHhWf06BLvR7YY9O1xj2I3cQtncBtuhn3SRPYAZmcCmvbM6nVE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310357; c=relaxed/simple; bh=WyJEus1uJnXy8m0ACloG+/0RKFyldGkXc9VBNsLAa5g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BB5+lb6scbWiAdKLaoAVHfvy+AK6NVClQ9Wpx2tnD9DEa7/oUrKVx9yMc7EB17j14aHGvAqTJW2jNrnJIrJUMhwy5Q1NMWIgxZq4pDgNj0f+q9QMvVHRCD09C9cX3H6TOjPhmPzAIrjUhZlD9oL/ejqFH41HIAnasOnA5iyEwko= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ajyUNY48; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ajyUNY48" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-35b977ffbcdso744594a91.3 for ; Mon, 23 Mar 2026 16:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310355; x=1774915155; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=CreqaGlwQEQK+YRyK6iGxzdx6WSNNVSy3f0PqjRZciY=; b=ajyUNY48YtupU10DQPf/1QAF7odcTf4OF8sqdhpBiGvFQLNJH9cTPvJZtO1s1753Pv MA6FnBthhJ1bbcJ9DEcL1fHOVJYpURhQf9RnV9nBBEtOw2q7JJalwEzS5KJBw5i9G1bv wH9kkAB42BSECyvkJuWH1d0ZrBE7IA+LhG4IV0Qsnt9At1aianoU9jmO2LEDq0TXFV63 WmhdKVW+Ug3oPAFbJ56TUEtM9ayhjVwxySBQS6Lb7all80wZ/2FkD4NIKRMaAAk8nsZh jfl2Q+dkxavhG/jk/LI0A0YlzDaIYKs5arvXLkRi308kvupxkiG9jWawFCjaYa9vO7Oy +Bgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310355; x=1774915155; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CreqaGlwQEQK+YRyK6iGxzdx6WSNNVSy3f0PqjRZciY=; b=pnspPhFr0kVBTFaK7Cdn9zFzSbnSrj6O2n42obJ4cCAEukxWCZ9jqm66JHRLARl82J P7s9ROIj04ElwMDmetB3N892h8x4ub9V3GK0UjLN5nRu2rjzD558fIc1oXVJKEhu3JL2 gHxUOfjrfYh6BpyO5f7VaZVRuofo1yjzsAGnsOopGUtaP4XeH59xn87AWev4F2D3ZXZ+ f7OsMd3N/44jv9Ctf8VZwQjBFioMn8XfMb7L//PCCiSzg6VL9jlqFOaPkFwolUTQJdSI lQb8ZWuKij6tApPqgzRav5BgCiCu9rbrVwNdRsYDzH+kc0xqnN6C81bQtIXy6QMEg5qL wHtA== X-Forwarded-Encrypted: i=1; AJvYcCUpX/csS/0daX4+Zoo+2hkoQq1h69paf+lE9NisvMMWZrx/C6BAw8A/wxj87xtX6SHfdIKxXFv4RSW+edc=@vger.kernel.org X-Gm-Message-State: AOJu0Yz5+FoPDTHfwqWS5Bbz/yHKylra7E+MX7+bYdhDbcwrj089mvR5 20xKQ8DUxPIZjMt2AQBMJiB2wFfYTDb1gwJGCPx0UJcER06H+ARSfS95IyDcZ3PTZvbz67U98hG kiWpH6khZmqTbUA== X-Received: from pjat16.prod.google.com ([2002:a17:90a:d10:b0:35b:a2ad:add2]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3d85:b0:359:1063:6aed with SMTP id 98e67ed59e1d1-35bd2cb3e52mr11055132a91.22.1774310354638; Mon, 23 Mar 2026 16:59:14 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:13 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-22-dmatlack@google.com> Subject: [PATCH v3 21/24] vfio: selftests: Expose iommu_modes to tests From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose the list of iommu_modes to enable tests that want to iterate through all possible iommu modes. Signed-off-by: David Matlack --- tools/testing/selftests/vfio/lib/include/libvfio/iommu.h | 2 ++ tools/testing/selftests/vfio/lib/iommu.c | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h b/too= ls/testing/selftests/vfio/lib/include/libvfio/iommu.h index e9a3386a4719..4b9cbe262159 100644 --- a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h +++ b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h @@ -15,6 +15,8 @@ struct iommu_mode { unsigned long iommu_type; }; =20 +extern const struct iommu_mode iommu_modes[]; +extern const int nr_iommu_modes; extern const char *default_iommu_mode; =20 struct dma_region { diff --git a/tools/testing/selftests/vfio/lib/iommu.c b/tools/testing/selft= ests/vfio/lib/iommu.c index 035dac069d60..95a494f829d2 100644 --- a/tools/testing/selftests/vfio/lib/iommu.c +++ b/tools/testing/selftests/vfio/lib/iommu.c @@ -23,7 +23,7 @@ const char *default_iommu_mode =3D MODE_IOMMUFD; =20 /* Reminder: Keep in sync with FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(). */ -static const struct iommu_mode iommu_modes[] =3D { +const struct iommu_mode iommu_modes[] =3D { { .name =3D MODE_VFIO_TYPE1_IOMMU, .container_path =3D "/dev/vfio/vfio", @@ -49,6 +49,8 @@ static const struct iommu_mode iommu_modes[] =3D { }, }; =20 +const int nr_iommu_modes =3D ARRAY_SIZE(iommu_modes); + static const struct iommu_mode *lookup_iommu_mode(const char *iommu_mode) { int i; --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 304E13DB65E for ; Mon, 23 Mar 2026 23:59:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310359; cv=none; b=BEJxM4SaJbl1sGjyiZ7wbbGNyB8B2kxftW9vrHc7GJcg6pn+x86EEyMnA40ZwJO/IGDvVwttV/3rhdBX+i9TYykS/QKgGej12NOMYJ90yEvpq3jKAiu/7yOakPjMq+uEMs0Ap2qbVrg90jpS2cCI+4FV3TQ4m+jwm8CPktdx3Wg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310359; c=relaxed/simple; bh=lvaufN7TJ12UOehxU4H8lGXb0W9AqjyOq1Y2ZRN7GDg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jT4q4MJVTa2NZuUA/qOHOZ8G9WpYV3AN4RZEN16p66blRHzZRQhgc21AD6+LN51BXYO/QnfAJhLibodC9k8KV41WsH3o5+0xH64Zej2tJ7lrP4PDVKmuiXueQ7nzFg6GMVuEdpSmxYjCHLD3RI6sPHm1N/UDY4eFNFHT3gJNKZ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GblrU+QO; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GblrU+QO" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-35b9246248dso4896343a91.0 for ; Mon, 23 Mar 2026 16:59:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310356; x=1774915156; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=a6ai3Q1Y7nW49bYKBphuq9xPRlXveX2MorY71NenjMU=; b=GblrU+QOeyYAi7Uxv/yN8eKWfVSEz0OGT4lyLInM1TUTJ/LI/pmkHNUXoXcSo7QaPm j41HcWgHt3g3ciNZVQX1sPFf/vgrCyOauzcRNmNTzuKjDSyqviDpmLaCN0ouzT09B8HK olJ9h6f3TeT0yLIC6wuW2iT4qBN3tvtgLFweGU5W3Itdw9v5Dw7pD9BFwDSWeBVWdmbh clKw8feAwtbtqwZRz8J4ipqL3kPjkDZuljBTisBZwOD0jZAHf7QLXEeGlxP7RaMXPp9p 35qUj9BvhZ3JZtzlEa0f0TswHXHVGdN5METHCzFKt2ird+/BxNAM5Lr7g2BB/L25QZrj eqjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310356; x=1774915156; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=a6ai3Q1Y7nW49bYKBphuq9xPRlXveX2MorY71NenjMU=; b=i9nglOVvW6MSY+rG5LSd9jOaBrr5sz9MIDmWGiqtBKLvJvK97Ks5HAgZu5gQgzTZCu v1tacl9EwkcqvG6jKAP+Ek9PAnxemkZJHZ3FQr2k0oY30/vrLKoc2g5YvCttpuflIFik oBDZRv3+JEquTkoENCrpI4ua42tCpVo0XxiSjb2wshDflKDOAj5qV0YQPIvSV9ktDDqD lTCc/p5Kvpo/aVHG8ZkiwtfNSFU2BPtff2R5FZyU/RKjXeuB1k/Y2PZwtoy/7oCXJYfS Bt0OyctuH5X0cISPD7x0MWiDT6IKzel+EqdZVHgdyIQNXLVaBQiUIEmy3nUi8J3BT/UY fdIg== X-Forwarded-Encrypted: i=1; AJvYcCUWa46ByZ0tTcKiuG57/hOWPdfnwgS5KDHmKQDkFJuRbOxdgcB22AzdHDixrjGikuZfEMTX4dKwKaw2lao=@vger.kernel.org X-Gm-Message-State: AOJu0YyuSs04h/KcZYZ3rJxwCCeQJqHyyFnFOFY5ATkQFXI66iCYKl/0 dlDOGYIdvKmMyQb4x9yzNkRELzTlcWmkBHt+EOrgrrNQA7A+ZeswLF2F05MywWoXNiy5B/GvxQV yTMWj3bRx9T/sqg== X-Received: from pjxu14.prod.google.com ([2002:a17:90a:db4e:b0:35b:a2a4:de5a]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3891:b0:35b:e85a:865e with SMTP id 98e67ed59e1d1-35be85a8693mr5878913a91.33.1774310356298; Mon, 23 Mar 2026 16:59:16 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:14 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-23-dmatlack@google.com> Subject: [PATCH v3 22/24] vfio: selftests: Expose low-level helper routines for setting up struct vfio_pci_device From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose a few low-level helper routings for setting up vfio_pci_device structs. These routines will be used in a subsequent commit to assert that VFIO_GROUP_GET_DEVICE_FD fails under certain conditions. Signed-off-by: David Matlack --- .../lib/include/libvfio/vfio_pci_device.h | 5 +++ .../selftests/vfio/lib/vfio_pci_device.c | 33 +++++++++++++------ 2 files changed, 28 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_devi= ce.h b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h index 896dfde88118..2389c7698335 100644 --- a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +++ b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h @@ -125,4 +125,9 @@ static inline bool vfio_pci_device_match(struct vfio_pc= i_device *device, =20 const char *vfio_pci_get_cdev_path(const char *bdf); =20 +/* Low-level routines for setting up a struct vfio_pci_device */ +struct vfio_pci_device *vfio_pci_device_alloc(const char *bdf, struct iomm= u *iommu); +void vfio_pci_group_setup(struct vfio_pci_device *device); +void vfio_pci_iommu_setup(struct vfio_pci_device *device); + #endif /* SELFTESTS_VFIO_LIB_INCLUDE_LIBVFIO_VFIO_PCI_DEVICE_H */ diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/tes= ting/selftests/vfio/lib/vfio_pci_device.c index e9215c712cda..66ee268110e2 100644 --- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c +++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c @@ -220,7 +220,7 @@ static unsigned int vfio_pci_get_group_from_dev(const c= har *bdf) return group; } =20 -static void vfio_pci_group_setup(struct vfio_pci_device *device, const cha= r *bdf) +void vfio_pci_group_setup(struct vfio_pci_device *device) { struct vfio_group_status group_status =3D { .argsz =3D sizeof(group_status), @@ -228,7 +228,7 @@ static void vfio_pci_group_setup(struct vfio_pci_device= *device, const char *bdf char group_path[32]; int group; =20 - group =3D vfio_pci_get_group_from_dev(bdf); + group =3D vfio_pci_get_group_from_dev(device->bdf); snprintf(group_path, sizeof(group_path), "/dev/vfio/%d", group); =20 device->group_fd =3D open(group_path, O_RDWR); @@ -240,14 +240,12 @@ static void vfio_pci_group_setup(struct vfio_pci_devi= ce *device, const char *bdf ioctl_assert(device->group_fd, VFIO_GROUP_SET_CONTAINER, &device->iommu->= container_fd); } =20 -static void vfio_pci_container_setup(struct vfio_pci_device *device, const= char *bdf) +void vfio_pci_iommu_setup(struct vfio_pci_device *device) { struct iommu *iommu =3D device->iommu; unsigned long iommu_type =3D iommu->mode->iommu_type; int ret; =20 - vfio_pci_group_setup(device, bdf); - ret =3D ioctl(iommu->container_fd, VFIO_CHECK_EXTENSION, iommu_type); VFIO_ASSERT_GT(ret, 0, "VFIO IOMMU type %lu not supported\n", iommu_type); =20 @@ -257,8 +255,14 @@ static void vfio_pci_container_setup(struct vfio_pci_d= evice *device, const char * because the IOMMU type is already set. */ (void)ioctl(iommu->container_fd, VFIO_SET_IOMMU, (void *)iommu_type); +} =20 - device->fd =3D ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, bdf); +static void vfio_pci_container_setup(struct vfio_pci_device *device) +{ + vfio_pci_group_setup(device); + vfio_pci_iommu_setup(device); + + device->fd =3D ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, device->= bdf); VFIO_ASSERT_GE(device->fd, 0); } =20 @@ -358,9 +362,7 @@ static void vfio_pci_iommufd_setup(struct vfio_pci_devi= ce *device, vfio_device_attach_iommufd_pt(device->fd, device->iommu->ioas_id); } =20 -struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, - struct iommu *iommu, - int device_fd) +struct vfio_pci_device *vfio_pci_device_alloc(const char *bdf, struct iomm= u *iommu) { struct vfio_pci_device *device; =20 @@ -371,9 +373,20 @@ struct vfio_pci_device *__vfio_pci_device_init(const c= har *bdf, device->iommu =3D iommu; device->bdf =3D bdf; =20 + return device; +} + +struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, + struct iommu *iommu, + int device_fd) +{ + struct vfio_pci_device *device; + + device =3D vfio_pci_device_alloc(bdf, iommu); + if (iommu->mode->container_path) { VFIO_ASSERT_EQ(device_fd, -1); - vfio_pci_container_setup(device, bdf); + vfio_pci_container_setup(device); } else { vfio_pci_iommufd_setup(device, bdf, device_fd); } --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D70C93E1D1A for ; Mon, 23 Mar 2026 23:59:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310360; cv=none; b=FuVjf9cCZqKC/4P2hc2uglviZcCajghbO3wiej8TWVHLEXyTonkGVPBIYjBXHJXJ9jzQmLLTQWfcNL1IjCgqAdPF1d9jO4IGPRPQHfZnfdGBnzVd62ZSJHrU6+J08jWDgy2jTwc1JgdkIr/c2OhVZsFdqhr8nC6bBG1fT+/rtl0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310360; c=relaxed/simple; bh=dNUCR5flvD4enDeDr1lXIjIuSGLMofR2iyil2GVOefA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Ws3sdGmOysR6g5Q8/7oi7zvD/7ruI7kEwdt+lV91h3LWjpuIBk3yvP/Ibn23DwKIZBwBDQzjqGpX3kk2n+BGJZA6djr7xQUpnS5a/YCr+82mb4QzX284MRcplowSpan2I1Rky2O028zfz1sefyfnGkp289B81m1jTuV4zSeUM7c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=maKUtm8k; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="maKUtm8k" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-35641c14663so6066086a91.2 for ; Mon, 23 Mar 2026 16:59:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310358; x=1774915158; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sR2QH2u4Z2w+nCWLa7H/5qSEeXCYEvILvD5RlzuPkZI=; b=maKUtm8kSCkVi+sgPD2pb803+yCt0hRk/DQU6+sN6WsQuJYMe+sWUL+H6fu96DFnPN CX4NorpQMP3GnyLMruYy/xH5RqI9dQnXF9pyfxlaMVHzEIvtQtJ85eEXCfAH9p9IQ/G0 +KIUEFwPOMgg5NbZPEw2KSlXnTnOpz4+IhLbk5pRAwR4Qw3dsEdegtqTw8bcrYrVt2TX mU8SvDI7weiDlnPS6/xpLU7TJ19UvRHL+qXIU+loVW7rUuLNwfyGb5Bw+3BD8pdcNBiR wErn7MUmTrRKtsQEHzoGcBU3ns4QzvplGAGz2cqLG/bd4PEu3swcNkzovToy+/Lsimgp JevA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310358; x=1774915158; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sR2QH2u4Z2w+nCWLa7H/5qSEeXCYEvILvD5RlzuPkZI=; b=SJIpbqcf+m6pE81vbQ6YXuCFeBFZAA5jtm5364DQvXPuLONbdZW0b4wE4i+97WiK+L H+O1vHtMRfgT5VyhtmxHdNDHMbDnmQbDw8EKjp9IXHQS58LEmnd3Ia+JHOK7HleTtdrz EqYJ+dna9ykCqmN2kxRGv00l3QLDCYF3i57aF1nkYj9jCDr/PBLG3bMW+OVUHzYCsXV2 AGu26xv5XjAa2nj03txYYwwqJl3sKJj37fz5X0z6LBFs//OpHLdorxNutILWwji1jp8q hBSeGjT04kIxi8LaDUlCRY/Z3oML9YeCKf8UuUzWWqKA5r6MilloERGvpMfhjgTJjs2X PR9w== X-Forwarded-Encrypted: i=1; AJvYcCXKwQ7UqBmB7lMuSiu7C3bf2CKkjHLMe5PSIiKcY+U/Loeo9GIZv+DbP+4zwG3ZC6sXlWYF8rvFzUhSZUg=@vger.kernel.org X-Gm-Message-State: AOJu0YzsNVir0zpnNLLBlVkj1mtDXfK1pdDLeHoHQThzMyeD+MeOZiNy cGtFJKgj41yC3weUdWi7qsviCfArbLYWhy0xrtjoXc/LTtz82p9dj4IVWXfru8kTaA0dS2jN3Hb xHt9ciqFGrZOTBw== X-Received: from plge5.prod.google.com ([2002:a17:902:cf45:b0:2a3:1bf9:d25]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e84f:b0:2b0:7114:23cf with SMTP id d9443c01a7336-2b0827dbc37mr127246635ad.36.1774310358090; Mon, 23 Mar 2026 16:59:18 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:15 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-24-dmatlack@google.com> Subject: [PATCH v3 23/24] vfio: selftests: Verify that opening VFIO device fails during Live Update From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Verify that opening a VFIO device through its cdev file and via VFIO_GROUP_GET_DEVICE_FD both fail with -EBUSY if the device was preserved across a Live Update. When a device file is preserve across a Live Update, the file must be retrieved from /dev/liveupdate, not from VFIO directly. Signed-off-by: David Matlack --- .../vfio/vfio_pci_liveupdate_kexec_test.c | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c = b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c index 15b3e3af91d1..65c48196e44e 100644 --- a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c @@ -36,6 +36,42 @@ static void before_kexec(int luo_fd) daemonize_and_wait(); } =20 +static void check_open_vfio_device_fails(void) +{ + const char *cdev_path =3D vfio_pci_get_cdev_path(device_bdf); + struct vfio_pci_device *device; + struct iommu *iommu; + int ret, i; + + printf("Checking open(%s) fails\n", cdev_path); + ret =3D open(cdev_path, O_RDWR); + VFIO_ASSERT_EQ(ret, -1); + VFIO_ASSERT_EQ(errno, EBUSY); + free((void *)cdev_path); + + for (i =3D 0; i < nr_iommu_modes; i++) { + if (!iommu_modes[i].container_path) + continue; + + iommu =3D iommu_init(iommu_modes[i].name); + + device =3D vfio_pci_device_alloc(device_bdf, iommu); + vfio_pci_group_setup(device); + vfio_pci_iommu_setup(device); + + printf("Checking ioctl(group_fd, VFIO_GROUP_GET_DEVICE_FD, \"%s\") fails= (%s)\n", + device_bdf, iommu_modes[i].name); + + ret =3D ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, device->bdf); + VFIO_ASSERT_EQ(ret, -1); + VFIO_ASSERT_EQ(errno, EBUSY); + + close(device->group_fd); + free(device); + iommu_cleanup(iommu); + } +} + static void after_kexec(int luo_fd, int state_session_fd) { struct vfio_pci_device *device; @@ -44,6 +80,8 @@ static void after_kexec(int luo_fd, int state_session_fd) int device_fd; int stage; =20 + check_open_vfio_device_fails(); + restore_and_read_stage(state_session_fd, STATE_TOKEN, &stage); VFIO_ASSERT_EQ(stage, 2); =20 --=20 2.53.0.983.g0bb29b3bc5-goog From nobody Thu Apr 2 20:26:42 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71E713E558F for ; Mon, 23 Mar 2026 23:59:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310362; cv=none; b=IWuCjXI6hXCBHlcG76tuiTVNcBbRGEDOA6x0MwX7JkDzhz0kfCTzIxc8k7hPObRs/jq5fWX0k6NctEaUvWv0PB9SkrHzxJV2c349vDXOBqETi2+vqwNR+FmKt07c9RL29TqUURDg0eiwGqAMx1z0XZmpEb3n0XOWlKpXaA4hRQ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774310362; c=relaxed/simple; bh=vpHjJ0pn077dUCj9wV/y1NEgQP72set6GB5RHBGwAdM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QCg1V15cZD8f9oKlUD8ulmAKlto5L4NL9bWyuQVfDE1r6XwWiisT90cZDmu0C9zBsM+BZvaqQ0b0ECYSoipRBt973k2sMN97SCE77dR4c6WCcWbL5wfPUON98rqRlWjd0TEjvxBsdZXaY34GNGzS1MZ7Gge9OKhQWrars/ItEHs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iHCuUcD1; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iHCuUcD1" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2b0601ff3d9so12989595ad.2 for ; Mon, 23 Mar 2026 16:59:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774310360; x=1774915160; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=emXTkfduCARpReH/0NkUqCqksAosTTU4j/dciIvOyCs=; b=iHCuUcD1MXluPlFGR9QaLpMovlhc43mYSiP5FnvngghaPKZCwxqWpBQ3l97AKVY4mm C+lIG/50aD4f5beKECih57U0jbIXYtpKSbNL3+jm78RPQgp4ZD1sCB3QcSUASbBfyFrU pefSw9WULtAHhzcEbXM26XiSMtvMc7jp3fVTiHPrBRyNigX5umOTiJeC8VHI5uUhK3FL tCXfDiVgjAzi7ThSb1TK6HACV9WyNK77Cv7Rg9BC5NOg9mquA+OSggYPcB8nrBshn3bU +ZFoZMPQxqpzmYW7U6y/rvzPLcolxmBFzUOpHiEDHe+/JH+7qv7Xv0tePHYuTu5JYwiF UM0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774310360; x=1774915160; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=emXTkfduCARpReH/0NkUqCqksAosTTU4j/dciIvOyCs=; b=ovxMw22C+PioQgy9vDmmzFl/cJrSiFdXkPUcvmc38szqKNjN6f223EiC4AMDRB872Q njhabZ76hMVp3HElerYixveloJTlEuWGzDwdUH8whggVIUz0c2VOTZefZZPwqri/z6Js pRsfDDKV4FahaSEiqgZBO94qms3mtdpJYt6KMHZDOm/nF8GkzuTesdlXOaSzzX/81M9+ w7elVirzR4nLjNo6866ZQxYbmLQOBPv26Oup62pTTzApdPb45wjBsTFf+d+PsLOkshn4 GBwgTbJPn0GCATLpB+4ispBF384juPpywlnnGI4Faz7v3+6w1UyL6jwi4JWBhYVqI0OG ly6A== X-Forwarded-Encrypted: i=1; AJvYcCUnjQRy9+EPeyzWIy4ExduplTDAXlSNG8JsvD8plFPGYZet03RhFbOfVcn9ohLPTOdLv20LYij6JqrFS3U=@vger.kernel.org X-Gm-Message-State: AOJu0YzQ74X5UkFPE70NBkG4U0T0WUNRZxR9yg7rnOQtiw21KcJjC9/k rKC971Jak0GIB+muAwS5hZtqZt9zLEOxN8zkeBzsypF3P7jyMwSIeT3y1ranyIpCFnIUXAgVYsO oc9WFsTV8iOIjEA== X-Received: from plcp2.prod.google.com ([2002:a17:902:e342:b0:2b0:6c44:fc55]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:d54a:b0:2ae:c9be:5f2c with SMTP id d9443c01a7336-2b08271d042mr130329495ad.21.1774310359658; Mon, 23 Mar 2026 16:59:19 -0700 (PDT) Date: Mon, 23 Mar 2026 23:58:16 +0000 In-Reply-To: <20260323235817.1960573-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260323235817.1960573-1-dmatlack@google.com> X-Mailer: git-send-email 2.53.0.983.g0bb29b3bc5-goog Message-ID: <20260323235817.1960573-25-dmatlack@google.com> Subject: [PATCH v3 24/24] vfio: selftests: Add continuous DMA to vfio_pci_liveupdate_kexec_test From: David Matlack To: Alex Williamson , Bjorn Helgaas Cc: Adithya Jayachandran , Alexander Graf , Alex Mastro , Andrew Morton , Ankit Agrawal , Arnd Bergmann , Askar Safin , "Borislav Petkov (AMD)" , Chris Li , Dapeng Mi , David Matlack , David Rientjes , Feng Tang , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Kees Cook , Kevin Tian , kexec@lists.infradead.org, kvm@vger.kernel.org, Leon Romanovsky , Leon Romanovsky , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Li RongQing , Lukas Wunner , Marco Elver , "=?UTF-8?q?Micha=C5=82=20Winiarski?=" , Mike Rapoport , Parav Pandit , Pasha Tatashin , "Paul E. McKenney" , Pawan Gupta , "Peter Zijlstra (Intel)" , Pranjal Shrivastava , Pratyush Yadav , Raghavendra Rao Ananta , Randy Dunlap , Rodrigo Vivi , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , Vivek Kasireddy , William Tu , Yi Liu , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a long-running DMA memcpy operation to vfio_pci_liveupdate_kexec_test so that the device attempts to perform DMAs continuously during the Live Update. At this point iommufd preservation is not supported and bus mastering is not kept enabled on the device during across the kexec, so most of these DMAs will be dropped. However this test ensures that the current device preservation support does not lead to system instability or crashes if the device is active. And once iommufd and bus mastering are preserved, this test can be relaxed to check that the DMA operations completed successfully. Signed-off-by: David Matlack --- .../vfio/vfio_pci_liveupdate_kexec_test.c | 129 ++++++++++++++++++ 1 file changed, 129 insertions(+) diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c = b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c index 65c48196e44e..36bddfbb88ed 100644 --- a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c @@ -1,8 +1,16 @@ // SPDX-License-Identifier: GPL-2.0-only =20 +#include +#include + #include #include =20 +#define MEMCPY_SIZE SZ_1G +#define DRIVER_SIZE SZ_1M +#define MEMFD_SIZE (MEMCPY_SIZE + DRIVER_SIZE) + +static struct dma_region memcpy_region; static const char *device_bdf; =20 static char state_session[LIVEUPDATE_SESSION_NAME_LENGTH]; @@ -11,8 +19,89 @@ static char device_session[LIVEUPDATE_SESSION_NAME_LENGT= H]; enum { STATE_TOKEN, DEVICE_TOKEN, + MEMFD_TOKEN, }; =20 +static void dma_memcpy_one(struct vfio_pci_device *device) +{ + void *src =3D memcpy_region.vaddr, *dst; + u64 size; + + size =3D min_t(u64, memcpy_region.size / 2, device->driver.max_memcpy_siz= e); + dst =3D src + size; + + memset(src, 1, size); + memset(dst, 0, size); + + printf("Kicking off 1 DMA memcpy operations of size 0x%lx...\n", size); + vfio_pci_driver_memcpy(device, + to_iova(device, src), + to_iova(device, dst), + size); + + VFIO_ASSERT_EQ(memcmp(src, dst, size), 0); +} + +static void dma_memcpy_start(struct vfio_pci_device *device) +{ + void *src =3D memcpy_region.vaddr, *dst; + u64 count, size; + + size =3D min_t(u64, memcpy_region.size / 2, device->driver.max_memcpy_siz= e); + dst =3D src + size; + + /* + * Rough Math: If we assume the device will perform memcpy at a rate of + * 30GB/s then 7200GB of transfers will run for about 4 minutes. + */ + count =3D (u64)7200 * SZ_1G / size; + count =3D min_t(u64, count, device->driver.max_memcpy_count); + + memset(src, 1, size / 2); + memset(dst, 0, size / 2); + + printf("Kicking off %lu DMA memcpy operations of size 0x%lx...\n", count,= size); + vfio_pci_driver_memcpy_start(device, + to_iova(device, src), + to_iova(device, dst), + size, count); +} + +static void dma_memfd_map(struct vfio_pci_device *device, int fd) +{ + void *vaddr; + + vaddr =3D mmap(NULL, MEMFD_SIZE, PROT_WRITE, MAP_SHARED, fd, 0); + VFIO_ASSERT_NE(vaddr, MAP_FAILED); + + memcpy_region.iova =3D SZ_4G; + memcpy_region.size =3D MEMCPY_SIZE; + memcpy_region.vaddr =3D vaddr; + iommu_map(device->iommu, &memcpy_region); + + device->driver.region.iova =3D memcpy_region.iova + memcpy_region.size; + device->driver.region.size =3D DRIVER_SIZE; + device->driver.region.vaddr =3D vaddr + memcpy_region.size; + iommu_map(device->iommu, &device->driver.region); +} + +static void dma_memfd_setup(struct vfio_pci_device *device, int session_fd) +{ + int fd, ret; + + fd =3D memfd_create("dma-buffer", 0); + VFIO_ASSERT_GE(fd, 0); + + ret =3D fallocate(fd, 0, 0, MEMFD_SIZE); + VFIO_ASSERT_EQ(ret, 0); + + printf("Preserving memfd of size 0x%x in session\n", MEMFD_SIZE); + ret =3D luo_session_preserve_fd(session_fd, fd, MEMFD_TOKEN); + VFIO_ASSERT_EQ(ret, 0); + + dma_memfd_map(device, fd); +} + static void before_kexec(int luo_fd) { struct vfio_pci_device *device; @@ -32,6 +121,27 @@ static void before_kexec(int luo_fd) ret =3D luo_session_preserve_fd(session_fd, device->fd, DEVICE_TOKEN); VFIO_ASSERT_EQ(ret, 0); =20 + dma_memfd_setup(device, session_fd); + + /* + * If the device has a selftests driver, kick off a long-running DMA + * operation to exercise the device trying to DMA during a Live Update. + * Since iommufd preservation is not supported yet, these DMAs should be + * dropped. So this is just looking to verify that the system does not + * fall over and crash as a result of a busy device being preserved. + */ + if (device->driver.ops) { + vfio_pci_driver_init(device); + dma_memcpy_start(device); + + /* + * Disable interrupts on the device or freeze() will fail. + * Unfortunately there isn't a way to easily have a test for + * that here since the check happens during shutdown. + */ + vfio_pci_msix_disable(device); + } + close(luo_fd); daemonize_and_wait(); } @@ -78,6 +188,7 @@ static void after_kexec(int luo_fd, int state_session_fd) struct iommu *iommu; int session_fd; int device_fd; + int memfd; int stage; =20 check_open_vfio_device_fails(); @@ -88,6 +199,10 @@ static void after_kexec(int luo_fd, int state_session_f= d) session_fd =3D luo_retrieve_session(luo_fd, device_session); VFIO_ASSERT_GE(session_fd, 0); =20 + printf("Retrieving memfd from LUO\n"); + memfd =3D luo_session_retrieve_fd(session_fd, MEMFD_TOKEN); + VFIO_ASSERT_GE(memfd, 0); + printf("Finishing the session before retrieving the device (should fail)\= n"); VFIO_ASSERT_NE(luo_session_finish(session_fd), 0); =20 @@ -109,9 +224,23 @@ static void after_kexec(int luo_fd, int state_session_= fd) */ device =3D __vfio_pci_device_init(device_bdf, iommu, device_fd); =20 + dma_memfd_map(device, memfd); + printf("Finishing the session\n"); VFIO_ASSERT_EQ(luo_session_finish(session_fd), 0); =20 + /* + * Once iommufd preservation is supported and the device is kept fully + * running across the Live Update, this should wait for the long- + * running DMA memcpy operation kicked off in before_kexec() to + * complete. But for now we expect the device to be reset so just + * trigger a single memcpy to make sure it's still functional. + */ + if (device->driver.ops) { + vfio_pci_driver_init(device); + dma_memcpy_one(device); + } + vfio_pci_device_cleanup(device); iommu_cleanup(iommu); } --=20 2.53.0.983.g0bb29b3bc5-goog