From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D392936A357 for ; Mon, 11 May 2026 23:48:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543313; cv=none; b=Y4+0yqD/qhp5j7Yda9CLCZV0eyS8TAOIgmpdi2TSgrxIUSU1Yadv4lrH1pyX9OoPtmjv2T500hqh2JTx7FxMOiR0w3fGbhmU0YnDyid2SZTZuAYzb9QiuHtJjdtEJZ5MXvCsi3HeIHquZBXyjttR1nD5n/Im2VNvpdAFd5q1GR4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543313; c=relaxed/simple; bh=tfZtedW2QWnLrxToR7VGh+7wrLdGvMWEp3d3S0hb/FM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gCcjvmF8cEoDFUBB4VuniqoJxRz7Mqu3iTnGcRP7B8bzVNhQONWrdBWUzdXmMSDvkq8EODMarKvzFXrOLRQVrDbe5n1AlFOHrXyiuhFibDjHeEgIdvYyoESs9xphYX6XLw5/s66Pf21qzBtUlms5UvLPELkSRiJFKGSD4HmQlnA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=B/p9Kbmh; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="B/p9Kbmh" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2baf7748d0aso41220985ad.1 for ; Mon, 11 May 2026 16:48:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543310; x=1779148110; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=g2q9EGS/hZY8b1wwaKYU2VBFf9wnakzcnkxpfxUOFzQ=; b=B/p9KbmhjKHSxXtxl/T3Ct4z6gXku9GPt1mDU9TSF2foN6Mqgvyi06BEw1FTQEGrUy flg4qsyVz7aYBuwjzfSECSHtNLKgHJgltsznmWHzvVCE8ymaiKSWUJIOETa7E/KcTeZe nPElRnUvJ1y5tKWQQFAhsIs6FzIDFW3/MoAGa1UeVIz6f1D2Ds5HM1Z9hSFbzKoE7kI3 MUV4LOfPqldlYjbupz9iG8GYi3nOb9MljPy8bUWrzGIoEnhh7n5uDT2rbDzoUprEJ0hI URv5djAU0YLeJGH7WXpFrbYEi4euPhjWWkmWpEbZIQSU4GQeaFqQWv181LpjkNhN44QZ US8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543310; x=1779148110; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=g2q9EGS/hZY8b1wwaKYU2VBFf9wnakzcnkxpfxUOFzQ=; b=O4itEom2HD6Bqlh29ZzGYhfbl7jWYr4FcGZPYbR8eKS3fGwXvqPc6z7lP07y0b8Unl WME5sl3DGAQDVnISJ2DCHQIM/ULlDh5KgUfhdUNlR3jCuJCWwB+f5UctRLEWBdtc/5sO HlS2ygc8H8sf6oqGkcyccpAEF2HlByNcBmx4wBI1JbY3syTQ8QHPVrHCmFBjk/dQU/QU wyqwTyWMj8VJQQZ9GuyB4hdjKrVWuZa+Ln3sAWkfkHW+CNKsCTugt5HWzf219PcBKxjA abMAPTrQuopJqGsriXGglrxAdymcgpkKxWQh3m1+uZE6K9Q2iePq4KLoQiAzcutPd95f K7gA== X-Forwarded-Encrypted: i=1; AFNElJ+sJknGs/g0Zw+gL26DoyiZVt+lYUILUpemE3tEO6SyLPIX/SrU3hV0Ih5tfyivzzE8TUXYt6xjJvf/jl4=@vger.kernel.org X-Gm-Message-State: AOJu0Yyfdec0OY+u7ou0iNq8u7Hl5FUGkC7Dbqd/LvQ3LWdb8z7Ck+sg kfYuHyZtwRD7smqFYc/x9S1cyuS5JxBn91FMdiFn+e35IHvewH6ZN20owKXXrq0vH9XVvMG8oeY zzNL3Sj/ZRg== X-Received: from plbv16.prod.google.com ([2002:a17:903:44d0:b0:2b2:488e:b71]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:1b28:b0:2ba:881f:6192 with SMTP id d9443c01a7336-2baf0dd629cmr168833395ad.22.1778543310123; Mon, 11 May 2026 16:48:30 -0700 (PDT) Date: Mon, 11 May 2026 16:47:47 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-2-vipinsh@google.com> Subject: [PATCH v4 01/16] vfio/pci: Register a file handler with Live Update Orchestrator From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Register a live update file handler for vfio-pci device files. Add stub implementations of all required callbacks so that registration does not fail (i.e. to avoid breaking git-bisect). This file handler will be extended in subsequent commits to enable a device bound to vfio-pci to run without interruption while the host is going through a kexec Live Update. Put this support behind a new Kconfig VFIO_PCI_LIVEUPDATE that is marked experimental and default-disabled until more of the device preservation support has landed in the kernel. Co-developed-by: David Matlack Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- MAINTAINERS | 1 + drivers/vfio/pci/Kconfig | 12 +++++ drivers/vfio/pci/Makefile | 1 + drivers/vfio/pci/vfio_pci.c | 11 +++- drivers/vfio/pci/vfio_pci_liveupdate.c | 70 ++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 14 ++++++ include/linux/kho/abi/vfio_pci.h | 28 +++++++++++ 7 files changed, 136 insertions(+), 1 deletion(-) create mode 100644 drivers/vfio/pci/vfio_pci_liveupdate.c create mode 100644 include/linux/kho/abi/vfio_pci.h diff --git a/MAINTAINERS b/MAINTAINERS index 42dbac2c2ed3..b6c33c2bcc7c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -27980,6 +27980,7 @@ F: Documentation/ABI/testing/debugfs-vfio F: Documentation/ABI/testing/sysfs-devices-vfio-dev F: Documentation/driver-api/vfio.rst F: drivers/vfio/ +F: include/linux/kho/abi/vfio_pci.h F: include/linux/vfio.h F: include/linux/vfio_pci_core.h F: include/uapi/linux/vfio.h diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index 296bf01e185e..c12d614fc6c4 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -42,6 +42,18 @@ config VFIO_PCI_IGD and LPC bridge config space. =20 To enable Intel IGD assignment through vfio-pci, say Y. + +config VFIO_PCI_LIVEUPDATE + bool "VFIO PCI support for Live Update (EXPERIMENTAL)" + depends on PCI_LIVEUPDATE + help + Support for preserving devices bound to vfio-pci across a Live + Update. This option should only be enabled by developers working on + implementing this support. Once enough support has landed in the + kernel, this option will no longer be marked EXPERIMENTAL. + + If you don't know what to do here, say N. + endif =20 config VFIO_PCI_ZDEV_KVM diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index 6138f1bf241d..bf296ab1c967 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_VFIO_PCI_CORE) +=3D vfio-pci-core.o =20 vfio-pci-y :=3D vfio_pci.o vfio-pci-$(CONFIG_VFIO_PCI_IGD) +=3D vfio_pci_igd.o +vfio-pci-$(CONFIG_VFIO_PCI_LIVEUPDATE) +=3D vfio_pci_liveupdate.o obj-$(CONFIG_VFIO_PCI) +=3D vfio-pci.o =20 obj-$(CONFIG_MLX5_VFIO_PCI) +=3D mlx5/ diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 0c771064c0b8..5038265f3af5 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -264,10 +264,14 @@ static int __init vfio_pci_init(void) =20 vfio_pci_core_set_params(nointxmask, is_disable_vga, disable_idle_d3); =20 + ret =3D vfio_pci_liveupdate_init(); + if (ret) + return ret; + /* Register and scan for devices */ ret =3D pci_register_driver(&vfio_pci_driver); if (ret) - return ret; + goto err_liveupdate_cleanup; =20 vfio_pci_fill_ids(); =20 @@ -275,12 +279,17 @@ static int __init vfio_pci_init(void) pr_warn("device denylist disabled.\n"); =20 return 0; + +err_liveupdate_cleanup: + vfio_pci_liveupdate_cleanup(); + return ret; } module_init(vfio_pci_init); =20 static void __exit vfio_pci_cleanup(void) { pci_unregister_driver(&vfio_pci_driver); + vfio_pci_liveupdate_cleanup(); } module_exit(vfio_pci_cleanup); =20 diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c new file mode 100644 index 000000000000..1c74854600ea --- /dev/null +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright (c) 2026, Google LLC. + * Vipin Sharma + * David Matlack + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include + +#include "vfio_pci_priv.h" + +static bool vfio_pci_liveupdate_can_preserve(struct liveupdate_file_handle= r *handler, + struct file *file) +{ + return false; +} + +static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *ar= gs) +{ + return -EOPNOTSUPP; +} + +static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args = *args) +{ +} + +static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) +{ + return -EOPNOTSUPP; +} + +static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) +{ +} + +static const struct liveupdate_file_ops vfio_pci_liveupdate_file_ops =3D { + .can_preserve =3D vfio_pci_liveupdate_can_preserve, + .preserve =3D vfio_pci_liveupdate_preserve, + .unpreserve =3D vfio_pci_liveupdate_unpreserve, + .retrieve =3D vfio_pci_liveupdate_retrieve, + .finish =3D vfio_pci_liveupdate_finish, + .owner =3D THIS_MODULE, +}; + +static struct liveupdate_file_handler vfio_pci_liveupdate_fh =3D { + .ops =3D &vfio_pci_liveupdate_file_ops, + .compatible =3D VFIO_PCI_LUO_FH_COMPATIBLE, +}; + +int __init vfio_pci_liveupdate_init(void) +{ + int ret; + + ret =3D liveupdate_register_file_handler(&vfio_pci_liveupdate_fh); + if (ret && ret !=3D -EOPNOTSUPP) + return ret; + + return 0; +} + +void vfio_pci_liveupdate_cleanup(void) +{ + liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); +} diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index fca9d0dfac90..c130ac53c4f9 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -137,4 +137,18 @@ static inline void vfio_pci_dma_buf_move(struct vfio_p= ci_core_device *vdev, } #endif =20 +#ifdef CONFIG_VFIO_PCI_LIVEUPDATE +int __init vfio_pci_liveupdate_init(void); +void vfio_pci_liveupdate_cleanup(void); +#else +static inline int vfio_pci_liveupdate_init(void) +{ + return 0; +} + +static inline void vfio_pci_liveupdate_cleanup(void) +{ +} +#endif /* CONFIG_VFIO_PCI_LIVEUPDATE */ + #endif diff --git a/include/linux/kho/abi/vfio_pci.h b/include/linux/kho/abi/vfio_= pci.h new file mode 100644 index 000000000000..67e8199eec59 --- /dev/null +++ b/include/linux/kho/abi/vfio_pci.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Copyright (c) 2026, Google LLC. + * Vipin Sharma + * David Matlack + */ + +#ifndef _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H +#define _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H + +/** + * DOC: VFIO PCI Live Update ABI + * + * VFIO uses the ABI defined below for preserving device files across a ke= xec + * reboot using LUO. + * + * Device metadata is serialized into memory which is then handed to the n= ext + * kernel via KHO. + * + * This interface is a contract. Any modification to any of the serializat= ion + * structs defined here constitutes a breaking change. Such changes require + * incrementing the version number in the VFIO_PCI_LUO_FH_COMPATIBLE strin= g. + */ + +#define VFIO_PCI_LUO_FH_COMPATIBLE "vfio-pci-v1" + +#endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */ --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DFA9536A36F for ; Mon, 11 May 2026 23:48:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543314; cv=none; b=esG7o4BMnCXj8wDH1+cNWrr2qqOdj3aK79A27ZsKXuDNtPQrJBPV+0tMOMBrJQbYljXqWSNxULwcOiQk63iM5P4PcM8lNzOaDppLXuLxS7xjGIL+tbxv62mxww6tJKC4CJvrBsNBI2kHE3doLvJvytY3I4DsW+FBiWITYCgyUvU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543314; c=relaxed/simple; bh=JQdvnfwZ1UiMBoJV/M1aimVlkpa2tFk4EK0bnL+eufI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RwFo9AY3wf9RmjmdGZnqc7R8pr33XafA1IIDGmRRRX7Gpz8qX6/j5JWaJZnkXGDwgnjTNoXtx1MuiBCz13421bCZ4SBUYfBj+Tict+fZvkuohJxQUefesU1PBXykO4bNDwIE5a0fKKFWZT96mf76xgPP/K4ArGDGmM8FJgluz54= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Ppf55UKC; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Ppf55UKC" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c828acf7c1dso2161605a12.3 for ; Mon, 11 May 2026 16:48:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543311; x=1779148111; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xFPnttZDgBRngzGtSguPdn1kleodxJkte/D1pn+GAEc=; b=Ppf55UKCqbj46/rnC9bvtQF5avovdtOg7P6XkMXQNuQtVQ1iLbp2tMOxXZDLH2GbL5 01xhazhjms71eNEjZQsNEwnRfSkrfE5kZTaFKkF/7+lc/WyN9J+w8EzuwVQTGcbD0XJF yipTRKa3j28kcZbUrfYkRdNIU7mHA42kKwDhJeyOR8PJ9xSjayHZc97qBV2ZgwhNLCb5 oc5vgGzluIg2PXz7vOHcr6aLq+dNUJDiNex4z8kMAqr9spuRdoCxm8iwrK09Y0mTkVC8 WuHCOqD5C6lllPTQrxuS+MHOlILzggUuorScq5lfuT/SOfbpLuSRJqp+mzfXM3JhT7mE JtvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543311; x=1779148111; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xFPnttZDgBRngzGtSguPdn1kleodxJkte/D1pn+GAEc=; b=jMOeoyh0yfoXdYlkDAnFZNTP83cDJyUqZTohXcmtwc1ehB/IH/hyadkGlxwhk+shzg CWcGqL0FWzeZeJSPVphFfteXcZ51WonrEFr8BJZlg9r3XAv8en3U85Qr54d1+OXDkdSG cTehNeGasdSlSnXubm5zLo6yE9bx2b5U9lLMwrZqcXRx2M1HOWhfX9rPAP3cp9QN2z4V MtGh0fHz0D4XaS0Yw8KZEI99fNbNEdqgYbTDY9Te2S5sobp7g2L6KVAgCEjUh4MTwaYB XRHvwvurCd0uFcXeVAJ6JIn5FVCbScrlq3oGgOq4SlNFjqzZGJj3ntRMA1Tiv63DqgrO Dbvw== X-Forwarded-Encrypted: i=1; AFNElJ+UCLnmq2L7VotdwH5xJ415MYVfcTUcWjvkW/i3W7pVsT2eHyQppdG56vqB5CalR2/kYKvm1voa6I8cn3M=@vger.kernel.org X-Gm-Message-State: AOJu0YxMAZQiZEOtaFWYbs9MxzkUhYNmyKGoDHSabBWyVJ9TFFIiQPmK it9cU0bYP3Yf71tKdAlGRVCfZiKXZ+tN+3wiik0aRU/LZvQXGs0qlH8X7RpzpSyqKufZZP/dqYh uQSEm40MacQ== X-Received: from pghz21.prod.google.com ([2002:a63:e115:0:b0:c7f:c4f1:ae17]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:3ca2:b0:398:bbd6:80b9 with SMTP id adf61e73a8af0-3ad966596abmr704170637.23.1778543311124; Mon, 11 May 2026 16:48:31 -0700 (PDT) Date: Mon, 11 May 2026 16:47:48 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-3-vipinsh@google.com> Subject: [PATCH v4 02/16] vfio/pci: Preserve vfio-pci device files across Live Update From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Implement the live update file handler callbacks to preserve a vfio-pci device across a Live Update. Subsequent commits will enable userspace to then retrieve this file after the Live Update. Live Update support is scoped only to cdev files (i.e. not VFIO_GROUP_GET_DEVICE_FD files). State about each device is serialized into a new ABI struct vfio_pci_core_device_ser. The contents of this struct are preserved across the Live Update to the next kernel using a combination of Kexec-Handover (KHO) to preserve the page(s) holding the struct and the Live Update Orchestrator (LUO) to preserve the physical address of the struct. For now the only contents of struct vfio_pci_core_device_ser the device's PCI segment number and BDF, so that the device can be uniquely identified after the Live Update. Require that userspace disables interrupts on the device prior to freeze() so that the device does not send any interrupts until new interrupt handlers have been set up by the next kernel. Reset the device and restore its state in the freeze() callback. This ensures the device can be received by the next kernel in a consistent state. Eventually this will be dropped and the device can be preserved across in a running state, but that requires further work in VFIO and the core PCI layer. Note that LUO holds a reference to this file when it is preserved. So VFIO is guaranteed that vfio_df_device_last_close() will not be called on this device until userspace has unpreserved the file using LUO. Co-developed-by: David Matlack Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- drivers/vfio/pci/Kconfig | 6 +- drivers/vfio/pci/vfio_pci.c | 2 +- drivers/vfio/pci/vfio_pci_core.c | 60 +++++---- drivers/vfio/pci/vfio_pci_liveupdate.c | 178 ++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_priv.h | 5 + drivers/vfio/vfio_main.c | 3 +- include/linux/kho/abi/vfio_pci.h | 15 +++ include/linux/vfio.h | 2 + 8 files changed, 242 insertions(+), 29 deletions(-) diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index c12d614fc6c4..019de053f116 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -45,13 +45,15 @@ config VFIO_PCI_IGD =20 config VFIO_PCI_LIVEUPDATE bool "VFIO PCI support for Live Update (EXPERIMENTAL)" - depends on PCI_LIVEUPDATE + depends on PCI_LIVEUPDATE && VFIO_DEVICE_CDEV help Support for preserving devices bound to vfio-pci across a Live Update. This option should only be enabled by developers working on implementing this support. Once enough support has landed in the kernel, this option will no longer be marked EXPERIMENTAL. =20 + Enabling this will disable support for VFIO PCI DMA buffer. + If you don't know what to do here, say N. =20 endif @@ -68,7 +70,7 @@ config VFIO_PCI_ZDEV_KVM To enable s390x KVM vfio-pci extensions, say Y. =20 config VFIO_PCI_DMABUF - def_bool y if VFIO_PCI_CORE && PCI_P2PDMA && DMA_SHARED_BUFFER + def_bool y if VFIO_PCI_CORE && PCI_P2PDMA && DMA_SHARED_BUFFER && !VFIO_P= CI_LIVEUPDATE =20 source "drivers/vfio/pci/mlx5/Kconfig" =20 diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 5038265f3af5..439d14d79d8f 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -125,7 +125,7 @@ static int vfio_pci_open_device(struct vfio_device *cor= e_vdev) return 0; } =20 -static const struct vfio_device_ops vfio_pci_ops =3D { +const struct vfio_device_ops vfio_pci_ops =3D { .name =3D "vfio-pci", .init =3D vfio_pci_core_init_dev, .release =3D vfio_pci_core_release_dev, diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index ad52abc46c04..1726d4d41b57 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -585,9 +585,42 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *= vdev) } EXPORT_SYMBOL_GPL(vfio_pci_core_enable); =20 +void vfio_pci_core_try_reset(struct vfio_pci_core_device *vdev) +{ + struct pci_dev *pdev =3D vdev->pdev; + struct pci_dev *bridge =3D pci_upstream_bridge(pdev); + + lockdep_assert_held(&vdev->vdev.dev_set->lock); + + if (!vdev->reset_works) + return; + + /* + * Try to get the locks ourselves to prevent a deadlock. The + * success of this is dependent on being able to lock the device, + * which is not always possible. + * + * We cannot use the "try" reset interface here, since that will + * overwrite the previously restored configuration information. + */ + if (bridge && !pci_dev_trylock(bridge)) + return; + + if (!pci_dev_trylock(pdev)) + goto out; + + if (!__pci_reset_function_locked(pdev)) + vdev->needs_reset =3D false; + + pci_dev_unlock(pdev); +out: + if (bridge) + pci_dev_unlock(bridge); +} +EXPORT_SYMBOL_GPL(vfio_pci_core_try_reset); + void vfio_pci_core_disable(struct vfio_pci_core_device *vdev) { - struct pci_dev *bridge; struct pci_dev *pdev =3D vdev->pdev; struct vfio_pci_dummy_resource *dummy_res, *tmp; struct vfio_pci_ioeventfd *ioeventfd, *ioeventfd_tmp; @@ -687,27 +720,7 @@ void vfio_pci_core_disable(struct vfio_pci_core_device= *vdev) */ pci_write_config_word(pdev, PCI_COMMAND, PCI_COMMAND_INTX_DISABLE); =20 - /* - * Try to get the locks ourselves to prevent a deadlock. The - * success of this is dependent on being able to lock the device, - * which is not always possible. - * We can not use the "try" reset interface here, which will - * overwrite the previously restored configuration information. - */ - if (vdev->reset_works) { - bridge =3D pci_upstream_bridge(pdev); - if (bridge && !pci_dev_trylock(bridge)) - goto out_restore_state; - if (pci_dev_trylock(pdev)) { - if (!__pci_reset_function_locked(pdev)) - vdev->needs_reset =3D false; - pci_dev_unlock(pdev); - } - if (bridge) - pci_dev_unlock(bridge); - } - -out_restore_state: + vfio_pci_core_try_reset(vdev); pci_restore_state(pdev); out: pci_disable_device(pdev); @@ -1613,7 +1626,7 @@ ssize_t vfio_pci_core_write(struct vfio_device *core_= vdev, const char __user *bu } EXPORT_SYMBOL_GPL(vfio_pci_core_write); =20 -static void vfio_pci_zap_bars(struct vfio_pci_core_device *vdev) +void vfio_pci_zap_bars(struct vfio_pci_core_device *vdev) { struct vfio_device *core_vdev =3D &vdev->vdev; loff_t start =3D VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_BAR0_REGION_INDEX); @@ -1622,6 +1635,7 @@ static void vfio_pci_zap_bars(struct vfio_pci_core_de= vice *vdev) =20 unmap_mapping_range(core_vdev->inode->i_mapping, start, len, true); } +EXPORT_SYMBOL_GPL(vfio_pci_zap_bars); =20 void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_core_device *= vdev) { diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 1c74854600ea..5cc57bf39dbe 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -6,28 +6,201 @@ * David Matlack */ =20 +/** + * DOC: VFIO PCI Preservation via LUO + * + * VFIO PCI devices can be preserved over a kexec using the Live Update + * Orchestrator (LUO) file preservation. This allows userspace (such as a = VMM) + * to transfer an in-use device to the next kernel. + * + * .. note:: + * The support for preserving VFIO PCI devices is currently *partial* a= nd + * should be considered *experimental*. It should only be used by devel= opers + * working on expanding the support for the time being. + * + * To avoid accidental usage while the support is still experimental, t= his + * support is hidden behind a default-disable config option + * ``CONFIG_VFIO_PCI_LIVEUPDATE``. Once the kernel support has stabiliz= ed and + * become complete, this option will be enabled by default when + * ``CONFIG_VFIO_PCI`` and ``CONFIG_LIVEUPDATE`` are enabled. + * + * Usage Example + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * VFIO PCI devices can be preserved across a kexec by preserving the file + * associated with the device in a LUO session:: + * + * device_fd =3D open("/dev/vfio/devices/vfioX"); + * ... + * ioctl(session_fd, LIVEUPDATE_SESSION_PRESERVE_FD, { ..., device_fd, .= ..}); + * + * .. note:: + * LUO will hold an extra reference to the device file for as long as i= t is + * preserved, so there is no way for the file to be destroyed or the de= vice + * to be unbound from the vfio-pci driver while it is preserved. + * + * Retrieving the file after kexec is not yet supported. + * + * Restrictions + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * The kernel imposes the following restrictions when preserving VFIO devi= ces: + * + * * The device must be bound to the ``vfio-pci`` driver. + * + * * ``CONFIG_VFIO_PCI_ZDEV_KVM`` must not be enabled. This may be relaxe= d in + * the future. + * + * * The device must not be an Intel display device. This may be relaxed = in + * the future. + * + * * No support for VFIO PCI DMA buffer. This is to keep code simpler, it= will + * be removed in future. + * + * * The device file descriptor must be obtained by opening the VFIO char= acter + * device (``/dev/vfio/devices/vfioX``) and not via + * ``VFIO_GROUP_GET_DEVICE_FD``. + * + * * The device must have interrupt disable prior to kexec. Failure to di= sable + * interrupts on the device will cause the ``reboot(LINUX_REBOOT_CMD_KE= XEC)`` + * syscall (to initiate the kexec) to fail. + * + * Preservation Behavior + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * The eventual goal of this support is to avoid disrupting the workload, = state, + * or configuration of each preserved device during a Live Update. This wo= uld + * include allowing the device to perform DMA to preserved memory buffers = and + * perform P2P DMA to other preserved devices. However, there are many pie= ces + * that still need to land in the kernel. + * + * For now, VFIO only preserves the following state for for devices: + * + * * The PCI Segment, Bus, Device, and Function numbers of the device. The + * kernel guarantees the these will not change across a kexec when a de= vice + * is preserved. + * + * Since the kernel is not yet prepared to preserve all parts of the devic= e and + * its dependencies (such as DMA mappings), VFIO currently resets and rest= ores + * preserved devices back into an idle state during kexec, before handing = off + * control to the next kernel. This will be relaxed in future versions of = the + * kernel once it is safe to allow the device to keep running across kexec. + */ + #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt =20 #include +#include #include #include #include +#include =20 #include "vfio_pci_priv.h" =20 static bool vfio_pci_liveupdate_can_preserve(struct liveupdate_file_handle= r *handler, struct file *file) { - return false; + struct vfio_device *device =3D vfio_device_from_file(file); + struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + + if (!device) + return false; + + /* Live Update support is limited to cdev files. */ + if (!vfio_device_cdev_opened(device)) + return false; + + if (device->ops !=3D &vfio_pci_ops) + return false; + + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + pdev =3D vdev->pdev; + + /* + * Don't support specialized vfio-pci devices for now since they haven't + * been tested. + */ + if (IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM) || vfio_pci_is_intel_display(pde= v)) + return false; + + return true; } =20 static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *ar= gs) { - return -EOPNOTSUPP; + struct vfio_device *device =3D vfio_device_from_file(args->file); + struct vfio_pci_core_device_ser *ser; + struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + pdev =3D vdev->pdev; + + ser =3D kho_alloc_preserve(sizeof(*ser)); + if (IS_ERR(ser)) + return PTR_ERR(ser); + + ser->bdf =3D pci_dev_id(pdev); + ser->domain =3D pci_domain_nr(pdev->bus); + + args->serialized_data =3D virt_to_phys(ser); + return 0; } =20 static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args = *args) { + kho_unpreserve_free(phys_to_virt(args->serialized_data)); +} + +static int vfio_pci_liveupdate_freeze(struct liveupdate_file_op_args *args) +{ + struct vfio_device *device =3D vfio_device_from_file(args->file); + struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + int ret; + + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + pdev =3D vdev->pdev; + + guard(mutex)(&device->dev_set->lock); + + /* + * Userspace must disable interrupts on the device prior to freeze so + * that the device does not send any interrupts until new interrupt + * handlers have been established by the next kernel. + */ + if (vdev->irq_type !=3D VFIO_PCI_NUM_IRQS) { + pci_err(pdev, "Freeze failed! Interrupts are still enabled.\n"); + return -EINVAL; + } + + guard(rwsem_write)(&vdev->memory_lock); + + /* + * Userspace must make sure device is not in the lower power state for + * live update. We may relax this in future. + */ + if (pdev->current_state !=3D PCI_D0) { + pci_err(pdev, "Freeze failed! Device not in D0 state.\n"); + return -EINVAL; + } + + /* + * Reset is a temporary measure to provide kernel after kexec a clean + * device while VFIO live update work is under development and not + * fully supported. It will go away once continuous DMA support is + * added to device preservation. + */ + vfio_pci_zap_bars(vdev); + ret =3D pci_load_saved_state(pdev, vdev->pci_saved_state); + if (ret) + return ret; + pci_clear_master(pdev); + vfio_pci_core_try_reset(vdev); + pci_restore_state(pdev); + return 0; } =20 static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) @@ -43,6 +216,7 @@ static const struct liveupdate_file_ops vfio_pci_liveupd= ate_file_ops =3D { .can_preserve =3D vfio_pci_liveupdate_can_preserve, .preserve =3D vfio_pci_liveupdate_preserve, .unpreserve =3D vfio_pci_liveupdate_unpreserve, + .freeze =3D vfio_pci_liveupdate_freeze, .retrieve =3D vfio_pci_liveupdate_retrieve, .finish =3D vfio_pci_liveupdate_finish, .owner =3D THIS_MODULE, diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index c130ac53c4f9..20d942a7d9b7 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -11,6 +11,10 @@ /* Cap maximum number of ioeventfds per device (arbitrary) */ #define VFIO_PCI_IOEVENTFD_MAX 1000 =20 +extern const struct vfio_device_ops vfio_pci_ops; + +void vfio_pci_core_try_reset(struct vfio_pci_core_device *vdev); + struct vfio_pci_ioeventfd { struct list_head next; struct vfio_pci_core_device *vdev; @@ -72,6 +76,7 @@ void vfio_pci_zap_and_down_write_memory_lock(struct vfio_= pci_core_device *vdev); u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_core_device *vdev); void vfio_pci_memory_unlock_and_restore(struct vfio_pci_core_device *vdev, u16 cmd); +void vfio_pci_zap_bars(struct vfio_pci_core_device *vdev); =20 #ifdef CONFIG_VFIO_PCI_IGD bool vfio_pci_is_intel_display(struct pci_dev *pdev); diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c index 8666f35fb3f0..2148b00491e4 100644 --- a/drivers/vfio/vfio_main.c +++ b/drivers/vfio/vfio_main.c @@ -1457,7 +1457,7 @@ const struct file_operations vfio_device_fops =3D { #endif }; =20 -static struct vfio_device *vfio_device_from_file(struct file *file) +struct vfio_device *vfio_device_from_file(struct file *file) { struct vfio_device_file *df =3D file->private_data; =20 @@ -1465,6 +1465,7 @@ static struct vfio_device *vfio_device_from_file(stru= ct file *file) return NULL; return df->device; } +EXPORT_SYMBOL_GPL(vfio_device_from_file); =20 /** * vfio_file_is_valid - True if the file is valid vfio file diff --git a/include/linux/kho/abi/vfio_pci.h b/include/linux/kho/abi/vfio_= pci.h index 67e8199eec59..229a13ac1e52 100644 --- a/include/linux/kho/abi/vfio_pci.h +++ b/include/linux/kho/abi/vfio_pci.h @@ -9,6 +9,9 @@ #ifndef _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H #define _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H =20 +#include +#include + /** * DOC: VFIO PCI Live Update ABI * @@ -25,4 +28,16 @@ =20 #define VFIO_PCI_LUO_FH_COMPATIBLE "vfio-pci-v1" =20 +/** + * struct vfio_pci_core_device_ser - Serialized state of a single VFIO PCI + * device. + * + * @domain: The device's PCI domain number (segment). + * @bdf: The device's PCI bus, device, and function number. + */ +struct vfio_pci_core_device_ser { + u32 domain; + u16 bdf; +} __packed; + #endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 31b826efba00..f5f7ece51f8f 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -81,6 +81,8 @@ struct vfio_device { struct dentry *debug_root; }; =20 +struct vfio_device *vfio_device_from_file(struct file *file); + /** * struct vfio_device_ops - VFIO bus driver device callbacks * --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E956036EA93 for ; Mon, 11 May 2026 23:48:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543315; cv=none; b=UM7JDc+5nBSw7fYs8tFw2OaC6VwxHb4u0gfwcnf7FUW1AMw/2qKwfw7rW1IbwYzCVXMNU24mWR/w1rbn2ERZ2bXVoUFPf2baZRaoue/6ZtLD0Pp9D+e8WnRQNmiPVlbqD7fnaXjvdv6H8QGg/ttpEFLdr4dr+A18ow5LBciPv9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543315; c=relaxed/simple; bh=oJsSrCLnq/hUpbpSIJ6e+j3j3wj1WKuepVreB9B1P5E=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BlTf349zYg7U7cfCUbVAqtDqqVcfsmWAuPYlxAIExxuHxSJNL4im6boQVi8UCZ1wljCwQeGClrR6mA5f3h/bHw8/+xTa221021QT2M+rZeh8GfaOaJ3LFR537BxNaGszEWhP1iTsJqC4jkJ7bgCjvfGX0WqnBjcoBvqPOQWVe94= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=agQzXT9O; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="agQzXT9O" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-365d4d2fa04so4665194a91.3 for ; Mon, 11 May 2026 16:48:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543312; x=1779148112; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8YUOBD+7hHFjPbisS9Y2OJf58q41fidHTUfr1qnzIcU=; b=agQzXT9Omert0RLr3XQwYaq31ZB1I8kXJ2QyZTn5jIcGwibzZ6EuyHVd3qXHvL9kLe N+Wt8rqPvu4v8sLKCsqSDFS+glITHaxJueyHEjWh7CKOcUKNgq6+7+DchwRbgJr5wzEG fcMyHppdM3D9qmfvhmaE4TBhHLhrKe43ZCVculgkJz/SRShn6mXUkBhN1Qhkg6YFihtA E513L8bUEPM8LOthHHETqchu5iyt1ZW/P289Z9HhgXnaLHyo3tnq588VpML/gvXIhWOs KsVzAcyxKaLJRzo/3bvSycv1HCnfoXmmDh/P1s2vUEZ9y/aWWu2XfBIGwsbdFAwQ+MKA 1hng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543312; x=1779148112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8YUOBD+7hHFjPbisS9Y2OJf58q41fidHTUfr1qnzIcU=; b=YWoQOaF3lz2PeV+KXQkq3KsFMt/tnel3K4WZZv7qcGjkuvOToCbaxDAkKS/lIUbBIu oXHuOqottELDh4eFdKZro4gzLUJvuDp7do02EepPDBsw4GWw501ncOSrRMnMjr98Ukbb 1Bbz4OEk+b7g9GqxBIRi+AWyx/4HBxmf3Js2rw+E2+GFUFkYaJonYInzXydGVQTcZ6dP dLoeAssJup1zfsm57fUYWQRiAQXr3Jb2BSZYuGrfb50T1UU0hPg7hC8BjylPs9g3857g gNDI2OPL3kF9u8khGL+WxlKd3GNq6aGA26TstZPzZx7EaRBEvAEwZGNuzAF69WGeIMJQ r3wQ== X-Forwarded-Encrypted: i=1; AFNElJ8yb9qOFEwkMq8vahd6YsAfiBIW6Dse0OLdmZISycMbHd3q+4ArIiWnh2HONw8ZyDE8CxAY36iXY96nbIs=@vger.kernel.org X-Gm-Message-State: AOJu0YwnAaan79/CdPmzaNXpAvbEi1QwfWa6/X/f2x92l2wCzCYSEp8O DYuef61kppvb2ecUlNWPnmT65gaRDhpQSFR162Hk4+rVlW6IMiz3v6abMFWry8TeUPfPN01uQBU PhE5/tgas5Q== X-Received: from pgdh5.prod.google.com ([2002:a05:6a02:5185:b0:c79:7107:a67a]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:6a15:b0:398:7d4a:c2c4 with SMTP id adf61e73a8af0-3ad9a95b960mr640565637.48.1778543312073; Mon, 11 May 2026 16:48:32 -0700 (PDT) Date: Mon, 11 May 2026 16:47:49 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-4-vipinsh@google.com> Subject: [PATCH v4 03/16] vfio/pci: Retrieve preserved device files after Live Update From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Enable userspace to retrieve preserved VFIO device files from VFIO after a Live Update by implementing the retrieve() and finish() file handler callbacks. Use an anonymous inode when creating the file, since the retrieved device file is not opened through any particular cdev inode, and the cdev inode does not matter in practice. For now the retrieved file is functionally equivalent a opening the corresponding VFIO cdev file. Subsequent commits will leverage the preserved state associated with the retrieved file to preserve bits of the device across Live Update. Co-developed-by: David Matlack Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- drivers/vfio/device_cdev.c | 59 ++++++++++++++++++++++---- drivers/vfio/pci/vfio_pci_liveupdate.c | 52 ++++++++++++++++++++++- drivers/vfio/vfio_main.c | 13 ++++++ include/linux/vfio.h | 11 +++++ 4 files changed, 124 insertions(+), 11 deletions(-) diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c index 8ceca24ac136..1ab07ccaf3ab 100644 --- a/drivers/vfio/device_cdev.c +++ b/drivers/vfio/device_cdev.c @@ -2,6 +2,7 @@ /* * Copyright (c) 2023 Intel Corporation. */ +#include #include #include =20 @@ -16,15 +17,10 @@ void vfio_init_device_cdev(struct vfio_device *device) device->cdev.owner =3D THIS_MODULE; } =20 -/* - * device access via the fd opened by this function is blocked until - * .open_device() is called successfully during BIND_IOMMUFD. - */ -int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep) +static int vfio_device_cdev_open(struct vfio_device *device, struct file *= *filep) { - struct vfio_device *device =3D container_of(inode->i_cdev, - struct vfio_device, cdev); struct vfio_device_file *df; + struct file *file =3D *filep; int ret; =20 /* Paired with the put in vfio_device_fops_release() */ @@ -37,22 +33,67 @@ int vfio_device_fops_cdev_open(struct inode *inode, str= uct file *filep) goto err_put_registration; } =20 - filep->private_data =3D df; + /* + * Simulate opening the character device using an anonymous inode. The + * returned file has the same properties as a cdev file (e.g. operations + * are blocked until BIND_IOMMUFD is called). + */ + if (!file) { + file =3D anon_inode_getfile_fmode("[vfio-device-liveupdate]", + &vfio_device_fops, NULL, + O_RDWR, FMODE_PREAD | FMODE_PWRITE); + + if (IS_ERR(file)) { + ret =3D PTR_ERR(file); + goto err_free_device_file; + } + + *filep =3D file; + } + + file->private_data =3D df; =20 /* * Use the pseudo fs inode on the device to link all mmaps * to the same address space, allowing us to unmap all vmas * associated to this device using unmap_mapping_range(). */ - filep->f_mapping =3D device->inode->i_mapping; + file->f_mapping =3D device->inode->i_mapping; =20 return 0; =20 +err_free_device_file: + kfree(df); err_put_registration: vfio_device_put_registration(device); return ret; } =20 +struct file *vfio_device_liveupdate_cdev_open(struct vfio_device *device) +{ + struct file *file =3D NULL; + int ret; + + ret =3D vfio_device_cdev_open(device, &file); + if (ret) + return ERR_PTR(ret); + + return file; +} +EXPORT_SYMBOL_GPL(vfio_device_liveupdate_cdev_open); + +/* + * device access via the fd opened by this function is blocked until + * .open_device() is called successfully during BIND_IOMMUFD. + */ +int vfio_device_fops_cdev_open(struct inode *inode, struct file *file) +{ + struct vfio_device *device =3D container_of(inode->i_cdev, + struct vfio_device, cdev); + + return vfio_device_cdev_open(device, &file); +} + static void vfio_df_get_kvm_safe(struct vfio_device_file *df) { spin_lock(&df->kvm_ref_lock); diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 5cc57bf39dbe..9142aaea94f2 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -39,7 +39,13 @@ * preserved, so there is no way for the file to be destroyed or the de= vice * to be unbound from the vfio-pci driver while it is preserved. * - * Retrieving the file after kexec is not yet supported. + * After kexec, the preserved VFIO device file can be retrieved from the s= ession + * just like any other preserved file:: + * + * ioctl(session_fd, LIVEUPDATE_SESSION_RETRIEVE_FD, &arg); + * device_fd =3D arg.fd; + * ... + * ioctl(session_fd, LIVEUPDATE_SESSION_FINISH, ...); * * Restrictions * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D @@ -90,6 +96,7 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt =20 #include +#include #include #include #include @@ -203,13 +210,53 @@ static int vfio_pci_liveupdate_freeze(struct liveupda= te_file_op_args *args) return 0; } =20 +static int match_device(struct device *dev, const void *arg) +{ + struct vfio_device *device =3D container_of(dev, struct vfio_device, devi= ce); + const struct vfio_pci_core_device_ser *ser =3D arg; + struct pci_dev *pdev; + + pdev =3D dev_is_pci(device->dev) ? to_pci_dev(device->dev) : NULL; + if (!pdev) + return false; + + return ser->bdf =3D=3D pci_dev_id(pdev) && ser->domain =3D=3D pci_domain_= nr(pdev->bus); +} + static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) { - return -EOPNOTSUPP; + struct vfio_pci_core_device_ser *ser; + struct vfio_device *device; + struct file *file; + int ret =3D 0; + + ser =3D phys_to_virt(args->serialized_data); + + device =3D vfio_find_device(ser, match_device); + if (!device) + return -ENODEV; + + file =3D vfio_device_liveupdate_cdev_open(device); + if (IS_ERR(file)) { + ret =3D PTR_ERR(file); + goto out; + } + + args->file =3D file; +out: + /* Drop the reference from vfio_find_device() */ + put_device(&device->device); + return ret; +} + +static bool vfio_pci_liveupdate_can_finish(struct liveupdate_file_op_args = *args) +{ + return args->retrieve_status > 0; } =20 static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) { + kho_restore_free(phys_to_virt(args->serialized_data)); } =20 static const struct liveupdate_file_ops vfio_pci_liveupdate_file_ops =3D { @@ -218,6 +265,7 @@ static const struct liveupdate_file_ops vfio_pci_liveup= date_file_ops =3D { .unpreserve =3D vfio_pci_liveupdate_unpreserve, .freeze =3D vfio_pci_liveupdate_freeze, .retrieve =3D vfio_pci_liveupdate_retrieve, + .can_finish =3D vfio_pci_liveupdate_can_finish, .finish =3D vfio_pci_liveupdate_finish, .owner =3D THIS_MODULE, }; diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c index 2148b00491e4..33e082af9a08 100644 --- a/drivers/vfio/vfio_main.c +++ b/drivers/vfio/vfio_main.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -1787,6 +1788,18 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr= _t iova, void *data, } EXPORT_SYMBOL(vfio_dma_rw); =20 +struct vfio_device *vfio_find_device(const void *data, device_match_t matc= h) +{ + struct device *device; + + device =3D class_find_device(vfio.device_class, NULL, data, match); + if (!device) + return NULL; + + return container_of(device, struct vfio_device, device); +} +EXPORT_SYMBOL_GPL(vfio_find_device); + /* * Module/class support */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index f5f7ece51f8f..0b61aeff8216 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -431,4 +431,15 @@ int vfio_virqfd_enable(void *opaque, int (*handler)(vo= id *, void *), void vfio_virqfd_disable(struct virqfd **pvirqfd); void vfio_virqfd_flush_thread(struct virqfd **pvirqfd); =20 +#if IS_ENABLED(CONFIG_VFIO_DEVICE_CDEV) +struct file *vfio_device_liveupdate_cdev_open(struct vfio_device *device); +#else +static inline struct file *vfio_device_liveupdate_cdev_open(struct vfio_de= vice *device) +{ + return ERR_PTR(-EOPNOTSUPP); +} +#endif /* IS_ENABLED(CONFIG_VFIO_DEVICE_CDEV) */ + +struct vfio_device *vfio_find_device(const void *data, device_match_t matc= h); + #endif /* VFIO_H */ --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF34E378826 for ; Mon, 11 May 2026 23:48:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543318; cv=none; b=mUc5zUxCE7xT30JcUJDCLKnZAsv9P52IPzeVq395vY8Ln1GsbaVisjHG4va5Wy7cEhbfuGZ+SrQzSnpcC/YMPHEbuUiQ0S8QYSxj/JEucqR4oQdFuU34mjfFCaRyCUTWnDiwskZPnLgm95gfXyl+syzAol6RQPFXFm91omp4shk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543318; c=relaxed/simple; bh=jYAESqc70AKAPm2uQN0+BGW14vc4dGTA7ACtaEyAaQA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=evjMXESSGAs1qleHErpginiA0ZGBKjptXJ/Z2T2Inilx+lBtu8qIf6b9IHE2xJZ7Dmy9ffId7XLNXKFo7O1L1yWo+YPXVfKsuJzSyRT7p0b8UPrKcNTk3x1IH0N1b6iBC7DHF9jsYFf3NewLCcwq4Vu7sMW/CA9iWbL0VKD5qK0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PFUjGoak; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PFUjGoak" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2ba86e35aa1so74122985ad.1 for ; Mon, 11 May 2026 16:48:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543313; x=1779148113; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=W3NVtd4OK2CvqntR6Z53hO/1J0Yder27WCm5j/oDPnQ=; b=PFUjGoakT9zJjtd/apU/V8phW/8AuEDg6bJ7t+CL0TXOi80ozxaiRICrr53BMsFs+V A0+iRwvKydGd7SysfEV9hNZmrtxfs1+m0vsXyfK0JvDkjohMFjsGZXVtid29OniBGlQx vCJPMme+keCjR03ubtOOV3qp45ZeBEYr6XP6kfGVHLiYes0cHtf0qDjshDopNaUNijrR p+vwsddhbxN33e2iwjF/N0guuMQW4ngRtVUsnzPSw85iIQjl9oVQK7AnmzV039OqWDua 0r1pW1MEXV6LcavxYYXlsCfjJ8CiLsagiKGxQ6hdLpZTm3T0dvF2Kz+R1YQytHgFwvB+ EU9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543313; x=1779148113; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=W3NVtd4OK2CvqntR6Z53hO/1J0Yder27WCm5j/oDPnQ=; b=CgTa3YLQL+5q8SVRg2QQatezp4Q1WtL7cer3GpjQpEYAfmyFtosbtHVEcOxPANOVB2 ug6xM506iQzm0zxr1J4YokHVm/sDYRr2RHCBva8Ua93o2bZqNi4ZctYPd6lMrxOzj6Dm E4U2Bsyypv2ivOxVMNtYfKOSGWc71rbAm7HuRmzVhmxId5K7/4yH+dnF2R+WF7jAICQ/ lxa0uZrPB9NPBRKaG2+B8rqN3/ttro6c77YPVmuqUUo16LT9gKaSx0CU2GLufIS3eTvT z5mYsoNn6+XPwliTVy9uEva3p8kuH6rwutX/z5jDPGHT5qs1cWosOxVbWPKiXsbMa6+y 1XJg== X-Forwarded-Encrypted: i=1; AFNElJ/3yoAv9mJu3MbdteEtJjjZ/qdr3P2Q2Zd5TjHXNh2bHGeg6Fhfcs4DiDAjd0n/Ab1cSstqQ/w4kKIBet4=@vger.kernel.org X-Gm-Message-State: AOJu0YynxcmAi7Iwtu+v82Ot6Nh2o02Nf2o12bedGnlBw8U7ATKhBXF0 +L6WUWyYT/OrrYVx6OlvSDL3TC4i6xQMnuA8yuAyW22Eo9KqWq1nhr0JK0JuH675b/BLmIwadsE xsbc4GDPMjA== X-Received: from plud14.prod.google.com ([2002:a17:903:f6e:b0:2ae:3bca:37a5]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ebd1:b0:2bc:db02:d1ed with SMTP id d9443c01a7336-2bd0134098dmr6418155ad.38.1778543313150; Mon, 11 May 2026 16:48:33 -0700 (PDT) Date: Mon, 11 May 2026 16:47:50 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-5-vipinsh@google.com> Subject: [PATCH v4 04/16] vfio/pci: Notify PCI subsystem about devices preserved across Live Update From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Notify the PCI subsystem about devices vfio-pci is preserving across Live Update by registering the vfio-pci liveupdate file handler with the PCI subsystem's FLB handler. Notably this will ensure that devices preserved through vfio-pci will have their PCI bus numbers preserved across Live Update, allowing VFIO to use BDF as a key to identify the device across the Live Update and (in the future) allow the device to continue DMA operations across the Live Update. This also enables VFIO to detect that a device was preserved before userspace first retrieves the file from it, which will be used in subsequent commits. Signed-off-by: David Matlack Co-developed-by: Vipin Sharma Signed-off-by: Vipin Sharma --- drivers/vfio/pci/vfio_pci_liveupdate.c | 44 +++++++++++++++++++++++--- 1 file changed, 40 insertions(+), 4 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 9142aaea94f2..11c3bc8a8dcd 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -71,6 +71,9 @@ * interrupts on the device will cause the ``reboot(LINUX_REBOOT_CMD_KE= XEC)`` * syscall (to initiate the kexec) to fail. * + * In addition, the device must meet all of the restrictions imposed by the + * core PCI layer documented at :doc:`/PCI/liveupdate`. + * * Preservation Behavior * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * @@ -141,23 +144,37 @@ static int vfio_pci_liveupdate_preserve(struct liveup= date_file_op_args *args) struct vfio_pci_core_device_ser *ser; struct vfio_pci_core_device *vdev; struct pci_dev *pdev; + int ret; =20 vdev =3D container_of(device, struct vfio_pci_core_device, vdev); pdev =3D vdev->pdev; =20 + ret =3D pci_liveupdate_preserve(pdev); + if (ret) + return ret; + ser =3D kho_alloc_preserve(sizeof(*ser)); - if (IS_ERR(ser)) - return PTR_ERR(ser); + if (IS_ERR(ser)) { + ret =3D PTR_ERR(ser); + goto err_unpreserve; + } =20 ser->bdf =3D pci_dev_id(pdev); ser->domain =3D pci_domain_nr(pdev->bus); =20 args->serialized_data =3D virt_to_phys(ser); return 0; + +err_unpreserve: + pci_liveupdate_unpreserve(pdev); + return ret; } =20 static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args = *args) { + struct vfio_device *device =3D vfio_device_from_file(args->file); + + pci_liveupdate_unpreserve(to_pci_dev(device->dev)); kho_unpreserve_free(phys_to_virt(args->serialized_data)); } =20 @@ -256,6 +273,15 @@ static bool vfio_pci_liveupdate_can_finish(struct live= update_file_op_args *args) =20 static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) { + struct vfio_device *device; + + if (!args->file) { + pr_err("Finish called on an unretrieved file.\n"); + return; + } + + device =3D vfio_device_from_file(args->file); + pci_liveupdate_finish(to_pci_dev(device->dev)); kho_restore_free(phys_to_virt(args->serialized_data)); } =20 @@ -280,13 +306,23 @@ int __init vfio_pci_liveupdate_init(void) int ret; =20 ret =3D liveupdate_register_file_handler(&vfio_pci_liveupdate_fh); - if (ret && ret !=3D -EOPNOTSUPP) - return ret; + if (ret) + goto err_return; + + ret =3D pci_liveupdate_register_flb(&vfio_pci_liveupdate_fh); + if (ret) + goto err_unregister; =20 return 0; + +err_unregister: + liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); +err_return: + return (ret =3D=3D -EOPNOTSUPP) ? 0 : ret; } =20 void vfio_pci_liveupdate_cleanup(void) { + pci_liveupdate_unregister_flb(&vfio_pci_liveupdate_fh); liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); } --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4D0F36A36D for ; Mon, 11 May 2026 23:48:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543316; cv=none; b=EciuEOeJnsOYOUhsYGTRoqY7AftZIlGQXaw4Bo2QmKwVdcUdovC0D0NmVolkBJM9ui6Qa5fuJpJyvEFuQ2bHiXfKSUVgyfPEUavlkTf79n/XaiQ+dT9FV3V8GM+qNer7XHJNjMhgSUNyISYhK7XCHXFGYUldY5YvMplzsyWiLUc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543316; c=relaxed/simple; bh=Ddf0Lby4MdD14pq6niokXK/rA0eJRVzLYmia/RpI6xo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cJ5ypCOnDc92+UQAdSNwmqgkJAOskNwusndodM90UxmqoqtXeAeJT8IrAPlgT+dZore3TEbEhYd33lKF8jPxhSb055jlGgIzpvPR3QWmwGO946pzX3CUMP3iRMh71YC+dB0abuavFPeUCLj/EmO2ZVzyJwFBt3CtyQVmXVp/dKA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=paOEj/na; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="paOEj/na" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-c82894155ceso2136383a12.1 for ; Mon, 11 May 2026 16:48:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543314; x=1779148114; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=e5EUY3ZlaySk7yEluRmGCiDPZ2d7y5O8uLlUW0QQA8U=; b=paOEj/naC9KSIGYgnctDtpeloG9k7E8iViHgX7L4b7CkWdQCFPj1Ly8qhwAKuIQaPp xtuNhQmFjZBPwG4huIH3lsU5U6XGdz1nyHckWCkGZCBNJG4JiGOcnag6bWtg8CD2BB4x tN+Xqh6/C7ng/7iLwr/h+nt57WHEu7yd9h+2fZnf5+FNC90WqaIZrdHoMzYdttO6ROx3 wDzu1ulbvJUgKA33t/LnKA8CvL37uTkdBZA82e4fN96ES1sJju26QjHl37kMGH/MAgVd tin00xL9tagafhNlbu1hFz7ffWH9Lp26wsoRgEcXKPY52wuDjHO8GBcX/SMSM+AZcICA Z2Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543314; x=1779148114; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=e5EUY3ZlaySk7yEluRmGCiDPZ2d7y5O8uLlUW0QQA8U=; b=McO6deAmeDSH/1cjQtgCTVvPmzX681GZai8bS2UXHQY0DlR0eZhDChv65D8NK6lLPW 6UBlMrm9UfISRd7PRFYNd6J0xuYSpD4Zr3ailFM0aVqfY2GjiihaoI6vQGdr6uqdiW3R F8Oio6kBErkzyTnjNGJIw9KZ4l2PU5l4Y/+fV66gjSn5isO3GkzIf4MaHCLplG5noZj0 Wv2kypDlQnAHOMaB3a2EbcBA6osL5qjTCWFygGD78fhJQtk4xeCy2W3DE5YBUf3wGVc7 87HWxPyMjovADbUvv1cKw3xJf+qZuO0wJ7G28AtcK89fiA7+jGdTWs5Av1bFtcklQ3kj P1xg== X-Forwarded-Encrypted: i=1; AFNElJ+AftR4WT/XMugB4EoTixXrFRWL9jEiOTesokwhGOFqYP8oZeyO8ku4g5TXo66QXFGwXi4+UmSbqIk7NK0=@vger.kernel.org X-Gm-Message-State: AOJu0YzDiWxcEBAYSTxtJ6EDLYYbq1xCNkZFHgrG2ae+lOixtYRS46WF 6fmTckdT5L1irdYOv1KEZn7EK5ET5eCg0K2ySm5nUKU25G9yDOi0EBC8FzVntC46trS4FW5c9WG lHQVkc1cfhQ== X-Received: from pgbcq5.prod.google.com ([2002:a05:6a02:4085:b0:c79:8a8e:b046]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:72a1:b0:39b:9644:6e94 with SMTP id adf61e73a8af0-3aad425fdf0mr13009821637.9.1778543314037; Mon, 11 May 2026 16:48:34 -0700 (PDT) Date: Mon, 11 May 2026 16:47:51 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-6-vipinsh@google.com> Subject: [PATCH v4 05/16] vfio: Enforce preserved devices are retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Enforce that files for incoming (preserved by previous kernel) VFIO devices are retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD rather than by opening the corresponding VFIO character device or via VFIO_GROUP_GET_DEVICE_FD. Both of these methods would result in VFIO initializing the device without access to the preserved state of the device passed by the previous kernel. Reviewed-by: Pranjal Shrivastava Signed-off-by: David Matlack Co-developed-by: Vipin Sharma Signed-off-by: Vipin Sharma --- drivers/vfio/device_cdev.c | 8 ++++++++ drivers/vfio/group.c | 9 +++++++++ drivers/vfio/pci/vfio_pci_liveupdate.c | 6 ++++++ drivers/vfio/vfio.h | 18 ++++++++++++++++++ 4 files changed, 41 insertions(+) diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c index 1ab07ccaf3ab..4df0495941c6 100644 --- a/drivers/vfio/device_cdev.c +++ b/drivers/vfio/device_cdev.c @@ -49,6 +49,14 @@ static int vfio_device_cdev_open(struct vfio_device *dev= ice, struct file **filep } =20 *filep =3D file; + } else if (vfio_liveupdate_incoming_is_preserved(device)) { + /* + * Since it is live update preserved device, it must be + * retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD instead of + * opening /dev/vfio/devices/vfioX. + */ + ret =3D -EBUSY; + goto err_free_device_file; } =20 file->private_data =3D df; diff --git a/drivers/vfio/group.c b/drivers/vfio/group.c index b2299e5bc6df..62b4eaabc829 100644 --- a/drivers/vfio/group.c +++ b/drivers/vfio/group.c @@ -316,6 +316,15 @@ static int vfio_group_ioctl_get_device_fd(struct vfio_= group *group, if (IS_ERR(device)) return PTR_ERR(device); =20 + /* + * This device was preserved across a Live Update. Accessing it via + * VFIO_GROUP_GET_DEVICE_FD is not allowed. + */ + if (vfio_liveupdate_incoming_is_preserved(device)) { + vfio_device_put_registration(device); + return -EBUSY; + } + fd =3D FD_ADD(O_CLOEXEC, vfio_device_open_file(device)); if (fd < 0) vfio_device_put_registration(device); diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 11c3bc8a8dcd..731a3e34085f 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -47,6 +47,12 @@ * ... * ioctl(session_fd, LIVEUPDATE_SESSION_FINISH, ...); * + * .. note:: + * After kexec, if a device was preserved by the previous kernel, attem= pting + * to open a new file for the device via its character device + * (``/dev/vfio/devices/X``) or via ``VFIO_GROUP_GET_DEVICE_FD`` will f= ail + * with ``-EBUSY``. + * * Restrictions * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h index 0854f3fa1a22..5269fe021ee3 100644 --- a/drivers/vfio/vfio.h +++ b/drivers/vfio/vfio.h @@ -11,6 +11,7 @@ #include #include #include +#include =20 struct iommufd_ctx; struct iommu_group; @@ -461,4 +462,21 @@ static inline void vfio_device_debugfs_init(struct vfi= o_device *vdev) { } static inline void vfio_device_debugfs_exit(struct vfio_device *vdev) { } #endif /* CONFIG_VFIO_DEBUGFS */ =20 +#ifdef CONFIG_PCI_LIVEUPDATE +static inline bool vfio_liveupdate_incoming_is_preserved(struct vfio_devic= e *device) +{ + struct device *d =3D device->dev; + + if (dev_is_pci(d)) + return to_pci_dev(d)->liveupdate_incoming; + + return false; +} +#else +static inline bool vfio_liveupdate_incoming_is_preserved(struct vfio_devic= e *device) +{ + return false; +} +#endif /* CONFIG_PCI_LIVEUPDATE */ + #endif --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7A1E3B7744 for ; Mon, 11 May 2026 23:48:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543318; cv=none; b=syobsOxDe2c1W2+hZr4GzAUlLKY9T2Ok/o+GzKaZxK78pCm/U5/Z9+5iY4nAKoE/2bKkTuFvaREhbvhjDGZ8NOYkINYh9juKnmbarCFH78bME5dDupwBAaPPo4/5Mg87EWUHOZWxiqkoX+1aTpAEvIOda+yAAuqz9nqO+6X0vZM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543318; c=relaxed/simple; bh=3ytzMoApkUjmFhiyRDC37NlP+bK8f+8Zi9rhfzytfEI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=q3AwbtLYVYzPLCuqVk1Ujts2XfzDGaCgWkHDwiF3RX+fc6hPkzm/Hxb0+LyWhiTNkNJbpXONPXDKlRCoQV8PEIdXvts9oRktTL1kmOa9SneeIl6O6spmpnrm765vxaGVBGaczhSupITRU/2YBI21qGmv8R/DWA+qHZU1dK8ntRg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XUAGvJYQ; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XUAGvJYQ" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-366ded5ab5fso6531703a91.0 for ; Mon, 11 May 2026 16:48:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543315; x=1779148115; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lsypqsh627xWDj14lndk3F1iTuQ4MYPrH0UMeFATjE8=; b=XUAGvJYQ8Tfrpf073ilcEYbRyiPMXrScNEZY2MjqOmmVKNcHeCIFhoDgdIuhQObH3e 1Y2X4+KSnwUg15JQiCbewy9WZ0lJ0Mm+NZS65AnNxxEpHBl0K/1OH6qRVg43//Xscr/M f1Qo0QEQAVH2fsIQ2ImKkldjbCdqf5COeLepE7yOvKT3o1DFK9wdCUNOu6nsIzdefbTw ghY6bqIMP8QOGSuZXLrpXwxdtaLwuhV7hM+XbI81+SdgbTlUkY8rySCCiXOygQTQGC55 ajIicHcKqiTwcLfXzsdSCntEz3ZC3ufdf9S178jLg6vx01XrQ4fl6SoDtHq70GlY69Gq 1e8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543315; x=1779148115; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lsypqsh627xWDj14lndk3F1iTuQ4MYPrH0UMeFATjE8=; b=FY0J0MDoAd+K0Ifd4gqo4l5xWzDbiINOv28H0r4WD9kJ2XPjG3l2p1czPqbZvnOS2o Qaf/Q4idN+ajU6/pikXZ9ht9PllhIRMU72Rvd/gGMb60wMozxngQfFZmGqvdp0/T3pM3 CHjLDhP0Da938ypYwuPHBnCll+ghAdmAUUCWbAotugJViCPV5er8Iappyf+OFFru+8V2 d1ya+5B8wr9dwKPQcIRCR6VRAzxVGdpsOZQ+Kd9pqGHOm/Vb08vLXVka9l6nVAv+5+G2 c9q9LneuznbZSdz9+buSWckut45oES35S4lGWmmj1IeGznnXpX50Se4lXJn9INyQRyKe sZfA== X-Forwarded-Encrypted: i=1; AFNElJ8s7sT0dL7+aJjSQ003pecXSpGjg7Y00oRuCt2WjbOLRaZ5qsLjKXO8cO8FmH2wAqv7BsJcrMpyO+qRuvA=@vger.kernel.org X-Gm-Message-State: AOJu0YwdyHdxksb5hIoD9WoIl/KVMuWS2vI5F1oWT4pDWdj1zy+I4Yr6 6p5QA8avl8tu89xAcDfIZiy7UFfvXf+OtY+DA2smpNxN61lyQyYG8CW5EVIFdCR9m6w/ufHtfms wWwHTpzxabQ== X-Received: from pjbkl7.prod.google.com ([2002:a17:90b:4987:b0:366:29cd:da0d]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1b0c:b0:35f:b5df:463 with SMTP id 98e67ed59e1d1-367d484654amr11616671a91.14.1778543315098; Mon, 11 May 2026 16:48:35 -0700 (PDT) Date: Mon, 11 May 2026 16:47:52 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-7-vipinsh@google.com> Subject: [PATCH v4 06/16] vfio/pci: Store incoming Live Update state in struct vfio_pci_core_device From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Stash a pointer to a device's incoming Live Updated state in struct vfio_pci_core_device. This will enable subsequent commits to use the preserved state when initializing the device. To enable VFIO to safely access this pointer during device enablement, require that the device is fully enabled before returning true from can_finish(). This is synchronized by vfio_pci_core.c setting vdev->liveupdate_incoming_state to NULL under dev_set lock once it's done using it. Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- drivers/vfio/pci/vfio_pci_core.c | 2 +- drivers/vfio/pci/vfio_pci_liveupdate.c | 17 ++++++++++++++++- include/linux/vfio_pci_core.h | 2 ++ 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 1726d4d41b57..da013b4b1e94 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -568,7 +568,7 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *v= dev) if (!vfio_vga_disabled() && vfio_pci_is_vga(pdev)) vdev->has_vga =3D true; =20 - + vdev->liveupdate_incoming_state =3D NULL; return 0; =20 out_free_zdev: diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 731a3e34085f..7b841a1e7fcc 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -249,6 +249,7 @@ static int match_device(struct device *dev, const void = *arg) static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) { struct vfio_pci_core_device_ser *ser; + struct vfio_pci_core_device *vdev; struct vfio_device *device; struct file *file; int ret =3D 0; @@ -265,6 +266,9 @@ static int vfio_pci_liveupdate_retrieve(struct liveupda= te_file_op_args *args) goto out; } =20 + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + vdev->liveupdate_incoming_state =3D ser; + args->file =3D file; out: /* Drop the reference from vfio_find_device() */ @@ -274,7 +278,18 @@ static int vfio_pci_liveupdate_retrieve(struct liveupd= ate_file_op_args *args) =20 static bool vfio_pci_liveupdate_can_finish(struct liveupdate_file_op_args = *args) { - return args->retrieve_status > 0; + struct vfio_pci_core_device *vdev; + struct vfio_device *device; + + if (args->retrieve_status <=3D 0) + return false; + + device =3D vfio_device_from_file(args->file); + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + + /* Check that vdev->liveupdate_incoming_state is no longer in use. */ + guard(mutex)(&device->dev_set->lock); + return !vdev->liveupdate_incoming_state; } =20 static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 2ebba746c18f..0c508dd8d1ac 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -26,6 +26,7 @@ #define VFIO_PCI_OFFSET_MASK (((u64)(1) << VFIO_PCI_OFFSET_SHIFT) - 1) =20 struct vfio_pci_core_device; +struct vfio_pci_core_device_ser; struct vfio_pci_region; struct p2pdma_provider; struct dma_buf_attachment; @@ -142,6 +143,7 @@ struct vfio_pci_core_device { struct notifier_block nb; struct rw_semaphore memory_lock; struct list_head dmabufs; + struct vfio_pci_core_device_ser *liveupdate_incoming_state; }; =20 enum vfio_pci_io_width { --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C6443BED74 for ; Mon, 11 May 2026 23:48:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543320; cv=none; b=IFfA4NQmEmuFnuggHLNP1Rj6e+k/5yBXyhfVob0CDPwj8SU56bpqyc7DnFcBX1k7wjVebxQ5qIqzMOr1TqMYIMnSubT540R+P2v/A/px46t9xFFHgu8itRqSQITfIZFFLCRurgwR8r/mL2s5PPRFSmCOKNn1uMh62yHi1AZ1xII= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543320; c=relaxed/simple; bh=2O3kiZbxsoqPdSkkFfZY/ZsPS5GCY/RmMZMzns7zjr4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qFpkJfnKgjxxTzmExwZOdwgm6QosczJtPQXQV1IvlUGSdzqsS+/mUd+gN4/i1Lx5Ajbb2mtGqjwxZ7v4bpiTmwqgYbn03pr4nysARyujisYgk+BexQt7C7R1qUPMlT96RZHUYx46rwMoE3Fo7axhow00vHq51P2w1GxJqYBt0/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ipDJcFIh; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ipDJcFIh" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-366122e01fcso5428327a91.2 for ; Mon, 11 May 2026 16:48:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543316; x=1779148116; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ciHF9s89MGSV42RioHjLjFryxlWaYoxn0O9vl4XFI1I=; b=ipDJcFIhdiURZAVFmzT9q6Vqy7RliouHFFE1Xj3ZlmXy1nMO7bMTgNrgVJiOODxoB9 at+NI71ZRYNF3bF25/xIP7qoMEjWIMGJgYyUof/kOkxEytVAw7jmKOnPxSPdGt8ikFCT ZGrC7WfjcKmDsPlDBnX5joQ1O2Gt3Ywuw8/9sWeZ8S2EVbSGYG3IhkonGJMIg1KhAaBJ jVBdAHRB1qgZ5qD1AbWLGXW5PYuSbZcbTziTAbidlENoM3ZDQF+etwybV2X56yHcyrt2 LvWhLQ6FhXIhVlqSkglp+DkPhJydEblQA4Fq3AHjhfzDSGsi69jYc5quRyZ81IwNtJx5 KQFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543316; x=1779148116; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ciHF9s89MGSV42RioHjLjFryxlWaYoxn0O9vl4XFI1I=; b=e8pZklDLey17ydYQeFUPtfoT7XEcpBOAGdF7JCcgjtqfnxxAEpLa3FwX9EkAT06mRe iA66QDt+Sg9IJ+LRfB6Z1agMwa3GICJ4iGpl6OicXidapiQun9GdL3MucGOHvjZJT4Fc fH+DALUXishrAOZBBfa/VMpMXwI8ZXYV/uUGkFfBTCMEpRDhgUjSEoZ++1YguY4sSCwq GdHPRwep1mAZw+VxBROntsUenAPgwZDYVnSE5CJruQ3QNLmarvNiwQaAUs+YtcXnpoey SPEfTzaKb2owzhZIccJc8EA2F4h9mCntqFZx+Cs8d/2vgyQmjIl7SzocbTPfpSuvk4Ey M2qA== X-Forwarded-Encrypted: i=1; AFNElJ8fn3Y/7eHIi1E4nUKIoJftzV+deOQj5jJn5SCEyPPUaIWhwsKsj/jMrHyXmTGnB4B33w39OQX4AZ/wzig=@vger.kernel.org X-Gm-Message-State: AOJu0YwrQzkluDabmUIG78enAuHxri8R01D5RyxdTv46nNsVr84oFAyq dZ6b1WKDXcuUYPPuUpCyh66G1zHVUiD6O3w10O7rgW3rm2SyVuVm70gUdn+5UK8CrMBzb/woDyr t1XvBVvhwNA== X-Received: from pjbdt11.prod.google.com ([2002:a17:90a:fa4b:b0:368:aec2:8d2f]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1cce:b0:368:5367:d67e with SMTP id 98e67ed59e1d1-368b24ffb1bmr755202a91.10.1778543316136; Mon, 11 May 2026 16:48:36 -0700 (PDT) Date: Mon, 11 May 2026 16:47:53 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-8-vipinsh@google.com> Subject: [PATCH v4 07/16] docs: liveupdate: Add documentation for VFIO PCI From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Add documentation for preserving VFIO device files across a Live Update, generated from the kernel-doc comments in the code. Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- Documentation/core-api/liveupdate.rst | 1 + .../driver-api/vfio_pci_liveupdate.rst | 23 +++++++++++++++++++ MAINTAINERS | 1 + 3 files changed, 25 insertions(+) create mode 100644 Documentation/driver-api/vfio_pci_liveupdate.rst diff --git a/Documentation/core-api/liveupdate.rst b/Documentation/core-api= /liveupdate.rst index d56a7760978a..c55d0d9d1d3b 100644 --- a/Documentation/core-api/liveupdate.rst +++ b/Documentation/core-api/liveupdate.rst @@ -34,6 +34,7 @@ The following types of file descriptors can be preserved :maxdepth: 1 =20 ../mm/memfd_preservation + ../driver-api/vfio_pci_liveupdate =20 Public API =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/Documentation/driver-api/vfio_pci_liveupdate.rst b/Documentati= on/driver-api/vfio_pci_liveupdate.rst new file mode 100644 index 000000000000..1098b84e5ecd --- /dev/null +++ b/Documentation/driver-api/vfio_pci_liveupdate.rst @@ -0,0 +1,23 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +VFIO PCI Device Preservation via LUO +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. kernel-doc:: drivers/vfio/pci/vfio_pci_liveupdate.c + :doc: VFIO PCI Preservation via LUO + +VFIO PCI Preservation ABI +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D + +.. kernel-doc:: include/linux/kho/abi/vfio_pci.h + :doc: VFIO PCI Live Update ABI + +.. kernel-doc:: include/linux/kho/abi/vfio_pci.h + :internal: + +See Also +=3D=3D=3D=3D=3D=3D=3D=3D + +- :doc:`/core-api/liveupdate` +- :doc:`/core-api/kho/index` diff --git a/MAINTAINERS b/MAINTAINERS index b6c33c2bcc7c..737c5ed1ce38 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -27979,6 +27979,7 @@ T: git https://github.com/awilliam/linux-vfio.git F: Documentation/ABI/testing/debugfs-vfio F: Documentation/ABI/testing/sysfs-devices-vfio-dev F: Documentation/driver-api/vfio.rst +F: Documentation/driver-api/vfio_pci_liveupdate.rst F: drivers/vfio/ F: include/linux/kho/abi/vfio_pci.h F: include/linux/vfio.h --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 191173C1412 for ; Mon, 11 May 2026 23:48:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543320; cv=none; b=S+h3it+YDau4ykRgeuHISgcdugaVolF0V39jeoqzFPTMAgkLMOrLCWBPKL506p75ylnr4n72mfajPF20DMiJ2VwCN9bYWq5yIRq79tz/hYjOo7f9CfYv3VUrirGrRN1VWAhstDMg5KELXPg6zOt/RYGjBz0m31Z19ZNezgEIg3E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543320; c=relaxed/simple; bh=21W6Ql13lrjgrDgG4oHa5rhHZslFZu4s8ImhCjMl8pw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ickIb/bg1HP+cMcTMXbEk//MPiQr30mjAT3s5fsmT5qZ8GxpkZQEDTlfxtEGqkz/oDyRYV3D9v/ALZ7EprnwSeg2826EtdTi0GVdUKvadfP2uSM7NnxolqaiqCP5cIci6RKjKrMDtY9uS539wbA30ztyS2dKgJrsoAbbXInK/a0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=DGhRHcFo; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="DGhRHcFo" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-367c2d149efso3322324a91.1 for ; Mon, 11 May 2026 16:48:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543317; x=1779148117; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7WP44kmfFcYhlyRsivHdlontJq1i9E72JydBw+wZ7hQ=; b=DGhRHcFoRSbWLKENOCnbGVN6u2kvANDJj1uUOhONgntWUtPEEKUpXJaeQrfElEMgOk N0Iv8SPEr3NY/kpJv40UOM9DknWpgpqUbQcKXFGXeSfnOg89yED2/9t2ubqKgeQ7z8Tm 5Sdg3lUCkmNuyQTmsj2cT7l//XqTHkN5ch2np2d7+kvU3vowjQ9KpugSxUQs67d2IMJV w5Uc+IXE9ex3FImVFsvcBhPxfiP3dYhA1+PMS1pgR3llekBVb/gClqelNHEn4HOYqlLS EPNQfCuWEsaoecSyMpo23Zojc21v8cpF4Mi+9uh2VdhRATEuRtmlueEGpZa/0TNdfvvP J2rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543317; x=1779148117; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7WP44kmfFcYhlyRsivHdlontJq1i9E72JydBw+wZ7hQ=; b=GNbJgItVj0juDKrq6OmFaMMmJT3gGFX2iEqmRUr2BkOBVo341nCaRv+YBnFHngVs+O yJrBmtia7cD6WvGnRFIJl94SJnl26LUMx/6RSqHqIPWdUEOUOakKuUP2UAvpsqVEFHHP KOBgYbrsANT984Md5Hx/U3vJ6GiFWxrBcCYHwdGI5AQQZR5NaaiTbPyg9xjh+XHXPiOp ACVNmUjLIR0912PIk94sPHMCsvXepUJJuQDDcm42STW8NcJWpegTlg8jdB4Vndide45x mIxXHmQqX4nbmJ33kkNZcQIyjuG2K6eE7X8p/tVOM4wV5uPiE7cJQww/Vx56az4cnFcq rSJw== X-Forwarded-Encrypted: i=1; AFNElJ+vwyw2RvBP33sHK3F6ybdW+mSc0SrsIspr3WovO0gfcZ/aR5aGKGLBFFMwax6jjhVH0MUcK1e65IdTc0I=@vger.kernel.org X-Gm-Message-State: AOJu0YwhWODD9UnFrm++xQuh17UXQgFPHOGq6tyoUIY9+UoebYoxTC83 ji6GDWa+aK/X5JL8HZWsPBTAxSW3LgHFG8byQ5BUWZys6Fh4uMSJCU13/vVlPx8TXsYrIrnaf1D VgFkUPf7Sqw== X-Received: from pgjw1.prod.google.com ([2002:a63:fb41:0:b0:c80:15fa:8bd5]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4b0a:b0:368:3030:4a65 with SMTP id 98e67ed59e1d1-36830304d85mr7869135a91.13.1778543317042; Mon, 11 May 2026 16:48:37 -0700 (PDT) Date: Mon, 11 May 2026 16:47:54 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-9-vipinsh@google.com> Subject: [PATCH v4 08/16] vfio: selftests: Build liveupdate library in VFIO selftests From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Import and build liveupdate selftest library in VFIO selftests. It allows to use liveupdate ioctls in VFIO selftests Co-developed-by: David Matlack Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- tools/testing/selftests/vfio/Makefile | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 0684932d91bf..7865b2aa011c 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -20,6 +20,7 @@ TEST_FILES +=3D scripts/setup.sh =20 include ../lib.mk include lib/libvfio.mk +include ../liveupdate/lib/libliveupdate.mk =20 CFLAGS +=3D -I$(top_srcdir)/tools/include CFLAGS +=3D -MD @@ -27,11 +28,15 @@ CFLAGS +=3D $(EXTRA_CFLAGS) =20 LDFLAGS +=3D -pthread =20 -$(TEST_GEN_PROGS): %: %.o $(LIBVFIO_O) - $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $< $(LIBVFIO_O) $(LDLIBS) -o $@ +LIBS_O :=3D $(LIBVFIO_O) +LIBS_O +=3D $(LIBLIVEUPDATE_O) + +$(TEST_GEN_PROGS): %: %.o $(LIBS_O) + $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIBS_O) $(LDLI= BS) -o $@ =20 TEST_GEN_PROGS_O =3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) -TEST_DEP_FILES =3D $(patsubst %.o, %.d, $(TEST_GEN_PROGS_O) $(LIBVFIO_O)) +TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(TEST_GEN_PROGS_O)) +TEST_DEP_FILES +=3D $(patsubst %.o, %.d, $(LIBS_O)) -include $(TEST_DEP_FILES) =20 EXTRA_CLEAN +=3D $(TEST_GEN_PROGS_O) $(TEST_DEP_FILES) --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E0C638A72C for ; Mon, 11 May 2026 23:48:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543322; cv=none; b=o3xy1+Fb804Ap/4WPE9edlSl40T2zIXKcFUg/jCUNTHVxzaub9X1SVBhaSanVZv46j4AucJWbZvpV1drxCtdlNkL5dyKHRXUCzC5U/FTLSkmRJHGscirR3mSJbq701U9Do1EI+FIDbujxGYCGV869K7GM+DjuGWFAdXxxDhqnaE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543322; c=relaxed/simple; bh=xXmk5pZJDMgIjU18pL7UFgkHN+p/KhwkXx8qVOwhLq4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sl1mDKhrwule+XUDl42/D6fqkC0rvKZC+VI5hq3o/ds/u6ikwxeeR750irvzsAdUxb70m3qwRTiIIBZq46shsBrJn+WVDBe4OBQ6uW4ruW7Ff2lyCb5oBH8zgNLmWIc2mTC/qGhkFaBoEVHigyrQYe2ABuj7Q6d82nuOnNcb0gc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Tx/te94p; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Tx/te94p" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-3662990c03fso3147589a91.0 for ; Mon, 11 May 2026 16:48:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543318; x=1779148118; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=MVGH18xW5LW+rwJCXWUBd4c2GaIpMJfCuXBDhFQuypI=; b=Tx/te94pjs2tHeUjndCYmfeoxFd2wE7IkAP7ThGVuqtOCmmZkIWJRiWWurj+dBCgcj wfrzeVeEmPdvxhVL4ziT45d6Q5IEdSXWTfhwdKDVpOeGNMG0bP/MpgQCFTYaQ3aS/apk /w5WoFbdH63GJ3VUZol2sjzrnP2rTGRYDZmuqaXVo0bukG1UMgk96NQOk7fQDnUvIXFu i2wdZgvGTn1CIo8ybuSgjhU8MLKH7nQQHX2XpuSJOdraRwjxceML90+tGGleSVJY1RUA 2ZT+2UrISQbarakfIkvypM0Jb4KP81Nz+lhdoQcQxTO0RQPHI4JKFE51Cc1WgMRvYBbq QWaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543318; x=1779148118; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MVGH18xW5LW+rwJCXWUBd4c2GaIpMJfCuXBDhFQuypI=; b=A8Hg4r6jipvtlR8DCcv+GaYyMKFjzqHjYPCxzALChrtA4kcN7Dwcb4qm0GoHV08r1i GjPE5hL6CWQYwJYvY1LOlmYQJnHEfw7lA8MoCqEVJIBXKWaeZjkaHClGQCI2dUGjqKtC +N+UWbbAehZ5z9EXePASkcPPCSDT896KdnrnKe9yoyfbZbvS0Y5t4CwYyhRiJuRDmsp2 t8zI2XPjSTumCugQArz2uhhKYzwtE6v/qbYBqZMt5BPeTfMPnjhYNTpuDnpbJuhToPyq SjR4jFRCXDAdNCYTlRqYGCWEivOnXx47VAJrYDzoiVkIFrBp5utv8KeSgkKk3Hamnlkt 9hEg== X-Forwarded-Encrypted: i=1; AFNElJ/zaU+XiRfq5EMYLG/gJLebLg+xZWR9yeHtFNiLjOcUmazqErG4HmB0vHLlXBQQIbHgPZoX6ZvRC9khZm0=@vger.kernel.org X-Gm-Message-State: AOJu0YyHrdp++gFDTciWnU5+12StUiap68obbG+d3HiBkbk11qRn0Yka 0ep3IgXx0G23jDlSt4LZ1Zo5TOavs3NYmyjE+95FOcpDQpL7S6VM6zuqnAXvjr7dkYPlCKFVQdu Q/+cSo/Mqjg== X-Received: from pjbsk11.prod.google.com ([2002:a17:90b:2dcb:b0:35e:58a0:798e]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4a0d:b0:368:2ba0:5e8a with SMTP id 98e67ed59e1d1-368ab863a90mr1167614a91.6.1778543318040; Mon, 11 May 2026 16:48:38 -0700 (PDT) Date: Mon, 11 May 2026 16:47:55 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-10-vipinsh@google.com> Subject: [PATCH v4 09/16] vfio: selftests: Add vfio_pci_liveupdate_uapi_test From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Add a selftest to exercise preserving a various VFIO files through /dev/liveupdate. Ensure that VFIO cdev device files can be preserved and everything else (group-based device files, group files, and container files) all fail. Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- tools/testing/selftests/vfio/Makefile | 1 + .../vfio/vfio_pci_liveupdate_uapi_test.c | 93 +++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_t= est.c diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 7865b2aa011c..df3e44660bd0 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -12,6 +12,7 @@ TEST_GEN_PROGS +=3D vfio_iommufd_setup_test TEST_GEN_PROGS +=3D vfio_pci_device_test TEST_GEN_PROGS +=3D vfio_pci_device_init_perf_test TEST_GEN_PROGS +=3D vfio_pci_driver_test +TEST_GEN_PROGS +=3D vfio_pci_liveupdate_uapi_test =20 TEST_FILES +=3D scripts/cleanup.sh TEST_FILES +=3D scripts/lib.sh diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c b= /tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c new file mode 100644 index 000000000000..1d89b08ab0a4 --- /dev/null +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include +#include + +static const char *device_bdf; + +FIXTURE(vfio_pci_liveupdate_uapi_test) { + int luo_fd; + int session_fd; + struct iommu *iommu; + struct vfio_pci_device *device; +}; + +FIXTURE_VARIANT(vfio_pci_liveupdate_uapi_test) { + const char *iommu_mode; +}; + +#define FIXTURE_VARIANT_ADD_IOMMU_MODE(_iommu_mode) \ +FIXTURE_VARIANT_ADD(vfio_pci_liveupdate_uapi_test, _iommu_mode) { \ + .iommu_mode =3D #_iommu_mode, \ +} + +FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(); +#undef FIXTURE_VARIANT_ADD_IOMMU_MODE + +FIXTURE_SETUP(vfio_pci_liveupdate_uapi_test) +{ + self->luo_fd =3D luo_open_device(); + ASSERT_GE(self->luo_fd, 0); + + self->session_fd =3D luo_create_session(self->luo_fd, "session"); + ASSERT_GE(self->session_fd, 0); + + self->iommu =3D iommu_init(variant->iommu_mode); + self->device =3D vfio_pci_device_init(device_bdf, self->iommu); +} + +FIXTURE_TEARDOWN(vfio_pci_liveupdate_uapi_test) +{ + vfio_pci_device_cleanup(self->device); + iommu_cleanup(self->iommu); + close(self->session_fd); + close(self->luo_fd); +} + +TEST_F(vfio_pci_liveupdate_uapi_test, preserve_device) +{ + int ret; + + ret =3D luo_session_preserve_fd(self->session_fd, self->device->fd, 0); + + /* Preservation should only be supported for VFIO cdev files. */ + ASSERT_EQ(ret, self->iommu->iommufd ? 0 : -ENOENT); +} + +TEST_F(vfio_pci_liveupdate_uapi_test, preserve_group_fails) +{ + int ret; + + if (self->iommu->iommufd) + SKIP(return, "iommufd-mode does not have group files"); + + ret =3D luo_session_preserve_fd(self->session_fd, self->device->group_fd,= 0); + ASSERT_EQ(ret, -ENOENT); +} + +TEST_F(vfio_pci_liveupdate_uapi_test, preserve_container_fails) +{ + int ret; + + if (self->iommu->iommufd) + SKIP(return, "iommufd-mode does not have container files"); + + ret =3D luo_session_preserve_fd(self->session_fd, self->iommu->container_= fd, 0); + ASSERT_EQ(ret, -ENOENT); +} + +int main(int argc, char *argv[]) +{ + int fd; + + fd =3D luo_open_device(); + if (fd < 0) + ksft_exit_skip("open(%s) failed: %s, skipping\n", + LUO_DEVICE, strerror(errno)); + + close(fd); + + device_bdf =3D vfio_selftests_get_bdf(&argc, argv); + return test_harness_run(argc, argv); +} --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 110BE3B7B7A for ; Mon, 11 May 2026 23:48:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543322; cv=none; b=BJ1YXbGwOpiFeh9kAzNHDjBRNrnXZ+luvrpxRqiY7DtZ8P0azxgaFbgjDh+bef3z+7AF/mjkHJFKUg8MkhpW5JB5TUtE4g6G5mynPdbutTaNW3el4yauNvBAA0vlIUFrMzZgY4zfYq7UFntjbFiIg8QHRPGnavPbGyJ/3w6nNVQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543322; c=relaxed/simple; bh=SpIYSVSldCEwiprj2FMflWFJ+YUNjxruLjwCB5+5c+Y=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=G2pL15/7TYtEp51vFld1a8aL8y9dxanFkFBy0QirS8jVgYuWxwLZ93uLjv4xJNfeF3n6DCBAET7nJEfoxzdnR9oeXWOfhblUEiVfz5eOZmaqtsTECnpmP5ZvIyir0kkH+owfVs5RtO0p7fQ2OQlOoUw5IfQDfF1Y9Kb1O3H7hG0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iIdQdAll; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iIdQdAll" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-365fc4636bbso9567926a91.2 for ; Mon, 11 May 2026 16:48:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543319; x=1779148119; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Lrg49k9+y9J8ukhGOYyOcoRGRzDx8kw+P2iL1XvrVyA=; b=iIdQdAllJGT5uRTSnlNfjA0jyVxxEaxv1PVl1YEqetXp9I428MPP1Y+4l//NcU3ptc MgTFqo9N59JxHD0WJFGV7Ro7TWjjE8jpDu2sI6XsT+PSZCTVZIbjfPhzN1zVJU4TWL8k x6miZHXAiDs9lz/ZGAlGe15fzmSxEqMXk1UD4ZJrKkfufilgIJuMSHPw92W4Y8ATLsku 5ZcAi35anfpFcKInOeoCYybxG6b10Kd4M6EyMMFLdk0dWOhXbqDRn452QnYOEb906sYH Stba+i56J7zM/PwHIKttxiPnkyMbdcDcryAe50LlroUanKComuRWamGG+vuuGm1hy6Aw j77Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543319; x=1779148119; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Lrg49k9+y9J8ukhGOYyOcoRGRzDx8kw+P2iL1XvrVyA=; b=gMAM7A0fRZewTzBSmkbmOsxPJcqu1TPoh2IH5KBQoNiDMaLhafUSqvR0GpkoQTsX8D KoxWtGLZ5W9sp7RypPb20idoatBrzB2OZO3gPOAzq9QXeJUh4TXA1k/BO/wkeq4SMBDJ Qz9Gx0LzKBV1JTO3wFEcOPThotZo2LQnTBYhLxWNjFkg8Gq7e/zRcHm3bd1gnAIMxmfi aVwynERFTrG+qykdo2+23BsE/vt6zzS390mT1QpyN5b1Err8iPzLqEG8zZRd791NGzwE kA3eufrzsIEir+4lfPxporr8FuxPY/92qmYD8z3ARZk7b5AFYVTjFoocwxWkAOlCxEZ+ YMwA== X-Forwarded-Encrypted: i=1; AFNElJ8WlXnBTFZD8fTPCdija5qoRFJJ9k+YrqC5FZziJDEfpXs3tYc38dfiJM26eFgxiB9R6HUD5sgzfaxKfHA=@vger.kernel.org X-Gm-Message-State: AOJu0YwitOQUTEtS4GvAnzfZ8WZHxfPZyxmVwzx6Od+nxUPr28d8f9ez 5q/NLZ9HgE2Y3P/QaECDeHg4zLb221B6fQmBszcJyD/SHMsGWK68+oB0lFbkfdFuAiHbplOpBUn Z4JLNSp586g== X-Received: from pjbil14.prod.google.com ([2002:a17:90b:164e:b0:368:bb57:bcb7]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2e87:b0:35e:30bc:96e2 with SMTP id 98e67ed59e1d1-368b24e55acmr760190a91.10.1778543319028; Mon, 11 May 2026 16:48:39 -0700 (PDT) Date: Mon, 11 May 2026 16:47:56 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-11-vipinsh@google.com> Subject: [PATCH v4 10/16] vfio: selftests: Initialize vfio_pci_device using a VFIO cdev FD From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the given VFIO cdev FD to initialize vfio_pci_device in VFIO selftests. Add the assertion to make sure that passed cdev FD is not used with legacy VFIO APIs. If VFIO cdev FD is provided then do not open the device instead use the FD for any interaction with the device. This API will allow to write selftests where VFIO device FD is preserved using liveupdate and retrieved later using liveupdate ioctl after kexec. Co-developed-by: David Matlack Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- .../lib/include/libvfio/vfio_pci_device.h | 3 ++ .../selftests/vfio/lib/vfio_pci_device.c | 33 ++++++++++++++----- 2 files changed, 27 insertions(+), 9 deletions(-) diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_devi= ce.h b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h index 2858885a89bb..896dfde88118 100644 --- a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +++ b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h @@ -38,6 +38,9 @@ struct vfio_pci_device { #define dev_info(_dev, _fmt, ...) printf("%s: " _fmt, (_dev)->bdf, ##__VA_= ARGS__) #define dev_err(_dev, _fmt, ...) fprintf(stderr, "%s: " _fmt, (_dev)->bdf,= ##__VA_ARGS__) =20 +struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, + struct iommu *iommu, + int device_fd); struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu= *iommu); void vfio_pci_device_cleanup(struct vfio_pci_device *device); =20 diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/tes= ting/selftests/vfio/lib/vfio_pci_device.c index fc75e04ef010..7a81edb3245e 100644 --- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c +++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c @@ -342,19 +342,27 @@ static void vfio_device_attach_iommufd_pt(int device_= fd, u32 pt_id) ioctl_assert(device_fd, VFIO_DEVICE_ATTACH_IOMMUFD_PT, &args); } =20 -static void vfio_pci_iommufd_setup(struct vfio_pci_device *device, const c= har *bdf) +static void vfio_pci_iommufd_setup(struct vfio_pci_device *device, + const char *bdf, int device_fd) { - const char *cdev_path =3D vfio_pci_get_cdev_path(bdf); + const char *cdev_path; =20 - device->fd =3D open(cdev_path, O_RDWR); - VFIO_ASSERT_GE(device->fd, 0); - free((void *)cdev_path); + if (device_fd >=3D 0) { + device->fd =3D device_fd; + } else { + cdev_path =3D vfio_pci_get_cdev_path(bdf); + device->fd =3D open(cdev_path, O_RDWR); + VFIO_ASSERT_GE(device->fd, 0); + free((void *)cdev_path); + } =20 vfio_device_bind_iommufd(device->fd, device->iommu->iommufd); vfio_device_attach_iommufd_pt(device->fd, device->iommu->ioas_id); } =20 -struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu= *iommu) +struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, + struct iommu *iommu, + int device_fd) { struct vfio_pci_device *device; =20 @@ -365,10 +373,12 @@ struct vfio_pci_device *vfio_pci_device_init(const ch= ar *bdf, struct iommu *iomm device->iommu =3D iommu; device->bdf =3D bdf; =20 - if (iommu->mode->container_path) + if (iommu->mode->container_path) { + VFIO_ASSERT_EQ(device_fd, -1); vfio_pci_container_setup(device, bdf); - else - vfio_pci_iommufd_setup(device, bdf); + } else { + vfio_pci_iommufd_setup(device, bdf, device_fd); + } =20 vfio_pci_device_setup(device); vfio_pci_driver_probe(device); @@ -376,6 +386,11 @@ struct vfio_pci_device *vfio_pci_device_init(const cha= r *bdf, struct iommu *iomm return device; } =20 +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu= *iommu) +{ + return __vfio_pci_device_init(bdf, iommu, /*device_fd=3D*/-1); +} + void vfio_pci_device_cleanup(struct vfio_pci_device *device) { int i; --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC3243CAE74 for ; Mon, 11 May 2026 23:48:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543322; cv=none; b=R3f/Yemj+Y+/1ebE5DHca2XmYFhc+MR4XRX4+w9ACVi5jF0PnVQJ/ph/vJdYxH2vpN9KXI7kPayUC0IkdOmxWNtaXffCouJ9n1rHMN4Vqa1d9MjQczOgx+yrAvPI8P2+Dlk5ZjL13hnBh126um7y58O5vtOMu4Ll6ALndTfp5lE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543322; c=relaxed/simple; bh=nkhAcR/a501eLQKC6FcCHdjPYtswgNMRUI2YsbdltG8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lF+J0gf9x3mrWfuZrsEWF6+SmQyQW3YUlgt+lkWvCQG3xNP6/Pwu3E3yPGR8S4g2W+tnfVKYaUFiPFeYFqZsSSrpXqeZum+IuBLOrPQ9Gh0j2QQfu2wDD9KOLBtGvVkcT/hCGDOr0pCXBSf5Y52O6Pko5n25uYoJl/NJE24Nvzk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mdK1nWgw; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mdK1nWgw" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2ba268cb5e6so38269095ad.1 for ; Mon, 11 May 2026 16:48:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543320; x=1779148120; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UlR7sP3TmIVuCVXTiNbI5bhyf9pBSqUWeLZMdRUzyiQ=; b=mdK1nWgwsfEfJWdHHftnr/w4TxqPPnnZXTjLi6bEQv1p1xKQMu/adtbyOdHuFFgfvf MEdzxt/EQ9IWRU4Nuw8YS4AfoeGJkduuLa0XJfFUZRIMC8K08y3qtr0ipQVbM1JIGCsd JSw4Sopsd8PyxTZVxAquO8eK/Um2GVGzWPDKLMg1r7cTo4/lC4gPio7HZ2f3qeM6Tf+U EyaOztW3Bs/b2svYVn/XKT/2WZodl0Gj9H8yBaCBucYLYjGte8BcwJugmqn60yngrxY5 8cOr7toZ8O6QJVh9DrfOTD72g0/OAke4CRJZgOvprBF3GRL5o86IFKSBw5ndd4l6Dp5b BidQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543320; x=1779148120; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UlR7sP3TmIVuCVXTiNbI5bhyf9pBSqUWeLZMdRUzyiQ=; b=KKcu0B766ITe13Wkn5NJpfUy68a0piQzsNZgdlSoBnimitgWbb8nZqNyqlE0+RKZ/y eJzexOmawtEXI1hhxCZpy8T2AuKaJB0Eouyi756GV7HJeRsE9hWp179sBbhUj06cLUEv LRjahUsLFOBveX4Q6fEG+JDMk5sM7w3/2ia0VVVfOjz2wFsyUiKEw/vOmOsBHlRfFdPk 4iSc4jgQCAm/QSOZuCbiBgL5r5/kxTk0FKP/IVBIDaA5fTeseYVTkSNErmassmFO1J0Z Vp6LS59rc15Shx4bNT6jLzFazrxaWxe3Fs+ZdzT5J5OAlvDZZMVtOv9edvsAvoijSEhn ArUA== X-Forwarded-Encrypted: i=1; AFNElJ9XsVW3S5Y5M6n7nMcN1ReUWQB9OzRU/ECfGdRpviwSS8e3psfqsR5HOlfEHun7S0fg1hv3FuWbo9tfBok=@vger.kernel.org X-Gm-Message-State: AOJu0YzNBHiTVIWRv+jFjZZiuC+LAwDPyCz9ol7NGbL6qudoUjzcEgDk 964KOZAYK1E6kdmm9RVL/fG/cQT3urT1aR++hR0dZr5ZU0MmYy6sGtr8M6kMjVvno/rwbfFfMDa uylImCwHr4g== X-Received: from pgwb24.prod.google.com ([2002:a65:6698:0:b0:c74:42:899a]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:37c5:b0:2ae:cb0e:fd59 with SMTP id d9443c01a7336-2bd010f3972mr6895335ad.8.1778543320035; Mon, 11 May 2026 16:48:40 -0700 (PDT) Date: Mon, 11 May 2026 16:47:57 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-12-vipinsh@google.com> Subject: [PATCH v4 11/16] vfio: selftests: Add Makefile support for TEST_GEN_PROGS_EXTENDED From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Add Makefile support for TEST_GEN_PROGS_EXTENDED targets. These tests are not run by default. TEST_GEN_PROGS_EXTENDED will be used for Live Update selftests in subsequent commits. These selftests must be run manually because they require the user/runner to perform additional actions, such as kexec, during the test. Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- tools/testing/selftests/vfio/Makefile | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index df3e44660bd0..ad081ec31fb2 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -32,14 +32,17 @@ LDFLAGS +=3D -pthread LIBS_O :=3D $(LIBVFIO_O) LIBS_O +=3D $(LIBLIVEUPDATE_O) =20 -$(TEST_GEN_PROGS): %: %.o $(LIBS_O) +$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): %: %.o $(LIBS_O) $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIBS_O) $(LDLI= BS) -o $@ =20 -TEST_GEN_PROGS_O =3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) -TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(TEST_GEN_PROGS_O)) +TESTS_O :=3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) +TESTS_O +=3D $(patsubst %, %.o, $(TEST_GEN_PROGS_EXTENDED)) + +TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(TESTS_O)) TEST_DEP_FILES +=3D $(patsubst %.o, %.d, $(LIBS_O)) -include $(TEST_DEP_FILES) =20 -EXTRA_CLEAN +=3D $(TEST_GEN_PROGS_O) $(TEST_DEP_FILES) +EXTRA_CLEAN +=3D $(TESTS_O) +EXTRA_CLEAN +=3D $(TEST_DEP_FILES) =20 endif --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57AA83CF688 for ; Mon, 11 May 2026 23:48:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543325; cv=none; b=fKwhEzAddFKAVxPiSMLIP2r4nOk0JNYsy9Rub8tsDVI6B+lkf6+wiQhs7x3hQv2Q307SVaIWh3A1se3dVyHUW3kW7J/9ZAvX3tE/Z2k72OTm/vQPvfOom445XyYp1BVBU1jHQV3T5yJHqaiMuZcgcGhUskNC4zHlVx6+Zieox8s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543325; c=relaxed/simple; bh=dUEtQv4m1bgdZfAKqo/nHhsJYCn9NG0HmQqPLRiSL/Y=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sos0gUSlJ55Af5+LVDFMIi2KVVq//wK4ry2US62mKC40yxhZMIiSda62so81+HU4FKUm5veopakFqVis47ew8/cH7Mczl1W5/1ApmYp7IW9QBL6APBcXEH471od6rRFDC7jCMG2kJniWW3ZjwfiDuydyy8TvEK5dcNq128VWvsE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oDPf/97L; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oDPf/97L" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2ba86e35aa1so74124595ad.1 for ; Mon, 11 May 2026 16:48:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543321; x=1779148121; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=f4gqIrYhtwiwQ5vr+LOFB75VLebAVjQ9EvxHYxdiZqo=; b=oDPf/97LvUOHZ5OcI55Qks487P0bJWmW5FdZ0FzyPzXIentRhbHxZh3GNq0ZUOOsVt HoS7tACCEcn6WiLpGIOEMHPZvaEQKody4Xqnm2+sCQSMvKgjiniQhXGMOrHucf6ZQbsR hN5as28RKi8PjdirojVVF2TFfTTYlUmR7VRUHDd22eaOfEZUZo5ME20sjGjyC1bFUO6j skaJAyTdkcfWRy9a5Li0b7OifNB7pGYSQzdqiECOL9l15/OEsEcTPtYrp3C4adM4SkeD FvomyZZoIMdM1zEtxTEO8m4snrxRVUk4wMRxNIpESneas4l7KvIOVdXONMOteNk5dStz rCIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543321; x=1779148121; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=f4gqIrYhtwiwQ5vr+LOFB75VLebAVjQ9EvxHYxdiZqo=; b=JITaEGwOS2FJ6svh/1EgSbxLbFWHhM+rWo/Uxf1or+lsESzM9p158sGqaelfJfSM0+ bWwgfCm4tRh+Vo9wxZQXTkKoqZ3SXZ0KyiGvq9qAnuTz1Jiz6PSLGyoi9SoQrRmzMV93 hiPY+X4WdDigiECit98oduxAxjShV9oH8jBoso2l15m38VyB8wJ2GZT63tuyM9hk4IPs PZkJSQMu3Q7HbyilBWFbOYnNNmyt6SiRz08Vh/t6DyN6+Wkyah2eHoOH2fIJii4cYw/+ HP+KKKF42kr8qCTq8UtcVbQEDvM2a3dTCv7E0bHWAtcnPwMEsV73E8M5ddTiUly86hGV v47g== X-Forwarded-Encrypted: i=1; AFNElJ/08q/1INT0cnA54XspmyTzxSOMr2gsDUwT14cBMMH7apN4A7MIGXmY2zWxt0edxpdrbq2/yGZt4xAMnco=@vger.kernel.org X-Gm-Message-State: AOJu0YwJ87AaScdOklw5fxZuweqf9hO0T/6+aOE4slJdchE27lfyJI4N uJ62al45ecKpw/yPvKVcrr8oA4MBQJDYFucipvVUiSwFfGVnk/5eqBw0hnZ6yNCoSylhTChP4A5 8MGXGdkZztg== X-Received: from plmk20.prod.google.com ([2002:a17:903:1814:b0:2ae:c50f:f4ec]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ed8b:b0:2b0:7531:b61e with SMTP id d9443c01a7336-2bd0135ec3bmr5062815ad.41.1778543321102; Mon, 11 May 2026 16:48:41 -0700 (PDT) Date: Mon, 11 May 2026 16:47:58 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-13-vipinsh@google.com> Subject: [PATCH v4 12/16] vfio: selftests: Add vfio_pci_liveupdate_kexec_test From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a selftest to exercise preserving a vfio-pci device across a Live Update. For now the test is extremely simple and just verifies that the device file can be preserved and retrieved. In the future this test will be extended to verify more parts about device preservation as they are implemented. This test is added to TEST_GEN_PROGS_EXTENDED since it must be run manually along with a kexec. To run this test manually: $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 1 00= 00:00:04.0 $ kexec ... # NOTE: Exact method will be distro-dependent $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 2 00= 00:00:04.0 The second call to setup.sh is necessary because preserved devices are not bound to a driver after Live Update. Such devices must be manually bound by userspace after Live Update via driver_override. This test is considered passing if all commands exit with 0. Co-developed-by: David Matlack Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- tools/testing/selftests/vfio/Makefile | 4 + .../vfio/vfio_pci_liveupdate_kexec_test.c | 89 +++++++++++++++++++ 2 files changed, 93 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_= test.c diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index ad081ec31fb2..17082a57aa9f 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -14,6 +14,10 @@ TEST_GEN_PROGS +=3D vfio_pci_device_init_perf_test TEST_GEN_PROGS +=3D vfio_pci_driver_test TEST_GEN_PROGS +=3D vfio_pci_liveupdate_uapi_test =20 +# This test must be run manually since it requires the user/automation to +# perform a kexec during the test. +TEST_GEN_PROGS_EXTENDED +=3D vfio_pci_liveupdate_kexec_test + TEST_FILES +=3D scripts/cleanup.sh TEST_FILES +=3D scripts/lib.sh TEST_FILES +=3D scripts/run.sh diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c = b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c new file mode 100644 index 000000000000..15b3e3af91d1 --- /dev/null +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include + +static const char *device_bdf; + +static char state_session[LIVEUPDATE_SESSION_NAME_LENGTH]; +static char device_session[LIVEUPDATE_SESSION_NAME_LENGTH]; + +enum { + STATE_TOKEN, + DEVICE_TOKEN, +}; + +static void before_kexec(int luo_fd) +{ + struct vfio_pci_device *device; + struct iommu *iommu; + int session_fd; + int ret; + + iommu =3D iommu_init("iommufd"); + device =3D vfio_pci_device_init(device_bdf, iommu); + + create_state_file(luo_fd, state_session, STATE_TOKEN, /*next_stage=3D*/2); + + session_fd =3D luo_create_session(luo_fd, device_session); + VFIO_ASSERT_GE(session_fd, 0); + + printf("Preserving device in session\n"); + ret =3D luo_session_preserve_fd(session_fd, device->fd, DEVICE_TOKEN); + VFIO_ASSERT_EQ(ret, 0); + + close(luo_fd); + daemonize_and_wait(); +} + +static void after_kexec(int luo_fd, int state_session_fd) +{ + struct vfio_pci_device *device; + struct iommu *iommu; + int session_fd; + int device_fd; + int stage; + + restore_and_read_stage(state_session_fd, STATE_TOKEN, &stage); + VFIO_ASSERT_EQ(stage, 2); + + session_fd =3D luo_retrieve_session(luo_fd, device_session); + VFIO_ASSERT_GE(session_fd, 0); + + printf("Finishing the session before retrieving the device (should fail)\= n"); + VFIO_ASSERT_NE(luo_session_finish(session_fd), 0); + + printf("Retrieving the device FD from LUO\n"); + device_fd =3D luo_session_retrieve_fd(session_fd, DEVICE_TOKEN); + VFIO_ASSERT_GE(device_fd, 0); + + printf("Finishing the session before binding to iommufd (should fail)\n"); + VFIO_ASSERT_NE(luo_session_finish(session_fd), 0); + + printf("Binding the device to an iommufd and setting it up\n"); + iommu =3D iommu_init("iommufd"); + + /* + * This will invoke various ioctls on device_fd such as + * VFIO_DEVICE_GET_INFO. So this is a decent sanity test + * that LUO actually handed us back a valid VFIO device + * file and not something else. + */ + device =3D __vfio_pci_device_init(device_bdf, iommu, device_fd); + + printf("Finishing the session\n"); + VFIO_ASSERT_EQ(luo_session_finish(session_fd), 0); + + vfio_pci_device_cleanup(device); + iommu_cleanup(iommu); +} + +int main(int argc, char *argv[]) +{ + device_bdf =3D vfio_selftests_get_bdf(&argc, argv); + + sprintf(device_session, "device-%s", device_bdf); + sprintf(state_session, "state-%s", device_bdf); + + return luo_test(argc, argv, state_session, before_kexec, after_kexec); +} --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23EAA36A363 for ; Mon, 11 May 2026 23:48:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543325; cv=none; b=e8ymSk81hRUBi5MyrOqk4ZKToOlagGXdcMQwRMgwdRYr4UJDV8CPJsxPqK5GLrQXTJIYRZsxxf10po7dF+EEWeXUUID9frzYPHM4CkfQrPsjSRAYKnQ/+NVNK08RqPUouR4ElA0zXdI39E9REG7PqrUYouIzQaL4Ionis22tbqc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543325; c=relaxed/simple; bh=9DBA5wHS1wz1qD/m2QL4eJhMj91b/joPGTcCpT2+Eks=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RHtwfr3UZiOIJbNFQTu0Ky17/vZOEfRd0F6DKgM28Itpn7GKy3UPn9kt85EES8awYHjNbIA6Zmbp9ntMFxBHGj9cWIBIeHGYbnDMgrStZ4+YbFnpxG1vkcpTDaVzGP6M+OpVUiW07vMaX5wzsnS5SEWa1EmFnrRzQP2/Wtt4r2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Z3TGi4A+; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Z3TGi4A+" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c828659ecd4so1029919a12.0 for ; Mon, 11 May 2026 16:48:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543322; x=1779148122; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=PEYeyguTQfOyWF9RyPuRLLMirOfxgeWU3fHWlC7AITo=; b=Z3TGi4A+laRAaDSxy3l9o55UVSOAbIbhFxmFJ9lmnpuPGDoSGtFdiJgymbhPdI8MFL VBkSCt7Bu9XAdELJ8wbnJZ34dTIJi391+3g/gmGDgMrVTI+ijFYKcgYtTAqpQSs+7Nl9 tWa+aBdV4cNr74Q0kq0BhVK2o+zh6O9I1uMiQWSVid2uoVms4iycS6MS9ybDDin0HyHB lcyaR/dPWbs9zIBxyFP651OuocVDDuTOCwo9/xkLz3Ths2blfgmNT/YcbxHsANOULY3H GHYHOcLswiPDVPF5qexFRkvbR6a3LnjT+vceL93rTEdz5rqzycmHzRwtAlqBUJUcKjmS 6fbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543322; x=1779148122; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PEYeyguTQfOyWF9RyPuRLLMirOfxgeWU3fHWlC7AITo=; b=jeA3gULMpcEiiUs5SmFzAjjFOR/Vl2tP8VfQ5xGm9myFnTKRgY/PUYhJ+B5q09iu6V dJiXkCqriMJPz/F/vUIpMW7sDbwtHgisYF6I1eBSbvrX2F42Wn+ZBHseHHR/jFjNxrf7 fPZN4QnN6DtcTTqEvmnHMWnfZrBXiM2+g/6OF9staG5U0ljI3uAKGVo0bOZX56QDwCUn 4snZNf75DAjqcgL+dBsKh5xZfK8ZlMYIsNirFY9Xd1+4C75Y1fmSOzUIA35MNxcRdajN jkuV0D0IqQAbf3tg1EuxpYoG0KVTQiC+jpKBUWuSZWT3fx0agPTm7JFTpx6besGH89pq EypA== X-Forwarded-Encrypted: i=1; AFNElJ+iPoDA+UZs89sDggZTLK37prabbFQNh4+B8Gjy2SaiatI45fOfQ3eQ3Y0AVT2VWJ97498WHOtJbTFb0/I=@vger.kernel.org X-Gm-Message-State: AOJu0Ywmt5lvbMVGFXgFj1t2klhCGEdpHyB2sjAsj8Xwkmi8VefIWsNT HDDHq8G2k4unGvqglWVdltJLx8auhrCUHqg5UcSyPvEt2iNpPJZPjBGQLT4zGU+yrd0eFzT2hQJ in5mXDxQXAA== X-Received: from pjtl10.prod.google.com ([2002:a17:90a:c58a:b0:366:22a3:3078]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1cc4:b0:368:b4a5:c4dd with SMTP id 98e67ed59e1d1-368b4a5c615mr642795a91.2.1778543322159; Mon, 11 May 2026 16:48:42 -0700 (PDT) Date: Mon, 11 May 2026 16:47:59 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-14-vipinsh@google.com> Subject: [PATCH v4 13/16] vfio: selftests: Expose iommu_modes to tests From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Expose the list of iommu_modes to enable tests that want to iterate through all possible iommu modes. Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- tools/testing/selftests/vfio/lib/include/libvfio/iommu.h | 2 ++ tools/testing/selftests/vfio/lib/iommu.c | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h b/too= ls/testing/selftests/vfio/lib/include/libvfio/iommu.h index e9a3386a4719..4b9cbe262159 100644 --- a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h +++ b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h @@ -15,6 +15,8 @@ struct iommu_mode { unsigned long iommu_type; }; =20 +extern const struct iommu_mode iommu_modes[]; +extern const int nr_iommu_modes; extern const char *default_iommu_mode; =20 struct dma_region { diff --git a/tools/testing/selftests/vfio/lib/iommu.c b/tools/testing/selft= ests/vfio/lib/iommu.c index 035dac069d60..95a494f829d2 100644 --- a/tools/testing/selftests/vfio/lib/iommu.c +++ b/tools/testing/selftests/vfio/lib/iommu.c @@ -23,7 +23,7 @@ const char *default_iommu_mode =3D MODE_IOMMUFD; =20 /* Reminder: Keep in sync with FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(). */ -static const struct iommu_mode iommu_modes[] =3D { +const struct iommu_mode iommu_modes[] =3D { { .name =3D MODE_VFIO_TYPE1_IOMMU, .container_path =3D "/dev/vfio/vfio", @@ -49,6 +49,8 @@ static const struct iommu_mode iommu_modes[] =3D { }, }; =20 +const int nr_iommu_modes =3D ARRAY_SIZE(iommu_modes); + static const struct iommu_mode *lookup_iommu_mode(const char *iommu_mode) { int i; --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80FFF3D2FFB for ; Mon, 11 May 2026 23:48:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543327; cv=none; b=iGrk5jH8ZTsMngU2DwTYJNW5TSBCP7uMWX4pzPNyDMJP6nJ14jTm8HKLfz1SHhrsk/bukM9kuHxzUsrj249G8rBj5vAG3IQw2NbikWePn/sx8g6l+vuy+OVy4lgMhvBIcmKmIJPPNLHEZSQqGF/hfoxRetinNofOGx9b39g5dXU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543327; c=relaxed/simple; bh=K2g0rDAceRP3rk+ss2Z6d44PvotU5vQgbAUUB2teEOg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=A14apJcCZRwZOLHnRkJGcxN2yjNMZiCu412Kh2eyhSJumbvIdUzVpBm/rr3TmrbqaLbufvLxsaLiv7pi4Q7zPhmYnyUp1jw5kUL+eXeZ0kq5ZVVh70AT6fpx/785ycWlNByoPxmmBwIVW5wLCvd6DZHbbDTjR8X9tXAGx7xJdjY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XdsUW6oX; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XdsUW6oX" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-82fa6c3a77cso3036442b3a.3 for ; Mon, 11 May 2026 16:48:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543323; x=1779148123; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=W808toPe+T7JqWu8fRVTH+oiRJhlVase5UG9iYYX56M=; b=XdsUW6oX+58WIemR0Y125/oqVAMxLJVGHEg81FigHMbI66BrxwWYRLSXBCJVCDUnPZ C3WOAepRuGw380XuwtpBVvkRJfPK1acRTWFysRodz36TaZV3R3cYaYz8USz75j+XyXax SJEcEi8siN2coJE0+CXce8BU3ponaO0YFBsTZpdmUroYLRjkt2czY81m5SwxgjXBzJDn nx5Dzv+aq9ZtZw6rYvfzjO6LR2oEmX1xCiU3gQXansMDu3AVYW8V1dmSMIN4xMV73d3v te5GwT+lAclYF3sbnweQnAWCLacSVJbT1bEf8u5kylqtxBB3AUHrXoes9p63stpR2g91 MDvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543323; x=1779148123; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=W808toPe+T7JqWu8fRVTH+oiRJhlVase5UG9iYYX56M=; b=IeA0FM+851rHKiwwXqgWWr2gYrdyXWMaTE2nT5mFdp9sjH7mLLzL9bFMWGkQiA46h1 Ma8E/FKbNDNxcmOCJ+vg4ZGH54UttJ95hnf49CZ5swf96b7Vc8WrXhXfCQO/xq+Thzkb WciPfnOpO3iEqh+lcXhqkl1ERe8zl/azEx1TSWNHtuIGn11kJMp9oCCP7yw+e/U61FF2 YsWSlVMc73bmieuF59xYCjogkhb0o0UDJxW/O30Q5EP/ramkYx9rnoMKWuLPT4/huJGB 4JcYY8qeCbY8zgOC359ZxWhxZSPpDCvHw2Ua8KQ4+IsgFF26b3p6ijxwRvObicXy4Z5w zGbA== X-Forwarded-Encrypted: i=1; AFNElJ9S6C+y2cSnEhKFV8bDAVpceE+hoin7diZem8UZyOWmGK3hiAZjzXhf7XTcCrZEfxaKQHfWVwuuSk2LNsc=@vger.kernel.org X-Gm-Message-State: AOJu0Yx3nKZ2FCsj/eAJweP6yvvRdi6oHSjrclbq1LZdYLYtefE+ceLt 9b5ufjbWZIGK4l/Znvh63R2akBNU+2QT3+z2hCobkRCzC+cv7HgaQLDzL5LCTOr8K9fCtXMXVR3 lVn2uEnJeBA== X-Received: from pfoc11.prod.google.com ([2002:aa7:880b:0:b0:838:5125:b5e4]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:a203:b0:835:4447:69d8 with SMTP id d2e1a72fcca58-83cf6a3a65emr15561909b3a.30.1778543323193; Mon, 11 May 2026 16:48:43 -0700 (PDT) Date: Mon, 11 May 2026 16:48:00 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-15-vipinsh@google.com> Subject: [PATCH v4 14/16] vfio: selftests: Expose low-level helper routines for setting up struct vfio_pci_device From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Expose a few low-level helper routings for setting up vfio_pci_device structs. These routines will be used in a subsequent commit to assert that VFIO_GROUP_GET_DEVICE_FD fails under certain conditions. Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- .../lib/include/libvfio/vfio_pci_device.h | 5 +++ .../selftests/vfio/lib/vfio_pci_device.c | 33 +++++++++++++------ 2 files changed, 28 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_devi= ce.h b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h index 896dfde88118..2389c7698335 100644 --- a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +++ b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h @@ -125,4 +125,9 @@ static inline bool vfio_pci_device_match(struct vfio_pc= i_device *device, =20 const char *vfio_pci_get_cdev_path(const char *bdf); =20 +/* Low-level routines for setting up a struct vfio_pci_device */ +struct vfio_pci_device *vfio_pci_device_alloc(const char *bdf, struct iomm= u *iommu); +void vfio_pci_group_setup(struct vfio_pci_device *device); +void vfio_pci_iommu_setup(struct vfio_pci_device *device); + #endif /* SELFTESTS_VFIO_LIB_INCLUDE_LIBVFIO_VFIO_PCI_DEVICE_H */ diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/tes= ting/selftests/vfio/lib/vfio_pci_device.c index 7a81edb3245e..3152196261a2 100644 --- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c +++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c @@ -222,7 +222,7 @@ static unsigned int vfio_pci_get_group_from_dev(const c= har *bdf) return group; } =20 -static void vfio_pci_group_setup(struct vfio_pci_device *device, const cha= r *bdf) +void vfio_pci_group_setup(struct vfio_pci_device *device) { struct vfio_group_status group_status =3D { .argsz =3D sizeof(group_status), @@ -230,7 +230,7 @@ static void vfio_pci_group_setup(struct vfio_pci_device= *device, const char *bdf char group_path[32]; int group; =20 - group =3D vfio_pci_get_group_from_dev(bdf); + group =3D vfio_pci_get_group_from_dev(device->bdf); snprintf(group_path, sizeof(group_path), "/dev/vfio/%d", group); =20 device->group_fd =3D open(group_path, O_RDWR); @@ -242,14 +242,12 @@ static void vfio_pci_group_setup(struct vfio_pci_devi= ce *device, const char *bdf ioctl_assert(device->group_fd, VFIO_GROUP_SET_CONTAINER, &device->iommu->= container_fd); } =20 -static void vfio_pci_container_setup(struct vfio_pci_device *device, const= char *bdf) +void vfio_pci_iommu_setup(struct vfio_pci_device *device) { struct iommu *iommu =3D device->iommu; unsigned long iommu_type =3D iommu->mode->iommu_type; int ret; =20 - vfio_pci_group_setup(device, bdf); - ret =3D ioctl(iommu->container_fd, VFIO_CHECK_EXTENSION, iommu_type); VFIO_ASSERT_GT(ret, 0, "VFIO IOMMU type %lu not supported\n", iommu_type); =20 @@ -259,8 +257,14 @@ static void vfio_pci_container_setup(struct vfio_pci_d= evice *device, const char * because the IOMMU type is already set. */ (void)ioctl(iommu->container_fd, VFIO_SET_IOMMU, (void *)iommu_type); +} =20 - device->fd =3D ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, bdf); +static void vfio_pci_container_setup(struct vfio_pci_device *device) +{ + vfio_pci_group_setup(device); + vfio_pci_iommu_setup(device); + + device->fd =3D ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, device->= bdf); VFIO_ASSERT_GE(device->fd, 0); } =20 @@ -360,9 +364,7 @@ static void vfio_pci_iommufd_setup(struct vfio_pci_devi= ce *device, vfio_device_attach_iommufd_pt(device->fd, device->iommu->ioas_id); } =20 -struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, - struct iommu *iommu, - int device_fd) +struct vfio_pci_device *vfio_pci_device_alloc(const char *bdf, struct iomm= u *iommu) { struct vfio_pci_device *device; =20 @@ -373,9 +375,20 @@ struct vfio_pci_device *__vfio_pci_device_init(const c= har *bdf, device->iommu =3D iommu; device->bdf =3D bdf; =20 + return device; +} + +struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, + struct iommu *iommu, + int device_fd) +{ + struct vfio_pci_device *device; + + device =3D vfio_pci_device_alloc(bdf, iommu); + if (iommu->mode->container_path) { VFIO_ASSERT_EQ(device_fd, -1); - vfio_pci_container_setup(device, bdf); + vfio_pci_container_setup(device); } else { vfio_pci_iommufd_setup(device, bdf, device_fd); } --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 437E93D34A6 for ; Mon, 11 May 2026 23:48:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543327; cv=none; b=fH6AAcbL4R8msGc8DDB0eWN7Kc3eexAGvZGBEHTOXjABKe3XVyofiCrxZS1ij176CcpD5qUG8hZ9T3PS7+5SlrwZK5osFiLllCELjQ9EFtptshuTV8q8BjpBzb5PqlCVA+OyVzon1xxnqARhbErsFxM1S9aLh4OwVUBfzpzqflU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543327; c=relaxed/simple; bh=RghH9FcYPgVqkLV+z4skVa4m4AD6epMoNgDgpNvGU6w=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jRkV671rJEmoUacPUd8lfJ2GAJ7KVG3cbcnM70QwHp5JVW9J7ANe0F/VyOwggr283ypclIXmod970BvXThUmCLX5w9j9wKKdWvui0BK5QnV7W246InK8Iv3THd19ZCXiCEM4lV8axUDAyFkicUU23pw7htJfHp9TtnQ1h2vyXSg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YInU6vj/; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YInU6vj/" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-c827adbf00cso1127396a12.2 for ; Mon, 11 May 2026 16:48:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543324; x=1779148124; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=VIWZ9kq9YD08PbxmWBTSSy3QBQyaG9ngqmDEoB6yjoA=; b=YInU6vj/Isv6MOjB6KT5+4SMSgr8gkH01ejqoq8UNGugd8yYwZzAq2sSrurTN2qRHH ZwSX+PKYz+PRRiCtAo7zfsUcFeSVVTZJpKrxeAwHgwJSDXC8vxfSh2XCvDzjCduizsmG bUSsnjPZ8D6MgDXHvIevdZuxbBYadfGOCOAChAl7a3EDRg+ilajf9kWEy/TGNMkI/yk9 T2gpm0OcQ2uE99pClBzcVwg+9/2JuPUWoi8oDW8GIxgoVPd9kB0IiqRt0XdQGnLNB8Ku ejtqbqz0205A2KABNKofJd6Tp4wgvewJWU95+EwEzftRZ+rznMcQwI9WmDWY6QHKZmpT QeAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543324; x=1779148124; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VIWZ9kq9YD08PbxmWBTSSy3QBQyaG9ngqmDEoB6yjoA=; b=Bt5CFsct1lcLtiItnxa9fBNAxfIw+Pw2zrVuLTXNGcKyy94MRrANvoCY2gl6TZzZvC zCIo9znnfJ2Z0NAwa7g7eRPBy4Ck3qWa5nywoYtJvHBJWdqYXrk2d6NkyxJdsdoCikPk F3ZBzwAAsG/wg2AwORpSJ1xmrC/Qi4pQ5l4lBzEOTV3fNy0PwpSQ4WRPia5NchRTaGbV +b5U1ON/2X8HXLuhQFb78yy1mq/jPfQ3GwtcxXWiAUFUnQDQuZjYt06LTEoUh7EEUv/I 3Zh1l9k7jiOwoFIMGjWCat2LvKZ8AnafO85uNzOoUGisuu3eYSvRo86nYMHYnzDVD4kE +XJA== X-Forwarded-Encrypted: i=1; AFNElJ/lOWkS6VV7IzppyRR3MBk5bEDTNKhRsHk+cej+n9bZZIWKN9UtsSRTG+akyRUxrRX/D/xkzyCabqYvTIY=@vger.kernel.org X-Gm-Message-State: AOJu0YyOHG4iN+mJYYTvVXYegdGw+cX5v23LPIzuwG8ykyJZ8RIeJpOS d7MK647+h+dnxi4gIhzQRZDcS0kvzc6+iMYoTZ7BYIhQJPgGpuxrPKciS7ucsz6Vc1oSEs0znPK +JwpYzUZWpA== X-Received: from pgap4.prod.google.com ([2002:a63:4204:0:b0:c6e:795e:f99a]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:9149:b0:39b:87f0:758e with SMTP id adf61e73a8af0-3aad42566ccmr12931272637.8.1778543324306; Mon, 11 May 2026 16:48:44 -0700 (PDT) Date: Mon, 11 May 2026 16:48:01 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-16-vipinsh@google.com> Subject: [PATCH v4 15/16] vfio: selftests: Verify that opening VFIO device fails during Live Update From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Verify that opening a VFIO device through its cdev file and via VFIO_GROUP_GET_DEVICE_FD both fail with -EBUSY if the device was preserved across a Live Update. When a device file is preserve across a Live Update, the file must be retrieved from /dev/liveupdate, not from VFIO directly. Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- .../vfio/vfio_pci_liveupdate_kexec_test.c | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c = b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c index 15b3e3af91d1..65c48196e44e 100644 --- a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c @@ -36,6 +36,42 @@ static void before_kexec(int luo_fd) daemonize_and_wait(); } =20 +static void check_open_vfio_device_fails(void) +{ + const char *cdev_path =3D vfio_pci_get_cdev_path(device_bdf); + struct vfio_pci_device *device; + struct iommu *iommu; + int ret, i; + + printf("Checking open(%s) fails\n", cdev_path); + ret =3D open(cdev_path, O_RDWR); + VFIO_ASSERT_EQ(ret, -1); + VFIO_ASSERT_EQ(errno, EBUSY); + free((void *)cdev_path); + + for (i =3D 0; i < nr_iommu_modes; i++) { + if (!iommu_modes[i].container_path) + continue; + + iommu =3D iommu_init(iommu_modes[i].name); + + device =3D vfio_pci_device_alloc(device_bdf, iommu); + vfio_pci_group_setup(device); + vfio_pci_iommu_setup(device); + + printf("Checking ioctl(group_fd, VFIO_GROUP_GET_DEVICE_FD, \"%s\") fails= (%s)\n", + device_bdf, iommu_modes[i].name); + + ret =3D ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, device->bdf); + VFIO_ASSERT_EQ(ret, -1); + VFIO_ASSERT_EQ(errno, EBUSY); + + close(device->group_fd); + free(device); + iommu_cleanup(iommu); + } +} + static void after_kexec(int luo_fd, int state_session_fd) { struct vfio_pci_device *device; @@ -44,6 +80,8 @@ static void after_kexec(int luo_fd, int state_session_fd) int device_fd; int stage; =20 + check_open_vfio_device_fails(); + restore_and_read_stage(state_session_fd, STATE_TOKEN, &stage); VFIO_ASSERT_EQ(stage, 2); =20 --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 23:48:35 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AB953D413F for ; Mon, 11 May 2026 23:48:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543329; cv=none; b=ofb+JSOx0ZehT72bhxvzDQ3kfUSpBmchUXTcXrjgsSZ1Tswj4XF5X/tJjtEkMgwFKZYBLjMVSRuw7jbmE65jTC40OTz7L6//9gbwHnVP62E1ae4hSyzdGrflDaFFe/tfk7Oz/0trIKXWAfqU8rVBpy4iAb56xJ/aTMoNltyPn1U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778543329; c=relaxed/simple; bh=mKQBR0rz/05lEuaw8S69iIUy1w9TkuyP0iz8peUQ0d4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=EvaYLXeF++8IHqW0k16+nsdm5+Rwi1Ge8bpZDbeGjfNOPvkcu9GRD+hEI8JROokCV/Z8XYswTro/5AoGzpj7sftKhW4FLbGgrQPxGP1/Dv0X+GPiPZ8Q6sqFbQ3zHiN+9IFfpe+osnzG88Dd6GEHLZxsUdqbPXAdnjrZTOj/UM0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eMvDrSGH; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vipinsh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eMvDrSGH" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c709551ec08so5975681a12.3 for ; Mon, 11 May 2026 16:48:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778543325; x=1779148125; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lJTGo5XDHXx2rcanjBQwin8Y3adz33qUp8/b4RzzHOA=; b=eMvDrSGH+5hP0K9N4d8LE6BlkhwFbeq3BYmFmOh7anwc7v+u8ytWNS7slzwgpZE/9k zLNm6S/pBbLF/RbAo4MuSNyuOdMOL7gFr2g11DVyl/z3sfV7ayq1KuvMAWoCT+SmK9zd OJ29Qv5seI4Ick43AuEuKTgt4DLo1CY4P20Ujr5CnRKVD+7sBIr+4W4jStI8IvifYs7g 7G+JRjw4MPZm5gV/BvgEAzaXsUJpP2PF+vbOc7HJCRTubcEWjuGYDmPuaVLeQnCRe4Jo +jFqFoIwd3wJ6wA0acjjGyG2iuwLk/Hr/2mkt2MnBkRRDeb6p6mdRdYsLv4Tyap/tQu1 vGaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778543325; x=1779148125; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lJTGo5XDHXx2rcanjBQwin8Y3adz33qUp8/b4RzzHOA=; b=FjXaRahWQAcOBOxzhxlw7HBPq6xkrY5CbxzGmXAyCOmgyK4oZFQzv8ybV/eFLkCdbV kHCkWQOfNHxkduJ0AwzZZypAi6sTEvPo5HdLgswQmtIcbe1wcxMgBbTn1+EWQpTQmSci bRIUVt7oEkIuuhMScM851ZAS0mGXtguiWS8wQGIw9vx3bYtdt41rWFcVaVP1WoAxa0tl HNhSO/SOQDLds1fyCjeoZDlmtFAqScu9mMu3bAe8F5yVSt8UyOnHpJHYdKIBXsmxYM6Z HR6krzV3oRqJ3ItDCdI985va3FDIFc3OBijMiFG2HBmyioT2Slq0MzufXhoVjczF3Wsb wEVA== X-Forwarded-Encrypted: i=1; AFNElJ9aq+ke1AsNlfKOJdOUA9bOqLQBXe6nN9hl1l9xZlx12VFJbUO7/tJubE3BYQ0MDsq7wvTtMhzvmpDEkMQ=@vger.kernel.org X-Gm-Message-State: AOJu0YwySMiiASx9pYb+rHSEDibRQ0nNjOokxpglVPmjuIjmEOpHNfNr J8rcJJk6MHzf2hl1UnDnlr/opCMy7nK9pC5RBQhLOLQ9KIGsyY/wGPlUyISzjzTkGGKsviLhDFi EP3TuFs55Rw== X-Received: from pfoo17.prod.google.com ([2002:a05:6a00:1a11:b0:82f:8e5d:a777]) (user=vipinsh job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:8014:b0:82c:dfea:9e2a with SMTP id d2e1a72fcca58-83a5b6d0adamr27939299b3a.2.1778543325352; Mon, 11 May 2026 16:48:45 -0700 (PDT) Date: Mon, 11 May 2026 16:48:02 -0700 In-Reply-To: <20260511234802.2280368-1-vipinsh@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260511234802.2280368-1-vipinsh@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260511234802.2280368-17-vipinsh@google.com> Subject: [PATCH v4 16/16] vfio: selftests: Add continuous DMA to vfio_pci_liveupdate_kexec_test From: Vipin Sharma To: kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org Cc: ajayachandra@nvidia.com, alex@shazbot.org, amastro@fb.com, ankita@nvidia.com, apopple@nvidia.com, chrisl@kernel.org, corbet@lwn.net, dmatlack@google.com, graf@amazon.com, jacob.pan@linux.microsoft.com, jgg@nvidia.com, jgg@ziepe.ca, jrhilke@google.com, julianr@linux.ibm.com, kevin.tian@intel.com, leon@kernel.org, leonro@nvidia.com, lukas@wunner.de, michal.winiarski@intel.com, parav@nvidia.com, pasha.tatashin@soleen.com, praan@google.com, pratyush@kernel.org, rananta@google.com, rientjes@google.com, rodrigo.vivi@intel.com, rppt@kernel.org, saeedm@nvidia.com, skhan@linuxfoundation.org, skhawaja@google.com, vipinsh@google.com, vivek.kasireddy@intel.com, witu@nvidia.com, yanjun.zhu@linux.dev, yi.l.liu@intel.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Matlack Add a long-running DMA memcpy operation to vfio_pci_liveupdate_kexec_test so that the device attempts to perform DMAs continuously during the Live Update. At this point iommufd preservation is not supported and bus mastering is not kept enabled on the device during across the kexec, so most of these DMAs will be dropped. However this test ensures that the current device preservation support does not lead to system instability or crashes if the device is active. And once iommufd and bus mastering are preserved, this test can be relaxed to check that the DMA operations completed successfully. Signed-off-by: David Matlack Signed-off-by: Vipin Sharma --- .../vfio/vfio_pci_liveupdate_kexec_test.c | 129 ++++++++++++++++++ 1 file changed, 129 insertions(+) diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c = b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c index 65c48196e44e..36bddfbb88ed 100644 --- a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c @@ -1,8 +1,16 @@ // SPDX-License-Identifier: GPL-2.0-only =20 +#include +#include + #include #include =20 +#define MEMCPY_SIZE SZ_1G +#define DRIVER_SIZE SZ_1M +#define MEMFD_SIZE (MEMCPY_SIZE + DRIVER_SIZE) + +static struct dma_region memcpy_region; static const char *device_bdf; =20 static char state_session[LIVEUPDATE_SESSION_NAME_LENGTH]; @@ -11,8 +19,89 @@ static char device_session[LIVEUPDATE_SESSION_NAME_LENGT= H]; enum { STATE_TOKEN, DEVICE_TOKEN, + MEMFD_TOKEN, }; =20 +static void dma_memcpy_one(struct vfio_pci_device *device) +{ + void *src =3D memcpy_region.vaddr, *dst; + u64 size; + + size =3D min_t(u64, memcpy_region.size / 2, device->driver.max_memcpy_siz= e); + dst =3D src + size; + + memset(src, 1, size); + memset(dst, 0, size); + + printf("Kicking off 1 DMA memcpy operations of size 0x%lx...\n", size); + vfio_pci_driver_memcpy(device, + to_iova(device, src), + to_iova(device, dst), + size); + + VFIO_ASSERT_EQ(memcmp(src, dst, size), 0); +} + +static void dma_memcpy_start(struct vfio_pci_device *device) +{ + void *src =3D memcpy_region.vaddr, *dst; + u64 count, size; + + size =3D min_t(u64, memcpy_region.size / 2, device->driver.max_memcpy_siz= e); + dst =3D src + size; + + /* + * Rough Math: If we assume the device will perform memcpy at a rate of + * 30GB/s then 7200GB of transfers will run for about 4 minutes. + */ + count =3D (u64)7200 * SZ_1G / size; + count =3D min_t(u64, count, device->driver.max_memcpy_count); + + memset(src, 1, size / 2); + memset(dst, 0, size / 2); + + printf("Kicking off %lu DMA memcpy operations of size 0x%lx...\n", count,= size); + vfio_pci_driver_memcpy_start(device, + to_iova(device, src), + to_iova(device, dst), + size, count); +} + +static void dma_memfd_map(struct vfio_pci_device *device, int fd) +{ + void *vaddr; + + vaddr =3D mmap(NULL, MEMFD_SIZE, PROT_WRITE, MAP_SHARED, fd, 0); + VFIO_ASSERT_NE(vaddr, MAP_FAILED); + + memcpy_region.iova =3D SZ_4G; + memcpy_region.size =3D MEMCPY_SIZE; + memcpy_region.vaddr =3D vaddr; + iommu_map(device->iommu, &memcpy_region); + + device->driver.region.iova =3D memcpy_region.iova + memcpy_region.size; + device->driver.region.size =3D DRIVER_SIZE; + device->driver.region.vaddr =3D vaddr + memcpy_region.size; + iommu_map(device->iommu, &device->driver.region); +} + +static void dma_memfd_setup(struct vfio_pci_device *device, int session_fd) +{ + int fd, ret; + + fd =3D memfd_create("dma-buffer", 0); + VFIO_ASSERT_GE(fd, 0); + + ret =3D fallocate(fd, 0, 0, MEMFD_SIZE); + VFIO_ASSERT_EQ(ret, 0); + + printf("Preserving memfd of size 0x%x in session\n", MEMFD_SIZE); + ret =3D luo_session_preserve_fd(session_fd, fd, MEMFD_TOKEN); + VFIO_ASSERT_EQ(ret, 0); + + dma_memfd_map(device, fd); +} + static void before_kexec(int luo_fd) { struct vfio_pci_device *device; @@ -32,6 +121,27 @@ static void before_kexec(int luo_fd) ret =3D luo_session_preserve_fd(session_fd, device->fd, DEVICE_TOKEN); VFIO_ASSERT_EQ(ret, 0); =20 + dma_memfd_setup(device, session_fd); + + /* + * If the device has a selftests driver, kick off a long-running DMA + * operation to exercise the device trying to DMA during a Live Update. + * Since iommufd preservation is not supported yet, these DMAs should be + * dropped. So this is just looking to verify that the system does not + * fall over and crash as a result of a busy device being preserved. + */ + if (device->driver.ops) { + vfio_pci_driver_init(device); + dma_memcpy_start(device); + + /* + * Disable interrupts on the device or freeze() will fail. + * Unfortunately there isn't a way to easily have a test for + * that here since the check happens during shutdown. + */ + vfio_pci_msix_disable(device); + } + close(luo_fd); daemonize_and_wait(); } @@ -78,6 +188,7 @@ static void after_kexec(int luo_fd, int state_session_fd) struct iommu *iommu; int session_fd; int device_fd; + int memfd; int stage; =20 check_open_vfio_device_fails(); @@ -88,6 +199,10 @@ static void after_kexec(int luo_fd, int state_session_f= d) session_fd =3D luo_retrieve_session(luo_fd, device_session); VFIO_ASSERT_GE(session_fd, 0); =20 + printf("Retrieving memfd from LUO\n"); + memfd =3D luo_session_retrieve_fd(session_fd, MEMFD_TOKEN); + VFIO_ASSERT_GE(memfd, 0); + printf("Finishing the session before retrieving the device (should fail)\= n"); VFIO_ASSERT_NE(luo_session_finish(session_fd), 0); =20 @@ -109,9 +224,23 @@ static void after_kexec(int luo_fd, int state_session_= fd) */ device =3D __vfio_pci_device_init(device_bdf, iommu, device_fd); =20 + dma_memfd_map(device, memfd); + printf("Finishing the session\n"); VFIO_ASSERT_EQ(luo_session_finish(session_fd), 0); =20 + /* + * Once iommufd preservation is supported and the device is kept fully + * running across the Live Update, this should wait for the long- + * running DMA memcpy operation kicked off in before_kexec() to + * complete. But for now we expect the device to be reset so just + * trigger a single memcpy to make sure it's still functional. + */ + if (device->driver.ops) { + vfio_pci_driver_init(device); + dma_memcpy_one(device); + } + vfio_pci_device_cleanup(device); iommu_cleanup(iommu); } --=20 2.54.0.563.g4f69b47b94-goog