From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 39A683148A6 for ; Wed, 26 Nov 2025 19:36:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185774; cv=none; b=Sr4i6sXw4tHxhlSlx4up8bzuLyv8qFU1OyF84OLAduCbutpwpPa2bQp0egml1xwli5Lh61841iZ15EewoyeS3Runb+JPA65oXfaPvcynQxc3SLszFy5riQxD21kOkmx5o06NPkOjm15qN8/0R0sRxOaj8Z78yP7TCE+Qh8d/6C4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185774; c=relaxed/simple; bh=/FXmHIzCD+k6Qaqzmfau8LRE8Vrb+9907NJenDBaRrw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MQojE9oFUSHXUqzBUmePm7k2c3mJ5a+IlaK9Rp/qE+PrklsWpjcrHp1FWBju24eF609Kw41qKPQMN2A+btXBOBTXjM27aX2hXnI8Ox0NFeM8rNJc2K7R1rjP6d+kuvIUo0o0qHNMTAu2ShVjOHURKLepFFl21H4QDhUpmZxAmEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2AyXwUI8; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2AyXwUI8" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7b89c1ce9cfso50483b3a.2 for ; Wed, 26 Nov 2025 11:36:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185772; x=1764790572; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uHriGM21Vxl9AOHAXOvBqeFMQcHzxMHlkr+LZIh84mE=; b=2AyXwUI8LHR5+vkf1wx8lh5FgwstwLcl+Q+eNcEIpRMuMqZGexLMIbWsdyC9hB0pUs zT0RbQRxVGqyV6fD15rInwB7XO4aYHVLtI7PkTxYlzEkMt8Ucr6bn98hAyYTvalCf9zj dNjQqzQe5FJzvgcplPHgsg27vO75UxHdNoVj3pDT0EgP0WaCq78htQHKGoi31krRRsqJ KyP6RfTip0i6X0kOcRKccOJgaG4fCwVhmNsakemqfiHPbjhwjDkBRZN+SqiYc/6F/8Hu HJW8lu+hd18exJvydsHeqhimsjI9IVG4zQoICUywd9ClyZ96qI1/suO3jWqkVDSwnG4K CJWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185772; x=1764790572; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uHriGM21Vxl9AOHAXOvBqeFMQcHzxMHlkr+LZIh84mE=; b=pSsoJTlZsicBbBZZvWaut7w+GLNknvs40i1VB5l5+jcyceofVSKxPI6Iy62R8kgGcq ROYF8/SxHU4jXCB3LuJM01TDGCzG0GSXd8oCn96BZrzjS/RUWQDS2uUbCRHzCI26bpjI kqAxdUVFBK6kqKmJWOOgnd00d73D9XZVWIiPJKgcLdACHYs1GdsvONUHsebNU7kzH934 YZrnfTj+iB6g+9gWJn7T1MKPi9KD+xzrlXdMENdihEMq+vy+m6ql4UyfGUH25xSA0U0Q lItIcJwbY4Y4mbwR3UMmuMa/el2jVr7dANUoJvRGs+F0jD6dokPnklk3V3GdvP1R4Jkq 0RcQ== X-Forwarded-Encrypted: i=1; AJvYcCWPbUlA0oRhgN8Jtb0YmeggDue6zNRZ4SUiU8fAG4dvCOxRN+a0TkyNNFfOsXnWVD3wFxaO2P4WUc3dXKA=@vger.kernel.org X-Gm-Message-State: AOJu0YyElAMBE/kIDXtaGvdzfmyR3LNoU27DqOSMNxFWMkAZFne7NtBj I1IHogAaOHxnCvn+HdsSH9ms7O9i7MkfvFM733M+DXb9yfM0C/9UYXBhW8BOnMHvIXbxGwHqqRT mSt2sJXvSF7hqZw== X-Google-Smtp-Source: AGHT+IGSiH44ozkyo+4RkUn4tpozCQb7RKk52SJWagBr1rO1D7Afqr3pd7mKB/7v6Lj8EhB8iNn4BVDHpADpeQ== X-Received: from pfbmu13.prod.google.com ([2002:a05:6a00:6e8d:b0:7bc:34a1:3751]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:bd92:b0:7b2:2d85:ae59 with SMTP id d2e1a72fcca58-7ca879ebbc9mr7460707b3a.11.1764185772505; Wed, 26 Nov 2025 11:36:12 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:48 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-2-dmatlack@google.com> Subject: [PATCH 01/21] liveupdate: luo_flb: Prevent retrieve() after finish() From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Prevent an incoming FLB from being retrieved after its finish() callback has been run. An incoming FLB should only retrieved once, and once its finish() callback has run the data has been freed and cannot be (and should not be) retrieved again. Fixes: e8c57e582167 ("liveupdate: luo_flb: Introduce File-Lifecycle-Bound g= lobal state") Signed-off-by: David Matlack --- include/linux/liveupdate.h | 3 +++ kernel/liveupdate/luo_flb.c | 4 ++++ 2 files changed, 7 insertions(+) diff --git a/include/linux/liveupdate.h b/include/linux/liveupdate.h index ed81e7b31a9f..b913d63eab5f 100644 --- a/include/linux/liveupdate.h +++ b/include/linux/liveupdate.h @@ -165,12 +165,15 @@ struct liveupdate_flb_ops { * @obj: The live kernel object returned by .preserve() or .retrieve(). * @lock: A mutex that protects all fields within this structure, providi= ng * the synchronization service for the FLB's ops. + * @finished: True once the FLB's finish() callback has run, to prevent an= FLB + * from being retrieve()'d again after finish() (incoming only). */ struct luo_flb_private_state { long count; u64 data; void *obj; struct mutex lock; + bool finished; }; =20 /* diff --git a/kernel/liveupdate/luo_flb.c b/kernel/liveupdate/luo_flb.c index e80ac5b575ec..072796fb75cb 100644 --- a/kernel/liveupdate/luo_flb.c +++ b/kernel/liveupdate/luo_flb.c @@ -155,6 +155,9 @@ static int luo_flb_retrieve_one(struct liveupdate_flb *= flb) =20 guard(mutex)(&private->incoming.lock); =20 + if (private->incoming.finished) + return -ENODATA; + if (private->incoming.obj) return 0; =20 @@ -213,6 +216,7 @@ static void luo_flb_file_finish_one(struct liveupdate_f= lb *flb) =20 private->incoming.data =3D 0; private->incoming.obj =3D NULL; + private->incoming.finished =3D true; } } } --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB28E340277 for ; Wed, 26 Nov 2025 19:36:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185777; cv=none; b=IAGffwsbR6RDJUVGbwi1o4LJz7vtjo7n81oTsntBslUgb7247z7vJB0LfriVuYIdBpKatmqKc8q9yhwx1BBV+a3CTn3Eu5MqQuB9NkxItFmQL2IrO+0Wchgh6Sax9yKhtswvSBXbBt8bK+xPaC6fpO2mge9I0L7rFi3aQjoeYN8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185777; c=relaxed/simple; bh=EmkP5+D+rlUjnYw1IRRpUkFiaMI5Frjr+rChuQWESLw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LeJpKwFCVGZkd4yMpdN85N4C4IasDBBkqWJkU63T3PuK5CPGx/6C7fTaMjduh6fEIGz5RzRetJHxok5HuMVfNHtoHgu+cflNUrcmw6yXskrugaNtEHS4vv0Gj+9LLyIT5xh/pTuC01HMs9VbID7kLbkwOG4zZIXPlYUVqcDSFbk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lAi+4Ix+; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lAi+4Ix+" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7be3d08f863so101458b3a.2 for ; Wed, 26 Nov 2025 11:36:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185774; x=1764790574; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KGzKyE17LTIjiDUTLc5iTgG4+2QVa3flPC17N65tMUo=; b=lAi+4Ix+i0j3r5ZqNhhkRCSsBdCQEMwWx0MgSC43hZbzTprTUERh2Wqbrtdv4CYSkt CPg+qK76fxzeRDOhUGun2hcVOEXNj1mucnDfpK3ie3AzdZc41h57o3RPGegP4aJ/ZG5I Jj+myTWz764PfBr44PpsE7UQAZq1ym9j6A8JTHDk2x9ZLEGsX+TYD5m8V5aPz6Z5XNnM IwTu5X+WZizro4uMw7g67d1L4TwL1DevQ4V5JUESMuvrQShMlQmymLETmUSuQOXxbl31 I8z6+CwUYAZdQrqb9CcT+fPZDbp81NdJubmh44JC9pqPTkhjPmbpPLOR9ETS+m+Z+vXe M/FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185774; x=1764790574; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KGzKyE17LTIjiDUTLc5iTgG4+2QVa3flPC17N65tMUo=; b=Jh65AhLvxbdHvW3mogzpMGB7cw6boaOnQlxfkjnO/muhkdYjXHWemGLSc5FbWWeAlB 9HyCRjxmZJmUkfPlKHUpnqkdUYgaROK43JT63GMESOlcmkfvYsijTYBjE25djpxL4Fel NgCUOjjEYJZMBcBaH2brAgP3uFA1Iye5G4Xbq+cSe7Ye/vnKpUW7AhuBVHfW93v2+dUr oVUirOULHuuAqAcSYAB/Rl17PHS5V0OzsJ96IKwJGQalqYZWjk6mMtiyxGzv7DbEIPsL ZL/83qPFoHb+0gUb1AtS5ypovGsAJYnbT+1cZqnBZn2BOdhoNYQNdL6BUFELmZzW4r03 6nlQ== X-Forwarded-Encrypted: i=1; AJvYcCVs4Qb6z6V6IlYGgWCMKqV+oyTQ1kgbXx92V+ifqGqiR9d4blN7RAaG124v4Ag8nXEbtD1VmlAtNoyGAS4=@vger.kernel.org X-Gm-Message-State: AOJu0YxpHZJGr9CGy/tfcdjvMt6bQooWBR2iHFvUgYy75JqlGh2Szd22 i2jC6eYjIawna46iw6JoU74uBk6KxKPykQiozzd2pIUqcPGfeGKpyRsefufNSquPYvEdNb/LNiY mX+ZxGseGV5xQhQ== X-Google-Smtp-Source: AGHT+IHGeSb0yVqfD3cs07aFTfYmCZ7mgD11B0dXTDhgzXpsQtWff0tzFWVLtBWXSh4eYpi5VvZWhIYsSuOdcA== X-Received: from pfus11.prod.google.com ([2002:a05:6a00:8cb:b0:7bf:76b4:31f8]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:23d5:b0:7ad:9e8a:1f72 with SMTP id d2e1a72fcca58-7c58c7a7601mr24343739b3a.14.1764185774060; Wed, 26 Nov 2025 11:36:14 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:49 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-3-dmatlack@google.com> Subject: [PATCH 02/21] PCI: Add API to track PCI devices preserved across Live Update From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an API to enable the PCI subsystem to track all devices that are preserved across a Live Update, including both incoming devices (passed from the previous kernel) and outgoing devices (passed to the next kernel). Use PCI segment number and BDF to keep track of devices across Live Update. This means the kernel must keep both identifiers constant across a Live Update for any preserved device. VFs are not supported for now, since that requires preserving SR-IOV state on the device to ensure the same number of VFs appear after kexec and with the same BDFs. Drivers that preserve devices across Live Update can now register their struct liveupdate_file_handler with the PCI subsystem so that the PCI subsystem can allocate and manage File-Lifecycle-Bound (FLB) global data to track the list of incoming and outgoing preserved devices. pci_liveupdate_register_fh(driver_fh) pci_liveupdate_unregister_fh(driver_fh) Drivers can notify the PCI subsystem whenever a device is preserved and unpreserved with the following APIs: pci_liveupdate_outgoing_preserve(pci_dev) pci_liveupdate_outgoing_unpreserve(pci_dev) After a Live Update, the PCI subsystem can fetch its FLB global data from the previous kernel from the Live Update Orchestrator (LUO) to determine which devices are preserved. This API is also made available for drivers to use to check if a device was preserved before userspace retrieves the file for it. pci_liveupdate_incoming_is_preserved(pci_dev) Once a driver has finished restoring an incoming preserved device, it can notify the PCI subsystem with the following call: pci_liveupdate_incoming_finish(pci_dev) This will be used in subsequent commits by the vfio-pci driver to preserve VFIO devices across Live Update. Signed-off-by: David Matlack --- drivers/pci/Makefile | 1 + drivers/pci/liveupdate.c | 248 ++++++++++++++++++++++++++++++++++++ include/linux/kho/abi/pci.h | 53 ++++++++ include/linux/pci.h | 38 ++++++ 4 files changed, 340 insertions(+) create mode 100644 drivers/pci/liveupdate.c create mode 100644 include/linux/kho/abi/pci.h diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 67647f1880fb..0cb43e10e71d 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -16,6 +16,7 @@ obj-$(CONFIG_PROC_FS) +=3D proc.o obj-$(CONFIG_SYSFS) +=3D pci-sysfs.o slot.o obj-$(CONFIG_ACPI) +=3D pci-acpi.o obj-$(CONFIG_GENERIC_PCI_IOMAP) +=3D iomap.o +obj-$(CONFIG_LIVEUPDATE) +=3D liveupdate.o endif =20 obj-$(CONFIG_OF) +=3D of.o diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c new file mode 100644 index 000000000000..f9bb97f3bada --- /dev/null +++ b/drivers/pci/liveupdate.c @@ -0,0 +1,248 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright (c) 2025, Google LLC. + * David Matlack + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static DEFINE_MUTEX(pci_flb_outgoing_lock); +static DEFINE_MUTEX(pci_flb_incoming_lock); + +static int pci_flb_preserve(struct liveupdate_flb_op_args *args) +{ + struct pci_dev *dev =3D NULL; + struct folio *folio; + unsigned int order; + int nr_devices =3D 0; + int ret; + + /* + * Calculate the maximum number of devices based on what's present + * on the system currently (including VFs) to size the folio holding + * struct pci_ser. This is not perfect given devices could be + * hotplugged, but it's also unlikely that all devices in the system are + * going to be preserved anyway. + */ + for_each_pci_dev(dev) { + if (dev->is_virtfn) + continue; + + nr_devices +=3D 1 + pci_sriov_get_totalvfs(dev); + } + + order =3D get_order(offsetof(struct pci_ser, devices[nr_devices + 1])); + + folio =3D folio_alloc(GFP_KERNEL | __GFP_ZERO, order); + if (!folio) + return -ENOMEM; + + ret =3D kho_preserve_folio(folio); + if (ret) { + folio_put(folio); + return ret; + } + + args->obj =3D folio_address(folio); + args->data =3D virt_to_phys(args->obj); + + return 0; +} + +static void pci_flb_unpreserve(struct liveupdate_flb_op_args *args) +{ + struct pci_ser *ser =3D args->obj; + struct folio *folio =3D virt_to_folio(ser); + + WARN_ON_ONCE(ser->nr_devices); + kho_unpreserve_folio(folio); + folio_put(folio); +} + +static int pci_flb_retrieve(struct liveupdate_flb_op_args *args) +{ + struct folio *folio; + + folio =3D kho_restore_folio(args->data); + if (!folio) + panic("Unable to restore preserved FLB data from KHO (0x%llx)\n", args->= data); + + args->obj =3D folio_address(folio); + return 0; +} + +static void pci_flb_finish(struct liveupdate_flb_op_args *args) +{ + struct pci_ser *ser =3D args->obj; + + /* + * Sanity check that all devices have been finished via + * pci_liveupdate_incoming_finish(). + */ + WARN_ON_ONCE(ser->nr_devices); + folio_put(virt_to_folio(ser)); +} + +static struct liveupdate_flb_ops pci_liveupdate_flb_ops =3D { + .preserve =3D pci_flb_preserve, + .unpreserve =3D pci_flb_unpreserve, + .retrieve =3D pci_flb_retrieve, + .finish =3D pci_flb_finish, + .owner =3D THIS_MODULE, +}; + +static struct liveupdate_flb pci_liveupdate_flb =3D { + .ops =3D &pci_liveupdate_flb_ops, + .compatible =3D PCI_LUO_FLB_COMPATIBLE, +}; + +#define INIT_PCI_DEV_SER(_dev) { \ + .domain =3D pci_domain_nr((_dev)->bus), \ + .bdf =3D pci_dev_id(_dev), \ +} + +static int pci_dev_ser_cmp(const void *__a, const void *__b) +{ + const struct pci_dev_ser *a =3D __a, *b =3D __b; + + return cmp_int(a->domain << 16 | a->bdf, b->domain << 16 | b->bdf); +} + +static struct pci_dev_ser *pci_ser_find(struct pci_ser *ser, struct pci_de= v *dev) +{ + const struct pci_dev_ser key =3D INIT_PCI_DEV_SER(dev); + + return bsearch(&key, ser->devices, ser->nr_devices, + sizeof(key), pci_dev_ser_cmp); +} + +static int pci_ser_delete(struct pci_ser *ser, struct pci_dev *dev) +{ + struct pci_dev_ser *dev_ser; + int i; + + dev_ser =3D pci_ser_find(ser, dev); + if (!dev_ser) + return -ENOENT; + + for (i =3D dev_ser - ser->devices; i < ser->nr_devices - 1; i++) + ser->devices[i] =3D ser->devices[i + 1]; + + ser->nr_devices--; + return 0; +} + +static int max_nr_devices(struct pci_ser *ser) +{ + u64 size; + + size =3D folio_size(virt_to_folio(ser)); + size -=3D offsetof(struct pci_ser, devices); + + return size / sizeof(struct pci_dev_ser); +} + +int pci_liveupdate_outgoing_preserve(struct pci_dev *dev) +{ + struct pci_dev_ser new =3D INIT_PCI_DEV_SER(dev); + struct pci_ser *ser; + int i, ret; + + /* VFs are not supported yet due to BDF instability across kexec */ + if (dev->is_virtfn) + return -EINVAL; + + guard(mutex)(&pci_flb_outgoing_lock); + + ret =3D liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser); + if (ret) + return ret; + + if (ser->nr_devices =3D=3D max_nr_devices(ser)) + return -E2BIG; + + for (i =3D ser->nr_devices; i > 0; i--) { + struct pci_dev_ser *prev =3D &ser->devices[i - 1]; + int cmp =3D pci_dev_ser_cmp(&new, prev); + + /* This device is already preserved. */ + if (cmp =3D=3D 0) + return 0; + + if (cmp > 0) + break; + + ser->devices[i] =3D *prev; + } + + ser->devices[i] =3D new; + ser->nr_devices++; + return 0; +} +EXPORT_SYMBOL_GPL(pci_liveupdate_outgoing_preserve); + +void pci_liveupdate_outgoing_unpreserve(struct pci_dev *dev) +{ + struct pci_ser *ser; + int ret; + + guard(mutex)(&pci_flb_outgoing_lock); + + ret =3D liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser); + if (WARN_ON_ONCE(ret)) + return; + + WARN_ON_ONCE(pci_ser_delete(ser, dev)); +} +EXPORT_SYMBOL_GPL(pci_liveupdate_outgoing_unpreserve); + +bool pci_liveupdate_incoming_is_preserved(struct pci_dev *dev) +{ + struct pci_ser *ser; + int ret; + + guard(mutex)(&pci_flb_incoming_lock); + + ret =3D liveupdate_flb_get_incoming(&pci_liveupdate_flb, (void **)&ser); + if (ret) + return false; + + return pci_ser_find(ser, dev); +} +EXPORT_SYMBOL_GPL(pci_liveupdate_incoming_is_preserved); + +void pci_liveupdate_incoming_finish(struct pci_dev *dev) +{ + struct pci_ser *ser; + int ret; + + guard(mutex)(&pci_flb_incoming_lock); + + ret =3D liveupdate_flb_get_incoming(&pci_liveupdate_flb, (void **)&ser); + if (WARN_ON_ONCE(ret)) + return; + + WARN_ON_ONCE(pci_ser_delete(ser, dev)); +} +EXPORT_SYMBOL_GPL(pci_liveupdate_incoming_finish); + +int pci_liveupdate_register_fh(struct liveupdate_file_handler *fh) +{ + return liveupdate_register_flb(fh, &pci_liveupdate_flb); +} +EXPORT_SYMBOL_GPL(pci_liveupdate_register_fh); + +int pci_liveupdate_unregister_fh(struct liveupdate_file_handler *fh) +{ + return liveupdate_unregister_flb(fh, &pci_liveupdate_flb); +} +EXPORT_SYMBOL_GPL(pci_liveupdate_unregister_fh); diff --git a/include/linux/kho/abi/pci.h b/include/linux/kho/abi/pci.h new file mode 100644 index 000000000000..53744b6f191a --- /dev/null +++ b/include/linux/kho/abi/pci.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Copyright (c) 2025, Google LLC. + * David Matlack + */ + +#ifndef _LINUX_KHO_ABI_PCI_H +#define _LINUX_KHO_ABI_PCI_H + +#include +#include + +/** + * DOC: PCI File-Lifecycle Bound (FLB) Live Update ABI + * + * This header defines the ABI for preserving core PCI state across kexec = using + * Live Update File-Lifecycle Bound (FLB) data. + * + * This interface is a contract. Any modification to any of the serializat= ion + * structs defined here constitutes a breaking change. Such changes require + * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE string. + */ + +#define PCI_LUO_FLB_COMPATIBLE "pci-v1" + +/** + * struct pci_dev_ser - Serialized state about a single PCI device. + * + * @domain: The device's PCI domain number (segment). + * @bdf: The device's PCI bus, device, and function number. + */ +struct pci_dev_ser { + u16 domain; + u16 bdf; +} __packed; + +/** + * struct pci_ser - PCI Subsystem Live Update State + * + * This struct tracks state about all devices that are being preserved acr= oss + * a Live Update for the next kernel. + * + * @nr_devices: The number of devices that were preserved. + * @devices: Flexible array of pci_dev_ser structs for each device. Guaran= teed + * to be sorted ascending by domain and bdf. + */ +struct pci_ser { + u64 nr_devices; + struct pci_dev_ser devices[]; +} __packed; + +#endif /* _LINUX_KHO_ABI_PCI_H */ diff --git a/include/linux/pci.h b/include/linux/pci.h index d1fdf81fbe1e..6a3c2d7e5b82 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -40,6 +40,7 @@ #include #include #include +#include =20 #include =20 @@ -2795,4 +2796,41 @@ void pci_uevent_ers(struct pci_dev *pdev, enum pci_= ers_result err_type); WARN_ONCE(condition, "%s %s: " fmt, \ dev_driver_string(&(pdev)->dev), pci_name(pdev), ##arg) =20 +#ifdef CONFIG_LIVEUPDATE +int pci_liveupdate_outgoing_preserve(struct pci_dev *dev); +void pci_liveupdate_outgoing_unpreserve(struct pci_dev *dev); +bool pci_liveupdate_incoming_is_preserved(struct pci_dev *dev); +void pci_liveupdate_incoming_finish(struct pci_dev *dev); +int pci_liveupdate_register_fh(struct liveupdate_file_handler *fh); +int pci_liveupdate_unregister_fh(struct liveupdate_file_handler *fh); +#else /* !CONFIG_LIVEUPDATE */ +static inline int pci_liveupdate_outgoing_preserve(struct pci_dev *dev) +{ + return -EOPNOTSUPP; +} + +static inline void pci_liveupdate_outgoing_unpreserve(struct pci_dev *dev) +{ +} + +static inline bool pci_liveupdate_incoming_is_preserved(struct pci_dev *de= v) +{ + return false; +} + +static inline void pci_liveupdate_incoming_finish(struct pci_dev *dev) +{ +} + +static inline int pci_liveupdate_register_fh(struct liveupdate_file_handle= r *fh) +{ + return -EOPNOTSUPP; +} + +static inline int pci_liveupdate_unregister_fh(struct liveupdate_file_hand= ler *fh) +{ + return -EOPNOTSUPP; +} +#endif /* !CONFIG_LIVEUPDATE */ + #endif /* LINUX_PCI_H */ --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8016F341050 for ; Wed, 26 Nov 2025 19:36:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185778; cv=none; b=mNGYBJKAE1Uj+PtRTLz+YVnYd5/IJ2UEO1i65VyMmR51Crxf8cywoaq0s/HbmaBPvCOAK/u4/vT6pHY9gE8rF5sRdyLpXttVfGlv/sdsZ4Ks1aiahOjQwyHQINOfpBT3EyCBHJBC/LU0FmLkU1gSsAGSFJ0Volvw0Ydfsd/VAJQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185778; c=relaxed/simple; bh=DgrTkiwsj7G+kzAv5Mmspt4Q8NbY6RXRBJtrpMsEy5E=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tw52kdUniYNj0BX/FcFVo0BOIcjl9FVnekxUmTEdXPxPBkZdJKWeiVBKz9lXyKDywbaq2OkI9PlNkFm7uYJy+Kbj8v/OboqU9O58hbQtQeBsrdiCWHt9WQPzzuKrNavOVC0GjqCRT/SDUAH0tEnWm9CG2gDP+7gphEmHsqSwTag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dbU6Adrw; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dbU6Adrw" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-297f8a2ba9eso1820345ad.3 for ; Wed, 26 Nov 2025 11:36:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185776; x=1764790576; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=G4W8jO8YwRXmYQwem/Oi26baRXbR//DiNTTxQHZyVKU=; b=dbU6Adrw/r4HbB2gp1axytAQ3o43CYiIxktTc1NaQUbbgieIqOqSyN5L8pEVth1EjJ F3b4+FXxQRnSrd1XwMjcNsmTfIsWkJgUX9+1PJ92FFNhD4SJ3SbeYKpNZXA++d90+Qn8 pPWLuJE005M8Mcnq/5AJppqd2ZQ8ix0Gj055ltkMvlss3XK+QHflkdaxzbiX1GRJkcBN 43HUbvtflmJJGE8XyGlFlpcLebVvL1NIhknhVDX0RaSiDIGy1BAeDGNVYh5DuQ9I8S3o Y+hYJ7aFFs7hR941kGZxxfOFA4W32vUZtMCkAHOjkAlfO8s+2qTXCQRpmBGjHL5S/06O WIjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185776; x=1764790576; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=G4W8jO8YwRXmYQwem/Oi26baRXbR//DiNTTxQHZyVKU=; b=cOLCJ2WAv9oNB7HTtnO0wRC5sqxVCj4b05Wez7W07hhPWnVkts2QFjUTG5MzKAY2wR DVep+ox8cFRu9Zn9uGvFgUQ+DJLbbGyXg4ROqIEHgSviKj1q20eD/4oau0j8Dt2QIirG WQpMNn2lr1zZpamYSkupd9ju2q5pYd+77aZdkTL6kspcaZjQHMHaTlMxsvzWSZk7s2EL Kh31tY3tZbVuG6o8rc6tndKBfV4D+niXSdLWwKG70QmQb1rbJvnV0y7zahZgBWcSsWKw OE9u7dcfQUEQaOIOpvS4KcKB2Hd8MsvX3jhcD6oB1MqOh0cQMrjfeXSiEDg1JmcjCgfo lX6w== X-Forwarded-Encrypted: i=1; AJvYcCVtGepL2+KAQXCIvN9nOcfRmixOMGAkZ7siRBQz5RC4KmWbEFN3UzIpr/opNgkB2vyhcbweDua0u+3Eiy8=@vger.kernel.org X-Gm-Message-State: AOJu0Yy+ysEvFtcr8g3fyHgQwYaMOFVlTQFrOAL9sNw0EmMDc/DLy2fW VtI68osP7bpOBLeTlt6GQAcPHccNnQ0EyeWc7XlEjfl582kaqaap5BK0+I9AdPI6YqZr5yB4z5Q hY71VZgj10jUt6A== X-Google-Smtp-Source: AGHT+IH14W8+tVRPyICSHFhnhdie4VNOZ11aj2/eJVs/11OeDSQhWpHoOWodZE+UlFUV7qImHq1Artm+rlwecg== X-Received: from plbjn3.prod.google.com ([2002:a17:903:503:b0:293:e00:bb82]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ce89:b0:298:485d:556b with SMTP id d9443c01a7336-29b6be8c617mr237546495ad.5.1764185775828; Wed, 26 Nov 2025 11:36:15 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:50 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-4-dmatlack@google.com> Subject: [PATCH 03/21] PCI: Require driver_override for incoming Live Update preserved devices From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Require driver_override to be set to bind an incoming Live Update preserved device to a driver. Auto-probing could lead to the device being bound to a different driver than what was bound to the device prior to Live Update. Delegate binding preserved devices to the right driver to userspace by requiring driver_override to be set on the device. This restriction is relaxed once a driver calls pci_liveupdate_incoming_finish(). Signed-off-by: David Matlack --- drivers/pci/pci-driver.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 302d61783f6c..294ce92331a8 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -420,18 +420,26 @@ static int __pci_device_probe(struct pci_driver *drv,= struct pci_dev *pci_dev) } =20 #ifdef CONFIG_PCI_IOV -static inline bool pci_device_can_probe(struct pci_dev *pdev) +static inline bool pci_iov_device_can_probe(struct pci_dev *pdev) { return (!pdev->is_virtfn || pdev->physfn->sriov->drivers_autoprobe || pdev->driver_override); } #else -static inline bool pci_device_can_probe(struct pci_dev *pdev) +static inline bool pci_iov_device_can_probe(struct pci_dev *pdev) { return true; } #endif =20 +static inline bool pci_device_can_probe(struct pci_dev *pdev) +{ + if (pci_liveupdate_incoming_is_preserved(pdev)) + return pdev->driver_override; + + return pci_iov_device_can_probe(pdev); +} + static int pci_device_probe(struct device *dev) { int error; --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 387CF340273 for ; Wed, 26 Nov 2025 19:36:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185780; cv=none; b=bjUe54BRT0lK/UIZZZYZmpNgRq+C7qw34ebpE0y8VF7YQDX8jqSl+iCSbNbaWM7qEssTOr70sDIsP15WIBxTyHX6QYtFQEidhwA3R1kF0wOTmyK3y+mSOzEv2I71JvpHsC+8d5Tjt05nMZ9ombOeyDwu6YBLTV0GzyPiJIMGEJA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185780; c=relaxed/simple; bh=MdHEo5JlzB38kfrUS/yfJXxeBIf14QgYazV3eFOuYps=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Cko+O+mY3NPs/sh0ImNqhCkpkC4AOiA5Ul4jHfaaA/CezXC973h/pPKLF2IRZxtrulK5U4RBdvZjoaSwm6dj9ewa17m0E882wDd+s6/5P0zIhRx5/XYNInWlJ3S3UeKCYKf0roRDwoY0xkZohXeH9L9y34U6N0pLsauJbpyKKNg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=z3gtCNHc; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="z3gtCNHc" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7a99a5f77e0so94811b3a.2 for ; Wed, 26 Nov 2025 11:36:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185777; x=1764790577; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=BAvCfHwQMnOs+cOiqWeL0elvv18GHJNsZ1eh/Y/HY0A=; b=z3gtCNHcf+fM7LoIsD3FuDVUkbiRFI6nYpU1cqvbOngvIsVwW3x+OZIFk5ANo94SMy yYlNgAtL2bsUiT34YMLA6criqC0v6i5Fhe2Sf6QgtQRY47fEeBBzodgDtCnAEHpuQcqT C/4Dw2dEVHOZbTSdikhSxybcswAK90d2ZxQ8vTg7kav1qWPkxJ7i0F/iJuvgmrpyOp3M 0WTbAOqHM4gKLfbgpiEOVU1p3IAEqX5JKUqujVf5Zq2DVLbjH0BJv/+XFyxza8PGr27R 6V9HjKd4GJNRVtCd8truRTaeZWSiXkYY/vt8vOCx1hX0YqC+X6sW9X9VMZprMoGlKzwJ DbTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185777; x=1764790577; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BAvCfHwQMnOs+cOiqWeL0elvv18GHJNsZ1eh/Y/HY0A=; b=NBJotHYccoOz1q1Pd19FOVH730OquuqR9qbCq18aFHukadOTEjVKtHW9Qa5vuju1qk W94jJRgfAtmJFnUhiKwgXOvAifgB7AByfQR7Anq6gVUMLQ7NtbMXu3s/mnQppjMYG0V1 mBcRbgXuOfmbsisgxtOnIIHqfs4S68SgkbC+ZBZnHBUNJJrjlhx8nKnJb70aQjXejHeU HnExjoGNH8QBg+sSPI3BeenxvIrIW0Ki49JlFWZOmMJrmR8gS2a/R0ygLCCJRUUJefSp fNwJDqYB63PHJx8uyaQ2KFj9TnfzySnxd3GvHbbbvnm5FaLFq2uuQRh2YDlEX2SjjTZ+ qxjg== X-Forwarded-Encrypted: i=1; AJvYcCU6E34rXBmCv22Z74cK1JOZx5MVWgbuFiUveHDy7Tqp4L9hWle0a5/sApQlJJYQ37dWaYQlYjO4ALUvRfY=@vger.kernel.org X-Gm-Message-State: AOJu0YzLDtV8KLQRMpQ3bBtqlA0UZiBC6i0zKyNI4QkinEt3CTpUjIw2 fFozVRZCNvGsGFYQoThGN9JEnA2sLRp5iRzG80EvGIUXHuOAuYyLJQlUUGTTQeGa8IOee77dT9D +gSX2plhTQ138JQ== X-Google-Smtp-Source: AGHT+IG+xXSTrXQR9csQs3WCqwSJan5Af+yQylnVLM5oBb0LBhg+2VUH2UWT8kYvjS2p2mBfoZH/ozR1D/SXjA== X-Received: from pfbna23.prod.google.com ([2002:a05:6a00:3e17:b0:7b9:7dd0:134d]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:17a3:b0:7b8:349:1b24 with SMTP id d2e1a72fcca58-7c58e116e23mr20953003b3a.22.1764185777352; Wed, 26 Nov 2025 11:36:17 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:51 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-5-dmatlack@google.com> Subject: [PATCH 04/21] vfio/pci: Register a file handler with Live Update Orchestrator From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Register a live update file handler for vfio-pci device files. Add stub implementations of all required callbacks so that registration does not fail (i.e. to avoid breaking git-bisect). This file handler will be extended in subsequent commits to enable a device bound to vfio-pci to run without interruption while the host is going through a kexec Live Update. Signed-off-by: Vipin Sharma Co-Developed-by: David Matlack Signed-off-by: David Matlack --- MAINTAINERS | 1 + drivers/vfio/pci/Makefile | 1 + drivers/vfio/pci/vfio_pci.c | 9 +++- drivers/vfio/pci/vfio_pci_liveupdate.c | 69 ++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 14 ++++++ include/linux/kho/abi/vfio_pci.h | 28 +++++++++++ 6 files changed, 121 insertions(+), 1 deletion(-) create mode 100644 drivers/vfio/pci/vfio_pci_liveupdate.c create mode 100644 include/linux/kho/abi/vfio_pci.h diff --git a/MAINTAINERS b/MAINTAINERS index 2722f98d0ed7..ff50977277c4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -26933,6 +26933,7 @@ F: Documentation/ABI/testing/debugfs-vfio F: Documentation/ABI/testing/sysfs-devices-vfio-dev F: Documentation/driver-api/vfio.rst F: drivers/vfio/ +F: include/linux/kho/abi/vfio_pci.h F: include/linux/vfio.h F: include/linux/vfio_pci_core.h F: include/uapi/linux/vfio.h diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index cf00c0a7e55c..929df22c079b 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -2,6 +2,7 @@ =20 vfio-pci-core-y :=3D vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio= _pci_config.o vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) +=3D vfio_pci_zdev.o +vfio-pci-core-$(CONFIG_LIVEUPDATE) +=3D vfio_pci_liveupdate.o obj-$(CONFIG_VFIO_PCI_CORE) +=3D vfio-pci-core.o =20 vfio-pci-y :=3D vfio_pci.o diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index ac10f14417f2..c2fe34a830d8 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -252,6 +252,10 @@ static int __init vfio_pci_init(void) int ret; bool is_disable_vga =3D true; =20 + ret =3D vfio_pci_liveupdate_init(); + if (ret) + return ret; + #ifdef CONFIG_VFIO_PCI_VGA is_disable_vga =3D disable_vga; #endif @@ -260,8 +264,10 @@ static int __init vfio_pci_init(void) =20 /* Register and scan for devices */ ret =3D pci_register_driver(&vfio_pci_driver); - if (ret) + if (ret) { + vfio_pci_liveupdate_cleanup(); return ret; + } =20 vfio_pci_fill_ids(); =20 @@ -275,6 +281,7 @@ module_init(vfio_pci_init); static void __exit vfio_pci_cleanup(void) { pci_unregister_driver(&vfio_pci_driver); + vfio_pci_liveupdate_cleanup(); } module_exit(vfio_pci_cleanup); =20 diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c new file mode 100644 index 000000000000..b84e63c0357b --- /dev/null +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright (c) 2025, Google LLC. + * Vipin Sharma + * David Matlack + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include + +#include "vfio_pci_priv.h" + +static bool vfio_pci_liveupdate_can_preserve(struct liveupdate_file_handle= r *handler, + struct file *file) +{ + return false; +} + +static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *ar= gs) +{ + return -EOPNOTSUPP; +} + +static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args = *args) +{ +} + +static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) +{ + return -EOPNOTSUPP; +} + +static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) +{ +} + +static const struct liveupdate_file_ops vfio_pci_liveupdate_file_ops =3D { + .can_preserve =3D vfio_pci_liveupdate_can_preserve, + .preserve =3D vfio_pci_liveupdate_preserve, + .unpreserve =3D vfio_pci_liveupdate_unpreserve, + .retrieve =3D vfio_pci_liveupdate_retrieve, + .finish =3D vfio_pci_liveupdate_finish, + .owner =3D THIS_MODULE, +}; + +static struct liveupdate_file_handler vfio_pci_liveupdate_fh =3D { + .ops =3D &vfio_pci_liveupdate_file_ops, + .compatible =3D VFIO_PCI_LUO_FH_COMPATIBLE, +}; + +int __init vfio_pci_liveupdate_init(void) +{ + if (!liveupdate_enabled()) + return 0; + + return liveupdate_register_file_handler(&vfio_pci_liveupdate_fh); +} + +void vfio_pci_liveupdate_cleanup(void) +{ + if (!liveupdate_enabled()) + return; + + liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); +} diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index a9972eacb293..b9f7c4e2b4df 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -107,4 +107,18 @@ static inline bool vfio_pci_is_vga(struct pci_dev *pde= v) return (pdev->class >> 8) =3D=3D PCI_CLASS_DISPLAY_VGA; } =20 +#ifdef CONFIG_LIVEUPDATE +int __init vfio_pci_liveupdate_init(void); +void vfio_pci_liveupdate_cleanup(void); +#else +static inline int vfio_pci_liveupdate_init(void) +{ + return 0; +} + +static inline void vfio_pci_liveupdate_cleanup(void) +{ +} +#endif /* CONFIG_LIVEUPDATE */ + #endif diff --git a/include/linux/kho/abi/vfio_pci.h b/include/linux/kho/abi/vfio_= pci.h new file mode 100644 index 000000000000..37a845eed972 --- /dev/null +++ b/include/linux/kho/abi/vfio_pci.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Copyright (c) 2025, Google LLC. + * Vipin Sharma + * David Matlack + */ + +#ifndef _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H +#define _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H + +/** + * DOC: VFIO PCI Live Update ABI + * + * This header defines the ABI for preserving the state of a VFIO PCI devi= ce + * files across a kexec reboot using LUO. + * + * Device metadata is serialized into memory which is then handed to the n= ext + * kernel via KHO. + * + * This interface is a contract. Any modification to any of the serializat= ion + * structs defined here constitutes a breaking change. Such changes require + * incrementing the version number in the VFIO_PCI_LUO_FH_COMPATIBLE strin= g. + */ + +#define VFIO_PCI_LUO_FH_COMPATIBLE "vfio-pci-v1" + +#endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */ --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D770D341ADF for ; Wed, 26 Nov 2025 19:36:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185782; cv=none; b=ni8GZmGxE3PdfeKHhbqRSZapETBYKRpAZOGxQxskIZrilH2zPLO7JQZgluFTRtF50wRh1USvaPaoIV7LhVo7l7K2Vdqtr5t7CLFe+A+2F4CisVWfomU4XOwjauBh7Poe/cKzukxHgUXMIeQNmYZbzQe4r1k2ZxVCMQecJ/n6pPY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185782; c=relaxed/simple; bh=eWQ0jCNtCej3K5+qd3kNS3unsOCSFfv5PnVYoXcfUwc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XaOgEYemL3aJ0SJRyu4AI98x6mjdPf6FfI8Frk5m+/Pm44i6TJiFvKmARAJ5BOw6PcjruSsF/i8OsjbXZ3bNZX3Ry/PSIRfkswLBQoWCbhOe7md7E/xLNottj1/5cqjOx6Hy6+73T2+Nix/Z4vfxV7B9lvVeN/s8Uze0tRYIcak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OF99GhsF; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OF99GhsF" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7be94e1a073so94036b3a.2 for ; Wed, 26 Nov 2025 11:36:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185779; x=1764790579; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0ewOyHN8sWLyC5m0IHbHnpnDsQX1yh6Ng3tYlExD0qM=; b=OF99GhsFHzEVXZ9zyJO3PtjymIap0eRmnwm+led79mMqBRUhG5nl0XcNzmaTmOaT6r BXoOSKlfRSzM/sMc2NvehaPD6M9EO1IoRt8jaUg2SBP3TdpKr+gnfT84zuLlm7PUPGn2 68Rinqs4jgSEQ6zr1cfCDqOFQ7w/azQYcSZ/IO3SrgcqmfMFkke+btb6TlOn6Q7hmW1m lwddOgTpWLmm+EtP1vjWlMFHJmkPYsB2Vy9wwgXt1pFPXrRjFCCU9+VFGc6Z3zWtZozd tMGaXmivG///ZYF0OTY1EX/V/+OXFfuN0VJkF75zsLWDJrKNvs5LCpL0fdqMvS5D/mKo K+ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185779; x=1764790579; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0ewOyHN8sWLyC5m0IHbHnpnDsQX1yh6Ng3tYlExD0qM=; b=IOLt3Qv8ITJ64XplDNqD9e98MxEpIdMHALljaCEfwJo2fWV0iyAmeN0OMPw9Op8sFj uvo7dMIUck/dJR/CnaO+Oxf89n0URGj27ksapjQ2F0UI3xj+BUZxCbYrtCgNS/L6WTxH h1tr5yyOem4MiD22TxhI+TmEXyitkjhL2EndvzzqEXIv1lR8jeVBTWM1doKDcvZFkN3d qTb7fUgpqrDJWA2oQTeAqW8BuahzEIo0QpnZZcGBL3BM54t+n92bCsxmhg7taaOlMAHI OEjB8V4/HH4t7oeSWKeHmeJYaOZEsk4W1OWfDlDyQ0VXyW3cJOvRk+dnmitF3FhKnbnb OcmQ== X-Forwarded-Encrypted: i=1; AJvYcCVinS224ujgDUeQqEbjXqpG988TIi6tukUYZUbfIhfZMKQ48CoKzeWgyVzL2p33iEfpDh1pW1Apes0cvx8=@vger.kernel.org X-Gm-Message-State: AOJu0YyrbgIuGtbNbyBDfCrkkIDBaNmejJzUyKSk8aag6Lbo+vnTiF8n MUc8eUXWdjXrOtOKH/szTh1HaLYlPvy+2DYG2DDY31mWW2+NgcmC5oiQtnj/haS9eUe3NCwSHwQ HE/EdQxTjBmmg+g== X-Google-Smtp-Source: AGHT+IHBXARD0M4TNVc+5iNMok8lRsm6YAxKL931j3Ozfj+tg8nBvG322NUGKmqWAR0CwYxApWkwx5vZFj9KpQ== X-Received: from pfblu1.prod.google.com ([2002:a05:6a00:7481:b0:7ba:6768:fe53]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3983:b0:7ac:78d6:5f00 with SMTP id d2e1a72fcca58-7c58effc48emr22677849b3a.31.1764185778900; Wed, 26 Nov 2025 11:36:18 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:52 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-6-dmatlack@google.com> Subject: [PATCH 05/21] vfio/pci: Preserve vfio-pci device files across Live Update From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Implement the live update file handler callbacks to preserve a vfio-pci device across a Live Update. Subsequent commits will enable userspace to then retrieve this file after the Live Update. Live Update support is scoped only to cdev files (i.e. not VFIO_GROUP_GET_DEVICE_FD files). State about each device is serialized into a new ABI struct vfio_pci_core_device_ser. The contents of this struct are preserved across the Live Update to the next kernel using a combination of Kexec-Handover (KHO) to preserve the page(s) holding the struct and the Live Update Orchestrator (LUO) to preserve the physical address of the struct. For now the only contents of struct vfio_pci_core_device_ser the device's PCI segment number and BDF, so that the device can be uniquely identified after the Live Update. Require that userspace disables interrupts on the device prior to freeze() so that the device does not send any interrupts until new interrupt handlers have been set up by the next kernel. Reset the device and restore its state in the freeze() callback. This ensures the device can be received by the next kernel in a consistent state. Eventually this will be dropped and the device can be preserved across in a running state, but that requires further work in VFIO and the core PCI layer. Note that LUO holds a reference to this file when it is preserved. So VFIO is guaranteed that vfio_df_device_last_close() will not be called on this device no matter what userspace does. Signed-off-by: Vipin Sharma Co-Developed-by: David Matlack Signed-off-by: David Matlack --- drivers/vfio/pci/vfio_pci.c | 2 +- drivers/vfio/pci/vfio_pci_liveupdate.c | 100 ++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_priv.h | 2 + drivers/vfio/vfio.h | 13 ---- drivers/vfio/vfio_main.c | 9 --- include/linux/kho/abi/vfio_pci.h | 15 ++++ include/linux/vfio.h | 28 +++++++ 7 files changed, 144 insertions(+), 25 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index c2fe34a830d8..281c69c086d3 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -125,7 +125,7 @@ static int vfio_pci_open_device(struct vfio_device *cor= e_vdev) return 0; } =20 -static const struct vfio_device_ops vfio_pci_ops =3D { +const struct vfio_device_ops vfio_pci_ops =3D { .name =3D "vfio-pci", .init =3D vfio_pci_core_init_dev, .release =3D vfio_pci_core_release_dev, diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index b84e63c0357b..a0147dee8c0f 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -8,25 +8,120 @@ =20 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt =20 +#include #include #include #include +#include =20 #include "vfio_pci_priv.h" =20 static bool vfio_pci_liveupdate_can_preserve(struct liveupdate_file_handle= r *handler, struct file *file) { - return false; + struct vfio_device_file *df =3D to_vfio_device_file(file); + + if (!df) + return false; + + /* Live Update support is limited to cdev files. */ + if (df->group) + return false; + + return df->device->ops =3D=3D &vfio_pci_ops; } =20 static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *ar= gs) { - return -EOPNOTSUPP; + struct vfio_device *device =3D vfio_device_from_file(args->file); + struct vfio_pci_core_device_ser *ser; + struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + struct folio *folio; + int err; + + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + pdev =3D vdev->pdev; + + if (IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)) + return -EINVAL; + + if (vfio_pci_is_intel_display(pdev)) + return -EINVAL; + + folio =3D folio_alloc(GFP_KERNEL | __GFP_ZERO, get_order(sizeof(*ser))); + if (!folio) + return -ENOMEM; + + ser =3D folio_address(folio); + + ser->bdf =3D pci_dev_id(pdev); + ser->domain =3D pci_domain_nr(pdev->bus); + + err =3D kho_preserve_folio(folio); + if (err) + goto error; + + args->serialized_data =3D virt_to_phys(ser); + return 0; + +error: + folio_put(folio); + return err; } =20 static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args = *args) { + struct vfio_pci_core_device_ser *ser =3D phys_to_virt(args->serialized_da= ta); + struct folio *folio =3D virt_to_folio(ser); + + kho_unpreserve_folio(folio); + folio_put(folio); +} + +static int vfio_pci_liveupdate_freeze(struct liveupdate_file_op_args *args) +{ + struct vfio_device *device =3D vfio_device_from_file(args->file); + struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + int ret; + + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + pdev =3D vdev->pdev; + + guard(mutex)(&device->dev_set->lock); + + /* + * Userspace must disable interrupts on the device prior to freeze so + * that the device does not send any interrupts until new interrupt + * handlers have been established by the next kernel. + */ + if (vdev->irq_type !=3D VFIO_PCI_NUM_IRQS) { + pci_err(pdev, "Freeze failed! Interrupts are still enabled.\n"); + return -EINVAL; + } + + pci_dev_lock(pdev); + + ret =3D pci_load_and_free_saved_state(pdev, &vdev->pci_saved_state); + if (ret) + goto out; + + /* + * Reset the device and restore it back to its original state before + * handing it to the next kernel. + * + * Eventually both of these should be dropped and the device should be + * kept running with its current state across the Live Update. + */ + if (vdev->reset_works) + ret =3D __pci_reset_function_locked(pdev); + + pci_restore_state(pdev); + +out: + pci_dev_unlock(pdev); + return ret; } =20 static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) @@ -42,6 +137,7 @@ static const struct liveupdate_file_ops vfio_pci_liveupd= ate_file_ops =3D { .can_preserve =3D vfio_pci_liveupdate_can_preserve, .preserve =3D vfio_pci_liveupdate_preserve, .unpreserve =3D vfio_pci_liveupdate_unpreserve, + .freeze =3D vfio_pci_liveupdate_freeze, .retrieve =3D vfio_pci_liveupdate_retrieve, .finish =3D vfio_pci_liveupdate_finish, .owner =3D THIS_MODULE, diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index b9f7c4e2b4df..7f189e5e6c0a 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -11,6 +11,8 @@ /* Cap maximum number of ioeventfds per device (arbitrary) */ #define VFIO_PCI_IOEVENTFD_MAX 1000 =20 +extern const struct vfio_device_ops vfio_pci_ops; + struct vfio_pci_ioeventfd { struct list_head next; struct vfio_pci_core_device *vdev; diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h index 50128da18bca..6b89edbbf174 100644 --- a/drivers/vfio/vfio.h +++ b/drivers/vfio/vfio.h @@ -16,17 +16,6 @@ struct iommufd_ctx; struct iommu_group; struct vfio_container; =20 -struct vfio_device_file { - struct vfio_device *device; - struct vfio_group *group; - - u8 access_granted; - u32 devid; /* only valid when iommufd is valid */ - spinlock_t kvm_ref_lock; /* protect kvm field */ - struct kvm *kvm; - struct iommufd_ctx *iommufd; /* protected by struct vfio_device_set::lock= */ -}; - void vfio_device_put_registration(struct vfio_device *device); bool vfio_device_try_get_registration(struct vfio_device *device); int vfio_df_open(struct vfio_device_file *df); @@ -34,8 +23,6 @@ void vfio_df_close(struct vfio_device_file *df); struct vfio_device_file * vfio_allocate_device_file(struct vfio_device *device); =20 -extern const struct file_operations vfio_device_fops; - #ifdef CONFIG_VFIO_NOIOMMU extern bool vfio_noiommu __read_mostly; #else diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c index 38c8e9350a60..9182dc46d73f 100644 --- a/drivers/vfio/vfio_main.c +++ b/drivers/vfio/vfio_main.c @@ -1386,15 +1386,6 @@ const struct file_operations vfio_device_fops =3D { #endif }; =20 -static struct vfio_device *vfio_device_from_file(struct file *file) -{ - struct vfio_device_file *df =3D file->private_data; - - if (file->f_op !=3D &vfio_device_fops) - return NULL; - return df->device; -} - /** * vfio_file_is_valid - True if the file is valid vfio file * @file: VFIO group file or VFIO device file diff --git a/include/linux/kho/abi/vfio_pci.h b/include/linux/kho/abi/vfio_= pci.h index 37a845eed972..9bf58a2f3820 100644 --- a/include/linux/kho/abi/vfio_pci.h +++ b/include/linux/kho/abi/vfio_pci.h @@ -9,6 +9,9 @@ #ifndef _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H #define _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H =20 +#include +#include + /** * DOC: VFIO PCI Live Update ABI * @@ -25,4 +28,16 @@ =20 #define VFIO_PCI_LUO_FH_COMPATIBLE "vfio-pci-v1" =20 +/** + * struct vfio_pci_core_device_ser - Serialized state of a single VFIO PCI + * device. + * + * @bdf: The device's PCI bus, device, and function number. + * @domain: The device's PCI domain number (segment). + */ +struct vfio_pci_core_device_ser { + u16 bdf; + u16 domain; +} __packed; + #endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index eb563f538dee..f09da3bdf786 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -80,6 +80,34 @@ struct vfio_device { #endif }; =20 +struct vfio_device_file { + struct vfio_device *device; + struct vfio_group *group; + + u8 access_granted; + u32 devid; /* only valid when iommufd is valid */ + spinlock_t kvm_ref_lock; /* protect kvm field */ + struct kvm *kvm; + struct iommufd_ctx *iommufd; /* protected by struct vfio_device_set::lock= */ +}; + +extern const struct file_operations vfio_device_fops; + +static inline struct vfio_device_file *to_vfio_device_file(struct file *fi= le) +{ + if (file->f_op !=3D &vfio_device_fops) + return NULL; + + return file->private_data; +} + +static inline struct vfio_device *vfio_device_from_file(struct file *file) +{ + struct vfio_device_file *df =3D to_vfio_device_file(file); + + return df ? df->device : NULL; +} + /** * struct vfio_device_ops - VFIO bus driver device callbacks * --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B6FE343216 for ; Wed, 26 Nov 2025 19:36:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185783; cv=none; b=djnLwwKHVD93cVJmerO6fVDXvHW4mlPep25ShNYvctEYkPhTlJIftexf2gwphS45+/l/+c9BI7pVZJ/NHRfjgkdZKvXKRxJINQ0Dg7PRDG5VI7RzqhEFkafgYsWuCT/XsKnTNLwsmEqtTuAuAGv3A03F/WBgVztjQUnwKFo8UMk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185783; c=relaxed/simple; bh=ujUPtKhH8eK0VUfq5+GCiwM/drW+JJnPW9gwQ6T/y+w=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rnlsk31HV8ufJujL1OlAD8IgzGSgRhFQuwm28zqYyQzMHQvLxk9qmgJb9K59tuBBRQISWnjXz8EMIjCtSD1jbm7MjDB2yDmPREybeC/nARkN08IQw41xfdZc7gaXx6dUYoqKiQiESkzkDHPd0VifaLyrDariRZ9hebn7fZlyLZg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LEPsZ/GI; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LEPsZ/GI" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7b82c2c2ca2so94725b3a.1 for ; Wed, 26 Nov 2025 11:36:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185780; x=1764790580; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2+rT/I6Er8D4dJfq77ubBBjkiST524x14FZV2ZAC1gg=; b=LEPsZ/GIoTjVMON6d8GulqhpcTtGgQToklb3aFboko4OK80K8fK8eYYFRUyn+9Txtg iOrnhfcqLt/zmOW0GH5yJxk+fpN0h5Z3uv1mon5bmdCh171YPNNBXflff2TQbnH4mi3k N7u0jDqplzZTMjj4hoOleyHm3+B/9MoGVuz7tzJvpAy/VfQ1dfTCUMRdwGh3jflfu1yw RU5BLp0SXxxtYnVrjOgkGrrlct5er12KeoT9o6rGPeTqM5AIx8sx7l3gJkqUQi3kAJhn YfjuxTBTaYp//2yCa2eL+7DHJNFuBBeH0DoT+3bURmaOzv13jzF8jQH5pdc8HTE0BP87 pMQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185780; x=1764790580; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2+rT/I6Er8D4dJfq77ubBBjkiST524x14FZV2ZAC1gg=; b=lNmDMYd+t+rlzzFku8DIj+/2/MX6ywxEnqVs2QQznmAxhQnICoiVaEsWUE+Gd2tJc0 rhWamU19XE4uME7XhG1ha+bmqq3doliY06BWsW5yxllw4DrfFQuNcXqq6xczCQvyKEsp aOdRJDJ3lF032vYqtgwhygNzFWHrdRVghMwqNqrfwznoUQGdYvYYB/R6xbbWDpR1+0g+ 3Kpvkr+AB2Pc5ZtQQtxe6tVx1LKEZW1m8EalEjwZP2PG0nstWbt4igTaafuHYEIGUO86 ObKO6rAIiWg0BDjwcur84iLmPoIax6bSMQ0gH17IV3MX/acCLt3dIJIm3hL1zwvCCBn0 bZMg== X-Forwarded-Encrypted: i=1; AJvYcCXF0D01/ekfRElYOsvtc3xhfuAc++qt1Tb9B9o6c4aeUWrZVSAbXbH6hdb87gIDyyofdZPsFBnrK5idkYw=@vger.kernel.org X-Gm-Message-State: AOJu0Yz2QK+wTBXTb3aaLs3QCKRXAzEV/x9MIrHfZIAuJGLvhZ8FbPz2 kkkrO15quSgAMhrezCXLefh8oLOumBFv56E7GNSWC2jc3J1VUUmlJGSBRYqRN9KycqICQp8pFvH YDDYOC7gXUW/cLQ== X-Google-Smtp-Source: AGHT+IGDRxLe3KOI/zDK03uAwR3U+/6DpvmuKHCLULTgg0s0yJtUlbehI9/yQQlCmgVCk4FPLCQg3k160b0cAA== X-Received: from pfgt42.prod.google.com ([2002:a05:6a00:13aa:b0:7ae:e321:13cf]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1399:b0:7b2:1fb0:6da3 with SMTP id d2e1a72fcca58-7c58e60111bmr20427615b3a.24.1764185780467; Wed, 26 Nov 2025 11:36:20 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:53 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-7-dmatlack@google.com> Subject: [PATCH 06/21] vfio/pci: Retrieve preserved device files after Live Update From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Enable userspace to retrieve preserved VFIO device files from VFIO after a Live Update by implementing the retrieve() and finish() file handler callbacks. Use an anonymous inode when creating the file, since the retrieved device file is not opened through any particular cdev inode, and the cdev inode does not matter in practice. For now the retrieved file is functionally equivalent a opening the corresponding VFIO cdev file. Subsequent commits will leverage the preserved state associated with the retrieved file to preserve bits of the device across Live Update. Signed-off-by: Vipin Sharma Co-Developed-by: David Matlack Signed-off-by: David Matlack --- drivers/vfio/device_cdev.c | 21 +++++--- drivers/vfio/pci/vfio_pci_liveupdate.c | 75 +++++++++++++++++++++++++- drivers/vfio/vfio_main.c | 13 +++++ include/linux/vfio.h | 12 +++++ 4 files changed, 113 insertions(+), 8 deletions(-) diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c index 480cac3a0c27..0a6e972f322b 100644 --- a/drivers/vfio/device_cdev.c +++ b/drivers/vfio/device_cdev.c @@ -16,14 +16,8 @@ void vfio_init_device_cdev(struct vfio_device *device) device->cdev.owner =3D THIS_MODULE; } =20 -/* - * device access via the fd opened by this function is blocked until - * .open_device() is called successfully during BIND_IOMMUFD. - */ -int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep) +int __vfio_device_fops_cdev_open(struct vfio_device *device, struct file *= filep) { - struct vfio_device *device =3D container_of(inode->i_cdev, - struct vfio_device, cdev); struct vfio_device_file *df; int ret; =20 @@ -52,6 +46,19 @@ int vfio_device_fops_cdev_open(struct inode *inode, stru= ct file *filep) vfio_device_put_registration(device); return ret; } +EXPORT_SYMBOL_GPL(__vfio_device_fops_cdev_open); + +/* + * device access via the fd opened by this function is blocked until + * .open_device() is called successfully during BIND_IOMMUFD. + */ +int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep) +{ + struct vfio_device *device =3D container_of(inode->i_cdev, + struct vfio_device, cdev); + + return __vfio_device_fops_cdev_open(device, filep); +} =20 static void vfio_df_get_kvm_safe(struct vfio_device_file *df) { diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index a0147dee8c0f..b7451007fca4 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -8,6 +8,8 @@ =20 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt =20 +#include +#include #include #include #include @@ -124,13 +126,83 @@ static int vfio_pci_liveupdate_freeze(struct liveupda= te_file_op_args *args) return ret; } =20 +static int match_device(struct device *dev, const void *arg) +{ + struct vfio_device *device =3D container_of(dev, struct vfio_device, devi= ce); + const struct vfio_pci_core_device_ser *ser =3D arg; + struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + pdev =3D vdev->pdev; + + return ser->bdf =3D=3D pci_dev_id(pdev) && ser->domain =3D=3D pci_domain_= nr(pdev->bus); +} + static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) { - return -EOPNOTSUPP; + struct vfio_pci_core_device_ser *ser; + struct vfio_device *device; + struct folio *folio; + struct file *file; + int ret; + + folio =3D kho_restore_folio(args->serialized_data); + if (!folio) + return -ENOENT; + + ser =3D folio_address(folio); + + device =3D vfio_find_device(ser, match_device); + if (!device) + return -ENODEV; + + /* + * During a Live Update userspace retrieves preserved VFIO cdev files by + * issuing an ioctl on /dev/liveupdate rather than by opening VFIO + * character devices. + * + * To handle that scenario, this routine simulates opening the VFIO + * character device for userspace with an anonymous inode. The returned + * file has the same properties as a cdev file (e.g. operations are + * blocked until BIND_IOMMUFD is called), aside from the inode + * association. + */ + file =3D anon_inode_getfile_fmode("[vfio-device-liveupdate]", + &vfio_device_fops, NULL, + O_RDWR, FMODE_PREAD | FMODE_PWRITE); + + if (IS_ERR(file)) { + ret =3D PTR_ERR(file); + goto out; + } + + ret =3D __vfio_device_fops_cdev_open(device, file); + if (ret) { + fput(file); + goto out; + } + + args->file =3D file; + +out: + /* Drop the reference from vfio_find_device() */ + put_device(&device->device); + + return ret; +} + +static bool vfio_pci_liveupdate_can_finish(struct liveupdate_file_op_args = *args) +{ + return args->retrieved; } =20 static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) { + struct folio *folio; + + folio =3D virt_to_folio(phys_to_virt(args->serialized_data)); + folio_put(folio); } =20 static const struct liveupdate_file_ops vfio_pci_liveupdate_file_ops =3D { @@ -139,6 +211,7 @@ static const struct liveupdate_file_ops vfio_pci_liveup= date_file_ops =3D { .unpreserve =3D vfio_pci_liveupdate_unpreserve, .freeze =3D vfio_pci_liveupdate_freeze, .retrieve =3D vfio_pci_liveupdate_retrieve, + .can_finish =3D vfio_pci_liveupdate_can_finish, .finish =3D vfio_pci_liveupdate_finish, .owner =3D THIS_MODULE, }; diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c index 9182dc46d73f..c5b5eb509474 100644 --- a/drivers/vfio/vfio_main.c +++ b/drivers/vfio/vfio_main.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -1706,6 +1707,18 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr= _t iova, void *data, } EXPORT_SYMBOL(vfio_dma_rw); =20 +struct vfio_device *vfio_find_device(const void *data, device_match_t matc= h) +{ + struct device *device; + + device =3D class_find_device(vfio.device_class, NULL, data, match); + if (!device) + return NULL; + + return container_of(device, struct vfio_device, device); +} +EXPORT_SYMBOL_GPL(vfio_find_device); + /* * Module/class support */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index f09da3bdf786..4e400a7219ea 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -413,4 +413,16 @@ int vfio_virqfd_enable(void *opaque, int (*handler)(vo= id *, void *), void vfio_virqfd_disable(struct virqfd **pvirqfd); void vfio_virqfd_flush_thread(struct virqfd **pvirqfd); =20 +#if IS_ENABLED(CONFIG_VFIO_DEVICE_CDEV) +int __vfio_device_fops_cdev_open(struct vfio_device *device, struct file *= filep); +#else +static inline int __vfio_device_fops_cdev_open(struct vfio_device *device, + struct file *filep) +{ + return -EOPNOTSUPP; +} +#endif /* IS_ENABLED(CONFIG_VFIO_DEVICE_CDEV) */ + +struct vfio_device *vfio_find_device(const void *data, device_match_t matc= h); + #endif /* VFIO_H */ --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B555634402F for ; Wed, 26 Nov 2025 19:36:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185784; cv=none; b=cAYYCLHlt1tWqFjRRveOqu9tejL3wm2wAkgUOqtkjLlCBJLfRQo/F0OqLWW8kI9WpUaEk6BkEoHGM5kd/SdVE+BM7s41VbiyVc3R3U5UJE/PabWzO7Bser4nkjlRyreUk5Lp5OvRZGrMrpJYls+oHPQMgERQ3fF5ELmHQZNwdhw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185784; c=relaxed/simple; bh=HzMrzWRCLtgSmquMMa2P4U7RJKBy8Y6HK2PaCjHpnGw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZpGKl2pb8kmF/cN2nFa64vRNWzShCF7Jv9SFN9pYtHHTjIxXxpMlW2sl4xyHS/8KdSw/d03q+SRyL3o/VDjg6PJTTWUx2zoI2OQdkauAYEz/CzCih2mKi8n3jSGu3joKLuI2dw0XlxxD5ZamvnMjdKazqxy8YwdY1wxX1HEIH0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=KjLgK782; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KjLgK782" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-297dde580c8so3707055ad.1 for ; Wed, 26 Nov 2025 11:36:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185782; x=1764790582; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Oq/yr6BWXlg1Xcrv4iWHhg15TfmkXzU0vPtqwC0SJKQ=; b=KjLgK782aOUBR9A8sLsTlj9ZVqmwzXa6sndJ6BVc6Ymg+Z4ej4B4qo8prYXdW0dCjW OaVzINZquF6F21gmQHkwqH7FTDx5GVAwNqcxlD6cIG8gFemTVA069qzvZLo3d2B0t3oR lVzxIsoS87jQufLANOOrun2IoZIcN10BiTeMLS/yIog8LC/T+z2wSb3i3jXxG4O53E3A WHg6Z0ijzn+XDpFIecUzw6/HeeCRldYnEgt5+Ni6MSGhUYeddVss04W79iI9sefyF6bT aPUX6RH3m+Llj4SaGM/TkDTVqcfclb85+vLb44i6uPKBYPCm000PSGv4MCw4b296TNKA OR4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185782; x=1764790582; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Oq/yr6BWXlg1Xcrv4iWHhg15TfmkXzU0vPtqwC0SJKQ=; b=onn5CvL3w88H9BxRQsBCt0JHiVkPoFq+QInCwFIq7zweU3wi/Brfvwv7iN5JTdOWVD DYr4ftNgFtsAprdPyJFVjDCrvnJR2Ch0L+P0EOHMuzYcxVYGUaFj0MFbkcTtg36p0TUv NmZcwl8rRjqY8Otp20Svb0Ndi20yyrLFaumnYevUN2dn/9Ta83uHSv1GB4/qOgGZupOA U4EfhZqsf0PZD78WiKRLQDIpYlb7/jMnS9tKF7jUCjrnbLqwF3yxjSmmjyWKBTikAL/W uCtUCCU3tVYFpPsv/ftQcE+Yjl0K4Z+rljeaavlMUgo+pVP6cNB4bqYrkn5ZroD7JNZU KvYQ== X-Forwarded-Encrypted: i=1; AJvYcCVmArsxg/On92yLbLtCkKDvKu7fnkv/OUKM6W2/JG7/dufLEx4SlkU7N566ufrHQKr/SirIq5aEHLLf+MA=@vger.kernel.org X-Gm-Message-State: AOJu0YzNkuej0s1BYVDtHJ9qBqmcdioFE599n6dRwQssJNijyZmIJONQ TScmp/M7BSoeWoG0ldLKNBDYltyhkB80hJGhaKcQjZnH16xHiY3jTE79j25tTNa3X1Tlcdmb3eG 5BBMbLB/BhDPL9Q== X-Google-Smtp-Source: AGHT+IFWOeNwF25aWnbYsfeGo4vdcfOTzNrl8hAriCkFnQsDtZekeoeRfF6D6GNT5dSECm+ci49u7/PdsBRstw== X-Received: from plblq5.prod.google.com ([2002:a17:903:1445:b0:268:4e0:9c09]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:dac3:b0:299:e031:16d with SMTP id d9443c01a7336-29bab148c3emr87405365ad.33.1764185782017; Wed, 26 Nov 2025 11:36:22 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:54 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-8-dmatlack@google.com> Subject: [PATCH 07/21] vfio/pci: Notify PCI subsystem about devices preserved across Live Update From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Notify the PCI subsystem about devices vfio-pci is preserving across Live Update by registering the vfio-pci liveupdate file handler with the PCI subsystem's FLB handler. Notably this will ensure that devices preserved through vfio-pci do not go through auto-probing in the next kernel (i.e. preventing them from being bound to a different driver). This also enables VFIO to detect that a device was preserved before userspace first retrieves the file from it, which will be used in subsequent commits. Signed-off-by: David Matlack --- drivers/vfio/pci/vfio_pci_liveupdate.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index b7451007fca4..7669c65bde17 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -64,6 +64,7 @@ static int vfio_pci_liveupdate_preserve(struct liveupdate= _file_op_args *args) if (err) goto error; =20 + pci_liveupdate_outgoing_preserve(pdev); args->serialized_data =3D virt_to_phys(ser); return 0; =20 @@ -75,8 +76,10 @@ static int vfio_pci_liveupdate_preserve(struct liveupdat= e_file_op_args *args) static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args = *args) { struct vfio_pci_core_device_ser *ser =3D phys_to_virt(args->serialized_da= ta); + struct vfio_device *device =3D vfio_device_from_file(args->file); struct folio *folio =3D virt_to_folio(ser); =20 + pci_liveupdate_outgoing_unpreserve(to_pci_dev(device->dev)); kho_unpreserve_folio(folio); folio_put(folio); } @@ -199,8 +202,11 @@ static bool vfio_pci_liveupdate_can_finish(struct live= update_file_op_args *args) =20 static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) { + struct vfio_device *device =3D vfio_device_from_file(args->file); struct folio *folio; =20 + pci_liveupdate_incoming_finish(to_pci_dev(device->dev)); + folio =3D virt_to_folio(phys_to_virt(args->serialized_data)); folio_put(folio); } @@ -223,10 +229,24 @@ static struct liveupdate_file_handler vfio_pci_liveup= date_fh =3D { =20 int __init vfio_pci_liveupdate_init(void) { + int ret; + if (!liveupdate_enabled()) return 0; =20 - return liveupdate_register_file_handler(&vfio_pci_liveupdate_fh); + ret =3D liveupdate_register_file_handler(&vfio_pci_liveupdate_fh); + if (ret) + return ret; + + ret =3D pci_liveupdate_register_fh(&vfio_pci_liveupdate_fh); + if (ret) + goto error; + + return 0; + +error: + liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); + return ret; } =20 void vfio_pci_liveupdate_cleanup(void) @@ -234,5 +254,6 @@ void vfio_pci_liveupdate_cleanup(void) if (!liveupdate_enabled()) return; =20 + WARN_ON_ONCE(pci_liveupdate_unregister_fh(&vfio_pci_liveupdate_fh)); liveupdate_unregister_file_handler(&vfio_pci_liveupdate_fh); } --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B30E340277 for ; Wed, 26 Nov 2025 19:36:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185786; cv=none; b=UKn6te0Wt5/R6K8TKpmjVjV6/3HmByydeOEWXnQvvPPDo577B3sXtxx+aIe6jA9bTKMfNvT+EtfmqlZsbJF619dFWJyOyCfxsrMZvxOI4jE8wI7Ucq9NT4IVCK/b+HlzyPdNk9goWLB3hCOK4bjA4CUh3INlLSK3pFXFxzzshMU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185786; c=relaxed/simple; bh=mCOKJ+2oazOul2zSOoTvvQYPlws6VrgIzEWD2SHP4h0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tGx5lZJsLJvzazhxkTo74hlHn6sZrF1FDPU400MKNJjfEg4DwEwz5EV57TyAyIAs4mP1JltGcIBQq5wQK4a0BzsmjIbprr1+kRtJK2lZ9Gm++Nq8sfgyWXqrknWZbHPT2razud/NckpauLX9PyWU1OldUa9GbuKbDnUxlyKCxEU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=leAra91s; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="leAra91s" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-34374bfbcccso78830a91.0 for ; Wed, 26 Nov 2025 11:36:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185784; x=1764790584; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=34dEI7XqQFY3PUssu08ewRf+hu3ICrVVHhgvFs2tITk=; b=leAra91seKzVgaVuc1yowBpyV16MZVEQ/3jryUfGEdiEb4Q3Ko2JxP/oPV93go6eOV rGzf0xj00ibva5m04HYWQMNtHdlfUI0xHJjaVKm7XiyOwnvsZJUgUAggVq3l6rkJOw1g 2zApUa/IIbAXWmSO9ccvS4EjN+D7sAxQDfq0pnyCf7kTfwzqdf2lrqu8DtPNujROkOve ARdeSCLlfpDROJmZ1Jw8Dcfk/xWjHx4es34lalz3/aGgqBWKLZC7F8D3x38RRxI2Yyho 4UPf5jeZHgg5JGUuf9j6ZGX/ZK40tbuih14vHz+uBcmy9R2nmAbajhlomhXH3gyfSlWJ LBiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185784; x=1764790584; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=34dEI7XqQFY3PUssu08ewRf+hu3ICrVVHhgvFs2tITk=; b=ZhHpMMdJ1a25vDQoY1SRIDP/myDK0B4Fr2orgGvLuV6uSGCTRQ7O+uWYbCY/a0ijEL b5eNf3MxPf0qLCcjD9lRzxBEqE11MqEZki1HFtRA6WbRrNGU2m6NZyntbHmz8d2+zpc4 l8aRLkcAkXYOO0PD2ZNx4ZXxNI/yuQhI7Q1OQ3ZJDXrDWXIc6J51HH+xz0P9Ov9/OE5t hkYMemYbANmhMruOVPUI8lTmvPV1s1Xk8OVg+vfIJIXTnOt1cVWZLojPTdk5WFmPuEW+ //Z8doExhaIeLp4ccEZf5vg3iR8WBiJZcjT7r6B0lyOe0ok0zbHOTIdi6nr7XSy0+LqL yHSg== X-Forwarded-Encrypted: i=1; AJvYcCWx0nHAJJGS+s4ri3Hd4qppSrXh4HV87zvX0+MDbtqe10kUCI3NnFWxZ42hkRTEPtxCxx81QDDHVGZp04s=@vger.kernel.org X-Gm-Message-State: AOJu0Yz11Boy7/1VB2ol7lUGviHF89LZ862rKIf8K6NEy8etpIZh0XYT vAZSNUD3mgAwNthrHNMBxXS0P+BXZZf+Q0EzSSM1lvOG2tIAdjR0qbKa6H1wS9fk7NHeQK/v20O JgBGrZkTOWBcOBQ== X-Google-Smtp-Source: AGHT+IEqIu3OlacY0Bm+LqAlpJoB8yetidaH8WntE+ttbQ3ZI60SDomuw3uasEM3kvlaRlv8+XaIAZZGkGEx9w== X-Received: from pfbfb24.prod.google.com ([2002:a05:6a00:2d98:b0:781:e8d:dd2b]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:6d96:b0:35e:6c3:c8d0 with SMTP id adf61e73a8af0-3637e0c5c31mr8915052637.54.1764185783558; Wed, 26 Nov 2025 11:36:23 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:55 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-9-dmatlack@google.com> Subject: [PATCH 08/21] vfio: Enforce preserved devices are retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Enforce that files for incoming (preserved by previous kernel) VFIO devices are retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD rather than by opening the corresponding VFIO character device or via VFIO_GROUP_GET_DEVICE_FD. Both of these methods would result in VFIO initializing the device without access to the preserved state of the device passed by the previous kernel. Signed-off-by: David Matlack --- drivers/vfio/device_cdev.c | 4 ++++ drivers/vfio/group.c | 9 +++++++++ include/linux/vfio.h | 11 +++++++++++ 3 files changed, 24 insertions(+) diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c index 0a6e972f322b..f1e54dcc04e7 100644 --- a/drivers/vfio/device_cdev.c +++ b/drivers/vfio/device_cdev.c @@ -57,6 +57,10 @@ int vfio_device_fops_cdev_open(struct inode *inode, stru= ct file *filep) struct vfio_device *device =3D container_of(inode->i_cdev, struct vfio_device, cdev); =20 + /* Device file must be retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD */ + if (vfio_liveupdate_incoming_is_preserved(device)) + return -EBUSY; + return __vfio_device_fops_cdev_open(device, filep); } =20 diff --git a/drivers/vfio/group.c b/drivers/vfio/group.c index c376a6279de0..93dbbf37ec4c 100644 --- a/drivers/vfio/group.c +++ b/drivers/vfio/group.c @@ -313,6 +313,15 @@ static int vfio_group_ioctl_get_device_fd(struct vfio_= group *group, if (IS_ERR(device)) return PTR_ERR(device); =20 + /* + * This device was preserved across a Live Update. Accessing it via + * VFIO_GROUP_GET_DEVICE_FD is not allowed. + */ + if (vfio_liveupdate_incoming_is_preserved(device)) { + ret =3D -EBUSY; + goto err_put_device; + } + fdno =3D get_unused_fd_flags(O_CLOEXEC); if (fdno < 0) { ret =3D fdno; diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 4e400a7219ea..040c2b2ee074 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -16,6 +16,7 @@ #include #include #include +#include =20 struct kvm; struct iommufd_ctx; @@ -425,4 +426,14 @@ static inline int __vfio_device_fops_cdev_open(struct = vfio_device *device, =20 struct vfio_device *vfio_find_device(const void *data, device_match_t matc= h); =20 +static inline bool vfio_liveupdate_incoming_is_preserved(struct vfio_devic= e *device) +{ + struct device *d =3D device->dev; + + if (dev_is_pci(d)) + return pci_liveupdate_incoming_is_preserved(to_pci_dev(d)); + + return false; +} + #endif /* VFIO_H */ --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B40B4343210 for ; Wed, 26 Nov 2025 19:36:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185787; cv=none; b=OIrKcjACiHMuKFKXWH9d17/D5hYYWSFUCOPywlAD32cpTkk9gMLeFxjpcRnxO1HfGf6E/tk51JxiQ5d0GlKQrKnXDJXvC+ah2GG/iCeof/h6Zta/MLs1KePVc+bX19Upu8Oq9a9GX6hW3G+HLfxVnREbYO6iAlXNL03nPbRNFf8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185787; c=relaxed/simple; bh=dYwaWyRIbjLSPJvPfwfi39/44Tve6NUTV8wbCVJRFAE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XjkE2+NXgXxmoOPyEBeYDcr7fMTP6JwBXgtO+rhv2agpMzNOgzt2MaeMsxu7AI+WaP3uFyf1cfGOmvmWLMXpiU9v1Nyi/Uw2JTxhHaKprS+zpODKQzDo3cn0wdblEtdrIfd7MXtcI8PluvIWCjF07gYEaJVoqso2cj+jC/n5mwA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=q4asNFnU; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="q4asNFnU" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2982dec5ccbso1690655ad.3 for ; Wed, 26 Nov 2025 11:36:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185785; x=1764790585; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=oupTdIgiJcobDgdLCKQ4bCK5hBmMGc7yRUjoOu6trk8=; b=q4asNFnUx0t6QIitaPgc/5GLpLQXvY3BXVGEPex4Bu52KS7bXAnIBQ8xm6c9ht6dt9 8GChiubJAM5zGLZI4WeOsIFLVFNucWneMjVUxodYmHjxJi1Qg8UQTKDKJ2C1DHA/yxTo roXPZ0lnlEGa7y2ekykT1ITMbHzFsxg1C7L35TFpNqrJOxgDxY+SU14bC+C6pb8hgOq7 jMFkyL0bDGbI5J0dRoSVZpQA3yr+buqGnRnJX7W6MNUz5sJciFAT/lfBPxGjjt15bFaR WAMC8IwrelECwS2OnPnfgThhaFUzwNmPF8JU68DvH9U3g5WHANnp85+lLI7NTdMDS1DG pINw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185785; x=1764790585; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oupTdIgiJcobDgdLCKQ4bCK5hBmMGc7yRUjoOu6trk8=; b=D27qk/4k/P43cJMUO5HwBRUNeMX+E+H/KXmE7j9gLTim7BCw9xH50pDH5jI09ShLDR z0Fvwea5y/DKtojNM/7A5XAtgevv+rVZrDyRyEJUHnYQzG2ZJo4Xv4zuMlBzNxLpQ/ol 0+0E6BIUwl4Z2bqMakPWIESH2H5BZOReRRDKyXIXdMyb+XbmhLXTeG5hc51q/cf0OREV e4CTcQkRS29RUAxOnbvdGOuT7Udrk1XcVqmCQpJqDRS4jxjQthvvp23eYJsRX09qdbtg gKtTKpQ1fKiMmOVIld1PNGy1RlZDx52j1D0ZV2s4CqrFSyB5mL2tTwucB75QDDCt8I6w 8a+g== X-Forwarded-Encrypted: i=1; AJvYcCVP1qL8Ye2ZQmSdcvC8HG0UKfq10ZsL9AqpoqTiS6LzKhjy91DiKEjwuWXR61lBr5KJWlGhnIbG0TeLeJg=@vger.kernel.org X-Gm-Message-State: AOJu0YwTkWbFR7TU3gR8q47G4w3d+N5KGISwUz/GGqDVREHqFGI+Yr88 ksEoJLi9DtUc0mDbUnKJWUo0vFGk4Vo9gRudebJ9u5De5IaEkRZx79KE05sV+Odb60IdheTDtMA nfsGfrZZ28t6tSQ== X-Google-Smtp-Source: AGHT+IF1dFRpdgEXNa+eADLtxGLANbMQ+v0Qp/LYDBPxS4oxJSeNtN05AVnZrSpiBB35n+k+Bh9kmPafil/o6A== X-Received: from pjbmf13.prod.google.com ([2002:a17:90b:184d:b0:341:c8a:bd40]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:ace:b0:295:350a:f9b8 with SMTP id d9443c01a7336-29b6c685132mr215591725ad.29.1764185785065; Wed, 26 Nov 2025 11:36:25 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:56 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-10-dmatlack@google.com> Subject: [PATCH 09/21] vfio/pci: Store Live Update state in struct vfio_pci_core_device From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Stash a pointer to a device's Live Updated state in struct vfio_pci_core_device. This will enable subsequent commits to use the preserved state when enabling the device. To enable VFIO to safely access this pointer during device enablement, require that the device is fully enabled before returning true from can_finish(). Signed-off-by: David Matlack --- drivers/vfio/pci/vfio_pci_core.c | 1 + drivers/vfio/pci/vfio_pci_liveupdate.c | 20 +++++++++++++++++++- include/linux/vfio_pci_core.h | 6 ++++++ 3 files changed, 26 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 7dcf5439dedc..b09fe0993e04 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -536,6 +536,7 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *v= dev) if (!vfio_vga_disabled() && vfio_pci_is_vga(pdev)) vdev->has_vga =3D true; =20 + vdev->liveupdate_state =3D NULL; =20 return 0; =20 diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 7669c65bde17..0fb29bd3ae3b 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -145,6 +145,7 @@ static int match_device(struct device *dev, const void = *arg) static int vfio_pci_liveupdate_retrieve(struct liveupdate_file_op_args *ar= gs) { struct vfio_pci_core_device_ser *ser; + struct vfio_pci_core_device *vdev; struct vfio_device *device; struct folio *folio; struct file *file; @@ -186,6 +187,9 @@ static int vfio_pci_liveupdate_retrieve(struct liveupda= te_file_op_args *args) goto out; } =20 + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + vdev->liveupdate_state =3D ser; + args->file =3D file; =20 out: @@ -197,7 +201,21 @@ static int vfio_pci_liveupdate_retrieve(struct liveupd= ate_file_op_args *args) =20 static bool vfio_pci_liveupdate_can_finish(struct liveupdate_file_op_args = *args) { - return args->retrieved; + struct vfio_pci_core_device *vdev; + struct vfio_device *device; + + if (!args->retrieved) + return false; + + device =3D vfio_device_from_file(args->file); + vdev =3D container_of(device, struct vfio_pci_core_device, vdev); + + /* + * Ensure VFIO is done using vdev->liveupdate_state, which means its + * safe for vfio_pci_liveupdate_finish() to free it. + */ + guard(mutex)(&device->dev_set->lock); + return !vdev->liveupdate_state; } =20 static void vfio_pci_liveupdate_finish(struct liveupdate_file_op_args *arg= s) diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index f541044e42a2..56ff6452562d 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -94,6 +94,12 @@ struct vfio_pci_core_device { struct vfio_pci_core_device *sriov_pf_core_dev; struct notifier_block nb; struct rw_semaphore memory_lock; + + /* + * State passed by the previous kernel during a Live Update. Only + * safe to access when first opening the device. + */ + struct vfio_pci_core_device_ser *liveupdate_state; }; =20 /* Will be exported for vfio pci drivers usage */ --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BB0E345CD6 for ; Wed, 26 Nov 2025 19:36:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185789; cv=none; b=sRIqfE9WSxOXmSb2NAv5srGyVuy4Iz1UvK9BYKauhKWZcPzGLlktgsBJaciciU8Vn7fwBcTLsK8JC4k5sTgYViZt5MyAfBFaq+/XJwL2rMc9dS0YHUTyvx4jOTZmclsNIxOUvq/wHBI8YUje6G0hkPwMALvDTFufzd5z5ZVV89w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185789; c=relaxed/simple; bh=J7pVIwlaC2a90b7nwtyp8yiJuohGfw3mOEce/82UxtI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qttdLdrJziUikYSdZfGsZqTlVuDIvHuMtUS8+9EaCDfes4mVyozUijMYXZ/DAY2yrb3xf5s7mdYp9fTzkCkHs5zWFhtdPlE9gyAK8vavshXrw9dQozyT1m9fm+qOX9ZQrR46maYauVNcBwCsRwCeiSmyCj86DHpd4HWCRjzi7Po= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dwVL6vOD; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dwVL6vOD" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-341616a6fb7so95820a91.0 for ; Wed, 26 Nov 2025 11:36:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185786; x=1764790586; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=qxDgVkCfvUIBjknMGYFc0YAO25izn9/Vb7Fr5WcLn8I=; b=dwVL6vODunu314pHy+b2J1h+eW4lRroED9tEH94HXDe+9H+ZEr68PVqSE2ZLm9Lo02 aAggeJRVjcR+sNsP6v5A2InKO889aGBQowHXlyVUtRiBAAkt37Wn3pz4vXAcLk2qzdje F9Ziizhs81Cw/8DFC8NJVZFWQjUZ/JWSXMvusLWp7kg+sI4ljYZCbUqSaJ0yPWk0kign qi5CB6ugbl4EVTJhtLLHkZGOQEqnt4YsbRPzqe/XXiyqCHNQICn4yALQjo6fSGoj0OH1 S3vp/Uv2qdT7dpWF26PPBnxXKVLXr4wTKyW4Xm7I+QZfrfwT6CY6OQz9D0xCl+tz3Ith Q5bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185786; x=1764790586; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qxDgVkCfvUIBjknMGYFc0YAO25izn9/Vb7Fr5WcLn8I=; b=VCEyBlenWP8zwa0VsOGBRugyk+psFJGg2+OJ8Ga+fEjlgeJrpH4bhl2dpbnYLRyr0E oCg9TJMup6i7RAxHjtIxcOnId4HpPWCuMWrmsvT3FqWg72vBPJRONxQV3N57zQo1AZWK Kv9CBvxz9BXqrR4nPWW5ycSZ4ZhK8EukM18t1ebC2MOgD1qXtAviesljSbRgpLd7ZYSA sjJwTF6wnRaYq3j6w0jkPv1xi2kRWsEGtSgj3RMFKFhzrrw6SyJICEhzUa8TrTZxJnmt R9MdCEFv7pf4mcxJjnMy70LQSBX4VqUSEKzGLLkBccFUdtPTSrRLsoYfoJDn91EvKvRM fQbg== X-Forwarded-Encrypted: i=1; AJvYcCU/ERhzKJi8mvq9ml6W0d1GHIDepVse5XyYQ+jlaTe3Omx42maFLWeidx5meFNTgcKzK0XE1kqJA8Zl65o=@vger.kernel.org X-Gm-Message-State: AOJu0YzMeXeMVotMpLaQmYdQYQX7ZfuWuyHdhuuFeXAs4uUq7+Q7ufbJ UtbH2ggDeqOVW6O8ILpufM9WfEtpyaowpeU5LSl73Oq4cXrxqTkVjHBDT7ZIm+Yg03X2X1L3bAC 3RCXRZkZaG4hTMQ== X-Google-Smtp-Source: AGHT+IGz6vSeW4fFPlsbH/YbzCbOjEAX4lyHpEhDzDf7WPVwipOugVcvT745U9YgzvArnPjB+ggooe0srOWMJQ== X-Received: from pjpo2.prod.google.com ([2002:a17:90a:9f82:b0:340:5c38:986c]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:394c:b0:340:54a1:d703 with SMTP id 98e67ed59e1d1-34733f5d318mr22023223a91.35.1764185786500; Wed, 26 Nov 2025 11:36:26 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:57 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-11-dmatlack@google.com> Subject: [PATCH 10/21] vfio/pci: Skip reset of preserved device after Live Update From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Do not reset the device when a Live Update preserved vfio-pci device is retrieved and first enabled. vfio_pci_liveupdate_freeze() guarantees the device is reset prior to Live Update, so there's no reason to reset it again after Live Update. Since VFIO normally uses the initial reset to detect if the device supports function resets, pass that from the previous kernel via struct vfio_pci_core_dev_ser. Signed-off-by: Vipin Sharma Signed-off-by: David Matlack --- drivers/vfio/pci/vfio_pci_core.c | 22 +++++++++++++++++----- drivers/vfio/pci/vfio_pci_liveupdate.c | 1 + include/linux/kho/abi/vfio_pci.h | 2 ++ include/linux/vfio_pci_core.h | 1 + 4 files changed, 21 insertions(+), 5 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index b09fe0993e04..c3b30f08a788 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -482,12 +482,24 @@ int vfio_pci_core_enable(struct vfio_pci_core_device = *vdev) if (ret) goto out_power; =20 - /* If reset fails because of the device lock, fail this path entirely */ - ret =3D pci_try_reset_function(pdev); - if (ret =3D=3D -EAGAIN) - goto out_disable_device; + if (vdev->liveupdate_state) { + /* + * This device was handed off by vfio-pci from a previous kernel + * via Live Update, so it does not need to be reset. + */ + vdev->reset_works =3D vdev->liveupdate_state->reset_works; + } else { + /* + * If reset fails because of the device lock, fail this path + * entirely. + */ + ret =3D pci_try_reset_function(pdev); + if (ret =3D=3D -EAGAIN) + goto out_disable_device; + + vdev->reset_works =3D !ret; + } =20 - vdev->reset_works =3D !ret; pci_save_state(pdev); vdev->pci_saved_state =3D pci_store_saved_state(pdev); if (!vdev->pci_saved_state) diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio= _pci_liveupdate.c index 0fb29bd3ae3b..bcaf9de8a823 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -59,6 +59,7 @@ static int vfio_pci_liveupdate_preserve(struct liveupdate= _file_op_args *args) =20 ser->bdf =3D pci_dev_id(pdev); ser->domain =3D pci_domain_nr(pdev->bus); + ser->reset_works =3D vdev->reset_works; =20 err =3D kho_preserve_folio(folio); if (err) diff --git a/include/linux/kho/abi/vfio_pci.h b/include/linux/kho/abi/vfio_= pci.h index 9bf58a2f3820..6c3d3c6dfc09 100644 --- a/include/linux/kho/abi/vfio_pci.h +++ b/include/linux/kho/abi/vfio_pci.h @@ -34,10 +34,12 @@ * * @bdf: The device's PCI bus, device, and function number. * @domain: The device's PCI domain number (segment). + * @reset_works: Non-zero if the device supports function resets. */ struct vfio_pci_core_device_ser { u16 bdf; u16 domain; + u8 reset_works; } __packed; =20 #endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */ diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 56ff6452562d..3421721a1615 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -15,6 +15,7 @@ #include #include #include +#include =20 #ifndef VFIO_PCI_CORE_H #define VFIO_PCI_CORE_H --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D093033FE12 for ; Wed, 26 Nov 2025 19:36:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185791; cv=none; b=HunPt/447eeVMKkUHHiMTHpN9ydlHpC2SZOXoSVWUCdUYs7QNmfyH/bVoZGnmA2iRQetRa+QhLhKFOduBVoZAYu/TNa+nCnaDnZD667ajD6PPnxePBwdLE2ssAN9nxgvho+2VwzNImkCGv4Wzgru5djZr6zSVE9nHURY7FdpqPc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185791; c=relaxed/simple; bh=xQdAIox14rL920kyxPeD7CGr8MVH1OzKyBlfF0iKpfE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WDsXqDpjXmk81EOJkLXwSKvVWvA0mJeNq9w33qrJBLU6zxsnMfKZ3JqnU5ktYSuhFwDpH0n++tCYAy1fY0eDAkHzOvv+8Q2tQLr5Jdr78ZbncDLSkWJdyl/ozVi7A+mFnEVuBOOadppl02p6HqxQ01l1KYUrUdAjYSIJI95ijJw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PVd+SVD9; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PVd+SVD9" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7b89ee2c1a4so76728b3a.2 for ; Wed, 26 Nov 2025 11:36:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185788; x=1764790588; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=il8MDXrCBxc/QPsV+lp75cRRTs/ibymz6EtH1r9GxH0=; b=PVd+SVD9/FIoYKdXiKq+36U1PqFFPos0JvjwDGRwbIgOKzCNnaWjPJ507HBKqbVjuu OBvdES4816PnOAe4b06Tu7oQtDG/WrN2jqpbMx+Jc8x3H9T/MSb3t0IACxj9NMydtiT8 qvjbwcDabu8DHvdkNGz72yUZVuEbrjIzHPjgTaZ7SnR6pyBb7jUONSiOEc0JvEmAkS6+ BSz8d8Hz5Gky7joHDFJ6TCEP80chvEl/dgz1Tg9TLCOaxMqTRmKKxqDfj2EnxOSqcz7a fjKIVsh/ZyLSGLtAvuDmu1D6/zvBAr3tL6jQ7PqwlXFivfDU8GuLsZOIC4jgfIl8ZrgB IiIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185788; x=1764790588; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=il8MDXrCBxc/QPsV+lp75cRRTs/ibymz6EtH1r9GxH0=; b=xRINr6OfSzN0TJXeUuSC2oHq1jABxY12rWLeGZoYVPG9WlADkd+qCDh2bKNYIpabzz WBe4L5o0FIwUxBhT+SprAZ0XCbFrj72Mo0SHS0rf97EZV4gG0Gt3hRg8ux9g0wUviNZv 0F0Clv861kfG8eNTGT+Ljj8QXqpfSY01swrZneIRm2CW+Q59CJRJUTyFAj71Ddf6Rk2b nkW6pXyK9B6cLR1KOvkWnLDstziIpvvxc7G22ZJmbB8hSor+hqgUveaC4Im1eoP1+/Dc NoaS0BWSiDE59kEsA0plOI4O11f4xE/PZndD9Es7N9vOSqX8QjMLFFuW+cboUT+uV4Bu 2uwQ== X-Forwarded-Encrypted: i=1; AJvYcCUc0CH5+FDmqIn4Swt/8ZGIUS1yKGSLIguj3+39j8nehfM19vsqubhw+9t1XNzm5zzrOocdlNG/Jg3jXNM=@vger.kernel.org X-Gm-Message-State: AOJu0Yxzm5M4oWb9A2PhruThmWSAj5trz6wRUaQvMO8KFXDYB0iZ1KVE OslAUfHfWw8FesgZELZmMVlB3UHDMnrNT3BTGUJZf8wLLV5EmuaLkZBPQclPEdxxMsy0VrzQ2r7 R9mTm9CiX0piZ0g== X-Google-Smtp-Source: AGHT+IFjRkYzrHjH75Y+Axw4RUqwa3zdxTT3QVQfwdPfSsZ2PKB+LzQLV1cCLFLD84k+auOPTtZD5gnENKFtpA== X-Received: from pfblo2.prod.google.com ([2002:a05:6a00:3d02:b0:7c5:ca53:260e]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1812:b0:7b8:8d43:fcd2 with SMTP id d2e1a72fcca58-7c58c7a883emr24782001b3a.14.1764185787996; Wed, 26 Nov 2025 11:36:27 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:58 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-12-dmatlack@google.com> Subject: [PATCH 11/21] selftests/liveupdate: Move luo_test_utils.* into a reusable library From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Move luo_test_utils.[ch] into a lib/ directory and pull the rules to build them out into a separate make script. This will enable these utilities to be also built by and used within other selftests (such as VFIO) in subsequent commits. No functional change intended. Signed-off-by: Vipin Sharma Co-Developed-by: David Matlack Signed-off-by: David Matlack --- tools/testing/selftests/liveupdate/.gitignore | 1 + tools/testing/selftests/liveupdate/Makefile | 14 ++++--------- .../include/libliveupdate.h} | 8 ++++---- .../selftests/liveupdate/lib/libliveupdate.mk | 20 +++++++++++++++++++ .../{luo_test_utils.c =3D> lib/liveupdate.c} | 2 +- .../selftests/liveupdate/luo_kexec_simple.c | 2 +- .../selftests/liveupdate/luo_multi_session.c | 2 +- 7 files changed, 32 insertions(+), 17 deletions(-) rename tools/testing/selftests/liveupdate/{luo_test_utils.h =3D> lib/inclu= de/libliveupdate.h} (87%) create mode 100644 tools/testing/selftests/liveupdate/lib/libliveupdate.mk rename tools/testing/selftests/liveupdate/{luo_test_utils.c =3D> lib/liveu= pdate.c} (99%) diff --git a/tools/testing/selftests/liveupdate/.gitignore b/tools/testing/= selftests/liveupdate/.gitignore index 661827083ab6..18a0c7036cf3 100644 --- a/tools/testing/selftests/liveupdate/.gitignore +++ b/tools/testing/selftests/liveupdate/.gitignore @@ -3,6 +3,7 @@ !/**/ !*.c !*.h +!*.mk !*.sh !.gitignore !config diff --git a/tools/testing/selftests/liveupdate/Makefile b/tools/testing/se= lftests/liveupdate/Makefile index 080754787ede..a060cc21f27f 100644 --- a/tools/testing/selftests/liveupdate/Makefile +++ b/tools/testing/selftests/liveupdate/Makefile @@ -1,7 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only =20 -LIB_C +=3D luo_test_utils.c - TEST_GEN_PROGS +=3D liveupdate =20 TEST_GEN_PROGS_EXTENDED +=3D luo_kexec_simple @@ -10,25 +8,21 @@ TEST_GEN_PROGS_EXTENDED +=3D luo_multi_session TEST_FILES +=3D do_kexec.sh =20 include ../lib.mk +include lib/libliveupdate.mk =20 CFLAGS +=3D $(KHDR_INCLUDES) CFLAGS +=3D -Wall -O2 -Wno-unused-function CFLAGS +=3D -MD =20 -LIB_O :=3D $(patsubst %.c, $(OUTPUT)/%.o, $(LIB_C)) TEST_O :=3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) TEST_O +=3D $(patsubst %, %.o, $(TEST_GEN_PROGS_EXTENDED)) =20 -TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(LIB_O)) +TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(LIBLIVEUPDATE_O)) TEST_DEP_FILES +=3D $(patsubst %.o, %.d, $(TEST_O)) -include $(TEST_DEP_FILES) =20 -$(LIB_O): $(OUTPUT)/%.o: %.c - $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@ - -$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/%: %.o $(LIB_O) - $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIB_O) $(LDLIB= S) -o $@ +$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/%: %.o $(LIBLIVEUP= DATE_O) + $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIBLIVEUPDATE_= O) $(LDLIBS) -o $@ =20 -EXTRA_CLEAN +=3D $(LIB_O) EXTRA_CLEAN +=3D $(TEST_O) EXTRA_CLEAN +=3D $(TEST_DEP_FILES) diff --git a/tools/testing/selftests/liveupdate/luo_test_utils.h b/tools/te= sting/selftests/liveupdate/lib/include/libliveupdate.h similarity index 87% rename from tools/testing/selftests/liveupdate/luo_test_utils.h rename to tools/testing/selftests/liveupdate/lib/include/libliveupdate.h index 90099bf49577..4390a2737930 100644 --- a/tools/testing/selftests/liveupdate/luo_test_utils.h +++ b/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h @@ -7,13 +7,13 @@ * Utility functions for LUO kselftests. */ =20 -#ifndef LUO_TEST_UTILS_H -#define LUO_TEST_UTILS_H +#ifndef SELFTESTS_LIVEUPDATE_LIB_LIVEUPDATE_H +#define SELFTESTS_LIVEUPDATE_LIB_LIVEUPDATE_H =20 #include #include #include -#include "../kselftest.h" +#include "../../../kselftest.h" =20 #define LUO_DEVICE "/dev/liveupdate" =20 @@ -41,4 +41,4 @@ typedef void (*luo_test_stage2_fn)(int luo_fd, int state_= session_fd); int luo_test(int argc, char *argv[], const char *state_session_name, luo_test_stage1_fn stage1, luo_test_stage2_fn stage2); =20 -#endif /* LUO_TEST_UTILS_H */ +#endif /* SELFTESTS_LIVEUPDATE_LIB_LIVEUPDATE_H */ diff --git a/tools/testing/selftests/liveupdate/lib/libliveupdate.mk b/tool= s/testing/selftests/liveupdate/lib/libliveupdate.mk new file mode 100644 index 000000000000..fffd95b085b6 --- /dev/null +++ b/tools/testing/selftests/liveupdate/lib/libliveupdate.mk @@ -0,0 +1,20 @@ +include $(top_srcdir)/scripts/subarch.include +ARCH ?=3D $(SUBARCH) + +LIBLIVEUPDATE_SRCDIR :=3D $(selfdir)/liveupdate/lib + +LIBLIVEUPDATE_C :=3D liveupdate.c + +LIBLIVEUPDATE_OUTPUT :=3D $(OUTPUT)/libliveupdate + +LIBLIVEUPDATE_O :=3D $(patsubst %.c, $(LIBLIVEUPDATE_OUTPUT)/%.o, $(LIBLIV= EUPDATE_C)) + +LIBLIVEUPDATE_O_DIRS :=3D $(shell dirname $(LIBLIVEUPDATE_O) | uniq) +$(shell mkdir -p $(LIBLIVEUPDATE_O_DIRS)) + +CFLAGS +=3D -I$(LIBLIVEUPDATE_SRCDIR)/include + +$(LIBLIVEUPDATE_O): $(LIBLIVEUPDATE_OUTPUT)/%.o : $(LIBLIVEUPDATE_SRCDIR)/= %.c + $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@ + +EXTRA_CLEAN +=3D $(LIBLIVEUPDATE_OUTPUT) diff --git a/tools/testing/selftests/liveupdate/luo_test_utils.c b/tools/te= sting/selftests/liveupdate/lib/liveupdate.c similarity index 99% rename from tools/testing/selftests/liveupdate/luo_test_utils.c rename to tools/testing/selftests/liveupdate/lib/liveupdate.c index 3c8721c505df..60121873f685 100644 --- a/tools/testing/selftests/liveupdate/luo_test_utils.c +++ b/tools/testing/selftests/liveupdate/lib/liveupdate.c @@ -21,7 +21,7 @@ #include #include =20 -#include "luo_test_utils.h" +#include =20 int luo_open_device(void) { diff --git a/tools/testing/selftests/liveupdate/luo_kexec_simple.c b/tools/= testing/selftests/liveupdate/luo_kexec_simple.c index d7ac1f3dc4cb..786ac93b9ae3 100644 --- a/tools/testing/selftests/liveupdate/luo_kexec_simple.c +++ b/tools/testing/selftests/liveupdate/luo_kexec_simple.c @@ -8,7 +8,7 @@ * across a single kexec reboot. */ =20 -#include "luo_test_utils.h" +#include =20 #define TEST_SESSION_NAME "test-session" #define TEST_MEMFD_TOKEN 0x1A diff --git a/tools/testing/selftests/liveupdate/luo_multi_session.c b/tools= /testing/selftests/liveupdate/luo_multi_session.c index 0ee2d795beef..aac24a5f5ce3 100644 --- a/tools/testing/selftests/liveupdate/luo_multi_session.c +++ b/tools/testing/selftests/liveupdate/luo_multi_session.c @@ -9,7 +9,7 @@ * files. */ =20 -#include "luo_test_utils.h" +#include =20 #define SESSION_EMPTY_1 "multi-test-empty-1" #define SESSION_EMPTY_2 "multi-test-empty-2" --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B5CF34677E for ; Wed, 26 Nov 2025 19:36:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185792; cv=none; b=FPJMs0aMqBjfIJLy9Dqu1p6B5nfkfkO4tArYTcPUu9Xi7Mb9nSKZOskjbi9/nEoVOTxy9fE13jAdsyG0/Ts+ObnKalW7q5m3boEkKFptpIzW6URQykTCatwwi53I7dx9XdIpd/oYIc4QLEQXmq1nTapJfbZAmy5gM6Dv9A1zBak= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185792; c=relaxed/simple; bh=AYgj+wirCLMCRR/8ot0I5xWDdqgGP4Qyh8BZEo8RH9k=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Bh5H/94CgTSKlWctZXFhEuk8lg2Pny+sJVvNhVtgnZqh+LH2Gen23h+KZno81X2vMYDQPE64vjYWycwGzhAfe6cvwwEWJa5pXcFmkipS4CXb/wgJboyETiHz6/1XpJMEHJGax5mtdgOSQxL+RUUxpuH5tzG9+hlQ6vHElnqlQlc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=F4xIFy1Z; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="F4xIFy1Z" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-345b4f2a8e8so120015a91.1 for ; Wed, 26 Nov 2025 11:36:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185790; x=1764790590; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hLCqoeSsQabfDeNa/xBdJZvFpiy30zTE4mNpE8T41QE=; b=F4xIFy1ZBSz9JKv+L6mOsKTCEVlxIOSesTCOjpH7sntxuFhsx7YANBgoHpAp4aI30g 0tTPgmJ5B7wp8f4GljZejz766K2u8Fb5vckdtbIrYYYCSuAsQMwySlxtuGw6kXNTc69S bYYRUutNbdGAio16YM4Ehv9UXcpeSOGbIK8sg4+WBT5Ap/etNwDG54soqW/xHdGTKNvT a2MhtYjCRcy189Bd+HGhGCAepLgcmVo3gOQ+Y4qrjJDcTs7r77GJcrrWLlzqcqKi+dcj HJEW9kf8aMpR4KDDuZkBhDMFQV1zFpks3I4/rHRMbXeflBUewbTM08i9ZeRoX4E4eWK/ ypow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185790; x=1764790590; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hLCqoeSsQabfDeNa/xBdJZvFpiy30zTE4mNpE8T41QE=; b=oX4puoCxdyHJgWeIGAMFmx/AspXVR/zs1BwLLbA+bPw0RUNgf5jFy2PXqA6WBuww3k 0UXRLs6d34Wvj0YmMhZJyBXgiNRJ8LNy/ru0PrODbIYSveU24T3O346NIi59b0Q9HYy2 5A32k1ToNX91HIYcuGp56SE/U8EMfeI697lpYh/8iu6HD7/0MNJCdqR5NHp8yCX7hDan oztMtECE4P504yqU4UEj7exoWS3xdHXhoOhugJYZPCVEBAkK9GwUo4L69HUz3RURS6pl 88GFb9hbA5o499mOpnDNUsZqUHK6A4+9/mgeh4PZa/KedZFxUMN1XkxCt7HUqUp/UX55 RIaQ== X-Forwarded-Encrypted: i=1; AJvYcCVHKv6v9EZ7mpJh1346cCj8RNg9MWhastjpDhwO58Op62qvpMCPVHGwrQluWcx8u7PQHmckFbhPKGm2Sw4=@vger.kernel.org X-Gm-Message-State: AOJu0YyZf6n034vCc0eDioNxvsW7H1t60nA+lEDtsTw9Q/yiTXV4R+0a 2ZVtqwaVSM/0dwM8umPo+BQBfKEa6rsGfXJfbhGkLS2HQScG5oMgODTLugy9W5PVKvKx4pOANYF /3nTUi1NfKcwwkQ== X-Google-Smtp-Source: AGHT+IGR5imVc4Gm8Q9WhaafA0U0GgE9yqjy//tdxegzmSxDm6Br74h7sG3/6ABHdOqP7Rab5/mVHLJp6Z1KeA== X-Received: from pjbsw4.prod.google.com ([2002:a17:90b:2c84:b0:32b:58d1:a610]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:17d1:b0:313:1c7b:fc62 with SMTP id 98e67ed59e1d1-34733f2d167mr18887345a91.22.1764185789612; Wed, 26 Nov 2025 11:36:29 -0800 (PST) Date: Wed, 26 Nov 2025 19:35:59 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-13-dmatlack@google.com> Subject: [PATCH 12/21] selftests/liveupdate: Add helpers to preserve/retrieve FDs From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Add helper functions to preserve and retrieve file descriptors from an LUO session. These will be used be used in subsequent commits to preserve FDs other than memfd. No functional change intended. Signed-off-by: Vipin Sharma Co-Developed-by: David Matlack Signed-off-by: David Matlack --- .../liveupdate/lib/include/libliveupdate.h | 3 ++ .../selftests/liveupdate/lib/liveupdate.c | 41 +++++++++++++++---- 2 files changed, 35 insertions(+), 9 deletions(-) diff --git a/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h= b/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h index 4390a2737930..4c93d043d2b3 100644 --- a/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h +++ b/tools/testing/selftests/liveupdate/lib/include/libliveupdate.h @@ -26,6 +26,9 @@ int luo_create_session(int luo_fd, const char *name); int luo_retrieve_session(int luo_fd, const char *name); int luo_session_finish(int session_fd); =20 +int luo_session_preserve_fd(int session_fd, int fd, int token); +int luo_session_retrieve_fd(int session_fd, int token); + int create_and_preserve_memfd(int session_fd, int token, const char *data); int restore_and_verify_memfd(int session_fd, int token, const char *expect= ed_data); =20 diff --git a/tools/testing/selftests/liveupdate/lib/liveupdate.c b/tools/te= sting/selftests/liveupdate/lib/liveupdate.c index 60121873f685..9bf4f16ca0a4 100644 --- a/tools/testing/selftests/liveupdate/lib/liveupdate.c +++ b/tools/testing/selftests/liveupdate/lib/liveupdate.c @@ -54,9 +54,35 @@ int luo_retrieve_session(int luo_fd, const char *name) return arg.fd; } =20 +int luo_session_preserve_fd(int session_fd, int fd, int token) +{ + struct liveupdate_session_preserve_fd arg =3D { + .size =3D sizeof(arg), + .fd =3D fd, + .token =3D token, + }; + + if (ioctl(session_fd, LIVEUPDATE_SESSION_PRESERVE_FD, &arg)) + return -errno; + + return 0; +} + +int luo_session_retrieve_fd(int session_fd, int token) +{ + struct liveupdate_session_retrieve_fd arg =3D { + .size =3D sizeof(arg), + .token =3D token, + }; + + if (ioctl(session_fd, LIVEUPDATE_SESSION_RETRIEVE_FD, &arg)) + return -errno; + + return arg.fd; +} + int create_and_preserve_memfd(int session_fd, int token, const char *data) { - struct liveupdate_session_preserve_fd arg =3D { .size =3D sizeof(arg) }; long page_size =3D sysconf(_SC_PAGE_SIZE); void *map =3D MAP_FAILED; int mfd =3D -1, ret =3D -1; @@ -75,9 +101,8 @@ int create_and_preserve_memfd(int session_fd, int token,= const char *data) snprintf(map, page_size, "%s", data); munmap(map, page_size); =20 - arg.fd =3D mfd; - arg.token =3D token; - if (ioctl(session_fd, LIVEUPDATE_SESSION_PRESERVE_FD, &arg) < 0) + ret =3D luo_session_preserve_fd(session_fd, mfd, token); + if (ret) goto out; =20 ret =3D 0; @@ -92,15 +117,13 @@ int create_and_preserve_memfd(int session_fd, int toke= n, const char *data) int restore_and_verify_memfd(int session_fd, int token, const char *expected_data) { - struct liveupdate_session_retrieve_fd arg =3D { .size =3D sizeof(arg) }; long page_size =3D sysconf(_SC_PAGE_SIZE); void *map =3D MAP_FAILED; int mfd =3D -1, ret =3D -1; =20 - arg.token =3D token; - if (ioctl(session_fd, LIVEUPDATE_SESSION_RETRIEVE_FD, &arg) < 0) - return -errno; - mfd =3D arg.fd; + mfd =3D luo_session_retrieve_fd(session_fd, token); + if (mfd < 0) + return mfd; =20 map =3D mmap(NULL, page_size, PROT_READ, MAP_SHARED, mfd, 0); if (map =3D=3D MAP_FAILED) --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C582341045 for ; Wed, 26 Nov 2025 19:36:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185794; cv=none; b=pJXA3W0Hsa0nb3euk4bQFBEzFaBBDD0eveLBuUHNdfGhyWc+Jp5nEn2IswS0P8dp5Lvtwro2SWzYnSL91Iy6pe06g0VKBfH/udWMY3SqrHnUXftKemtYWNRWMpbJklL7tLQA8lsYZ7x7cf7OfqAFfYTLCZlIpydMA9b+cACwFKI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185794; c=relaxed/simple; bh=2C91XLgizRzxrlYCyNq+ckn1edftrGddg/9ARO85DKg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=i4uxKztA/6aAf5JQ/ejQHparT4orCFu8HSgradHMUXIT0TiHEFqDBAd0wDw34JGeqTkxcRJsjdcmw1wlaMCODsfUCWpo+TJPUFExa6eRDtQddW2Mmp1iRdeT2PZ44qbshIdZR4uWzYMxaSWZnxSsXFQ2sCo9T7uLfoA2r0izWsg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=c7OdERCp; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="c7OdERCp" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-3439fe6229aso83597a91.1 for ; Wed, 26 Nov 2025 11:36:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185791; x=1764790591; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+RPQzdHtB0lCBw9QZTDQVrKsYUywm29Ltc1imWZcj7k=; b=c7OdERCp4K4Lst3D604HWJ1Hdu08QBB7S6QuQtNhXD22a/ZdUDs5lj4EygqUPNnxYQ ZiLWizqSKzZoR1Vb04qx4wrTV/HNWXXyL14cm3ndPgHC9OX0+br9ROhT4YPLjeDJJgHu WpHPKJapZJbi1tcyWCGoFw8Z4+m9ESspCNL0TGsjFVnXr04XuhUChKT1+memJo2AQuic Zj3ExfPQ9Y9t5g520Hfjsyvp/evzGcy1r73mBSnsUwM5P51wc6XdiVhAwgq5FwOM7sE2 Dcn+5ddwbsy3vpur5Zo3pSby7cSbO6r4KKmJI84qz7zCBJZxEEZwlux+SlymgRQ0l523 Xcig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185791; x=1764790591; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+RPQzdHtB0lCBw9QZTDQVrKsYUywm29Ltc1imWZcj7k=; b=E8wvHh9JsG5lfCM7EEPHR8g+wruRf2kzKBASYIoNS8bem2AFmr/hpGbxhRlFDSkZhm Aal2A0LdsBDGopAVjPZqeJ4PW2r2YsCSlHJp4wpbmsPThXYRIUIMuhq6ITS63Zbdnb3d 1MRY6789KEx3xZv7gZH7EjIrw0s/EdUcPTS3e4zfTad7SR/r7gukpFnH59a+n9OxwNj5 YNY8pGWKEYi2lnM6XtvX19YORWJrqxT9feo0hCySYcQ460xDSONrBd1bo7S5pSVhj9// KZ0dG+Urxj/zyu6V+BXYb9YXQ3NdxXjP/jhMFH7ioaLNUamQph6IaN7ch4Zyg1vEWh0D EhBA== X-Forwarded-Encrypted: i=1; AJvYcCWMEUQ1hRXMd0LnSZlWGiGfWuwS5JbFBIDQIFIpOj2Q2LQywAmEX9EmMxiohuI7EjnHgjQTSEOOW/qO/3Q=@vger.kernel.org X-Gm-Message-State: AOJu0YxTqFuq/55vNPk1ppB/pjmqTevfdkzTQBtoTJyyKmhTG8CxbiJ1 jUl3uQK7N9cmAIi+SSzs33jZF7KCwCtn512IqVGZsnxhJur8SI7/CF5SjFgIAiupwlGBv0Muy9m olzpm3MHJGFG5zQ== X-Google-Smtp-Source: AGHT+IGJKvfpLG416BGAixM+h/ZidHbUlKEaobKY4EEVzwZO2KNhNqrg2ZdlLp8R7DryVcFhcu4pbfPELtL2yQ== X-Received: from pjboi12.prod.google.com ([2002:a17:90b:3a0c:b0:33b:9921:8e9a]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:dfcb:b0:343:7711:127d with SMTP id 98e67ed59e1d1-347298a9fabmr23577307a91.9.1764185791103; Wed, 26 Nov 2025 11:36:31 -0800 (PST) Date: Wed, 26 Nov 2025 19:36:00 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-14-dmatlack@google.com> Subject: [PATCH 13/21] vfio: selftests: Build liveupdate library in VFIO selftests From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Import and build liveupdate selftest library in VFIO selftests. It allows to use liveupdate ioctls in VFIO selftests Signed-off-by: Vipin Sharma Signed-off-by: David Matlack --- tools/testing/selftests/vfio/Makefile | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 8bb0b1e2d3a3..8e6e2cc2d8fd 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -11,6 +11,7 @@ TEST_PROGS_EXTENDED :=3D scripts/run.sh TEST_PROGS_EXTENDED :=3D scripts/setup.sh include ../lib.mk include lib/libvfio.mk +include ../liveupdate/lib/libliveupdate.mk =20 CFLAGS +=3D -I$(top_srcdir)/tools/include CFLAGS +=3D -MD @@ -18,11 +19,15 @@ CFLAGS +=3D $(EXTRA_CFLAGS) =20 LDFLAGS +=3D -pthread =20 -$(TEST_GEN_PROGS): %: %.o $(LIBVFIO_O) - $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $< $(LIBVFIO_O) $(LDLIBS) -o $@ +LIBS_O :=3D $(LIBVFIO_O) +LIBS_O +=3D $(LIBLIVEUPDATE_O) + +$(TEST_GEN_PROGS): %: %.o $(LIBS_O) + $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIBS_O) $(LDLI= BS) -o $@ =20 TEST_GEN_PROGS_O =3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) -TEST_DEP_FILES =3D $(patsubst %.o, %.d, $(TEST_GEN_PROGS_O) $(LIBVFIO_O)) +TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(TEST_GEN_PROGS_O)) +TEST_DEP_FILES +=3D $(patsubst %.o, %.d, $(LIBS_O)) -include $(TEST_DEP_FILES) =20 EXTRA_CLEAN +=3D $(TEST_GEN_PROGS_O) $(TEST_DEP_FILES) --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A758E34886F for ; Wed, 26 Nov 2025 19:36:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185796; cv=none; b=elTP5APqE9JLGQdRR7TzDfd8MtcUFRArkGNmBwg9IVrIP76Tqh8OLCBWhHhcZoAMqEC+ZSK+BIwEfeV+MZCs8WGNx1qFb85z4jZ8/Hee5pBl7AiLqoOFWWXuVDFHJwdA0w61Jcj5XulevPthTE6eHhRAre7Wr+V+IcJqKWjYvdo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185796; c=relaxed/simple; bh=LFB03j9PyPhBeg5BZcBtpcmf2J/k1pT0XFgVYS2afs0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=nZCrJabAt9BuXnbl9l3mt7DoGU9lGmkyHaEQHc62NIfVhJSqgtFHykJAPSdjuJKx10+WD08hzFMHYnMIUeQMUUdsDR6eHwaNry4h/cVkFoUWtQp6U/Lm6GIbeHLreuZjxIjcz2+rig2+r9nHSwxfKY4hYwns3FsM2WvnRCxMno8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=waKmfsHn; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="waKmfsHn" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-34374bfbcccso78959a91.0 for ; Wed, 26 Nov 2025 11:36:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185793; x=1764790593; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=//f12vbddhO9W/g+moaY6VE/bxAGwxrPiDFdKTDdMcM=; b=waKmfsHnHN8Qp4RhFB59K/gDbuRnMgiIYepTlz7bbV0o4GkBzC3d7zIYCJyZg59L7P k/gixU//jivw4qIRUD4wrQ5XfjZd89bPqTIa52V/bHqXt0peGTzcI63o+vP8ycv7d69f XNqkmRN3mfJlQG1SOyWVJTWmJ3FoyuQbcnbO7eTT9tdsCn+lRwRVs5UgFXusMTHO7U6u 1GbugwU+TdP5cmCpI6+GfmlSK+vaKuYitP9vHFgHfP/T6+lAa5P0k40NwuJFbA5KPFT9 0BS+5tXeLYhMVCNrGy+LVLgTy0CgzAq2PQkSaYlaB+D+0ZcJd33kA3crdW20RXust5og /P3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185793; x=1764790593; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=//f12vbddhO9W/g+moaY6VE/bxAGwxrPiDFdKTDdMcM=; b=YgFmi35jjIP3UbJNThf2PRKpBgexw8hyxSnKT+XHN49esYP30h6q0zxyJpTZEkQ6pZ PKZHXb6SQi7WrZ+6yvpTuIjLolpun8Gs8UW79Ang5gWdMrtvDL84gSln+MShUa8rfIkt EU7OUaygUhwxA5f62A7NAb2LdE+1frMCk7r56jxx+9dcKZoxnysofnkyWyCtEwjy7tiZ 5OnPmj/w4EvRrfPfxCMqH309ulPtKmRFUsSZx6BpFfKZjoF4O15EGlWy7ob1KITgSEtE Mp8LlLb4Nj5rp5BLeQU3dkHYGzXHgud9vTMTwiFAoMJXRa79AgN7m92jVOIFkbAuACMU 5rmg== X-Forwarded-Encrypted: i=1; AJvYcCVyxsMjFdSqaC6rVLpvdU8XgLmcitOSL93cP+W/E61D1U1z338yMjylZSsE8knbfHRYD2/yYc47KRRzIvc=@vger.kernel.org X-Gm-Message-State: AOJu0YyUnT9TvgASVeC3JkIcPWJhenruc5HALnVCT1GLkiKp+JUdFmoz zCYmsJ3dEyKWwnOIH97vyGuNILHR1aCrDpqLBoieUeXWOt9zlVX1tRHXmKb92lZOZPbX4o4FvHg pBXihugvym0/tTA== X-Google-Smtp-Source: AGHT+IHkKNcU5wbIyBeDLHpLT8jvQv8C3h64QCysVij0yc88H1dhFuteaCqt+2xeK1d/SIGvHBUqJRnU1SfUBQ== X-Received: from pfbly20.prod.google.com ([2002:a05:6a00:7594:b0:7b9:55bc:4970]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:329b:b0:35d:d477:a7e0 with SMTP id adf61e73a8af0-3637dafc474mr9254985637.15.1764185792706; Wed, 26 Nov 2025 11:36:32 -0800 (PST) Date: Wed, 26 Nov 2025 19:36:01 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-15-dmatlack@google.com> Subject: [PATCH 14/21] vfio: selftests: Add Makefile support for TEST_GEN_PROGS_EXTENDED From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add Makefile support for TEST_GEN_PROGS_EXTENDED targets. These tests are not run by default. TEST_GEN_PROGS_EXTENDED will be used for Live Update selftests in subsequent commits. These selftests must be run manually because they require the user/runner to perform additional actions, such as kexec, during the test. Signed-off-by: David Matlack --- tools/testing/selftests/vfio/Makefile | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 8e6e2cc2d8fd..424649049d94 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -22,12 +22,15 @@ LDFLAGS +=3D -pthread LIBS_O :=3D $(LIBVFIO_O) LIBS_O +=3D $(LIBLIVEUPDATE_O) =20 -$(TEST_GEN_PROGS): %: %.o $(LIBS_O) +$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): %: %.o $(LIBS_O) $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIBS_O) $(LDLI= BS) -o $@ =20 -TEST_GEN_PROGS_O =3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) -TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(TEST_GEN_PROGS_O)) +TESTS_O :=3D $(patsubst %, %.o, $(TEST_GEN_PROGS)) +TESTS_O +=3D $(patsubst %, %.o, $(TEST_GEN_PROGS_EXTENDED)) + +TEST_DEP_FILES :=3D $(patsubst %.o, %.d, $(TESTS_O)) TEST_DEP_FILES +=3D $(patsubst %.o, %.d, $(LIBS_O)) -include $(TEST_DEP_FILES) =20 -EXTRA_CLEAN +=3D $(TEST_GEN_PROGS_O) $(TEST_DEP_FILES) +EXTRA_CLEAN +=3D $(TESTS_O) +EXTRA_CLEAN +=3D $(TEST_DEP_FILES) --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5847C3469F3 for ; Wed, 26 Nov 2025 19:36:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185797; cv=none; b=JrmJN35JfiSrNxDFbbpBsp3y8fxcc+CEfrSgxQ0qQU9O5pQ1DIti4ppFhEYLVk2x9rjhOoFOpvQT+IGWlOVaYNLjT0j1P8PgI85k1vkJB3R8LyXxigYk/q6MhW0gJyKujK8jGr3Zji2vNKaDZvE9bFzE+SYKID8PKaHCr3qnqNE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185797; c=relaxed/simple; bh=vBI+BcPeBVK4GSwEmi25dYSa3GjI0+koQ6NIxM3yr7I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RABaJuJupkEdQX0fFOd4lQlBNkHya8qLcDNOOz6eWVjdMJ9t1+yg3NmAV+/t0TddpjaUZB/uGTyJ0/DTkvlBUELN5beLZ9DvdUVow6offBrm1ZjYsQDZK8MGiEvIaY5+jlTtziHeInyvCaINUF9nBZaFSDsOibmkpzVp3ar5E3o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RvtO+qoB; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RvtO+qoB" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7ba9c366057so182336b3a.1 for ; Wed, 26 Nov 2025 11:36:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185794; x=1764790594; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=JlttO+kP+T+xZD5DVn0EpdpTFcNTZGOHePuSNnW/JoA=; b=RvtO+qoBrysg8qXH+T1HcibT/haYUCXdYRZzrf+VRexDeAoIGQoHBl1kMqo4HBnMG2 8SDdxIj2SfP85X6DHNUYKI6NQ+HmELeu01y9/JjkS1kA0m4QxWam7p6OijuuSyTWmKX3 +HPPTvwdNSUkWHjNCfyS5772Nm3pn0nHsEy4AiT1iIPOk++NOFy3QU21LtS53LcabZ0h WjLafKukrIFarSkIuAfS8qf3k0c0xy/rth2+7JWfML1Pw+A6WKXSfTRLz90Zbx2WUcLc szmooieqXrzroM7+C8PZyDGbcHa/Cx2ayhs86kGqK1Pi2lmDYLnrEIfciCa4gCt62IlO 1fow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185794; x=1764790594; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JlttO+kP+T+xZD5DVn0EpdpTFcNTZGOHePuSNnW/JoA=; b=OHdlalGftLiNbyrUBAwAulc0KDXbFwGp1Xc+lQvJSVvPDeWqfbayrCKb9JBjCkb71b Vjpo5R3ZFIGOf89hvZ+zyyJ9B3zfCX5nqPHLttcNLesVVgjl2vH4ZtwSvajaaOWclnti mbCs5Vseb2SfCPw9/cDxPbPcUCQfVOWX4vfdzlY+xPe+90RcZCMMTvx+IsAq4dxVP3IM zvL+sDBjcw3p6RXgoT5KwBokUeEZ+Q9ot3qrQVjGfr09oJaf2/QqowkTcdL968D6F3Tt rn/uCalrInZXNwRGEsiSyAln6w8Ww1HKJ2Hc0oxjwtSlesfeHoRWsN4yHtm+65nXidTn P+GQ== X-Forwarded-Encrypted: i=1; AJvYcCU+lbVQ9J8QUgEI3hoGFQGCt8PC1PWOmmzveomnujiW+S1gViBfGD3eKIVTFyOZNuHLM3dbo5iTP++uYGo=@vger.kernel.org X-Gm-Message-State: AOJu0Yw0U6V4SYTXu1v9MAFUAvDAAxv39bzwFvI+yFHzmWXiq5trPMcr r+cCkEOWNYmw+aONQUnfUoR1q0LihqlmTPReoiPOqMZc864ePrMH/ysrE+bWsG5wO7/c0BWx8lg +M0X7Y9mzwVmcuQ== X-Google-Smtp-Source: AGHT+IGfJFqnQcEqTjyto3NMliFrj7gIasa9HojRwUWAbHy3ogFhO2JBnuthvTdBNfMOSov8pJvPknTpGK+cJQ== X-Received: from pfbfj40.prod.google.com ([2002:a05:6a00:3a28:b0:781:1659:e630]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:23d5:b0:7b8:758c:7e7f with SMTP id d2e1a72fcca58-7ca877f7eabmr8568252b3a.7.1764185794404; Wed, 26 Nov 2025 11:36:34 -0800 (PST) Date: Wed, 26 Nov 2025 19:36:02 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-16-dmatlack@google.com> Subject: [PATCH 15/21] vfio: selftests: Add vfio_pci_liveupdate_uapi_test From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a selftest to exercise preserving a various VFIO files through /dev/liveupdate. Ensure that VFIO cdev device files can be preserved and everything else (group-based device files, group files, and container files) all fail. Signed-off-by: David Matlack --- tools/testing/selftests/vfio/Makefile | 1 + .../vfio/vfio_pci_liveupdate_uapi_test.c | 93 +++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_t= est.c diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 424649049d94..9b2a4f10c558 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -4,6 +4,7 @@ TEST_GEN_PROGS +=3D vfio_iommufd_setup_test TEST_GEN_PROGS +=3D vfio_pci_device_test TEST_GEN_PROGS +=3D vfio_pci_device_init_perf_test TEST_GEN_PROGS +=3D vfio_pci_driver_test +TEST_GEN_PROGS +=3D vfio_pci_liveupdate_uapi_test =20 TEST_PROGS_EXTENDED :=3D scripts/cleanup.sh TEST_PROGS_EXTENDED :=3D scripts/lib.sh diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c b= /tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c new file mode 100644 index 000000000000..3b4276b2532c --- /dev/null +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include +#include + +static const char *device_bdf; + +FIXTURE(vfio_pci_liveupdate_uapi_test) { + int luo_fd; + int session_fd; + struct iommu *iommu; + struct vfio_pci_device *device; +}; + +FIXTURE_VARIANT(vfio_pci_liveupdate_uapi_test) { + const char *iommu_mode; +}; + +#define FIXTURE_VARIANT_ADD_IOMMU_MODE(_iommu_mode) \ +FIXTURE_VARIANT_ADD(vfio_pci_liveupdate_uapi_test, _iommu_mode) { \ + .iommu_mode =3D #_iommu_mode, \ +} + +FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(); +#undef FIXTURE_VARIANT_ADD_IOMMU_MODE + +FIXTURE_SETUP(vfio_pci_liveupdate_uapi_test) +{ + self->luo_fd =3D luo_open_device(); + ASSERT_GE(self->luo_fd, 0); + + self->session_fd =3D luo_create_session(self->luo_fd, "session"); + ASSERT_GE(self->session_fd, 0); + + self->iommu =3D iommu_init(variant->iommu_mode); + self->device =3D vfio_pci_device_init(device_bdf, self->iommu); +} + +FIXTURE_TEARDOWN(vfio_pci_liveupdate_uapi_test) +{ + vfio_pci_device_cleanup(self->device); + iommu_cleanup(self->iommu); + close(self->session_fd); + close(self->luo_fd); +} + +TEST_F(vfio_pci_liveupdate_uapi_test, preserve_device) +{ + int ret; + + ret =3D luo_session_preserve_fd(self->session_fd, self->device->fd, 0); + + /* Preservation should only be supported for VFIO cdev files. */ + ASSERT_EQ(ret, self->iommu->iommufd ? 0 : -ENOENT); +} + +TEST_F(vfio_pci_liveupdate_uapi_test, preserve_group_fails) +{ + int ret; + + if (self->iommu->iommufd) + return; + + ret =3D luo_session_preserve_fd(self->session_fd, self->device->group_fd,= 0); + ASSERT_EQ(ret, -ENOENT); +} + +TEST_F(vfio_pci_liveupdate_uapi_test, preserve_container_fails) +{ + int ret; + + if (self->iommu->iommufd) + return; + + ret =3D luo_session_preserve_fd(self->session_fd, self->iommu->container_= fd, 0); + ASSERT_EQ(ret, -ENOENT); +} + +int main(int argc, char *argv[]) +{ + int fd; + + fd =3D luo_open_device(); + if (fd < 0) { + printf("open(%s) failed: %s, skipping\n", LUO_DEVICE, strerror(errno)); + return KSFT_SKIP; + } + close(fd); + + device_bdf =3D vfio_selftests_get_bdf(&argc, argv); + return test_harness_run(argc, argv); +} --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13F39341642 for ; Wed, 26 Nov 2025 19:36:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185799; cv=none; b=OMa5IsEjZsYPgHzB06IaYC57VjjMKnmW63v162W78pSMShr/wRhuRem0R3pf18D5VMgYFMzOPd8oxXQVsnTxuAHZsnGLMl3+1YtN8f87ny79S2fbYRGOo2bmXCEZi7bYaqKAzWkRAxSKyNHSvd5i44/sJf43g+tdV+2napYCXcs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185799; c=relaxed/simple; bh=oOTZ22wFJZqnFzsFStI6K/6j0+r7OjYCkSOC13WPK5c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=d+2mz8GD0e4ErzbQ8QeAGl7pkJ8/AWjJebeEF+Z9FYwbF6JhsoAY2OlkCOvxw02iSF/HcpblgU+a0NCd1rsCTmtE6OPy0UJNC2niAE7mF1QmBrC5+vy7RUordeuh1zeEC4N04kKJ+gibtk0NjOg2GQYLMJhsjURQRrMKlQjfQGE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=DTHOE9Hi; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="DTHOE9Hi" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7b89ee2c1a4so76918b3a.2 for ; Wed, 26 Nov 2025 11:36:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185796; x=1764790596; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=JNFE9+5HOL1VTfFvE91RbwOqejjTuBWxPiJYobw2Kcg=; b=DTHOE9HimPqnsmGO6KjbvxuhCOECIH5aP141Aqw88qkgl6OjQ/aonnhvd3T1slhAQ7 zBUK7gmd6ghG/GH2jPBWTHoUDeSuChKWwmm2+AzYrL/XB6eFCwQRVy1I/+VA+NZYZMRK HKRnSVQga9HjIrnis7Uycvjamsaea+CHWIyRI5M97xgqVWAwxARlG0GlRzxQrR0iVdyA /bROYq3pk8j8XhOnWoMwpZY9W1UzpnC3Py2OcW0iNfEd78pk1+vp8IjPipFXenTadPdj 9zAWfR+KXH9iCYV5yHBISKm6QDihYJqf32tPuugvozilzqVpr/g1kbIF3I11epmvepsW v0GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185796; x=1764790596; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JNFE9+5HOL1VTfFvE91RbwOqejjTuBWxPiJYobw2Kcg=; b=rFSE+Oi5qd0DeAB0qHSnD8Y6ar0YoWl77MHgw6Lpwphgz0nQN1B86a7ql3OKKd+zo7 qBgzXFwYdOIidN5Fhgm62n0uOZTGSb1f0itQe0QtbUn1jGCli2pkY5hpLJM7GHS9Lpx2 Pa7g5D09/NugaXmvkVOQgrb98d94cnR3HIFDDOo9VQ/uDZUob1c1kfGt1rfg0b43GauX 5RJm0Ijp0OnDSSsl42YWHlfy7YCevvszHcLmoXQuutEvAPPqXYDCXxdW6RAS24ICsOxc GEdqUYxdskoKGhpa8N/V9jTOxGd5iYQQC3WBXc4VQwABbfK5paaXSMrokeLvcbWoOp3t 6t+g== X-Forwarded-Encrypted: i=1; AJvYcCVePwk0Sa7bEwS6b3uyKrDdMsRS3emFIXM9aGPw+gNQ1rmWKVETF30202PD2s8x2RYM6/XyIfe42a4BIw8=@vger.kernel.org X-Gm-Message-State: AOJu0Yyjhl+U+WgKFx3QdcKEantEj9FYUTjZl4UvHOMI0r1gR/Us/iSV ENZxsMDfvi/osWx9hHzkVOJMbVPAihqiRzhCEwaiMmRgJxSHMRIieaO3NwsCpplUHzbru1P3r63 FWPqIy/Fe8vDWLA== X-Google-Smtp-Source: AGHT+IFQhjKrX/N6hJ5PO1HA/1hqWlujUuZce1Edh5MBtvzNhWFpXfCucZ7C6spTR8QDg6JjwdBie16yKE0H0A== X-Received: from pfbmy26-n2.prod.google.com ([2002:a05:6a00:6d5a:20b0:7ba:3994:c73f]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2d88:b0:7b9:ef46:ec70 with SMTP id d2e1a72fcca58-7c58e50c16emr18773625b3a.23.1764185795950; Wed, 26 Nov 2025 11:36:35 -0800 (PST) Date: Wed, 26 Nov 2025 19:36:03 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-17-dmatlack@google.com> Subject: [PATCH 16/21] vfio: selftests: Initialize vfio_pci_device using a VFIO cdev FD From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Use the given VFIO cdev FD to initialize vfio_pci_device in VFIO selftests. Add the assertion to make sure that passed cdev FD is not used with legacy VFIO APIs. If VFIO cdev FD is provided then do not open the device instead use the FD for any interaction with the device. This API will allow to write selftests where VFIO device FD is preserved using liveupdate and retrieved later using liveupdate ioctl after kexec. Signed-off-by: Vipin Sharma Co-Developed-by: David Matlack Signed-off-by: David Matlack --- .../lib/include/libvfio/vfio_pci_device.h | 3 ++ .../selftests/vfio/lib/vfio_pci_device.c | 33 ++++++++++++++----- 2 files changed, 27 insertions(+), 9 deletions(-) diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_devi= ce.h b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h index 2858885a89bb..896dfde88118 100644 --- a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +++ b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h @@ -38,6 +38,9 @@ struct vfio_pci_device { #define dev_info(_dev, _fmt, ...) printf("%s: " _fmt, (_dev)->bdf, ##__VA_= ARGS__) #define dev_err(_dev, _fmt, ...) fprintf(stderr, "%s: " _fmt, (_dev)->bdf,= ##__VA_ARGS__) =20 +struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, + struct iommu *iommu, + int device_fd); struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu= *iommu); void vfio_pci_device_cleanup(struct vfio_pci_device *device); =20 diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/tes= ting/selftests/vfio/lib/vfio_pci_device.c index 468ee1c61b0c..e9423dc3864a 100644 --- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c +++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c @@ -319,19 +319,27 @@ static void vfio_device_attach_iommufd_pt(int device_= fd, u32 pt_id) ioctl_assert(device_fd, VFIO_DEVICE_ATTACH_IOMMUFD_PT, &args); } =20 -static void vfio_pci_iommufd_setup(struct vfio_pci_device *device, const c= har *bdf) +static void vfio_pci_iommufd_setup(struct vfio_pci_device *device, + const char *bdf, int device_fd) { - const char *cdev_path =3D vfio_pci_get_cdev_path(bdf); + const char *cdev_path; =20 - device->fd =3D open(cdev_path, O_RDWR); - VFIO_ASSERT_GE(device->fd, 0); - free((void *)cdev_path); + if (device_fd >=3D 0) { + device->fd =3D device_fd; + } else { + cdev_path =3D vfio_pci_get_cdev_path(bdf); + device->fd =3D open(cdev_path, O_RDWR); + VFIO_ASSERT_GE(device->fd, 0); + free((void *)cdev_path); + } =20 vfio_device_bind_iommufd(device->fd, device->iommu->iommufd); vfio_device_attach_iommufd_pt(device->fd, device->iommu->ioas_id); } =20 -struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu= *iommu) +struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, + struct iommu *iommu, + int device_fd) { struct vfio_pci_device *device; =20 @@ -341,10 +349,12 @@ struct vfio_pci_device *vfio_pci_device_init(const ch= ar *bdf, struct iommu *iomm device->bdf =3D bdf; device->iommu =3D iommu; =20 - if (device->iommu->mode->container_path) + if (device->iommu->mode->container_path) { + VFIO_ASSERT_EQ(device_fd, -1); vfio_pci_container_setup(device, bdf); - else - vfio_pci_iommufd_setup(device, bdf); + } else { + vfio_pci_iommufd_setup(device, bdf, device_fd); + } =20 vfio_pci_device_setup(device); vfio_pci_driver_probe(device); @@ -352,6 +362,11 @@ struct vfio_pci_device *vfio_pci_device_init(const cha= r *bdf, struct iommu *iomm return device; } =20 +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu= *iommu) +{ + return __vfio_pci_device_init(bdf, iommu, /*device_fd=3D*/-1); +} + void vfio_pci_device_cleanup(struct vfio_pci_device *device) { int i; --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48E1634B676 for ; Wed, 26 Nov 2025 19:36:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185800; cv=none; b=UyRhWDMcWiA6ybBB9CNGpgNLzVcd8Ov32wDPQbJ0oW84TGZQ1Q5fbA3jwoJzeDWGKoxJ3hG3DHh49odKVhUHWSGUdR+cW7rvylde5nunIb88V9KRfLbqq9k7boEitGXKmlyl2me2ZIjNxlgxkpjq2ytJKtvpHpI6SJrw1rV02EI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185800; c=relaxed/simple; bh=VROY2nVPYZk7HCEafGudTR0uHJjXvZb2PCdkyXcjk7g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IXTxUz/gvBHhoZlRnrqtDvNVqJ0Y8ZGA3yE7LTUCC12h75EiYZ7NUJhVACFi0btfx2crufAlpY7TtrEhCKA927G/gbqSQVCq61X5emjD8ffQB7zqcAWuPhbFTf3aCsw4Hs3zFkfZTkAcOchJLLG5xToZLEg5LD4MZ5lQvQ6BelQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=d57oHvVc; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="d57oHvVc" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7baaf371585so687679b3a.0 for ; Wed, 26 Nov 2025 11:36:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185798; x=1764790598; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=AglQTQPHcs2CoAEoAnmh5ycsdEepcN2ki5EQBy88w+U=; b=d57oHvVc/6WDHyZbceYYnuh5C7m2bimKY2G4L9k/ngbK+i8gtVmUkzUFoxwCBg2mcl TN6QtpyPIhWIKFLj0V6bTZa0IOrA888ZSdkeTWFAqJes2Iked6utR3Sh9y6X0T2GIEco o87/KFQW2j8/u9SlXEAtv/vQdtXVnMaArnyxMUazuz9hjgsIedlgLmlQJx6d3puihjFv LW3izZuZVFb3O+yQkxegEycoq/HFcnH778pzAVkYrwRSVegAKok8HEHGGo6wUAzwrRHF VXkSfVPMs6YLNsBFP7eKPq4gRQouy4YDlKW9rQtRQlkNzz9c6bVt3bFX9jMPypBiC/Ka XP8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185798; x=1764790598; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AglQTQPHcs2CoAEoAnmh5ycsdEepcN2ki5EQBy88w+U=; b=BgW/TMz4N8Rz/fpd47CntDKMDIpnTv6WQaNLGjx2UihdkUrLTBuf76LTdvFcaX18ht PnwZo5R2JeoquiiBTyfwfO2LwCuA/6qcyJyqQAD1wF1r+15nC7bLaKnsOGaeww8C7TOh NKyVTj0OF8RWNMiPgyDLEvr42NHhyo7jKzFE0R+GXUsS7AC7fZxTsMcmjOZ6tqFiiJfD AWlFPe+lkOTBNIPLpDP/Zvpyhljrdc5C1WrCsl+mEikDLrXHrQC9nvd+jr0bGqmTLlMo YPbeyffhLpCBAcf66MEt5YazTGam3fGKCiXNSHKSze1KM9SmjX+wObCzibaw1fE0nFkm zWJQ== X-Forwarded-Encrypted: i=1; AJvYcCWLgOnLe85G/p94vY/JTp5sGRAEWS4vOZunSP4kWEHMbiAMZmXI3J9q2nX+bpw4l3ix6h/O+YGdG3XbEW4=@vger.kernel.org X-Gm-Message-State: AOJu0YxyNxScPBEa5Bf5JfZT3xcV6K/EQ63UItFkG7yEROWQywhS6j8S +rCCmAujivdAqnFQt1divl8YlAvhb2RlgcZE4gmZOS/vF1ol3K5N2MX7fkTu7S5nc8kbXcvPCEW ffjubY69m02jrJw== X-Google-Smtp-Source: AGHT+IHTAVGn798p3HwWv0BX7bvliQCFJ5hk7JsHWhTZ3epmSpJF+jTGo/Caibt0EEi13ZZvg8+CBEgvtlv2Hg== X-Received: from pfoo16.prod.google.com ([2002:a05:6a00:1a10:b0:7b0:bc2e:9599]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:3391:b0:35d:8881:e69b with SMTP id adf61e73a8af0-3614f4a255bmr24108621637.18.1764185797563; Wed, 26 Nov 2025 11:36:37 -0800 (PST) Date: Wed, 26 Nov 2025 19:36:04 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-18-dmatlack@google.com> Subject: [PATCH 17/21] vfio: selftests: Add vfio_pci_liveupdate_kexec_test From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vipin Sharma Add a selftest to exercise preserving a vfio-pci device across a Live Update. For now the test is extremely simple and just verifies that the device file can be preserved and retrieved. In the future this test will be extended to verify more parts about device preservation as they are implemented. This test is added to TEST_GEN_PROGS_EXTENDED since it must be run manually along with a kexec. To run this test manually: $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 1 00= 00:00:04.0 $ kexec ... # NOTE: Exact method will be distro-dependent $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 2 00= 00:00:04.0 The second call to setup.sh is necessary because preserved devices are not bound to a driver after Live Update. Such devices must be manually bound by userspace after Live Update via driver_override. This test is considered passing if all commands exit with 0. Signed-off-by: Vipin Sharma Co-Developed-by: David Matlack Signed-off-by: David Matlack --- tools/testing/selftests/vfio/Makefile | 4 + .../vfio/vfio_pci_liveupdate_kexec_test.c | 82 +++++++++++++++++++ 2 files changed, 86 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_= test.c diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 9b2a4f10c558..177aa5cd4376 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -6,6 +6,10 @@ TEST_GEN_PROGS +=3D vfio_pci_device_init_perf_test TEST_GEN_PROGS +=3D vfio_pci_driver_test TEST_GEN_PROGS +=3D vfio_pci_liveupdate_uapi_test =20 +# This test must be run manually since it requires the user/automation to +# perform a kexec during the test. +TEST_GEN_PROGS_EXTENDED +=3D vfio_pci_liveupdate_kexec_test + TEST_PROGS_EXTENDED :=3D scripts/cleanup.sh TEST_PROGS_EXTENDED :=3D scripts/lib.sh TEST_PROGS_EXTENDED :=3D scripts/run.sh diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c = b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c new file mode 100644 index 000000000000..95644c3bd2d3 --- /dev/null +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include + +static const char *device_bdf; + +#define STATE_SESSION "567e1b8d-daf1-4a4a-ac13-35d6d275f646" +#define DEVICE_SESSION "751ad95f-1621-4d68-8b5a-f9c2e460425f" + +enum { + STATE_TOKEN, + DEVICE_TOKEN, +}; + +static void before_kexec(int luo_fd) +{ + struct vfio_pci_device *device; + struct iommu *iommu; + int session_fd; + int ret; + + iommu =3D iommu_init("iommufd"); + device =3D vfio_pci_device_init(device_bdf, iommu); + + create_state_file(luo_fd, STATE_SESSION, STATE_TOKEN, /*next_stage=3D*/2); + + session_fd =3D luo_create_session(luo_fd, DEVICE_SESSION); + VFIO_ASSERT_GE(session_fd, 0); + + printf("Preserving device in session\n"); + ret =3D luo_session_preserve_fd(session_fd, device->fd, DEVICE_TOKEN); + VFIO_ASSERT_EQ(ret, 0); + + close(luo_fd); + daemonize_and_wait(); +} + +static void after_kexec(int luo_fd, int state_session_fd) +{ + struct vfio_pci_device *device; + struct iommu *iommu; + int session_fd; + int device_fd; + int stage; + + restore_and_read_stage(state_session_fd, STATE_TOKEN, &stage); + VFIO_ASSERT_EQ(stage, 2); + + session_fd =3D luo_retrieve_session(luo_fd, DEVICE_SESSION); + VFIO_ASSERT_GE(session_fd, 0); + + printf("Finishing the session before retrieving the device (should fail)\= n"); + VFIO_ASSERT_NE(luo_session_finish(session_fd), 0); + + printf("Retrieving the device FD from LUO\n"); + device_fd =3D luo_session_retrieve_fd(session_fd, DEVICE_TOKEN); + VFIO_ASSERT_GE(device_fd, 0); + + printf("Binding the device to an iommufd and setting it up\n"); + iommu =3D iommu_init("iommufd"); + + /* + * This will invoke various ioctls on device_fd such as + * VFIO_DEVICE_GET_INFO. So this is a decent sanity test + * that LUO actually handed us back a valid VFIO device + * file and not something else. + */ + device =3D __vfio_pci_device_init(device_bdf, iommu, device_fd); + + printf("Finishing the session\n"); + VFIO_ASSERT_EQ(luo_session_finish(session_fd), 0); + + vfio_pci_device_cleanup(device); + iommu_cleanup(iommu); +} + +int main(int argc, char *argv[]) +{ + device_bdf =3D vfio_selftests_get_bdf(&argc, argv); + return luo_test(argc, argv, STATE_SESSION, before_kexec, after_kexec); +} --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E8B234C838 for ; Wed, 26 Nov 2025 19:36:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185802; cv=none; b=cGZGkwTYfx5Lz8GwKAOiNeDt2mq7HTzcUAW981AZYChiueMmgYx1YPJAEdfHHsDEtDRT96McTJ3W1Ej4tQEqvXKRz2HvGDU4uCrNlc0YqMjKXeLwQ+epCHYq5/xlbXqasOnRO5LIqtVf7g24Xm6EmjJN7b9Z8fPwkaRRM0AzRU4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185802; c=relaxed/simple; bh=K+uixgzOmOQZujrzd9MR8iSY8J1cEIiUFF0kkwJVoPg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tOp96hFjVJKLrBDRl45qD3z10j5mk0xBuDUnQyIOIFIeZnGUCIkWvExAIbfH2DXeTRhQthBJ8dRRKmkzWMi0u6omCIwCZhN1bCfSTfpEAWybHryllS2aL7JPUAzlJLpswKp6gUWy6W0V0QcHPxKi14cwcAWGzocgeLyHog4i8vA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BEI2UU2O; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BEI2UU2O" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7b82c2c2ca2so95130b3a.1 for ; Wed, 26 Nov 2025 11:36:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185799; x=1764790599; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ivqiOtkW8ET2HoL0E9M1Cs+J6gVjRV1eIqgG0sXk3/w=; b=BEI2UU2OlIZagsPVOGiJKKZrxCWK0uxCEzo9to9wwrLweYys2rEICxP3Y7ngBxO+fm f3QiVKJQEGGCyV/CysV7TtFVrGvWx6EMLKH6BKaIlMc3VGZoHNbtZrflwoV88DynHYSF 9V4gXbQ/x3jjURWe3eFZZVYnYb/hB8LM8pjdPM6aAhEViE/jNk/QvL3CnuFtMRJvgD83 S/EyGFBq+7jIKn/OGlmLuZ8FOwper2QPtGWRZElYDV3YtwYXXOB7Vq9A4U/AV9YgTN3R nmBvVC0gHG8Y8cI0oles7TB2kdVuuuN9OJ/V4GhIpNDjPeKepyPcRA2gvZcv6RgU7cWU 4H1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185799; x=1764790599; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ivqiOtkW8ET2HoL0E9M1Cs+J6gVjRV1eIqgG0sXk3/w=; b=mcf+DNzlf5Ga2al+9eLzxfEsDQ3aBN+oFLcnpKT5HTcCV7q/sHEJjEPGajmDnyY/Fr am1tVE8ga7195vQULZ9+02LSFPP06u49ted1kHrbTTLjwwiwELTONaCocN4PkxpU6kO1 zMwWIZ/3V0TTMy6Sd8rIUUDGBKZmLu74C6sTzVUXTsRRHmS5YRbNjH22/6nFg190u6CK SyLRw9SVzCCbpfLiK2qL68UxJuAdPSF47+fH5icdRarSyRK27qa14wWqvlauyIdbGW1V /xyzKghgW1oqyplldSxuV67bcE0sC/7lMZrcL/u9EshkggkQBz8RON8FvsftDvua5tC4 IxDg== X-Forwarded-Encrypted: i=1; AJvYcCX/bkkrPmwQ5kw5d1sFJ1mP0P22s9C00ubyS4Zx6kcWXHKf3qkkLKq33fMyKLTUV0zzW8jY9azgzvchqlk=@vger.kernel.org X-Gm-Message-State: AOJu0YyLeJ5TV3v9JFzTqiuTg74Mo5LTvCa4pKTHVVHZBHAKuGxZW+7C CzKMzlA7Wn//oMxDuJplvtx1+ZAe8/jJF08wGy2BCSa1tCJD+43zf9feN+19z787j9MgnP4TDhc HUpKAcVTBzols6w== X-Google-Smtp-Source: AGHT+IE3jtexGsrRDjIT0Tyf8nE5Fmimh71e5TPCAkGjiZc+TJVpvLRXUcY5i8nAhMcct73UaVxxGUfD8Z9rpw== X-Received: from pfblc3.prod.google.com ([2002:a05:6a00:4f43:b0:77f:1f29:5399]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:23cb:b0:7ad:f6e8:d013 with SMTP id d2e1a72fcca58-7c58eaf4174mr21909766b3a.32.1764185799181; Wed, 26 Nov 2025 11:36:39 -0800 (PST) Date: Wed, 26 Nov 2025 19:36:05 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-19-dmatlack@google.com> Subject: [PATCH 18/21] vfio: selftests: Expose iommu_modes to tests From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose the list of iommu_modes to enable tests that want to iterate through all possible iommu modes. Signed-off-by: David Matlack --- tools/testing/selftests/vfio/lib/include/libvfio/iommu.h | 2 ++ tools/testing/selftests/vfio/lib/iommu.c | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h b/too= ls/testing/selftests/vfio/lib/include/libvfio/iommu.h index 5c9b9dc6d993..a03ff2281f11 100644 --- a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h +++ b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h @@ -15,6 +15,8 @@ struct iommu_mode { unsigned long iommu_type; }; =20 +extern const struct iommu_mode iommu_modes[]; +extern const int nr_iommu_modes; extern const char *default_iommu_mode; =20 struct dma_region { diff --git a/tools/testing/selftests/vfio/lib/iommu.c b/tools/testing/selft= ests/vfio/lib/iommu.c index 8079d43523f3..bf54f95ec833 100644 --- a/tools/testing/selftests/vfio/lib/iommu.c +++ b/tools/testing/selftests/vfio/lib/iommu.c @@ -24,7 +24,7 @@ const char *default_iommu_mode =3D "iommufd"; =20 /* Reminder: Keep in sync with FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(). */ -static const struct iommu_mode iommu_modes[] =3D { +const struct iommu_mode iommu_modes[] =3D { { .name =3D "vfio_type1_iommu", .container_path =3D "/dev/vfio/vfio", @@ -50,6 +50,8 @@ static const struct iommu_mode iommu_modes[] =3D { }, }; =20 +const int nr_iommu_modes =3D ARRAY_SIZE(iommu_modes); + static const struct iommu_mode *lookup_iommu_mode(const char *iommu_mode) { int i; --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6D7D34CFD9 for ; Wed, 26 Nov 2025 19:36:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185804; cv=none; b=uQ3drlmBFxCxbl16oaEflMKDFITp6bGBQu6zyzd8qWBGL7L/aoCdZE1lnFfSweJvpldJImRQ89qN5oKFRiJsdDZ0D4OQS00J/dDSwLaqRjUJv0Xx/+xiNdOZrCFrdALfsVZnUS/Sb17tUd707TNmaDs/8JfK8kuzd3F9eiB2ezs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185804; c=relaxed/simple; bh=pY5ouGUCyRUEJXsQXzAc3UxouobBtC66I9KMuiV6pDg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Of0ig7ghJj5GEHB9+S2BLfRtRb6BW7tf/tqR8vgKQcZIK9VR3xvLNhvIsOE7ghhV/PeaiSn8DhwIDE1ovTZ54P21PjQ1yzDZI22yf1RRMFYncpbRAiGlT/uBoeehZfFTM8q5ydiQTrKhtuv0CeBhTlQqRY2pl3HCqtileRYit5Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pHWrfcrC; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pHWrfcrC" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-bddf8a4dd94so65033a12.1 for ; Wed, 26 Nov 2025 11:36:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185801; x=1764790601; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=m959yfvIsSxpy5q7Eg0smleSkVURLFC+s19Qeh/qusk=; b=pHWrfcrCCPVOmeBv0AXoHItMQ/DNkudESfPMrswO4cgI7+NRV6hU8l70LGFfSBEiSv DQ1eDgKOBEOZ+pEuwwiF9w32FfooK6K/YzdbjHXqLEVkQn/mtJfQ6uxX7E6zbu07vrik DyJGx8pT2BcATAp/Cz23dNdsj8/bYIq8Fd90/r3jG2SlNhuazdk+l8wv2HqiejJ1NAcI LuMQZq14+6JUYV2ZgpCxnKz0GcKO0qEttdSRwvrMIjYyV5tIB2djsJ4lRvE0nljBIqtN ydXJEHYteKOmGd3bYg6P55iZsCBP/Bhafhfa+Cr8Cwtk0mY9m4oWt9mva+yblU/LWBbK U8PQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185801; x=1764790601; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=m959yfvIsSxpy5q7Eg0smleSkVURLFC+s19Qeh/qusk=; b=fcXAWK5vAQvyyFcj+9HrRurua3hwqO2/+hTKVK+fQsC6HayuZ/DDQtSrNtiBOKBEu/ W/aAwU521D31HgSPfCxzIWzxLnGgIp6lDhs8JPRFpBfGpIGH2O99/zshtx7znB1TD/lB Liumjnr+79USCcO7gCdGxhBfUph0zoOrO9l1Yx9f4LRdFHpvcSocDw0jyQACDRhoI5dy umLC9EJS10wpAFILPqQwXilLhLDUW4sxSUh7JE3yedPNjoPZPALIiHDHBe6iBqc6IiHm u6KA2FAUvUFV+IjgCGsRCHLQvCO3B5f1jXOIvh1Cpu0BSoRQ2yBxfkeBoj9Al0xviyye ILoA== X-Forwarded-Encrypted: i=1; AJvYcCXoG+L0tCBMbvnWBIZvpZUIHovpNVUzpFfjPLF3GKjfiPYZwFGkJ1qyIcezltIQZtg34CLAWVh+eHktwXc=@vger.kernel.org X-Gm-Message-State: AOJu0Yxs2QNMuqaUb4Yzn9xvD8TZH7zNJZcX0pYAG49X89UqwQQtvScO 6sJFHlgEgGJUL8aDXBStJnfnaE6i3c4zgb1O52D9DHLUhVH0/v7TIImP5YLRDWv7hIEdTv1i+lE 3X6lQDEs1oBJKCA== X-Google-Smtp-Source: AGHT+IFHEidLnQkcHmYLLGmMjLxzA1O3bNZ0x68Qy0dH/iUpCs9DOSGGUEyYCq/FUTYeONv/KIiX9w+sQ3BkIA== X-Received: from pfcs3.prod.google.com ([2002:a05:6a00:6fc3:b0:7b8:8099:b70b]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:2590:b0:35d:1bcd:687f with SMTP id adf61e73a8af0-3614f2550a8mr23300680637.9.1764185800837; Wed, 26 Nov 2025 11:36:40 -0800 (PST) Date: Wed, 26 Nov 2025 19:36:06 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-20-dmatlack@google.com> Subject: [PATCH 19/21] vfio: selftests: Expose low-level helper routines for setting up struct vfio_pci_device From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose a few low-level helper routings for setting up vfio_pci_device structs. These routines will be used in a subsequent commit to assert that VFIO_GROUP_GET_DEVICE_FD fails under certain conditions. Signed-off-by: David Matlack --- .../lib/include/libvfio/vfio_pci_device.h | 5 +++ .../selftests/vfio/lib/vfio_pci_device.c | 33 +++++++++++++------ 2 files changed, 28 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_devi= ce.h b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h index 896dfde88118..2389c7698335 100644 --- a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +++ b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h @@ -125,4 +125,9 @@ static inline bool vfio_pci_device_match(struct vfio_pc= i_device *device, =20 const char *vfio_pci_get_cdev_path(const char *bdf); =20 +/* Low-level routines for setting up a struct vfio_pci_device */ +struct vfio_pci_device *vfio_pci_device_alloc(const char *bdf, struct iomm= u *iommu); +void vfio_pci_group_setup(struct vfio_pci_device *device); +void vfio_pci_iommu_setup(struct vfio_pci_device *device); + #endif /* SELFTESTS_VFIO_LIB_INCLUDE_LIBVFIO_VFIO_PCI_DEVICE_H */ diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/tes= ting/selftests/vfio/lib/vfio_pci_device.c index e9423dc3864a..c1a3886dee30 100644 --- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c +++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c @@ -199,7 +199,7 @@ static unsigned int vfio_pci_get_group_from_dev(const c= har *bdf) return group; } =20 -static void vfio_pci_group_setup(struct vfio_pci_device *device, const cha= r *bdf) +void vfio_pci_group_setup(struct vfio_pci_device *device) { struct vfio_group_status group_status =3D { .argsz =3D sizeof(group_status), @@ -207,7 +207,7 @@ static void vfio_pci_group_setup(struct vfio_pci_device= *device, const char *bdf char group_path[32]; int group; =20 - group =3D vfio_pci_get_group_from_dev(bdf); + group =3D vfio_pci_get_group_from_dev(device->bdf); snprintf(group_path, sizeof(group_path), "/dev/vfio/%d", group); =20 device->group_fd =3D open(group_path, O_RDWR); @@ -219,14 +219,12 @@ static void vfio_pci_group_setup(struct vfio_pci_devi= ce *device, const char *bdf ioctl_assert(device->group_fd, VFIO_GROUP_SET_CONTAINER, &device->iommu->= container_fd); } =20 -static void vfio_pci_container_setup(struct vfio_pci_device *device, const= char *bdf) +void vfio_pci_iommu_setup(struct vfio_pci_device *device) { struct iommu *iommu =3D device->iommu; unsigned long iommu_type =3D iommu->mode->iommu_type; int ret; =20 - vfio_pci_group_setup(device, bdf); - ret =3D ioctl(iommu->container_fd, VFIO_CHECK_EXTENSION, iommu_type); VFIO_ASSERT_GT(ret, 0, "VFIO IOMMU type %lu not supported\n", iommu_type); =20 @@ -236,8 +234,14 @@ static void vfio_pci_container_setup(struct vfio_pci_d= evice *device, const char * because the IOMMU type is already set. */ (void)ioctl(iommu->container_fd, VFIO_SET_IOMMU, (void *)iommu_type); +} =20 - device->fd =3D ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, bdf); +static void vfio_pci_container_setup(struct vfio_pci_device *device) +{ + vfio_pci_group_setup(device); + vfio_pci_iommu_setup(device); + + device->fd =3D ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, device->= bdf); VFIO_ASSERT_GE(device->fd, 0); } =20 @@ -337,9 +341,7 @@ static void vfio_pci_iommufd_setup(struct vfio_pci_devi= ce *device, vfio_device_attach_iommufd_pt(device->fd, device->iommu->ioas_id); } =20 -struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, - struct iommu *iommu, - int device_fd) +struct vfio_pci_device *vfio_pci_device_alloc(const char *bdf, struct iomm= u *iommu) { struct vfio_pci_device *device; =20 @@ -349,9 +351,20 @@ struct vfio_pci_device *__vfio_pci_device_init(const c= har *bdf, device->bdf =3D bdf; device->iommu =3D iommu; =20 + return device; +} + +struct vfio_pci_device *__vfio_pci_device_init(const char *bdf, + struct iommu *iommu, + int device_fd) +{ + struct vfio_pci_device *device; + + device =3D vfio_pci_device_alloc(bdf, iommu); + if (device->iommu->mode->container_path) { VFIO_ASSERT_EQ(device_fd, -1); - vfio_pci_container_setup(device, bdf); + vfio_pci_container_setup(device); } else { vfio_pci_iommufd_setup(device, bdf, device_fd); } --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4CE2434D4C4 for ; Wed, 26 Nov 2025 19:36:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185805; cv=none; b=nYcIR0UDLeMJd4REIz8CIQlIfM8iqy3VNfC13kxAk4H75nE/egeZF6pxBEafgbOklerHvy2iifOcaL+56jXQA7QDWtqHm6d090sl6KxcTiv26hOjLLgoAJwGNeRM76xBnrpERrYaZCKF7HkP3+hWEmsV8HNuVGxzQzsfSqq8o2E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185805; c=relaxed/simple; bh=AKg0E0qnouOBnaL90KIbwoCnCOQpIGGOxs9nH2+RlA4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dgbxkS88WLZ/ZIMzJ4BRLloCVk1RdnFuGY7efpkF6bXmIAAWF8gUTxOT49MgIUKQ+0m1aAf4bAnyjjOrT1m0MuWuKR0oqThjEZuXmfoElkEExSadqAKmfFv3DR7C9PlC14IaFxv9ED/ZEsUG3rJhAgvcrSZ3ntDauOcHcRSDVn0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Mk7cMxbX; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Mk7cMxbX" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7bb2303fe94so55898b3a.3 for ; Wed, 26 Nov 2025 11:36:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185803; x=1764790603; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2mMhadFzAEncdROdPe4thGWEZeQry8cuGMCxEDY4Otw=; b=Mk7cMxbXLvjHycUbk+yGO+lx4g0Rp90S77qZcZed7HSDZVtbNy+Bg6WbU3PrT7Et3+ /sl2RqW9TDxBvcqUGvqA2aa1ZzgPZKh8QoVbop0w6i0nWEcuu0emi1+fViNhaQKT+vNz 2XZl/42BBvSoCrSY8FK2NDGKAhlDfet0SCQs9p9CXthktdSy4xHa5VHVwvi1dEvE2Nxy 99t0wEaEQya1qQBhU5LT5RRYOeOLhWfRyy9GjecfuvDKSXL3MDMO68TH8YTELD/6/FgL Rg13Pj0nrYKOZEDsBwOrdQeg0BxPJUViz1EtrpLbfa0o2xVwDkSPERny40htaNYMeogA dkbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185803; x=1764790603; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2mMhadFzAEncdROdPe4thGWEZeQry8cuGMCxEDY4Otw=; b=AyI8Bd21iQUfFjcEqwFeFLNxEW1MOatzM6dtThrfgs5Mvfo43fNvXxWRXfnQOXQGk+ 7JGQdY7HUPWNXVuYIOKwf3S5Swmdy1D/LLR0B19jCrf2FGyzeUfOxABzdR3ic0d5YWh9 PDctQIxY+EXkmn4KnTXC7izD/tV9GpjL8nLEq6Njg378wIhN3Yl+/FeTPEq6pP+3sh4M h2xxml4agtoAL1mpDOR6FjqAyUQdcnC+ZUiOSLTqu/FE5g7dhPh2i1NG70sGuTbLJNxX UgITt3CsQmS9MjY8pI2Ecck/VwKC29R4KF2kTgigMhyTXnoqwmfeju0L8+SLRbBHmDMF 998w== X-Forwarded-Encrypted: i=1; AJvYcCVH8hYBY+d77oemlv9x4iyNfZQvvo5DzxZnz6ro5wVRW6/pQNVblHYvD+iutE2z6lDi5rqCphzQMDV2P5Q=@vger.kernel.org X-Gm-Message-State: AOJu0YxlX2ZBLR4dWSl6UmGdI5iNzPWEZJ92qzvWbnj7i3Uvv7kS8TT7 6x2Ovn5mTGj3A+z/Kgb017eDTUl8Ab8t3heghyzHA64oqBJSulEqf3K8mLn4zSSMbqF36Qzesng sRBSfxBPJwyR+tw== X-Google-Smtp-Source: AGHT+IG0nWsINIf/GruvQ2HPIp2jBBinQeZ2Mfu1D9y30mRN7EoyzQVebfADvYhxmegugqK6CpozEorpQsDNYQ== X-Received: from pfbcw14.prod.google.com ([2002:a05:6a00:450e:b0:7ba:8e7:5499]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:32a2:b0:340:db9b:cfe8 with SMTP id adf61e73a8af0-36150e1fc11mr23324744637.12.1764185802504; Wed, 26 Nov 2025 11:36:42 -0800 (PST) Date: Wed, 26 Nov 2025 19:36:07 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-21-dmatlack@google.com> Subject: [PATCH 20/21] vfio: selftests: Verify that opening VFIO device fails during Live Update From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Verify that opening a VFIO device through its cdev file and via VFIO_GROUP_GET_DEVICE_FD both fail with -EBUSY if the device was preserved across a Live Update. When a device file is preserve across a Live Update, the file must be retrieved from /dev/liveupdate, not from VFIO directly. Signed-off-by: David Matlack --- .../vfio/vfio_pci_liveupdate_kexec_test.c | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c = b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c index 95644c3bd2d3..925c5fc30d56 100644 --- a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c @@ -36,6 +36,42 @@ static void before_kexec(int luo_fd) daemonize_and_wait(); } =20 +static void check_open_vfio_device_fails(void) +{ + const char *cdev_path =3D vfio_pci_get_cdev_path(device_bdf); + struct vfio_pci_device *device; + struct iommu *iommu; + int ret, i; + + printf("Checking open(%s) fails\n", cdev_path); + ret =3D open(cdev_path, O_RDWR); + VFIO_ASSERT_EQ(ret, -1); + VFIO_ASSERT_EQ(errno, EBUSY); + free((void *)cdev_path); + + for (i =3D 0; i < nr_iommu_modes; i++) { + if (!iommu_modes[i].container_path) + continue; + + iommu =3D iommu_init(iommu_modes[i].name); + + device =3D vfio_pci_device_alloc(device_bdf, iommu); + vfio_pci_group_setup(device); + vfio_pci_iommu_setup(device); + + printf("Checking ioctl(group_fd, VFIO_GROUP_GET_DEVICE_FD, \"%s\") fails= (%s)\n", + device_bdf, iommu_modes[i].name); + + ret =3D ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, device->bdf); + VFIO_ASSERT_EQ(ret, -1); + VFIO_ASSERT_EQ(errno, EBUSY); + + close(device->group_fd); + free(device); + iommu_cleanup(iommu); + } +} + static void after_kexec(int luo_fd, int state_session_fd) { struct vfio_pci_device *device; @@ -44,6 +80,8 @@ static void after_kexec(int luo_fd, int state_session_fd) int device_fd; int stage; =20 + check_open_vfio_device_fails(); + restore_and_read_stage(state_session_fd, STATE_TOKEN, &stage); VFIO_ASSERT_EQ(stage, 2); =20 --=20 2.52.0.487.g5c8c507ade-goog From nobody Mon Dec 1 22:37:06 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 094D334D4FD for ; Wed, 26 Nov 2025 19:36:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185807; cv=none; b=lL82r58Dw2fMxjtHwGhINHQ9L6LuAtBrKlTmyGKGzR8A0C6G2cG2Lv0qZgGEnwdMEwGVb/NMQ/1IcL1QEpcsHTVz/J8jHq8WL/Hj8RrKpnCMcGlfuGVV5BWxfN+sG98cbQpnmjLQk98ztSnZaGrEc+a/DIGPK1FKEc2fKIEJExc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764185807; c=relaxed/simple; bh=KlrakXRYDAwae9AantcdeyMmPCGsY3ONeKbMm2nz5i0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HiGn0OEXdMrpSmJJUWM3du+Yb/igTy4STuFoA7FLsTqdRUQ3sUWa+KS5nk5nKGtMsGsYOikiWk7W1ICZbBT7F1CR9UAn11dmzLkK6DvTs2XKyVeVtz+qOnYwlNnsV3x66cQCOkKSFMpsKlvCKJlw7fmBlw9y+yTHOUSdrngNUGM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WWTiWNxf; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WWTiWNxf" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7a998ab7f87so73081b3a.3 for ; Wed, 26 Nov 2025 11:36:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764185804; x=1764790604; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=F4HruGk4w1YQ8UWc4dTnuGXBMMZoDm4CrvXWdnupA6c=; b=WWTiWNxfaNnX+oBcLgUNkYafPNVUbL0RUQAhNMs6LbIoWTkYvBzvKvGou9CwkxQdEl hix409K4K9TiANobe19fQJCMJhc3QJzPv6hqy8QJJkmTtBBkvh/NMgul1tb4TnhnzsON YCf2Ug00lvbBFF3qZcsInQB6+bcuXS9XJ0gXVgKExcMWgfU5mQ76v9VuM2t0Ft0UpPC6 pJ1djSga2WnvBHAxMskx2fxW42RE/5NJqhHN5Bqa39DdNiM+27DnOC/CqdJLi9dvwq9z L8suHOGkhtCOw4A/gZ6CXYFE4ZAsaI/7ldJqg0FwLUTHmoRtUoRjrCOnovelx/T13YHR RuDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764185804; x=1764790604; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=F4HruGk4w1YQ8UWc4dTnuGXBMMZoDm4CrvXWdnupA6c=; b=O3K+hhkIFNbYT6selX7nA0kSM4HR7J/953Jl2RwhYi8sCL4Tcvrs9tGqdOUUi2mIMt gY+wab/bsIwnxk1Gvm4mHGYLjOKcBxGPznqjCzFHmXxH4neZ0t2okV8Kr35vPOGwlPWG tb4bUl3O3dpIA2f8v+fzbUkJR08NpAuT7J/5rqZvNiHjh0O3NXl6Qx2NKGVLtuteWlSB CMbM5l8G1kcGjtphyX47TIA/wliafosFZvkb4LgH7itBc9mu7pM1BKhziM25BPGNefAE XNhFI9/Mqa/Cgm7aMTcfNPmP4c/1rH1nMtSZ4nmC+PigcEwY9ODBl6uefiHichj/7Rzr EXuQ== X-Forwarded-Encrypted: i=1; AJvYcCWcRA8jFOpGZgb3gW9LGuIug5qQgwVNMw5tQehGFfQh61FxsCQRmxj73HIj9U9GmXg1HREwYSZ6y/MK2bU=@vger.kernel.org X-Gm-Message-State: AOJu0YysLMtJr8cDkdsHEsFwP6q4v+2BeHCAL5ejO08LAAWwYu7Xy8zO eGPxnOiNDzt4MRmBDpUbQQW0w9Mx4rOT5WuhGNpHMkJP28cnovMvLYdsqdWDJ2euYL6D2AJ24F8 IbroM10uMHtG6Tw== X-Google-Smtp-Source: AGHT+IEBIWWppu0CTBFQwqlZd62yUWkoTSQEwzYofKMCO8/xCUHYiOx68XHiZKglrx1LrJ62PFpPkJgu1NINKA== X-Received: from pfbei35.prod.google.com ([2002:a05:6a00:80e3:b0:7be:dfde:de14]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:3ca8:b0:35f:c643:105e with SMTP id adf61e73a8af0-3614eda6de1mr22522832637.34.1764185804103; Wed, 26 Nov 2025 11:36:44 -0800 (PST) Date: Wed, 26 Nov 2025 19:36:08 +0000 In-Reply-To: <20251126193608.2678510-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251126193608.2678510-1-dmatlack@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251126193608.2678510-22-dmatlack@google.com> Subject: [PATCH 21/21] vfio: selftests: Add continuous DMA to vfio_pci_liveupdate_kexec_test From: David Matlack To: Alex Williamson Cc: Adithya Jayachandran , Alex Mastro , Alistair Popple , Andrew Morton , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jason Gunthorpe , Josh Hilke , Kevin Tian , kvm@vger.kernel.org, Leon Romanovsky , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-pci@vger.kernel.org, Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Philipp Stanner , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Tomita Moeko , Vipin Sharma , William Tu , Yi Liu , Yunxiang Li , Zhu Yanjun Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a long-running DMA memcpy operation to vfio_pci_liveupdate_kexec_test so that the device attempts to perform DMAs continuously during the Live Update. At this point iommufd preservation is not supported and bus mastering is not kept enabled on the device during across the kexec, so most of these DMAs will be dropped. However this test ensures that the current device preservation support does not lead to system instability or crashes if the device is active. And once iommufd and bus mastering are preserved, this test can be relaxed to check that the DMA operations completed successfully. Signed-off-by: David Matlack --- .../vfio/vfio_pci_liveupdate_kexec_test.c | 135 ++++++++++++++++++ 1 file changed, 135 insertions(+) diff --git a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c = b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c index 925c5fc30d56..3af50110a65d 100644 --- a/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c +++ b/tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c @@ -1,8 +1,16 @@ // SPDX-License-Identifier: GPL-2.0-only =20 +#include +#include + #include #include =20 +#define MEMCPY_SIZE SZ_1G +#define DRIVER_SIZE SZ_1M +#define MEMFD_SIZE (MEMCPY_SIZE + DRIVER_SIZE) + +static struct dma_region memcpy_region; static const char *device_bdf; =20 #define STATE_SESSION "567e1b8d-daf1-4a4a-ac13-35d6d275f646" @@ -11,8 +19,100 @@ static const char *device_bdf; enum { STATE_TOKEN, DEVICE_TOKEN, + MEMFD_TOKEN, }; =20 +static void dma_memcpy_one(struct vfio_pci_device *device) +{ + void *src =3D memcpy_region.vaddr, *dst; + u64 size; + + size =3D min_t(u64, memcpy_region.size / 2, device->driver.max_memcpy_siz= e); + dst =3D src + size; + + memset(src, 1, size); + memset(dst, 0, size); + + printf("Kicking off 1 DMA memcpy operations of size 0x%lx...\n", size); + vfio_pci_driver_memcpy(device, + to_iova(device, src), + to_iova(device, dst), + size); + + VFIO_ASSERT_EQ(memcmp(src, dst, size), 0); +} + +static void dma_memcpy_start(struct vfio_pci_device *device) +{ + void *src =3D memcpy_region.vaddr, *dst; + u64 count, size; + + size =3D min_t(u64, memcpy_region.size / 2, device->driver.max_memcpy_siz= e); + dst =3D src + size; + + /* + * Rough Math: If we assume the device will perform memcpy at a rate of + * 30GB/s then 7200GB of transfers will run for about 4 minutes. + */ + count =3D (u64)7200 * SZ_1G / size; + count =3D min_t(u64, count, device->driver.max_memcpy_count); + + memset(src, 1, size / 2); + memset(dst, 0, size / 2); + + printf("Kicking off %lu DMA memcpy operations of size 0x%lx...\n", count,= size); + vfio_pci_driver_memcpy_start(device, + to_iova(device, src), + to_iova(device, dst), + size, count); +} + +static void dma_memfd_map(struct vfio_pci_device *device, int fd) +{ + void *vaddr; + + vaddr =3D mmap(NULL, MEMFD_SIZE, PROT_WRITE, MAP_SHARED, fd, 0); + VFIO_ASSERT_NE(vaddr, MAP_FAILED); + + memcpy_region.iova =3D SZ_4G; + memcpy_region.size =3D MEMCPY_SIZE; + memcpy_region.vaddr =3D vaddr; + iommu_map(device->iommu, &memcpy_region); + + device->driver.region.iova =3D memcpy_region.iova + memcpy_region.size; + device->driver.region.size =3D DRIVER_SIZE; + device->driver.region.vaddr =3D vaddr + memcpy_region.size; + iommu_map(device->iommu, &device->driver.region); +} + +static void dma_memfd_setup(struct vfio_pci_device *device, int session_fd) +{ + int fd, ret; + + fd =3D memfd_create("dma-buffer", 0); + VFIO_ASSERT_GE(fd, 0); + + ret =3D fallocate(fd, 0, 0, MEMFD_SIZE); + VFIO_ASSERT_EQ(ret, 0); + + printf("Preserving memfd of size 0x%x in session\n", MEMFD_SIZE); + ret =3D luo_session_preserve_fd(session_fd, fd, MEMFD_TOKEN); + VFIO_ASSERT_EQ(ret, 0); + + dma_memfd_map(device, fd); +} + +static void dma_memfd_restore(struct vfio_pci_device *device, int session_= fd) +{ + int fd; + + printf("Retrieving memfd from LUO\n"); + fd =3D luo_session_retrieve_fd(session_fd, MEMFD_TOKEN); + VFIO_ASSERT_GE(fd, 0); + + dma_memfd_map(device, fd); +} + static void before_kexec(int luo_fd) { struct vfio_pci_device *device; @@ -32,6 +132,27 @@ static void before_kexec(int luo_fd) ret =3D luo_session_preserve_fd(session_fd, device->fd, DEVICE_TOKEN); VFIO_ASSERT_EQ(ret, 0); =20 + dma_memfd_setup(device, session_fd); + + /* + * If the device has a selftests driver, kick off a long-running DMA + * operation to exercise the device trying to DMA during a Live Update. + * Since iommufd preservation is not supported yet, these DMAs should be + * dropped. So this is just looking to verify that the system does not + * fall over and crash as a result of a busy device being preserved. + */ + if (device->driver.ops) { + vfio_pci_driver_init(device); + dma_memcpy_start(device); + + /* + * Disable interrupts on the device or freeze() will fail. + * Unfortunately there isn't a way to easily have a test for + * that here since the check happens during shutdown. + */ + vfio_pci_msix_disable(device); + } + close(luo_fd); daemonize_and_wait(); } @@ -106,9 +227,23 @@ static void after_kexec(int luo_fd, int state_session_= fd) */ device =3D __vfio_pci_device_init(device_bdf, iommu, device_fd); =20 + dma_memfd_restore(device, session_fd); + printf("Finishing the session\n"); VFIO_ASSERT_EQ(luo_session_finish(session_fd), 0); =20 + /* + * Once iommufd preservation is supported and the device is kept fully + * running across the Live Update, this should wait for the long- + * running DMA memcpy operation kicked off in before_kexec() to + * complete. But for now we expect the device to be reset so just + * trigger a single memcpy to make sure it's still functional. + */ + if (device->driver.ops) { + vfio_pci_driver_init(device); + dma_memcpy_one(device); + } + vfio_pci_device_cleanup(device); iommu_cleanup(iommu); } --=20 2.52.0.487.g5c8c507ade-goog