From nobody Sat May 4 11:49:18 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1659326555; cv=none; d=zohomail.com; s=zohoarc; b=j6tPductH85A5sXMZEkmFbrjBxf8lcdy9yMR7ZyAXEEk27HZcthBkOl62AHA00FZy5jt2EQ3fHoZy7WMCNwAoGJLDR1x8I1CbbC8qzagQcfOcparImpI8KmZgHV6ON9VJW0E8lb3dG8O3ebOB/xf93a+L4MBIV6h1ta3l/Tc5aA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1659326555; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=dx4xTvH8gaFN/8ZUhABbVcrb7hIGrIJRRFknfpbEFJQ=; b=cdz3tzW+UX8JmJt8MjPAW0QNUOEPQX1vMG37WQIMbs12j5cGGoN9CxPqGmdL26Z1koDFPhEUHdoJQogrUIdGS7S14PIRZDvpW6wlR01yH9ELYij4Z+KcdlM7Sha2ZGZV6ebo8svSbObHXbbrL/Jt28ooyNz0AbCKVQgKjo44OEc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1659326555092419.6254542669884; Sun, 31 Jul 2022 21:02:35 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-100-6r9O5Y62OkCXGCPDSDSW2Q-1; Mon, 01 Aug 2022 00:02:27 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4054A85A58A; Mon, 1 Aug 2022 04:02:25 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 79B352166B26; Mon, 1 Aug 2022 04:02:23 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 572C61945D87; Mon, 1 Aug 2022 04:02:23 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id AD5711945D83 for ; Mon, 1 Aug 2022 04:02:22 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 8FB4B2166B2A; Mon, 1 Aug 2022 04:02:22 +0000 (UTC) Received: from vhost3.router.laine.org (unknown [10.2.16.49]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6A53D2166B26; Mon, 1 Aug 2022 04:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1659326553; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=dx4xTvH8gaFN/8ZUhABbVcrb7hIGrIJRRFknfpbEFJQ=; b=J0hN0edzEAlg0x9/vlS4paYWGKPAXt1/Bt5EZCo7iS4ljgGVP2hEZxhuAPsL0H3jlHEihT yDZSPHBJGEULhSF7cMrh839kAf03eVl6el8921LQBuMoOyPTztHUReOMoDdqqhb5iQluVE b/0CkHXW2x7QJFHBBgHq4/dTaWCGKiI= X-MC-Unique: 6r9O5Y62OkCXGCPDSDSW2Q-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Laine Stump To: libvir-list@redhat.com Subject: [PATCH] util: basic support for vendor-specific vfio drivers Date: Mon, 1 Aug 2022 00:02:22 -0400 Message-Id: <20220801040222.675916-1-laine@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1659326556273100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Before a PCI device can be assigned to a guest with VFIO, that device must be bound to the vfio-pci driver rather than to the device's normal driver. The vfio-pci driver provides APIs that permit QEMU to perform all the necessary operations to make the device accessible to the guest. There has been kernel work recently to support vendor/device-specific VFIO drivers that provide the basic vfio-pci driver functionality while adding support for device-specific operations (for example these device-specific drivers are planned to support live migration of certain devices). All that will be needed to make this functionality available will be to bind the new vendor-specific driver to the device (rather than the generic vfio-pci driver, which will continue to work just without the extra functionality). But until now libvirt has required that all PCI devices being assigned to a guest with VFIO specifically have the "vfio-pci" driver bound to the device. So even if the user manually binds a shiny new vendor-specific driver to the device (and puts "managed=3D'no'" in the config to prevent libvirt from changing that), libvirt will just fail during startup of the guest (or during hotplug) because the driver bound to the device isn't named exactly "vfio-pci". Fortunately these new vendor/device-specific drivers can be easily identified as being "vfio-pci + extra stuff" - all that's needed is to look at the output of the "modinfo $driver_name" command to see if "vfio_pci" is in the alias list for the driver. That's what this patch does. When libvirt checks the driver bound to a device (either to decide if it needs to bind to a different driver or perform some other operation, or if the current driver is acceptable as-is), if the driver isn't specifically "vfio-pci", then it will look at the output of modinfo for the driver that *is* bound to the device; if modinfo shows vfio_pci as an alias for that device, then we'll behave as if the driver was exactly "vfio-pci". The effect of this patch is that users will now be able to pre-setup a device to be bound to a vendor-specific driver, then put "managed=3D'no'" in the config and libvirt will allow that driver. What this patch does *not* do is handle automatically determining the proper/best vendor-specific driver and binding to it in the case of "managed=3D'yes'". This will be implemented later when there is a widely available driver / device combo we can use for testing. This initial simple patch is just something simple that will permit initial testing of the new drivers' functionality. (I personally had to add an extra patch playing with driver names to my build just to test that everything was working as expected; that's okay for a patch as simple as this, but wouldn't be acceptable testing for anything more complex.) Signed-off-by: Laine Stump --- meson.build | 1 + src/hypervisor/virhostdev.c | 26 ++++------- src/util/virpci.c | 90 ++++++++++++++++++++++++++++++++++--- src/util/virpci.h | 3 ++ 4 files changed, 97 insertions(+), 23 deletions(-) diff --git a/meson.build b/meson.build index de59b1be9c..9d96eb3ee3 100644 --- a/meson.build +++ b/meson.build @@ -822,6 +822,7 @@ optional_programs =3D [ 'iscsiadm', 'mdevctl', 'mm-ctl', + 'modinfo', 'modprobe', 'ovs-vsctl', 'pdwtags', diff --git a/src/hypervisor/virhostdev.c b/src/hypervisor/virhostdev.c index c0ce867596..15b35fa75e 100644 --- a/src/hypervisor/virhostdev.c +++ b/src/hypervisor/virhostdev.c @@ -747,9 +747,8 @@ virHostdevPreparePCIDevicesImpl(virHostdevManager *mgr, mgr->inactivePCIHostdevs) < 0) goto reattachdevs; } else { - g_autofree char *driverPath =3D NULL; - g_autofree char *driverName =3D NULL; - int stub; + g_autofree char *drvName =3D NULL; + virPCIStubDriver drvType; =20 /* Unmanaged devices should already have been marked as * inactive: if that's the case, we can simply move on */ @@ -769,18 +768,14 @@ virHostdevPreparePCIDevicesImpl(virHostdevManager *mg= r, * information about active / inactive device across * daemon restarts has been implemented */ =20 - if (virPCIDeviceGetDriverPathAndName(pci, - &driverPath, &driverName)= < 0) + if (virPCIDeviceGetDriverNameAndType(pci, &drvName, &drvType) = < 0) goto reattachdevs; =20 - stub =3D virPCIStubDriverTypeFromString(driverName); - - if (stub > VIR_PCI_STUB_DRIVER_NONE && - stub < VIR_PCI_STUB_DRIVER_LAST) { + if (drvType > VIR_PCI_STUB_DRIVER_NONE) { =20 /* The device is bound to a known stub driver: store this * information and add a copy to the inactive list */ - virPCIDeviceSetStubDriver(pci, stub); + virPCIDeviceSetStubDriver(pci, drvType); =20 VIR_DEBUG("Adding PCI device %s to inactive list", virPCIDeviceGetName(pci)); @@ -2292,18 +2287,13 @@ virHostdevPrepareOneNVMeDevice(virHostdevManager *h= ostdev_mgr, /* Let's check if all PCI devices are NVMe disks. */ for (i =3D 0; i < virPCIDeviceListCount(pciDevices); i++) { virPCIDevice *pci =3D virPCIDeviceListGet(pciDevices, i); - g_autofree char *drvPath =3D NULL; g_autofree char *drvName =3D NULL; - int stub =3D VIR_PCI_STUB_DRIVER_NONE; + virPCIStubDriver drvType; =20 - if (virPCIDeviceGetDriverPathAndName(pci, &drvPath, &drvName) < 0) + if (virPCIDeviceGetDriverNameAndType(pci, &drvName, &drvType) < 0) goto cleanup; =20 - if (drvName) - stub =3D virPCIStubDriverTypeFromString(drvName); - - if (stub =3D=3D VIR_PCI_STUB_DRIVER_VFIO || - STREQ_NULLABLE(drvName, "nvme")) + if (drvType =3D=3D VIR_PCI_STUB_DRIVER_VFIO || STREQ_NULLABLE(drvN= ame, "nvme")) continue; =20 VIR_WARN("Suspicious NVMe disk assignment. PCI device " diff --git a/src/util/virpci.c b/src/util/virpci.c index 7800966963..8b714d3ddc 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -37,6 +37,7 @@ #include "virstring.h" #include "viralloc.h" #include "virpcivpd.h" +#include "vircommand.h" =20 VIR_LOG_INIT("util.pci"); =20 @@ -277,6 +278,84 @@ virPCIDeviceGetDriverPathAndName(virPCIDevice *dev, ch= ar **path, char **name) } =20 =20 +/** + * virPCIDeviceGetDriverNameAndType: + * @dev: virPCIDevice object to examine + * @drvName: returns name of driver bound to this device (if any) + * @drvType: returns type of driver if it is a known stub driver type + * + * Find the name of the driver bound to @dev (if any) and the type of + * the driver if it is a known/recognized "stub" driver (based on the + * name). If the name doesn't match one of the virPCIStubDrivers + * exactly, check the output of "modinfo vfio-pci" to see if + * "vfio_pci" is included in the driver's list of aliases; if so, then + * this driver has all the functionality of the basic vfio_pci driver, + * so it should be considered of the type VIR_PCI_STUB_DRIVER_VFIO. + * + * Return 0 on success, -1 on failure. If -1 is returned, then an error + * message has been logged. + */ +int +virPCIDeviceGetDriverNameAndType(virPCIDevice *dev, + char **drvName, + virPCIStubDriver *drvType) +{ + g_autofree char *drvPath =3D NULL; + g_autoptr(virCommand) cmd =3D NULL; + g_autofree char *output =3D NULL; + g_autoptr(GRegex) regex =3D NULL; + g_autoptr(GError) err =3D NULL; + g_autoptr(GMatchInfo) info =3D NULL; + int exit; + int tmpType; + + if (virPCIDeviceGetDriverPathAndName(dev, &drvPath, drvName) < 0) + return -1; + + if (!*drvName) { + *drvType =3D VIR_PCI_STUB_DRIVER_NONE; + return 0; + } + + tmpType =3D virPCIStubDriverTypeFromString(*drvName); + + if (tmpType > VIR_PCI_STUB_DRIVER_NONE) { + *drvType =3D tmpType; + return 0; /* exact match of a known driver name (or no name) */ + } + + /* Check the output of "modinfo $drvName" to see if it has + * "vfio_pci" as an alias. If it does, then this driver should + * also be considered as a vfio-pci driver, because it implements + * all the functionality of the basic vfio-pci (plus additional + * device-specific stuff). + */ + + cmd =3D virCommandNewArgList(MODINFO, *drvName, NULL); + virCommandSetOutputBuffer(cmd, &output); + if (virCommandRun(cmd, &exit) < 0) + return -1; + + regex =3D g_regex_new("^alias: +vfio_pci:", G_REGEX_MULTILINE, 0, &err= ); + if (!regex) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Failed to compile regex %s"), err->message); + return -1; + } + + g_regex_match(regex, output, 0, &info); + if (g_match_info_matches(info)) { + VIR_DEBUG("Driver %s is a vfio_pci driver", *drvName); + *drvType =3D VIR_PCI_STUB_DRIVER_VFIO; + } else { + VIR_DEBUG("Driver %s is NOT a vfio_pci driver", *drvName); + *drvType =3D VIR_PCI_STUB_DRIVER_NONE; + } + + return 0; +} + + static int virPCIDeviceConfigOpenInternal(virPCIDevice *dev, bool readonly, bool fata= l) { @@ -1004,8 +1083,8 @@ virPCIDeviceReset(virPCIDevice *dev, virPCIDeviceList *activeDevs, virPCIDeviceList *inactiveDevs) { - g_autofree char *drvPath =3D NULL; g_autofree char *drvName =3D NULL; + virPCIStubDriver drvType; int ret =3D -1; int fd =3D -1; int hdrType =3D -1; @@ -1032,15 +1111,16 @@ virPCIDeviceReset(virPCIDevice *dev, * reset it whenever appropriate, so doing it ourselves would just * be redundant. */ - if (virPCIDeviceGetDriverPathAndName(dev, &drvPath, &drvName) < 0) + if (virPCIDeviceGetDriverNameAndType(dev, &drvName, &drvType) < 0) goto cleanup; =20 - if (virPCIStubDriverTypeFromString(drvName) =3D=3D VIR_PCI_STUB_DRIVER= _VFIO) { - VIR_DEBUG("Device %s is bound to vfio-pci - skip reset", - dev->name); + if (drvType =3D=3D VIR_PCI_STUB_DRIVER_VFIO) { + + VIR_DEBUG("Device %s is bound to %s - skip reset", dev->name, drvN= ame); ret =3D 0; goto cleanup; } + VIR_DEBUG("Resetting device %s", dev->name); =20 if ((fd =3D virPCIDeviceConfigOpenWrite(dev)) < 0) diff --git a/src/util/virpci.h b/src/util/virpci.h index 4d9193f24e..0532b90f90 100644 --- a/src/util/virpci.h +++ b/src/util/virpci.h @@ -280,6 +280,9 @@ int virPCIDeviceRebind(virPCIDevice *dev); int virPCIDeviceGetDriverPathAndName(virPCIDevice *dev, char **path, char **name); +int virPCIDeviceGetDriverNameAndType(virPCIDevice *dev, + char **drvName, + virPCIStubDriver *drvType); =20 int virPCIDeviceIsPCIExpress(virPCIDevice *dev); int virPCIDeviceHasPCIExpressLink(virPCIDevice *dev); --=20 2.35.3