From nobody Tue Nov 26 17:29:19 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1570482852; cv=none; d=zoho.com; s=zohoarc; b=S64gRRnqmVClo/Y6g8uywP7nDf+eO1kYI0SegLYX+xdMm/WETwPxSvHqW2McpRWxPxkyOMJNWAOhTGVAinvxMmxzaIE86ojggL/UDtb9FJtBPdYOmqNOQvoCBNayUoyyQC2uPrmJskAk+JRXvE3xkquvfVCYxqwTS7INciPkRmM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1570482852; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=xtUd5MBWU6jKW+vwJYgXs8SDNyxJWSgmaNf7ZfhqG5s=; b=MbvCoKlSCOHCQ36yx4eryafKFvYm1EczDltBg4ZVYICjveogNJRU8kKe1Cc7hQXzFinCKWhBC1+Z/VBBXGBM7MezG4bR9fc1EfbMMXio63Gi8Xfw1t5MqmVwo6mnombeII6PspOXcdX/K4lOw9rRo9gkSQHgNzPUPEyvvatyq10= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1570482852015569.1591435072257; Mon, 7 Oct 2019 14:14:12 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 16E08302245A; Mon, 7 Oct 2019 21:14:10 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E568A60C5E; Mon, 7 Oct 2019 21:14:09 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id A342E1803B48; Mon, 7 Oct 2019 21:14:09 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x97LBvmf012234 for ; Mon, 7 Oct 2019 17:11:58 -0400 Received: by smtp.corp.redhat.com (Postfix) id EF9535EE1D; Mon, 7 Oct 2019 21:11:57 +0000 (UTC) Received: from mx1.redhat.com (ext-mx20.extmail.prod.ext.phx2.redhat.com [10.5.110.49]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E97BD5D772 for ; Mon, 7 Oct 2019 21:11:55 +0000 (UTC) Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5CDC5302245A for ; Mon, 7 Oct 2019 21:11:54 +0000 (UTC) Received: by mail-qk1-f176.google.com with SMTP id h126so14081030qke.10 for ; Mon, 07 Oct 2019 14:11:54 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:431:c7c6:e8e7:3739:3e28:be31:8030]) by smtp.gmail.com with ESMTPSA id i30sm11333409qte.27.2019.10.07.14.11.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Oct 2019 14:11:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4eeHXQ75aFKz3EuSJMlNDQnpPrNjQcyheeXaKroYmLo=; b=jeJ8FdB06I0jm6ej1Oe9uo2QizEL/Wy2w4p5mxNOOMKoq6RO02uu5/Otp+zsas4chq CNAQgFQpTcPiFTtuRsg+ADOnzcwwmle6LfcdbnExcjwRAXbrX3Gshd7r0a9A1nb0WI4S oJbKb3fHqSuUqY+lKRHkQGTkLCBZDVhLy0GFCBMzfM+LxrMzuwUTV33LE+vTR2+AoJjP OqGH7zsP26w3/sWlqxktb2zxGX7DdjU/Th60LmiLijVfpojrQ8GRsamyl9YpvQ6HeugJ mS/2GRyi6CgSC6LoZwq+WhmDbdfSnFXfdP3Vi4VwnR/WFo0ualcR8mHdtutBiGqw7XnC 0QLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4eeHXQ75aFKz3EuSJMlNDQnpPrNjQcyheeXaKroYmLo=; b=dthcQYxqi/eNKNhRkrPrvz1pCYIit/UCgeTdHJbN1g3+zB74rstLi0EBiPa2f7H2Rj tn5tEyI4/JcgzcH8LJi0rEZiXOyyDTnovr2SVm165ywzA3YnffanEkb/dxExYfSZHSnD ObQq5gU92cIMe6xDuIju8IRpnia3SD4ioqiMIo4j2EVLn9CAor2l8HqO7wBTQwrbqYUg oevLl3+nkpZElMhMuhDmSdOSy0/D4H/BfYLyetWPdHVymFLnoBWxy5OaCz6hP1R/9ox5 OelDe4i3NUoj7yGEgQljml0zTe2ft2bjDy5Sl7aRtAV5NW0NsgGYMGHAz5Lv3G09RsUp NeMw== X-Gm-Message-State: APjAAAXcS0yuf3EDjZ6XpUs+3HgUeYeprLm6BKnKK6ZpJYZxcka0NWbA KJVHf3Di4XQ7Iul3SK2tfYjnpUOl X-Google-Smtp-Source: APXvYqz9uIICTX5OrKOs3dBsPO2XNu26yRJN0w2sk++LYRQyTi4M2jIWxTQ3a1XZjKLbWz/90mTOVg== X-Received: by 2002:a37:a849:: with SMTP id r70mr25508400qke.37.1570482713345; Mon, 07 Oct 2019 14:11:53 -0700 (PDT) From: Daniel Henrique Barboza To: libvir-list@redhat.com Date: Mon, 7 Oct 2019 18:11:36 -0300 Message-Id: <20191007211136.16730-5-danielhb413@gmail.com> In-Reply-To: <20191007211136.16730-1-danielhb413@gmail.com> References: <20191007211136.16730-1-danielhb413@gmail.com> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Mon, 07 Oct 2019 21:11:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Mon, 07 Oct 2019 21:11:54 +0000 (UTC) for IP:'209.85.222.176' DOMAIN:'mail-qk1-f176.google.com' HELO:'mail-qk1-f176.google.com' FROM:'danielhb413@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.15 (DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS) 209.85.222.176 mail-qk1-f176.google.com 209.85.222.176 mail-qk1-f176.google.com X-Scanned-By: MIMEDefang 2.84 on 10.5.110.49 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Cc: Daniel Henrique Barboza , laine@laine.org Subject: [libvirt] [PATCH 4/4] virhostdev: all functions of a PCI multifunction dev must be added X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Mon, 07 Oct 2019 21:14:10 +0000 (UTC) X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This is the last piece for a full fledged PCI multifunction partial assignment support. Now that we have a defined XML format and QEMU is ignoring the existence of the hostdev that isn't being assigned, let's enforce that all functions must be declared in the domain XML for such devices. This will make sure that we'll always detach/re-attach the whole IOMMU and will force the user to be aware of what is going to be detached from the host. Signed-off-by: Daniel Henrique Barboza --- src/util/virhostdev.c | 64 ++++++++++++++++++- ...ostdev-pci-multifunction-partial-fail.args | 31 +++++++++ ...hostdev-pci-multifunction-partial-fail.xml | 35 ++++++++++ tests/qemuxml2argvtest.c | 4 ++ 4 files changed, 132 insertions(+), 2 deletions(-) create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction-partia= l-fail.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction-partia= l-fail.xml diff --git a/src/util/virhostdev.c b/src/util/virhostdev.c index 1aa8e9729d..a85860b34f 100644 --- a/src/util/virhostdev.c +++ b/src/util/virhostdev.c @@ -56,6 +56,13 @@ struct virHostdevIsPCINodeDeviceUsedData { bool usesVFIO; }; =20 +struct virHostdevIsAllIOMMUGroupUsedData { + virPCIDeviceListPtr pcidevs; + const char *domainName; + const char *deviceName; +}; + + /* This module makes heavy use of bookkeeping lists contained inside a * virHostdevManager instance to keep track of the devices' status. To make * it easy to spot potential ownership errors when moving devices from one @@ -114,6 +121,26 @@ static int virHostdevIsPCINodeDeviceUsed(virPCIDeviceA= ddressPtr devAddr, void *o return ret; } =20 +static int virHostdevIsAllIOMMUGroupUsed(virPCIDeviceAddressPtr devAddr, v= oid *opaque) +{ + struct virHostdevIsAllIOMMUGroupUsedData *helperData =3D opaque; + virPCIDevicePtr actual; + + actual =3D virPCIDeviceListFindByIDs(helperData->pcidevs, + devAddr->domain, devAddr->bus, + devAddr->slot, devAddr->function); + if (actual) { + return 0; + } else { + virReportError(VIR_ERR_OPERATION_INVALID, + _("All devices of the same IOMMU group as the " + "multifunction PCI device %s must belong to " + "domain %s"), + helperData->deviceName, helperData->domainName); + return -1; + } +} + static int virHostdevManagerOnceInit(void) { if (!VIR_CLASS_NEW(virHostdevManager, virClassForObject())) @@ -714,9 +741,10 @@ virHostdevPreparePCIDevices(virHostdevManagerPtr mgr, unsigned int flags) { VIR_AUTOUNREF(virPCIDeviceListPtr) pcidevs =3D NULL; + VIR_AUTOFREE(unsigned int *) searchedIOMMUs =3D NULL; int last_processed_hostdev_vf =3D -1; - size_t i; - int ret =3D -1; + size_t i, j; + int ret =3D -1, nSearchedIOMMUs =3D 0; virPCIDeviceAddressPtr devAddr =3D NULL; =20 if (!nhostdevs) @@ -725,6 +753,9 @@ virHostdevPreparePCIDevices(virHostdevManagerPtr mgr, if (!(pcidevs =3D virHostdevGetPCIHostDeviceList(hostdevs, nhostdevs))) return -1; =20 + if (VIR_ALLOC_N(searchedIOMMUs, virPCIDeviceListCount(pcidevs))) + return -1; + virObjectLock(mgr->activePCIHostdevs); virObjectLock(mgr->inactivePCIHostdevs); =20 @@ -767,6 +798,35 @@ virHostdevPreparePCIDevices(virHostdevManagerPtr mgr, if (virHostdevIsPCINodeDeviceUsed(devAddr, &data)) goto cleanup; =20 + /* Multifunction PCI devices are more restrict than regular + * VFIO devices, since it requires *all* devices from the same + * IOMMU to belong to the same domain - even if not all of + * them are assigned to the guest. + */ + if (virPCIDeviceIsMultifunction(pci)) { + struct virHostdevIsAllIOMMUGroupUsedData helper =3D { + pcidevs, dom_name, virPCIDeviceGetName(pci)}; + int devIOMMUGroup =3D virPCIDeviceAddressGetIOMMUGroupNum(devA= ddr); + bool alreadySearched =3D false; + + for (j =3D 0; j < nSearchedIOMMUs; j++) { + if (devIOMMUGroup =3D=3D searchedIOMMUs[j]) { + alreadySearched =3D true; + break; + } + } + + if (alreadySearched) + continue; + + if (virPCIDeviceAddressIOMMUGroupIterate( + devAddr, virHostdevIsAllIOMMUGroupUsed, &helper) < 0) + goto cleanup; + + searchedIOMMUs[nSearchedIOMMUs++] =3D devIOMMUGroup; + continue; + } + /* VFIO devices belonging to same IOMMU group can't be * shared across guests. Check if that's the case. */ if (usesVFIO) { diff --git a/tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.= args b/tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.args new file mode 100644 index 0000000000..4074968c84 --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.args @@ -0,0 +1,31 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-delete \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-delete/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-delete/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-delete/.config \ +QEMU_AUDIO_DRV=3Dnone \ +/usr/bin/qemu-system-x86_64 \ +-name delete \ +-S \ +-machine pc,accel=3Dkvm,usb=3Doff,dump-guest-core=3Doff \ +-m 256 \ +-realtime mlock=3Doff \ +-smp 4,sockets=3D4,cores=3D1,threads=3D1 \ +-uuid 583a8e8e-f0ce-4f53-89ab-092862148b25 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,path=3D/tmp/lib/domain--1-delete/monitor.= sock,\ +server,nowait \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-usb \ +-device vfio-pci,host=3D0005:90:01.0,id=3Dhostdev0,bus=3Dpci.0,addr=3D0x3 \ +-device vfio-pci,host=3D0005:90:01.1,id=3Dhostdev1,bus=3Dpci.0,addr=3D0x4 \ +-device vfio-pci,host=3D0005:90:01.3,id=3Dhostdev2,bus=3Dpci.0,addr=3D0x5 \ +-device virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.0,addr=3D0x6 diff --git a/tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.= xml b/tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.xml new file mode 100644 index 0000000000..fc3e3bb520 --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.xml @@ -0,0 +1,35 @@ + + delete + 583a8e8e-f0ce-4f53-89ab-092862148b25 + 262144 + 4 + + hvm + + + /usr/bin/qemu-system-x86_64 + + + + + + + + +
+ + + + + +
+ + + + + +
+ + + + diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 434e01308c..462852c690 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1361,6 +1361,10 @@ mymain(void) QEMU_CAPS_KVM, QEMU_CAPS_DEVICE_VFIO_PCI); =20 + DO_TEST("hostdev-pci-multifunction-partial-fail", + QEMU_CAPS_KVM, + QEMU_CAPS_DEVICE_VFIO_PCI); + DO_TEST("serial-file-log", QEMU_CAPS_CHARDEV_FILE_APPEND, QEMU_CAPS_DEVICE_ISA_SERIAL, --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list