From nobody Sun Feb 8 06:54:17 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1561577952; cv=none; d=zoho.com; s=zohoarc; b=nR8dlCcktUCHZXwhjgeWB0cLAkkcVxPeRg+Awn6RaovVe7WlTJ4kzifg7SwAFjO640yqliE8pl1KWu51F30VPCxd6DhQkKq//J6jrjSRT1qlMsaxAuTiEU+Z0La08w/CtQJEDPGtSSYMftZ6SDqI147NoiVpb/pe+N3veVW7kAk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1561577952; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=F27awY90gc6WNM+Fvf3dKnx/iBztgemLcv1s+Brlws8=; b=a9hpwYjMCQf3GLqYgQ57ZDm8KAl8giMdY7VdjoWM2cuD5V/NtjYmDfC48tHwVWKj1kJg767aMNN4t8S31F+5IXi3KND0C6fzYZTNMX40jANGsqKGr0wM8fx1Zf+lBSPhTASmzGW8RWxYd1rxOq8JdouHfWlDV0T6QzQ5W+0DLdc= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1561577952030731.076454629486; Wed, 26 Jun 2019 12:39:12 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E58DD66995; Wed, 26 Jun 2019 19:39:07 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B27D9608C1; Wed, 26 Jun 2019 19:39:06 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6D540206D2; Wed, 26 Jun 2019 19:39:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x5QJaXtU013059 for ; Wed, 26 Jun 2019 15:36:33 -0400 Received: by smtp.corp.redhat.com (Postfix) id F25765D9D6; Wed, 26 Jun 2019 19:36:32 +0000 (UTC) Received: from mx1.redhat.com (ext-mx04.extmail.prod.ext.phx2.redhat.com [10.5.110.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EAAEE5D9C6; Wed, 26 Jun 2019 19:36:32 +0000 (UTC) Received: from mail-qt1-f195.google.com (mail-qt1-f195.google.com [209.85.160.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 96FFD8830C; Wed, 26 Jun 2019 19:36:21 +0000 (UTC) Received: by mail-qt1-f195.google.com with SMTP id x47so3682833qtk.11; Wed, 26 Jun 2019 12:36:21 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:18:805:41b2:608e:c97b:9c84:d48f]) by smtp.gmail.com with ESMTPSA id k40sm10706658qta.50.2019.06.26.12.36.19 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 26 Jun 2019 12:36:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lncVBjV6lVg71V6AJRDhmReAL3GIHke3KXetA3IJ5Vg=; b=obOZzNtrEvvKptnc/C4LjbSUL1JdJ8Pax4nNKEj90bfu4iEvRzCc6LUw6imcMdvVz7 F026vN29hst8HX+4YbIUHb4pL1TsvqbvbMlTWL9H4dEtpFCPfWESoe/01ARshEq5Mj3K Y2eLCbcBzYweNFmTbiGRH6BhbXu+j3bmCgX379ncG/70afSSYubgfGep51p3RyZmpP3E NXUn/ILP/vXcCWbUxf/3qDPCaoaLy+Z7jApSNTHcucpw34OSLCZ4PlfeOlMpETAh3wng dgibwErIo+GCoi68YOCAAn+hTalD9C4ZmoonbryW05j2IhlQIbrqJRt/t8VEcppkIUFs rwSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lncVBjV6lVg71V6AJRDhmReAL3GIHke3KXetA3IJ5Vg=; b=ED74IkGSl1fW7S0ZB1Wka0SkP2FFWnzmQiRdbFeOKTkh92V8ZAxvtmr6OC03hZECbW la6IbYFl7UBNpq2ckQXEx8+uydrjG8G9FueVzoIqL3mHg4wmhMC5nYjos4dqnxNfyoUE xP0ogIDrA6UD38kzrEk7kNcVolUeYfrYSWvA4HFRMyE3Tz6gsJFBtDI0EcvLTNHwyW5K VOxpJK943LAWqeMLe6sxk2ju1rrcHYt6IbRhIIF7W8CTekXpNFvaFx42jRLzhUWMGV4/ imJJojEKFciLTlh+MbUzy6/lNrKGkNM2diGcPmeIJ/Hb3v54Hz4w3Lc9JEQVKZCsSo5o SVeA== X-Gm-Message-State: APjAAAVu/52sHApB8Uq8CRASFCW92dtAeFkC5hcd1aI86WyA9+a8uoH3 tKaINZFEL2R0f3zuH2oDuPyjds5miiE= X-Google-Smtp-Source: APXvYqwf11OU1FVpJU6kEUvJXvrla1OK5ZOkcbleecRxd/CXzWNIbzpfNqC2iqJRjGl1UFJrLv1Emg== X-Received: by 2002:a0c:b084:: with SMTP id o4mr5098474qvc.227.1561577780715; Wed, 26 Jun 2019 12:36:20 -0700 (PDT) From: Daniel Henrique Barboza To: libvir-list@redhat.com Date: Wed, 26 Jun 2019 16:35:55 -0300 Message-Id: <20190626193559.24431-7-danielhb413@gmail.com> In-Reply-To: <20190626193559.24431-1-danielhb413@gmail.com> References: <20190626193559.24431-1-danielhb413@gmail.com> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 26 Jun 2019 19:36:21 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 26 Jun 2019 19:36:21 +0000 (UTC) for IP:'209.85.160.195' DOMAIN:'mail-qt1-f195.google.com' HELO:'mail-qt1-f195.google.com' FROM:'danielhb413@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.141 (DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS) 209.85.160.195 mail-qt1-f195.google.com 209.85.160.195 mail-qt1-f195.google.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.28 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Cc: mprivozn@redhat.com, Daniel Henrique Barboza , laine@laine.org Subject: [libvirt] [PATCH v1 06/10] util/virhostdev: enhance VFIO device is in use detection X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 26 Jun 2019 19:39:10 +0000 (UTC) X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The current virHostdevPreparePCIDevices code fails to detect an unmanaged VFIO device that is in the activePCIHostdevs, and active in the same domain name as dom_name, as a device in use. Considering a call to this function, when activePCIHostdevs has a VFIO deviceA in the list, and deviceA is active in domain 'dom_name', this is what happens: - At step 1, the code goes to the 'if (usesVFIO)' block. All devices in the same IOMMU group of deviceA are used as argument of virHostdevIsPCINodeDeviceUsed, via the IOMMUGroupIterate function; - Inside virHostdevIsPCINodeDeviceUsed, an 'usesVFIO' verification is made, following to an 'iommu_owner' jump that sets ret=3D0. This will happen to all devices of the IOMMU group that belongs to the same domain as dom_name, including deviceA; - Step 2 starts, thinking that all devices inside hostdevs are available, which is not the case. This error was detected when changing tests/virhostdev.c to use vfio-pci instead of pci-stub (a change that will follow this one). The side effect observed was a test failure in testVirHostdevPreparePCIHostdevs_unmanaged, result of the behavior mentioned above: Unexpected count of items in mgr->inactivePCIHostdevs: 1, expecting 0 This patch fixes virHostdevIsPCINodeDeviceUsed to consider the case of a VFIO device that is already active in the domain. First, check if the device is in the activePCIHostdev list and bail immediately if true. Otherwise, in case of VFIO, check for IOMMU group ownership of the domain. This is done by a new callback function for IOMMUGroupIterate. If the VFIO device isn't in the active list and its domain has ownership of the IOMMU, then it is unused. Signed-off-by: Daniel Henrique Barboza --- src/util/virhostdev.c | 74 +++++++++++++++++++++++++++++++++---------- 1 file changed, 57 insertions(+), 17 deletions(-) diff --git a/src/util/virhostdev.c b/src/util/virhostdev.c index a3647a6cf4..35be8fede1 100644 --- a/src/util/virhostdev.c +++ b/src/util/virhostdev.c @@ -55,6 +55,46 @@ struct virHostdevIsPCINodeDeviceUsedData { const bool usesVFIO; }; =20 + +static virPCIDevicePtr +virHostdevFindActivePCIDevWithAddr(virPCIDeviceAddressPtr devAddr, + virHostdevManagerPtr mgr) +{ + virPCIDevicePtr ret =3D NULL; + + ret =3D virPCIDeviceListFindByIDs(mgr->activePCIHostdevs, + devAddr->domain, devAddr->bus, + devAddr->slot, devAddr->function); + + return ret; +} + +/* Callback to be used inside virHostdevIsPCINodeDeviceUsed to check + * for IOMMU ownership of a domain given by helperData->domainName. */ +static int +virHostdevDomainHasIOMMUOwnershipCb(virPCIDeviceAddressPtr devAddr, void *= opaque) +{ + struct virHostdevIsPCINodeDeviceUsedData *helperData =3D opaque; + virPCIDevicePtr actual; + int ret =3D 0; + + actual =3D virHostdevFindActivePCIDevWithAddr(devAddr, helperData->mgr= ); + if (!actual) + return ret; + + if (helperData->usesVFIO) { + const char *actual_drvname =3D NULL; + const char *actual_domname =3D NULL; + virPCIDeviceGetUsedBy(actual, &actual_drvname, &actual_domname); + + if ((actual_domname && helperData->domainName) && + (STRNEQ(actual_domname, helperData->domainName))) + ret =3D -1; + } + + return ret; +} + /* This module makes heavy use of bookkeeping lists contained inside a * virHostdevManager instance to keep track of the devices' status. To make * it easy to spot potential ownership errors when moving devices from one @@ -82,19 +122,13 @@ static int virHostdevIsPCINodeDeviceUsed(virPCIDeviceA= ddressPtr devAddr, void *o int ret =3D -1; struct virHostdevIsPCINodeDeviceUsedData *helperData =3D opaque; =20 - actual =3D virPCIDeviceListFindByIDs(helperData->mgr->activePCIHostdev= s, - devAddr->domain, devAddr->bus, - devAddr->slot, devAddr->function); + actual =3D virHostdevFindActivePCIDevWithAddr(devAddr, helperData->mgr= ); + if (actual) { const char *actual_drvname =3D NULL; const char *actual_domname =3D NULL; virPCIDeviceGetUsedBy(actual, &actual_drvname, &actual_domname); =20 - if (helperData->usesVFIO && - (actual_domname && helperData->domainName) && - (STREQ(actual_domname, helperData->domainName))) - goto iommu_owner; - if (actual_drvname && actual_domname) virReportError(VIR_ERR_OPERATION_INVALID, _("PCI device %s is in use by " @@ -105,9 +139,20 @@ static int virHostdevIsPCINodeDeviceUsed(virPCIDeviceA= ddressPtr devAddr, void *o virReportError(VIR_ERR_OPERATION_INVALID, _("PCI device %s is in use"), virPCIDeviceGetName(actual)); + goto cleanup; } - iommu_owner: + + /* For VFIO devices, the domain helperData->domainName must have owner= ship + * of the IOMMU group that contains devAddr. Otherwise, even if the de= vAddr + * is not in use, another device of that IOMMU group is in use by anot= herdomain, + * forbidding devAddr to be used in helperData->domainName. */ + if (helperData->usesVFIO) + if (virPCIDeviceAddressIOMMUGroupIterate(devAddr, + virHostdevDomainHasIOMMUO= wnershipCb, + helperData) < 0) + goto cleanup; + ret =3D 0; cleanup: return ret; @@ -673,17 +718,12 @@ virHostdevPreparePCIDevices(virHostdevManagerPtr mgr, /* The device is in use by other active domain if * the dev is in list activePCIHostdevs. VFIO devices * belonging to same iommu group can't be shared - * across guests. + * across guests. virHostdevIsPCINodeDeviceUsedData handles + * both cases. */ devAddr =3D virPCIDeviceGetAddress(pci); - if (usesVFIO) { - if (virPCIDeviceAddressIOMMUGroupIterate(devAddr, - virHostdevIsPCINodeDe= viceUsed, - &data) < 0) - goto cleanup; - } else if (virHostdevIsPCINodeDeviceUsed(devAddr, &data)) { + if (virHostdevIsPCINodeDeviceUsed(devAddr, &data)) goto cleanup; - } } =20 /* Step 1.5: For non-802.11Qbh SRIOV network devices, save the --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list