From nobody Sun Feb 8 04:11:41 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1564078221; cv=none; d=zoho.com; s=zohoarc; b=WItIkcA7F5I26fHRE63LKLRiJvZi3IA414xMEuybhqzEla2jvesEfq4BYT5rUKszkY+cRFTXg9vXC/R9q/Y+qboDwC7ez3Qra4MQ9d0cRrxX5LV7bleFa3Jm8AcULNDwD7KOPRFuRXXTTwxdISyQ4u+s0ehi9zl5TxYXEdK7Zfs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1564078221; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=pu2xdew1Bd9tGZe/iIhsNWLnPnldkcA3TQDxmOxapNo=; b=S2PXa9ctZKsU5HULiP3zVT32SBpJk5s0/0EzS2UEDnqZ0QuJqcuQkhCfLiT6+POC7AUitL29I69es3Winl/GfhwmUX4U9tYEMRMd2cDP/cipW3dsQCahSWveFLa9uxPXCEIo62SroipmjPQZPau/gIL4sOXxaT4BFO5cu3PyzqY= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1564078221733467.6092463748398; Thu, 25 Jul 2019 11:10:21 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3B22644BD1; Thu, 25 Jul 2019 18:10:20 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 10135379C; Thu, 25 Jul 2019 18:10:20 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B9C353CBC; Thu, 25 Jul 2019 18:10:19 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x6PIA9Ve031544 for ; Thu, 25 Jul 2019 14:10:09 -0400 Received: by smtp.corp.redhat.com (Postfix) id 2BFCF60C18; Thu, 25 Jul 2019 18:10:09 +0000 (UTC) Received: from mx1.redhat.com (ext-mx20.extmail.prod.ext.phx2.redhat.com [10.5.110.49]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 25C2760C61 for ; Thu, 25 Jul 2019 18:10:07 +0000 (UTC) Received: from mail-qt1-f196.google.com (mail-qt1-f196.google.com [209.85.160.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BA97D3086272 for ; Thu, 25 Jul 2019 18:10:06 +0000 (UTC) Received: by mail-qt1-f196.google.com with SMTP id h21so49942439qtn.13 for ; Thu, 25 Jul 2019 11:10:06 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:431:c7c6:30ed:e8ca:caf:6199:5dc7]) by smtp.gmail.com with ESMTPSA id e8sm21639968qkn.95.2019.07.25.11.10.04 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 25 Jul 2019 11:10:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=87mZdswjKO9+uCqN0YTAwsUyM5wPp2iF1vHp767fhPM=; b=PQ32XjMOcpIQ21hcN4W+cob3KVI5uf3GMLZpvQrzP9cbr3b55O8WxFNM3zalCTTq1y mY/iHLJoB0UxY1h1Ujksvo2c/UBRrwqTs6o1appjrTwmMRMgAWNq5PBSZKWOgI7om4+w 2H9pCCMuaJtRL390JpMs4MbRuSomO11C3Jxh5vk999YGnOSxz/ihTQQG04prQ7aCPlXB Zwh9i/FqLyBek1I51N3quNZQ0YcwvRD5Kkcd7xglm1bjnfDH1ssBsVKzUeZOk19RGbyk g3FdmbZHOGy1QZVd6arZg5ewUfym7SxoNVggpEPt2vgyILuPcSegqDrUHZ3LDJ2UsCH+ YDzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=87mZdswjKO9+uCqN0YTAwsUyM5wPp2iF1vHp767fhPM=; b=T25r73D5sBgPj0VxLYJp8JRwwywLYBGpDSpSb9uibAQNHer+QYonzYfCOm4TCIvuEb Qg7jqLNT0smOIIF57ICz7587Wx4BEtBIwoXUeNj0F8NNLBghP8mK6Cw2QWr+VlWTgICe a26ZK5L/JM3XJJRyEu/B7byadPpZv+/KuPFynehmVyWzUX8neIeQ/66wjM9USRv17/Ub bb1FFLx88GoIiGcdhgdtM7LDmScjIrpQtBoQR9k0ZO220EyMM4Q4oQJxBrX+HYlhSSUH C7ALaUlQhs1q1AHVSLUbUd68TwCkdP+JsQsNun99xEE+K7QWiXyqeS50N1o9D6NZZKqo BCxQ== X-Gm-Message-State: APjAAAUY7URKfz35H7IJUdFa0MIeI5GcZaZk9vndpZ5jp/W+GANrle66 STpLsoTRlzYtI6q/Onl0N4g0VK1O X-Google-Smtp-Source: APXvYqzHyS0bWKX3JHSvlyf2X6K1TErD/Z8iHLKAnIUoCku7qXENr67jmGMXf4JFJAzIGr3cpuOccw== X-Received: by 2002:aed:306e:: with SMTP id 101mr62349318qte.102.1564078205850; Thu, 25 Jul 2019 11:10:05 -0700 (PDT) From: Daniel Henrique Barboza To: libvir-list@redhat.com Date: Thu, 25 Jul 2019 15:09:47 -0300 Message-Id: <20190725180951.8926-4-danielhb413@gmail.com> In-Reply-To: <20190725180951.8926-1-danielhb413@gmail.com> References: <20190725180951.8926-1-danielhb413@gmail.com> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Thu, 25 Jul 2019 18:10:06 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Thu, 25 Jul 2019 18:10:06 +0000 (UTC) for IP:'209.85.160.196' DOMAIN:'mail-qt1-f196.google.com' HELO:'mail-qt1-f196.google.com' FROM:'danielhb413@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.153 (DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS) 209.85.160.196 mail-qt1-f196.google.com 209.85.160.196 mail-qt1-f196.google.com X-Scanned-By: MIMEDefang 2.84 on 10.5.110.49 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: libvir-list@redhat.com Cc: Daniel Henrique Barboza Subject: [libvirt] [PATCH v2 3/7] util/virhostdev: enhance VFIO device is in use detection X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 25 Jul 2019 18:10:20 +0000 (UTC) X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The current virHostdevPreparePCIDevices code fails to detect an unmanaged VFIO device that is in the activePCIHostdevs, and active in the same domain name as dom_name, as a device in use. Considering a call to this function, when activePCIHostdevs has a VFIO deviceA in the list, and deviceA is active in domain 'dom_name', this is what happens: - At step 1, the code goes to the 'if (usesVFIO)' block. All devices in the same IOMMU group of deviceA are used as argument of virHostdevIsPCINodeDeviceUsed, via the IOMMUGroupIterate function; - Inside virHostdevIsPCINodeDeviceUsed, an 'usesVFIO' verification is made, following to an 'iommu_owner' jump that sets ret=3D0. This will happen to all devices of the IOMMU group that belongs to the same domain as dom_name, including deviceA; - Step 2 starts, thinking that all devices inside hostdevs are available, which is not the case. This error was detected when changing tests/virhostdev.c to use vfio-pci instead of pci-stub (a change that will follow this one). The side effect observed was a test failure in testVirHostdevPreparePCIHostdevs_unmanaged, result of the behavior mentioned above: Unexpected count of items in mgr->inactivePCIHostdevs: 1, expecting 0 This patch fixes virHostdevIsPCINodeDeviceUsed to consider the case of a VFIO device that is already active in the domain. First, check if the device is in the activePCIHostdev list and bail immediately if true. Otherwise, in case of VFIO, check for IOMMU group ownership of the domain. This is done by a new callback function for IOMMUGroupIterate. If the VFIO device isn't in the active list and its domain has ownership of the IOMMU, then it is unused. Signed-off-by: Daniel Henrique Barboza --- src/util/virhostdev.c | 74 +++++++++++++++++++++++++++++++++---------- 1 file changed, 57 insertions(+), 17 deletions(-) diff --git a/src/util/virhostdev.c b/src/util/virhostdev.c index a3647a6cf4..35be8fede1 100644 --- a/src/util/virhostdev.c +++ b/src/util/virhostdev.c @@ -55,6 +55,46 @@ struct virHostdevIsPCINodeDeviceUsedData { const bool usesVFIO; }; =20 + +static virPCIDevicePtr +virHostdevFindActivePCIDevWithAddr(virPCIDeviceAddressPtr devAddr, + virHostdevManagerPtr mgr) +{ + virPCIDevicePtr ret =3D NULL; + + ret =3D virPCIDeviceListFindByIDs(mgr->activePCIHostdevs, + devAddr->domain, devAddr->bus, + devAddr->slot, devAddr->function); + + return ret; +} + +/* Callback to be used inside virHostdevIsPCINodeDeviceUsed to check + * for IOMMU ownership of a domain given by helperData->domainName. */ +static int +virHostdevDomainHasIOMMUOwnershipCb(virPCIDeviceAddressPtr devAddr, void *= opaque) +{ + struct virHostdevIsPCINodeDeviceUsedData *helperData =3D opaque; + virPCIDevicePtr actual; + int ret =3D 0; + + actual =3D virHostdevFindActivePCIDevWithAddr(devAddr, helperData->mgr= ); + if (!actual) + return ret; + + if (helperData->usesVFIO) { + const char *actual_drvname =3D NULL; + const char *actual_domname =3D NULL; + virPCIDeviceGetUsedBy(actual, &actual_drvname, &actual_domname); + + if ((actual_domname && helperData->domainName) && + (STRNEQ(actual_domname, helperData->domainName))) + ret =3D -1; + } + + return ret; +} + /* This module makes heavy use of bookkeeping lists contained inside a * virHostdevManager instance to keep track of the devices' status. To make * it easy to spot potential ownership errors when moving devices from one @@ -82,19 +122,13 @@ static int virHostdevIsPCINodeDeviceUsed(virPCIDeviceA= ddressPtr devAddr, void *o int ret =3D -1; struct virHostdevIsPCINodeDeviceUsedData *helperData =3D opaque; =20 - actual =3D virPCIDeviceListFindByIDs(helperData->mgr->activePCIHostdev= s, - devAddr->domain, devAddr->bus, - devAddr->slot, devAddr->function); + actual =3D virHostdevFindActivePCIDevWithAddr(devAddr, helperData->mgr= ); + if (actual) { const char *actual_drvname =3D NULL; const char *actual_domname =3D NULL; virPCIDeviceGetUsedBy(actual, &actual_drvname, &actual_domname); =20 - if (helperData->usesVFIO && - (actual_domname && helperData->domainName) && - (STREQ(actual_domname, helperData->domainName))) - goto iommu_owner; - if (actual_drvname && actual_domname) virReportError(VIR_ERR_OPERATION_INVALID, _("PCI device %s is in use by " @@ -105,9 +139,20 @@ static int virHostdevIsPCINodeDeviceUsed(virPCIDeviceA= ddressPtr devAddr, void *o virReportError(VIR_ERR_OPERATION_INVALID, _("PCI device %s is in use"), virPCIDeviceGetName(actual)); + goto cleanup; } - iommu_owner: + + /* For VFIO devices, the domain helperData->domainName must have owner= ship + * of the IOMMU group that contains devAddr. Otherwise, even if the de= vAddr + * is not in use, another device of that IOMMU group is in use by anot= herdomain, + * forbidding devAddr to be used in helperData->domainName. */ + if (helperData->usesVFIO) + if (virPCIDeviceAddressIOMMUGroupIterate(devAddr, + virHostdevDomainHasIOMMUO= wnershipCb, + helperData) < 0) + goto cleanup; + ret =3D 0; cleanup: return ret; @@ -673,17 +718,12 @@ virHostdevPreparePCIDevices(virHostdevManagerPtr mgr, /* The device is in use by other active domain if * the dev is in list activePCIHostdevs. VFIO devices * belonging to same iommu group can't be shared - * across guests. + * across guests. virHostdevIsPCINodeDeviceUsedData handles + * both cases. */ devAddr =3D virPCIDeviceGetAddress(pci); - if (usesVFIO) { - if (virPCIDeviceAddressIOMMUGroupIterate(devAddr, - virHostdevIsPCINodeDe= viceUsed, - &data) < 0) - goto cleanup; - } else if (virHostdevIsPCINodeDeviceUsed(devAddr, &data)) { + if (virHostdevIsPCINodeDeviceUsed(devAddr, &data)) goto cleanup; - } } =20 /* Step 1.5: For non-802.11Qbh SRIOV network devices, save the --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list