From nobody Fri Nov 14 16:37:27 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1589224786; cv=none; d=zohomail.com; s=zohoarc; b=H1tIDLPxBI7krHRadw/LfnUwrSqr0v+uFy+SFsAPvQ3IKvHyQk51MqLevtVNQXUTPQOC5+ClXYiIJDtHmcJCOmxr/9FWqfxzOpBO5c64D4S9kv2Dk/aLIGT5dju8TYkXFDAhM63iL5JAjVziRuwsVG8Yfts71GAoqsJw2+fD0cE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589224786; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=N6rESFNnjMacDs9b4dinWIplxxVHkhfEbXE4KWxFc8s=; b=K25/psFomSzlCG+sm2fIrMbM5pBwlNbzLKmcsMh3XaKiWmqakSrUxWF/JoJLbjWSY+SU+k8hHyHj/jswR1ytQGWJY0If2YpHYLDwOHXrO0AKFU+dd9YJpW1/lWnUB+MujDgt9zqXxyeTg9RadJAyR+bl+JloUZHTkN+b9ah40r8= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1589224786465902.6733781096815; Mon, 11 May 2020 12:19:46 -0700 (PDT) Received: from localhost ([::1]:58378 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYDxp-0005GB-8v for importer@patchew.org; Mon, 11 May 2020 15:19:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51378) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYDvg-0001hK-4t for qemu-devel@nongnu.org; Mon, 11 May 2020 15:17:32 -0400 Received: from mga18.intel.com ([134.134.136.126]:15020) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYDve-0001Yf-JA for qemu-devel@nongnu.org; Mon, 11 May 2020 15:17:31 -0400 Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 May 2020 12:17:21 -0700 Received: from unknown (HELO localhost.lm.intel.com) ([10.232.116.74]) by fmsmga006.fm.intel.com with ESMTP; 11 May 2020 12:17:20 -0700 IronPort-SDR: UmWxfTJi9+JGAbgCCKy8U7dRxwsFWcATeS0AsoduDY3PvJMZUq7HGE7S6H2LAArJdwgWTjB5Ru 2DacLNDkdsYg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False IronPort-SDR: gX6s2I+xvu25lTfrvc9VVp+S/yrqlBXmbMfI0HTgFWAq92SJ/DiAQ0/Syae+h9aifloYXAd5EY cJX/Yuy7wvyw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,381,1583222400"; d="scan'208";a="463494242" From: Jon Derrick To: , qemu-devel@nongnu.org Subject: [PATCH v2 1/2] PCI: vmd: Filter resource type bits from shadow register Date: Mon, 11 May 2020 15:01:28 -0400 Message-Id: <20200511190129.9313-3-jonathan.derrick@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20200511190129.9313-1-jonathan.derrick@intel.com> References: <20200511190129.9313-1-jonathan.derrick@intel.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=134.134.136.126; envelope-from=jonathan.derrick@intel.com; helo=mga18.intel.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/11 15:17:21 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lorenzo Pieralisi , virtualization@lists.linux-foundation.org, Andrzej Jakowski , Bjorn Helgaas , Christoph Hellwig , Jon Derrick Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Versions of VMD with the Host Physical Address shadow register use this register to calculate the bus address offset needed to do guest passthrough of the domain. This register shadows the Host Physical Address registers including the resource type bits. After calculating the offset, the extra resource type bits lead to the VMD resources being over-provisioned at the front and under-provisioned at the back. Example: pci 10000:80:02.0: reg 0x10: [mem 0xf801fffc-0xf803fffb 64bit] Expected: pci 10000:80:02.0: reg 0x10: [mem 0xf8020000-0xf803ffff 64bit] If other devices are mapped in the over-provisioned front, it could lead to resource conflict issues with VMD or those devices. Fixes: a1a30170138c9 ("PCI: vmd: Fix shadow offsets to reflect spec changes= ") Signed-off-by: Jon Derrick --- drivers/pci/controller/vmd.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index dac91d60701d..e386d4eac407 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -445,9 +445,11 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsi= gned long features) if (!membar2) return -ENOMEM; offset[0] =3D vmd->dev->resource[VMD_MEMBAR1].start - - readq(membar2 + MB2_SHADOW_OFFSET); + (readq(membar2 + MB2_SHADOW_OFFSET) & + PCI_BASE_ADDRESS_MEM_MASK); offset[1] =3D vmd->dev->resource[VMD_MEMBAR2].start - - readq(membar2 + MB2_SHADOW_OFFSET + 8); + (readq(membar2 + MB2_SHADOW_OFFSET + 8) & + PCI_BASE_ADDRESS_MEM_MASK); pci_iounmap(vmd->dev, membar2); } } --=20 2.18.1 From nobody Fri Nov 14 16:37:27 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1589224753; cv=none; d=zohomail.com; s=zohoarc; b=AT8uUE/ZQkwZRNVTdo/ZmikCnHNEQLdT3Bf4FFUc+lqJQtyr8JJBK8HuO0f32gJ/onz+VWyW4bp1ZpjH2Eg8UonJa6eab4pJf6epwrWm1McXKG78Qcxb7cRno7O0buTGwc/tjHHWpd06EWGR/bLu4OBjHjGdA/JMABXCnteTPC0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589224753; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=Ppz6uakLijxwZeON9MZj23KvwN9PM/hQHdZUjUUzoLY=; b=PNaltWORJll5CAaQKGd+Q4x/qtLYc1qqwUOGBVHaQ05EqAxH7nfMD3dZz5E2oJEEWndTU9IHW9r3g9KKnLYOny31vvzoskICX1UCIqGfYFfOktLTdlDkWL3U9Syk3MDGX9uwoHoYkesWYP5hs9/DypnOGh2Ri9g85gKdOeb5KF0= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1589224753699398.2380079664108; Mon, 11 May 2020 12:19:13 -0700 (PDT) Received: from localhost ([::1]:55154 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYDxI-0003jy-CD for importer@patchew.org; Mon, 11 May 2020 15:19:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51382) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYDvg-0001hS-Lo for qemu-devel@nongnu.org; Mon, 11 May 2020 15:17:32 -0400 Received: from mga18.intel.com ([134.134.136.126]:15027) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYDvf-0001bB-Dx for qemu-devel@nongnu.org; Mon, 11 May 2020 15:17:32 -0400 Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 May 2020 12:17:21 -0700 Received: from unknown (HELO localhost.lm.intel.com) ([10.232.116.74]) by fmsmga006.fm.intel.com with ESMTP; 11 May 2020 12:17:21 -0700 IronPort-SDR: OK1SdtoHI4Z94rsvnymImHXLFEeGy4tp334Z5hrx9WANmyACIs7VRg4epDSUkFP7hW39utoaLz alfbqXGpFG2Q== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False IronPort-SDR: roRqwOjLYhkQ32MOBY1NWaG8Rq4NfXfjhv0xHiAE0sHLAo09c8dlxaMkE1RMVc64sRzQVz9coz nUt+Po+4mQ4w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,381,1583222400"; d="scan'208";a="463494245" From: Jon Derrick To: , qemu-devel@nongnu.org Subject: [PATCH v2 2/2] PCI: vmd: Use Shadow MEMBAR registers for QEMU/KVM guests Date: Mon, 11 May 2020 15:01:29 -0400 Message-Id: <20200511190129.9313-4-jonathan.derrick@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20200511190129.9313-1-jonathan.derrick@intel.com> References: <20200511190129.9313-1-jonathan.derrick@intel.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=134.134.136.126; envelope-from=jonathan.derrick@intel.com; helo=mga18.intel.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/11 15:17:21 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lorenzo Pieralisi , virtualization@lists.linux-foundation.org, Andrzej Jakowski , Bjorn Helgaas , Christoph Hellwig , Jon Derrick Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" VMD device 28C0 natively assists guest passthrough of the VMD endpoint through the use of shadow registers that provide Host Physical Addresses to correctly assign bridge windows. These shadow registers are only available if VMD config space register 0x70, bit 1 is set. For existing VMD which don't natively support the shadow register, VMD config space register 0x70 is reserved and will return 0. Future VMD will have these registers natively in hardware, but existing VMD can still use this feature by emulating the config space register and shadow registers. QEMU has been modified to emulate this config space register and the shadow membar registers for VMDs which don't natively support this feature. This patch updates the supported device list to allow this feature to be used on these VMDs. Signed-off-by: Jon Derrick --- drivers/pci/controller/vmd.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index e386d4eac407..ee71d0989875 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -600,6 +600,7 @@ static irqreturn_t vmd_irq(int irq, void *data) static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id) { struct vmd_dev *vmd; + unsigned long features =3D id->driver_data; int i, err; =20 if (resource_size(&dev->resource[VMD_CFGBAR]) < (1 << 20)) @@ -652,7 +653,7 @@ static int vmd_probe(struct pci_dev *dev, const struct = pci_device_id *id) =20 spin_lock_init(&vmd->cfg_lock); pci_set_drvdata(dev, vmd); - err =3D vmd_enable_domain(vmd, (unsigned long) id->driver_data); + err =3D vmd_enable_domain(vmd, features); if (err) return err; =20 @@ -716,16 +717,20 @@ static int vmd_resume(struct device *dev) static SIMPLE_DEV_PM_OPS(vmd_dev_pm_ops, vmd_suspend, vmd_resume); =20 static const struct pci_device_id vmd_ids[] =3D { - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_201D),}, + {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_201D), + .driver_data =3D VMD_FEAT_HAS_MEMBAR_SHADOW,}, {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0), .driver_data =3D VMD_FEAT_HAS_MEMBAR_SHADOW | VMD_FEAT_HAS_BUS_RESTRICTIONS,}, {PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x467f), - .driver_data =3D VMD_FEAT_HAS_BUS_RESTRICTIONS,}, + .driver_data =3D VMD_FEAT_HAS_MEMBAR_SHADOW | + VMD_FEAT_HAS_BUS_RESTRICTIONS,}, {PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x4c3d), - .driver_data =3D VMD_FEAT_HAS_BUS_RESTRICTIONS,}, + .driver_data =3D VMD_FEAT_HAS_MEMBAR_SHADOW | + VMD_FEAT_HAS_BUS_RESTRICTIONS,}, {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B), - .driver_data =3D VMD_FEAT_HAS_BUS_RESTRICTIONS,}, + .driver_data =3D VMD_FEAT_HAS_MEMBAR_SHADOW | + VMD_FEAT_HAS_BUS_RESTRICTIONS,}, {0,} }; MODULE_DEVICE_TABLE(pci, vmd_ids); --=20 2.18.1 From nobody Fri Nov 14 16:37:27 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1589224753; cv=none; d=zohomail.com; s=zohoarc; b=NRjbOfeEbDiq0sWgHiMckYUwKUrqIyZmVk7n1BJdiD2Y6zy6e7dBtT6dUm+Umg4HqC/UeS8HGwNkCwHY9+bscu6lQoIr34U62+ya3tkG5zvrth9bvhHpr6n0FIr+zBiZ5lHtShtk5vclBeoEZ8Tae8taTtrpZyT6JmcAVVgAsk0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589224753; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=TK79VYQMfDvHc2Tf+rWMF/f+ZqI8wg+lVvUYKFzm+wA=; b=Q2hTYx5OXCY4dQmdBfATaYO0aDSfzo+sZ0KMhsmcqBg//PEs7sKtIWeSYetUmHhCjoipLjkwr9JIjAsMgN+8KvThN6Izqm4j2aXNp1tDSicQxbZ7vm1NrShfg5wjoYx28IDqxh+7OkFPtixvsFsoadzhiXu7I2YQnrZYpdksmvs= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1589224753347353.65948217998937; Mon, 11 May 2020 12:19:13 -0700 (PDT) Received: from localhost ([::1]:55078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYDxH-0003fd-Qo for importer@patchew.org; Mon, 11 May 2020 15:19:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51374) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYDvf-0001ge-G5 for qemu-devel@nongnu.org; Mon, 11 May 2020 15:17:31 -0400 Received: from mga18.intel.com ([134.134.136.126]:15021) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYDve-0001Ys-0U for qemu-devel@nongnu.org; Mon, 11 May 2020 15:17:31 -0400 Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 May 2020 12:17:20 -0700 Received: from unknown (HELO localhost.lm.intel.com) ([10.232.116.74]) by fmsmga006.fm.intel.com with ESMTP; 11 May 2020 12:17:20 -0700 IronPort-SDR: 3AF57s3TCpHH4AMsW2Owkcui+iY1GHJlRZmXytTj2DzQdZOSmXCqTBFmsQ5kZ0tG+WiwRB/3lS mA9125Bnt2Vg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False IronPort-SDR: J1dl4aw3xiNFGvwucvJnxSy7hn4p9TQdkAA/AN+skUvWZgoHHx+3AVCX5+Gaf7cNBFaI/pIN7I AeD6RKVGKtww== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,381,1583222400"; d="scan'208";a="463494237" From: Jon Derrick To: , qemu-devel@nongnu.org Subject: [PATCH for QEMU v2] hw/vfio: Add VMD Passthrough Quirk Date: Mon, 11 May 2020 15:01:27 -0400 Message-Id: <20200511190129.9313-2-jonathan.derrick@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20200511190129.9313-1-jonathan.derrick@intel.com> References: <20200511190129.9313-1-jonathan.derrick@intel.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=134.134.136.126; envelope-from=jonathan.derrick@intel.com; helo=mga18.intel.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/11 15:17:21 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lorenzo Pieralisi , virtualization@lists.linux-foundation.org, Andrzej Jakowski , Bjorn Helgaas , Christoph Hellwig , Jon Derrick Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The VMD endpoint provides a real PCIe domain to the guest, including bridges and endpoints. Because the VMD domain is enumerated by the guest kernel, the guest kernel will assign Guest Physical Addresses to the downstream endpoint BARs and bridge windows. When the guest kernel performs MMIO to VMD sub-devices, IOMMU will translate from the guest address space to the physical address space. Because the bridges have been programmed with guest addresses, the bridges will reject the transaction containing physical addresses. VMD device 28C0 natively assists passthrough by providing the Host Physical Address in shadow registers accessible to the guest for bridge window assignment. The shadow registers are valid if bit 1 is set in VMD VMLOCK config register 0x70. Future VMDs will also support this feature. Existing VMDs have config register 0x70 reserved, and will return 0 on reads. In order to support existing VMDs, this quirk emulates the VMLOCK and HPA shadow registers for all VMD device ids which don't natively assist with passthrough. The Linux VMD driver is updated to allow existing VMD devices to query VMLOCK for passthrough support. Signed-off-by: Jon Derrick --- hw/vfio/pci-quirks.c | 103 +++++++++++++++++++++++++++++++++++++++++++ hw/vfio/pci.c | 7 +++ hw/vfio/pci.h | 2 + hw/vfio/trace-events | 3 ++ 4 files changed, 115 insertions(+) diff --git a/hw/vfio/pci-quirks.c b/hw/vfio/pci-quirks.c index 2d348f8237..4060a6a95d 100644 --- a/hw/vfio/pci-quirks.c +++ b/hw/vfio/pci-quirks.c @@ -1709,3 +1709,106 @@ free_exit: =20 return ret; } + +/* + * The VMD endpoint provides a real PCIe domain to the guest and the guest + * kernel performs enumeration of the VMD sub-device domain. Guest transac= tions + * to VMD sub-devices go through IOMMU translation from guest addresses to + * physical addresses. When MMIO goes to an endpoint after being translate= d to + * physical addresses, the bridge rejects the transaction because the wind= ow + * has been programmed with guest addresses. + * + * VMD can use the Host Physical Address in order to correctly program the + * bridge windows in its PCIe domain. VMD device 28C0 has HPA shadow regis= ters + * located at offset 0x2000 in MEMBAR2 (BAR 4). The shadow registers are v= alid + * if bit 1 is set in the VMD VMLOCK config register 0x70. VMD devices wit= hout + * this native assistance can have these registers safely emulated as these + * registers are reserved. + */ +typedef struct VFIOVMDQuirk { + VFIOPCIDevice *vdev; + uint64_t membar_phys[2]; +} VFIOVMDQuirk; + +static uint64_t vfio_vmd_quirk_read(void *opaque, hwaddr addr, unsigned si= ze) +{ + VFIOVMDQuirk *data =3D opaque; + uint64_t val =3D 0; + + memcpy(&val, (void *)data->membar_phys + addr, size); + return val; +} + +static const MemoryRegionOps vfio_vmd_quirk =3D { + .read =3D vfio_vmd_quirk_read, + .endianness =3D DEVICE_LITTLE_ENDIAN, +}; + +#define VMD_VMLOCK 0x70 +#define VMD_SHADOW 0x2000 +#define VMD_MEMBAR2 4 + +static int vfio_vmd_emulate_shadow_registers(VFIOPCIDevice *vdev) +{ + VFIOQuirk *quirk; + VFIOVMDQuirk *data; + PCIDevice *pdev =3D &vdev->pdev; + int ret; + + data =3D g_malloc0(sizeof(*data)); + ret =3D pread(vdev->vbasedev.fd, data->membar_phys, 16, + vdev->config_offset + PCI_BASE_ADDRESS_2); + if (ret !=3D 16) { + error_report("VMD %s cannot read MEMBARs (%d)", + vdev->vbasedev.name, ret); + g_free(data); + return -EFAULT; + } + + quirk =3D vfio_quirk_alloc(1); + quirk->data =3D data; + data->vdev =3D vdev; + + /* Emulate Shadow Registers */ + memory_region_init_io(quirk->mem, OBJECT(vdev), &vfio_vmd_quirk, data, + "vfio-vmd-quirk", sizeof(data->membar_phys)); + memory_region_add_subregion_overlap(vdev->bars[VMD_MEMBAR2].region.mem, + VMD_SHADOW, quirk->mem, 1); + memory_region_set_readonly(quirk->mem, true); + memory_region_set_enabled(quirk->mem, true); + + QLIST_INSERT_HEAD(&vdev->bars[VMD_MEMBAR2].quirks, quirk, next); + + trace_vfio_pci_vmd_quirk_shadow_regs(vdev->vbasedev.name, + data->membar_phys[0], + data->membar_phys[1]); + + /* Advertise Shadow Register support */ + pci_byte_test_and_set_mask(pdev->config + VMD_VMLOCK, 0x2); + pci_set_byte(pdev->wmask + VMD_VMLOCK, 0); + pci_set_byte(vdev->emulated_config_bits + VMD_VMLOCK, 0x2); + + trace_vfio_pci_vmd_quirk_vmlock(vdev->vbasedev.name, + pci_get_byte(pdev->config + VMD_VMLOCK= )); + + return 0; +} + +int vfio_pci_vmd_init(VFIOPCIDevice *vdev) +{ + int ret =3D 0; + + switch (vdev->device_id) { + case 0x28C0: /* Native passthrough support */ + break; + /* Emulates Native passthrough support */ + case 0x201D: + case 0x467F: + case 0x4C3D: + case 0x9A0B: + ret =3D vfio_vmd_emulate_shadow_registers(vdev); + break; + } + + return ret; +} diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 5e75a95129..85425a1a6f 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3024,6 +3024,13 @@ static void vfio_realize(PCIDevice *pdev, Error **er= rp) } } =20 + if (vdev->vendor_id =3D=3D PCI_VENDOR_ID_INTEL) { + ret =3D vfio_pci_vmd_init(vdev); + if (ret) { + error_report("Failed to setup VMD"); + } + } + vfio_register_err_notifier(vdev); vfio_register_req_notifier(vdev); vfio_setup_resetfn_quirk(vdev); diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 0da7a20a7e..e8632d806b 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -217,6 +217,8 @@ int vfio_pci_igd_opregion_init(VFIOPCIDevice *vdev, int vfio_pci_nvidia_v100_ram_init(VFIOPCIDevice *vdev, Error **errp); int vfio_pci_nvlink2_init(VFIOPCIDevice *vdev, Error **errp); =20 +int vfio_pci_vmd_init(VFIOPCIDevice *vdev); + void vfio_display_reset(VFIOPCIDevice *vdev); int vfio_display_probe(VFIOPCIDevice *vdev, Error **errp); void vfio_display_finalize(VFIOPCIDevice *vdev); diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index b1ef55a33f..ed64e477db 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -90,6 +90,9 @@ vfio_pci_nvidia_gpu_setup_quirk(const char *name, uint64_= t tgt, uint64_t size) " vfio_pci_nvlink2_setup_quirk_ssatgt(const char *name, uint64_t tgt, uint64= _t size) "%s tgt=3D0x%"PRIx64" size=3D0x%"PRIx64 vfio_pci_nvlink2_setup_quirk_lnkspd(const char *name, uint32_t link_speed)= "%s link_speed=3D0x%x" =20 +vfio_pci_vmd_quirk_shadow_regs(const char *name, uint64_t mb1, uint64_t mb= 2) "%s membar1_phys=3D0x%"PRIx64" membar2_phys=3D0x%"PRIx64 +vfio_pci_vmd_quirk_vmlock(const char *name, uint8_t vmlock) "%s vmlock=3D0= x%x" + # common.c vfio_region_write(const char *name, int index, uint64_t addr, uint64_t dat= a, unsigned size) " (%s:region%d+0x%"PRIx64", 0x%"PRIx64 ", %d)" vfio_region_read(char *name, int index, uint64_t addr, unsigned size, uint= 64_t data) " (%s:region%d+0x%"PRIx64", %d) =3D 0x%"PRIx64 --=20 2.18.1