From nobody Sun Feb 8 14:20:17 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1709827731; cv=none; d=zohomail.com; s=zohoarc; b=DAa0ZS6DPGtWqR8blGsBMfWzhoAUNeU4sLQXd/ZUm/LH5mAWUqe6dpg/e13bFgk+CDFBwKBFJ/qzcd+52NshzLIQEdQvn0q+AO1+ag78OKlzAaryB7lbPMIHnaeQ6obvbSbROo0tdohHZ9+5mNm+4y++mq9eEBag24cFJw3lwUo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709827731; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=1aXRP2KVTe35bDOq459aCYA3XDBPGwRmCn+nfWu9Dtg=; b=BTutqGHCcuC0XGKQo3IsHcbrEmsESNlNM97XJCbQh6j2j2NyoKwOSk32cu//Rrhzi6h9QTFhcc5FAqdiihQ3x3IqcFF1xKHBe6P8C745Lfu/VF4zdK3zFGHN9zjT01h3lpGy7Ab6xy9fEbrt1ZooVc53FiEkwGOtRBihpqqPYXc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709827731205390.0425074680536; Thu, 7 Mar 2024 08:08:51 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1riGEd-0006Kg-BI; Thu, 07 Mar 2024 11:04:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1riGET-0006E9-7x for qemu-devel@nongnu.org; Thu, 07 Mar 2024 11:04:33 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1riGEK-0006TV-Ob for qemu-devel@nongnu.org; Thu, 07 Mar 2024 11:04:31 -0500 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1dc5d0162bcso9386705ad.0 for ; Thu, 07 Mar 2024 08:04:24 -0800 (PST) Received: from grind.dc1.ventanamicro.com ([177.94.15.159]) by smtp.gmail.com with ESMTPSA id u11-20020a170903124b00b001db2b8b2da7sm14663136plh.122.2024.03.07.08.04.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Mar 2024 08:04:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1709827460; x=1710432260; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1aXRP2KVTe35bDOq459aCYA3XDBPGwRmCn+nfWu9Dtg=; b=K2L8Ji48bxE+Tf21r93ixkNye5ZgEwesNZrDQRieyvwplDZSq+N133WLQSVhpmMiVF RaLuVpETuSRtS0x81m9zzaLhLRpUcbIerW6xbPp70PGTmYqo8thl/mTiI+Law+zVjzIS FHki9PYt6S2vusWW3d85LBQy0HM+jtHEcMvdW90YRVgB9ACmNJsADoc2qKsZXt03soLK 4dOYKyKag+Vmr5jY6useJwQ8Z1JBHJ3D9t9wdR3/jqq5Dqu1sdSkpBuaa+hEpCKFdzB7 npGUbXg6uar3DCBaAvCn6cuEACI6z7sQR4MdW/tGrwB++7zDab/ZfiXKjlZ4hHw++L4I 3hBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709827460; x=1710432260; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1aXRP2KVTe35bDOq459aCYA3XDBPGwRmCn+nfWu9Dtg=; b=U0HbA+HGZjaEz7AMvVLtqxe/aN0yL1QRSUwdpjhdcYhY0WtUtAn8/VW2xBDnjR13I/ iP08p0YhcXYtaL9OsAZfBoRuaemk7FlvC8jAq21+dBkN9MLvZ131S5DFJLnwVHcIZgUH QvGjQebfstYkB3DTWGIhQpC+5JcS5dt0zQ9ATdPlhBCGqa+kyvonuwdlaa9rih7Wlg6G OYjhoFgwrXHaYkebwbJjGWIriIPFzTIUCe6ldhFuXYkz95xBxaemRj43zyPUX+NiS7wm TeMm6EDb+YmtL1UprvR7mxo1XnsvPnw/TrJHZBVwPF+VBDo5y+z5BRph2p7Az8CfSPMG vicA== X-Gm-Message-State: AOJu0YzR/Z6Quqye2mVEHxaj2Jievi7rOf1ENx9lm0J5nXKIKJPU8L+1 rEzKuIBLpzz3EpKS0Jf/S1R0ZdPqlCly1oUxnIxTJ/0YS3rudYrFt1HOP/UHKBpcoT29oB/bStu d X-Google-Smtp-Source: AGHT+IGoFS2MF0reIo/AKEmeI92FN3H728A+uxneGTTHUYFyLi7GdBPRQBQNqP0YBROznPlPMSgQKQ== X-Received: by 2002:a17:902:7c92:b0:1dc:cc09:ebad with SMTP id y18-20020a1709027c9200b001dccc09ebadmr6875109pll.28.1709827459524; Thu, 07 Mar 2024 08:04:19 -0800 (PST) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Cc: qemu-riscv@nongnu.org, alistair.francis@wdc.com, bmeng@tinylab.org, liwei1518@gmail.com, zhiwei_liu@linux.alibaba.com, palmer@rivosinc.com, ajones@ventanamicro.com, tjeznach@rivosinc.com Subject: [PATCH v2 15/15] hw/misc: EDU: add ATS/PRI capability Date: Thu, 7 Mar 2024 13:03:18 -0300 Message-ID: <20240307160319.675044-16-dbarboza@ventanamicro.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240307160319.675044-1-dbarboza@ventanamicro.com> References: <20240307160319.675044-1-dbarboza@ventanamicro.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=dbarboza@ventanamicro.com; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ventanamicro.com) X-ZM-MESSAGEID: 1709827733340100003 Content-Type: text/plain; charset="utf-8" From: Tomasz Jeznach Mimic ATS interface with IOMMU translate request with IOMMU_NONE. If mapping exists, translation service will return current permission flags, otherwise will report no permissions. Implement and register the IOMMU memory region listener to be notified whenever an ATS invalidation request is sent from the IOMMU. Implement and register the IOMMU memory region listener to be notified whenever an ATS page request group response is triggered from the IOMMU. Introduces a retry mechanism to the timer design so that any page that's not available should be only accessed after the PRGR notification has been received. Signed-off-by: Tomasz Jeznach Signed-off-by: Sebastien Boeuf --- hw/misc/edu.c | 258 ++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 251 insertions(+), 7 deletions(-) diff --git a/hw/misc/edu.c b/hw/misc/edu.c index 522cec85b3..f4f6c15ec6 100644 --- a/hw/misc/edu.c +++ b/hw/misc/edu.c @@ -45,6 +45,14 @@ DECLARE_INSTANCE_CHECKER(EduState, EDU, #define DMA_START 0x40000 #define DMA_SIZE 4096 =20 +/* + * Number of tries before giving up on page request group response. + * Given the timer callback is scheduled to be run again after 100ms, + * 10 tries give roughly a second for the PRGR notification to be + * received. + */ +#define NUM_TRIES 10 + struct EduState { PCIDevice pdev; MemoryRegion mmio; @@ -55,6 +63,7 @@ struct EduState { bool stopping; =20 bool enable_pasid; + uint32_t try; =20 uint32_t addr4; uint32_t fact; @@ -81,6 +90,20 @@ struct EduState { QEMUTimer dma_timer; char dma_buf[DMA_SIZE]; uint64_t dma_mask; + + MemoryListener iommu_listener; + QLIST_HEAD(, edu_iommu) iommu_list; + + bool prgr_rcvd; + bool prgr_success; +}; + +struct edu_iommu { + EduState *edu; + IOMMUMemoryRegion *iommu_mr; + hwaddr iommu_offset; + IOMMUNotifier n; + QLIST_ENTRY(edu_iommu) iommu_next; }; =20 static bool edu_msi_enabled(EduState *edu) @@ -136,11 +159,65 @@ static dma_addr_t edu_clamp_addr(const EduState *edu,= dma_addr_t addr) return res; } =20 +static bool __find_iommu_mr_cb(Int128 start, Int128 len, const MemoryRegio= n *mr, + hwaddr offset_in_region, void *opaque) +{ + IOMMUMemoryRegion **iommu_mr =3D opaque; + *iommu_mr =3D memory_region_get_iommu((MemoryRegion *)mr); + return *iommu_mr !=3D NULL; +} + +static int pci_dma_perm(PCIDevice *pdev, dma_addr_t iova, MemTxAttrs attrs) +{ + IOMMUMemoryRegion *iommu_mr =3D NULL; + IOMMUMemoryRegionClass *imrc; + int iommu_idx; + FlatView *fv; + EduState *edu =3D EDU(pdev); + struct edu_iommu *iommu; + + RCU_READ_LOCK_GUARD(); + + fv =3D address_space_to_flatview(pci_get_address_space(pdev)); + + /* Find first IOMMUMemoryRegion */ + flatview_for_each_range(fv, __find_iommu_mr_cb, &iommu_mr); + + if (iommu_mr) { + imrc =3D memory_region_get_iommu_class_nocheck(iommu_mr); + + /* IOMMU Index is mapping to memory attributes (PASID, etc) */ + iommu_idx =3D imrc->attrs_to_index ? + imrc->attrs_to_index(iommu_mr, attrs) : 0; + + /* Update IOMMU notifiers with proper index */ + QLIST_FOREACH(iommu, &edu->iommu_list, iommu_next) { + if (iommu->iommu_mr =3D=3D iommu_mr && + iommu->n.iommu_idx !=3D iommu_idx) { + memory_region_unregister_iommu_notifier( + MEMORY_REGION(iommu->iommu_mr), &iommu->n); + iommu->n.iommu_idx =3D iommu_idx; + memory_region_register_iommu_notifier( + MEMORY_REGION(iommu->iommu_mr), &iommu->n, NULL); + } + } + + /* Translate request with IOMMU_NONE is an ATS request */ + IOMMUTLBEntry iotlb =3D imrc->translate(iommu_mr, iova, IOMMU_NONE, + iommu_idx); + + return iotlb.perm; + } + + return IOMMU_NONE; +} + static void edu_dma_timer(void *opaque) { EduState *edu =3D opaque; bool raise_irq =3D false; MemTxAttrs attrs =3D MEMTXATTRS_UNSPECIFIED; + MemTxResult res; =20 if (!(edu->dma.cmd & EDU_DMA_RUN)) { return; @@ -155,18 +232,70 @@ static void edu_dma_timer(void *opaque) =20 if (EDU_DMA_DIR(edu->dma.cmd) =3D=3D EDU_DMA_FROM_PCI) { uint64_t dst =3D edu->dma.dst; + uint64_t src =3D edu_clamp_addr(edu, edu->dma.src); edu_check_range(dst, edu->dma.cnt, DMA_START, DMA_SIZE); dst -=3D DMA_START; - pci_dma_rw(&edu->pdev, edu_clamp_addr(edu, edu->dma.src), - edu->dma_buf + dst, edu->dma.cnt, - DMA_DIRECTION_TO_DEVICE, attrs); + if (edu->try-- =3D=3D NUM_TRIES) { + edu->prgr_rcvd =3D false; + if (!(pci_dma_perm(&edu->pdev, src, attrs) & IOMMU_RO)) { + timer_mod(&edu->dma_timer, + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + 100); + return; + } + } else if (edu->try) { + if (!edu->prgr_rcvd) { + timer_mod(&edu->dma_timer, + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + 100); + return; + } + if (!edu->prgr_success) { + /* PRGR failure, fail DMA. */ + edu->dma.cmd &=3D ~EDU_DMA_RUN; + return; + } + } else { + /* timeout, fail DMA. */ + edu->dma.cmd &=3D ~EDU_DMA_RUN; + return; + } + res =3D pci_dma_rw(&edu->pdev, src, edu->dma_buf + dst, edu->dma.c= nt, + DMA_DIRECTION_TO_DEVICE, attrs); + if (res !=3D MEMTX_OK) { + hw_error("EDU: DMA transfer TO 0x%"PRIx64" failed.\n", dst); + } } else { uint64_t src =3D edu->dma.src; + uint64_t dst =3D edu_clamp_addr(edu, edu->dma.dst); edu_check_range(src, edu->dma.cnt, DMA_START, DMA_SIZE); src -=3D DMA_START; - pci_dma_rw(&edu->pdev, edu_clamp_addr(edu, edu->dma.dst), - edu->dma_buf + src, edu->dma.cnt, - DMA_DIRECTION_FROM_DEVICE, attrs); + if (edu->try-- =3D=3D NUM_TRIES) { + edu->prgr_rcvd =3D false; + if (!(pci_dma_perm(&edu->pdev, dst, attrs) & IOMMU_WO)) { + timer_mod(&edu->dma_timer, + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + 100); + return; + } + } else if (edu->try) { + if (!edu->prgr_rcvd) { + timer_mod(&edu->dma_timer, + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + 100); + return; + } + if (!edu->prgr_success) { + /* PRGR failure, fail DMA. */ + edu->dma.cmd &=3D ~EDU_DMA_RUN; + return; + } + } else { + /* timeout, fail DMA. */ + edu->dma.cmd &=3D ~EDU_DMA_RUN; + return; + } + res =3D pci_dma_rw(&edu->pdev, dst, edu->dma_buf + src, edu->dma.c= nt, + DMA_DIRECTION_FROM_DEVICE, attrs); + if (res !=3D MEMTX_OK) { + hw_error("EDU: DMA transfer FROM 0x%"PRIx64" failed.\n", src); + } } =20 edu->dma.cmd &=3D ~EDU_DMA_RUN; @@ -193,6 +322,7 @@ static void dma_rw(EduState *edu, bool write, dma_addr_= t *val, dma_addr_t *dma, } =20 if (timer) { + edu->try =3D NUM_TRIES; timer_mod(&edu->dma_timer, qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) += 100); } } @@ -376,9 +506,92 @@ static void *edu_fact_thread(void *opaque) return NULL; } =20 +static void edu_iommu_ats_prgr_notify(IOMMUNotifier *n, IOMMUTLBEntry *iot= lb) +{ + struct edu_iommu *iommu =3D container_of(n, struct edu_iommu, n); + EduState *edu =3D iommu->edu; + edu->prgr_success =3D (iotlb->perm !=3D IOMMU_NONE); + barrier(); + edu->prgr_rcvd =3D true; +} + +static void edu_iommu_ats_inval_notify(IOMMUNotifier *n, + IOMMUTLBEntry *iotlb) +{ + +} + +static void edu_iommu_region_add(MemoryListener *listener, + MemoryRegionSection *section) +{ + EduState *edu =3D container_of(listener, EduState, iommu_listener); + struct edu_iommu *iommu; + Int128 end; + int iommu_idx; + IOMMUMemoryRegion *iommu_mr; + + if (!memory_region_is_iommu(section->mr)) { + return; + } + + iommu_mr =3D IOMMU_MEMORY_REGION(section->mr); + + /* Register ATS.INVAL notifier */ + iommu =3D g_malloc0(sizeof(*iommu)); + iommu->iommu_mr =3D iommu_mr; + iommu->iommu_offset =3D section->offset_within_address_space - + section->offset_within_region; + iommu->edu =3D edu; + end =3D int128_add(int128_make64(section->offset_within_region), + section->size); + end =3D int128_sub(end, int128_one()); + iommu_idx =3D memory_region_iommu_attrs_to_index(iommu_mr, + MEMTXATTRS_UNSPECIFIED); + iommu_notifier_init(&iommu->n, edu_iommu_ats_inval_notify, + IOMMU_NOTIFIER_DEVIOTLB_UNMAP, + section->offset_within_region, + int128_get64(end), + iommu_idx); + memory_region_register_iommu_notifier(section->mr, &iommu->n, NULL); + QLIST_INSERT_HEAD(&edu->iommu_list, iommu, iommu_next); + + /* Register ATS.PRGR notifier */ + iommu =3D g_memdup2(iommu, sizeof(*iommu)); + iommu_notifier_init(&iommu->n, edu_iommu_ats_prgr_notify, + IOMMU_NOTIFIER_MAP, + section->offset_within_region, + int128_get64(end), + iommu_idx); + memory_region_register_iommu_notifier(section->mr, &iommu->n, NULL); + QLIST_INSERT_HEAD(&edu->iommu_list, iommu, iommu_next); +} + +static void edu_iommu_region_del(MemoryListener *listener, + MemoryRegionSection *section) +{ + EduState *edu =3D container_of(listener, EduState, iommu_listener); + struct edu_iommu *iommu; + + if (!memory_region_is_iommu(section->mr)) { + return; + } + + QLIST_FOREACH(iommu, &edu->iommu_list, iommu_next) { + if (MEMORY_REGION(iommu->iommu_mr) =3D=3D section->mr && + iommu->n.start =3D=3D section->offset_within_region) { + memory_region_unregister_iommu_notifier(section->mr, + &iommu->n); + QLIST_REMOVE(iommu, iommu_next); + g_free(iommu); + break; + } + } +} + static void pci_edu_realize(PCIDevice *pdev, Error **errp) { EduState *edu =3D EDU(pdev); + AddressSpace *dma_as =3D NULL; uint8_t *pci_conf =3D pdev->config; int pos; =20 @@ -390,9 +603,28 @@ static void pci_edu_realize(PCIDevice *pdev, Error **e= rrp) pos =3D PCI_CONFIG_SPACE_SIZE; if (edu->enable_pasid) { /* PCIe Spec 7.8.9 PASID Extended Capability Structure */ - pcie_add_capability(pdev, 0x1b, 1, pos, 8); + pcie_add_capability(pdev, PCI_EXT_CAP_ID_PASID, 1, pos, 8); pci_set_long(pdev->config + pos + 4, 0x00001400); pci_set_long(pdev->wmask + pos + 4, 0xfff0ffff); + pos +=3D 8; + + /* ATS Capability */ + pcie_ats_init(pdev, pos, true); + pos +=3D PCI_EXT_CAP_ATS_SIZEOF; + + /* PRI Capability */ + pcie_add_capability(pdev, PCI_EXT_CAP_ID_PRI, 1, pos, 16); + /* PRI STOPPED */ + pci_set_long(pdev->config + pos + 4, 0x01000000); + /* PRI ENABLE bit writable */ + pci_set_long(pdev->wmask + pos + 4, 0x00000001); + /* PRI Capacity Supported */ + pci_set_long(pdev->config + pos + 8, 0x00000080); + /* PRI Allocations Allowed, 32 */ + pci_set_long(pdev->config + pos + 12, 0x00000040); + pci_set_long(pdev->wmask + pos + 12, 0x0000007f); + + pos +=3D 8; } =20 if (msi_init(pdev, 0, 1, true, false, errp)) { @@ -409,12 +641,24 @@ static void pci_edu_realize(PCIDevice *pdev, Error **= errp) memory_region_init_io(&edu->mmio, OBJECT(edu), &edu_mmio_ops, edu, "edu-mmio", 1 * MiB); pci_register_bar(pdev, 0, PCI_BASE_ADDRESS_SPACE_MEMORY, &edu->mmio); + + /* Register IOMMU listener */ + edu->iommu_listener =3D (MemoryListener) { + .name =3D "edu-iommu", + .region_add =3D edu_iommu_region_add, + .region_del =3D edu_iommu_region_del, + }; + + dma_as =3D pci_device_iommu_address_space(pdev); + memory_listener_register(&edu->iommu_listener, dma_as); } =20 static void pci_edu_uninit(PCIDevice *pdev) { EduState *edu =3D EDU(pdev); =20 + memory_listener_unregister(&edu->iommu_listener); + qemu_mutex_lock(&edu->thr_mutex); edu->stopping =3D true; qemu_mutex_unlock(&edu->thr_mutex); --=20 2.43.2