From nobody Tue Apr 23 07:10:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526481118481610.2218943529458; Wed, 16 May 2018 07:31:58 -0700 (PDT) Received: from localhost ([::1]:48963 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIxTB-0006GZ-Ki for importer@patchew.org; Wed, 16 May 2018 10:31:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42598) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIxOd-0002oh-1D for qemu-devel@nongnu.org; Wed, 16 May 2018 10:27:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIxOb-0000RL-Ej for qemu-devel@nongnu.org; Wed, 16 May 2018 10:27:15 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:51916 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIxOU-0000Ou-NN; Wed, 16 May 2018 10:27:06 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 18D4540201A6; Wed, 16 May 2018 14:27:06 +0000 (UTC) Received: from dhcp19-241-177.khw3.lab.eng.bos.redhat.com (gigabyte-r120-16.khw3.lab.eng.bos.redhat.com [10.19.240.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id D04C5112D199; Wed, 16 May 2018 14:27:05 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 16 May 2018 14:31:12 -0400 Message-Id: <1526495474-26394-2-git-send-email-eric.auger@redhat.com> In-Reply-To: <1526495474-26394-1-git-send-email-eric.auger@redhat.com> References: <1526495474-26394-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 16 May 2018 14:27:06 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 16 May 2018 14:27:06 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH 1/3] hw/arm/smmuv3: Cache/invalidate config data X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Let's cache config data to avoid fetching and parsing STE/CD structures on each translation. We invalidate them on data structure invalidation commands. Signed-off-by: Eric Auger --- v11 -> v12: - only insert the new config if decode_cfg succeeds - use smmu_get_sid for trace_* and store hits/misses in the SMMUDevice - s/smmuv3_put_config/smmuv3_flush_config - document smmuv3_get_config - removing the mutex as BQL does the job --- hw/arm/smmu-common.c | 26 ++++++++- hw/arm/smmuv3.c | 130 +++++++++++++++++++++++++++++++++++++++= ++-- hw/arm/trace-events | 6 ++ include/hw/arm/smmu-common.h | 5 ++ 4 files changed, 159 insertions(+), 8 deletions(-) diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index 3c5f724..7e9827d 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -297,6 +297,8 @@ static AddressSpace *smmu_find_add_as(PCIBus *bus, void= *opaque, int devfn) sdev->smmu =3D s; sdev->bus =3D bus; sdev->devfn =3D devfn; + sdev->cfg_cache_misses =3D 0; + sdev->cfg_cache_hits =3D 0; =20 memory_region_init_iommu(&sdev->iommu, sizeof(sdev->iommu), s->mrtypename, @@ -310,6 +312,24 @@ static AddressSpace *smmu_find_add_as(PCIBus *bus, voi= d *opaque, int devfn) return &sdev->as; } =20 +IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid) +{ + uint8_t bus_n, devfn; + SMMUPciBus *smmu_bus; + SMMUDevice *smmu; + + bus_n =3D PCI_BUS_NUM(sid); + smmu_bus =3D smmu_find_smmu_pcibus(s, bus_n); + if (smmu_bus) { + devfn =3D sid & 0x7; + smmu =3D smmu_bus->pbdev[devfn]; + if (smmu) { + return &smmu->iommu; + } + } + return NULL; +} + static void smmu_base_realize(DeviceState *dev, Error **errp) { SMMUState *s =3D ARM_SMMU(dev); @@ -321,7 +341,7 @@ static void smmu_base_realize(DeviceState *dev, Error *= *errp) error_propagate(errp, local_err); return; } - + s->configs =3D g_hash_table_new_full(NULL, NULL, NULL, g_free); s->smmu_pcibus_by_busptr =3D g_hash_table_new(NULL, NULL); =20 if (s->primary_bus) { @@ -333,7 +353,9 @@ static void smmu_base_realize(DeviceState *dev, Error *= *errp) =20 static void smmu_base_reset(DeviceState *dev) { - /* will be filled later on */ + SMMUState *s =3D ARM_SMMU(dev); + + g_hash_table_remove_all(s->configs); } =20 static Property smmu_dev_properties[] =3D { diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index 42dc521..d3b64c2 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -537,6 +537,58 @@ static int smmuv3_decode_config(IOMMUMemoryRegion *mr,= SMMUTransCfg *cfg, return decode_cd(cfg, &cd, event); } =20 +/** + * smmuv3_get_config - Look up for a cached copy of configuration data for + * @sdev and on cache miss performs a configuration structure decoding from + * guest RAM. + * + * @sdev: SMMUDevice handle + * @event: output event info + * + * The configuration cache contains data resulting from both STE and CD + * decoding under the form of an SMMUTransCfg struct. The hash table is in= dexed + * by the SMMUDevice handle. + */ +static SMMUTransCfg *smmuv3_get_config(SMMUDevice *sdev, SMMUEventInfo *ev= ent) +{ + SMMUv3State *s =3D sdev->smmu; + SMMUState *bc =3D &s->smmu_state; + SMMUTransCfg *cfg; + + cfg =3D g_hash_table_lookup(bc->configs, sdev); + if (cfg) { + sdev->cfg_cache_hits +=3D 1; + trace_smmuv3_config_cache_hit(smmu_get_sid(sdev), + sdev->cfg_cache_hits, sdev->cfg_cache_misses, + 100 * sdev->cfg_cache_hits / + (sdev->cfg_cache_hits + sdev->cfg_cache_misses= )); + } else { + sdev->cfg_cache_misses +=3D 1; + trace_smmuv3_config_cache_miss(smmu_get_sid(sdev), + sdev->cfg_cache_hits, sdev->cfg_cache_misses, + 100 * sdev->cfg_cache_hits / + (sdev->cfg_cache_hits + sdev->cfg_cache_misses= )); + cfg =3D g_new0(SMMUTransCfg, 1); + + if (!smmuv3_decode_config(&sdev->iommu, cfg, event)) { + g_hash_table_insert(bc->configs, sdev, cfg); + } else { + g_free(cfg); + cfg =3D NULL; + } + } + return cfg; +} + +static void smmuv3_flush_config(SMMUDevice *sdev) +{ + SMMUv3State *s =3D sdev->smmu; + SMMUState *bc =3D &s->smmu_state; + + trace_smmuv3_config_cache_inv(smmu_get_sid(sdev)); + g_hash_table_remove(bc->configs, sdev); +} + static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, IOMMUAccessFlags flag) { @@ -545,7 +597,7 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion= *mr, hwaddr addr, uint32_t sid =3D smmu_get_sid(sdev); SMMUEventInfo event =3D {.type =3D SMMU_EVT_OK, .sid =3D sid}; SMMUPTWEventInfo ptw_info =3D {}; - SMMUTransCfg cfg =3D {}; + SMMUTransCfg *cfg =3D NULL; IOMMUTLBEntry entry =3D { .target_as =3D &address_space_memory, .iova =3D addr, @@ -559,16 +611,17 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegi= on *mr, hwaddr addr, goto out; } =20 - ret =3D smmuv3_decode_config(mr, &cfg, &event); - if (ret) { + cfg =3D smmuv3_get_config(sdev, &event); + if (!cfg) { + ret =3D -EINVAL; goto out; } =20 - if (cfg.aborted) { + if (cfg->aborted) { goto out; } =20 - ret =3D smmu_ptw(&cfg, addr, flag, &entry, &ptw_info); + ret =3D smmu_ptw(cfg, addr, flag, &entry, &ptw_info); if (ret) { switch (ptw_info.type) { case SMMU_PTW_ERR_WALK_EABT: @@ -617,7 +670,7 @@ out: mr->parent_obj.name, addr, ret); entry.perm =3D IOMMU_NONE; smmuv3_record_event(s, &event); - } else if (!cfg.aborted) { + } else if (!cfg->aborted) { entry.perm =3D flag; trace_smmuv3_translate(mr->parent_obj.name, sid, addr, entry.translated_addr, entry.perm); @@ -628,6 +681,7 @@ out: =20 static int smmuv3_cmdq_consume(SMMUv3State *s) { + SMMUState *bs =3D ARM_SMMU(s); SMMUCmdError cmd_error =3D SMMU_CERROR_NONE; SMMUQueue *q =3D &s->cmdq; SMMUCommandType type =3D 0; @@ -670,10 +724,74 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) break; case SMMU_CMD_PREFETCH_CONFIG: case SMMU_CMD_PREFETCH_ADDR: + break; case SMMU_CMD_CFGI_STE: + { + uint32_t sid =3D CMD_SID(&cmd); + IOMMUMemoryRegion *mr =3D smmu_iommu_mr(bs, sid); + SMMUDevice *sdev; + + if (CMD_SSEC(&cmd)) { + cmd_error =3D SMMU_CERROR_ILL; + break; + } + + if (!mr) { + break; + } + + trace_smmuv3_cmdq_cfgi_ste(sid); + sdev =3D container_of(mr, SMMUDevice, iommu); + smmuv3_flush_config(sdev); + + break; + } case SMMU_CMD_CFGI_STE_RANGE: /* same as SMMU_CMD_CFGI_ALL */ + { + uint32_t start =3D CMD_SID(&cmd), end, i; + uint8_t range =3D CMD_STE_RANGE(&cmd); + + if (CMD_SSEC(&cmd)) { + cmd_error =3D SMMU_CERROR_ILL; + break; + } + + end =3D start + (1 << (range + 1)) - 1; + trace_smmuv3_cmdq_cfgi_ste_range(start, end); + + for (i =3D start; i <=3D end; i++) { + IOMMUMemoryRegion *mr =3D smmu_iommu_mr(bs, i); + SMMUDevice *sdev; + + if (!mr) { + continue; + } + sdev =3D container_of(mr, SMMUDevice, iommu); + smmuv3_flush_config(sdev); + } + break; + } case SMMU_CMD_CFGI_CD: case SMMU_CMD_CFGI_CD_ALL: + { + uint32_t sid =3D CMD_SID(&cmd); + IOMMUMemoryRegion *mr =3D smmu_iommu_mr(bs, sid); + SMMUDevice *sdev; + + if (CMD_SSEC(&cmd)) { + cmd_error =3D SMMU_CERROR_ILL; + break; + } + + if (!mr) { + break; + } + + trace_smmuv3_cmdq_cfgi_cd(sid); + sdev =3D container_of(mr, SMMUDevice, iommu); + smmuv3_flush_config(sdev); + break; + } case SMMU_CMD_TLBI_NH_ALL: case SMMU_CMD_TLBI_NH_ASID: case SMMU_CMD_TLBI_NH_VA: diff --git a/hw/arm/trace-events b/hw/arm/trace-events index 2d92727..fe4c2a8 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -39,3 +39,9 @@ smmuv3_get_cd(uint64_t addr) "CD addr: 0x%"PRIx64 smmuv3_translate(const char *n, uint16_t sid, uint64_t iova, uint64_t tran= slated, int perm) "%s sid=3D%d iova=3D0x%"PRIx64" translated=3D0x%"PRIx64" = perm=3D0x%x" smmuv3_decode_cd(uint32_t oas) "oas=3D%d" smmuv3_decode_cd_tt(int i, uint32_t tsz, uint64_t ttb, uint32_t granule_sz= ) "TT[%d]:tsz:%d ttb:0x%"PRIx64" granule_sz:%d" +smmuv3_cmdq_cfgi_ste(int streamid) " |_ streamid =3D%d" +smmuv3_cmdq_cfgi_ste_range(int start, int end) " |_ start=3D0x%d - end= =3D0x%d" +smmuv3_cmdq_cfgi_cd(uint32_t sid) " |_ streamid =3D %d" +smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, floa= t perc) "Config cache HIT for sid %d (hits=3D%d, misses=3D%d, hit rate=3D%.= 1f)" +smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, flo= at perc) "Config cache MISS for sid %d (hits=3D%d, misses=3D%d, hit rate=3D= %.1f)" +smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid %d" diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index c41eb5c..7ce95ca 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -75,6 +75,8 @@ typedef struct SMMUDevice { int devfn; IOMMUMemoryRegion iommu; AddressSpace as; + uint32_t cfg_cache_hits; + uint32_t cfg_cache_misses; } SMMUDevice; =20 typedef struct SMMUNotifierNode { @@ -142,4 +144,7 @@ int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUA= ccessFlags perm, */ SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova); =20 +/* Return the iommu mr associated to @sid, or NULL if none */ +IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid); + #endif /* HW_ARM_SMMU_COMMON */ --=20 1.8.3.1 From nobody Tue Apr 23 07:10:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526481240565656.9281464952751; Wed, 16 May 2018 07:34:00 -0700 (PDT) Received: from localhost ([::1]:49094 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIxV1-0007cb-QT for importer@patchew.org; Wed, 16 May 2018 10:33:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42630) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIxOh-0002sH-A7 for qemu-devel@nongnu.org; Wed, 16 May 2018 10:27:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIxOa-0000Qe-2d for qemu-devel@nongnu.org; Wed, 16 May 2018 10:27:19 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:55244 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIxOU-0000P3-Te; Wed, 16 May 2018 10:27:07 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6382781FE17A; Wed, 16 May 2018 14:27:06 +0000 (UTC) Received: from dhcp19-241-177.khw3.lab.eng.bos.redhat.com (gigabyte-r120-16.khw3.lab.eng.bos.redhat.com [10.19.240.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id 255E710EE961; Wed, 16 May 2018 14:27:06 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 16 May 2018 14:31:13 -0400 Message-Id: <1526495474-26394-3-git-send-email-eric.auger@redhat.com> In-Reply-To: <1526495474-26394-1-git-send-email-eric.auger@redhat.com> References: <1526495474-26394-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 16 May 2018 14:27:06 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 16 May 2018 14:27:06 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH 2/3] hw/arm/smmuv3: IOTLB emulation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We emulate a TLB cache of size SMMU_IOTLB_MAX_SIZE=3D256. It is implemented as a hash table whose key is a combination of the 16b asid and 48b IOVA. Entries are invalidated on TLB invalidation commands, either globally, or per asid, or per asid/iova. Signed-off-by: Eric Auger --- v11 -> v12: - Add new trace point when smmu is bypassed - s/iotlb_miss/iotlb_misses, s/iotlb_hit/iotlb_hits - use SMMUIOTLBKey as a key Credit to Tomasz Nowicki who did the first implementation of this IOTLB implementation, inspired of intel_iommu implementation. --- hw/arm/smmu-common.c | 59 ++++++++++++++++++++++++ hw/arm/smmuv3.c | 105 +++++++++++++++++++++++++++++++++++++++= +--- hw/arm/trace-events | 10 +++++ include/hw/arm/smmu-common.h | 13 ++++++ 4 files changed, 181 insertions(+), 6 deletions(-) diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index 7e9827d..e4408c4 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -24,11 +24,43 @@ #include "qom/cpu.h" #include "hw/qdev-properties.h" #include "qapi/error.h" +#include "qemu/jhash.h" =20 #include "qemu/error-report.h" #include "hw/arm/smmu-common.h" #include "smmu-internal.h" =20 +/* IOTLB Management */ + +inline void smmu_iotlb_inv_all(SMMUState *s) +{ + trace_smmu_iotlb_inv_all(); + g_hash_table_remove_all(s->iotlb); +} + +static gboolean smmu_hash_remove_by_asid(gpointer key, gpointer value, + gpointer user_data) +{ + uint16_t asid =3D *(uint16_t *)user_data; + SMMUIOTLBKey *iotlb_key =3D (SMMUIOTLBKey *)key; + + return iotlb_key->asid =3D=3D asid; +} + +inline void smmu_iotlb_inv_iova(SMMUState *s, uint16_t asid, dma_addr_t io= va) +{ + SMMUIOTLBKey key =3D {.asid =3D asid, .iova =3D iova}; + + trace_smmu_iotlb_inv_iova(asid, iova); + g_hash_table_remove(s->iotlb, &key); +} + +inline void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid) +{ + trace_smmu_iotlb_inv_asid(asid); + g_hash_table_foreach_remove(s->iotlb, smmu_hash_remove_by_asid, &asid); +} + /* VMSAv8-64 Translation */ =20 /** @@ -330,6 +362,30 @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_= t sid) return NULL; } =20 +static guint smmu_iotlb_key_hash(gconstpointer v) +{ + SMMUIOTLBKey *key =3D (SMMUIOTLBKey *)v; + uint32_t a, b, c; + + a =3D b =3D c =3D JHASH_INITVAL + sizeof(*key); + a +=3D key->asid; + b +=3D extract64(key->iova, 0, 32); + c +=3D extract64(key->iova, 32, 32); + + __jhash_mix(a, b, c); + __jhash_final(a, b, c); + + return c; +} + +static gboolean smmu_iotlb_key_equal(gconstpointer v1, gconstpointer v2) +{ + SMMUIOTLBKey *k1 =3D (SMMUIOTLBKey *)v1; + SMMUIOTLBKey *k2 =3D (SMMUIOTLBKey *)v2; + + return (k1->asid =3D=3D k2->asid) && (k1->iova =3D=3D k2->iova); +} + static void smmu_base_realize(DeviceState *dev, Error **errp) { SMMUState *s =3D ARM_SMMU(dev); @@ -342,6 +398,8 @@ static void smmu_base_realize(DeviceState *dev, Error *= *errp) return; } s->configs =3D g_hash_table_new_full(NULL, NULL, NULL, g_free); + s->iotlb =3D g_hash_table_new_full(smmu_iotlb_key_hash, smmu_iotlb_key= _equal, + g_free, g_free); s->smmu_pcibus_by_busptr =3D g_hash_table_new(NULL, NULL); =20 if (s->primary_bus) { @@ -356,6 +414,7 @@ static void smmu_base_reset(DeviceState *dev) SMMUState *s =3D ARM_SMMU(dev); =20 g_hash_table_remove_all(s->configs); + g_hash_table_remove_all(s->iotlb); } =20 static Property smmu_dev_properties[] =3D { diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index d3b64c2..6c89a71 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -572,6 +572,8 @@ static SMMUTransCfg *smmuv3_get_config(SMMUDevice *sdev= , SMMUEventInfo *event) =20 if (!smmuv3_decode_config(&sdev->iommu, cfg, event)) { g_hash_table_insert(bc->configs, sdev, cfg); + cfg->iotlb_misses =3D 0; + cfg->iotlb_hits =3D 0; } else { g_free(cfg); cfg =3D NULL; @@ -595,8 +597,12 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegio= n *mr, hwaddr addr, SMMUDevice *sdev =3D container_of(mr, SMMUDevice, iommu); SMMUv3State *s =3D sdev->smmu; uint32_t sid =3D smmu_get_sid(sdev); + SMMUState *bs =3D ARM_SMMU(s); SMMUEventInfo event =3D {.type =3D SMMU_EVT_OK, .sid =3D sid}; + uint64_t page_mask, aligned_addr; + IOMMUTLBEntry *cached_entry =3D NULL; SMMUPTWEventInfo ptw_info =3D {}; + SMMUTransTableInfo *tt; SMMUTransCfg *cfg =3D NULL; IOMMUTLBEntry entry =3D { .target_as =3D &address_space_memory, @@ -605,10 +611,13 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegi= on *mr, hwaddr addr, .addr_mask =3D ~(hwaddr)0, .perm =3D IOMMU_NONE, }; + SMMUIOTLBKey key, *new_key; int ret =3D 0; =20 if (!smmu_enabled(s)) { - goto out; + entry.perm =3D flag; + trace_smmuv3_translate_pt(mr->parent_obj.name, sid, addr, flag); + return entry; } =20 cfg =3D smmuv3_get_config(sdev, &event); @@ -621,8 +630,61 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegio= n *mr, hwaddr addr, goto out; } =20 - ret =3D smmu_ptw(cfg, addr, flag, &entry, &ptw_info); - if (ret) { + tt =3D select_tt(cfg, addr); + if (!tt) { + if (event.record_trans_faults) { + event.type =3D SMMU_EVT_F_TRANSLATION; + event.u.f_translation.addr =3D addr; + event.u.f_translation.rnw =3D flag & 0x1; + } + ret =3D -EINVAL; + goto out; + } + + page_mask =3D (1ULL << (tt->granule_sz)) - 1; + aligned_addr =3D addr & ~page_mask; + + key.asid =3D cfg->asid; + key.iova =3D aligned_addr; + + cached_entry =3D g_hash_table_lookup(bs->iotlb, &key); + if (cached_entry) { + cfg->iotlb_hits +=3D 1; + trace_smmu_iotlb_cache_hit(cfg->asid, aligned_addr, + cfg->iotlb_hits, cfg->iotlb_misses, + 100 * cfg->iotlb_hits / + (cfg->iotlb_hits + cfg->iotlb_misses)); + if ((flag & IOMMU_WO) && !(cached_entry->perm & IOMMU_WO)) { + ret =3D -EPERM; + if (event.record_trans_faults) { + event.type =3D SMMU_EVT_F_PERMISSION; + event.u.f_permission.addr =3D addr; + event.u.f_permission.rnw =3D flag & 0x1; + } + } + goto out; + } + + cfg->iotlb_misses +=3D 1; + trace_smmu_iotlb_cache_miss(cfg->asid, addr & ~page_mask, + cfg->iotlb_hits, cfg->iotlb_misses, + 100 * cfg->iotlb_hits / + (cfg->iotlb_hits + cfg->iotlb_misses)); + + if (g_hash_table_size(bs->iotlb) >=3D SMMU_IOTLB_MAX_SIZE) { + smmu_iotlb_inv_all(bs); + } + + cached_entry =3D g_new0(IOMMUTLBEntry, 1); + + ret =3D smmu_ptw(cfg, aligned_addr, flag, cached_entry, &ptw_info); + if (!ret) { + new_key =3D g_new0(SMMUIOTLBKey, 1); + new_key->asid =3D cfg->asid; + new_key->iova =3D aligned_addr; + g_hash_table_insert(bs->iotlb, new_key, cached_entry); + } else { + g_free(cached_entry); switch (ptw_info.type) { case SMMU_PTW_ERR_WALK_EABT: event.type =3D SMMU_EVT_F_WALK_EABT; @@ -672,6 +734,9 @@ out: smmuv3_record_event(s, &event); } else if (!cfg->aborted) { entry.perm =3D flag; + entry.translated_addr =3D cached_entry->translated_addr + + (addr & page_mask); + entry.addr_mask =3D cached_entry->addr_mask; trace_smmuv3_translate(mr->parent_obj.name, sid, addr, entry.translated_addr, entry.perm); } @@ -792,10 +857,39 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) smmuv3_flush_config(sdev); break; } - case SMMU_CMD_TLBI_NH_ALL: case SMMU_CMD_TLBI_NH_ASID: - case SMMU_CMD_TLBI_NH_VA: + { + uint16_t asid =3D CMD_ASID(&cmd); + + trace_smmuv3_cmdq_tlbi_nh_asid(asid); + smmu_iotlb_inv_asid(bs, asid); + break; + } + case SMMU_CMD_TLBI_NH_ALL: + case SMMU_CMD_TLBI_NSNH_ALL: + trace_smmuv3_cmdq_tlbi_nh(); + smmu_iotlb_inv_all(bs); + break; case SMMU_CMD_TLBI_NH_VAA: + { + dma_addr_t addr =3D CMD_ADDR(&cmd); + uint16_t vmid =3D CMD_VMID(&cmd); + + trace_smmuv3_cmdq_tlbi_nh_vaa(vmid, addr); + smmu_iotlb_inv_all(bs); + break; + } + case SMMU_CMD_TLBI_NH_VA: + { + uint16_t asid =3D CMD_ASID(&cmd); + uint16_t vmid =3D CMD_VMID(&cmd); + dma_addr_t addr =3D CMD_ADDR(&cmd); + bool leaf =3D CMD_LEAF(&cmd); + + trace_smmuv3_cmdq_tlbi_nh_va(vmid, asid, addr, leaf); + smmu_iotlb_inv_iova(bs, asid, addr); + break; + } case SMMU_CMD_TLBI_EL3_ALL: case SMMU_CMD_TLBI_EL3_VA: case SMMU_CMD_TLBI_EL2_ALL: @@ -804,7 +898,6 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) case SMMU_CMD_TLBI_EL2_VAA: case SMMU_CMD_TLBI_S12_VMALL: case SMMU_CMD_TLBI_S2_IPA: - case SMMU_CMD_TLBI_NSNH_ALL: case SMMU_CMD_ATC_INV: case SMMU_CMD_PRI_RESP: case SMMU_CMD_RESUME: diff --git a/hw/arm/trace-events b/hw/arm/trace-events index fe4c2a8..3eb9180 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -12,6 +12,11 @@ smmu_ptw_invalid_pte(int stage, int level, uint64_t base= addr, uint64_t pteaddr, smmu_ptw_page_pte(int stage, int level, uint64_t iova, uint64_t baseaddr,= uint64_t pteaddr, uint64_t pte, uint64_t address) "stage=3D%d level=3D%d i= ova=3D0x%"PRIx64" base@=3D0x%"PRIx64" pte@=3D0x%"PRIx64" pte=3D0x%"PRIx64" = page address =3D 0x%"PRIx64 smmu_ptw_block_pte(int stage, int level, uint64_t baseaddr, uint64_t ptead= dr, uint64_t pte, uint64_t iova, uint64_t gpa, int bsize_mb) "stage=3D%d le= vel=3D%d base@=3D0x%"PRIx64" pte@=3D0x%"PRIx64" pte=3D0x%"PRIx64" iova=3D0x= %"PRIx64" block address =3D 0x%"PRIx64" block size =3D %d MiB" smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte)= "baseaddr=3D0x%"PRIx64" index=3D0x%x, pteaddr=3D0x%"PRIx64", pte=3D0x%"PRI= x64 +smmu_iotlb_cache_hit(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t = miss, float p) "IOTLB cache HIT asid=3D%d addr=3D0x%"PRIx64" hit=3D%d miss= =3D%d hit rate=3D%.1f" +smmu_iotlb_cache_miss(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t= miss, float p) "IOTLB cache MISS asid=3D%d addr=3D0x%"PRIx64" hit=3D%d mis= s=3D%d hit rate=3D%.1f" +smmu_iotlb_inv_all(void) "IOTLB invalidate all" +smmu_iotlb_inv_asid(uint16_t asid) "IOTLB invalidate asid=3D%d" +smmu_iotlb_inv_iova(uint16_t asid, uint64_t addr) "IOTLB invalidate asid= =3D%d addr=3D0x%"PRIx64 =20 #hw/arm/smmuv3.c smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "= addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)" @@ -37,10 +42,15 @@ smmuv3_translate_bypass(const char *n, uint16_t sid, ui= nt64_t addr, bool is_writ smmuv3_translate_in(uint16_t sid, int pci_bus_num, uint64_t strtab_base) "= SID:0x%x bus:%d strtab_base:0x%"PRIx64 smmuv3_get_cd(uint64_t addr) "CD addr: 0x%"PRIx64 smmuv3_translate(const char *n, uint16_t sid, uint64_t iova, uint64_t tran= slated, int perm) "%s sid=3D%d iova=3D0x%"PRIx64" translated=3D0x%"PRIx64" = perm=3D0x%x" +smmuv3_translate_pt(const char *n, uint16_t sid, uint64_t iova, int perm) = "%s sid=3D%d iova=3D0x%"PRIx64" perm=3D0x%x PASSTHROUGH" smmuv3_decode_cd(uint32_t oas) "oas=3D%d" smmuv3_decode_cd_tt(int i, uint32_t tsz, uint64_t ttb, uint32_t granule_sz= ) "TT[%d]:tsz:%d ttb:0x%"PRIx64" granule_sz:%d" smmuv3_cmdq_cfgi_ste(int streamid) " |_ streamid =3D%d" smmuv3_cmdq_cfgi_ste_range(int start, int end) " |_ start=3D0x%d - end= =3D0x%d" +smmuv3_cmdq_tlbi_nh_va(int vmid, int asid, uint64_t addr, bool leaf) " = |_ vmid =3D%d asid =3D%d addr=3D0x%"PRIx64" leaf=3D%d" +smmuv3_cmdq_tlbi_nh_vaa(int vmid, uint64_t addr) " |_ vmid =3D%d addr= =3D0x%"PRIx64 +smmuv3_cmdq_tlbi_nh(void) "" +smmuv3_cmdq_tlbi_nh_asid(uint16_t asid) "asid=3D%d" smmuv3_cmdq_cfgi_cd(uint32_t sid) " |_ streamid =3D %d" smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, floa= t perc) "Config cache HIT for sid %d (hits=3D%d, misses=3D%d, hit rate=3D%.= 1f)" smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, flo= at perc) "Config cache MISS for sid %d (hits=3D%d, misses=3D%d, hit rate=3D= %.1f)" diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index 7ce95ca..d173806 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -67,6 +67,8 @@ typedef struct SMMUTransCfg { uint8_t tbi; /* Top Byte Ignore */ uint16_t asid; SMMUTransTableInfo tt[2]; + uint32_t iotlb_hits; /* counts IOTLB hits for this asid */ + uint32_t iotlb_misses; /* counts IOTLB misses for this asid */ } SMMUTransCfg; =20 typedef struct SMMUDevice { @@ -89,6 +91,11 @@ typedef struct SMMUPciBus { SMMUDevice *pbdev[0]; /* Parent array is sparse, so dynamically allo= c */ } SMMUPciBus; =20 +typedef struct SMMUIOTLBKey { + uint64_t iova; + uint16_t asid; +} SMMUIOTLBKey; + typedef struct SMMUState { /* */ SysBusDevice dev; @@ -147,4 +154,10 @@ SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_a= ddr_t iova); /* Return the iommu mr associated to @sid, or NULL if none */ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid); =20 +#define SMMU_IOTLB_MAX_SIZE 256 + +void smmu_iotlb_inv_all(SMMUState *s); +void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid); +void smmu_iotlb_inv_iova(SMMUState *s, uint16_t asid, dma_addr_t iova); + #endif /* HW_ARM_SMMU_COMMON */ --=20 1.8.3.1 From nobody Tue Apr 23 07:10:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526481129829982.5660699453555; Wed, 16 May 2018 07:32:09 -0700 (PDT) Received: from localhost ([::1]:48985 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIxTM-0006Sz-Pl for importer@patchew.org; Wed, 16 May 2018 10:32:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42642) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIxOi-0002tg-Q5 for qemu-devel@nongnu.org; Wed, 16 May 2018 10:27:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIxOe-0000SS-E7 for qemu-devel@nongnu.org; Wed, 16 May 2018 10:27:20 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:60724 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIxOV-0000P7-3l; Wed, 16 May 2018 10:27:07 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AF945BB40E; Wed, 16 May 2018 14:27:06 +0000 (UTC) Received: from dhcp19-241-177.khw3.lab.eng.bos.redhat.com (gigabyte-r120-16.khw3.lab.eng.bos.redhat.com [10.19.240.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6FD39112D199; Wed, 16 May 2018 14:27:06 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 16 May 2018 14:31:14 -0400 Message-Id: <1526495474-26394-4-git-send-email-eric.auger@redhat.com> In-Reply-To: <1526495474-26394-1-git-send-email-eric.auger@redhat.com> References: <1526495474-26394-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 16 May 2018 14:27:06 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 16 May 2018 14:27:06 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH 3/3] hw/arm/smmuv3: Add notifications on invalidation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" On TLB invalidation commands, let's call registered IOMMU notifiers. Those can only be UNMAP notifiers. SMMUv3 does not support notification on MAP (VFIO). This patch allows vhost use case where IOTLB API is notified on each guest IOTLB invalidation. Signed-off-by: Eric Auger --- hw/arm/smmu-common.c | 34 +++++++++++++++ hw/arm/smmuv3.c | 99 ++++++++++++++++++++++++++++++++++++++++= +++- hw/arm/trace-events | 5 +++ include/hw/arm/smmu-common.h | 6 +++ 4 files changed, 142 insertions(+), 2 deletions(-) diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index e4408c4..9caa149 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -386,6 +386,40 @@ static gboolean smmu_iotlb_key_equal(gconstpointer v1,= gconstpointer v2) return (k1->asid =3D=3D k2->asid) && (k1->iova =3D=3D k2->iova); } =20 +/* Unmap the whole notifier's range */ +static void smmu_unmap_notifier_range(IOMMUNotifier *n) +{ + IOMMUTLBEntry entry; + + entry.target_as =3D &address_space_memory; + entry.iova =3D n->start; + entry.perm =3D IOMMU_NONE; + entry.addr_mask =3D n->end - n->start; + + memory_region_notify_one(n, &entry); +} + +/* Unmap all notifiers attached to @mr */ +inline void smmu_inv_notifiers_mr(IOMMUMemoryRegion *mr) +{ + IOMMUNotifier *n; + + trace_smmu_inv_notifiers_mr(mr->parent_obj.name); + IOMMU_NOTIFIER_FOREACH(n, mr) { + smmu_unmap_notifier_range(n); + } +} + +/* Unmap all notifiers of all mr's */ +void smmu_inv_notifiers_all(SMMUState *s) +{ + SMMUNotifierNode *node; + + QLIST_FOREACH(node, &s->notifiers_list, next) { + smmu_inv_notifiers_mr(&node->sdev->iommu); + } +} + static void smmu_base_realize(DeviceState *dev, Error **errp) { SMMUState *s =3D ARM_SMMU(dev); diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index 6c89a71..01ec6ee 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -744,6 +744,68 @@ out: return entry; } =20 +/** + * smmuv3_notify_iova - call the notifier @n for a given + * @asid and @iova tuple. + * + * @mr: IOMMU mr region handle + * @n: notifier to be called + * @asid: address space ID or negative value if we don't care + * @iova: iova + */ +static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, + IOMMUNotifier *n, + int asid, + dma_addr_t iova) +{ + SMMUDevice *sdev =3D container_of(mr, SMMUDevice, iommu); + SMMUEventInfo event =3D {}; + SMMUTransTableInfo *tt; + SMMUTransCfg *cfg; + IOMMUTLBEntry entry; + + cfg =3D smmuv3_get_config(sdev, &event); + if (!cfg) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s error decoding the configuration for iommu mr=3D= %s\n", + __func__, mr->parent_obj.name); + return; + } + + if (asid >=3D 0 && cfg->asid !=3D asid) { + return; + } + + tt =3D select_tt(cfg, iova); + if (!tt) { + return; + } + + entry.target_as =3D &address_space_memory; + entry.iova =3D iova; + entry.addr_mask =3D (1 << tt->granule_sz) - 1; + entry.perm =3D IOMMU_NONE; + + memory_region_notify_one(n, &entry); +} + +/* invalidate an asid/iova tuple in all mr's */ +static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t i= ova) +{ + SMMUNotifierNode *node; + + QLIST_FOREACH(node, &s->notifiers_list, next) { + IOMMUMemoryRegion *mr =3D &node->sdev->iommu; + IOMMUNotifier *n; + + trace_smmuv3_inv_notifiers_iova(mr->parent_obj.name, asid, iova); + + IOMMU_NOTIFIER_FOREACH(n, mr) { + smmuv3_notify_iova(mr, n, asid, iova); + } + } +} + static int smmuv3_cmdq_consume(SMMUv3State *s) { SMMUState *bs =3D ARM_SMMU(s); @@ -862,12 +924,14 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) uint16_t asid =3D CMD_ASID(&cmd); =20 trace_smmuv3_cmdq_tlbi_nh_asid(asid); + smmu_inv_notifiers_all(&s->smmu_state); smmu_iotlb_inv_asid(bs, asid); break; } case SMMU_CMD_TLBI_NH_ALL: case SMMU_CMD_TLBI_NSNH_ALL: trace_smmuv3_cmdq_tlbi_nh(); + smmu_inv_notifiers_all(&s->smmu_state); smmu_iotlb_inv_all(bs); break; case SMMU_CMD_TLBI_NH_VAA: @@ -876,6 +940,7 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) uint16_t vmid =3D CMD_VMID(&cmd); =20 trace_smmuv3_cmdq_tlbi_nh_vaa(vmid, addr); + smmuv3_inv_notifiers_iova(bs, -1, addr); smmu_iotlb_inv_all(bs); break; } @@ -887,6 +952,7 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) bool leaf =3D CMD_LEAF(&cmd); =20 trace_smmuv3_cmdq_tlbi_nh_va(vmid, asid, addr, leaf); + smmuv3_inv_notifiers_iova(bs, asid, addr); smmu_iotlb_inv_iova(bs, asid, addr); break; } @@ -1362,9 +1428,38 @@ static void smmuv3_notify_flag_changed(IOMMUMemoryRe= gion *iommu, IOMMUNotifierFlag old, IOMMUNotifierFlag new) { + SMMUDevice *sdev =3D container_of(iommu, SMMUDevice, iommu); + SMMUv3State *s3 =3D sdev->smmu; + SMMUState *s =3D &(s3->smmu_state); + SMMUNotifierNode *node =3D NULL; + SMMUNotifierNode *next_node =3D NULL; + + if (new =3D=3D IOMMU_NOTIFIER_MAP) { + int bus_num =3D pci_bus_num(sdev->bus); + PCIDevice *pcidev =3D pci_find_device(sdev->bus, bus_num, sdev->de= vfn); + + warn_report("SMMUv3 does not support notification on MAP: " + "device %s will not function properly", pcidev->name); + } + if (old =3D=3D IOMMU_NOTIFIER_NONE) { - warn_report("SMMUV3 does not support vhost/vfio integration yet: " - "devices of those types will not function properly"); + trace_smmuv3_notify_flag_add(iommu->parent_obj.name); + node =3D g_malloc0(sizeof(*node)); + node->sdev =3D sdev; + QLIST_INSERT_HEAD(&s->notifiers_list, node, next); + return; + } + + /* update notifier node with new flags */ + QLIST_FOREACH_SAFE(node, &s->notifiers_list, next, next_node) { + if (node->sdev =3D=3D sdev) { + if (new =3D=3D IOMMU_NOTIFIER_NONE) { + trace_smmuv3_notify_flag_del(iommu->parent_obj.name); + QLIST_REMOVE(node, next); + g_free(node); + } + return; + } } } =20 diff --git a/hw/arm/trace-events b/hw/arm/trace-events index 3eb9180..f1198f7 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -17,6 +17,7 @@ smmu_iotlb_cache_miss(uint16_t asid, uint64_t addr, uint3= 2_t hit, uint32_t miss, smmu_iotlb_inv_all(void) "IOTLB invalidate all" smmu_iotlb_inv_asid(uint16_t asid) "IOTLB invalidate asid=3D%d" smmu_iotlb_inv_iova(uint16_t asid, uint64_t addr) "IOTLB invalidate asid= =3D%d addr=3D0x%"PRIx64 +smmu_inv_notifiers_mr(const char *name) "iommu mr=3D%s" =20 #hw/arm/smmuv3.c smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "= addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)" @@ -55,3 +56,7 @@ smmuv3_cmdq_cfgi_cd(uint32_t sid) " |_ streamid =3D %= d" smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, floa= t perc) "Config cache HIT for sid %d (hits=3D%d, misses=3D%d, hit rate=3D%.= 1f)" smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, flo= at perc) "Config cache MISS for sid %d (hits=3D%d, misses=3D%d, hit rate=3D= %.1f)" smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid %d" +smmuv3_notify_flag_add(const char *iommu) "ADD SMMUNotifier node for iommu= mr=3D%s" +smmuv3_notify_flag_del(const char *iommu) "DEL SMMUNotifier node for iommu= mr=3D%s" +smmuv3_inv_notifiers_iova(const char *name, uint16_t asid, uint64_t iova) = "iommu mr=3D%s asid=3D%d iova=3D0x%"PRIx64 + diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index d173806..50e2912 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -160,4 +160,10 @@ void smmu_iotlb_inv_all(SMMUState *s); void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid); void smmu_iotlb_inv_iova(SMMUState *s, uint16_t asid, dma_addr_t iova); =20 +/* Unmap the range of all the notifiers registered to any IOMMU mr */ +void smmu_inv_notifiers_all(SMMUState *s); + +/* Unmap the range of all the notifiers registered to @mr */ +void smmu_inv_notifiers_mr(IOMMUMemoryRegion *mr); + #endif /* HW_ARM_SMMU_COMMON */ --=20 1.8.3.1