From nobody Mon Oct 27 18:34:20 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521549649666252.29150017191364; Tue, 20 Mar 2018 05:40:49 -0700 (PDT) Received: from localhost ([::1]:48213 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyGZM-0000t9-Lt for importer@patchew.org; Tue, 20 Mar 2018 08:40:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42719) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyGWw-0007Oq-Sw for qemu-devel@nongnu.org; Tue, 20 Mar 2018 08:38:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eyGWt-0003Jf-D5 for qemu-devel@nongnu.org; Tue, 20 Mar 2018 08:38:18 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:44098 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eyGWo-0003DB-6d; Tue, 20 Mar 2018 08:38:10 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9B4468163C52; Tue, 20 Mar 2018 12:38:09 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-135.ams2.redhat.com [10.36.116.135]) by smtp.corp.redhat.com (Postfix) with ESMTP id BB75394583; Tue, 20 Mar 2018 12:37:57 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, prem.mallappa@gmail.com Date: Tue, 20 Mar 2018 13:37:25 +0100 Message-Id: <1521549456-14072-4-git-send-email-eric.auger@redhat.com> In-Reply-To: <1521549456-14072-1-git-send-email-eric.auger@redhat.com> References: <1521549456-14072-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 20 Mar 2018 12:38:09 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 20 Mar 2018 12:38:09 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v10 03/14] hw/arm/smmu-common: VMSAv8-64 page table walk X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, tn@semihalf.com, peterx@redhat.com, alex.williamson@redhat.com, linuc.decode@gmail.com, bharat.bhushan@nxp.com, jintack@cs.columbia.edu Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch implements the page table walk for VMSAv8-64. Signed-off-by: Eric Auger Signed-off-by: Prem Mallappa --- v9 -> v10: - Add 64b single-copy atomicity comment related to PTE fetch - remove checks in get_block_pte_address and use formulae to compute block address offset - remove check_perm - fix select_tt - in trace-events use PRIx64 instead of 0x%lx - commented the fact we cannot discriminate between user and priviledged transactions v8 -> v9: - remove guest error log on PTE fetch fault - rename trace functions - fix smmu_page_walk_level_res_invalid_pte last arg - fix PTE_ADDRESS - turn functions into macros - make sure to return the actual pte access permission into tlbe->perm - change proto of smmu_ptw* v7 -> v8: - rework get_pte - use LOG_LEVEL_ERROR - remove error checking in get_block_pte_address - page table walk simplified (no VFIO replay anymore) - handle PTW error events - use dma_memory_read v6 -> v7: - fix wrong error handling in walk_page_table - check perm in smmu_translate v5 -> v6: - use IOMMUMemoryRegion - remove initial_lookup_level() - fix block replay v4 -> v5: - add initial level in translation config - implement block pte - rename must_translate into nofail - introduce call_entry_hook - small changes to dynamic traces - smmu_page_walk code moved from smmuv3.c to this file - remove smmu_translate* v3 -> v4: - reworked page table walk to prepare for VFIO integration (capability to scan a range of IOVA). Same function is used for translate for a single iova. This is largely inspired from intel_iommu.c - as the translate function was not straightforward to me, I tried to stick more closely to the VMSA spec. - remove support of nested stage (kernel driver does not support it anyway) - use error_report and trace events - add aa64[] field in SMMUTransCfg --- hw/arm/smmu-common.c | 209 +++++++++++++++++++++++++++++++++++++++= ++++ hw/arm/smmu-internal.h | 102 +++++++++++++++++++++ hw/arm/trace-events | 9 +- include/hw/arm/smmu-common.h | 14 +++ 4 files changed, 333 insertions(+), 1 deletion(-) create mode 100644 hw/arm/smmu-internal.h diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index fe0cd70..c7a9027 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -27,6 +27,215 @@ =20 #include "qemu/error-report.h" #include "hw/arm/smmu-common.h" +#include "smmu-internal.h" + +/* VMSAv8-64 Translation */ + +/** + * get_pte - Get the content of a page table entry located t + * @base_addr[@index] + */ +static int get_pte(dma_addr_t baseaddr, uint32_t index, uint64_t *pte, + SMMUPTWEventInfo *info) +{ + int ret; + dma_addr_t addr =3D baseaddr + index * sizeof(*pte); + + /* TODO: guarantee 64-bit single-copy atomicity */ + ret =3D dma_memory_read(&address_space_memory, addr, + (uint8_t *)pte, sizeof(*pte)); + + if (ret !=3D MEMTX_OK) { + info->type =3D SMMU_PTW_ERR_WALK_EABT; + info->addr =3D addr; + return -EINVAL; + } + trace_smmu_get_pte(baseaddr, index, addr, *pte); + return 0; +} + +/* VMSAv8-64 Translation Table Format Descriptor Decoding */ + +/** + * get_page_pte_address - returns the L3 descriptor output address, + * ie. the page frame + * ARM ARM spec: Figure D4-17 VMSAv8-64 level 3 descriptor format + */ +static inline hwaddr get_page_pte_address(uint64_t pte, int granule_sz) +{ + return PTE_ADDRESS(pte, granule_sz); +} + +/** + * get_table_pte_address - return table descriptor output address, + * ie. address of next level table + * ARM ARM Figure D4-16 VMSAv8-64 level0, level1, and level 2 descriptor f= ormats + */ +static inline hwaddr get_table_pte_address(uint64_t pte, int granule_sz) +{ + return PTE_ADDRESS(pte, granule_sz); +} + +/** + * get_block_pte_address - return block descriptor output address and bloc= k size + * ARM ARM Figure D4-16 VMSAv8-64 level0, level1, and level 2 descriptor f= ormats + */ +static inline hwaddr get_block_pte_address(uint64_t pte, int level, + int granule_sz, uint64_t *bsz) +{ + int n =3D (granule_sz - 3) * (4 - level) + 3; + + *bsz =3D 1 << n; + return PTE_ADDRESS(pte, n); +} + +SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova) +{ + bool tbi =3D extract64(iova, 55, 1) ? TBI1(cfg->tbi) : TBI0(cfg->tbi); + uint8_t tbi_byte =3D tbi * 8; + + if (!extract64(iova, 64 - cfg->tt[0].tsz, cfg->tt[0].tsz - tbi_byte)) { + return &cfg->tt[0]; + } else if (!extract64(iova, 64 - cfg->tt[1].tsz, + cfg->tt[1].tsz - tbi_byte)) { + return &cfg->tt[1]; + } + return NULL; +} + +/** + * smmu_ptw_64 - VMSAv8-64 Walk of the page tables for a given IOVA + * @cfg: translation config + * @iova: iova to translate + * @perm: access type + * @tlbe: IOMMUTLBEntry (out) + * @info: handle to an error info + * + * Return 0 on success, < 0 on error. In case of error, @info is filled + * and tlbe->perm is set to IOMMU_NONE. + * Upon success, @tlbe is filled with translated_addr and entry + * permission rights. + */ +static int smmu_ptw_64(SMMUTransCfg *cfg, + dma_addr_t iova, IOMMUAccessFlags perm, + IOMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) +{ + dma_addr_t baseaddr; + int stage =3D cfg->stage; + SMMUTransTableInfo *tt =3D select_tt(cfg, iova); + uint8_t level; + uint8_t granule_sz; + + if (!tt || tt->disabled) { + info->type =3D SMMU_PTW_ERR_TRANSLATION; + goto error; + } + + level =3D tt->initial_level; + granule_sz =3D tt->granule_sz; + baseaddr =3D tt->ttb; + + tlbe->iova =3D iova; + tlbe->addr_mask =3D (1 << granule_sz) - 1; + + while (level <=3D 3) { + uint64_t subpage_size =3D 1ULL << level_shift(level, granule_sz); + uint64_t mask =3D subpage_size - 1; + uint32_t offset =3D iova_level_offset(iova, level, granule_sz); + uint64_t pte; + dma_addr_t pte_addr =3D baseaddr + offset * sizeof(pte); + uint8_t ap; + + if (get_pte(baseaddr, offset, &pte, info)) { + goto error; + } + trace_smmu_ptw_level(level, iova, subpage_size, + baseaddr, offset, pte); + + if (is_invalid_pte(pte) || is_reserved_pte(pte, level)) { + trace_smmu_ptw_invalid_pte(stage, level, baseaddr, + pte_addr, offset, pte); + info->type =3D SMMU_PTW_ERR_TRANSLATION; + goto error; + } + + if (is_page_pte(pte, level)) { + uint64_t gpa =3D get_page_pte_address(pte, granule_sz); + + ap =3D PTE_AP(pte); + if (is_permission_fault(ap, perm)) { + info->type =3D SMMU_PTW_ERR_PERMISSION; + goto error; + } + + tlbe->translated_addr =3D gpa + (iova & mask); + tlbe->perm =3D PTE_AP_TO_PERM(ap); + trace_smmu_ptw_page_pte(stage, level, iova, + baseaddr, pte_addr, pte, gpa); + return 0; + } + if (is_block_pte(pte, level)) { + uint64_t block_size; + hwaddr gpa =3D get_block_pte_address(pte, level, granule_sz, + &block_size); + + ap =3D PTE_AP(pte); + if (is_permission_fault(ap, perm)) { + info->type =3D SMMU_PTW_ERR_PERMISSION; + goto error; + } + + trace_smmu_ptw_block_pte(stage, level, baseaddr, + pte_addr, pte, iova, gpa, + block_size >> 20); + + tlbe->translated_addr =3D gpa + (iova & mask); + tlbe->perm =3D PTE_AP_TO_PERM(ap); + return 0; + } + + /* table pte */ + ap =3D PTE_APTABLE(pte); + + if (is_permission_fault(ap, perm)) { + info->type =3D SMMU_PTW_ERR_PERMISSION; + goto error; + } + baseaddr =3D get_table_pte_address(pte, granule_sz); + level++; + } + + info->type =3D SMMU_PTW_ERR_TRANSLATION; + +error: + tlbe->perm =3D IOMMU_NONE; + return -EINVAL; +} + +/** + * smmu_ptw - Walk the page tables for an IOVA, according to @cfg + * + * @cfg: translation configuration + * @iova: iova to translate + * @perm: tentative access type + * @tlbe: returned entry + * @info: ptw event handle + * + * return 0 on success + */ +inline int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags p= erm, + IOMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) +{ + if (!cfg->aa64) { + /* + * This code path is not entered as we check this while decoding + * the configuration data in the derived SMMU model. + */ + assert(0); + } + + return smmu_ptw_64(cfg, iova, perm, tlbe, info); +} =20 /** * The bus number is used for lookup when SID based invalidatation occurs. diff --git a/hw/arm/smmu-internal.h b/hw/arm/smmu-internal.h new file mode 100644 index 0000000..3a3a9f8 --- /dev/null +++ b/hw/arm/smmu-internal.h @@ -0,0 +1,102 @@ +/* + * ARM SMMU support - Internal API + * + * Copyright (c) 2017 Red Hat, Inc. + * Copyright (C) 2014-2016 Broadcom Corporation + * Written by Prem Mallappa, Eric Auger + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, see . + */ + +#ifndef HW_ARM_SMMU_INTERNAL_H +#define HW_ARM_SMMU_INTERNAL_H + +#define TBI0(tbi) ((tbi) & 0x1) +#define TBI1(tbi) ((tbi) & 0x2 >> 1) + +/* PTE Manipulation */ + +#define ARM_LPAE_PTE_TYPE_SHIFT 0 +#define ARM_LPAE_PTE_TYPE_MASK 0x3 + +#define ARM_LPAE_PTE_TYPE_BLOCK 1 +#define ARM_LPAE_PTE_TYPE_TABLE 3 + +#define ARM_LPAE_L3_PTE_TYPE_RESERVED 1 +#define ARM_LPAE_L3_PTE_TYPE_PAGE 3 + +#define ARM_LPAE_PTE_VALID (1 << 0) + +#define PTE_ADDRESS(pte, shift) \ + (extract64(pte, shift, 47 - shift + 1) << shift) + +#define is_invalid_pte(pte) (!(pte & ARM_LPAE_PTE_VALID)) + +#define is_reserved_pte(pte, level) \ + ((level =3D=3D 3) && = \ + ((pte & ARM_LPAE_PTE_TYPE_MASK) =3D=3D ARM_LPAE_L3_PTE_TYPE_RESERVED)) + +#define is_block_pte(pte, level) \ + ((level < 3) && \ + ((pte & ARM_LPAE_PTE_TYPE_MASK) =3D=3D ARM_LPAE_PTE_TYPE_BLOCK)) + +#define is_table_pte(pte, level) \ + ((level < 3) && \ + ((pte & ARM_LPAE_PTE_TYPE_MASK) =3D=3D ARM_LPAE_PTE_TYPE_TABLE)) + +#define is_page_pte(pte, level) \ + ((level =3D=3D 3) && = \ + ((pte & ARM_LPAE_PTE_TYPE_MASK) =3D=3D ARM_LPAE_L3_PTE_TYPE_PAGE)) + +/* access permissions */ + +#define PTE_AP(pte) \ + (extract64(pte, 6, 2)) + +#define PTE_APTABLE(pte) \ + (extract64(pte, 61, 2)) + +/* + * TODO: At the moment all transactions are considered as priviledged (EL1) + * as IOMMU translation callback does not pass user/priv attributes. + */ +#define is_permission_fault(ap, perm) \ + (((perm) & IOMMU_WO) && ((ap) & 0x2)) + +#define PTE_AP_TO_PERM(ap) \ + (IOMMU_ACCESS_FLAG(true, !((ap) & 0x2))) + +/* Level Indexing */ + +static inline int level_shift(int level, int granule_sz) +{ + return granule_sz + (3 - level) * (granule_sz - 3); +} + +static inline uint64_t level_page_mask(int level, int granule_sz) +{ + return ~(MAKE_64BIT_MASK(0, level_shift(level, granule_sz))); +} + +/** + * TODO: handle the case where the level resolves less than + * granule_sz -3 IA bits. + */ +static inline +uint64_t iova_level_offset(uint64_t iova, int level, int granule_sz) +{ + return (iova >> level_shift(level, granule_sz)) & + MAKE_64BIT_MASK(0, granule_sz - 3); +} + +#endif diff --git a/hw/arm/trace-events b/hw/arm/trace-events index 8e8b53c..524964a 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -4,4 +4,11 @@ virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." =20 # hw/arm/smmu-common.c -smmu_add_mr(const char *name) "%s" \ No newline at end of file +smmu_add_mr(const char *name) "%s" +smmu_page_walk(int stage, uint64_t baseaddr, int first_level, uint64_t sta= rt, uint64_t end) "stage=3D%d, baseaddr=3D0x%"PRIx64", first level=3D%d, st= art=3D0x%"PRIx64", end=3D0x%"PRIx64 +smmu_lookup_table(int level, uint64_t baseaddr, int granule_sz, uint64_t s= tart, uint64_t end, int flags, uint64_t subpage_size) "level=3D%d baseaddr= =3D0x%"PRIx64" granule=3D%d, start=3D0x%"PRIx64" end=3D0x%"PRIx64" flags=3D= %d subpage_size=3D0x%"PRIx64 +smmu_ptw_level(int level, uint64_t iova, size_t subpage_size, uint64_t bas= eaddr, uint32_t offset, uint64_t pte) "level=3D%d iova=3D0x%"PRIx64" subpag= e_sz=3D0x%lx baseaddr=3D0x%"PRIx64" offset=3D%d =3D> pte=3D0x%"PRIx64 +smmu_ptw_invalid_pte(int stage, int level, uint64_t baseaddr, uint64_t pte= addr, uint32_t offset, uint64_t pte) "stage=3D%d level=3D%d base@=3D0x%"PRI= x64" pte@=3D0x%"PRIx64" offset=3D%d pte=3D0x%"PRIx64 +smmu_ptw_page_pte(int stage, int level, uint64_t iova, uint64_t baseaddr,= uint64_t pteaddr, uint64_t pte, uint64_t address) "stage=3D%d level=3D%d i= ova=3D0x%"PRIx64" base@=3D0x%"PRIx64" pte@=3D0x%"PRIx64" pte=3D0x%"PRIx64" = page address =3D 0x%"PRIx64 +smmu_ptw_block_pte(int stage, int level, uint64_t baseaddr, uint64_t ptead= dr, uint64_t pte, uint64_t iova, uint64_t gpa, int bsize_mb) "stage=3D%d le= vel=3D%d base@=3D0x%"PRIx64" pte@=3D0x%"PRIx64" pte=3D0x%"PRIx64" iova=3D0x= %"PRIx64" block address =3D 0x%"PRIx64" block size =3D %d MiB" +smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte)= "baseaddr=3D0x%"PRIx64" index=3D0x%x, pteaddr=3D0x%"PRIx64", pte=3D0x%"PRI= x64 diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index 76cf4aa..4ccd131 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -129,4 +129,18 @@ static inline uint16_t smmu_get_sid(SMMUDevice *sdev) { return ((pci_bus_num(sdev->bus) & 0xff) << 8) | sdev->devfn; } + +/** + * smmu_ptw - Perform the page table walk for a given iova / access flags + * pair, according to @cfg translation config + */ +int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags perm, + IOMMUTLBEntry *tlbe, SMMUPTWEventInfo *info); + +/** + * select_tt - compute which translation table shall be used according + * the input iova and tranlsation config and return the TT specific info + */ +SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova); + #endif /* HW_ARM_SMMU_COMMON */ --=20 2.5.5