From nobody Tue Feb 10 01:31:08 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623662238903478.1886275659398; Mon, 14 Jun 2021 02:17:18 -0700 (PDT) Received: from localhost ([::1]:55786 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsiib-0004Wv-23 for importer@patchew.org; Mon, 14 Jun 2021 05:17:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50650) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lsiXa-00016F-0j for qemu-devel@nongnu.org; Mon, 14 Jun 2021 05:05:55 -0400 Received: from foss.arm.com ([217.140.110.172]:50934) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsiXX-0005rg-E8 for qemu-devel@nongnu.org; Mon, 14 Jun 2021 05:05:53 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 192511FB; Mon, 14 Jun 2021 02:05:50 -0700 (PDT) Received: from e112269-lin.arm.com (autoplooker.cambridge.arm.com [10.1.194.57]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4C3F83F694; Mon, 14 Jun 2021 02:05:47 -0700 (PDT) From: Steven Price To: Catalin Marinas , Marc Zyngier , Will Deacon Subject: [PATCH v15 3/7] KVM: arm64: Introduce MTE VM feature Date: Mon, 14 Jun 2021 10:05:21 +0100 Message-Id: <20210614090525.4338-4-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210614090525.4338-1-steven.price@arm.com> References: <20210614090525.4338-1-steven.price@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=217.140.110.172; envelope-from=steven.price@arm.com; helo=foss.arm.com X-Spam_score_int: -68 X-Spam_score: -6.9 X-Spam_bar: ------ X-Spam_report: (-6.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Peter Maydell , "Dr. David Alan Gilbert" , Andrew Jones , Haibo Xu , Suzuki K Poulose , qemu-devel@nongnu.org, Dave Martin , Juan Quintela , Richard Henderson , linux-kernel@vger.kernel.org, Steven Price , James Morse , Julien Thierry , Thomas Gleixner , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Add a new VM feature 'KVM_ARM_CAP_MTE' which enables memory tagging for a VM. This will expose the feature to the guest and automatically tag memory pages touched by the VM as PG_mte_tagged (and clear the tag storage) to ensure that the guest cannot see stale tags, and so that the tags are correctly saved/restored across swap. Actually exposing the new capability to user space happens in a later patch. Reviewed-by: Catalin Marinas Signed-off-by: Steven Price --- arch/arm64/include/asm/kvm_emulate.h | 3 ++ arch/arm64/include/asm/kvm_host.h | 3 ++ arch/arm64/include/asm/mte.h | 4 +++ arch/arm64/kernel/mte.c | 17 +++++++++++ arch/arm64/kvm/hyp/exception.c | 3 +- arch/arm64/kvm/mmu.c | 42 +++++++++++++++++++++++++++- arch/arm64/kvm/sys_regs.c | 7 +++++ include/uapi/linux/kvm.h | 1 + 8 files changed, 78 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/= kvm_emulate.h index f612c090f2e4..6bf776c2399c 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -84,6 +84,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu) if (cpus_have_const_cap(ARM64_MISMATCHED_CACHE_TYPE) || vcpu_el1_is_32bit(vcpu)) vcpu->arch.hcr_el2 |=3D HCR_TID2; + + if (kvm_has_mte(vcpu->kvm)) + vcpu->arch.hcr_el2 |=3D HCR_ATA; } =20 static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 7cd7d5c8c4bc..afaa5333f0e4 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -132,6 +132,8 @@ struct kvm_arch { =20 u8 pfr0_csv2; u8 pfr0_csv3; + /* Memory Tagging Extension enabled for the guest */ + bool mte_enabled; }; =20 struct kvm_vcpu_fault_info { @@ -769,6 +771,7 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu); #define kvm_arm_vcpu_sve_finalized(vcpu) \ ((vcpu)->arch.flags & KVM_ARM64_VCPU_SVE_FINALIZED) =20 +#define kvm_has_mte(kvm) (system_supports_mte() && (kvm)->arch.mte_enabled) #define kvm_vcpu_has_pmu(vcpu) \ (test_bit(KVM_ARM_VCPU_PMU_V3, (vcpu)->arch.features)) =20 diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 347ef38a35f7..be1de541a11c 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -37,6 +37,7 @@ void mte_free_tag_storage(char *storage); /* track which pages have valid allocation tags */ #define PG_mte_tagged PG_arch_2 =20 +void mte_prepare_page_tags(struct page *page); void mte_sync_tags(pte_t old_pte, pte_t pte); void mte_copy_page_tags(void *kto, const void *kfrom); void mte_thread_init_user(void); @@ -53,6 +54,9 @@ int mte_ptrace_copy_tags(struct task_struct *child, long = request, /* unused if !CONFIG_ARM64_MTE, silence the compiler */ #define PG_mte_tagged 0 =20 +static inline void mte_prepare_page_tags(struct page *page) +{ +} static inline void mte_sync_tags(pte_t old_pte, pte_t pte) { } diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index ae0a3c68fece..b120f82a2258 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -72,6 +72,23 @@ static void mte_sync_page_tags(struct page *page, pte_t = old_pte, spin_unlock_irqrestore(&tag_sync_lock, flags); } =20 +void mte_prepare_page_tags(struct page *page) +{ + unsigned long flags; + + spin_lock_irqsave(&tag_sync_lock, flags); + + /* Recheck with the lock held */ + if (test_bit(PG_mte_tagged, &page->flags)) + goto out; + + mte_clear_page_tags(page_address(page)); + set_bit(PG_mte_tagged, &page->flags); + +out: + spin_unlock_irqrestore(&tag_sync_lock, flags); +} + void mte_sync_tags(pte_t old_pte, pte_t pte) { struct page *page =3D pte_page(pte); diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c index 73629094f903..56426565600c 100644 --- a/arch/arm64/kvm/hyp/exception.c +++ b/arch/arm64/kvm/hyp/exception.c @@ -112,7 +112,8 @@ static void enter_exception64(struct kvm_vcpu *vcpu, un= signed long target_mode, new |=3D (old & PSR_C_BIT); new |=3D (old & PSR_V_BIT); =20 - // TODO: TCO (if/when ARMv8.5-MemTag is exposed to guests) + if (kvm_has_mte(vcpu->kvm)) + new |=3D PSR_TCO_BIT; =20 new |=3D (old & PSR_DIT_BIT); =20 diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index c5d1f3c87dbd..ed7c624e7362 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -822,6 +822,36 @@ transparent_hugepage_adjust(struct kvm_memory_slot *me= mslot, return PAGE_SIZE; } =20 +static int sanitise_mte_tags(struct kvm *kvm, kvm_pfn_t pfn, + unsigned long size) +{ + unsigned long i, nr_pages =3D size >> PAGE_SHIFT; + struct page *page; + + if (!kvm_has_mte(kvm)) + return 0; + + /* + * The page will be mapped in stage 2 as Normal Cacheable, so + * the VM will be able to see the page's tags and therefore + * they must be initialised first. If PG_mte_tagged is set, + * tags have already been initialised. + * pfn_to_online_page() is used to reject ZONE_DEVICE pages + * that may not support tags. + */ + page =3D pfn_to_online_page(pfn); + + if (!page) + return -EFAULT; + + for (i =3D 0; i < nr_pages; i++, page++) { + if (!test_bit(PG_mte_tagged, &page->flags)) + mte_prepare_page_tags(page); + } + + return 0; +} + static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, struct kvm_memory_slot *memslot, unsigned long hva, unsigned long fault_status) @@ -971,8 +1001,13 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys= _addr_t fault_ipa, if (writable) prot |=3D KVM_PGTABLE_PROT_W; =20 - if (fault_status !=3D FSC_PERM && !device) + if (fault_status !=3D FSC_PERM && !device) { + ret =3D sanitise_mte_tags(kvm, pfn, vma_pagesize); + if (ret) + goto out_unlock; + clean_dcache_guest_page(pfn, vma_pagesize); + } =20 if (exec_fault) { prot |=3D KVM_PGTABLE_PROT_X; @@ -1168,12 +1203,17 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kv= m_gfn_range *range) bool kvm_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range) { kvm_pfn_t pfn =3D pte_pfn(range->pte); + int ret; =20 if (!kvm->arch.mmu.pgt) return 0; =20 WARN_ON(range->end - range->start !=3D 1); =20 + ret =3D sanitise_mte_tags(kvm, pfn, PAGE_SIZE); + if (ret) + return false; + /* * We've moved a page around, probably through CoW, so let's treat it * just like a translation fault and clean the cache to the PoC. diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 76ea2800c33e..4a98902eaf1a 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1047,6 +1047,13 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu, break; case SYS_ID_AA64PFR1_EL1: val &=3D ~FEATURE(ID_AA64PFR1_MTE); + if (kvm_has_mte(vcpu->kvm)) { + u64 pfr, mte; + + pfr =3D read_sanitised_ftr_reg(SYS_ID_AA64PFR1_EL1); + mte =3D cpuid_feature_extract_unsigned_field(pfr, ID_AA64PFR1_MTE_SHIFT= ); + val |=3D FIELD_PREP(FEATURE(ID_AA64PFR1_MTE), mte); + } break; case SYS_ID_AA64ISAR1_EL1: if (!vcpu_has_ptrauth(vcpu)) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 3fd9a7e9d90c..8c95ba0fadda 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1082,6 +1082,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_SGX_ATTRIBUTE 196 #define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197 #define KVM_CAP_PTP_KVM 198 +#define KVM_CAP_ARM_MTE 199 =20 #ifdef KVM_CAP_IRQ_ROUTING =20 --=20 2.20.1