From nobody Mon Feb 9 16:18:38 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=1; a=rsa-sha256; t=1623065151; cv=none; d=zohomail.com; s=zohoarc; b=btmjcMj/bSOKdoAQS74wmNfl7FN3H4H5Upurxa70CQLQ9ox7iT2+myX8QxrKpZYou7HuYvmL5qQvFJc+TVatNi+heqfu1BCMXOK+y64O35XwWJ1wklsNVw72xam5sHUhepaqtPue7tj61W91wvXSF2SZqVoZsgAIGIXwPMvMu6Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1623065151; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vxIavHtmihUmrktQctHOMaLN+Bl0nZnAsfJRs9t2uzo=; b=nuA66JQsHUOiwk5qjX5AhVpGwM4gS/qDp+MxxcHfFZ75V/yT12CwSQBTo0DJvZicORqBtBqkFmVy23tMlQrp6ZAMeWDtqeY/Zc7qTd8/mLi9+BCdxNP+iO9rsPp7UJeWuwe7KUyWbjwZCaUQCc52NNvpfz8/e6L2/6GJqL5UCvg= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623065151159502.09413315152153; Mon, 7 Jun 2021 04:25:51 -0700 (PDT) Received: from localhost ([::1]:53888 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lqDOA-000824-0t for importer@patchew.org; Mon, 07 Jun 2021 07:25:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48424) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lqD7j-0002UL-Gr for qemu-devel@nongnu.org; Mon, 07 Jun 2021 07:08:51 -0400 Received: from foss.arm.com ([217.140.110.172]:51218) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lqD7d-0007aK-Ro for qemu-devel@nongnu.org; Mon, 07 Jun 2021 07:08:51 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3673D11D4; Mon, 7 Jun 2021 04:08:45 -0700 (PDT) Received: from e112269-lin.arm.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E9B0D3F73D; Mon, 7 Jun 2021 04:08:41 -0700 (PDT) From: Steven Price To: Catalin Marinas , Marc Zyngier , Will Deacon Subject: [PATCH v14 4/8] KVM: arm64: Introduce MTE VM feature Date: Mon, 7 Jun 2021 12:08:12 +0100 Message-Id: <20210607110816.25762-5-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210607110816.25762-1-steven.price@arm.com> References: <20210607110816.25762-1-steven.price@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=217.140.110.172; envelope-from=steven.price@arm.com; helo=foss.arm.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Peter Maydell , "Dr. David Alan Gilbert" , Andrew Jones , Haibo Xu , Suzuki K Poulose , qemu-devel@nongnu.org, Dave Martin , Juan Quintela , Richard Henderson , linux-kernel@vger.kernel.org, Steven Price , James Morse , Julien Thierry , Thomas Gleixner , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Add a new VM feature 'KVM_ARM_CAP_MTE' which enables memory tagging for a VM. This will expose the feature to the guest and automatically tag memory pages touched by the VM as PG_mte_tagged (and clear the tag storage) to ensure that the guest cannot see stale tags, and so that the tags are correctly saved/restored across swap. Actually exposing the new capability to user space happens in a later patch. Reviewed-by: Catalin Marinas Signed-off-by: Steven Price --- arch/arm64/include/asm/kvm_emulate.h | 3 ++ arch/arm64/include/asm/kvm_host.h | 3 ++ arch/arm64/include/asm/mte.h | 4 +++ arch/arm64/kernel/mte.c | 17 +++++++++++ arch/arm64/kvm/hyp/exception.c | 3 +- arch/arm64/kvm/mmu.c | 42 +++++++++++++++++++++++++++- arch/arm64/kvm/sys_regs.c | 7 +++++ include/uapi/linux/kvm.h | 1 + 8 files changed, 78 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/= kvm_emulate.h index f612c090f2e4..6bf776c2399c 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -84,6 +84,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu) if (cpus_have_const_cap(ARM64_MISMATCHED_CACHE_TYPE) || vcpu_el1_is_32bit(vcpu)) vcpu->arch.hcr_el2 |=3D HCR_TID2; + + if (kvm_has_mte(vcpu->kvm)) + vcpu->arch.hcr_el2 |=3D HCR_ATA; } =20 static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 7cd7d5c8c4bc..afaa5333f0e4 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -132,6 +132,8 @@ struct kvm_arch { =20 u8 pfr0_csv2; u8 pfr0_csv3; + /* Memory Tagging Extension enabled for the guest */ + bool mte_enabled; }; =20 struct kvm_vcpu_fault_info { @@ -769,6 +771,7 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu); #define kvm_arm_vcpu_sve_finalized(vcpu) \ ((vcpu)->arch.flags & KVM_ARM64_VCPU_SVE_FINALIZED) =20 +#define kvm_has_mte(kvm) (system_supports_mte() && (kvm)->arch.mte_enabled) #define kvm_vcpu_has_pmu(vcpu) \ (test_bit(KVM_ARM_VCPU_PMU_V3, (vcpu)->arch.features)) =20 diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 347ef38a35f7..be1de541a11c 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -37,6 +37,7 @@ void mte_free_tag_storage(char *storage); /* track which pages have valid allocation tags */ #define PG_mte_tagged PG_arch_2 =20 +void mte_prepare_page_tags(struct page *page); void mte_sync_tags(pte_t old_pte, pte_t pte); void mte_copy_page_tags(void *kto, const void *kfrom); void mte_thread_init_user(void); @@ -53,6 +54,9 @@ int mte_ptrace_copy_tags(struct task_struct *child, long = request, /* unused if !CONFIG_ARM64_MTE, silence the compiler */ #define PG_mte_tagged 0 =20 +static inline void mte_prepare_page_tags(struct page *page) +{ +} static inline void mte_sync_tags(pte_t old_pte, pte_t pte) { } diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index ae0a3c68fece..b120f82a2258 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -72,6 +72,23 @@ static void mte_sync_page_tags(struct page *page, pte_t = old_pte, spin_unlock_irqrestore(&tag_sync_lock, flags); } =20 +void mte_prepare_page_tags(struct page *page) +{ + unsigned long flags; + + spin_lock_irqsave(&tag_sync_lock, flags); + + /* Recheck with the lock held */ + if (test_bit(PG_mte_tagged, &page->flags)) + goto out; + + mte_clear_page_tags(page_address(page)); + set_bit(PG_mte_tagged, &page->flags); + +out: + spin_unlock_irqrestore(&tag_sync_lock, flags); +} + void mte_sync_tags(pte_t old_pte, pte_t pte) { struct page *page =3D pte_page(pte); diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c index 73629094f903..56426565600c 100644 --- a/arch/arm64/kvm/hyp/exception.c +++ b/arch/arm64/kvm/hyp/exception.c @@ -112,7 +112,8 @@ static void enter_exception64(struct kvm_vcpu *vcpu, un= signed long target_mode, new |=3D (old & PSR_C_BIT); new |=3D (old & PSR_V_BIT); =20 - // TODO: TCO (if/when ARMv8.5-MemTag is exposed to guests) + if (kvm_has_mte(vcpu->kvm)) + new |=3D PSR_TCO_BIT; =20 new |=3D (old & PSR_DIT_BIT); =20 diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index c5d1f3c87dbd..ed7c624e7362 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -822,6 +822,36 @@ transparent_hugepage_adjust(struct kvm_memory_slot *me= mslot, return PAGE_SIZE; } =20 +static int sanitise_mte_tags(struct kvm *kvm, kvm_pfn_t pfn, + unsigned long size) +{ + unsigned long i, nr_pages =3D size >> PAGE_SHIFT; + struct page *page; + + if (!kvm_has_mte(kvm)) + return 0; + + /* + * The page will be mapped in stage 2 as Normal Cacheable, so + * the VM will be able to see the page's tags and therefore + * they must be initialised first. If PG_mte_tagged is set, + * tags have already been initialised. + * pfn_to_online_page() is used to reject ZONE_DEVICE pages + * that may not support tags. + */ + page =3D pfn_to_online_page(pfn); + + if (!page) + return -EFAULT; + + for (i =3D 0; i < nr_pages; i++, page++) { + if (!test_bit(PG_mte_tagged, &page->flags)) + mte_prepare_page_tags(page); + } + + return 0; +} + static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, struct kvm_memory_slot *memslot, unsigned long hva, unsigned long fault_status) @@ -971,8 +1001,13 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys= _addr_t fault_ipa, if (writable) prot |=3D KVM_PGTABLE_PROT_W; =20 - if (fault_status !=3D FSC_PERM && !device) + if (fault_status !=3D FSC_PERM && !device) { + ret =3D sanitise_mte_tags(kvm, pfn, vma_pagesize); + if (ret) + goto out_unlock; + clean_dcache_guest_page(pfn, vma_pagesize); + } =20 if (exec_fault) { prot |=3D KVM_PGTABLE_PROT_X; @@ -1168,12 +1203,17 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kv= m_gfn_range *range) bool kvm_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range) { kvm_pfn_t pfn =3D pte_pfn(range->pte); + int ret; =20 if (!kvm->arch.mmu.pgt) return 0; =20 WARN_ON(range->end - range->start !=3D 1); =20 + ret =3D sanitise_mte_tags(kvm, pfn, PAGE_SIZE); + if (ret) + return false; + /* * We've moved a page around, probably through CoW, so let's treat it * just like a translation fault and clean the cache to the PoC. diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 76ea2800c33e..4a98902eaf1a 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1047,6 +1047,13 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu, break; case SYS_ID_AA64PFR1_EL1: val &=3D ~FEATURE(ID_AA64PFR1_MTE); + if (kvm_has_mte(vcpu->kvm)) { + u64 pfr, mte; + + pfr =3D read_sanitised_ftr_reg(SYS_ID_AA64PFR1_EL1); + mte =3D cpuid_feature_extract_unsigned_field(pfr, ID_AA64PFR1_MTE_SHIFT= ); + val |=3D FIELD_PREP(FEATURE(ID_AA64PFR1_MTE), mte); + } break; case SYS_ID_AA64ISAR1_EL1: if (!vcpu_has_ptrauth(vcpu)) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 3fd9a7e9d90c..8c95ba0fadda 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1082,6 +1082,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_SGX_ATTRIBUTE 196 #define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197 #define KVM_CAP_PTP_KVM 198 +#define KVM_CAP_ARM_MTE 199 =20 #ifdef KVM_CAP_IRQ_ROUTING =20 --=20 2.20.1