From nobody Sun May 12 16:00:17 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=1; a=rsa-sha256; t=1610724603; cv=none; d=zohomail.com; s=zohoarc; b=HbaOwJCCYm4LMl/Gt1GMQnBp4+Uz16D19eHldWV+hddbWN5x6KvRJll14WYHrOHgsyg8FBRU4IjK3y3xARKoOJam+/aq77kH11NN8Sxu16xg7994E+J9TpfyeCnZyX59BF6nXULTcKGmfVqNiAPYYi1gfeHpccEZ89SYK61vpjM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1610724603; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Z04pTKEvkRVXTqRQAZ9+QsZzIgW9jrN3IzwAVfstlmM=; b=WPXoSGm5a06JElmN05LGZ8qYJZCPNy66caJGzJ41IhktFojlKR3AIATc/rGz58wIkeGSuloYLMP7zoyj321dYafHK+6KfRkTtMqnksjJ2bQXAOYo+Dg6AegvAwCim4XngYnf4pjDJAIGm9dh/60V0OSkjyvsiyPTfQkngrn0A5E= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1610724603348369.84713645743943; Fri, 15 Jan 2021 07:30:03 -0800 (PST) Received: from localhost ([::1]:45030 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0R33-0003gA-VN for importer@patchew.org; Fri, 15 Jan 2021 10:30:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41644) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0R1b-0001ym-TC for qemu-devel@nongnu.org; Fri, 15 Jan 2021 10:28:31 -0500 Received: from foss.arm.com ([217.140.110.172]:35494) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0R1V-0008LL-FQ for qemu-devel@nongnu.org; Fri, 15 Jan 2021 10:28:31 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DEAAA11B3; Fri, 15 Jan 2021 07:28:23 -0800 (PST) Received: from e112269-lin.arm.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2AC903F70D; Fri, 15 Jan 2021 07:28:21 -0800 (PST) From: Steven Price To: Catalin Marinas , Marc Zyngier , Will Deacon Subject: [PATCH v7 1/3] arm64: kvm: Save/restore MTE registers Date: Fri, 15 Jan 2021 15:28:09 +0000 Message-Id: <20210115152811.8398-2-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210115152811.8398-1-steven.price@arm.com> References: <20210115152811.8398-1-steven.price@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=217.140.110.172; envelope-from=steven.price@arm.com; helo=foss.arm.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Peter Maydell , "Dr. David Alan Gilbert" , Andrew Jones , Haibo Xu , Suzuki K Poulose , qemu-devel@nongnu.org, Dave Martin , Juan Quintela , Richard Henderson , linux-kernel@vger.kernel.org, Steven Price , James Morse , Julien Thierry , Thomas Gleixner , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Define the new system registers that MTE introduces and context switch them. The MTE feature is still hidden from the ID register as it isn't supported in a VM yet. Signed-off-by: Steven Price --- arch/arm64/include/asm/kvm_host.h | 4 ++ arch/arm64/include/asm/kvm_mte.h | 74 ++++++++++++++++++++++ arch/arm64/include/asm/sysreg.h | 3 +- arch/arm64/kernel/asm-offsets.c | 3 + arch/arm64/kvm/hyp/entry.S | 7 ++ arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 4 ++ arch/arm64/kvm/sys_regs.c | 14 ++-- 7 files changed, 104 insertions(+), 5 deletions(-) create mode 100644 arch/arm64/include/asm/kvm_mte.h diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 11beda85ee7e..51590a397e4b 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -148,6 +148,8 @@ enum vcpu_sysreg { SCTLR_EL1, /* System Control Register */ ACTLR_EL1, /* Auxiliary Control Register */ CPACR_EL1, /* Coprocessor Access Control */ + RGSR_EL1, /* Random Allocation Tag Seed Register */ + GCR_EL1, /* Tag Control Register */ ZCR_EL1, /* SVE Control */ TTBR0_EL1, /* Translation Table Base Register 0 */ TTBR1_EL1, /* Translation Table Base Register 1 */ @@ -164,6 +166,8 @@ enum vcpu_sysreg { TPIDR_EL1, /* Thread ID, Privileged */ AMAIR_EL1, /* Aux Memory Attribute Indirection Register */ CNTKCTL_EL1, /* Timer Control Register (EL1) */ + TFSRE0_EL1, /* Tag Fault Status Register (EL0) */ + TFSR_EL1, /* Tag Fault Stauts Register (EL1) */ PAR_EL1, /* Physical Address Register */ MDSCR_EL1, /* Monitor Debug System Control Register */ MDCCINT_EL1, /* Monitor Debug Comms Channel Interrupt Enable Reg */ diff --git a/arch/arm64/include/asm/kvm_mte.h b/arch/arm64/include/asm/kvm_= mte.h new file mode 100644 index 000000000000..62bbfae77f33 --- /dev/null +++ b/arch/arm64/include/asm/kvm_mte.h @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020 ARM Ltd. + */ +#ifndef __ASM_KVM_MTE_H +#define __ASM_KVM_MTE_H + +#ifdef __ASSEMBLY__ + +#include + +#ifdef CONFIG_ARM64_MTE + +.macro mte_switch_to_guest g_ctxt, h_ctxt, reg1 +alternative_if_not ARM64_MTE + b .L__skip_switch\@ +alternative_else_nop_endif + mrs \reg1, hcr_el2 + and \reg1, \reg1, #(HCR_ATA) + cbz \reg1, .L__skip_switch\@ + + mrs_s \reg1, SYS_RGSR_EL1 + str \reg1, [\h_ctxt, #CPU_RGSR_EL1] + mrs_s \reg1, SYS_GCR_EL1 + str \reg1, [\h_ctxt, #CPU_GCR_EL1] + mrs_s \reg1, SYS_TFSRE0_EL1 + str \reg1, [\h_ctxt, #CPU_TFSRE0_EL1] + + ldr \reg1, [\g_ctxt, #CPU_RGSR_EL1] + msr_s SYS_RGSR_EL1, \reg1 + ldr \reg1, [\g_ctxt, #CPU_GCR_EL1] + msr_s SYS_GCR_EL1, \reg1 + ldr \reg1, [\g_ctxt, #CPU_TFSRE0_EL1] + msr_s SYS_TFSRE0_EL1, \reg1 + +.L__skip_switch\@: +.endm + +.macro mte_switch_to_hyp g_ctxt, h_ctxt, reg1 +alternative_if_not ARM64_MTE + b .L__skip_switch\@ +alternative_else_nop_endif + mrs \reg1, hcr_el2 + and \reg1, \reg1, #(HCR_ATA) + cbz \reg1, .L__skip_switch\@ + + mrs_s \reg1, SYS_RGSR_EL1 + str \reg1, [\g_ctxt, #CPU_RGSR_EL1] + mrs_s \reg1, SYS_GCR_EL1 + str \reg1, [\g_ctxt, #CPU_GCR_EL1] + mrs_s \reg1, SYS_TFSRE0_EL1 + str \reg1, [\g_ctxt, #CPU_TFSRE0_EL1] + + ldr \reg1, [\h_ctxt, #CPU_RGSR_EL1] + msr_s SYS_RGSR_EL1, \reg1 + ldr \reg1, [\h_ctxt, #CPU_GCR_EL1] + msr_s SYS_GCR_EL1, \reg1 + ldr \reg1, [\h_ctxt, #CPU_TFSRE0_EL1] + msr_s SYS_TFSRE0_EL1, \reg1 + +.L__skip_switch\@: +.endm + +#else /* CONFIG_ARM64_MTE */ + +.macro mte_switch_to_guest g_ctxt, h_ctxt, reg1 +.endm + +.macro mte_switch_to_hyp g_ctxt, h_ctxt, reg1 +.endm + +#endif /* CONFIG_ARM64_MTE */ +#endif /* __ASSEMBLY__ */ +#endif /* __ASM_KVM_MTE_H */ diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysre= g.h index 8b5e7e5c3cc8..0a01975d331d 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -574,7 +574,8 @@ #define SCTLR_ELx_M (BIT(0)) =20 #define SCTLR_ELx_FLAGS (SCTLR_ELx_M | SCTLR_ELx_A | SCTLR_ELx_C | \ - SCTLR_ELx_SA | SCTLR_ELx_I | SCTLR_ELx_IESB) + SCTLR_ELx_SA | SCTLR_ELx_I | SCTLR_ELx_IESB | \ + SCTLR_ELx_ITFSB) =20 /* SCTLR_EL2 specific flags. */ #define SCTLR_EL2_RES1 ((BIT(4)) | (BIT(5)) | (BIT(11)) | (BIT(16)) | \ diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offset= s.c index f42fd9e33981..801531e1fa5c 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -105,6 +105,9 @@ int main(void) DEFINE(VCPU_WORKAROUND_FLAGS, offsetof(struct kvm_vcpu, arch.workaround_= flags)); DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2)); DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_cpu_context, regs)); + DEFINE(CPU_RGSR_EL1, offsetof(struct kvm_cpu_context, sys_regs[RGSR_EL1= ])); + DEFINE(CPU_GCR_EL1, offsetof(struct kvm_cpu_context, sys_regs[GCR_EL1])= ); + DEFINE(CPU_TFSRE0_EL1, offsetof(struct kvm_cpu_context, sys_regs[TFSRE0_= EL1])); DEFINE(CPU_APIAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIA= KEYLO_EL1])); DEFINE(CPU_APIBKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIB= KEYLO_EL1])); DEFINE(CPU_APDAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APDA= KEYLO_EL1])); diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index b0afad7a99c6..c67582c6dd55 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -13,6 +13,7 @@ #include #include #include +#include #include =20 .text @@ -51,6 +52,9 @@ alternative_else_nop_endif =20 add x29, x0, #VCPU_CONTEXT =20 + // mte_switch_to_guest(g_ctxt, h_ctxt, tmp1) + mte_switch_to_guest x29, x1, x2 + // Macro ptrauth_switch_to_guest format: // ptrauth_switch_to_guest(guest cxt, tmp1, tmp2, tmp3) // The below macro to restore guest keys is not implemented in C code @@ -140,6 +144,9 @@ SYM_INNER_LABEL(__guest_exit, SYM_L_GLOBAL) // when this feature is enabled for kernel code. ptrauth_switch_to_hyp x1, x2, x3, x4, x5 =20 + // mte_switch_to_hyp(g_ctxt, h_ctxt, reg1) + mte_switch_to_hyp x1, x2, x3 + // Restore hyp's sp_el0 restore_sp_el0 x2, x3 =20 diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hy= p/include/hyp/sysreg-sr.h index cce43bfe158f..94d9736f0133 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -45,6 +45,8 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu= _context *ctxt) ctxt_sys_reg(ctxt, CNTKCTL_EL1) =3D read_sysreg_el1(SYS_CNTKCTL); ctxt_sys_reg(ctxt, PAR_EL1) =3D read_sysreg_par(); ctxt_sys_reg(ctxt, TPIDR_EL1) =3D read_sysreg(tpidr_el1); + if (system_supports_mte()) + ctxt_sys_reg(ctxt, TFSR_EL1) =3D read_sysreg_el1(SYS_TFSR); =20 ctxt_sys_reg(ctxt, SP_EL1) =3D read_sysreg(sp_el1); ctxt_sys_reg(ctxt, ELR_EL1) =3D read_sysreg_el1(SYS_ELR); @@ -106,6 +108,8 @@ static inline void __sysreg_restore_el1_state(struct kv= m_cpu_context *ctxt) write_sysreg_el1(ctxt_sys_reg(ctxt, CNTKCTL_EL1), SYS_CNTKCTL); write_sysreg(ctxt_sys_reg(ctxt, PAR_EL1), par_el1); write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL1), tpidr_el1); + if (system_supports_mte()) + write_sysreg_el1(ctxt_sys_reg(ctxt, TFSR_EL1), SYS_TFSR); =20 if (!has_vhe() && cpus_have_final_cap(ARM64_WORKAROUND_SPECULATIVE_AT) && diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 3313dedfa505..88d4f360949e 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1281,6 +1281,12 @@ static bool access_ccsidr(struct kvm_vcpu *vcpu, str= uct sys_reg_params *p, return true; } =20 +static unsigned int mte_visibility(const struct kvm_vcpu *vcpu, + const struct sys_reg_desc *rd) +{ + return REG_HIDDEN; +} + /* sys_reg_desc initialiser for known cpufeature ID registers */ #define ID_SANITISED(name) { \ SYS_DESC(SYS_##name), \ @@ -1449,8 +1455,8 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { { SYS_DESC(SYS_ACTLR_EL1), access_actlr, reset_actlr, ACTLR_EL1 }, { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 }, =20 - { SYS_DESC(SYS_RGSR_EL1), undef_access }, - { SYS_DESC(SYS_GCR_EL1), undef_access }, + { SYS_DESC(SYS_RGSR_EL1), undef_access, reset_unknown, RGSR_EL1, .visibil= ity =3D mte_visibility }, + { SYS_DESC(SYS_GCR_EL1), undef_access, reset_unknown, GCR_EL1, .visibilit= y =3D mte_visibility }, =20 { SYS_DESC(SYS_ZCR_EL1), NULL, reset_val, ZCR_EL1, 0, .visibility =3D sve= _visibility }, { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 }, @@ -1476,8 +1482,8 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { { SYS_DESC(SYS_ERXMISC0_EL1), trap_raz_wi }, { SYS_DESC(SYS_ERXMISC1_EL1), trap_raz_wi }, =20 - { SYS_DESC(SYS_TFSR_EL1), undef_access }, - { SYS_DESC(SYS_TFSRE0_EL1), undef_access }, + { SYS_DESC(SYS_TFSR_EL1), undef_access, reset_unknown, TFSR_EL1, .visibil= ity =3D mte_visibility }, + { SYS_DESC(SYS_TFSRE0_EL1), undef_access, reset_unknown, TFSRE0_EL1, .vis= ibility =3D mte_visibility }, =20 { SYS_DESC(SYS_FAR_EL1), access_vm_reg, reset_unknown, FAR_EL1 }, { SYS_DESC(SYS_PAR_EL1), NULL, reset_unknown, PAR_EL1 }, --=20 2.20.1 From nobody Sun May 12 16:00:17 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=1; a=rsa-sha256; t=1610724604; cv=none; d=zohomail.com; s=zohoarc; b=Q/VdAVJoflPOMzQWScg1XRq1bQU+74YdJzLIrKGQwW1KXu/E/ECN4+bjv5R23p7kVj4/wEJq+oxaZObwtspMiCy2z5XDnf1nPIoPcHMxBNXPuYD7mbTkwjHhft9/fwKKwmI2Ud7B4Wnb7xlUd2mHKHx3B8bna6aIFBamej2GD+M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1610724604; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wUvyjFSusM9sSTTa1gQ4DRvKseyExxoCsMz30fuIj7Y=; b=IH/b/3UVSP64nf/rlssywDpHQO/8W0ITJrTiKTlguinHBrAkLUMWdFQOYdCzZ0xAYrFw/vAMrUqd9lFFvJVAcD/+kwf2b7Olibh1L62XO4a7iZJlueoXBjRRJ2rci5bSwwJoKngMY4gEY5PjKhHu00Jxu4yy3GwfKK3W1iyjaSs= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1610724604727892.1739567906584; Fri, 15 Jan 2021 07:30:04 -0800 (PST) Received: from localhost ([::1]:45180 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0R35-0003jd-Bk for importer@patchew.org; Fri, 15 Jan 2021 10:30:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41670) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0R1e-0001zy-2e for qemu-devel@nongnu.org; Fri, 15 Jan 2021 10:28:35 -0500 Received: from foss.arm.com ([217.140.110.172]:35518) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0R1X-0008MY-Vi for qemu-devel@nongnu.org; Fri, 15 Jan 2021 10:28:33 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D676F11FB; Fri, 15 Jan 2021 07:28:26 -0800 (PST) Received: from e112269-lin.arm.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 24E153F70D; Fri, 15 Jan 2021 07:28:24 -0800 (PST) From: Steven Price To: Catalin Marinas , Marc Zyngier , Will Deacon Subject: [PATCH v7 2/3] arm64: kvm: Introduce MTE VCPU feature Date: Fri, 15 Jan 2021 15:28:10 +0000 Message-Id: <20210115152811.8398-3-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210115152811.8398-1-steven.price@arm.com> References: <20210115152811.8398-1-steven.price@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=217.140.110.172; envelope-from=steven.price@arm.com; helo=foss.arm.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Peter Maydell , "Dr. David Alan Gilbert" , Andrew Jones , Haibo Xu , Suzuki K Poulose , qemu-devel@nongnu.org, Dave Martin , Juan Quintela , Richard Henderson , linux-kernel@vger.kernel.org, Steven Price , James Morse , Julien Thierry , Thomas Gleixner , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Add a new VM feature 'KVM_ARM_CAP_MTE' which enables memory tagging for a VM. This exposes the feature to the guest and automatically tags memory pages touched by the VM as PG_mte_tagged (and clears the tags storage) to ensure that the guest cannot see stale tags, and so that the tags are correctly saved/restored across swap. Signed-off-by: Steven Price --- arch/arm64/include/asm/kvm_emulate.h | 3 +++ arch/arm64/include/asm/kvm_host.h | 3 +++ arch/arm64/include/asm/pgtable.h | 2 +- arch/arm64/kernel/mte.c | 36 +++++++++++++++++----------- arch/arm64/kvm/arm.c | 9 +++++++ arch/arm64/kvm/hyp/exception.c | 3 ++- arch/arm64/kvm/mmu.c | 16 +++++++++++++ arch/arm64/kvm/sys_regs.c | 6 ++++- include/uapi/linux/kvm.h | 1 + 9 files changed, 62 insertions(+), 17 deletions(-) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/= kvm_emulate.h index f612c090f2e4..6bf776c2399c 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -84,6 +84,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu) if (cpus_have_const_cap(ARM64_MISMATCHED_CACHE_TYPE) || vcpu_el1_is_32bit(vcpu)) vcpu->arch.hcr_el2 |=3D HCR_TID2; + + if (kvm_has_mte(vcpu->kvm)) + vcpu->arch.hcr_el2 |=3D HCR_ATA; } =20 static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 51590a397e4b..1ca5785fb0e9 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -132,6 +132,8 @@ struct kvm_arch { =20 u8 pfr0_csv2; u8 pfr0_csv3; + /* Memory Tagging Extension enabled for the guest */ + bool mte_enabled; }; =20 struct kvm_vcpu_fault_info { @@ -749,6 +751,7 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu); #define kvm_arm_vcpu_sve_finalized(vcpu) \ ((vcpu)->arch.flags & KVM_ARM64_VCPU_SVE_FINALIZED) =20 +#define kvm_has_mte(kvm) (system_supports_mte() && (kvm)->arch.mte_enabled) #define kvm_vcpu_has_pmu(vcpu) \ (test_bit(KVM_ARM_VCPU_PMU_V3, (vcpu)->arch.features)) =20 diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 501562793ce2..27416d52f6a9 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -312,7 +312,7 @@ static inline void set_pte_at(struct mm_struct *mm, uns= igned long addr, __sync_icache_dcache(pte); =20 if (system_supports_mte() && - pte_present(pte) && pte_tagged(pte) && !pte_special(pte)) + pte_present(pte) && pte_valid_user(pte) && !pte_special(pte)) mte_sync_tags(ptep, pte); =20 __check_racy_pte_update(mm, ptep, pte); diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index dc9ada64feed..f9e089be1603 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -25,27 +25,33 @@ =20 u64 gcr_kernel_excl __ro_after_init; =20 -static void mte_sync_page_tags(struct page *page, pte_t *ptep, bool check_= swap) +static void mte_sync_page_tags(struct page *page, pte_t *ptep, bool check_= swap, + bool pte_is_tagged) { pte_t old_pte =3D READ_ONCE(*ptep); =20 if (check_swap && is_swap_pte(old_pte)) { swp_entry_t entry =3D pte_to_swp_entry(old_pte); =20 - if (!non_swap_entry(entry) && mte_restore_tags(entry, page)) + if (!non_swap_entry(entry) && mte_restore_tags(entry, page)) { + set_bit(PG_mte_tagged, &page->flags); return; + } } =20 - page_kasan_tag_reset(page); - /* - * We need smp_wmb() in between setting the flags and clearing the - * tags because if another thread reads page->flags and builds a - * tagged address out of it, there is an actual dependency to the - * memory access, but on the current thread we do not guarantee that - * the new page->flags are visible before the tags were updated. - */ - smp_wmb(); - mte_clear_page_tags(page_address(page)); + if (pte_is_tagged) { + set_bit(PG_mte_tagged, &page->flags); + page_kasan_tag_reset(page); + /* + * We need smp_wmb() in between setting the flags and clearing the + * tags because if another thread reads page->flags and builds a + * tagged address out of it, there is an actual dependency to the + * memory access, but on the current thread we do not guarantee that + * the new page->flags are visible before the tags were updated. + */ + smp_wmb(); + mte_clear_page_tags(page_address(page)); + } } =20 void mte_sync_tags(pte_t *ptep, pte_t pte) @@ -53,11 +59,13 @@ void mte_sync_tags(pte_t *ptep, pte_t pte) struct page *page =3D pte_page(pte); long i, nr_pages =3D compound_nr(page); bool check_swap =3D nr_pages =3D=3D 1; + bool pte_is_tagged =3D pte_tagged(pte); =20 /* if PG_mte_tagged is set, tags have already been initialised */ for (i =3D 0; i < nr_pages; i++, page++) { - if (!test_and_set_bit(PG_mte_tagged, &page->flags)) - mte_sync_page_tags(page, ptep, check_swap); + if (!test_bit(PG_mte_tagged, &page->flags)) + mte_sync_page_tags(page, ptep, check_swap, + pte_is_tagged); } } =20 diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 6e637d2b4cfb..f4c2fd2e7c49 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -97,6 +97,12 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, r =3D 0; kvm->arch.return_nisv_io_abort_to_user =3D true; break; + case KVM_CAP_ARM_MTE: + if (!system_supports_mte() || kvm->created_vcpus) + return -EINVAL; + r =3D 0; + kvm->arch.mte_enabled =3D true; + break; default: r =3D -EINVAL; break; @@ -238,6 +244,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long = ext) */ r =3D 1; break; + case KVM_CAP_ARM_MTE: + r =3D system_supports_mte(); + break; case KVM_CAP_STEAL_TIME: r =3D kvm_arm_pvtime_supported(); break; diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c index 73629094f903..56426565600c 100644 --- a/arch/arm64/kvm/hyp/exception.c +++ b/arch/arm64/kvm/hyp/exception.c @@ -112,7 +112,8 @@ static void enter_exception64(struct kvm_vcpu *vcpu, un= signed long target_mode, new |=3D (old & PSR_C_BIT); new |=3D (old & PSR_V_BIT); =20 - // TODO: TCO (if/when ARMv8.5-MemTag is exposed to guests) + if (kvm_has_mte(vcpu->kvm)) + new |=3D PSR_TCO_BIT; =20 new |=3D (old & PSR_DIT_BIT); =20 diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 7d2257cc5438..b9f9fb462de6 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -879,6 +879,22 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_= addr_t fault_ipa, if (vma_pagesize =3D=3D PAGE_SIZE && !force_pte) vma_pagesize =3D transparent_hugepage_adjust(memslot, hva, &pfn, &fault_ipa); + + if (kvm_has_mte(kvm) && pfn_valid(pfn)) { + /* + * VM will be able to see the page's tags, so we must ensure + * they have been initialised. + */ + struct page *page =3D pfn_to_page(pfn); + long i, nr_pages =3D compound_nr(page); + + /* if PG_mte_tagged is set, tags have already been initialised */ + for (i =3D 0; i < nr_pages; i++, page++) { + if (!test_and_set_bit(PG_mte_tagged, &page->flags)) + mte_clear_page_tags(page_address(page)); + } + } + if (writable) { prot |=3D KVM_PGTABLE_PROT_W; kvm_set_pfn_dirty(pfn); diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 88d4f360949e..57e5be14f1cc 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1029,7 +1029,8 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu, val &=3D ~(0xfUL << ID_AA64PFR0_CSV3_SHIFT); val |=3D ((u64)vcpu->kvm->arch.pfr0_csv3 << ID_AA64PFR0_CSV3_SHIFT); } else if (id =3D=3D SYS_ID_AA64PFR1_EL1) { - val &=3D ~(0xfUL << ID_AA64PFR1_MTE_SHIFT); + if (!kvm_has_mte(vcpu->kvm)) + val &=3D ~(0xfUL << ID_AA64PFR1_MTE_SHIFT); } else if (id =3D=3D SYS_ID_AA64ISAR1_EL1 && !vcpu_has_ptrauth(vcpu)) { val &=3D ~((0xfUL << ID_AA64ISAR1_APA_SHIFT) | (0xfUL << ID_AA64ISAR1_API_SHIFT) | @@ -1284,6 +1285,9 @@ static bool access_ccsidr(struct kvm_vcpu *vcpu, stru= ct sys_reg_params *p, static unsigned int mte_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd) { + if (kvm_has_mte(vcpu->kvm)) + return 0; + return REG_HIDDEN; } =20 diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 886802b8ffba..de737d5102ca 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1056,6 +1056,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190 #define KVM_CAP_SYS_HYPERV_CPUID 191 #define KVM_CAP_DIRTY_LOG_RING 192 +#define KVM_CAP_ARM_MTE 193 =20 #ifdef KVM_CAP_IRQ_ROUTING =20 --=20 2.20.1 From nobody Sun May 12 16:00:17 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=1; a=rsa-sha256; t=1610724790; cv=none; d=zohomail.com; s=zohoarc; b=HPTpQQGgnlOLIeN5dZMDliUP8PDtghmScUklDr3YWi3TCVemfslG16OFlfcZpHlK1IKDphOl0WZmqZhjJnlhGyh7KfkwXVx5+F8P/u8G1LOEi6cLVaFcQKRRQdAYzngdjDKgODh6TyWkS/diguSvSRud8ULwxiEDrQvmcE4dkNE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1610724790; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=r4mopuIYdgY29/mJOiyglh2LMl/cgdw0glhjiP5Lulk=; b=e+YbaOAT0NxeKiN0gy7a+C/+76vrao2ZYq7N0zqtRC2RFA9DS1oMtYfPmBu1Y9v/Hc3TA6KBtghLEPg5ZFB0FDfPX/x0+/ch+6UQbTNtkD2PQ7loejRwewLoUj1MABfC/8MmdjyMoyBG4Re0kXUJ4uE5hIIWQBDearGv6DklHbc= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1610724790293528.7864757769332; Fri, 15 Jan 2021 07:33:10 -0800 (PST) Received: from localhost ([::1]:53238 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0R64-0007GF-Ul for importer@patchew.org; Fri, 15 Jan 2021 10:33:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41678) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0R1f-00020D-Iy for qemu-devel@nongnu.org; Fri, 15 Jan 2021 10:28:35 -0500 Received: from foss.arm.com ([217.140.110.172]:35542) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0R1b-0008Mj-OE for qemu-devel@nongnu.org; Fri, 15 Jan 2021 10:28:35 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CC61412FC; Fri, 15 Jan 2021 07:28:29 -0800 (PST) Received: from e112269-lin.arm.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1D39C3F70D; Fri, 15 Jan 2021 07:28:26 -0800 (PST) From: Steven Price To: Catalin Marinas , Marc Zyngier , Will Deacon Subject: [RFC PATCH v7 3/3] KVM: arm64: ioctl to fetch/store tags in a guest Date: Fri, 15 Jan 2021 15:28:11 +0000 Message-Id: <20210115152811.8398-4-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210115152811.8398-1-steven.price@arm.com> References: <20210115152811.8398-1-steven.price@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=217.140.110.172; envelope-from=steven.price@arm.com; helo=foss.arm.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Peter Maydell , "Dr. David Alan Gilbert" , Andrew Jones , Haibo Xu , Suzuki K Poulose , qemu-devel@nongnu.org, Dave Martin , Juan Quintela , Richard Henderson , linux-kernel@vger.kernel.org, Steven Price , James Morse , Julien Thierry , Thomas Gleixner , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" The VMM may not wish to have it's own mapping of guest memory mapped with PROT_MTE because this causes problems if the VMM has tag checking enabled (the guest controls the tags in physical RAM and it's unlikely the tags are correct for the VMM). Instead add a new ioctl which allows the VMM to easily read/write the tags from guest memory, allowing the VMM's mapping to be non-PROT_MTE while the VMM can still read/write the tags for the purpose of migration. Signed-off-by: Steven Price --- arch/arm64/include/uapi/asm/kvm.h | 13 +++++++ arch/arm64/kvm/arm.c | 59 +++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 1 + 3 files changed, 73 insertions(+) diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/as= m/kvm.h index 24223adae150..5fc2534ac5df 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -184,6 +184,19 @@ struct kvm_vcpu_events { __u32 reserved[12]; }; =20 +struct kvm_arm_copy_mte_tags { + __u64 guest_ipa; + __u64 length; + union { + void __user *addr; + __u64 padding; + }; + __u64 flags; +}; + +#define KVM_ARM_TAGS_TO_GUEST 0 +#define KVM_ARM_TAGS_FROM_GUEST 1 + /* If you need to interpret the index values, here is the key: */ #define KVM_REG_ARM_COPROC_MASK 0x000000000FFF0000 #define KVM_REG_ARM_COPROC_SHIFT 16 diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index f4c2fd2e7c49..d6dd6b79bb77 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1303,6 +1303,55 @@ static int kvm_vm_ioctl_set_device_addr(struct kvm *= kvm, } } =20 +static int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, + struct kvm_arm_copy_mte_tags *copy_tags) +{ + gpa_t guest_ipa =3D copy_tags->guest_ipa; + size_t length =3D copy_tags->length; + void __user *tags =3D copy_tags->addr; + gpa_t gfn; + size_t pages; + bool write =3D !(copy_tags->flags & KVM_ARM_TAGS_FROM_GUEST); + + if (copy_tags->flags & ~KVM_ARM_TAGS_FROM_GUEST) + return -EINVAL; + + if (length & ~PAGE_MASK || guest_ipa & ~PAGE_MASK) + return -EINVAL; + + gfn =3D gpa_to_gfn(guest_ipa); + pages =3D length >> PAGE_SHIFT; + + while (length > 0) { + kvm_pfn_t pfn =3D gfn_to_pfn_prot(kvm, gfn, write, NULL); + void *maddr; + unsigned long num_tags =3D PAGE_SIZE / MTE_GRANULE_SIZE; + + if (is_error_noslot_pfn(pfn)) + return -ENOENT; + + maddr =3D page_address(pfn_to_page(pfn)); + + if (!write) { + num_tags =3D mte_copy_tags_to_user(tags, maddr, num_tags); + kvm_release_pfn_clean(pfn); + } else { + num_tags =3D mte_copy_tags_from_user(maddr, tags, + num_tags); + kvm_release_pfn_dirty(pfn); + } + + if (num_tags !=3D PAGE_SIZE / MTE_GRANULE_SIZE) + return -EFAULT; + + gfn++; + tags +=3D num_tags; + length -=3D PAGE_SIZE; + } + + return 0; +} + long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { @@ -1339,6 +1388,16 @@ long kvm_arch_vm_ioctl(struct file *filp, =20 return 0; } + case KVM_ARM_MTE_COPY_TAGS: { + struct kvm_arm_copy_mte_tags copy_tags; + + if (!kvm_has_mte(kvm)) + return -EINVAL; + + if (copy_from_user(©_tags, argp, sizeof(copy_tags))) + return -EFAULT; + return kvm_vm_ioctl_mte_copy_tags(kvm, ©_tags); + } default: return -EINVAL; } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index de737d5102ca..76fccb33d025 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1397,6 +1397,7 @@ struct kvm_s390_ucas_mapping { /* Available with KVM_CAP_PMU_EVENT_FILTER */ #define KVM_SET_PMU_EVENT_FILTER _IOW(KVMIO, 0xb2, struct kvm_pmu_event_= filter) #define KVM_PPC_SVM_OFF _IO(KVMIO, 0xb3) +#define KVM_ARM_MTE_COPY_TAGS _IOR(KVMIO, 0xb4, struct kvm_arm_copy_mte= _tags) =20 /* ioctl for vm fd */ #define KVM_CREATE_DEVICE _IOWR(KVMIO, 0xe0, struct kvm_create_device) --=20 2.20.1