From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A671269B0D; Tue, 2 Sep 2025 11:44:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813467; cv=none; b=enwlRRFWFFLm9Od2nMdOORWZOnR+61Pto94QiUWjn49l4dyrbpvWsksINGb/KWW70sK+lnKW6bndX1JHt48n1hnFjlKiOndo53nXJZnXyjlqqqNl8psQ2xA1aaM0+17ilX2O55bYMic+hV/FZduVWgs2Rp9c4NsVMmeL/pWYt38= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813467; c=relaxed/simple; bh=G6Ht9INfK364jYp0ofZOIq49bsCw7bcFjAmT7TRHKx8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=lch0X9E9GKPRPyGfLHE0REQ9gglU2tvHPBohSgEJF+UzQVWdesokWxAkj3E87c1VmT0NyFjOknZtV63rj6ZRON2Cy4x4oyW9ikNLm+y3n25LX9kl/ezP3CEFo4Eakw2iX3qm4x4zjSg3YMM/MTv7Z/ozYmQBaEeCK/JEgE6SO7U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SJIhnvVY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SJIhnvVY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8A08C4CEF8; Tue, 2 Sep 2025 11:44:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813465; bh=G6Ht9INfK364jYp0ofZOIq49bsCw7bcFjAmT7TRHKx8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SJIhnvVYQuFR9TyxTzl3Kl/dkzJQBx0GRXQOcgm3qAQHYpngqcPgEfq3e2fUJJSm3 9+YTHYRmssQPG6azFD5RIRiYQp1nEEx1OnR3iPIflpAdQsTgvPWP1VXqA9QylndJBN qkBofMd3J79V6SMXhwA+XfbQzNLtv20skjt7ab+4gEB09M2gBZbqEyaepbW3zmhztS HZAYmn9NbGk4iBWmrcZN/IAM6XVE0jNZP+ACrXee1aep7KsapDcYGrzMmad8PFwdx+ Ys7hXRDF7AcTQCIrO9LROqUI0IBzRCUzgeWOn0a+XtZAFBPx/57if5ic27b4ds/tTN 3gyJ0U2QXf8ig== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:04 +0100 Subject: [PATCH v8 01/29] arm64/sysreg: Update SMIDR_EL1 to DDI0601 2025-06 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-1-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=933; i=broonie@kernel.org; h=from:subject:message-id; bh=G6Ht9INfK364jYp0ofZOIq49bsCw7bcFjAmT7TRHKx8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBotth6k+2hoSImM/h32YbvKVktd3/3yir2UgAmJ scXtjTK5UaJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYegAKCRAk1otyXVSH 0NxdB/9u21D4uvIMz1tC7G+VW0wrH0PjUX2Cu34VnLsxk0WqrsW4URtOXQmUKwJvW2ah0BGmP6X J5d6odTp5J/yYibDWw7QpAq4zYu54l4fmE8RfTE3Og+W2/fTIdPCTW0uaBYw9WP1oUqfbjdkZCb 4RUkXRjFP0eb/D8djZouXRY9tFhvXu4pJdiDMdkYQF6io2pEHKE7Ts1pDJnLAmlusopxQxWcQjX bEjuhtL+n6AytMok8B6FHRX6YarG3rPC4kUK5qpKn63P5+oaIE/WpAShqzioqYMHZq/XF4iBjSZ zxoK7e/peNR8mJJHA1qKO55MoHjS085cXTHp+HlzR4uo+G8W X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Update the definiton of SMIDR_EL1 in the sysreg definition to reflect the information in DD0601 2025-06. This includes somewhat more generic ways of describing the sharing of SMCUs, more information on supported priorities and provides additional resolution for describing affinity groups. Signed-off-by: Mark Brown --- arch/arm64/tools/sysreg | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg index 696ab1f32a67..9a1adb234ff0 100644 --- a/arch/arm64/tools/sysreg +++ b/arch/arm64/tools/sysreg @@ -3659,11 +3659,15 @@ Field 3:0 BS EndSysreg =20 Sysreg SMIDR_EL1 3 1 0 0 6 -Res0 63:32 +Res0 63:60 +Field 59:56 NSMC +Field 55:52 HIP +Field 51:32 AFFINITY2 Field 31:24 IMPLEMENTER Field 23:16 REVISION Field 15 SMPS -Res0 14:12 +Field 14:13 SH +Res0 12 Field 11:0 AFFINITY EndSysreg =20 --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 046332FABED; Tue, 2 Sep 2025 11:44:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813470; cv=none; b=oDhTjzEjoP//J04XxTmMS/fjDD16SQeJVlRjHeStekzlRfxEwySLIUG+11kMdU0q9xv1fUQvLghncYdF4iFG+Bl05CvudhMxTrL/XTb6DDxMFw1Oggz+Q+Mnw/a+cb5310XVTPPD+lbO2x89cCY36uvcUm1fR2N+Rfw6+bWr38s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813470; c=relaxed/simple; bh=84pzZs2B5XlM7NI3iiTFhdL+enlb8CJyDRVL7RoB/Dw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YV9x//RMsIv2yTRDmg/YxUaB+w53p8MOXJsT+f7Hf7VWXdjjz3AcBg/0Rhw21w4qHDc/uwvjdiKVcaRljO88ehidI/kaw+v1b/XqmrhZFyaLbwdQ4x8vNNHwOJYDiC29Ku4PXnfkxmqQc5dA8+hLANFrhE5QF9yHDgo1H4rz4wY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SCjY40uk; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SCjY40uk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 18D51C4CEF5; Tue, 2 Sep 2025 11:44:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813469; bh=84pzZs2B5XlM7NI3iiTFhdL+enlb8CJyDRVL7RoB/Dw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SCjY40ukWpJUT8EcC5agRxRrlPfCAe5S/rRV7pH6/kuzga2TvDjE8wsflJIyuOod3 FHNZy+ed4ylbZ9y9/bx7HrUhXcX71ZFprdyMv8aMU5d9enrTkeirJKUKW6SdqbEU+N YYcUbMKLu2mnbk95k7kw55UUDRKHUHwkyK1mLHy2rBSdnhBAxsjD4r4EgV50ARMy9f EMfiJ6l2TVAHAfxIcYErqRIVyzrJTSwG7O7YBwFpZEtY2iuUhNCXlAspRi7Icr3j9X CRZlM6lpJgb3j6GBvFy24NqMaFrlPNooWsZhrGFUd3DZ2nTJn7QUsvrZ/hoWTXu71a 7VUukpow8oXrQ== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:05 +0100 Subject: [PATCH v8 02/29] arm64/fpsimd: Update FA64 and ZT0 enables when loading SME state Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-2-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=5757; i=broonie@kernel.org; h=from:subject:message-id; bh=84pzZs2B5XlM7NI3iiTFhdL+enlb8CJyDRVL7RoB/Dw=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBotth7qkrxDzdPqe9mpmGIgLKBMlbhGEYFvQrPw hWjiFc0jTKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYewAKCRAk1otyXVSH 0KQbB/99fq/7tUclHQv8VSWp4/W9ljkvkEbdQOQ9k5J7ULxJJ89NOu3XO+9HP0RxKgNj/NdhDBJ QtUtaUqIKUcdUFv9p5DAspL4tIScta+oRJ2b3vmalp8p9dfhvZAraoC3EmBqCaiQE9Inf1MO0FF xVnmZDVXnXEMHJjnB/0lxFAVGvR3u24D/mg16qw1nxg9BQ6LbQIMWL8yoJ46ZDW7L2kdUR5/0/L 7s7ssyDf3C0g65ZnCX6pd6X/YWIBYm3HyPBZf4QY2lltaFZxYR6hFUtQcpOkggfhDMlt8GPanB9 0cQ3jcckI+r3MIQxSigCcFbKRqypwkvlMtGcFwYA2owbIdfs X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently we enable EL0 and EL1 access to FA64 and ZT0 at boot and leave them enabled throughout the runtime of the system. When we add KVM support we will need to make this configuration dynamic, these features may be disabled for some KVM guests. Since the host kernel saves the floating point state for non-protected guests and we wish to avoid KVM having to reload the floating point state needlessly on guest reentry let's move the configuration of these enables to the floating point state reload. We provide a helper which does the configuration as part of a read/modify/write operation along with the configuration of the task VL, then update the floating point state load and SME access trap to use it. We also remove the setting of the enable bits from the CPU feature identification and resume paths. There will be a small overhead from setting the enables one at a time but this should be negligable in the context of the state load or access trap. In order to avoid compiler warnings due to unused variables in !CONFIG_ARM64_SME cases we avoid storing the vector length in temporary variables. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 14 ++++++++++++ arch/arm64/kernel/cpufeature.c | 2 -- arch/arm64/kernel/fpsimd.c | 47 +++++++++++--------------------------= ---- 3 files changed, 26 insertions(+), 37 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsim= d.h index b8cf0ea43cc0..b4359f942621 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -428,6 +428,18 @@ static inline size_t sme_state_size(struct task_struct= const *task) return __sme_state_size(task_get_sme_vl(task)); } =20 +#define sme_cond_update_smcr(vl, fa64, zt0, reg) \ + do { \ + u64 __old =3D read_sysreg_s((reg)); \ + u64 __new =3D vl; \ + if (fa64) \ + __new |=3D SMCR_ELx_FA64; \ + if (zt0) \ + __new |=3D SMCR_ELx_EZT0; \ + if (__old !=3D __new) \ + write_sysreg_s(__new, (reg)); \ + } while (0) + #else =20 static inline void sme_user_disable(void) { BUILD_BUG(); } @@ -456,6 +468,8 @@ static inline size_t sme_state_size(struct task_struct = const *task) return 0; } =20 +#define sme_cond_update_smcr(val, fa64, zt0, reg) do { } while (0) + #endif /* ! CONFIG_ARM64_SME */ =20 /* For use by EFI runtime services calls only */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 9ad065f15f1d..85d091d3b279 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2961,7 +2961,6 @@ static const struct arm64_cpu_capabilities arm64_feat= ures[] =3D { .type =3D ARM64_CPUCAP_SYSTEM_FEATURE, .capability =3D ARM64_SME_FA64, .matches =3D has_cpuid_feature, - .cpu_enable =3D cpu_enable_fa64, ARM64_CPUID_FIELDS(ID_AA64SMFR0_EL1, FA64, IMP) }, { @@ -2969,7 +2968,6 @@ static const struct arm64_cpu_capabilities arm64_feat= ures[] =3D { .type =3D ARM64_CPUCAP_SYSTEM_FEATURE, .capability =3D ARM64_SME2, .matches =3D has_cpuid_feature, - .cpu_enable =3D cpu_enable_sme2, ARM64_CPUID_FIELDS(ID_AA64PFR1_EL1, SME, SME2) }, #endif /* CONFIG_ARM64_SME */ diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index c37f02d7194e..653c0dec6b18 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -392,11 +392,15 @@ static void task_fpsimd_load(void) =20 /* Restore SME, override SVE register configuration if needed */ if (system_supports_sme()) { - unsigned long sme_vl =3D task_get_sme_vl(current); - - /* Ensure VL is set up for restoring data */ + /* + * Ensure VL is set up for restoring data. KVM might + * disable subfeatures so we reset them each time. + */ if (test_thread_flag(TIF_SME)) - sme_set_vq(sve_vq_from_vl(sme_vl) - 1); + sme_cond_update_smcr(sve_vq_from_vl(task_get_sme_vl(current)) - 1, + system_supports_fa64(), + system_supports_sme2(), + SYS_SMCR_EL1); =20 write_sysreg_s(current->thread.svcr, SYS_SVCR); =20 @@ -1237,26 +1241,6 @@ void cpu_enable_sme(const struct arm64_cpu_capabilit= ies *__always_unused p) isb(); } =20 -void cpu_enable_sme2(const struct arm64_cpu_capabilities *__always_unused = p) -{ - /* This must be enabled after SME */ - BUILD_BUG_ON(ARM64_SME2 <=3D ARM64_SME); - - /* Allow use of ZT0 */ - write_sysreg_s(read_sysreg_s(SYS_SMCR_EL1) | SMCR_ELx_EZT0_MASK, - SYS_SMCR_EL1); -} - -void cpu_enable_fa64(const struct arm64_cpu_capabilities *__always_unused = p) -{ - /* This must be enabled after SME */ - BUILD_BUG_ON(ARM64_SME_FA64 <=3D ARM64_SME); - - /* Allow use of FA64 */ - write_sysreg_s(read_sysreg_s(SYS_SMCR_EL1) | SMCR_ELx_FA64_MASK, - SYS_SMCR_EL1); -} - void __init sme_setup(void) { struct vl_info *info =3D &vl_info[ARM64_VEC_SME]; @@ -1300,17 +1284,9 @@ void __init sme_setup(void) =20 void sme_suspend_exit(void) { - u64 smcr =3D 0; - if (!system_supports_sme()) return; =20 - if (system_supports_fa64()) - smcr |=3D SMCR_ELx_FA64; - if (system_supports_sme2()) - smcr |=3D SMCR_ELx_EZT0; - - write_sysreg_s(smcr, SYS_SMCR_EL1); write_sysreg_s(0, SYS_SMPRI_EL1); } =20 @@ -1425,9 +1401,10 @@ void do_sme_acc(unsigned long esr, struct pt_regs *r= egs) WARN_ON(1); =20 if (!test_thread_flag(TIF_FOREIGN_FPSTATE)) { - unsigned long vq_minus_one =3D - sve_vq_from_vl(task_get_sme_vl(current)) - 1; - sme_set_vq(vq_minus_one); + sme_cond_update_smcr(sve_vq_from_vl(task_get_sme_vl(current)) - 1, + system_supports_fa64(), + system_supports_sme2(), + SYS_SMCR_EL1); =20 fpsimd_bind_task_to_cpu(); } else { --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D80A2F8BD0; Tue, 2 Sep 2025 11:44:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813474; cv=none; b=VpPe1Q5fPy/x3NALmkXls1dkM8Av8Y692wWRSJh0HnCrOeDyS6A8ZYVUCI0URVsHeqF4OzoRJxSn2JMy0/HZJgCP1p/dglnq3tmOL/UzYuuS7A1pF553oRKfnLm8Ac4+JsJt8p8E4B0/r2zHlKUu8sZCEhtoGsmzZcPYnIj4ZPo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813474; c=relaxed/simple; bh=tDa3Sk8XxTm+Aaxc1UJdjATH+x+7vPOsHdbcIxBkYo8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bioh8ORLOYszcvJ1MxaUSe/ptoVzZTv6mmdzqxIGhXumf5na0O/19AM7EvfmW0BiQuy/EsbFq/CW+XBOjE6hvmFFAdBAj2erCPPjFjM60TfKlP/Fx2eyKfAFA0S22dBUvr+rmbvxSJwGJgqhicU2cbrXZTrWqiHCTWTdHZEyPoQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QumXOoYl; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QumXOoYl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 49F2CC4CEF5; Tue, 2 Sep 2025 11:44:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813474; bh=tDa3Sk8XxTm+Aaxc1UJdjATH+x+7vPOsHdbcIxBkYo8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QumXOoYloKKun156twQI0T4S7LRNQb8YF7ctqGvDc2K9ZgHag2BOSdTV70x7CpALy 4yfSZ6kClvP7QP7yZbotUPIsIRkzGG6oRE4Req3fh0+a0BohPY82+eUZ8l0PVHQwNv pSCWSBz2CoA41fZ3ItB2F0BYlhKj75llk3GGkNYxedQaZbSnxHM+dF/ijfc7rtK/CM ZkohTueUr7XR68QNS1L/s8NIlTZxY69appZ7HqRjVxsS5UvsENub+JXRpkUotBpzpu MtJGfhNekpEAmoHpKVHj2x5IyDGjL5lky+IMZ1vIyqr+1E3l8io35Pw+3fGOn0Eid0 fBrZSAUCXipHw== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:06 +0100 Subject: [PATCH v8 03/29] arm64/fpsimd: Decide to save ZT0 and streaming mode FFR at bind time Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-3-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=3253; i=broonie@kernel.org; h=from:subject:message-id; bh=tDa3Sk8XxTm+Aaxc1UJdjATH+x+7vPOsHdbcIxBkYo8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBotth8ditRMtDuLvggdD2r+WCiNbXhCyjDKZFV5 ekI3TZJ3JmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYfAAKCRAk1otyXVSH 0CJgB/93wLP5ln56mIw0GDElH3woq2lftowDHQ2xYMVsC8HN3kKR+9SSIzk+AUg2So9qg4PtzTX bn3CGbEI2ifWDrvqgBk4wohx+K0L/yh+qwt4TTgwN+4LWZrBfL4dKziYpvQVfA8cAlaxyUgfEzs 8kkt3VNFPVvm2QkENSBnAFVsTHgYtA3w/Gu6hqsbibPC6qK5ylcbUbGibdLpGObZp5xjT3f1efq Lh0KAM148dOO94Qr3hA8LxVXyOkJIFxSIvoqXw82cwwoBwnZ5D21Nerk4RHAforvAMMiU5Sk07K 95jNzlwsfaPOLGe5SwVHxMfUJAdRYcgGbwRHXUoYf6uWwJub X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Some parts of the SME state are optional, enabled by additional features on top of the base FEAT_SME and controlled with enable bits in SMCR_ELx. We unconditionally enable these for the host but for KVM we will allow the feature set exposed to guests to be restricted by the VMM. These are the FFR register (FEAT_SME_FA64) and ZT0 (FEAT_SME2). We defer saving of guest floating point state for non-protected guests to the host kernel. We also want to avoid having to reconfigure the guest floating point state if nothing used the floating point state while running the host. If the guest was running with the optional features disabled then traps will be enabled for them so the host kernel will need to skip accessing that state when saving state for the guest. Support this by moving the decision about saving this state to the point where we bind floating point state to the CPU, adding a new variable to the cpu_fp_state which uses the enable bits in SMCR_ELx to flag which features are enabled. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 1 + arch/arm64/kernel/fpsimd.c | 10 ++++++++-- arch/arm64/kvm/fpsimd.c | 1 + 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsim= d.h index b4359f942621..0ecdd7dcf623 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -87,6 +87,7 @@ struct cpu_fp_state { void *sme_state; u64 *svcr; u64 *fpmr; + u64 sme_features; unsigned int sve_vl; unsigned int sme_vl; enum fp_type *fp_type; diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 653c0dec6b18..77f9dfaffe8b 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -477,12 +477,12 @@ static void fpsimd_save_user_state(void) =20 if (*svcr & SVCR_ZA_MASK) sme_save_state(last->sme_state, - system_supports_sme2()); + last->sme_features & SMCR_ELx_EZT0); =20 /* If we are in streaming mode override regular SVE. */ if (*svcr & SVCR_SM_MASK) { save_sve_regs =3D true; - save_ffr =3D system_supports_fa64(); + save_ffr =3D last->sme_features & SMCR_ELx_FA64; vl =3D last->sme_vl; } } @@ -1655,6 +1655,12 @@ static void fpsimd_bind_task_to_cpu(void) last->to_save =3D FP_STATE_CURRENT; current->thread.fpsimd_cpu =3D smp_processor_id(); =20 + last->sme_features =3D 0; + if (system_supports_fa64()) + last->sme_features |=3D SMCR_ELx_FA64; + if (system_supports_sme2()) + last->sme_features |=3D SMCR_ELx_EZT0; + /* * Toggle SVE and SME trapping for userspace if needed, these * are serialsied by ret_to_user(). diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 15e17aca1dec..9158353d8be3 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -80,6 +80,7 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) fp_state.svcr =3D __ctxt_sys_reg(&vcpu->arch.ctxt, SVCR); fp_state.fpmr =3D __ctxt_sys_reg(&vcpu->arch.ctxt, FPMR); fp_state.fp_type =3D &vcpu->arch.fp_type; + fp_state.sme_features =3D 0; =20 if (vcpu_has_sve(vcpu)) fp_state.to_save =3D FP_STATE_SVE; --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEC1E2F8BF1; Tue, 2 Sep 2025 11:44:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813478; cv=none; b=t9ryoSyGLdwhQqUxiPEmPcTmIMLuAyVp+OUH8cMvsLavRIrrTTAjJ8wJx2AL+3gfQ9Hyfq3ReKc9vXjZDYyHsCX1q12IjXkSR61M1Ux8krxIfIU4FTYYxNbHBmQLlBvIXaibz5jYGiVO447KjbdSIEbwN9skfIupOxvp5G6wZKs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813478; c=relaxed/simple; bh=Rg1wsJ3gpmGTtJKL2YMiJAj+B1RPC9bu8N2e6PnFCu0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=q+JXp97fhFw2wY0u281dmihpvSP2TBbQ0ofa0NlkY2ObOHDv8NwFdxqYXspcT6Gp2ZXdvKSXxv8QKN04yCpNzBeyj2QnbbIelW3hyCS5loNuyPFTdrDl/ctY6ke6hmuUm1t/1BDaWxJIQ46O6zcht6Nj6fOaaIRWOP4k9rOW8Fo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=r1ovcLNf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="r1ovcLNf" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6F1E8C4CEF7; Tue, 2 Sep 2025 11:44:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813478; bh=Rg1wsJ3gpmGTtJKL2YMiJAj+B1RPC9bu8N2e6PnFCu0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=r1ovcLNfwVDXL+bXxzf5PSa1rUsPdW+5HC0nyoDrUBhf8Q7fZkimaoas8YJ0nLoEF msIIKeGVrnqVEEx4qvi9IaN5J3huOh9ZMNGzM2EkSGmSEIUa0+aKXesdbNnfhZHkfY 5fs723DlvsUqcy4gYffMWWSegvDrEgFpe3XuzbHkCmki4tn2HvOEWBvEiwCk0GEQKz xLv/FCTxT9yI4FEXwaGwF6a27wPBvyx3a6pIWPqZUay5aAPsuehyWZgdqlyoZp9uHz lPkUAQpntbVdlaPWRh1GF0yxr4ywWpbccDEOrm4lpbxExtJARuwicq2tOFrEyYnwfr kyV2hO8I4mUcg== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:07 +0100 Subject: [PATCH v8 04/29] arm64/fpsimd: Check enable bit for FA64 when saving EFI state Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-4-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=1451; i=broonie@kernel.org; h=from:subject:message-id; bh=Rg1wsJ3gpmGTtJKL2YMiJAj+B1RPC9bu8N2e6PnFCu0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBotth9vlQp3HEcNB3+1aXZEyLLW8Xg5amiroyT3 Uc8RlPjvACJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYfQAKCRAk1otyXVSH 0NlcB/9vVs/g0odZcyA1QsPFpOgkZ8RK1EFUEQuzEG+2XZdcsZ+x/l64QDrBpjtGxfUI9iRdMsf 6kV6wWYZ8CbZ5en6OkpoKOVbaS0uVoveAvevSmis/ZH2z+63X1rj49SfTIhaXRD/MY/I80AM/y1 Rx7ZxRWKFZ0cbZyVwLsRjzEq6F3/UzcJkRxQJ0Cpk+QdOKs/JbkcnFN7n2VQXpPgi5p1IQGJwCq T8sUEs+3f08NUp94pEr/szprnByKwuh/R7/sXJKl1o4lXX3bD16XA7MYnUWo0vpVD0MYMqyA34T LF0fDuJvEgDEsGIhZ1N1z1ttj73BN/Jhf4LnSW5Z1/Wo6x3k X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently when deciding if we need to save FFR when in streaming mode prior to EFI calls we check if FA64 is supported by the system. Since KVM guest support will mean that FA64 might be enabled and disabled at runtime switch to checking if traps for FA64 are enabled in SMCR_EL1 instead. Signed-off-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 77f9dfaffe8b..de2897d6208c 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1908,6 +1908,11 @@ static bool efi_sm_state; * either doing something wrong or you need to propose some refactoring. */ =20 +static bool fa64_enabled(void) +{ + return read_sysreg_s(SYS_SMCR_EL1) & SMCR_ELx_FA64; +} + /* * __efi_fpsimd_begin(): prepare FPSIMD for making an EFI runtime services= call */ @@ -1940,7 +1945,7 @@ void __efi_fpsimd_begin(void) * Unless we have FA64 FFR does not * exist in streaming mode. */ - if (!system_supports_fa64()) + if (!fa64_enabled()) ffr =3D !(svcr & SVCR_SM_MASK); } =20 @@ -1988,7 +1993,7 @@ void __efi_fpsimd_end(void) * Unless we have FA64 FFR does not * exist in streaming mode. */ - if (!system_supports_fa64()) + if (!fa64_enabled()) ffr =3D false; } } --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 754892F8BF1; Tue, 2 Sep 2025 11:44:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813482; cv=none; b=LmqxFEx4PPx/3txv6KA/gfq/ZriQT6PjV0BIWO7jmDT4zKtXnqCb6IJWGgkqsA2o2SgnNXIP4LTqqZaW912z/lAOHkrInlorCjhyzsqFitWapNNkdkLOJqwj7jE5Cu7czgb+NfNrJIrHQmTh2qn5NtbiNcAsZBLQB0FfqlHxgw8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813482; c=relaxed/simple; bh=d5H7XLcR4Rtj+Rj6OZQAl1e1bSehEWOuVd1drKJeL5c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Y5F8AZxIkYjFBeh6g+bl6Z5D0UPFmFdFVEyLI5MRh0B8DDuXITaCeo9dNrlgQ6sAmX8JKF3KA+dH7qozHLQZBHZdUThtXIl4SoAIX9HjFIVAwUB6FYHPCbe8mE1vxJ+uDnIcBbAdkIQvtRitRHq8U91TEppur+ZKJF1qUoXSZIY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qh6xYzRT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qh6xYzRT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 942F3C4CEF5; Tue, 2 Sep 2025 11:44:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813482; bh=d5H7XLcR4Rtj+Rj6OZQAl1e1bSehEWOuVd1drKJeL5c=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=qh6xYzRTWnyrgR8C6AJNbxZKm0nchFbf+33TeM9zOXW4aS1eWLkDwDmHGqiniFSOx jhIXOeyHaNm47WlrlDM543R6MO3rdI43xAk9F8pHLi+K0d72HxjEGhzWcx3Oxq9UFd 8jWT3r+2el6uhBkbK6ZoOVJ/PiCVWDFfBI++KivbSTeHig9gl+lTwQ+vzR7+e/Zh5D 6n9UOvAv1hoxCbP+tAMG269yy1qMGRLTzN5voxlC3f56bVfPIiN8zLwD4Qk9Ldm00o nx8lrYGR9xIWYUxwQt8mb26c8sfUPf3tdkqdU6m+nZpw/uJk9DcT0K9aR6BRDKoQMq 9QCRr3gUd1mvw== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:08 +0100 Subject: [PATCH v8 05/29] arm64/fpsimd: Determine maximum virtualisable SME vector length Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-5-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=2177; i=broonie@kernel.org; h=from:subject:message-id; bh=d5H7XLcR4Rtj+Rj6OZQAl1e1bSehEWOuVd1drKJeL5c=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBotth9g2ETHDRccMXUN7ZClfqhFnpIGrt3f0eib Yyte/XvC+SJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYfQAKCRAk1otyXVSH 0P5bB/94F+nLQ4trESn0bLg/Okw1dl+INFEsHerSNtcHuaiKUM+hz719h/I6wq3epOs+YJMR+Vv PwGUERXckqH+BNJ6gEYRyc9/449vijyzsnuBly/10GbNh6vVcBE6JJHX+IJ/OlgGRxLRYGQ4wg5 S8XsirfTjhQ+q4o5UMUjrqgvjOHNXqvBIVmdRE9g6hNzgqg8E+xhMnfTVCmIlY+eIy/IcmnYaZI h16N8bNAUMdSEGhjFMIyDyAc59cx2Z1kCMylcQtbOo2yczHXhxM6KNzS4YLikss5iVKP7MalFcI Z+OnpXnpsLKqXQaaNuaOC7C3ap7aWYITRpVEjX3pk+L9V9N0 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB As with SVE we can only virtualise SME vector lengths that are supported by all CPUs in the system, implement similar checks to those for SVE. Since unlike SVE there are no specific vector lengths that are architecturally required the handling is subtly different, we report a system where this happens with a maximum vector length of -1. Signed-off-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index de2897d6208c..fbc586813f6a 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1244,7 +1244,8 @@ void cpu_enable_sme(const struct arm64_cpu_capabiliti= es *__always_unused p) void __init sme_setup(void) { struct vl_info *info =3D &vl_info[ARM64_VEC_SME]; - int min_bit, max_bit; + DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); + int min_bit, max_bit, b; =20 if (!system_supports_sme()) return; @@ -1274,12 +1275,32 @@ void __init sme_setup(void) */ set_sme_default_vl(find_supported_vector_length(ARM64_VEC_SME, 32)); =20 + bitmap_andnot(tmp_map, info->vq_partial_map, info->vq_map, + SVE_VQ_MAX); + + b =3D find_last_bit(tmp_map, SVE_VQ_MAX); + if (b >=3D SVE_VQ_MAX) + /* All VLs virtualisable */ + info->max_virtualisable_vl =3D SVE_VQ_MAX; + else if (b =3D=3D SVE_VQ_MAX - 1) + /* No virtualisable VLs */ + info->max_virtualisable_vl =3D -1; + else + info->max_virtualisable_vl =3D sve_vl_from_vq(__bit_to_vq(b + 1)); + + if (info->max_virtualisable_vl > info->max_vl) + info->max_virtualisable_vl =3D info->max_vl; + pr_info("SME: minimum available vector length %u bytes per vector\n", info->min_vl); pr_info("SME: maximum available vector length %u bytes per vector\n", info->max_vl); pr_info("SME: default vector length %u bytes per vector\n", get_sme_default_vl()); + + /* KVM decides whether to support mismatched systems. Just warn here: */ + if (info->max_virtualisable_vl < info->max_vl) + pr_warn("SME: unvirtualisable vector lengths present\n"); } =20 void sme_suspend_exit(void) --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9F8F2F9C3E; Tue, 2 Sep 2025 11:44:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813487; cv=none; b=X0fG2u1ixaZUt0GWiehjP6stPGySX8h7DD07KdTdmWD59sKq0k+CVRCtSjhqmT52a3qH6VwMJoWfFOtvtHUxj1rfLc/alwJsjvXaiMkzrrKgtegu34YPPKUdb1DQxPq9mpSbRmH3StUavAPE/SOPIIeJ/178C+CchGi3dzh5P0A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813487; c=relaxed/simple; bh=dbI2jtQQauIA3bdjDRznGzLHLedqf9IfSzISWZL55Y8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=l1xjDCuDw65zkXtR0ubhg4LKjye2OiO8bhBo1IYlhoVlAB3GnfstzPynHj6CGHXZ7jxiY0u1OLVea1Y0tWp3AEaK6pStqB3aECsb4MYhMg4cQys526PURxK+l+D0BECx38dU7iWAtW54pNIUqMwJdhm0XU+NbkeHryW9n9iSA0U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CEPxVLc+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CEPxVLc+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE559C4CEF5; Tue, 2 Sep 2025 11:44:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813486; bh=dbI2jtQQauIA3bdjDRznGzLHLedqf9IfSzISWZL55Y8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=CEPxVLc+Vl9vWaY9RmfDlC16U3O2k3JgV3xBmu/NBNY4Trq8wwbWqnP6VXWoGwnq9 7XbMpWTO7gerYOzQfPP3GyWMky0jjh85jCeQaQ23/L1gsOW6QxkCmpAV4v5OT393uD VmCtB8O2XRyzj+iI+mstJsPXND+7mZJt0Om3pMrVDwmeuREY1faGgOgmDP6po7shNi xJZgRQA4bmTP1qIUswl2tfGDsxPlISjMhfqNpjY2wssPcXTZozsZVal5scArDIczMH 2CVDOIby76jlUHyDMQCT3BNjnfR6z4QE34RquVZ+jJ67rMJ6Pw0EX7mZkyzRMXrvV0 UuYBSk9z4Rf+g== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:09 +0100 Subject: [PATCH v8 06/29] KVM: arm64: Introduce non-UNDEF FGT control Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-6-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=2799; i=broonie@kernel.org; h=from:subject:message-id; bh=dbI2jtQQauIA3bdjDRznGzLHLedqf9IfSzISWZL55Y8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBotth+tJnXowwXsMWuGwEIdyHOnkrvT98GfQg5e ttpOQ1xj9mJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYfgAKCRAk1otyXVSH 0AWGB/9S5rlx1MKoPilf5QfFv4ZvmsRoqSmfY10BOm1SGXi15EKmsQ9vQCv5C1FqT5aoYR5z/Nf BwnpHqKJFi+kBYvd3SbKuTE93ebqUXn0dwWukCotNvaslzhqgyjtyFBGA/QDMGVD2GXAyotnCqX 0U94z1W4rg9I00hdDn51Gv4wE5j9N9uSvboX7riZiizIfZt3Dmb9XXBhDaR8gSfqT28+B/KMvRq YV++ye1df9d3iS6BTbwliYf0jonei8HBl/ZUgpF3a6dqozfbT9DlX0sUTTsk5MPEdIZq9kCiq4j gGIyqHim1hyI95bnxbDDKQvwQePEaJT0IEbHgZ+HCke02qsS X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB We have support for determining a set of fine grained traps to enable for the guest which is tied to the support for injecting UNDEFs for undefined features. This means that we can't use the mechanism for system registers which should be present but need emulation, such as SMPRI_EL1 which should be accessible when SME is present but if SME priority support is absent SMPRI_EL1.Priority should be RAZ. Add an additional set of fine grained traps fgt, mirroring the existing fgu array. We use the same format where we always set the bit for the trap in the array as for FGU. This makes it clear what is being explicitly managed and keeps the code consistent. We do not convert the handling of ARM_WORKAROUND_AMPERE_ACO3_CPU_38 to this mechanism since this only enables a write trap and when implementing the existing UNDEF that we would share the read and write trap enablement (this being the overwhelmingly common case). Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 6 ++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 7 ++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 2f2394cce24e..b501c2880ba2 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -302,6 +302,12 @@ struct kvm_arch { */ u64 fgu[__NR_FGT_GROUP_IDS__]; =20 + /* + * Additional FGTs to enable for the guests, eg. for emulated + * registers, + */ + u64 fgt[__NR_FGT_GROUP_IDS__]; + /* * Stage 2 paging state for VMs with nested S2 using a virtual * VMID. diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/i= nclude/hyp/switch.h index 84ec4e100fbb..d5620e02787f 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -283,9 +283,9 @@ static inline void __deactivate_cptr_traps(struct kvm_v= cpu *vcpu) id; \ }) =20 -#define compute_undef_clr_set(vcpu, kvm, reg, clr, set) \ +#define compute_trap_clr_set(vcpu, kvm, trap, reg, clr, set) \ do { \ - u64 hfg =3D kvm->arch.fgu[reg_to_fgt_group_id(reg)]; \ + u64 hfg =3D kvm->arch.trap[reg_to_fgt_group_id(reg)]; \ struct fgt_masks *m =3D reg_to_fgt_masks(reg); \ set |=3D hfg & m->mask; \ clr |=3D hfg & m->nmask; \ @@ -301,7 +301,8 @@ static inline void __deactivate_cptr_traps(struct kvm_v= cpu *vcpu) if (is_nested_ctxt(vcpu)) \ compute_clr_set(vcpu, reg, c, s); \ \ - compute_undef_clr_set(vcpu, kvm, reg, c, s); \ + compute_trap_clr_set(vcpu, kvm, fgu, reg, c, s); \ + compute_trap_clr_set(vcpu, kvm, fgt, reg, c, s); \ \ val =3D m->nmask; \ val |=3D s; \ --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5E832FC895; Tue, 2 Sep 2025 11:44:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813490; cv=none; b=FM796NkcV56P7sjiSs+49a/HCgNkmlm8JhIcRcinusvL0KEkhrhqaZH/1rABxXuFt0RNl3tVX/5xI31Qvf/cVS4jqZG/KheRiz6+5GD92sUgy3wDFXiGq5hJ4OjQTFNMMUPNQvX9/74sxd+lwm+vfh4CWgksCRaI5vBVZ4bssfA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813490; c=relaxed/simple; bh=LDt1xNEXDV4dCIkz7nCmkUUILCzxDoXGLDJ0dFTAhvY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TKpuC3XfRY9BKO2Jm4PTDWWDK2k1J3fvrDRtDBEqhIObFyH+O3jCPApq6WIMAzdfnj1rLFqeQ8tQtsBPf7SY81A11tyu0qNrkFthSQDpC9KPfXMFd3X5cabThp1w17ySB67wlh2W01rVh5KCiYEJfuE0CTqPJVluIAwdcjLC7g4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=McaPTu/3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="McaPTu/3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E2D16C4CEED; Tue, 2 Sep 2025 11:44:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813490; bh=LDt1xNEXDV4dCIkz7nCmkUUILCzxDoXGLDJ0dFTAhvY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=McaPTu/3nEsNP8++iWdFRglKCDsQdyR+NL93ZI4OrRVu2Q9xUK+u2VvR5p/3Z37f4 3vU/tgooF3fpSsDozPJhilz1IA0RdIWBePkbYvwUfd8WlWFX11pj8ZibuLecwLQJg9 8ZAVJNt+qGBDsKfQQ3RXYJ1lZeuMwrnQRDRmeopYzLPHmrr0lwlPTN6iwg+FsQ9t9g Pz1Pis4LoJXlxAVknm4hVi/zYTcw5qmEOJzBmyHWCPTeYcCThgFzTEVyYxwt6/pdCe lzsbrB5g3BGFSYDFIRSHUXO+7IojOPZ5+Mx8/AqHMCrq356OAMfMEoCzhpQc+6a0vQ mwmif7Fgre9PA== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:10 +0100 Subject: [PATCH v8 07/29] KVM: arm64: Pay attention to FFR parameter in SVE save and load Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-7-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=1028; i=broonie@kernel.org; h=from:subject:message-id; bh=LDt1xNEXDV4dCIkz7nCmkUUILCzxDoXGLDJ0dFTAhvY=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBotth/9YhjtgkxhNW/f037K41pEV8WSI0IuBOzW PYA2fg7tr2JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYfwAKCRAk1otyXVSH 0HWuB/4/sgZvXMFiO8HnjLPW/KnLnt7j3M2mbyq6B6N2WFGNoUMmKbnOpewTSJK3AebyJsxuZRz BQk0Yeskp7W3e70mCa/gkCkc2WToJKwGF+jI+XyLQyJ7AqybIfI4T8RVr+ZBUqSpiZrSJq7jagD m0Vtrne+Prso7Qr/nHlKBxL0z4G2kgleznHl2E3zJeyNTkaV4ZH8c6h34Y1zGY2tSbycgVlazU8 AOgwzTIRRn+wbLrlOigGn+24TNsfWm3e6BbiE2ql3JvCGvA2g3A33zQuikym6t7lVt/tItParw2 PB+F05oat9VglRwcHIlD582ek3of50kZnirNTxo/rpgagrUD X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The hypervisor copies of the SVE save and load functions are prototyped with third arguments specifying FFR should be accessed but the assembly functions overwrite whatever is supplied to unconditionally access FFR. Remove this and use the supplied parameter. This has no effect currently since FFR is always present for SVE but will be important for SME. Signed-off-by: Mark Brown --- arch/arm64/kvm/hyp/fpsimd.S | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index e950875e31ce..6e16cbfc5df2 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -21,13 +21,11 @@ SYM_FUNC_START(__fpsimd_restore_state) SYM_FUNC_END(__fpsimd_restore_state) =20 SYM_FUNC_START(__sve_restore_state) - mov x2, #1 sve_load 0, x1, x2, 3 ret SYM_FUNC_END(__sve_restore_state) =20 SYM_FUNC_START(__sve_save_state) - mov x2, #1 sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C74442F0664; Tue, 2 Sep 2025 11:44:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813495; cv=none; b=Aolzm/EPAxwD3VsqCq0blGUIHyEaXG+APWyX9Til7ZONVULV0Ql3OkMItaWRyPk0Rr07UcIimnkHIGGevT/hia0ilYXngWCJk9YFu1Fpno/rjwkYZfJsWnAjXKVTJUgWHJKCiV+ErQuDMaNQDsupw+THKq1nDv20w2dIi8bqm7g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813495; c=relaxed/simple; bh=Sp+TAelZmX24aLyaDocHISqg813AdGCzZMNr5P6CYI4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=nni71y4jc4MiO2oTM3mPm6MbJCe1gL+et2pcFdC39O/DpFklc3xvdGngN4jOkPmiAjCtXV1CMBOBZiV7SfoeZjF93dkcTQrqHzVxr8fCrTGOVeszIH6Bi1ANsnVjf6QUbMBGxvr7+znMNY/GO5MKxYknwLKmtQLCX4GSLkEjDsY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YHAgqOE7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YHAgqOE7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 12673C4CEED; Tue, 2 Sep 2025 11:44:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813494; bh=Sp+TAelZmX24aLyaDocHISqg813AdGCzZMNr5P6CYI4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=YHAgqOE7KQtKY7UsOu/+ZFQI7lrbO8HMaKS9ToJzVO6BjFO18dynGNlDq9+Ka1cGD iudfJAC8rK0ka3LMnYKE/gidF/QDLWZ5o5i8bCtnvPAZz0GQNzGFqbj/d5yZ1CArDD c+G/rhRJM713NtQhARKr4kVp8O3+E+lS3Fefm9S6XOxfhIiVFSH5c/zOKfLBFQFT0R 7FnmhxDipReRc4GxgjmeZVBxzLlxV3PRhBEkTVyOOKAfwpRCtXT2FaoTefr5LcfEOg 4Rb6AkO3QTlV+JLRu7TmuBHJrCbuLWf2nbXriCiIRV2AnWQOveOBAMHqJsuVAObBvy /ImwJdI9DL3Ng== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:11 +0100 Subject: [PATCH v8 08/29] KVM: arm64: Pull ctxt_has_ helpers to start of sysreg-sr.h Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-8-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=3907; i=broonie@kernel.org; h=from:subject:message-id; bh=Sp+TAelZmX24aLyaDocHISqg813AdGCzZMNr5P6CYI4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiAK2q+ieSftP3JR5svFfItgXatZ39qk/zGd PtPYIYgO0iJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYgAAKCRAk1otyXVSH 0HJhCACA30PK9at39oFg90KacChbs0sADn+mpW6Jo0390EGQbphyyC/+K1gsocs7LFdd7VJ0Jwz ZMr0qkcmFhr3w0UrhdAceYzB6GblriRRheA8G6uE5xhCspowyxweVhB85A/J8GT9vABBOqpQqfQ K9HcMizFkem7RJxW5Vuxgl9TZAwZ/cG2Sqlj4ywasUB27UyqVoqwaFvUfst4NTzJ7g4nqDGL3vr cKRgJUzFaT1vajh2Aqn8EiqvdhSJ487Cc2Cf44TzBbSu1DLeFkNglQmM4Jf4duiD53wuE0c2nrr YRNx5Ed+EgsMg1/6dgC1uV1Qz1yepptm4iJjRWkxUkU5KAMO X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Rather than add earlier prototypes of specific ctxt_has_ helpers let's just pull all their definitions to the top of sysreg-sr.h so they're all available to all the individual save/restore functions. Signed-off-by: Mark Brown --- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 84 +++++++++++++++-----------= ---- 1 file changed, 41 insertions(+), 43 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hy= p/include/hyp/sysreg-sr.h index a17cbe7582de..5624fd705ae3 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -16,8 +16,6 @@ #include #include =20 -static inline bool ctxt_has_s1poe(struct kvm_cpu_context *ctxt); - static inline struct kvm_vcpu *ctxt_to_vcpu(struct kvm_cpu_context *ctxt) { struct kvm_vcpu *vcpu =3D ctxt->__hyp_running_vcpu; @@ -28,47 +26,6 @@ static inline struct kvm_vcpu *ctxt_to_vcpu(struct kvm_c= pu_context *ctxt) return vcpu; } =20 -static inline bool ctxt_is_guest(struct kvm_cpu_context *ctxt) -{ - return host_data_ptr(host_ctxt) !=3D ctxt; -} - -static inline u64 *ctxt_mdscr_el1(struct kvm_cpu_context *ctxt) -{ - struct kvm_vcpu *vcpu =3D ctxt_to_vcpu(ctxt); - - if (ctxt_is_guest(ctxt) && kvm_host_owns_debug_regs(vcpu)) - return &vcpu->arch.external_mdscr_el1; - - return &ctxt_sys_reg(ctxt, MDSCR_EL1); -} - -static inline u64 ctxt_midr_el1(struct kvm_cpu_context *ctxt) -{ - struct kvm *kvm =3D kern_hyp_va(ctxt_to_vcpu(ctxt)->kvm); - - if (!(ctxt_is_guest(ctxt) && - test_bit(KVM_ARCH_FLAG_WRITABLE_IMP_ID_REGS, &kvm->arch.flags))) - return read_cpuid_id(); - - return kvm_read_vm_id_reg(kvm, SYS_MIDR_EL1); -} - -static inline void __sysreg_save_common_state(struct kvm_cpu_context *ctxt) -{ - *ctxt_mdscr_el1(ctxt) =3D read_sysreg(mdscr_el1); - - // POR_EL0 can affect uaccess, so must be saved/restored early. - if (ctxt_has_s1poe(ctxt)) - ctxt_sys_reg(ctxt, POR_EL0) =3D read_sysreg_s(SYS_POR_EL0); -} - -static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt) -{ - ctxt_sys_reg(ctxt, TPIDR_EL0) =3D read_sysreg(tpidr_el0); - ctxt_sys_reg(ctxt, TPIDRRO_EL0) =3D read_sysreg(tpidrro_el0); -} - static inline bool ctxt_has_mte(struct kvm_cpu_context *ctxt) { struct kvm_vcpu *vcpu =3D ctxt_to_vcpu(ctxt); @@ -131,6 +88,47 @@ static inline bool ctxt_has_sctlr2(struct kvm_cpu_conte= xt *ctxt) return kvm_has_sctlr2(kern_hyp_va(vcpu->kvm)); } =20 +static inline bool ctxt_is_guest(struct kvm_cpu_context *ctxt) +{ + return host_data_ptr(host_ctxt) !=3D ctxt; +} + +static inline u64 *ctxt_mdscr_el1(struct kvm_cpu_context *ctxt) +{ + struct kvm_vcpu *vcpu =3D ctxt_to_vcpu(ctxt); + + if (ctxt_is_guest(ctxt) && kvm_host_owns_debug_regs(vcpu)) + return &vcpu->arch.external_mdscr_el1; + + return &ctxt_sys_reg(ctxt, MDSCR_EL1); +} + +static inline u64 ctxt_midr_el1(struct kvm_cpu_context *ctxt) +{ + struct kvm *kvm =3D kern_hyp_va(ctxt_to_vcpu(ctxt)->kvm); + + if (!(ctxt_is_guest(ctxt) && + test_bit(KVM_ARCH_FLAG_WRITABLE_IMP_ID_REGS, &kvm->arch.flags))) + return read_cpuid_id(); + + return kvm_read_vm_id_reg(kvm, SYS_MIDR_EL1); +} + +static inline void __sysreg_save_common_state(struct kvm_cpu_context *ctxt) +{ + *ctxt_mdscr_el1(ctxt) =3D read_sysreg(mdscr_el1); + + // POR_EL0 can affect uaccess, so must be saved/restored early. + if (ctxt_has_s1poe(ctxt)) + ctxt_sys_reg(ctxt, POR_EL0) =3D read_sysreg_s(SYS_POR_EL0); +} + +static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt) +{ + ctxt_sys_reg(ctxt, TPIDR_EL0) =3D read_sysreg(tpidr_el0); + ctxt_sys_reg(ctxt, TPIDRRO_EL0) =3D read_sysreg(tpidrro_el0); +} + static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) { ctxt_sys_reg(ctxt, SCTLR_EL1) =3D read_sysreg_el1(SYS_SCTLR); --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6EEEA2FE56B; Tue, 2 Sep 2025 11:44:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813499; cv=none; b=VQqK6jxPju1dN6h1sWNb/SQRoBYsq7MIcq3qHad04cIbtxmgFQWOOq2jKfcwXQtqTIM8tw/MZ2GR+6AMuHikq8LFDCtBSkCzSL3iUGfeeJY8SqNZ98Z3FNMPiIeYKrZPkmLpc+ZjhPlLunBscY7wgGCGAqpiPSqIT6OxejM4J28= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813499; c=relaxed/simple; bh=ZG4fxyvl+fJjEtNXLvLt23cYfT+WmIeEoPP0DgaTqa0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JgqAwFMcMW6Fh1p/8JBnHUu747XqIE1Xjg857QqSro06wi3GVUxxgPrANbM0QqAYF5A0rfPQ4SD2Pepk8Ea2Y+5ehlQYCjv5ZyzQm92ZmIg7GzLIxVnCZRS3qRlVCMXUb2hPTYfbsRZ3ETq0u/CpIJ92zXh2QOSmPzxEL03irEY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=eia4wU0m; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="eia4wU0m" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 35B7AC4CEF7; Tue, 2 Sep 2025 11:44:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813498; bh=ZG4fxyvl+fJjEtNXLvLt23cYfT+WmIeEoPP0DgaTqa0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=eia4wU0m9xg4EvTo7aFBbY8yYebfjj5jR1h5o1hy2Jf3tRdg3i/5cnqqTe2kELgFo c40Eb2ufXtdwL3Hiw/WCvLQiwu7PX2DN6ZDN2kH+q8iCCxSXSRKDr07ktwhJUCbqUa vpmYEwQt4L16gcge0vbi4jaAlwXgDMNFAIrdr6m+tCJoXGK4TTmZ3eZXweJbI3g8Il sDciLua1yjnYcZ2ZJzVhuFaktklIgVEgYbwKC7eijDGMKyyk3BKpsCbIM7CrnhP7yo kazC8NoH53xSqsYWsW6dtzn2KfAsXPLRZTgI29kPSZwcT5Ksq6I231OCHy2Sa3y8Yy twp8dc72tO55A== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:12 +0100 Subject: [PATCH v8 09/29] KVM: arm64: Move SVE state access macros after feature test macros Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-9-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=2722; i=broonie@kernel.org; h=from:subject:message-id; bh=ZG4fxyvl+fJjEtNXLvLt23cYfT+WmIeEoPP0DgaTqa0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiAa5JvxgG0URYLkAdE3e1wR5kq07dyBcx8i ynF7f4rKKyJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYgAAKCRAk1otyXVSH 0JesB/0UtIZpn6yk/kUasJvMR/f+f+jq1DUI8iOVHrDqUNrfo8I/IQ/aZIQUOacwLyD8O0lohpg ib5n7N2Q9AGT1GJ8IxFSPrAslVi2kaYqbPQn+qiy0E14o/TvuztarwxeGc64RMV2t0a16kwQmR6 KLZUVyt70irxOHilX1lyc2doFOkpovMfvtAITwTZb/jfwgytDugfEG8Im9NFkr1fPCpcdj+GG1q YxdQzbzLtckzk57D/1FSfgQWspap4MLk/6xMNI5DFHZ2hlSt75ysY6Y6fAQYRAnoiBw+jjaUJTZ MwBsM5AjhilE+PdjnWauLn/H8qJs2ooJwKZo4wN5mvkDZAVC X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In preparation for SME support move the macros used to access SVE state after the feature test macros, we will need to test for SME subfeatures to determine the size of the SME state. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 50 +++++++++++++++++++----------------= ---- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index b501c2880ba2..8f48c2a488f7 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1045,31 +1045,6 @@ struct kvm_vcpu_arch { #define NESTED_SERROR_PENDING __vcpu_single_flag(sflags, BIT(8)) =20 =20 -/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ -#define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ - sve_ffr_offset((vcpu)->arch.sve_max_vl)) - -#define vcpu_sve_max_vq(vcpu) sve_vq_from_vl((vcpu)->arch.sve_max_vl) - -#define vcpu_sve_zcr_elx(vcpu) \ - (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) - -#define sve_state_size_from_vl(sve_max_vl) ({ \ - size_t __size_ret; \ - unsigned int __vq; \ - \ - if (WARN_ON(!sve_vl_valid(sve_max_vl))) { \ - __size_ret =3D 0; \ - } else { \ - __vq =3D sve_vq_from_vl(sve_max_vl); \ - __size_ret =3D SVE_SIG_REGS_SIZE(__vq); \ - } \ - \ - __size_ret; \ -}) - -#define vcpu_sve_state_size(vcpu) sve_state_size_from_vl((vcpu)->arch.sve_= max_vl) - #define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \ KVM_GUESTDBG_USE_SW_BP | \ KVM_GUESTDBG_USE_HW | \ @@ -1105,6 +1080,31 @@ struct kvm_vcpu_arch { =20 #define vcpu_gp_regs(v) (&(v)->arch.ctxt.regs) =20 +/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ +#define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ + sve_ffr_offset((vcpu)->arch.sve_max_vl)) + +#define vcpu_sve_max_vq(vcpu) sve_vq_from_vl((vcpu)->arch.sve_max_vl) + +#define vcpu_sve_zcr_elx(vcpu) \ + (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) + +#define sve_state_size_from_vl(sve_max_vl) ({ \ + size_t __size_ret; \ + unsigned int __vq; \ + \ + if (WARN_ON(!sve_vl_valid(sve_max_vl))) { \ + __size_ret =3D 0; \ + } else { \ + __vq =3D sve_vq_from_vl(sve_max_vl); \ + __size_ret =3D SVE_SIG_REGS_SIZE(__vq); \ + } \ + \ + __size_ret; \ +}) + +#define vcpu_sve_state_size(vcpu) sve_state_size_from_vl((vcpu)->arch.sve_= max_vl) + /* * Only use __vcpu_sys_reg/ctxt_sys_reg if you know you want the * memory backed version of a register, and not the one most recently --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7595A26F298; Tue, 2 Sep 2025 11:45:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813503; cv=none; b=FQPr6qY0QgFu1qgmFylfMh7OqELXwIR23XvB926cWE1ej9h6nNI/FttICer17tn+Js+I+VknR/YzVNIR875N4u7NF8HxDYSAXpWufnDcWiL11n66mgKN4TU/L9ddc4V3F/EHWb2m/GFK3q3X+OR7sGWdP7Ppf5PKFD0DAvkUn/I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813503; c=relaxed/simple; bh=8axVTKd1b6ikITueHwT3N1OlRgy3HWVT9HvnlpgFOig=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=l8+Lsu3LJzVOyGORRxj++9Q5DKt6SmpYWsXTMJSOR4pj0+50rCmcHepjx5n/pPMPUYaG7rBlFLDXgpTAekIx5EGO5iGRAR9RqQd0Jdntk2SJ+P46OBkah529N/F9JPRqQIUOK/CWIRqCvMc7mj/BzLCRv6v+9h/03va8B4aM1Jw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PM42uS5q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PM42uS5q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5D87CC4CEED; Tue, 2 Sep 2025 11:44:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813503; bh=8axVTKd1b6ikITueHwT3N1OlRgy3HWVT9HvnlpgFOig=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=PM42uS5qyIDudCvwoqan3/zmhZWu66pe+HV4vdHLketCqU2znvLkY62oQTRR5jX0X rNbVNjrv5mSvJWOvuBbk/IcTK7yjagmtw/hSgTTgvu1nmjr2C0yGZKMMhyiV+kvD7Z qU/ri+lFs9M4GKp56CSwCLLB3pBKc0Xm93YzsLb/qf10suV+1nYHMIJE6WgjMpvSmx fwJTc2WsVuCJ6uQc+R0jr/rJsC9KgXq8tHymKrTC3Sdk9ajefJFHtTH+dalKIcNyEt ti/8ImK3a0nGiRS8vXT/T4I6mOimM30XgnmyzUpHuzyLNSYvRoWXoPm7rt9mD0uTYz 7/QbyAUsNE+Kw== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:13 +0100 Subject: [PATCH v8 10/29] KVM: arm64: Rename SVE finalization constants to be more general Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-10-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=7220; i=broonie@kernel.org; h=from:subject:message-id; bh=8axVTKd1b6ikITueHwT3N1OlRgy3HWVT9HvnlpgFOig=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiBNpUJKUYVUW70gYO8L3dsHxzEFA8ZSklq+ 5zRBZrUJk+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYgQAKCRAk1otyXVSH 0KkpB/4rIjZdmXiI4bS8bUHzYODmGOe77oE9WWqwe9nBVBkJLh7lXq8BAyDVp+T8t7dF0Ho/I3D jgggQ72DNEJHEd/WWY/eCGAmsTSl/FMc/W72GvbnlZjXjW1Wetf1SN1McpAur6ZVs+la7TYpBPv alQk8kKCVSo2OyIOfwtWh8wIKB0+F8dqVlKN6bIjnN+s9ZvyNkB4Yg2evW2r8t7oKHZbiAehrRw xo6LoGtVIhJSiZhmeeLDM0OFimcxspgXLKnR1gNAwndb1SgErguqkzQ0jwrja0FnJwYYUtCiCPR XzovtyJKouOea0RwcY7FCjnZItLw80wE6ehkWd2lDeV1Hd3A X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Due to the overlap between SVE and SME vector length configuration created by streaming mode SVE we will finalize both at once. Rename the existing finalization to use _VEC (vector) for the naming to avoid confusion. Since this includes the userspace API we create an alias KVM_ARM_VCPU_VEC for the existing KVM_ARM_VCPU_SVE capability, existing code which does not enable SME will be unaffected and any SME only code will not need to use SVE constants. No functional change. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 8 +++++--- arch/arm64/include/uapi/asm/kvm.h | 6 ++++++ arch/arm64/kvm/guest.c | 10 +++++----- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 20 ++++++++++---------- 5 files changed, 27 insertions(+), 19 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 8f48c2a488f7..88d184aac789 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -985,8 +985,8 @@ struct kvm_vcpu_arch { =20 /* KVM_ARM_VCPU_INIT completed */ #define VCPU_INITIALIZED __vcpu_single_flag(cflags, BIT(0)) -/* SVE config completed */ -#define VCPU_SVE_FINALIZED __vcpu_single_flag(cflags, BIT(1)) +/* Vector config completed */ +#define VCPU_VEC_FINALIZED __vcpu_single_flag(cflags, BIT(1)) /* pKVM VCPU setup completed */ #define VCPU_PKVM_FINALIZED __vcpu_single_flag(cflags, BIT(2)) =20 @@ -1059,6 +1059,8 @@ struct kvm_vcpu_arch { #define vcpu_has_sve(vcpu) kvm_has_sve((vcpu)->kvm) #endif =20 +#define vcpu_has_vec(vcpu) vcpu_has_sve(vcpu) + #ifdef CONFIG_ARM64_PTR_AUTH #define vcpu_has_ptrauth(vcpu) \ ((cpus_have_final_cap(ARM64_HAS_ADDRESS_AUTH) || \ @@ -1561,7 +1563,7 @@ struct kvm *kvm_arch_alloc_vm(void); int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature); bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu); =20 -#define kvm_arm_vcpu_sve_finalized(vcpu) vcpu_get_flag(vcpu, VCPU_SVE_FINA= LIZED) +#define kvm_arm_vcpu_vec_finalized(vcpu) vcpu_get_flag(vcpu, VCPU_VEC_FINA= LIZED) =20 #define kvm_has_mte(kvm) \ (system_supports_mte() && \ diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/as= m/kvm.h index ed5f3892674c..4d789871bec1 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -107,6 +107,12 @@ struct kvm_regs { #define KVM_ARM_VCPU_HAS_EL2 7 /* Support nested virtualization */ #define KVM_ARM_VCPU_HAS_EL2_E2H0 8 /* Limit NV support to E2H RES0 */ =20 +/* + * An alias for _SVE since we finalize VL configuration for both SVE and S= ME + * simultaneously. + */ +#define KVM_ARM_VCPU_VEC KVM_ARM_VCPU_SVE + struct kvm_vcpu_init { __u32 target; __u32 features[7]; diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 16ba5e9ac86c..f38b95a73169 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -342,7 +342,7 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) if (!vcpu_has_sve(vcpu)) return -ENOENT; =20 - if (kvm_arm_vcpu_sve_finalized(vcpu)) + if (kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; /* too late! */ =20 if (WARN_ON(vcpu->arch.sve_state)) @@ -497,7 +497,7 @@ static int get_sve_reg(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) if (ret) return ret; =20 - if (!kvm_arm_vcpu_sve_finalized(vcpu)) + if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; =20 if (copy_to_user(uptr, vcpu->arch.sve_state + region.koffset, @@ -523,7 +523,7 @@ static int set_sve_reg(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) if (ret) return ret; =20 - if (!kvm_arm_vcpu_sve_finalized(vcpu)) + if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; =20 if (copy_from_user(vcpu->arch.sve_state + region.koffset, uptr, @@ -657,7 +657,7 @@ static unsigned long num_sve_regs(const struct kvm_vcpu= *vcpu) return 0; =20 /* Policed by KVM_GET_REG_LIST: */ - WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu)); + WARN_ON(!kvm_arm_vcpu_vec_finalized(vcpu)); =20 return slices * (SVE_NUM_PREGS + SVE_NUM_ZREGS + 1 /* FFR */) + 1; /* KVM_REG_ARM64_SVE_VLS */ @@ -675,7 +675,7 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *= vcpu, return 0; =20 /* Policed by KVM_GET_REG_LIST: */ - WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu)); + WARN_ON(!kvm_arm_vcpu_vec_finalized(vcpu)); =20 /* * Enumerate this first, so that userspace can save/restore in diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 338505cb0171..a461f192230a 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -420,7 +420,7 @@ static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp= _vcpu, struct kvm_vcpu *h int ret =3D 0; =20 if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) { - vcpu_clear_flag(vcpu, VCPU_SVE_FINALIZED); + vcpu_clear_flag(vcpu, VCPU_VEC_FINALIZED); return 0; } =20 diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 959532422d3a..f7c63e145d54 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -92,7 +92,7 @@ static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) * Finalize vcpu's maximum SVE vector length, allocating * vcpu->arch.sve_state as necessary. */ -static int kvm_vcpu_finalize_sve(struct kvm_vcpu *vcpu) +static int kvm_vcpu_finalize_vec(struct kvm_vcpu *vcpu) { void *buf; unsigned int vl; @@ -122,21 +122,21 @@ static int kvm_vcpu_finalize_sve(struct kvm_vcpu *vcp= u) } =09 vcpu->arch.sve_state =3D buf; - vcpu_set_flag(vcpu, VCPU_SVE_FINALIZED); + vcpu_set_flag(vcpu, VCPU_VEC_FINALIZED); return 0; } =20 int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) { switch (feature) { - case KVM_ARM_VCPU_SVE: - if (!vcpu_has_sve(vcpu)) + case KVM_ARM_VCPU_VEC: + if (!vcpu_has_vec(vcpu)) return -EINVAL; =20 - if (kvm_arm_vcpu_sve_finalized(vcpu)) + if (kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; =20 - return kvm_vcpu_finalize_sve(vcpu); + return kvm_vcpu_finalize_vec(vcpu); } =20 return -EINVAL; @@ -144,7 +144,7 @@ int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int fe= ature) =20 bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu) { - if (vcpu_has_sve(vcpu) && !kvm_arm_vcpu_sve_finalized(vcpu)) + if (vcpu_has_vec(vcpu) && !kvm_arm_vcpu_vec_finalized(vcpu)) return false; =20 return true; @@ -163,7 +163,7 @@ void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu) kfree(vcpu->arch.ccsidr); } =20 -static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu) +static void kvm_vcpu_reset_vec(struct kvm_vcpu *vcpu) { if (vcpu_has_sve(vcpu)) memset(vcpu->arch.sve_state, 0, vcpu_sve_state_size(vcpu)); @@ -203,11 +203,11 @@ void kvm_reset_vcpu(struct kvm_vcpu *vcpu) if (loaded) kvm_arch_vcpu_put(vcpu); =20 - if (!kvm_arm_vcpu_sve_finalized(vcpu)) { + if (!kvm_arm_vcpu_vec_finalized(vcpu)) { if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) kvm_vcpu_enable_sve(vcpu); } else { - kvm_vcpu_reset_sve(vcpu); + kvm_vcpu_reset_vec(vcpu); } =20 if (vcpu_el1_is_32bit(vcpu)) --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F5E13009EF; Tue, 2 Sep 2025 11:45:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813508; cv=none; b=B9M8mobgNiB4gU7ST7DgpEaAFcdQ6HKBMIslTWY0/J+TZU9KMrCLrS1qY/2fZCoGNQ3pZ5rB3Nh1gDjuoMhurNobavsBtXHyhAxaip5W/4swKee74QG/IXKcqQIGorX3DMUkcBTP1wwK784hiir+LoAdyghW0+TIuY/Tpxm6AOs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813508; c=relaxed/simple; bh=rH9XhgZQjPqeyn6F/14r2eQpDtg8OcPKafzcMuvwg9Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HHws7UD08q1XbECDeoQP/ZINcwYrja7Ah0UBAJUJ8IEO+R5lGLYbODG6jdyo1L9olTmoOxq0Dmjk1a8YA1d/r07p3K/mAvhtfwoVzWZ52hvsY30CpYJ2jV95L5jPuuWVTPpFDSPr5jDYC1dfLbmymMr0ucRy0s9ps0ENTznxf1I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Aspx+jBo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Aspx+jBo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8431BC4CEF8; Tue, 2 Sep 2025 11:45:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813508; bh=rH9XhgZQjPqeyn6F/14r2eQpDtg8OcPKafzcMuvwg9Y=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Aspx+jBogE0Yab7GP1A39kEqFlA5TIxA8bTHyZkhSYtzwNnkVPLXlzzrFQx6eDPtQ D7wQIs71HcIV31O+Ift0jWXt4i+ttZLd0DhRPLEMS8xkCbk9Gk1YCimIsyl7AjdxBc EiWdJwQ9ugQ1qC/NATXW3EjqH3hlpDw37HZlGDdd/hHQzRs7bnw0FjmNDHxNf2EtAP G3G1byPlsdurp5k5/nyp9VrHeV5LKHwrEKAYEenmBZEkkzlFRv86bEHJp1DnV2gEzI aQvsHCrHqqfIb/v70lOnqNaxhtUWpmiueDKDyE7vB6a0lg/HQK9vCpnvsFlfZYtkjj HbiVw5jBTE6Xg== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:14 +0100 Subject: [PATCH v8 11/29] KVM: arm64: Document the KVM ABI for SME Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-11-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=13253; i=broonie@kernel.org; h=from:subject:message-id; bh=rH9XhgZQjPqeyn6F/14r2eQpDtg8OcPKafzcMuvwg9Y=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiCaWSJ78p62BFgyuPzqc0yuQRvQzsxHwPYa 0qgXjhYuJ2JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYggAKCRAk1otyXVSH 0GFJB/9Pco9YWrN6dV58T0Q6JrCmysPVZAJFs9sMsCCy3wGaDztvDTiilnsEodlpBEogrBCGKnI +aF6+LimnvadNMfYUU+0hBxzOZl/lDOJg6VO0oRgqn47W5LUnAQ+KEpOeHUPU1atptr3fhjjbtW SlM/olzWwD4rU1JOj00gh8olIsRFSg1jeaLJ1O6G/G9+lspgahwd6Cy3+u8/lcZHiS8UgZXRA5g 3zwHAlQzHftvUmjafozH6mP9EdfvbBYCBel/1up4Hi9JJ+iouXLajPTRJfv9aNYgP3pGVG8tOlq OUiOp5mVoVXusWLMEPySm0fcwCdwNI3NYRP/qUBY0Cpdttc/ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME, the Scalable Matrix Extension, is an arm64 extension which adds support for matrix operations, with core concepts patterned after SVE. SVE introduced some complication in the ABI since it adds new vector floating point registers with runtime configurable size, the size being controlled by a parameter called the vector length (VL). To provide control of this to VMMs we offer two phase configuration of SVE, SVE must first be enabled for the vCPU with KVM_ARM_VCPU_INIT(KVM_ARM_VCPU_SVE), after which vector length may then be configured but the configurably sized floating point registers are inaccessible until finalized with a call to KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE) after which the configurably sized registers can be accessed. SME introduces an additional independent configurable vector length which as well as controlling the size of the new ZA register also provides an alternative view of the configurably sized SVE registers (known as streaming mode) with the guest able to switch between the two modes as it pleases. There is also a fixed sized register ZT0 introduced in SME2. As well as streaming mode the guest may enable and disable ZA and (where SME2 is available) ZT0 dynamically independently of streaming mode. These modes are controlled via the system register SVCR. We handle the configuration of the vector length for SME in a similar manner to SVE, requiring initialization and finalization of the feature with a pseudo register controlling the available SME vector lengths as for SVE. Further, if the guest has both SVE and SME then finalizing one prevents further configuration of the vector length for the other. Where both SVE and SME are configured for the guest we always present the SVE registers to userspace as having the larger of the configured maximum SVE and SME vector lengths, discarding extra data at load time and zero padding on read as required if the active vector length is lower. Note that this means that enabling or disabling streaming mode while the guest is stopped will not zero Zn or Pn as it will when the guest is running, but it does allow SVCR, Zn and Pn to be read and written in any order. Userspace access to ZA and (if configured) ZT0 is always available, they will be zeroed when the guest runs if disabled in SVCR and the value read will be zero if the guest stops with them disabled. This mirrors the behaviour of the architecture, enabling access causes ZA and ZT0 to be zeroed, while allowing access to SVCR, ZA and ZT0 to be performed in any order. If SME is enabled for a guest without SVE then the FPSIMD Vn registers must be accessed via the low 128 bits of the SVE Zn registers as is the case when SVE is enabled. This is not ideal but allows access to SVCR and the registers in any order without duplication or ambiguity about which values should take effect. This may be an issue for VMMs that are unaware of SME on systems that implement it without SVE if they let SME be enabled, the lack of access to Vn may surprise them, but it seems like an unusual implementation choice. For SME unware VMMs on systems with both SVE and SME support the SVE registers may be larger than expected, this should be less disruptive than on a system without SVE as they will simply ignore the high bits of the registers. Signed-off-by: Mark Brown --- Documentation/virt/kvm/api.rst | 115 +++++++++++++++++++++++++++++--------= ---- 1 file changed, 81 insertions(+), 34 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 6aa40ee05a4a..94a22407a1d4 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -406,7 +406,7 @@ Errors: instructions from device memory (arm64) ENOSYS data abort outside memslots with no syndrome info and KVM_CAP_ARM_NISV_TO_USER not enabled (arm64) - EPERM SVE feature set but not finalized (arm64) + EPERM SVE or SME feature set but not finalized (arm64) =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 This ioctl is used to run a guest virtual cpu. While there are no @@ -2601,11 +2601,11 @@ Specifically: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D= =3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D =20 .. [1] These encodings are not accepted for SVE-enabled vcpus. See - :ref:`KVM_ARM_VCPU_INIT`. + :ref:`KVM_ARM_VCPU_INIT`. They are also not accepted when SME is + enabled without SVE and the vcpu is in streaming mode. =20 The equivalent register content can be accessed via bits [127:0] of - the corresponding SVE Zn registers instead for vcpus that have SVE - enabled (see below). + the corresponding SVE Zn registers in these cases (see below). =20 arm64 CCSIDR registers are demultiplexed by CSSELR value:: =20 @@ -2636,24 +2636,34 @@ arm64 SVE registers have the following bit patterns= :: 0x6050 0000 0015 060 FFR bits[256*slice + 255 : 256*sli= ce] 0x6060 0000 0015 ffff KVM_REG_ARM64_SVE_VLS pseudo-regis= ter =20 -Access to register IDs where 2048 * slice >=3D 128 * max_vq will fail with -ENOENT. max_vq is the vcpu's maximum supported vector length in 128-bit -quadwords: see [2]_ below. +arm64 SME registers have the following bit patterns: + + 0x6080 0000 0017 00 ZA.H[n] bits[2048*slice + 2047 : 2= 048*slice] + 0x60XX 0000 0017 0100 ZT0 + 0x6060 0000 0017 fffe KVM_REG_ARM64_SME_VLS pseudo-regis= ter + +Access to Z, P or ZA register IDs where 2048 * slice >=3D 128 * max_vq +will fail with ENOENT. max_vq is the vcpu's maximum supported vector +length in 128-bit quadwords: see [2]_ below. + +Access to the ZA and ZT0 registers is only available if SVCR.ZA is set +to 1. =20 These registers are only accessible on vcpus for which SVE is enabled. See KVM_ARM_VCPU_INIT for details. =20 -In addition, except for KVM_REG_ARM64_SVE_VLS, these registers are not -accessible until the vcpu's SVE configuration has been finalized -using KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE). See KVM_ARM_VCPU_INIT -and KVM_ARM_VCPU_FINALIZE for more information about this procedure. +In addition, except for KVM_REG_ARM64_SVE_VLS and +KVM_REG_ARM64_SME_VLS, these registers are not accessible until the +vcpu's SVE and SME configuration has been finalized using +KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC). See KVM_ARM_VCPU_INIT and +KVM_ARM_VCPU_FINALIZE for more information about this procedure. =20 -KVM_REG_ARM64_SVE_VLS is a pseudo-register that allows the set of vector -lengths supported by the vcpu to be discovered and configured by -userspace. When transferred to or from user memory via KVM_GET_ONE_REG -or KVM_SET_ONE_REG, the value of this register is of type -__u64[KVM_ARM64_SVE_VLS_WORDS], and encodes the set of vector lengths as -follows:: +KVM_REG_ARM64_SVE_VLS and KVM_ARM64_VCPU_SME_VLS are pseudo-registers +that allows the set of vector lengths supported by the vcpu to be +discovered and configured by userspace. When transferred to or from +user memory via KVM_GET_ONE_REG or KVM_SET_ONE_REG, the value of this +register is of type __u64[KVM_ARM64_SVE_VLS_WORDS], and encodes the +set of vector lengths as follows:: =20 __u64 vector_lengths[KVM_ARM64_SVE_VLS_WORDS]; =20 @@ -2665,19 +2675,25 @@ follows:: /* Vector length vq * 16 bytes not supported */ =20 .. [2] The maximum value vq for which the above condition is true is - max_vq. This is the maximum vector length available to the guest on - this vcpu, and determines which register slices are visible through - this ioctl interface. + max_vq. This is the maximum vector length currently available to + the guest on this vcpu, and determines which register slices are + visible through this ioctl interface. + + If SME is supported then the max_vq used for the Z and P registers + while SVCR.SM is 1 this vector length will be the maximum SME + vector length available for the guest, otherwise it will be the + maximum SVE vector length available. =20 (See Documentation/arch/arm64/sve.rst for an explanation of the "vq" nomenclature.) =20 -KVM_REG_ARM64_SVE_VLS is only accessible after KVM_ARM_VCPU_INIT. -KVM_ARM_VCPU_INIT initialises it to the best set of vector lengths that -the host supports. +KVM_REG_ARM64_SVE_VLS and KVM_REG_ARM_SME_VLS are only accessible +after KVM_ARM_VCPU_INIT. KVM_ARM_VCPU_INIT initialises them to the +best set of vector lengths that the host supports. =20 -Userspace may subsequently modify it if desired until the vcpu's SVE -configuration is finalized using KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE). +Userspace may subsequently modify these registers if desired until the +vcpu's SVE and SME configuration is finalized using +KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC). =20 Apart from simply removing all vector lengths from the host set that exceed some value, support for arbitrarily chosen sets of vector lengths @@ -2685,8 +2701,8 @@ is hardware-dependent and may not be available. Atte= mpting to configure an invalid set of vector lengths via KVM_SET_ONE_REG will fail with EINVAL. =20 -After the vcpu's SVE configuration is finalized, further attempts to -write this register will fail with EPERM. +After the vcpu's SVE or SME configuration is finalized, further +attempts to write these registers will fail with EPERM. =20 arm64 bitmap feature firmware pseudo-registers have the following bit patt= ern:: =20 @@ -3469,6 +3485,7 @@ The initial values are defined as: - General Purpose registers, including PC and SP: set to 0 - FPSIMD/NEON registers: set to 0 - SVE registers: set to 0 + - SME registers: set to 0 - System registers: Reset to their architecturally defined values as for a warm reset to EL1 (resp. SVC) or EL2 (in the case of EL2 being enabled). @@ -3512,7 +3529,7 @@ Possible features: =20 - KVM_ARM_VCPU_SVE: Enables SVE for the CPU (arm64 only). Depends on KVM_CAP_ARM_SVE. - Requires KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE): + Requires KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): =20 * After KVM_ARM_VCPU_INIT: =20 @@ -3520,7 +3537,7 @@ Possible features: initial value of this pseudo-register indicates the best set of vector lengths possible for a vcpu on this host. =20 - * Before KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE): + * Before KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): =20 - KVM_RUN and KVM_GET_REG_LIST are not available; =20 @@ -3533,11 +3550,40 @@ Possible features: KVM_SET_ONE_REG, to modify the set of vector lengths available for the vcpu. =20 - * After KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE): + * After KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): =20 - the KVM_REG_ARM64_SVE_VLS pseudo-register is immutable, and can no longer be written using KVM_SET_ONE_REG. =20 + - KVM_ARM_VCPU_SME: Enables SME for the CPU (arm64 only). + Depends on KVM_CAP_ARM_SME. + Requires KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): + + * After KVM_ARM_VCPU_INIT: + + - KVM_REG_ARM64_SME_VLS may be read using KVM_GET_ONE_REG: the + initial value of this pseudo-register indicates the best set of + vector lengths possible for a vcpu on this host. + + * Before KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): + + - KVM_RUN and KVM_GET_REG_LIST are not available; + + - KVM_GET_ONE_REG and KVM_SET_ONE_REG cannot be used to access + the scalable architectural SVE registers + KVM_REG_ARM64_SVE_ZREG(), KVM_REG_ARM64_SVE_PREG() or + KVM_REG_ARM64_SVE_FFR, the matrix register + KVM_REG_ARM64_SME_ZA() or the LUT register KVM_REG_ARM64_ZT(); + + - KVM_REG_ARM64_SME_VLS may optionally be written using + KVM_SET_ONE_REG, to modify the set of vector lengths available + for the vcpu. + + * After KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): + + - the KVM_REG_ARM64_SME_VLS pseudo-register is immutable, and can + no longer be written using KVM_SET_ONE_REG. + - KVM_ARM_VCPU_HAS_EL2: Enable Nested Virtualisation support, booting the guest from EL2 instead of EL1. Depends on KVM_CAP_ARM_EL2. @@ -5120,11 +5166,12 @@ Errors: =20 Recognised values for feature: =20 - =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D - arm64 KVM_ARM_VCPU_SVE (requires KVM_CAP_ARM_SVE) - =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D + =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + arm64 KVM_ARM_VCPU_VEC (requires KVM_CAP_ARM_SVE or KVM_CAP_ARM_SME) + arm64 KVM_ARM_VCPU_SVE (alias for KVM_ARM_VCPU_VEC) + =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -Finalizes the configuration of the specified vcpu feature. +Finalizes the configuration of the specified vcpu features. =20 The vcpu must already have been initialised, enabling the affected feature= , by means of a successful :ref:`KVM_ARM_VCPU_INIT ` call wi= th the --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B36792FB625; Tue, 2 Sep 2025 11:45:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813513; cv=none; b=cg8phgSG9lrWpahplaEke55SvIVhYj77LISy5ii5Q4lI29kOfQHYr5Kd35gYoiiKx/Fwb8FRQalAei/7tpbjYCTfHq6cxLPqV2TEI4mgccePIRVKCKyteoiYaUTR1BgtygbcIAm9r98RKPi1PtCY9blB0eOfEsdIkk8I2fe9yrA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813513; c=relaxed/simple; bh=m/G5Vt7qKGaxjn4aWQkuI02V12W35pAJXakUbFGbMwk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=R3hl+mzjB/udgECDXyxNSAAFXZr+QZgTbNJmadDUK2UT5zCj2BxO/gtKScGcdwkzIYGlHNmMJxnZ1Ug2qwXEW1L0Xl5U7hzwm0tDjA6iKQ5OTIkN03UtNiopuEv6wC7AnEl1ZoURPE2Egwfxk9aBNJcILwF0CODkHidsF9zfSj0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VnMP5hy+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VnMP5hy+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 17513C4CEF5; Tue, 2 Sep 2025 11:45:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813513; bh=m/G5Vt7qKGaxjn4aWQkuI02V12W35pAJXakUbFGbMwk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=VnMP5hy+Of/j3nsHNGWOvYB50pr+13Ql5nplOhXaHoEPS45+xLU6M3ADH99JQ7rT8 AyDJQ3RzlSThcz87Fc2of048Fwi42wIMhB3HVLge+gzE8OG06TVnaUbEy/fWhDZ4B4 ELjAA4BdYe4MYei5Nx1ntubDnom4W2YXF8dVP0iBiXgq879chNCRFsnkUbOZokYrZp Sklf7ytanQyY4AOowMA6mjxg1fJMMmSVFA7Ql2aAUaIur/gn4zs23wlKMXg+heJX3a Ojj2wttnO/sE0nlUCNs6cZAylUV/pfwoOhBYHzWy3l7CCSYQTEz0QNX4F2UdYKlCzR QVzZCEcat3I3Q== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:15 +0100 Subject: [PATCH v8 12/29] KVM: arm64: Define internal features for SME Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-12-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=3292; i=broonie@kernel.org; h=from:subject:message-id; bh=m/G5Vt7qKGaxjn4aWQkuI02V12W35pAJXakUbFGbMwk=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiDI1QSeHkxBm6R7x1P8j8whI7B6mNwDowf3 A8T6xtcfNqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYgwAKCRAk1otyXVSH 0GgvB/oC4OCLayBnzN0pYrmDlfdx2pr4aooOPClEy+ilszzRnxoeLECQgVeuZJVeyzsvb1PQk2M GV3Y2k1oytxjIh9ju0zOV2aUNS7gqR675qv3xvK0o5SmKs2zxH5FAIGb6+eccFdUK09gbwT4+Ep 2mAwa81qicF+n+69QO+R3bsLTOz1UdNmzlTs12/soLJbr211dqq0xC3ygWy6c9pUYwALeBp/MUR uqtfQGjv55fYAG+YECWErPnhzacGZ0hRbZUS2UhQL3NRvaAmbhdvai3FgElpYiQ4Vu2khl95kRw MX/2C50RqpIhZJd7+/tlv3OtUnaEKGzTN1EmMJI+YD69TXh6 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In order to simplify interdependencies in the rest of the series define the feature detection for SME and it's subfeatures. Due to the need for vector length configuration we define a flag for SME like for SVE. We also have two subfeatures which add architectural state, FA64 and SME2, which are configured via the normal ID register scheme. Also provide helpers which check if the vCPU is in streaming mode or has ZA enabled. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 35 ++++++++++++++++++++++++++++++++++- arch/arm64/kvm/sys_regs.c | 2 +- 2 files changed, 35 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 88d184aac789..98761b601457 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -355,6 +355,8 @@ struct kvm_arch { #define KVM_ARCH_FLAG_GUEST_HAS_SVE 9 /* MIDR_EL1, REVIDR_EL1, and AIDR_EL1 are writable from userspace */ #define KVM_ARCH_FLAG_WRITABLE_IMP_ID_REGS 10 + /* SME exposed to guest */ +#define KVM_ARCH_FLAG_GUEST_HAS_SME 11 unsigned long flags; =20 /* VM-wide vCPU feature set */ @@ -1059,7 +1061,16 @@ struct kvm_vcpu_arch { #define vcpu_has_sve(vcpu) kvm_has_sve((vcpu)->kvm) #endif =20 -#define vcpu_has_vec(vcpu) vcpu_has_sve(vcpu) +#define kvm_has_sme(kvm) (system_supports_sme() && \ + test_bit(KVM_ARCH_FLAG_GUEST_HAS_SME, &(kvm)->arch.flags)) + +#ifdef __KVM_NVHE_HYPERVISOR__ +#define vcpu_has_sme(vcpu) kvm_has_sme(kern_hyp_va((vcpu)->kvm)) +#else +#define vcpu_has_sme(vcpu) kvm_has_sme((vcpu)->kvm) +#endif + +#define vcpu_has_vec(vcpu) (vcpu_has_sve(vcpu) || vcpu_has_sme(vcpu)) =20 #ifdef CONFIG_ARM64_PTR_AUTH #define vcpu_has_ptrauth(vcpu) \ @@ -1705,6 +1716,28 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64= val); #define kvm_has_sctlr2(k) \ (kvm_has_feat((k), ID_AA64MMFR3_EL1, SCTLRX, IMP)) =20 +#define kvm_has_fa64(k) \ + (system_supports_fa64() && \ + kvm_has_feat((k), ID_AA64SMFR0_EL1, FA64, IMP)) + +#define kvm_has_sme2(k) \ + (system_supports_sme2() && \ + kvm_has_feat((k), ID_AA64PFR1_EL1, SME, SME2)) + +#ifdef __KVM_NVHE_HYPERVISOR__ +#define vcpu_has_sme2(vcpu) kvm_has_sme2(kern_hyp_va((vcpu)->kvm)) +#define vcpu_has_fa64(vcpu) kvm_has_fa64(kern_hyp_va((vcpu)->kvm)) +#else +#define vcpu_has_sme2(vcpu) kvm_has_sme2((vcpu)->kvm) +#define vcpu_has_fa64(vcpu) kvm_has_fa64((vcpu)->kvm) +#endif + +#define vcpu_in_streaming_mode(vcpu) \ + (__vcpu_sys_reg(vcpu, SVCR) & SVCR_SM_MASK) + +#define vcpu_za_enabled(vcpu) \ + (__vcpu_sys_reg(vcpu, SVCR) & SVCR_ZA_MASK) + static inline bool kvm_arch_has_irq_bypass(void) { return true; diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 82ffb3b3b3cf..111b07a78787 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1782,7 +1782,7 @@ static unsigned int sve_visibility(const struct kvm_v= cpu *vcpu, static unsigned int sme_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd) { - if (kvm_has_feat(vcpu->kvm, ID_AA64PFR1_EL1, SME, IMP)) + if (vcpu_has_sme(vcpu)) return 0; =20 return REG_HIDDEN; --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68AAE2F618B; Tue, 2 Sep 2025 11:45:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813519; cv=none; b=oBFCtD0OYA04fK+1xsfr64wsw+6v3AtJW92y7tWpsnecWEESl2jknP43gnBA3T3JJ/D1JMGjtMINPK7ZH3zXZx+Lt2MEjMIf8pYMxtxwdTamcgwZzE4DRPi7xIatAChI1jTsP9P5+8zPTLHjX9elUF1Xy3HAKnPFotyuNHdbUd4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813519; c=relaxed/simple; bh=JcuB5ad9okGlXfOvq/M2PLI+WEiTQG4p3nyWXh3yr4c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BMufDyhfbGgJL1wmHSp0qJNTANjrdw4JDMPIIbD5XbhDcurb1A6V2Pj6B04mLYg2GW3btiQcWBPljX0+XTgO9tn1PikvG5tV+4OZ4F0XBaCx9FDYKWlS1Xa0fSX8/SupJ8TNvfUx4X1oQ5I3KEmhjL+4I7hEUvvXucayzhdNb40= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HeapaFPs; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HeapaFPs" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CC115C4CEED; Tue, 2 Sep 2025 11:45:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813519; bh=JcuB5ad9okGlXfOvq/M2PLI+WEiTQG4p3nyWXh3yr4c=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=HeapaFPsOuaJuO8zENcjOE77JPfBIwTOCMRrOra9z3WeFMngcAPJKqaFJY/e/yfYM mNt/A7C0RKNhGCBWgCtniznsK6akAv2OIxdm2b6h4KnFVklRih0/y7S6NBs0vv452k a7HG+FK3Xbf4M7DORn781fDwLR0AdilYZVtS4BfK2WqNEdTYWspQ6pFPCJ8gTaaeZp 4OWMDXRpeRgbLUvCYbFutsjDwnbgmgxfGYc9XXde3+C1UjylpQbjDj+EZJ0wW2bKS0 xGkcV7p2F4TwnCwl8yjV/MkAMnEQTpNGW0K1P2eogzF9JCtWisQ205ueDpmsyOJiP0 62RyA2f5gIxiQ== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:16 +0100 Subject: [PATCH v8 13/29] KVM: arm64: Rename sve_state_reg_region Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-13-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=2352; i=broonie@kernel.org; h=from:subject:message-id; bh=JcuB5ad9okGlXfOvq/M2PLI+WEiTQG4p3nyWXh3yr4c=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiEVhFwdz/qXdy0wHnP9UFZExCHy+Qz/8oSH DRaKuGMIy+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYhAAKCRAk1otyXVSH 0AQ1B/9U8a+0/H9lS9eFV2eD1eghMokg0qYk+XDngVDPJbeIAVtzGUxmw0PRQ3a7Fi3IUQnkEiB sHmCZGWKhUu9sYMMPQUir4KMBl0OdEHARSv5E964K8JjF1alpWxdweqJWWb8zQ23LjmfDbvbXCY UsEFyjrr8Pa4HAjIfxApuT9a9b7sdO9AWAqsDWYxmNdMF7dbcPBCDu34S2DrSLW/iJUuY7o+e3W METIAtW0nlQoSZxWPhcPaamINw0qKMm7nWlp7T12Km2L3KYPCQaKpJNUE7LIV5E2GKOrBuYaXk9 QYhxDMus1mGYBXj7BagdJR6gkrTyIlD+Ao9GwXMF+g8KXXBh X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB As for SVE we will need to pull parts of dynamically sized registers out of a block of memory for SME so we will use a similar code pattern for this. Rename the current struct sve_state_reg_region in preparation for this. No functional change. Signed-off-by: Mark Brown --- arch/arm64/kvm/guest.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index f38b95a73169..6eb273b525d5 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -404,9 +404,9 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) */ #define vcpu_sve_slices(vcpu) 1 =20 -/* Bounds of a single SVE register slice within vcpu->arch.sve_state */ -struct sve_state_reg_region { - unsigned int koffset; /* offset into sve_state in kernel memory */ +/* Bounds of a single register slice within vcpu->arch.s[mv]e_state */ +struct vec_state_reg_region { + unsigned int koffset; /* offset into s[mv]e_state in kernel memory */ unsigned int klen; /* length in kernel memory */ unsigned int upad; /* extra trailing padding in user memory */ }; @@ -415,7 +415,7 @@ struct sve_state_reg_region { * Validate SVE register ID and get sanitised bounds for user/kernel SVE * register copy */ -static int sve_reg_to_region(struct sve_state_reg_region *region, +static int sve_reg_to_region(struct vec_state_reg_region *region, struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) { @@ -485,7 +485,7 @@ static int sve_reg_to_region(struct sve_state_reg_regio= n *region, static int get_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) { int ret; - struct sve_state_reg_region region; + struct vec_state_reg_region region; char __user *uptr =3D (char __user *)reg->addr; =20 /* Handle the KVM_REG_ARM64_SVE_VLS pseudo-reg as a special case: */ @@ -511,7 +511,7 @@ static int get_sve_reg(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) static int set_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) { int ret; - struct sve_state_reg_region region; + struct vec_state_reg_region region; const char __user *uptr =3D (const char __user *)reg->addr; =20 /* Handle the KVM_REG_ARM64_SVE_VLS pseudo-reg as a special case: */ --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 686292F8BEE; Tue, 2 Sep 2025 11:45:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813525; cv=none; b=AQhhKKgrpnfd7W3F6WwpLT3Qrzv1xlYzTmvZUuupNEDyp4Z7MQXx6rN+dl+drZpbXQZemlRaYZYAXorVFVkai9MkSWT96+oMDf7jyAjLAJ0mkbMqpjArflRjEP1OQAXj6g6nBDYaZ69RTbAt0AsdYL6Lh0N/cWSSR4g/uKcpR7A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813525; c=relaxed/simple; bh=c+TpgbYUfXA/JhZ2KpmwCWf1keJZ+h/x6ZLMET666u0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iVSb3MXg1d3xvQPjz7XrzNzsKn3DFoEnB8JU4V9c2unE4932CsPTPhsmbHESQDRcMRKdFccw9V0/fCCbiR9V62m1RVYtVY8KQWT/HwcSwNic/UEibxB6KGvOkWIH5+9cn5Trg7uDKmA/nVmv8poNYCd2fACKrZ2g7zgvIjoKmbM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=U+0AuHea; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="U+0AuHea" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A20EDC4CEED; Tue, 2 Sep 2025 11:45:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813525; bh=c+TpgbYUfXA/JhZ2KpmwCWf1keJZ+h/x6ZLMET666u0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=U+0AuHea+AC+BDHKULXKdIhT/qDeUageGweMVGosgH9D+Sk2xPgxCqYHjOIOtkFFf /EWYeJBDQ7jkPCj9YD0ttr0ReudQRsVev4ypKjWFiUw4DDqxEpwqLLtRz70QoW/5GX AwdfzEl9AOLqQ4A0My+Kn0LPS9+CmRtw87uoS7r7Qds4GrdGdp5jjEagVZRiFVnrgh V6hzuRp7XIkvZmDixM+l6MtijtXsPBbsASYWbTz2pk43VMX0IE1io0MnLEW/bg7WSJ 6UhyjVE5hUh1CgFicfgMbPdplE5TTxw/OhMWh1ETndMAbnKww6fRbpuhxf23zXoKv7 HnmU97miKh48Q== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:17 +0100 Subject: [PATCH v8 14/29] KVM: arm64: Store vector lengths in an array Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-14-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=11784; i=broonie@kernel.org; h=from:subject:message-id; bh=c+TpgbYUfXA/JhZ2KpmwCWf1keJZ+h/x6ZLMET666u0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiELqduM5iwPZFWrzka7g4gqwpJkj5F6APwb HK/QLrDvlCJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYhAAKCRAk1otyXVSH 0JoGB/9c5RpcH5ij4KJewUF0qEFlmZ0Uf8MXpxkKZZQ8PpRuYxSREyGLgmQs9Ts7OpE8JXaPccS 1zlJ3s9QWhGmOi2825zqsl8UXg6DNxKtJlNh/HrrZ5Y+L9g8t4M12UTDSmtlZs5rIhFHd/n8jF8 0edFxJJ4j1gCXm1PmkfaS0Y7Y9Ap1qkPrucNoWychhQl0c2+jZyrqL1L2aOm2hzyfeQZK2wINSJ yGbhdpt4nuydbvUm1L+SZEAXePA8xu3rBRpe0tcmdL+kDo+0dwAhdWnlCobFH6Lf6oUYkHAs5NM uyO1KPigWnVpGoFEeZZ3eoc3b6C9YWb7H6Xtb7VgitKlB1O+ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME adds a second vector length configured in a very similar way to the SVE vector length, in order to facilitate future code sharing for SME refactor our storage of vector lengths to use an array like the host does. We do not yet take much advantage of this so the intermediate code is not as clean as might be. No functional change. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 17 +++++++++++------ arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/fpsimd.c | 2 +- arch/arm64/kvm/guest.c | 6 +++--- arch/arm64/kvm/hyp/include/hyp/switch.h | 6 +++--- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 +++--- arch/arm64/kvm/hyp/nvhe/pkvm.c | 7 ++++--- arch/arm64/kvm/reset.c | 22 +++++++++++----------- 9 files changed, 38 insertions(+), 32 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 98761b601457..59b88206d14c 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -76,8 +76,10 @@ enum kvm_mode kvm_get_mode(void); static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; }; #endif =20 -extern unsigned int __ro_after_init kvm_sve_max_vl; -extern unsigned int __ro_after_init kvm_host_sve_max_vl; +extern unsigned int __ro_after_init kvm_max_vl[ARM64_VEC_MAX]; +extern unsigned int __ro_after_init kvm_host_max_vl[ARM64_VEC_MAX]; +DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); + int __init kvm_arm_init_sve(void); =20 u32 __attribute_const__ kvm_target_cpu(void); @@ -813,7 +815,7 @@ struct kvm_vcpu_arch { */ void *sve_state; enum fp_type fp_type; - unsigned int sve_max_vl; + unsigned int max_vl[ARM64_VEC_MAX]; =20 /* Stage 2 paging state used by the hardware on next switch */ struct kvm_s2_mmu *hw_mmu; @@ -1095,9 +1097,12 @@ struct kvm_vcpu_arch { =20 /* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ #define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ - sve_ffr_offset((vcpu)->arch.sve_max_vl)) + sve_ffr_offset((vcpu)->arch.max_vl[ARM64_VEC_SVE])) + +#define vcpu_vec_max_vq(vcpu, type) sve_vq_from_vl((vcpu)->arch.max_vl[typ= e]) + +#define vcpu_sve_max_vq(vcpu) vcpu_vec_max_vq(vcpu, ARM64_VEC_SVE) =20 -#define vcpu_sve_max_vq(vcpu) sve_vq_from_vl((vcpu)->arch.sve_max_vl) =20 #define vcpu_sve_zcr_elx(vcpu) \ (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) @@ -1116,7 +1121,7 @@ struct kvm_vcpu_arch { __size_ret; \ }) =20 -#define vcpu_sve_state_size(vcpu) sve_state_size_from_vl((vcpu)->arch.sve_= max_vl) +#define vcpu_sve_state_size(vcpu) sve_state_size_from_vl((vcpu)->arch.max_= vl[ARM64_VEC_SVE]) =20 /* * Only use __vcpu_sys_reg/ctxt_sys_reg if you know you want the diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_= hyp.h index e6be1f5d0967..0ad5a66e0d25 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -145,6 +145,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); =20 extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); -extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +extern unsigned int kvm_nvhe_sym(kvm_host_max_vl[ARM64_VEC_MAX]); =20 #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm= _pkvm.h index ea58282f59bb..6925606f2263 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -166,7 +166,7 @@ static inline size_t pkvm_host_sve_state_size(void) return 0; =20 return size_add(sizeof(struct cpu_sve_state), - SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]))); } =20 struct pkvm_mapping { diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 9158353d8be3..1f4fcc8b5554 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -75,7 +75,7 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) */ fp_state.st =3D &vcpu->arch.ctxt.fp_regs; fp_state.sve_state =3D vcpu->arch.sve_state; - fp_state.sve_vl =3D vcpu->arch.sve_max_vl; + fp_state.sve_vl =3D vcpu->arch.max_vl[ARM64_VEC_SVE]; fp_state.sme_state =3D NULL; fp_state.svcr =3D __ctxt_sys_reg(&vcpu->arch.ctxt, SVCR); fp_state.fpmr =3D __ctxt_sys_reg(&vcpu->arch.ctxt, FPMR); diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 6eb273b525d5..81f2b1111180 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -318,7 +318,7 @@ static int get_sve_vls(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) if (!vcpu_has_sve(vcpu)) return -ENOENT; =20 - if (WARN_ON(!sve_vl_valid(vcpu->arch.sve_max_vl))) + if (WARN_ON(!sve_vl_valid(vcpu->arch.max_vl[ARM64_VEC_SVE]))) return -EINVAL; =20 memset(vqs, 0, sizeof(vqs)); @@ -356,7 +356,7 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) if (vq_present(vqs, vq)) max_vq =3D vq; =20 - if (max_vq > sve_vq_from_vl(kvm_sve_max_vl)) + if (max_vq > sve_vq_from_vl(kvm_max_vl[ARM64_VEC_SVE])) return -EINVAL; =20 /* @@ -375,7 +375,7 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) return -EINVAL; =20 /* vcpu->arch.sve_state will be alloc'd by kvm_vcpu_finalize_sve() */ - vcpu->arch.sve_max_vl =3D sve_vl_from_vq(max_vq); + vcpu->arch.max_vl[ARM64_VEC_SVE] =3D sve_vl_from_vq(max_vq); =20 return 0; } diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/i= nclude/hyp/switch.h index d5620e02787f..cd76129c6630 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -578,8 +578,8 @@ static inline void __hyp_sve_save_host(void) struct cpu_sve_state *sve_state =3D *host_data_ptr(sve_state); =20 sve_state->zcr_el1 =3D read_sysreg_el1(SYS_ZCR); - write_sysreg_s(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, SYS_ZCR_EL2); - __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl= ), + write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZC= R_EL2); + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_max_vl[ARM= 64_VEC_SVE]), &sve_state->fpsr, true); } @@ -634,7 +634,7 @@ static inline void fpsimd_lazy_switch_to_host(struct kv= m_vcpu *vcpu) zcr_el2 =3D vcpu_sve_max_vq(vcpu) - 1; write_sysreg_el2(zcr_el2, SYS_ZCR); } else { - zcr_el2 =3D sve_vq_from_vl(kvm_host_sve_max_vl) - 1; + zcr_el2 =3D sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1; write_sysreg_el2(zcr_el2, SYS_ZCR); =20 zcr_el1 =3D vcpu_sve_max_vq(vcpu) - 1; diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h= yp-main.c index 3206b2c07f82..76be13efcfcb 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -34,7 +34,7 @@ static void __hyp_sve_save_guest(struct kvm_vcpu *vcpu) */ sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); __sve_save_state(vcpu_sve_pffr(vcpu), &vcpu->arch.ctxt.fp_regs.fpsr, true= ); - write_sysreg_s(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, SYS_ZCR_EL2); + write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZC= R_EL2); } =20 static void __hyp_sve_restore_host(void) @@ -50,8 +50,8 @@ static void __hyp_sve_restore_host(void) * that was discovered, if we wish to use larger VLs this will * need to be revisited. */ - write_sysreg_s(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, SYS_ZCR_EL2); - __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max= _vl), + write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZC= R_EL2); + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_max_vl[= ARM64_VEC_SVE]), &sve_state->fpsr, true); write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index a461f192230a..65c49a5c7091 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -20,7 +20,7 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; =20 -unsigned int kvm_host_sve_max_vl; +unsigned int kvm_host_max_vl[ARM64_VEC_MAX]; =20 /* * The currently loaded hyp vCPU for each physical CPU. Used only when @@ -425,7 +425,8 @@ static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp= _vcpu, struct kvm_vcpu *h } =20 /* Limit guest vector length to the maximum supported by the host. */ - sve_max_vl =3D min(READ_ONCE(host_vcpu->arch.sve_max_vl), kvm_host_sve_ma= x_vl); + sve_max_vl =3D min(READ_ONCE(host_vcpu->arch.max_vl[ARM64_VEC_SVE]), + kvm_host_max_vl[ARM64_VEC_SVE]); sve_state_size =3D sve_state_size_from_vl(sve_max_vl); sve_state =3D kern_hyp_va(READ_ONCE(host_vcpu->arch.sve_state)); =20 @@ -439,7 +440,7 @@ static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp= _vcpu, struct kvm_vcpu *h goto err; =20 vcpu->arch.sve_state =3D sve_state; - vcpu->arch.sve_max_vl =3D sve_max_vl; + vcpu->arch.max_vl[ARM64_VEC_SVE] =3D sve_max_vl; =20 return 0; err: diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index f7c63e145d54..a8684a1346ec 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,7 +32,7 @@ =20 /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; -unsigned int __ro_after_init kvm_host_sve_max_vl; +unsigned int __ro_after_init kvm_host_max_vl[ARM64_VEC_MAX]; =20 /* * ARMv8 Reset Values @@ -46,14 +46,14 @@ unsigned int __ro_after_init kvm_host_sve_max_vl; #define VCPU_RESET_PSTATE_SVC (PSR_AA32_MODE_SVC | PSR_AA32_A_BIT | \ PSR_AA32_I_BIT | PSR_AA32_F_BIT) =20 -unsigned int __ro_after_init kvm_sve_max_vl; +unsigned int __ro_after_init kvm_max_vl[ARM64_VEC_MAX]; =20 int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { - kvm_sve_max_vl =3D sve_max_virtualisable_vl(); - kvm_host_sve_max_vl =3D sve_max_vl(); - kvm_nvhe_sym(kvm_host_sve_max_vl) =3D kvm_host_sve_max_vl; + kvm_max_vl[ARM64_VEC_SVE] =3D sve_max_virtualisable_vl(); + kvm_host_max_vl[ARM64_VEC_SVE] =3D sve_max_vl(); + kvm_nvhe_sym(kvm_host_max_vl[ARM64_VEC_SVE]) =3D kvm_host_max_vl[ARM64_V= EC_SVE]; =20 /* * The get_sve_reg()/set_sve_reg() ioctl interface will need @@ -61,16 +61,16 @@ int __init kvm_arm_init_sve(void) * order to support vector lengths greater than * VL_ARCH_MAX: */ - if (WARN_ON(kvm_sve_max_vl > VL_ARCH_MAX)) - kvm_sve_max_vl =3D VL_ARCH_MAX; + if (WARN_ON(kvm_max_vl[ARM64_VEC_SVE] > VL_ARCH_MAX)) + kvm_max_vl[ARM64_VEC_SVE] =3D VL_ARCH_MAX; =20 /* * Don't even try to make use of vector lengths that * aren't available on all CPUs, for now: */ - if (kvm_sve_max_vl < sve_max_vl()) + if (kvm_max_vl[ARM64_VEC_SVE] < sve_max_vl()) pr_warn("KVM: SVE vector length for guests limited to %u bytes\n", - kvm_sve_max_vl); + kvm_max_vl[ARM64_VEC_SVE]); } =20 return 0; @@ -78,7 +78,7 @@ int __init kvm_arm_init_sve(void) =20 static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) { - vcpu->arch.sve_max_vl =3D kvm_sve_max_vl; + vcpu->arch.max_vl[ARM64_VEC_SVE] =3D kvm_max_vl[ARM64_VEC_SVE]; =20 /* * Userspace can still customize the vector lengths by writing @@ -99,7 +99,7 @@ static int kvm_vcpu_finalize_vec(struct kvm_vcpu *vcpu) size_t reg_sz; int ret; =20 - vl =3D vcpu->arch.sve_max_vl; + vl =3D vcpu->arch.max_vl[ARM64_VEC_SVE]; =20 /* * Responsibility for these properties is shared between --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 480242FB61A; Tue, 2 Sep 2025 11:45:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813530; cv=none; b=Smr8VNv+ELTuDB2NsqQmUdUpnPeBfMG/gwd09C929MnBVvBkX78cXTqxMP5eVQkb2FKkWiT/j6SjaXAA+s9yA+ZyVZQPSHKQsoBgiAC0HG4NZ++sIDbV5xA9/K99ZDn8dmFae35P02mzIWvT6njLzU7kOKTHPCghP/5jUeb2RJE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813530; c=relaxed/simple; bh=MHP76p9ZYAtyF4GOnpjIu8qVdvdMlEFvQbs8UnH2ZLM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=O3PT0Gv39m2sBhCx0N1gOXE/CiHqRNJOaCbe8KqE85KYjGmvxfcNOsvZIBhi66BwFbZCMJgeXOUQPZ+4S9/AKjKQUzaYeXrXb8tu62ZkiGfsrK4Pye2ic6hpqYAuNoFbJUhkdTRdOSEC0kUC+kpk7TB8g6DQ2E0YJR+St8uiJS0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=olq4Bvf2; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="olq4Bvf2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 91E64C4CEED; Tue, 2 Sep 2025 11:45:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813529; bh=MHP76p9ZYAtyF4GOnpjIu8qVdvdMlEFvQbs8UnH2ZLM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=olq4Bvf2LzQ1ygewu7iLnhAG1XDNmaUIMJ86p7+adfMPEn4hDoAHIWNLeGOpYYYWv xMmEvzpBy+HFUp/7IUN0k2gUmUWTA9P/Qccg8rlhRkzrMyfN6SqWF70ys2bigtRmmA loMbU/eEqgNEFiwhpjPwVwNvgBUCSpgXvykY0SSiZEbQ/blB7wrgFl5YHk6v+izCNm /4wIHV5IVA0S0FJ86gqEAgejJPyEBxmJp+ayRx90vZFXiPeHBimJ/fOaFIhRy4auvm WFSPmyKyt5gWWrCv39XkzYe7J2V1OCuW4YcV6NnUS2JTf7GZqHbisGDu1XzuD3nEP0 hbX6vmRbsi+pg== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:18 +0100 Subject: [PATCH v8 15/29] KVM: arm64: Implement SME vector length configuration Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-15-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=10123; i=broonie@kernel.org; h=from:subject:message-id; bh=MHP76p9ZYAtyF4GOnpjIu8qVdvdMlEFvQbs8UnH2ZLM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiFnGQXsJT3lPq5cxMpNG9uZl1IM08g364Ob baG1pfkffeJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYhQAKCRAk1otyXVSH 0D2MB/9CRX8bWn+fm6K9CkWYVEOokM0BmHExbEgLjz6SwPJLx0S4bFxzENCb85qf77fPlGXmBDX BCj3sr7A2bVfXogJ4HzVZp0yo/9VPkIPUEYvjzQMlMyRCG3NiJzgBvn63hhpDwGRJmzJzZSumo6 sCLdvBSzCBCWjFa4RMtyQDV8MSFuYbukRjF3r9xu29hoLooNc4kJZj6RAH6MgURZtF936siBs85 Y/5kHZ74NBUSMZ82AEcsOSvnzfa/TZYjFQr2Wvh8+u36bYPNLbg/KIAtZl95uVr3CW5jo2CJQsd /WNZ/7n5CaTPg5HphPuHydtDC7qkFuWpQwydjnxoo04DPSmJ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME implements a vector length which architecturally looks very similar to that for SVE, configured in a very similar manner. This controls the vector length used for the ZA matrix register, and for the SVE vector and predicate registers when in streaming mode. The only substantial difference is that unlike SVE the architecture does not guarantee that any particular vector length will be implemented. Configuration for SME vector lengths is done using a virtual register as for SVE, hook up the implementation for the virtual register. Since we do not yet have support for any of the new SME registers stub register access functions are provided that only allow VL configuration. These will be extended as the SME specific registers, as for SVE. Since vq_available() is currently only defined for CONFIG_SVE add a stub for builds where that is disabled. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 1 + arch/arm64/include/asm/kvm_host.h | 24 ++++++++++-- arch/arm64/include/uapi/asm/kvm.h | 9 +++++ arch/arm64/kvm/guest.c | 82 +++++++++++++++++++++++++++++++----= ---- 4 files changed, 96 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsim= d.h index 0ecdd7dcf623..38c24c6485ad 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -340,6 +340,7 @@ static inline int sve_max_vl(void) return -EINVAL; } =20 +static inline bool vq_available(enum vec_type type, unsigned int vq) { ret= urn false; } static inline bool sve_vq_available(unsigned int vq) { return false; } =20 static inline void sve_user_disable(void) { BUILD_BUG(); } diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 59b88206d14c..4348f297b5ae 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -812,8 +812,15 @@ struct kvm_vcpu_arch { * low 128 bits of the SVE Z registers. When the core * floating point code saves the register state of a task it * records which view it saved in fp_type. + * + * If SME support is also present then it provides an + * alternative view of the SVE registers accessed as for the Z + * registers when PSTATE.SM is 1, plus an additional set of + * SME specific state in the matrix register ZA and LUT + * register ZT0. */ void *sve_state; + void *sme_state; enum fp_type fp_type; unsigned int max_vl[ARM64_VEC_MAX]; =20 @@ -1095,14 +1102,23 @@ struct kvm_vcpu_arch { =20 #define vcpu_gp_regs(v) (&(v)->arch.ctxt.regs) =20 -/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ -#define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ - sve_ffr_offset((vcpu)->arch.max_vl[ARM64_VEC_SVE])) - #define vcpu_vec_max_vq(vcpu, type) sve_vq_from_vl((vcpu)->arch.max_vl[typ= e]) =20 #define vcpu_sve_max_vq(vcpu) vcpu_vec_max_vq(vcpu, ARM64_VEC_SVE) +#define vcpu_sme_max_vq(vcpu) vcpu_vec_max_vq(vcpu, ARM64_VEC_SME) + +#define vcpu_sve_max_vl(vcpu) ((vcpu)->arch.max_vl[ARM64_VEC_SVE]) +#define vcpu_sme_max_vl(vcpu) ((vcpu)->arch.max_vl[ARM64_VEC_SME]) =20 +#define vcpu_max_vl(vcpu) max(vcpu_sve_max_vl(vcpu), vcpu_sme_max_vl(vcpu)) +#define vcpu_max_vq(vcpu) sve_vq_from_vl(vcpu_max_vl(vcpu)) + +#define vcpu_cur_sve_vl(vcpu) (vcpu_in_streaming_mode(vcpu) ? \ + vcpu_sme_max_vl(vcpu) : vcpu_sve_max_vl(vcpu)) + +/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ +#define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ + sve_ffr_offset(vcpu_cur_sve_vl(vcpu))) =20 #define vcpu_sve_zcr_elx(vcpu) \ (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/as= m/kvm.h index 4d789871bec1..5220797361e0 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -354,6 +354,15 @@ struct kvm_arm_counter_offset { #define KVM_ARM64_SVE_VLS_WORDS \ ((KVM_ARM64_SVE_VQ_MAX - KVM_ARM64_SVE_VQ_MIN) / 64 + 1) =20 +/* SME registers */ +#define KVM_REG_ARM64_SME (0x17 << KVM_REG_ARM_COPROC_SHIFT) + +/* Vector lengths pseudo-register: */ +#define KVM_REG_ARM64_SME_VLS (KVM_REG_ARM64 | KVM_REG_ARM64_SME | \ + KVM_REG_SIZE_U512 | 0xffff) +#define KVM_ARM64_SME_VLS_WORDS \ + ((KVM_ARM64_SVE_VQ_MAX - KVM_ARM64_SVE_VQ_MIN) / 64 + 1) + /* Bitmap feature firmware registers */ #define KVM_REG_ARM_FW_FEAT_BMAP (0x0016 << KVM_REG_ARM_COPROC_SHIFT) #define KVM_REG_ARM_FW_FEAT_BMAP_REG(r) (KVM_REG_ARM64 | KVM_REG_SIZE_U64= | \ diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 81f2b1111180..bcb0d565f1a4 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -310,22 +310,20 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const = struct kvm_one_reg *reg) #define vq_mask(vq) ((u64)1 << ((vq) - SVE_VQ_MIN) % 64) #define vq_present(vqs, vq) (!!((vqs)[vq_word(vq)] & vq_mask(vq))) =20 -static int get_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) +static int get_vec_vls(enum vec_type vec_type, struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg) { unsigned int max_vq, vq; u64 vqs[KVM_ARM64_SVE_VLS_WORDS]; =20 - if (!vcpu_has_sve(vcpu)) - return -ENOENT; - - if (WARN_ON(!sve_vl_valid(vcpu->arch.max_vl[ARM64_VEC_SVE]))) + if (WARN_ON(!sve_vl_valid(vcpu->arch.max_vl[vec_type]))) return -EINVAL; =20 memset(vqs, 0, sizeof(vqs)); =20 - max_vq =3D vcpu_sve_max_vq(vcpu); + max_vq =3D vcpu_vec_max_vq(vcpu, vec_type); for (vq =3D SVE_VQ_MIN; vq <=3D max_vq; ++vq) - if (sve_vq_available(vq)) + if (vq_available(vec_type, vq)) vqs[vq_word(vq)] |=3D vq_mask(vq); =20 if (copy_to_user((void __user *)reg->addr, vqs, sizeof(vqs))) @@ -334,40 +332,41 @@ static int get_sve_vls(struct kvm_vcpu *vcpu, const s= truct kvm_one_reg *reg) return 0; } =20 -static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) +static int set_vec_vls(enum vec_type vec_type, struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg) { unsigned int max_vq, vq; u64 vqs[KVM_ARM64_SVE_VLS_WORDS]; =20 - if (!vcpu_has_sve(vcpu)) - return -ENOENT; - if (kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; /* too late! */ =20 - if (WARN_ON(vcpu->arch.sve_state)) + if (WARN_ON(!sve_vl_valid(vcpu->arch.max_vl[vec_type]))) return -EINVAL; =20 if (copy_from_user(vqs, (const void __user *)reg->addr, sizeof(vqs))) return -EFAULT; =20 + if (WARN_ON(vcpu->arch.sve_state || vcpu->arch.sme_state)) + return -EINVAL; + max_vq =3D 0; for (vq =3D SVE_VQ_MIN; vq <=3D SVE_VQ_MAX; ++vq) if (vq_present(vqs, vq)) max_vq =3D vq; =20 - if (max_vq > sve_vq_from_vl(kvm_max_vl[ARM64_VEC_SVE])) + if (max_vq > sve_vq_from_vl(kvm_max_vl[vec_type])) return -EINVAL; =20 /* * Vector lengths supported by the host can't currently be * hidden from the guest individually: instead we can only set a - * maximum via ZCR_EL2.LEN. So, make sure the available vector + * maximum via xCR_EL2.LEN. So, make sure the available vector * lengths match the set requested exactly up to the requested * maximum: */ for (vq =3D SVE_VQ_MIN; vq <=3D max_vq; ++vq) - if (vq_present(vqs, vq) !=3D sve_vq_available(vq)) + if (vq_present(vqs, vq) !=3D vq_available(vec_type, vq)) return -EINVAL; =20 /* Can't run with no vector lengths at all: */ @@ -375,11 +374,27 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const s= truct kvm_one_reg *reg) return -EINVAL; =20 /* vcpu->arch.sve_state will be alloc'd by kvm_vcpu_finalize_sve() */ - vcpu->arch.max_vl[ARM64_VEC_SVE] =3D sve_vl_from_vq(max_vq); + vcpu->arch.max_vl[vec_type] =3D sve_vl_from_vq(max_vq); =20 return 0; } =20 +static int get_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) +{ + if (!vcpu_has_sve(vcpu)) + return -ENOENT; + + return get_vec_vls(ARM64_VEC_SVE, vcpu, reg); +} + +static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) +{ + if (!vcpu_has_sve(vcpu)) + return -ENOENT; + + return set_vec_vls(ARM64_VEC_SVE, vcpu, reg); +} + #define SVE_REG_SLICE_SHIFT 0 #define SVE_REG_SLICE_BITS 5 #define SVE_REG_ID_SHIFT (SVE_REG_SLICE_SHIFT + SVE_REG_SLICE_BITS) @@ -533,6 +548,39 @@ static int set_sve_reg(struct kvm_vcpu *vcpu, const st= ruct kvm_one_reg *reg) return 0; } =20 +static int get_sme_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) +{ + if (!vcpu_has_sme(vcpu)) + return -ENOENT; + + return get_vec_vls(ARM64_VEC_SME, vcpu, reg); +} + +static int set_sme_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) +{ + if (!vcpu_has_sme(vcpu)) + return -ENOENT; + + return set_vec_vls(ARM64_VEC_SME, vcpu, reg); +} + +static int get_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) +{ + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ + if (reg->id =3D=3D KVM_REG_ARM64_SME_VLS) + return get_sme_vls(vcpu, reg); + + return -EINVAL; +} + +static int set_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) +{ + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ + if (reg->id =3D=3D KVM_REG_ARM64_SME_VLS) + return set_sme_vls(vcpu, reg); + + return -EINVAL; +} int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *r= egs) { return -EINVAL; @@ -775,6 +823,7 @@ int kvm_arm_get_reg(struct kvm_vcpu *vcpu, const struct= kvm_one_reg *reg) case KVM_REG_ARM_FW_FEAT_BMAP: return kvm_arm_get_fw_reg(vcpu, reg); case KVM_REG_ARM64_SVE: return get_sve_reg(vcpu, reg); + case KVM_REG_ARM64_SME: return get_sme_reg(vcpu, reg); } =20 if (is_timer_reg(reg->id)) @@ -795,6 +844,7 @@ int kvm_arm_set_reg(struct kvm_vcpu *vcpu, const struct= kvm_one_reg *reg) case KVM_REG_ARM_FW_FEAT_BMAP: return kvm_arm_set_fw_reg(vcpu, reg); case KVM_REG_ARM64_SVE: return set_sve_reg(vcpu, reg); + case KVM_REG_ARM64_SME: return set_sme_reg(vcpu, reg); } =20 if (is_timer_reg(reg->id)) --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 553772FC00F; Tue, 2 Sep 2025 11:45:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813534; cv=none; b=L6zUO6X2aOrnTatQYrlmG8HKBsA91CshzPAHgA48AJ8yjNY3cZf+efWIG2DwHe7Bqx7fcN22g601zQPDjokpixalF0D4t3dBlDLciV9hcGKcWgTXjZptsUheOldzPvMPZAsQe8c13odXRCdJ88sGg3BWesdGcZ122AZ5Ks4iopU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813534; c=relaxed/simple; bh=0fLGhfB6VaFN9V2OPaprNenzTUhxjwQ4WMNFeuSowXY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oXihA+frJFbHsLjUD03naDDbmBjUYFD3hODK4G9to4VvzfEQJsjiMVRsNQ5VtTPjtRnpjFz0l23uG251rdrzge+Z3FtW3fl7E1K78StRlsbOnZzPBN9yM4sNYv2i9KY7evmGSEZpjSvp5NPSnxQA+cEsnVYdZtCc2FWgAA1uqGc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=g4kXt23b; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="g4kXt23b" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 46704C4CEF8; Tue, 2 Sep 2025 11:45:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813534; bh=0fLGhfB6VaFN9V2OPaprNenzTUhxjwQ4WMNFeuSowXY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=g4kXt23bpso+nNFv4THNUTimu6aAahPoUlS7Z6aoNlCLT/j2+CtW5dCfj6EDDQroY CxDvLmbUpyO8y75GuVNZap4vAKva9rzeu3dNgaGiyhurMFWMxwHNlovKuxO5HZwwtq 0ONwLNEAqk/SNy61KJnLH9//DsOWm/WCTb5BZ+tWfDqGaRkOpVXE9tlMkKGdfloha9 x+pjinSnKXPE7ehi67TFp0oreZBtsnXmoNlBOcyjmQtOXg+VTWsDsvbcvp+4D+tLr/ IUEiZkIJrWyaJsOnCvNGmXtkL593y9itmTIi6gJoL+PYXN83vJGorcyJ8/jiFUOux8 XnMV8IpdTsOLg== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:19 +0100 Subject: [PATCH v8 16/29] KVM: arm64: Support SME control registers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-16-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=4807; i=broonie@kernel.org; h=from:subject:message-id; bh=0fLGhfB6VaFN9V2OPaprNenzTUhxjwQ4WMNFeuSowXY=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiGSWd/Ihob9g8NK9RPBSZUvgmJDyCN/VXIh FyC9PxLim2JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYhgAKCRAk1otyXVSH 0GEnB/9nLGGwCKUeQvIiZCjBrlVztcGG0JqnYGJjaypFwfonrlb2ZStP+KZEN5/SScNrtdA0qV3 mn/WSpC+Pkd2G9b/FmbIC4y2xUWcLkmYpcDbDFGRfjz02ORZztG2wcby4L7ZamjBoY4UCPkD4Be cB4jm8DBF3Dz2R/xZcGE3T5FVMpn/9kqyq8BCl8D9qefupeJD83xNF2WpAODL1oZqqGj7mufFB4 /o7HQ2IzCvt6AaJ2DqGIEvoHiDZvI6vG45poloILrogZWjhjjYk3DohpEUNwIxT0Rfg8ELHxltu FDdwqlR6m7/rc1hn8kBRGlYXRq5yZ79HJgJMc74ps2IfSYnk X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME is configured by the system registers SMCR_EL1 and SMCR_EL2, add definitions and userspace access for them. These control the SME vector length in a manner similar to that for SVE and also have feature enable bits for SME2 and FA64. A subsequent patch will add management of them for guests as part of the general floating point context switch, as is done for the equivalent SVE registers. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/include/asm/vncr_mapping.h | 1 + arch/arm64/kvm/sys_regs.c | 37 +++++++++++++++++++++++++++++++= +++- 3 files changed, 39 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 4348f297b5ae..edb2acdb4bd1 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -506,6 +506,7 @@ enum vcpu_sysreg { CPTR_EL2, /* Architectural Feature Trap Register (EL2) */ HACR_EL2, /* Hypervisor Auxiliary Control Register */ ZCR_EL2, /* SVE Control Register (EL2) */ + SMCR_EL2, /* SME Control Register (EL2) */ TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */ TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */ TCR_EL2, /* Translation Control Register (EL2) */ @@ -544,6 +545,7 @@ enum vcpu_sysreg { VNCR(ACTLR_EL1),/* Auxiliary Control Register */ VNCR(CPACR_EL1),/* Coprocessor Access Control */ VNCR(ZCR_EL1), /* SVE Control */ + VNCR(SMCR_EL1), /* SME Control */ VNCR(TTBR0_EL1),/* Translation Table Base Register 0 */ VNCR(TTBR1_EL1),/* Translation Table Base Register 1 */ VNCR(TCR_EL1), /* Translation Control Register */ diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm= /vncr_mapping.h index f6ec500ad3fa..5ab6a964bccf 100644 --- a/arch/arm64/include/asm/vncr_mapping.h +++ b/arch/arm64/include/asm/vncr_mapping.h @@ -44,6 +44,7 @@ #define VNCR_HDFGWTR_EL2 0x1D8 #define VNCR_ZCR_EL1 0x1E0 #define VNCR_HAFGRTR_EL2 0x1E8 +#define VNCR_SMCR_EL1 0x1F0 #define VNCR_TTBR0_EL1 0x200 #define VNCR_TTBR1_EL1 0x210 #define VNCR_FAR_EL1 0x220 diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 111b07a78787..81742e9237c0 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -142,6 +142,7 @@ static bool get_el2_to_el1_mapping(unsigned int reg, MAPPED_EL2_SYSREG(ELR_EL2, ELR_EL1, NULL ); MAPPED_EL2_SYSREG(SPSR_EL2, SPSR_EL1, NULL ); MAPPED_EL2_SYSREG(ZCR_EL2, ZCR_EL1, NULL ); + MAPPED_EL2_SYSREG(SMCR_EL2, SMCR_EL1, NULL ); MAPPED_EL2_SYSREG(CONTEXTIDR_EL2, CONTEXTIDR_EL1, NULL ); MAPPED_EL2_SYSREG(SCTLR2_EL2, SCTLR2_EL1, NULL ); default: @@ -2589,6 +2590,37 @@ static bool access_gic_elrsr(struct kvm_vcpu *vcpu, return true; } =20 +static unsigned int sme_el2_visibility(const struct kvm_vcpu *vcpu, + const struct sys_reg_desc *rd) +{ + return __el2_visibility(vcpu, rd, sme_visibility); +} + +static bool access_smcr_el2(struct kvm_vcpu *vcpu, + struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + unsigned int vq; + u64 smcr; + + if (guest_hyp_sve_traps_enabled(vcpu)) { + kvm_inject_nested_sve_trap(vcpu); + return true; + } + + if (!p->is_write) { + p->regval =3D vcpu_read_sys_reg(vcpu, SMCR_EL2); + return true; + } + + smcr =3D p->regval; + vq =3D SYS_FIELD_GET(SMCR_ELx, LEN, smcr) + 1; + vq =3D min(vq, vcpu_sme_max_vq(vcpu)); + vcpu_write_sys_reg(vcpu, SYS_FIELD_PREP(SMCR_ELx, LEN, vq - 1), + SMCR_EL2); + return true; +} + static unsigned int s1poe_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd) { @@ -3035,7 +3067,7 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { { SYS_DESC(SYS_ZCR_EL1), NULL, reset_val, ZCR_EL1, 0, .visibility =3D sve= _visibility }, { SYS_DESC(SYS_TRFCR_EL1), undef_access }, { SYS_DESC(SYS_SMPRI_EL1), undef_access }, - { SYS_DESC(SYS_SMCR_EL1), undef_access }, + { SYS_DESC(SYS_SMCR_EL1), NULL, reset_val, SMCR_EL1, 0, .visibility =3D s= me_visibility }, { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 }, { SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 }, { SYS_DESC(SYS_TCR_EL1), access_vm_reg, reset_val, TCR_EL1, 0 }, @@ -3391,6 +3423,9 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { =20 EL2_REG_VNCR(HCRX_EL2, reset_val, 0), =20 + EL2_REG_FILTERED(SMCR_EL2, access_smcr_el2, reset_val, 0, + sme_el2_visibility), + EL2_REG(TTBR0_EL2, access_rw, reset_val, 0), EL2_REG(TTBR1_EL2, access_rw, reset_val, 0), EL2_REG(TCR_EL2, access_rw, reset_val, TCR_EL2_RES1), --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02BAE2FC020; Tue, 2 Sep 2025 11:45:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813540; cv=none; b=PA5DuptkHQXqfX4OAXzoEj98wBfG5yWrAP8KDfPXKiRZHBOyNmFVPwVGE5jBFjk/XI1X2KcH0scNh+ES/GdbUOdCnzv9k9x12G3jAkpImTsTyKAUPzP0dfzztIGGnZRGUB5yplG0cINft36NdFY4ZhfJ6WILcDg17aJf9KyD1iI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813540; c=relaxed/simple; bh=s9OgbUV3jMegR4yK5YJZn39bA8MN2bqc20M/qGI/rlg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=p6eSV7pUlMw32pP7iPQL04hxPV6nCvx2V28cd2an5caFEipSek6HN3FZX85/ZkbRUd6YJGzCN/HgADjMLBdnLw4itg8RogIBRQ3xXCp3jVYfxkNN5hQV6tciBzGmBBHA+KRTb6V0mRwmpO94uK8Z87yz5C3bJOeQLq7CjLCQ5j8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QZuYqXi+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QZuYqXi+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95C9CC4CEED; Tue, 2 Sep 2025 11:45:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813538; bh=s9OgbUV3jMegR4yK5YJZn39bA8MN2bqc20M/qGI/rlg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QZuYqXi+TSZTdOoPLIGC+xq12gG0BUGZV/l6O6cGxTjuMso+gHFB60OFQQWy4bZwG SCl+AxUwSejvbC60tDeruUtR24GXcbQQ6POOapbxD9gl7Qk9KIvGndoHeGR0HcMPFK owi4z68OUMZ8Y665JLPPIoRATIwi/q7V/JuGTDYsluS/CBGfNB4lt5BmnaxVDgjxVg J2WY1CgntRGRyILwRZB42gohn2824HLssRVhlsGmJUQER/Br6V7UJ9DEAk7q9tbmMr yvK8ZWRDLOCLzs+9IWrHM4BfqjRE4vUFyYbr2efzrX3AqrZ4pXQHu5v78xaWCvMfJu UFcE5fo+5oUUg== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:20 +0100 Subject: [PATCH v8 17/29] KVM: arm64: Support TPIDR2_EL0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-17-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=3465; i=broonie@kernel.org; h=from:subject:message-id; bh=s9OgbUV3jMegR4yK5YJZn39bA8MN2bqc20M/qGI/rlg=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiHDFKJXw5f2H21IFF1uQvm/fPZsanmuSrom /a5i86XmZ6JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYhwAKCRAk1otyXVSH 0OahB/9y3zF/euhwkQBojIIq0LMsL+jbp3v2b9QbC0+ZlZGtcktmT5/5mY8wKvAJGw/Tgu0/Q+Z 5YLpJf5cp+gcsIV93+PFIPh42owo6MklqRVtxOj/8RF4g2vC3akJV1XD1lrZpZe/CZRXF7J+vDU F1TcDxb6Vh781UNrLAs9cTMINcu0R3QJXaCySGQ0WFb6tbvPPj+urfNZYwHq9/1rMKCKN7olb1X +bredmoG7fhmcywiAEgR0n8kF3V2SIiDigFPkMjT812i+nJqbGeXRVe6YE9rh+tQBPuXTR+deMH OVLTPtaUy6h8TRHTboPrmy7/OQaEJU05Iec6EPI2ZssuiBGw X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME adds a new thread ID register, TPIDR2_EL0. This is used in userspace for delayed saving of the ZA state but in terms of the architecture is not really connected to SME other than being part of FEAT_SME. It has an independent fine grained trap and the runtime connection with the rest of SME is purely software defined. Expose the register as a system register if the guest supports SME, context switching it along with the other EL0 TPIDRs. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 15 +++++++++++++++ arch/arm64/kvm/sys_regs.c | 3 ++- 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index edb2acdb4bd1..632c74397fa9 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -450,6 +450,7 @@ enum vcpu_sysreg { CSSELR_EL1, /* Cache Size Selection Register */ TPIDR_EL0, /* Thread ID, User R/W */ TPIDRRO_EL0, /* Thread ID, User R/O */ + TPIDR2_EL0, /* Thread ID, Register 2 */ TPIDR_EL1, /* Thread ID, Privileged */ CNTKCTL_EL1, /* Timer Control Register (EL1) */ PAR_EL1, /* Physical Address Register */ diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hy= p/include/hyp/sysreg-sr.h index 5624fd705ae3..8c3b3d6df99f 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -88,6 +88,17 @@ static inline bool ctxt_has_sctlr2(struct kvm_cpu_contex= t *ctxt) return kvm_has_sctlr2(kern_hyp_va(vcpu->kvm)); } =20 +static inline bool ctxt_has_sme(struct kvm_cpu_context *ctxt) +{ + struct kvm_vcpu *vcpu; + + if (!system_supports_sme()) + return false; + + vcpu =3D ctxt_to_vcpu(ctxt); + return kvm_has_sme(kern_hyp_va(vcpu->kvm)); +} + static inline bool ctxt_is_guest(struct kvm_cpu_context *ctxt) { return host_data_ptr(host_ctxt) !=3D ctxt; @@ -127,6 +138,8 @@ static inline void __sysreg_save_user_state(struct kvm_= cpu_context *ctxt) { ctxt_sys_reg(ctxt, TPIDR_EL0) =3D read_sysreg(tpidr_el0); ctxt_sys_reg(ctxt, TPIDRRO_EL0) =3D read_sysreg(tpidrro_el0); + if (ctxt_has_sme(ctxt)) + ctxt_sys_reg(ctxt, TPIDR2_EL0) =3D read_sysreg_s(SYS_TPIDR2_EL0); } =20 static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) @@ -204,6 +217,8 @@ static inline void __sysreg_restore_user_state(struct k= vm_cpu_context *ctxt) { write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL0), tpidr_el0); write_sysreg(ctxt_sys_reg(ctxt, TPIDRRO_EL0), tpidrro_el0); + if (ctxt_has_sme(ctxt)) + write_sysreg_s(ctxt_sys_reg(ctxt, TPIDR2_EL0), SYS_TPIDR2_EL0); } =20 static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt, diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 81742e9237c0..4166d396450d 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -3243,7 +3243,8 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { .visibility =3D s1poe_visibility }, { SYS_DESC(SYS_TPIDR_EL0), NULL, reset_unknown, TPIDR_EL0 }, { SYS_DESC(SYS_TPIDRRO_EL0), NULL, reset_unknown, TPIDRRO_EL0 }, - { SYS_DESC(SYS_TPIDR2_EL0), undef_access }, + { SYS_DESC(SYS_TPIDR2_EL0), NULL, reset_unknown, TPIDR2_EL0, + .visibility =3D sme_visibility}, =20 { SYS_DESC(SYS_SCXTNUM_EL0), undef_access }, =20 --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB9A92F5484; Tue, 2 Sep 2025 11:45:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813542; cv=none; b=WtAPkHC9zFv70TJm7flxqyNItcdton6Py1A7JRv2UQ1dMxTZO/BO/nuiURsSbb98CudRVxTyFgpE5zrbnh2A8qweSNlPCO50N9hKuGWnZ8bcaBPwtWLnv6lXOn+eJk243emIUiAILqzC7jC17nfa1TUzDI6G4PqxwP37QwFUs9E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813542; c=relaxed/simple; bh=IOvRUXTwez4vce4kM+TbWeJxICUO2t5XVZ3KjPI4DQM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OSAdY/XBbKCm1d+5r9S0r6wnSQuUZIpaRZ40iKuIPgz+hZTbEh/L8lZwf1p1ZiKcdpKKcMZVzPRIY5pLZDvHY0hUD8rg3K9G6paNMsJiuKf86fCSfCErslfjVl/APrVFv3U3udi1Awr56qXSEHhgnb8AicR1FYXbLMvKTHrekEI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UpaCi98s; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UpaCi98s" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F272FC4CEF5; Tue, 2 Sep 2025 11:45:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813542; bh=IOvRUXTwez4vce4kM+TbWeJxICUO2t5XVZ3KjPI4DQM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=UpaCi98s6fPka0+erYNtNcDAX8/3Et25DXn+qMm58Tm4pqx4H4KBboTCtoMUbJWVc SjIdJpk6LJyxyVG6WN1bkYKfry4veUTdfJ6TdPe+epw9BhzonbJqF5J/eRMZcTGLH+ fLzhjmdoJ+6/8bHyA8JfTpWhkAXqWC8jEpeATbSZxkwYlIsxUMBDjcqZ+paqyMSQhV cEj0Avozb6kKrZjU5g/uhsuVuUhgDJp7HSVTJZPWttYuefNbYWKOIxwCREYhgv1tkK aLE5tyEKakGavOhBuqzXNcCsJIx5T2KQ5/+9ColYWaAyx3Oie4wqi9QYPbN6NeDWxC oISHNJfwnH4OA== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:21 +0100 Subject: [PATCH v8 18/29] KVM: arm64: Support SME identification registers for guests Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-18-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=8730; i=broonie@kernel.org; h=from:subject:message-id; bh=IOvRUXTwez4vce4kM+TbWeJxICUO2t5XVZ3KjPI4DQM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiH0RF71Uziy+zSJEfHp66qaaMSnQCV0xwMQ T6hpEmJOWyJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYhwAKCRAk1otyXVSH 0CVcB/99MT8EGZWMnh4l5mgyn/GrjswQZsMK0qjufEghAgQz2ODg2Z6w/4pAvsDcP8CYjNrhgEV ENIDGxaTve3aT5jDaoc7pr7CS5DiqAGO7acmb/BSjCqj9JXUaSFb9Ilq903SO3XxApTsedvYvTo 08Fj28i0M3BF8mebJlU1jYQEu/6LpNbrblPu2DGQha/vsKVI58KThCRsEkyRLC52M7AMyB79rGx GDoDePXrdcI+ZYEmxTtRyJ1Ktqys4uKAbF9Mjl61H0DPBSBGNDpngfkRYywhKOYLZQMdDl8ypjI 0MXtCFOGcAvrtn3uHMoQ5cZID2i+P2PXXk3AIbX/7U/tkAo8 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The primary register for identifying SME is ID_AA64PFR1_EL1.SME. This is hidden from guests unless SME is enabled by the VMM. When it is visible it is writable and can be used to control the availability of SME2. There is also a new register ID_AA64SMFR0_EL1 which we make writable, forcing it to all bits 0 if SME is disabled. This includes the field SMEver giving the SME version, userspace is responsible for ensuring the value is consistent with ID_AA64PFR1_EL1.SME. It also includes FA64, a separately enableable extension which provides the full FPSIMD and SVE instruction set including FFR in streaming mode. Userspace can control the availability of FA64 by writing to this field. The other features enumerated there only add new instructions, there are no architectural controls for these. There is a further identification register SMIDR_EL1 which provides a basic description of the SME microarchitecture, in a manner similar to MIDR_EL1 for the PE. It also describes support for priority management and a basic affinity description for shared SME units, plus some RES0 space. We do not support priority management for guests so this is hidden from guests, along with any new fields. As for MIDR_EL1 and REVIDR_EL1 we expose the implementer and revision information to guests with the raw value from the CPU we are running on, this may present issues for asymmetric systems or for migration as it does for the existing registers. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 3 +++ arch/arm64/kvm/config.c | 8 +------ arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 11 +++++++++ arch/arm64/kvm/hyp/nvhe/pkvm.c | 4 +++- arch/arm64/kvm/sys_regs.c | 37 ++++++++++++++++++++++++++= ---- 5 files changed, 51 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 632c74397fa9..5225dd9b752c 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -402,6 +402,7 @@ struct kvm_arch { u64 revidr_el1; u64 aidr_el1; u64 ctr_el0; + u64 smidr_el1; =20 /* Masks for VNCR-backed and general EL2 sysregs */ struct kvm_sysreg_masks *sysreg_masks; @@ -1646,6 +1647,8 @@ static inline u64 *__vm_id_reg(struct kvm_arch *ka, u= 32 reg) return &ka->revidr_el1; case SYS_AIDR_EL1: return &ka->aidr_el1; + case SYS_SMIDR_EL1: + return &ka->smidr_el1; default: WARN_ON_ONCE(1); return NULL; diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c index da66c4a14775..a08e89e210d8 100644 --- a/arch/arm64/kvm/config.c +++ b/arch/arm64/kvm/config.c @@ -208,14 +208,8 @@ static bool feat_anerr(struct kvm *kvm) =20 static bool feat_sme_smps(struct kvm *kvm) { - /* - * Revists this if KVM ever supports SME -- this really should - * look at the guest's view of SMIDR_EL1. Funnily enough, this - * is not captured in the JSON file, but only as a note in the - * ARM ARM. - */ return (kvm_has_feat(kvm, FEAT_SME) && - (read_sysreg_s(SYS_SMIDR_EL1) & SMIDR_EL1_SMPS)); + (kvm_read_vm_id_reg(kvm, SYS_SMIDR_EL1) & SMIDR_EL1_SMPS)); } =20 static bool feat_spe_fds(struct kvm *kvm) diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hy= p/include/hyp/sysreg-sr.h index 8c3b3d6df99f..d921db152119 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -125,6 +125,17 @@ static inline u64 ctxt_midr_el1(struct kvm_cpu_context= *ctxt) return kvm_read_vm_id_reg(kvm, SYS_MIDR_EL1); } =20 +static inline u64 ctxt_smidr_el1(struct kvm_cpu_context *ctxt) +{ + struct kvm *kvm =3D kern_hyp_va(ctxt_to_vcpu(ctxt)->kvm); + + if (!(ctxt_is_guest(ctxt) && + test_bit(KVM_ARCH_FLAG_WRITABLE_IMP_ID_REGS, &kvm->arch.flags))) + return read_sysreg_s(SYS_SMIDR_EL1); + + return kvm_read_vm_id_reg(kvm, SYS_SMIDR_EL1); +} + static inline void __sysreg_save_common_state(struct kvm_cpu_context *ctxt) { *ctxt_mdscr_el1(ctxt) =3D read_sysreg(mdscr_el1); diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 65c49a5c7091..fc9fb2e693f6 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -338,8 +338,10 @@ static void pkvm_init_features_from_host(struct pkvm_h= yp_vm *hyp_vm, const struc host_kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES); =20 - if (test_bit(KVM_ARCH_FLAG_WRITABLE_IMP_ID_REGS, &host_arch_flags)) + if (test_bit(KVM_ARCH_FLAG_WRITABLE_IMP_ID_REGS, &host_arch_flags)) { hyp_vm->kvm.arch.midr_el1 =3D host_kvm->arch.midr_el1; + hyp_vm->kvm.arch.smidr_el1 =3D host_kvm->arch.smidr_el1; + } =20 return; } diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 4166d396450d..36ba1986e7a1 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1612,7 +1612,9 @@ static u64 __kvm_read_sanitised_id_reg(const struct k= vm_vcpu *vcpu, val &=3D ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_MTE_frac); } =20 - val &=3D ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_SME); + if (!vcpu_has_sme(vcpu)) + val &=3D ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_SME); + val &=3D ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_RNDR_trap); val &=3D ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_NMI); val &=3D ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_GCS); @@ -1731,6 +1733,10 @@ static unsigned int id_visibility(const struct kvm_v= cpu *vcpu, if (!vcpu_has_sve(vcpu)) return REG_RAZ; break; + case SYS_ID_AA64SMFR0_EL1: + if (!vcpu_has_sme(vcpu)) + return REG_RAZ; + break; } =20 return 0; @@ -1758,10 +1764,25 @@ static unsigned int raz_visibility(const struct kvm= _vcpu *vcpu, =20 /* cpufeature ID register access trap handlers */ =20 +static bool hidden_id_reg(struct kvm_vcpu *vcpu, + struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + switch (reg_to_encoding(r)) { + case SYS_SMIDR_EL1: + return !vcpu_has_sme(vcpu); + default: + return false; + } +} + static bool access_id_reg(struct kvm_vcpu *vcpu, struct sys_reg_params *p, const struct sys_reg_desc *r) { + if (hidden_id_reg(vcpu, p, r)) + return bad_trap(vcpu, p, r, "write to hidden ID register"); + if (p->is_write) return write_to_read_only(vcpu, p, r); =20 @@ -2789,12 +2810,15 @@ static bool access_imp_id_reg(struct kvm_vcpu *vcpu, static u64 __ro_after_init boot_cpu_midr_val; static u64 __ro_after_init boot_cpu_revidr_val; static u64 __ro_after_init boot_cpu_aidr_val; +static u64 __ro_after_init boot_cpu_smidr_val; =20 static void init_imp_id_regs(void) { boot_cpu_midr_val =3D read_sysreg(midr_el1); boot_cpu_revidr_val =3D read_sysreg(revidr_el1); boot_cpu_aidr_val =3D read_sysreg(aidr_el1); + if (system_supports_sme()) + boot_cpu_smidr_val =3D read_sysreg_s(SYS_SMIDR_EL1); } =20 static u64 reset_imp_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_de= sc *r) @@ -2806,6 +2830,8 @@ static u64 reset_imp_id_reg(struct kvm_vcpu *vcpu, co= nst struct sys_reg_desc *r) return boot_cpu_revidr_val; case SYS_AIDR_EL1: return boot_cpu_aidr_val; + case SYS_SMIDR_EL1: + return boot_cpu_smidr_val; default: KVM_BUG_ON(1, vcpu->kvm); return 0; @@ -2972,7 +2998,6 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { ID_AA64PFR1_EL1_MTE_frac | ID_AA64PFR1_EL1_NMI | ID_AA64PFR1_EL1_RNDR_trap | - ID_AA64PFR1_EL1_SME | ID_AA64PFR1_EL1_RES0 | ID_AA64PFR1_EL1_MPAM_frac | ID_AA64PFR1_EL1_RAS_frac | @@ -2983,7 +3008,7 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { ID_AA64PFR2_EL1_MTESTOREONLY), ID_UNALLOCATED(4,3), ID_WRITABLE(ID_AA64ZFR0_EL1, ~ID_AA64ZFR0_EL1_RES0), - ID_HIDDEN(ID_AA64SMFR0_EL1), + ID_WRITABLE(ID_AA64SMFR0_EL1, ~ID_AA64SMFR0_EL1_RES0), ID_UNALLOCATED(4,6), ID_WRITABLE(ID_AA64FPFR0_EL1, ~ID_AA64FPFR0_EL1_RES0), =20 @@ -3185,7 +3210,11 @@ static const struct sys_reg_desc sys_reg_descs[] =3D= { { SYS_DESC(SYS_CLIDR_EL1), access_clidr, reset_clidr, CLIDR_EL1, .set_user =3D set_clidr, .val =3D ~CLIDR_EL1_RES0 }, { SYS_DESC(SYS_CCSIDR2_EL1), undef_access }, - { SYS_DESC(SYS_SMIDR_EL1), undef_access }, + IMPLEMENTATION_ID(SMIDR_EL1, (SMIDR_EL1_NSMC | SMIDR_EL1_HIP | + SMIDR_EL1_AFFINITY2 | + SMIDR_EL1_IMPLEMENTER | + SMIDR_EL1_REVISION | SMIDR_EL1_SH | + SMIDR_EL1_AFFINITY)), IMPLEMENTATION_ID(AIDR_EL1, GENMASK_ULL(63, 0)), { SYS_DESC(SYS_CSSELR_EL1), access_csselr, reset_unknown, CSSELR_EL1 }, ID_FILTERED(CTR_EL0, ctr_el0, --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04F053054FA; Tue, 2 Sep 2025 11:45:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813547; cv=none; b=NPaKjuOsGMDCF6L7+NyujZithdlmApm8778mSdqhuzd/vKm9+l9fFrimX1bnArkVSt21atR/FRke2eTHZ5UQaxDr4CQlOxA550nfgElqFHd/8EpHCQeGaicFEw/tpYcpAw9Y6S0hQlp2wdbnAwI4My91fCCxETax/+DIXwfiW6E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813547; c=relaxed/simple; bh=MAZZlV6eUub7krRK8EsIsDbLy5E7LRegIDDazNCSgqc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=F9yVZjaf6eUSBsnzFJZOs7yA61OCj1Ay1b0k5IPeyZ0/CQjAebfFAn6DiHNR05mTlaFNG5yiiRpgxwwGQ8dS9FnUEw6GfaOTRnGScvr1IkYESjdZad9+ykeCWfdXw5NNyOmpfOfVYGtcePAkJmt35Lo51Z67b+OrtEGp3PZO86A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dZDzTuom; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dZDzTuom" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 237A5C4CEED; Tue, 2 Sep 2025 11:45:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813546; bh=MAZZlV6eUub7krRK8EsIsDbLy5E7LRegIDDazNCSgqc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=dZDzTuom2qz2uW2MEKgtqYLcfmdX9wZ8o6ekzPqwooJzUVyxUIztXUcqWgyaWgtxL GkT7Abdu3VvsVwMXbt7RmOmj7t0GaW9n/BosaZfHRQH3llNm4mK2L2clQkKRrKQhMH 8s1AA/icDyjCx6szb4WMJJ4WgYWWRyoDaEgk3XMI29igOYcgRWaVRocKDe99dk7N1O R0B8nkpDRNCpb4+Ig+sxoXUQQ6VfoDFPVA9xBnZowrVvu9jolijzSaNRver+kO2mgr 0vQz6iYEfVp2hdp2yuCuk3UkGZmdkp6ftz75k8feNkzUnaj/2aCsPFEjDv9CtdCaQv bqvl7nMd0NoQw== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:22 +0100 Subject: [PATCH v8 19/29] KVM: arm64: Support SME priority registers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-19-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=5778; i=broonie@kernel.org; h=from:subject:message-id; bh=MAZZlV6eUub7krRK8EsIsDbLy5E7LRegIDDazNCSgqc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiIjqDWmUKV1ALhUPmh+OIgBl44LpEhQFqdF feXJKlkIfqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYiAAKCRAk1otyXVSH 0HlMB/0eZhL87ixROGalEh58FQLrmayJWWrGEC2AF222L8xuItLI2FbKCTZZLod2QMjR38IyqAx DctAIfO/OEv2o2l/HzhspOxCtZakpLB/Z5jTsnvSpt8RUxDoh+HHAJbYFOeytBPaKZaXiEIlSHf sVpiOKRcHRtTZq/kd64kWHRtKlHE4vVACzwGMK+SP38BH5CVpLer/TJawz5ZIe0GsWCV3OHvahi 0nr+VB3GEk3uEh65FSnT0KHmWjeBZG0WjSuv3G+dMfhaUjh2FgSy3nqrfbYtcbBB6WNgYqzCvXY rzB83ODSlxZ1n0Iaaxkn2Rp4Q0hlY2eaI26Nw1kObP5dVJ8Z X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME has optional support for configuring the relative priorities of PEs in systems where they share a single SME hardware block, known as a SMCU. Currently we do not have any support for this in Linux and will also hide it from KVM guests, pending experience with practical implementations. The interface for configuring priority support is via two new system registers, these registers are always defined when SME is available. The register SMPRI_EL1 allows control of SME execution priorities. Since we disable SME priority support for guests this register is RES0, define it as such and enable fine grained traps for SMPRI_EL1 to ensure that guests can't write to it even if the hardware supports priorites. Since the register should be readable with fixed contents we only trap writes, not reads. There is also an EL2 register SMPRIMAP_EL2 for virtualisation of priorities, this is RES0 when priority configuration is not supported but has no specific traps available. When saving state from a nested guest we overwite any value the guest stored. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/vncr_mapping.h | 1 + arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 7 +++++++ arch/arm64/kvm/sys_regs.c | 33 +++++++++++++++++++++++++++++++= +- 4 files changed, 41 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 5225dd9b752c..ebc516fe09db 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -548,6 +548,7 @@ enum vcpu_sysreg { VNCR(CPACR_EL1),/* Coprocessor Access Control */ VNCR(ZCR_EL1), /* SVE Control */ VNCR(SMCR_EL1), /* SME Control */ + VNCR(SMPRIMAP_EL2), /* Streaming Mode Priority Mapping Register */ VNCR(TTBR0_EL1),/* Translation Table Base Register 0 */ VNCR(TTBR1_EL1),/* Translation Table Base Register 1 */ VNCR(TCR_EL1), /* Translation Control Register */ diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm= /vncr_mapping.h index 5ab6a964bccf..670a2f81847c 100644 --- a/arch/arm64/include/asm/vncr_mapping.h +++ b/arch/arm64/include/asm/vncr_mapping.h @@ -45,6 +45,7 @@ #define VNCR_ZCR_EL1 0x1E0 #define VNCR_HAFGRTR_EL2 0x1E8 #define VNCR_SMCR_EL1 0x1F0 +#define VNCR_SMPRIMAP_EL2 0x1F0 #define VNCR_TTBR0_EL1 0x200 #define VNCR_TTBR1_EL1 0x210 #define VNCR_FAR_EL1 0x220 diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sy= sreg-sr.c index f28c6cf4fe1b..07aa4378c58a 100644 --- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c @@ -80,6 +80,13 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vc= pu) =20 if (ctxt_has_sctlr2(&vcpu->arch.ctxt)) __vcpu_assign_sys_reg(vcpu, SCTLR2_EL2, read_sysreg_el1(SYS_SCTLR2)); + + /* + * We block SME priorities so SMPRIMAP_EL2 is RES0, however we + * do not have traps to block access so the guest might have + * updated the state, overwrite anything there. + */ + __vcpu_assign_sys_reg(vcpu, SMPRIMAP_EL2, 0); } =20 static void __sysreg_restore_vel2_state(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 36ba1986e7a1..7284705a5f0a 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -552,6 +552,15 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu, return read_zero(vcpu, p); } =20 +static int set_res0(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd, + u64 val) +{ + if (val) + return -EINVAL; + + return 0; +} + /* * ARMv8.1 mandates at least a trivial LORegion implementation, where all = the * RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0 @@ -1819,6 +1828,15 @@ static unsigned int fp8_visibility(const struct kvm_= vcpu *vcpu, return REG_HIDDEN; } =20 +static unsigned int sme_raz_visibility(const struct kvm_vcpu *vcpu, + const struct sys_reg_desc *rd) +{ + if (vcpu_has_sme(vcpu)) + return REG_RAZ; + + return REG_HIDDEN; +} + static u64 sanitise_id_aa64pfr0_el1(const struct kvm_vcpu *vcpu, u64 val) { if (!vcpu_has_sve(vcpu)) @@ -3091,7 +3109,14 @@ static const struct sys_reg_desc sys_reg_descs[] =3D= { =20 { SYS_DESC(SYS_ZCR_EL1), NULL, reset_val, ZCR_EL1, 0, .visibility =3D sve= _visibility }, { SYS_DESC(SYS_TRFCR_EL1), undef_access }, - { SYS_DESC(SYS_SMPRI_EL1), undef_access }, + + /* + * SMPRI_EL1 is UNDEF when SME is disabled, the UNDEF is + * handled via FGU which is handled without consulting this + * table. + */ + { SYS_DESC(SYS_SMPRI_EL1), trap_raz_wi, .visibility =3D sme_raz_visibilit= y }, + { SYS_DESC(SYS_SMCR_EL1), NULL, reset_val, SMCR_EL1, 0, .visibility =3D s= me_visibility }, { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 }, { SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 }, @@ -3453,6 +3478,9 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { =20 EL2_REG_VNCR(HCRX_EL2, reset_val, 0), =20 + { SYS_DESC(SYS_SMPRIMAP_EL2), .reg =3D SMPRIMAP_EL2, + .access =3D trap_raz_wi, .set_user =3D set_res0, .reset =3D reset_val, + .val =3D 0, .visibility =3D sme_el2_visibility }, EL2_REG_FILTERED(SMCR_EL2, access_smcr_el2, reset_val, 0, sme_el2_visibility), =20 @@ -5379,6 +5407,9 @@ void kvm_calculate_traps(struct kvm_vcpu *vcpu) compute_fgu(kvm, HFGITR2_GROUP); compute_fgu(kvm, HDFGRTR2_GROUP); =20 + if (kvm_has_feat(kvm, ID_AA64PFR1_EL1, SME, IMP)) + kvm->arch.fgt[HFGWTR_GROUP] |=3D HFGWTR_EL2_nSMPRI_EL1_MASK; + set_bit(KVM_ARCH_FLAG_FGU_INITIALIZED, &kvm->arch.flags); out: mutex_unlock(&kvm->arch.config_lock); --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38B6C2F7452; Tue, 2 Sep 2025 11:45:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813551; cv=none; b=iB+wGk8QhfLRsCDjpy28esabWSRhH1xldqT3K6I09r8eOe2rkSguSdXyPBqlk/wcR1fdRRuNfv2oeXhEdOAMtH7in3+0n/LdtvW/QT00eisopftmC4vKjW3dBZ3mEcZnz+j3fU6Hb5wBvLTOdM3CUReKOiMeenu1w8A8c1s57KM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813551; c=relaxed/simple; bh=eko0M/1IoNzmkVmBa1f5aR4hGKuIltnrGwhCOTIwAnI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=b9TPRHx0As8PZNuo80Z7panmAYuTUYq1nhQIblWcvdGYoXHAee2ngMqmXWNYoBUXGo5NuhsD48nLh7naPAJKcnvfIV2i2ZdbwCCpLyxwSojLo6KmVpihfbhNxA4cgmDgjtG+vIAQI6c6z0ojOnjAyfeVygqSiN+SF8Gr9E5Nlyw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cBRYj6VW; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cBRYj6VW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4D7BAC4CEED; Tue, 2 Sep 2025 11:45:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813551; bh=eko0M/1IoNzmkVmBa1f5aR4hGKuIltnrGwhCOTIwAnI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=cBRYj6VWD8fF6HBWd1yUzFlLKMsLQBI5Io8P84mfdeJZCc6lGqJy4fg4mmzr5Pan4 BtS4ldqYhZvfjHjkAuGrne2k1pxExaAjn5mnOuVgZ/Rd3vhVQ57NXveRi3pbYo90kk FjtivsPIiKte5eSkbBoopdcEaxwSnnZVbPoMOrEog+FcQw849qJgjhr2RNvV1IQmKP 0dOopDnUXOY9x4v2zUklI0Hs9T6IUlLYs6FlFfZHtt5IfcaNwCGW6JjaLQ7rkWrQYA 9BolGYAjSoEmWnWg1dK6GX1GQCkFh9v7mLvyH4wHtYu1UUVf4Gzoy8qyqMyiWKx306 zi8BIGXb2rY6w== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:23 +0100 Subject: [PATCH v8 20/29] KVM: arm64: Provide assembly for SME register access Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-20-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=1851; i=broonie@kernel.org; h=from:subject:message-id; bh=eko0M/1IoNzmkVmBa1f5aR4hGKuIltnrGwhCOTIwAnI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiJvkHNyZ0bEhIAFv5mvwACf4J2Z1oPmYc3H v/gCxw87IuJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYiQAKCRAk1otyXVSH 0DI9CACGbb8IBp+/xZtmi1XfvQdy54WecZTRmvQ3x6QMt88KfTaWZ1z5+VpWKlnZEiN3ymhO6ps HF5b2q7mKKOAVLp3SZBIRjVhqRi3Vw//FHiecwMVQcOxKeA50zfbIc5lZPxS14YS1FpcP29wLuz dPjWkc5HogT99QehSooI1ZqQ3OEPVqjXvj/+ouKkv5PZrWUZKDSvW7q5wtgn7R9fSSSxF8V6hJ9 hO54p5nnoNhprkkLYK/kW/NaApddkZyppXqth3c07VhSo+gamKqJ0AyNVtbEYk1SNWjC9JM3sh0 AaFqC478LysO6ApnOhJGkfp/L1matQt757ssB73WwmWu3JoF X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Provide versions of the SME state save and restore functions for the hypervisor to allow it to restore ZA and ZT for guests. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_hyp.h | 3 +++ arch/arm64/kvm/hyp/fpsimd.S | 26 ++++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_= hyp.h index 0ad5a66e0d25..1c31d8b26aa9 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -115,6 +115,9 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_r= egs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); +int __sve_get_vl(void); +void __sme_save_state(void const *state, bool restore_zt); +void __sme_restore_state(void const *state, bool restore_zt); =20 u64 __guest_enter(struct kvm_vcpu *vcpu); =20 diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index 6e16cbfc5df2..44a1b0a483da 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -29,3 +29,29 @@ SYM_FUNC_START(__sve_save_state) sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) + +SYM_FUNC_START(__sve_get_vl) + _sve_rdvl 0, 1 + ret +SYM_FUNC_END(__sve_get_vl) + +SYM_FUNC_START(__sme_save_state) + _sme_rdsvl 2, 1 // x2 =3D VL/8 + sme_save_za 0, x2, 12 // Leaves x0 pointing to the end of ZA + + cbz x1, 1f + _str_zt 0 +1: + ret +SYM_FUNC_END(__sme_save_state) + +SYM_FUNC_START(__sme_restore_state) + _sme_rdsvl 2, 1 // x2 =3D VL/8 + sme_load_za 0, x2, 12 // Leaves x0 pointing to end of ZA + + cbz x1, 1f + _ldr_zt 0 + +1: + ret +SYM_FUNC_END(__sme_restore_state) --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E89E2FC874; Tue, 2 Sep 2025 11:45:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813555; cv=none; b=OgiKjz3dpou8YgctlRKe0CzJxJde9Pdh29rhncPhiZvnpZ4IoY3EnCqjtzJwwRwf3tjFpJkUrWSg9Yr7FUVhIbxXRLQ35THfEukB3GKCQAG3nEo4+sEUXuepRX0+OniG+dt45rnWfQswfN9sd8qXbf5/yHK0me7GGGhA5cFJLJQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813555; c=relaxed/simple; bh=00XRQFjc7Pzb86pKVTknY1kmQGmcMmO43+orrAnZ8z0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Hc3ivBCvp88WOVlB6ulvNV1oSsT22wKtOyQtCsnNIKMY8AZ2w8WgCMOpRfD6/HC/2eKsdDA62THRIhYuoNtpN7TmF0QWYN6cuU1fNaHJR/RmZUAkIQpl56T2VpxxipYU3+LR19Lrbf0sDSAayYnsMRx51kuiphvjBO6T1fbwP7Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Bc3K9+Fy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Bc3K9+Fy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 77FFDC4CEED; Tue, 2 Sep 2025 11:45:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813555; bh=00XRQFjc7Pzb86pKVTknY1kmQGmcMmO43+orrAnZ8z0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Bc3K9+Fy8PnmzG/5c1f8roAtthQakNkGu0n7K46iCpBr8qi1fuL5ShIFxbtrFmkZb DnQw4KkdcUIl+fKeGK/XT20uixb05sE8sySDll797XIGvU8pjgIW5kP+ZBde0DEnDb EO5GN1gaf/h8/CA/5GKGZrqR6KXEIY4ByvxIXUasSLqr3DM74ngUaj6pmNdrYHm5nN Y9Itpi/D6o1sw7esljE+Hs7LWdBjchmPnGY0BKhbulTe0ZCO7DpancSn4sIHLI/Yb1 tShYzo1VXUYoc0bNZXACksc7gn8wDeodpqH66iOizGgv9gWyqF9+LPYbLXXK18a0S0 Eg1H3pm/ZeNxw== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:24 +0100 Subject: [PATCH v8 21/29] KVM: arm64: Support userspace access to streaming mode Z and P registers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-21-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=4047; i=broonie@kernel.org; h=from:subject:message-id; bh=00XRQFjc7Pzb86pKVTknY1kmQGmcMmO43+orrAnZ8z0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiKrgNjNlnrrVsEXG1Jj2vvs1ae3X06tYQ09 nDAX7lLn6yJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYigAKCRAk1otyXVSH 0P3IB/oD2Yx6x2B28OALAcF4QgCyoYTG5z+lM940bUezYR1iqHzbfwEirXY3b9SQrlC9b4w72dQ +OGfu3iu6GCcggVpHW5tspk4frkaCHVSB+nddgxBQADfp7aOrvtG3IjfQCPj1WKgeOy24KBdRN+ StzFm0mfmv5G5a4jVXZEZdAgL0SisARDbkQfdBejtTlCZY3CkMf7w7ECbBZF4HHJnuKSu27lkSA ZMh2eT3Lpb2Fz59RLMcYhL0aODJ6PFCcsWF5tqKAVptaM3MaCU++Xm/LPdSaNGpBaQqgYCpCfrM SUCDnMCMgwzOWUqvHpRySY+OwSpm/3DMamXFcM8FiM5pudIb X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME introduces a mode called streaming mode where the Z, P and optionally FFR registers can be accessed using the SVE instructions but with the SME vector length. Reflect this in the ABI for accessing the guest registers by making the vector length for the vcpu reflect the vector length that would be seen by the guest were it running, using the SME vector length when the guest is configured for streaming mode. Since SME may be present without SVE we also update the existing checks for access to the Z, P and V registers to check for either SVE or streaming mode. When not in streaming mode the guest floating point state may be accessed via the V registers. Any VMM that supports SME must be aware of the need to configure streaming mode prior to writing the floating point registers that this creates. Signed-off-by: Mark Brown --- arch/arm64/kvm/guest.c | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index bcb0d565f1a4..013d66febf10 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -73,6 +73,11 @@ static u64 core_reg_offset_from_id(u64 id) return id & ~(KVM_REG_ARCH_MASK | KVM_REG_SIZE_MASK | KVM_REG_ARM_CORE); } =20 +static bool vcpu_has_sve_regs(const struct kvm_vcpu *vcpu) +{ + return vcpu_has_sve(vcpu) || vcpu_in_streaming_mode(vcpu); +} + static int core_reg_size_from_offset(const struct kvm_vcpu *vcpu, u64 off) { int size; @@ -110,9 +115,10 @@ static int core_reg_size_from_offset(const struct kvm_= vcpu *vcpu, u64 off) /* * The KVM_REG_ARM64_SVE regs must be used instead of * KVM_REG_ARM_CORE for accessing the FPSIMD V-registers on - * SVE-enabled vcpus: + * SVE-enabled vcpus or when a SME enabled vcpu is in + * streaming mode: */ - if (vcpu_has_sve(vcpu) && core_reg_offset_is_vreg(off)) + if (vcpu_has_sve_regs(vcpu) && core_reg_offset_is_vreg(off)) return -EINVAL; =20 return size; @@ -426,6 +432,24 @@ struct vec_state_reg_region { unsigned int upad; /* extra trailing padding in user memory */ }; =20 +/* + * We represent the Z and P registers to userspace using either the + * SVE or SME vector length, depending on which features the guest has + * and if the guest is in streaming mode. + */ +static unsigned int vcpu_sve_cur_vq(struct kvm_vcpu *vcpu) +{ + unsigned int vq =3D 0; + + if (vcpu_has_sve(vcpu)) + vq =3D vcpu_sve_max_vq(vcpu); + + if (vcpu_in_streaming_mode(vcpu)) + vq =3D vcpu_sme_max_vq(vcpu); + + return vq; +} + /* * Validate SVE register ID and get sanitised bounds for user/kernel SVE * register copy @@ -466,7 +490,7 @@ static int sve_reg_to_region(struct vec_state_reg_regio= n *region, if (!vcpu_has_sve(vcpu) || (reg->id & SVE_REG_SLICE_MASK) > 0) return -ENOENT; =20 - vq =3D vcpu_sve_max_vq(vcpu); + vq =3D vcpu_sve_cur_vq(vcpu); =20 reqoffset =3D SVE_SIG_ZREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET; @@ -476,7 +500,7 @@ static int sve_reg_to_region(struct vec_state_reg_regio= n *region, if (!vcpu_has_sve(vcpu) || (reg->id & SVE_REG_SLICE_MASK) > 0) return -ENOENT; =20 - vq =3D vcpu_sve_max_vq(vcpu); + vq =3D vcpu_sve_cur_vq(vcpu); =20 reqoffset =3D SVE_SIG_PREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET; @@ -515,6 +539,9 @@ static int get_sve_reg(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; =20 + if (!vcpu_has_sve_regs(vcpu)) + return -EBUSY; + if (copy_to_user(uptr, vcpu->arch.sve_state + region.koffset, region.klen) || clear_user(uptr + region.klen, region.upad)) @@ -541,6 +568,9 @@ static int set_sve_reg(struct kvm_vcpu *vcpu, const str= uct kvm_one_reg *reg) if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; =20 + if (!vcpu_has_sve_regs(vcpu)) + return -EBUSY; + if (copy_from_user(vcpu->arch.sve_state + region.koffset, uptr, region.klen)) return -EFAULT; --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6B642FCBE8; Tue, 2 Sep 2025 11:45:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813559; cv=none; b=MOGBoP0VePxeWJtvzJPLbEtSXH+2dcc4gpPDuO3rv5tMEZuDJJ4Nie4Exsr7BjEnFX0aqRW5HsfU50Y8c29e1+30lC3pw86Bu377mEh5myHrd2trmr1TrSMbn8aqFI9usd0B13mr9LH3w/9QkXq40+GCN0S1zbtPUDRVUXCo5EM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813559; c=relaxed/simple; bh=K/O9CttzW0zt3QhDrAZF6NQXIZmkZnmyhsSNH3lWRrU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OIouAYDA0Q6gYLHGy5uEV0Lwq4u4I7x299JAsXAgrPEVQeESG13Q2G2/ZepL4/jq4uXMF+vy5mHeSTKmwq943neaIdiTnnawqz+aXBGrcZx3wYqlou5ockeVsYxyH2qSg5QK3i6z94j2BJoblM/g9IWEN7TY/xz570ieS9FqMIg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cyysmErd; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cyysmErd" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F7FCC4CEED; Tue, 2 Sep 2025 11:45:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813559; bh=K/O9CttzW0zt3QhDrAZF6NQXIZmkZnmyhsSNH3lWRrU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=cyysmErdQnsGhAI7LauWgMOqAJlIKVs243APJWcei6qn/NMTkucSU8IVpqOFYPLll WvrIOjTPqL1dkjOmdw/EMjoRvW7D2/E7dFTBSyzOStBly12lZJQXLeZk1RynYZ1MUB 2E5b/JBEr4cxcXEgD5VIDYgwUgNAJgblDtQpQ3cngM1sw+aGY+HN++A8CxhVBe8eP8 j4z4oYffv1m9O8uJjJqnlhPMr8GhsIJ5S9+Z+qjYNyWf7+rUFi/Dm95F9tyFMMCh/z d2xWngqlXVNd/rSiU3fCpiuWBcsoVsOOjoEfYtpU6aJa12ixXxO3J761TW4CHxXmMV yWvF15t7+eopA== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:25 +0100 Subject: [PATCH v8 22/29] KVM: arm64: Flush register state on writes to SVCR.SM and SVCR.ZA Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-22-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=3621; i=broonie@kernel.org; h=from:subject:message-id; bh=K/O9CttzW0zt3QhDrAZF6NQXIZmkZnmyhsSNH3lWRrU=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiLOOREPgtGmGQzcmsCTjpuRPxvjG7kfaCVp 9oPVplO1wOJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYiwAKCRAk1otyXVSH 0AXUB/0TZZCGtlyShi6U3mavSZg5/Dy1snG0FfS5N64yz7AiGhEnPZd4q2TUg0snTlhMnBKdAxO nN1yTY/Iv5rfj+9F94WeqlDXXB3m4FKe1UUXQKgYxX6rtlc20n/36bJPjyZuUhUk7zeMEMkUeYS /NHfPoD9vbJZNJdneUhiXK1MobDbRx1nysapJ1IqohY46kK8M8FeIs7KR1MUre1GuUU3+tvxtF/ RvZ5AU44Kt59uGKDR8S48C6dO532CHgE6VMItwNC/TVWH+e542fGsBMbF9u3/Wdq9pELVTTgqvP AZudWOfPcg6zgSoomH3jBOHjcVZKN0pFg88PhK+z5eM5LPhw X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Writes to the physical SVCR.SM and SVCR.ZA change the state of PSTATE.SM and PSTATE.ZA, causing other floating point state to reset. Emulate this behaviour for writes done via the KVM userspace ABI. Setting PSTATE.ZA to 1 causes ZA and ZT0 to be reset to 0, these are stored in sme_state. Setting PSTATE.ZA to 0 causes ZA and ZT0 to become inaccesible so no reset is needed. Any change in PSTATE.SM causes the V, Z, P, FFR and FPMR registers to be reset to 0 and FPSR to be reset to 0x800009f. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 24 ++++++++++++++++++++++++ arch/arm64/kvm/sys_regs.c | 29 ++++++++++++++++++++++++++++- 2 files changed, 52 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index ebc516fe09db..8e5c16372b83 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1144,6 +1144,30 @@ struct kvm_vcpu_arch { =20 #define vcpu_sve_state_size(vcpu) sve_state_size_from_vl((vcpu)->arch.max_= vl[ARM64_VEC_SVE]) =20 +#define vcpu_sme_state(vcpu) (kern_hyp_va((vcpu)->arch.sme_state)) + +#define sme_state_size_from_vl(vl, sme2) ({ \ + size_t __size_ret; \ + unsigned int __vq; \ + \ + if (WARN_ON(!sve_vl_valid(vl))) { \ + __size_ret =3D 0; \ + } else { \ + __vq =3D sve_vq_from_vl(vl); \ + __size_ret =3D ZA_SIG_REGS_SIZE(__vq); \ + if (sme2) \ + __size_ret +=3D ZT_SIG_REG_SIZE; \ + } \ + \ + __size_ret; \ +}) + +#define vcpu_sme_state_size(vcpu) ({ \ + unsigned long __vl; \ + __vl =3D (vcpu)->arch.max_vl[ARM64_VEC_SME]; \ + sme_state_size_from_vl(__vl, vcpu_has_sme2(vcpu)); \ +}) + /* * Only use __vcpu_sys_reg/ctxt_sys_reg if you know you want the * memory backed version of a register, and not the one most recently diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 7284705a5f0a..0752e119c056 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -788,6 +788,33 @@ static unsigned int hidden_visibility(const struct kvm= _vcpu *vcpu, return REG_HIDDEN; } =20 +static int set_svcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd, + u64 val) +{ + u64 old =3D __vcpu_sys_reg(vcpu, rd->reg); + + if (val & SVCR_RES0) + return -EINVAL; + + if ((val & SVCR_ZA) && !(old & SVCR_ZA) && vcpu->arch.sme_state) + memset(vcpu->arch.sme_state, 0, vcpu_sme_state_size(vcpu)); + + if ((val & SVCR_SM) !=3D (old & SVCR_SM)) { + memset(vcpu->arch.ctxt.fp_regs.vregs, 0, + sizeof(vcpu->arch.ctxt.fp_regs.vregs)); + + if (vcpu->arch.sve_state) + memset(vcpu->arch.sve_state, 0, + vcpu_sve_state_size(vcpu)); + + __vcpu_assign_sys_reg(vcpu, FPMR, 0); + vcpu->arch.ctxt.fp_regs.fpsr =3D 0x800009f; + } + + __vcpu_assign_sys_reg(vcpu, rd->reg, val); + return 0; +} + static unsigned int pmu_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { @@ -3248,7 +3275,7 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { CTR_EL0_DminLine_MASK | CTR_EL0_L1Ip_MASK | CTR_EL0_IminLine_MASK), - { SYS_DESC(SYS_SVCR), undef_access, reset_val, SVCR, 0, .visibility =3D s= me_visibility }, + { SYS_DESC(SYS_SVCR), undef_access, reset_val, SVCR, 0, .visibility =3D s= me_visibility, .set_user =3D set_svcr }, { SYS_DESC(SYS_FPMR), undef_access, reset_val, FPMR, 0, .visibility =3D f= p8_visibility }, =20 { PMU_SYS_REG(PMCR_EL0), .access =3D access_pmcr, .reset =3D reset_pmcr, --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16214308F07; Tue, 2 Sep 2025 11:46:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813564; cv=none; b=TVkppzZNBQuvwFx4hL/Nt80uNykhh3unCvIneHePLZyGJHcMxBs1+OA+9Bz34G8Rk3AdhVsY9/+46EmY9BUpgtY04lZO5qAT2IMBeCOMQ3Wc21gIM0iylTRGDJoRxPgTSEXWo+d5Dr2h8+lj6f8fi+DGXucKjo51fZii8pThrNc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813564; c=relaxed/simple; bh=n9Gv38zt8zDnvl+mS5Ep0oghSoUg5vKLb32ofFnUTn0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HSqtpiQkOMiot8YvqA7OBIH5nAZhc8DTPJOUfRy42CQgHUI6610a7E6T5GezZ8gsuqYSYlhzsNid1hPnKGX2bHw15eBzc+P4CP0OdVhO3qy/GrFP0kF0Vh55MsM2NZDsdgklqmSGzgjkLnncUDpiQhaV2CCiBEuxLAtBlhTOfgI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ofeJM14g; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ofeJM14g" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C5A38C4CEF5; Tue, 2 Sep 2025 11:45:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813563; bh=n9Gv38zt8zDnvl+mS5Ep0oghSoUg5vKLb32ofFnUTn0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ofeJM14gdgDf2pe6FEgJB8hmzU6BmyNYQDN1Nr9Rg2dbfrv5Q6BOXnVIHGHXUSEsf IAgkRaCwwE7TQvVeIFyTt0SKKA7rMQ06NHfnZls0WucwaggIjUyitft2vO7OwnVkXE MvY9xjdn+J8H5FozY0R5Wq78VtAjnS79BVixngT+kUOb4EEBbtu79jMoCG6oTynGAH p/k1jtl939ni1Sw6HSL3O7JNfGtmttJj+ZoTFr0c6U7TezAtssq5tgjGoMHAhLsq6k UPviUrekGduQVfPBVkAQqa7goQdQ5j3hGFRwQsrvYC6yPTdR7vqZJqFuCnzqTzoOts IkOe2U4D8NaNw== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:26 +0100 Subject: [PATCH v8 23/29] KVM: arm64: Expose SME specific state to userspace Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-23-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=6845; i=broonie@kernel.org; h=from:subject:message-id; bh=n9Gv38zt8zDnvl+mS5Ep0oghSoUg5vKLb32ofFnUTn0=; b=owGbwMvMwMWocq27KDak/QLjabUkhoxtN7qfKlvu3CrWp2NXd7vkh01kY77Z+egYzTjO2ZGeu Q0ia006GY1ZGBi5GGTFFFnWPstYlR4usXX+o/mvYAaxMoFMYeDiFICJJG5l/2fy4e4bqUKBmRIv 1C1n3XrmXf2+wFpfJGeZ1yJ+06dXTR0L/0708/b9uyii1YixvEX7tf9V+cKT98yvn+Ro2ha3z9B 94fYnP7uE47vYMxd2tvTUmzg51ARctNfJzFlu0VymxBL68rDr3kVfkx/bnV+/Wzrc8tN0/vm7wy qs3tvcXfro/YVVsjESH0ze3U6d8+iSY+ip8tWdb0r1f/yRWawT8tvy6ZoS2RLVksbWlsqgpWnXf vw0m6z+SlkyP/C9Q9w7gZIm851iRu3WTrcmTPkxYc3ZgLkext+ndZzaJeW3rZj73z310jnXDteo R7Ubzzi419Aho7XDTy+EY06JoX1JblbcVjXRcMfFHIKvAQ== X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME introduces two new registers, the ZA matrix register and the ZT0 LUT register. Both of these registers are only accessible when PSTATE.ZA is set and ZT0 is only present if SME2 is enabled for the guest. Provide support for configuring these from VMMs. The ZA matrix is a single SVL*SVL register which is available when PSTATE.ZA is set. We follow the pattern established by the architecture itself and expose this to userspace as a series of horizontal SVE vectors with the streaming mode vector length, using the format already established for the SVE vectors themselves. ZT0 is a single register with a refreshingly fixed size 512 bit register which is like ZA accessible only when PSTATE.ZA is set. Add support for it to the userspace API, as with ZA we allow the register to be read or written regardless of the state of PSTATE.ZA in order to simplify userspace usage. The value will be reset to 0 whenever PSTATE.ZA changes from 0 to 1, userspace can read stale values but these are not observable by the guest without manipulation of PSTATE.ZA by userspace. While there is currently only one ZT register the naming as ZT0 and the instruction encoding clearly leave room for future extensions adding more ZT registers. This encoding can readily support such an extension if one is introduced. Signed-off-by: Mark Brown --- arch/arm64/include/uapi/asm/kvm.h | 17 ++++++ arch/arm64/kvm/guest.c | 114 ++++++++++++++++++++++++++++++++++= +++- 2 files changed, 129 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/as= m/kvm.h index 5220797361e0..cf75a830f17a 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -357,6 +357,23 @@ struct kvm_arm_counter_offset { /* SME registers */ #define KVM_REG_ARM64_SME (0x17 << KVM_REG_ARM_COPROC_SHIFT) =20 +#define KVM_ARM64_SME_VQ_MIN __SVE_VQ_MIN +#define KVM_ARM64_SME_VQ_MAX __SVE_VQ_MAX + +/* ZA and ZTn occupy blocks at the following offsets within this range: */ +#define KVM_REG_ARM64_SME_ZA_BASE 0 +#define KVM_REG_ARM64_SME_ZT_BASE 0x600 + +#define KVM_ARM64_SME_MAX_ZAHREG (__SVE_VQ_BYTES * KVM_ARM64_SME_VQ_MAX) + +#define KVM_REG_ARM64_SME_ZAHREG(n, i) \ + (KVM_REG_ARM64 | KVM_REG_ARM64_SME | KVM_REG_ARM64_SME_ZA_BASE | \ + KVM_REG_SIZE_U2048 | \ + (((n) & (KVM_ARM64_SME_MAX_ZAHREG - 1)) << 5) | \ + ((i) & (KVM_ARM64_SVE_MAX_SLICES - 1))) + +#define KVM_REG_ARM64_SME_ZTREG_SIZE (512 / 8) + /* Vector lengths pseudo-register: */ #define KVM_REG_ARM64_SME_VLS (KVM_REG_ARM64 | KVM_REG_ARM64_SME | \ KVM_REG_SIZE_U512 | 0xffff) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 013d66febf10..734120a76623 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -594,23 +594,133 @@ static int set_sme_vls(struct kvm_vcpu *vcpu, const = struct kvm_one_reg *reg) return set_vec_vls(ARM64_VEC_SME, vcpu, reg); } =20 +/* + * Validate SVE register ID and get sanitised bounds for user/kernel SVE + * register copy + */ +static int sme_reg_to_region(struct vec_state_reg_region *region, + struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg) +{ + /* reg ID ranges for ZA.H[n] registers */ + unsigned int vq =3D vcpu_sme_max_vq(vcpu) - 1; + const u64 za_h_max =3D vq * __SVE_VQ_BYTES; + const u64 zah_id_min =3D KVM_REG_ARM64_SME_ZAHREG(0, 0); + const u64 zah_id_max =3D KVM_REG_ARM64_SME_ZAHREG(za_h_max - 1, + SVE_NUM_SLICES - 1); + unsigned int reg_num; + + unsigned int reqoffset, reqlen; /* User-requested offset and length */ + unsigned int maxlen; /* Maximum permitted length */ + + size_t sme_state_size; + + reg_num =3D (reg->id & SVE_REG_ID_MASK) >> SVE_REG_ID_SHIFT; + + if (reg->id >=3D zah_id_min && reg->id <=3D zah_id_max) { + if (!vcpu_has_sme(vcpu) || (reg->id & SVE_REG_SLICE_MASK) > 0) + return -ENOENT; + + /* ZA is exposed as SVE vectors ZA.H[n] */ + reqoffset =3D ZA_SIG_ZAV_OFFSET(vq, reg_num) - + ZA_SIG_REGS_OFFSET; + reqlen =3D KVM_SVE_ZREG_SIZE; + maxlen =3D SVE_SIG_ZREG_SIZE(vq); + } else if (reg->id =3D=3D KVM_REG_ARM64_SME_ZT_BASE) { + /* ZA is exposed as SVE vectors ZA.H[n] */ + if (!kvm_has_feat(vcpu->kvm, ID_AA64PFR1_EL1, SME, SME2) || + (reg->id & SVE_REG_SLICE_MASK) > 0 || + reg_num > 0) + return -ENOENT; + + /* ZT0 is stored after ZA */ + reqlen =3D KVM_REG_ARM64_SME_ZTREG_SIZE; + maxlen =3D KVM_REG_ARM64_SME_ZTREG_SIZE; + } else { + return -EINVAL; + } + + sme_state_size =3D vcpu_sme_state_size(vcpu); + if (WARN_ON(!sme_state_size)) + return -EINVAL; + + region->koffset =3D array_index_nospec(reqoffset, sme_state_size); + region->klen =3D min(maxlen, reqlen); + region->upad =3D reqlen - region->klen; + + return 0; +} + +/* + * ZA is exposed as an array of horizontal vectors with the same + * format as SVE, mirroring the architecture's LDR ZA[Wv, offs], [Xn] + * instruction. + */ + static int get_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) { + int ret; + struct vec_state_reg_region region; + char __user *uptr =3D (char __user *)reg->addr; + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ if (reg->id =3D=3D KVM_REG_ARM64_SME_VLS) return get_sme_vls(vcpu, reg); =20 - return -EINVAL; + /* Try to interpret reg ID as an architectural SME register... */ + ret =3D sme_reg_to_region(®ion, vcpu, reg); + if (ret) + return ret; + + if (!kvm_arm_vcpu_vec_finalized(vcpu)) + return -EPERM; + + /* + * None of the SME specific registers are accessible unless + * PSTATE.ZA is set. + */ + if (!vcpu_za_enabled(vcpu)) + return -EINVAL; + + if (copy_from_user(vcpu->arch.sme_state + region.koffset, uptr, + region.klen)) + return -EFAULT; + + return 0; } =20 static int set_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) { + int ret; + struct vec_state_reg_region region; + char __user *uptr =3D (char __user *)reg->addr; + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ if (reg->id =3D=3D KVM_REG_ARM64_SME_VLS) return set_sme_vls(vcpu, reg); =20 - return -EINVAL; + /* Try to interpret reg ID as an architectural SME register... */ + ret =3D sme_reg_to_region(®ion, vcpu, reg); + if (ret) + return ret; + + if (!kvm_arm_vcpu_vec_finalized(vcpu)) + return -EPERM; + + /* + * None of the SME specific registers are accessible unless + * PSTATE.ZA is set. + */ + if (!vcpu_za_enabled(vcpu)) + return -EINVAL; + + if (copy_from_user(vcpu->arch.sme_state + region.koffset, uptr, + region.klen)) + return -EFAULT; + + return 0; } + int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *r= egs) { return -EINVAL; --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 642A93093B8; Tue, 2 Sep 2025 11:46:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813568; cv=none; b=NfWbLaBycmDtdV2RYSdRaPe0FaLWGiVGOYmrYusuQI+DFfSb35glQusEKO/qxTYZ90VvDzn9L17E06TOnVF7C37HzQOqFy97CTr2FoLPMC+d2QDWuRDWS3odVL5U0anYaK7mMWS+bHx2BYRqQc9KAzk3qJ/E9IXHb/wUQnLHXVc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813568; c=relaxed/simple; bh=uilO8cZAVKYy6IefMKWSiW3palj0kkYE94hzaj93Noc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aeQu48CI4VHQflVI6X3rFcN21VvycNMQF3NbtGMmyGtmrw4xcQQu0dm989la3JAvgZmj3Z/RBSXqzLbTcDsL2fHXwmkPw/hFiJTy13NonVvouEw0J/YWIy7iTITD2Y68PzCg7m8FUTeyCpOGLJFxcVUwGVu8q75oFDY71JOYy4A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NtoB+9Jp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NtoB+9Jp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 09093C4CEF7; Tue, 2 Sep 2025 11:46:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813567; bh=uilO8cZAVKYy6IefMKWSiW3palj0kkYE94hzaj93Noc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=NtoB+9JpmYN9Ngh8KRDnDEekwkmQjtPweOm84GIUVdfkMTY45kNYETkIgrqJRAVMP P9AeEeathG8tKnxc5tlTOCoWR4mMYflNMHhHONZiz/8FscYssrgtSdCzEMTkE7GS0b ZeLh7ET5gBFSbi6sXdUYkLhyP7lXPxXByJE5d6xuJs5puZckHTNy9PlMztuKn9zbQ8 0rtbU0bqKhPoqRqBu3zs/+jxADqCwN/A+E38GN9PgKTq8HDYrkittSG+8vpTVmUkvb RqYLoqgcpeBLqfjVvNDliukHeT0LKqcD07HG6lrUWh1FfuAs0xusP0FiPudDrw4n/O p0TPwXVNMc8MQ== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:27 +0100 Subject: [PATCH v8 24/29] KVM: arm64: Context switch SME state for guests Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-24-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=18718; i=broonie@kernel.org; h=from:subject:message-id; bh=uilO8cZAVKYy6IefMKWSiW3palj0kkYE94hzaj93Noc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiMgJfPAWP7CVFOXjJg5jZfXY7dl9zsbkBpM FgDEqcfVdqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYjAAKCRAk1otyXVSH 0KxHB/0fRhiJdF2QK7j0bcwgjvP1CYs2dwSRPs4RKa7zsfNowwd0LlIr/KS6qNhouJg6k14oBO7 sRHi3YfLtai+kECD10461Bh6t/Gjech1s5RXNREeJgpCgPixqfDEOhauhMBpXI9J85bfKqFwGjz zLtAMNyuvGqp5c4RMQfgmEQnPxnF3erY3GmHqqP5xBFfGWwaDJj0ZWXKYZkptaTH2BH8iJsTqyZ Kfyved77W80FNN2VQah74RtSfnmPqj1Wok/ew//C4JxdAiRdtZsVzU0JRJOHW6qsAW8C4v/70Ef mjF+KJLrxA7Zb+t29rtsHyqUHYNTQFYd7G0a+SLJWMupBlaw X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB If the guest has SME state we need to context switch that state, provide support for that for normal guests. SME has three sets of registers, ZA, ZT (only present for SME2) and also streaming SVE which replaces the standard floating point registers when active. The first two are fairly straightforward, they are accessible only when PSTATE.ZA is set and we can reuse the assembly from the host to save and load them from a single contiguous buffer. When PSTATE.ZA is not set then these registers are inaccessible, when the guest enables PSTATE.ZA all bits will be set to 0 by that and nothing is required on restore. Streaming mode is slightly more complicated, when enabled via PSTATE.SM it provides a version of the SVE registers using the SME vector length and may optionally omit the FFR register. SME may also be present without SVE. The register state is stored in sve_state as for non-streaming SVE mode, we make an initial selection of registers to update based on the guest SVE support and then override this when loading SVCR if streaming mode is enabled. A further complication is that when the hardware is in streaming mode guest operations that are invalid in in streaming mode will generate SME exceptions. There are also subfeature exceptions for SME2 controlled via SMCR which generate distinct exception codes. In many situations these exceptions are routed directly to the lower ELs with no opportunity for the hypervisor to intercept. So that guests do not see unexpected exception types due to the actual hardware configuration not being what the guest configured we update the SMCRs and SVCR even if the guest does not own the registers. Since in order to avoid duplication with SME we now restore the register state outside of the SVE specific restore function we need to move the restore of the effective VL for nested guests to a separate restore function run after loading the floating point register state, along with the similar handling required for SME. The selection of which vector length to use is handled by vcpu_sve_pffr(). Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 10 +++ arch/arm64/include/asm/kvm_emulate.h | 6 ++ arch/arm64/include/asm/kvm_host.h | 4 + arch/arm64/kvm/fpsimd.c | 25 ++++-- arch/arm64/kvm/hyp/include/hyp/switch.h | 151 ++++++++++++++++++++++++++++= ++-- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 80 +++++++++++++++-- 6 files changed, 255 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsim= d.h index 38c24c6485ad..40b56fba9c54 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -442,6 +442,15 @@ static inline size_t sme_state_size(struct task_struct= const *task) write_sysreg_s(__new, (reg)); \ } while (0) =20 +#define sme_cond_update_smcr_vq(val, reg) \ + do { \ + u64 __smcr =3D read_sysreg_s((reg)); \ + u64 __new =3D __smcr & ~SMCR_ELx_LEN_MASK; \ + __new |=3D (val) & SMCR_ELx_LEN_MASK; \ + if (__smcr !=3D __new) \ + write_sysreg_s(__new, (reg)); \ + } while (0) + #else =20 static inline void sme_user_disable(void) { BUILD_BUG(); } @@ -471,6 +480,7 @@ static inline size_t sme_state_size(struct task_struct = const *task) } =20 #define sme_cond_update_smcr(val, fa64, zt0, reg) do { } while (0) +#define sme_cond_update_smcr_vq(val, reg) do { } while (0) =20 #endif /* ! CONFIG_ARM64_SME */ =20 diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/= kvm_emulate.h index fa8a08a1ccd5..d71ccea726f2 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -674,4 +674,10 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu) vcpu->arch.hcrx_el2 |=3D HCRX_EL2_SCTLR2En; } } + +static inline bool guest_hyp_sme_traps_enabled(const struct kvm_vcpu *vcpu) +{ + return __guest_hyp_cptr_xen_trap_enabled(vcpu, SMEN); +} + #endif /* __ARM64_KVM_EMULATE_H__ */ diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 8e5c16372b83..fa646da80e3e 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -738,6 +738,7 @@ struct kvm_host_data { =20 /* Used by pKVM only. */ u64 fpmr; + u64 smcr_el1; =20 /* Ownership of the FP regs */ enum { @@ -1128,6 +1129,9 @@ struct kvm_vcpu_arch { #define vcpu_sve_zcr_elx(vcpu) \ (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) =20 +#define vcpu_sme_smcr_elx(vcpu) \ + (unlikely(is_hyp_ctxt(vcpu)) ? SMCR_EL2 : SMCR_EL1) + #define sve_state_size_from_vl(sve_max_vl) ({ \ size_t __size_ret; \ unsigned int __vq; \ diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 1f4fcc8b5554..8fb8c55e50b3 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -69,19 +69,25 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) WARN_ON_ONCE(!irqs_disabled()); =20 if (guest_owns_fp_regs()) { - /* - * Currently we do not support SME guests so SVCR is - * always 0 and we just need a variable to point to. - */ fp_state.st =3D &vcpu->arch.ctxt.fp_regs; fp_state.sve_state =3D vcpu->arch.sve_state; fp_state.sve_vl =3D vcpu->arch.max_vl[ARM64_VEC_SVE]; - fp_state.sme_state =3D NULL; + fp_state.sme_state =3D vcpu->arch.sme_state; + fp_state.sme_vl =3D vcpu->arch.max_vl[ARM64_VEC_SME]; fp_state.svcr =3D __ctxt_sys_reg(&vcpu->arch.ctxt, SVCR); fp_state.fpmr =3D __ctxt_sys_reg(&vcpu->arch.ctxt, FPMR); fp_state.fp_type =3D &vcpu->arch.fp_type; + fp_state.sme_features =3D 0; + if (kvm_has_fa64(vcpu->kvm)) + fp_state.sme_features |=3D SMCR_ELx_FA64; + if (kvm_has_sme2(vcpu->kvm)) + fp_state.sme_features |=3D SMCR_ELx_EZT0; =20 + /* + * For SME only hosts fpsimd_save() will override the + * state selection if we are in streaming mode. + */ if (vcpu_has_sve(vcpu)) fp_state.to_save =3D FP_STATE_SVE; else @@ -90,6 +96,15 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) fpsimd_bind_state_to_cpu(&fp_state); =20 clear_thread_flag(TIF_FOREIGN_FPSTATE); + } else { + /* + * We might have enabled SME to configure traps but + * insist the host doesn't run the hypervisor with SME + * enabled, ensure it's disabled again. + */ + if (system_supports_sme()) { + sme_smstop(); + } } } =20 diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/i= nclude/hyp/switch.h index cd76129c6630..c128b4d25a2d 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -551,6 +551,29 @@ static inline bool kvm_hyp_handle_mops(struct kvm_vcpu= *vcpu, u64 *exit_code) return true; } =20 +static inline void __hyp_sme_restore_guest(struct kvm_vcpu *vcpu, + bool *restore_sve, + bool *restore_ffr) +{ + bool has_fa64 =3D vcpu_has_fa64(vcpu); + bool has_sme2 =3D vcpu_has_sme2(vcpu); + + sme_cond_update_smcr(vcpu_sme_max_vq(vcpu) - 1, has_fa64, has_sme2, + SYS_SMCR_EL2); + + write_sysreg_el1(__vcpu_sys_reg(vcpu, SMCR_EL1), SYS_SMCR); + + write_sysreg_s(__vcpu_sys_reg(vcpu, SVCR), SYS_SVCR); + + if (vcpu_in_streaming_mode(vcpu)) { + *restore_sve =3D true; + *restore_ffr =3D has_fa64; + } + + if (vcpu_za_enabled(vcpu)) + __sme_restore_state(vcpu_sme_state(vcpu), has_sme2); +} + static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) { /* @@ -558,19 +581,25 @@ static inline void __hyp_sve_restore_guest(struct kvm= _vcpu *vcpu) * vCPU. Start off with the max VL so we can load the SVE state. */ sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); - __sve_restore_state(vcpu_sve_pffr(vcpu), - &vcpu->arch.ctxt.fp_regs.fpsr, - true); =20 + write_sysreg_el1(__vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)), SYS_ZCR); +} + +static inline void __hyp_nv_restore_guest_vls(struct kvm_vcpu *vcpu) +{ /* * The effective VL for a VM could differ from the max VL when running a * nested guest, as the guest hypervisor could select a smaller VL. Slap * that into hardware before wrapping up. */ - if (is_nested_ctxt(vcpu)) + if (!is_nested_ctxt(vcpu)) + return; + + if (vcpu_has_sve(vcpu)) sve_cond_update_zcr_vq(__vcpu_sys_reg(vcpu, ZCR_EL2), SYS_ZCR_EL2); =20 - write_sysreg_el1(__vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)), SYS_ZCR); + if (vcpu_has_sme(vcpu)) + sme_cond_update_smcr_vq(__vcpu_sys_reg(vcpu, SMCR_EL2), SYS_SMCR_EL2); } =20 static inline void __hyp_sve_save_host(void) @@ -584,10 +613,40 @@ static inline void __hyp_sve_save_host(void) true); } =20 +static inline void kvm_sme_configure_traps(struct kvm_vcpu *vcpu) +{ + u64 smcr_el1, smcr_el2; + u64 svcr; + + if (!vcpu_has_sme(vcpu)) + return; + + /* A guest hypervisor may restrict the effective max VL. */ + if (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu)) + smcr_el2 =3D __vcpu_sys_reg(vcpu, SMCR_EL2); + else + smcr_el2 =3D vcpu_sme_max_vq(vcpu) - 1; + + if (vcpu_has_fa64(vcpu)) + smcr_el2 |=3D SMCR_ELx_FA64; + if (vcpu_has_sme2(vcpu)) + smcr_el2 |=3D SMCR_ELx_EZT0; + + write_sysreg_el2(smcr_el2, SYS_SMCR); + + smcr_el1 =3D __vcpu_sys_reg(vcpu, vcpu_sme_smcr_elx(vcpu)); + write_sysreg_el1(smcr_el1, SYS_SMCR); + + svcr =3D __vcpu_sys_reg(vcpu, SVCR); + write_sysreg_s(svcr, SYS_SVCR); +} + static inline void fpsimd_lazy_switch_to_guest(struct kvm_vcpu *vcpu) { u64 zcr_el1, zcr_el2; =20 + kvm_sme_configure_traps(vcpu); + if (!guest_owns_fp_regs()) return; =20 @@ -607,8 +666,50 @@ static inline void fpsimd_lazy_switch_to_guest(struct = kvm_vcpu *vcpu) =20 static inline void fpsimd_lazy_switch_to_host(struct kvm_vcpu *vcpu) { + u64 smcr_el1, smcr_el2; u64 zcr_el1, zcr_el2; =20 + if (vcpu_has_sme(vcpu)) { + /* + * __deactivate_cptr_traps( disabled traps), but there + * hasn't necessarily been a context synchronization + * event yet. + */ + isb(); + + smcr_el1 =3D read_sysreg_el1(SYS_SMCR); + __vcpu_assign_sys_reg(vcpu, vcpu_sme_smcr_elx(vcpu), smcr_el1); + + smcr_el2 =3D 0; + if (system_supports_fa64()) + smcr_el2 |=3D SMCR_ELx_FA64; + if (system_supports_sme2()) + smcr_el2 |=3D SMCR_ELx_EZT0; + + /* + * The guest's state is always saved using the guest's max VL. + * Ensure that the host has the guest's max VL active such that + * the host can save the guest's state lazily, but don't + * artificially restrict the host to the guest's max VL. + */ + if (has_vhe()) { + smcr_el2 |=3D vcpu_sme_max_vq(vcpu) - 1; + write_sysreg_el2(smcr_el2, SYS_SMCR); + } else { + smcr_el1 =3D smcr_el2; + smcr_el2 |=3D sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SME]) - 1; + write_sysreg_el2(smcr_el2, SYS_SMCR); + + smcr_el1 |=3D vcpu_sve_max_vq(vcpu) - 1; + write_sysreg_el1(smcr_el1, SYS_SMCR); + } + + if (guest_owns_fp_regs()) { + u64 svcr =3D read_sysreg_s(SYS_SVCR); + __vcpu_assign_sys_reg(vcpu, SVCR, svcr); + } + } + if (!guest_owns_fp_regs()) return; =20 @@ -645,6 +746,16 @@ static inline void fpsimd_lazy_switch_to_host(struct k= vm_vcpu *vcpu) =20 static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu) { + /* + * The hypervisor refuses to run if streaming mode or ZA is + * enabled, we only need to save SMCR_EL1 for SME. For pKVM + * we will restore this, reset SMCR_EL2 to a fixed value and + * disable streaming mode and ZA to avoid any state being + * leaked. + */ + if (system_supports_sme()) + *host_data_ptr(smcr_el1) =3D read_sysreg_el1(SYS_SMCR); + /* * Non-protected kvm relies on the host restoring its sve state. * Protected kvm restores the host's sve state as not to reveal that @@ -669,14 +780,17 @@ static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu = *vcpu) */ static inline bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_= code) { - bool sve_guest; - u8 esr_ec; + bool restore_sve, restore_ffr; + bool sve_guest, sme_guest; + u8 esr_ec, esr_iss_smtc; =20 if (!system_supports_fpsimd()) return false; =20 sve_guest =3D vcpu_has_sve(vcpu); + sme_guest =3D vcpu_has_sme(vcpu); esr_ec =3D kvm_vcpu_trap_get_class(vcpu); + esr_iss_smtc =3D ESR_ELx_SME_ISS_SMTC((kvm_vcpu_get_esr(vcpu))); =20 /* Only handle traps the vCPU can support here: */ switch (esr_ec) { @@ -695,6 +809,15 @@ static inline bool kvm_hyp_handle_fpsimd(struct kvm_vc= pu *vcpu, u64 *exit_code) if (guest_hyp_sve_traps_enabled(vcpu)) return false; break; + case ESR_ELx_EC_SME: + if (!sme_guest) + return false; + if (guest_hyp_sme_traps_enabled(vcpu)) + return false; + if (!kvm_has_sme2(vcpu->kvm) && + (esr_iss_smtc =3D=3D ESR_ELx_SME_ISS_SMTC_ZT_DISABLED)) + return false; + break; default: return false; } @@ -710,8 +833,20 @@ static inline bool kvm_hyp_handle_fpsimd(struct kvm_vc= pu *vcpu, u64 *exit_code) kvm_hyp_save_fpsimd_host(vcpu); =20 /* Restore the guest state */ + + /* These may be overridden for a SME guest */ + restore_sve =3D sve_guest; + restore_ffr =3D sve_guest; + if (sve_guest) __hyp_sve_restore_guest(vcpu); + if (sme_guest) + __hyp_sme_restore_guest(vcpu, &restore_sve, &restore_ffr); + + if (restore_sve) + __sve_restore_state(vcpu_sve_pffr(vcpu), + &vcpu->arch.ctxt.fp_regs.fpsr, + restore_ffr); else __fpsimd_restore_state(&vcpu->arch.ctxt.fp_regs); =20 @@ -722,6 +857,8 @@ static inline bool kvm_hyp_handle_fpsimd(struct kvm_vcp= u *vcpu, u64 *exit_code) if (!(read_sysreg(hcr_el2) & HCR_RW)) write_sysreg(__vcpu_sys_reg(vcpu, FPEXC32_EL2), fpexc32_el2); =20 + __hyp_nv_restore_guest_vls(vcpu); + *host_data_ptr(fp_owner) =3D FP_STATE_GUEST_OWNED; =20 /* diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h= yp-main.c index 76be13efcfcb..ccbe0389c0b7 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -26,14 +26,17 @@ void __kvm_hyp_host_forward_smc(struct kvm_cpu_context = *host_ctxt); =20 static void __hyp_sve_save_guest(struct kvm_vcpu *vcpu) { + bool save_ffr =3D !vcpu_in_streaming_mode(vcpu) || vcpu_has_fa64(vcpu); + __vcpu_assign_sys_reg(vcpu, ZCR_EL1, read_sysreg_el1(SYS_ZCR)); + /* * On saving/restoring guest sve state, always use the maximum VL for * the guest. The layout of the data when saving the sve state depends * on the VL, so use a consistent (i.e., the maximum) guest VL. */ sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); - __sve_save_state(vcpu_sve_pffr(vcpu), &vcpu->arch.ctxt.fp_regs.fpsr, true= ); + __sve_save_state(vcpu_sve_pffr(vcpu), &vcpu->arch.ctxt.fp_regs.fpsr, save= _ffr); write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZC= R_EL2); } =20 @@ -57,9 +60,63 @@ static void __hyp_sve_restore_host(void) write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); } =20 -static void fpsimd_sve_flush(void) +static void __hyp_sme_save_guest(struct kvm_vcpu *vcpu) { - *host_data_ptr(fp_owner) =3D FP_STATE_HOST_OWNED; + __vcpu_assign_sys_reg(vcpu, SMCR_EL1, read_sysreg_el1(SYS_SMCR)); + __vcpu_assign_sys_reg(vcpu, SVCR, read_sysreg_s(SYS_SVCR)); + + /* + * On saving/restoring guest sve state, always use the maximum VL for + * the guest. The layout of the data when saving the sve state depends + * on the VL, so use a consistent (i.e., the maximum) guest VL. + * + * We restore the FA64 and SME2 enables for the host since we + * will always restore the host configuration so if host and + * guest VLs are the same we might suppress an update. + */ + sme_cond_update_smcr(vcpu_sme_max_vq(vcpu) - 1, system_supports_fa64(), + system_supports_sme2(), SYS_SMCR_EL2); + + if (vcpu_za_enabled(vcpu)) + __sme_save_state(vcpu_sme_state(vcpu), vcpu_has_sme2(vcpu)); +} + +static void __hyp_sme_restore_host(void) +{ + /* + * The hypervisor refuses to run if we are in streaming mode + * or have ZA enabled so there is no SME specific state to + * restore other than the system registers. + * + * Note that this constrains the PE to the maximum shared VL + * that was discovered, if we wish to use larger VLs this will + * need to be revisited. + */ + sme_cond_update_smcr(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SME]) - 1, + cpus_have_final_cap(ARM64_SME_FA64), + cpus_have_final_cap(ARM64_SME2), SYS_SMCR_EL2); + + write_sysreg_el1(*host_data_ptr(smcr_el1), SYS_SMCR); + + sme_smstop(); +} + +static void fpsimd_sve_flush(struct kvm_vcpu *vcpu) +{ + /* + * If the guest has SME then we need to restore the trap + * controls in SMCR and mode in SVCR in order to ensure that + * traps generated directly to EL1 have the correct types, + * otherwise we can defer until we load the guest state. + */ + if (vcpu_has_sme(vcpu)) { + kvm_hyp_save_fpsimd_host(vcpu); + kvm_sme_configure_traps(vcpu); + + *host_data_ptr(fp_owner) =3D FP_STATE_FREE; + } else { + *host_data_ptr(fp_owner) =3D FP_STATE_HOST_OWNED; + } } =20 static void fpsimd_sve_sync(struct kvm_vcpu *vcpu) @@ -75,7 +132,10 @@ static void fpsimd_sve_sync(struct kvm_vcpu *vcpu) */ isb(); =20 - if (vcpu_has_sve(vcpu)) + if (vcpu_has_sme(vcpu)) + __hyp_sme_save_guest(vcpu); + + if (vcpu_has_sve(vcpu) || vcpu_in_streaming_mode(vcpu)) __hyp_sve_save_guest(vcpu); else __fpsimd_save_state(&vcpu->arch.ctxt.fp_regs); @@ -84,6 +144,9 @@ static void fpsimd_sve_sync(struct kvm_vcpu *vcpu) if (has_fpmr) __vcpu_assign_sys_reg(vcpu, FPMR, read_sysreg_s(SYS_FPMR)); =20 + if (system_supports_sme()) + __hyp_sme_restore_host(); + if (system_supports_sve()) __hyp_sve_restore_host(); else @@ -121,7 +184,7 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vc= pu) { struct kvm_vcpu *host_vcpu =3D hyp_vcpu->host_vcpu; =20 - fpsimd_sve_flush(); + fpsimd_sve_flush(host_vcpu); flush_debug_state(hyp_vcpu); =20 hyp_vcpu->vcpu.arch.ctxt =3D host_vcpu->arch.ctxt; @@ -203,10 +266,9 @@ static void handle___kvm_vcpu_run(struct kvm_cpu_conte= xt *host_ctxt) struct pkvm_hyp_vcpu *hyp_vcpu =3D pkvm_get_loaded_hyp_vcpu(); =20 /* - * KVM (and pKVM) doesn't support SME guests for now, and - * ensures that SME features aren't enabled in pstate when - * loading a vcpu. Therefore, if SME features enabled the host - * is misbehaving. + * KVM (and pKVM) refuses to run if PSTATE.{SM,ZA} are + * enabled. Therefore, if SME features enabled the + * host is misbehaving. */ if (unlikely(system_supports_sme() && read_sysreg_s(SYS_SVCR))) { ret =3D -EINVAL; --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9874D30DD1A; Tue, 2 Sep 2025 11:46:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813574; cv=none; b=HZiU2lSAAIUcGfjbWwQVYOxK2PZBvfdCJe1lRL7M4q+f4aXqnon66vU+8kDuVSHXmIwkDZis2b0GiAEn3hTN2gSoAEmGh3dqK9dsc8JNaQEig2Nt7m4BpyqPWmJ4LP3BYUB9VumDklHGgvxKW2c3gxOKYuc0jhtxZwYHrqnL3Xo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813574; c=relaxed/simple; bh=NTFrR4Ul2FjKNCZg2te+RhXOLPTLiTRbz6mCjfo2Vis=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=SETSIt4/r9cqY/YIZDDQeRwLRZzyL/NIMD1Q7xP6lIWZN1W+8D+09SEeb+TpktdF3XLExKLAa5lw2uGPYGUuT8vk1yOyRicU4e4UrbX7YvjgrKFV/NXMyFlLg7LjMKpG5dbJWmEmAE5MQR6zRyEBSIJeNXH/Eq2qk9jGZNbFp0Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LU/fN+g1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LU/fN+g1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5C12DC4CEED; Tue, 2 Sep 2025 11:46:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813572; bh=NTFrR4Ul2FjKNCZg2te+RhXOLPTLiTRbz6mCjfo2Vis=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=LU/fN+g127OF0iEEH5mh9j4gTeSNuspKfz0iLbiV0e7pJj7Bjg6o4A504i1I6hSXc Or3FyGb/wisfYdoq+OF/yxKWZmUEbdzJTAfU+PZR2lrsEWY0p+DOmPlugmJAazoJVc NTipps/n2A9xgky2mxR/b8jsZ13ai6vSjJ4caGM9RUmu7eHPcPLWPhfaXXbhAFOi+1 3JyoUtkLUXrV8BOMJfjuI641GRCoOEQGmFtBhkEY7qPYEmce5eEZshZ7+/BnDN5jpV hEP9j9cCdsKvRS3y6Fmz5rR1ZaKYIAQOinPhxkCisp0XdvQkMHUAqgdHz4aGK+agRa ZJ1Fnr6doa2zw== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:28 +0100 Subject: [PATCH v8 25/29] KVM: arm64: Handle SME exceptions Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-25-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=6175; i=broonie@kernel.org; h=from:subject:message-id; bh=NTFrR4Ul2FjKNCZg2te+RhXOLPTLiTRbz6mCjfo2Vis=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiNqgPfkaWMCSeQ2d8u5PqsX4XybQjxBRtlB s0L51CavqyJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYjQAKCRAk1otyXVSH 0KJWB/43c6WsW+Bi69WMyiwE1PeIMs5+KboO9kaMDTUp1dRAzMKf6SZW+Nt/TgMvtyZaRzdXzqT r6xVWyojMTDtY1Wr8Nnc9l7PR7sQMVOGzs0lYu57GoMW4OnsQtxCLxtxu28NxdJhClfAbXHCYBp rQ2waBf+wppakR4Lq/FYARQiWMxPJbTu127zGR70Q8uR8W9vL4TIFYN3w7Lw+OFAQjF+OJf0FcX jHNTuyKtEXry/58P3RRVw7z8ySGfX3xkytQLxdJaI7VECxm/2G478BsP83fgBUv6CjlQT3WpOJT 40mITMw3kI53loFSXuNaw9qi6Z0mezwN+1qKBkoJfYqyStvA X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The access control for SME follows the same structure as for the base FP and SVE extensions, with control being via CPACR_ELx.SMEN and CPTR_EL2.TSM mirroring the equivalent FPSIMD and SVE controls in those registers. Add handling for these controls and exceptions mirroring the existing handling for FPSIMD and SVE. Signed-off-by: Mark Brown --- arch/arm64/kvm/handle_exit.c | 14 ++++++++++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 11 ++++++----- arch/arm64/kvm/hyp/nvhe/switch.c | 4 +++- arch/arm64/kvm/hyp/vhe/switch.c | 17 ++++++++++++----- 4 files changed, 35 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index a598072f36d2..d96f3a585d70 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -232,6 +232,19 @@ static int handle_sve(struct kvm_vcpu *vcpu) return 1; } =20 +/* + * Guest access to SME registers should be routed to this handler only + * when the system doesn't support SME. + */ +static int handle_sme(struct kvm_vcpu *vcpu) +{ + if (guest_hyp_sme_traps_enabled(vcpu)) + return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu)); + + kvm_inject_undefined(vcpu); + return 1; +} + /* * Two possibilities to handle a trapping ptrauth instruction: * @@ -385,6 +398,7 @@ static exit_handle_fn arm_exit_handlers[] =3D { [ESR_ELx_EC_SVC64] =3D handle_svc, [ESR_ELx_EC_SYS64] =3D kvm_handle_sys_reg, [ESR_ELx_EC_SVE] =3D handle_sve, + [ESR_ELx_EC_SME] =3D handle_sme, [ESR_ELx_EC_ERET] =3D kvm_handle_eret, [ESR_ELx_EC_IABT_LOW] =3D kvm_handle_guest_abort, [ESR_ELx_EC_DABT_LOW] =3D kvm_handle_guest_abort, diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/i= nclude/hyp/switch.h index c128b4d25a2d..9375afa96b71 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -69,11 +69,8 @@ static inline void __activate_cptr_traps_nvhe(struct kvm= _vcpu *vcpu) { u64 val =3D CPTR_NVHE_EL2_RES1 | CPTR_EL2_TAM | CPTR_EL2_TTA; =20 - /* - * Always trap SME since it's not supported in KVM. - * TSM is RES1 if SME isn't implemented. - */ - val |=3D CPTR_EL2_TSM; + if (!vcpu_has_sme(vcpu) || !guest_owns_fp_regs()) + val |=3D CPTR_EL2_TSM; =20 if (!vcpu_has_sve(vcpu) || !guest_owns_fp_regs()) val |=3D CPTR_EL2_TZ; @@ -101,6 +98,8 @@ static inline void __activate_cptr_traps_vhe(struct kvm_= vcpu *vcpu) val |=3D CPACR_EL1_FPEN; if (vcpu_has_sve(vcpu)) val |=3D CPACR_EL1_ZEN; + if (vcpu_has_sme(vcpu)) + val |=3D CPACR_EL1_SMEN; } =20 if (!vcpu_has_nv(vcpu)) @@ -142,6 +141,8 @@ static inline void __activate_cptr_traps_vhe(struct kvm= _vcpu *vcpu) val &=3D ~CPACR_EL1_FPEN; if (!(SYS_FIELD_GET(CPACR_EL1, ZEN, cptr) & BIT(0))) val &=3D ~CPACR_EL1_ZEN; + if (!(SYS_FIELD_GET(CPACR_EL1, SMEN, cptr) & BIT(0))) + val &=3D ~CPACR_EL1_SMEN; =20 if (kvm_has_feat(vcpu->kvm, ID_AA64MMFR3_EL1, S2POE, IMP)) val |=3D cptr & CPACR_EL1_E0POE; diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/swi= tch.c index ccd575d5f6de..79a3e5c290f9 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -175,6 +175,7 @@ static const exit_handler_fn hyp_exit_handlers[] =3D { [ESR_ELx_EC_CP15_32] =3D kvm_hyp_handle_cp15_32, [ESR_ELx_EC_SYS64] =3D kvm_hyp_handle_sysreg, [ESR_ELx_EC_SVE] =3D kvm_hyp_handle_fpsimd, + [ESR_ELx_EC_SME] =3D kvm_hyp_handle_fpsimd, [ESR_ELx_EC_FP_ASIMD] =3D kvm_hyp_handle_fpsimd, [ESR_ELx_EC_IABT_LOW] =3D kvm_hyp_handle_iabt_low, [ESR_ELx_EC_DABT_LOW] =3D kvm_hyp_handle_dabt_low, @@ -186,7 +187,8 @@ static const exit_handler_fn pvm_exit_handlers[] =3D { [0 ... ESR_ELx_EC_MAX] =3D NULL, [ESR_ELx_EC_SYS64] =3D kvm_handle_pvm_sys64, [ESR_ELx_EC_SVE] =3D kvm_handle_pvm_restricted, - [ESR_ELx_EC_FP_ASIMD] =3D kvm_hyp_handle_fpsimd, + [ESR_ELx_EC_SME] =3D kvm_handle_pvm_restricted, + [ESR_ELx_EC_FP_ASIMD] =3D kvm_handle_pvm_restricted, [ESR_ELx_EC_IABT_LOW] =3D kvm_hyp_handle_iabt_low, [ESR_ELx_EC_DABT_LOW] =3D kvm_hyp_handle_dabt_low, [ESR_ELx_EC_WATCHPT_LOW] =3D kvm_hyp_handle_watchpt_low, diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switc= h.c index e482181c6632..86a892966a18 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -448,22 +448,28 @@ static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu = *vcpu, u64 *exit_code) return true; } =20 -static bool kvm_hyp_handle_zcr_el2(struct kvm_vcpu *vcpu, u64 *exit_code) +static bool kvm_hyp_handle_vec_cr_el2(struct kvm_vcpu *vcpu, u64 *exit_cod= e) { u32 sysreg =3D esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu)); =20 if (!vcpu_has_nv(vcpu)) return false; =20 - if (sysreg !=3D SYS_ZCR_EL2) + switch (sysreg) { + case SYS_ZCR_EL2: + case SYS_SMCR_EL2: + break; + default: return false; + } =20 if (guest_owns_fp_regs()) return false; =20 /* - * ZCR_EL2 traps are handled in the slow path, with the expectation - * that the guest's FP context has already been loaded onto the CPU. + * ZCR_EL2 and SMCR_EL2 traps are handled in the slow path, + * with the expectation that the guest's FP context has + * already been loaded onto the CPU. * * Load the guest's FP context and unconditionally forward to the * slow path for handling (i.e. return false). @@ -483,7 +489,7 @@ static bool kvm_hyp_handle_sysreg_vhe(struct kvm_vcpu *= vcpu, u64 *exit_code) if (kvm_hyp_handle_cpacr_el1(vcpu, exit_code)) return true; =20 - if (kvm_hyp_handle_zcr_el2(vcpu, exit_code)) + if (kvm_hyp_handle_vec_cr_el2(vcpu, exit_code)) return true; =20 return kvm_hyp_handle_sysreg(vcpu, exit_code); @@ -512,6 +518,7 @@ static const exit_handler_fn hyp_exit_handlers[] =3D { [0 ... ESR_ELx_EC_MAX] =3D NULL, [ESR_ELx_EC_CP15_32] =3D kvm_hyp_handle_cp15_32, [ESR_ELx_EC_SYS64] =3D kvm_hyp_handle_sysreg_vhe, + [ESR_ELx_EC_SME] =3D kvm_hyp_handle_fpsimd, [ESR_ELx_EC_SVE] =3D kvm_hyp_handle_fpsimd, [ESR_ELx_EC_FP_ASIMD] =3D kvm_hyp_handle_fpsimd, [ESR_ELx_EC_IABT_LOW] =3D kvm_hyp_handle_iabt_low, --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D5D230EF90; Tue, 2 Sep 2025 11:46:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813576; cv=none; b=TbdrbakFqoeE9wULvZH4LVHVjNZzTZkGxnQkM5U2SiAWhmRg8DSbdln0/CSzoZxQgfafVYwXj0bo23CCkhu/GIPqA12ewdu8LhaTlgqcRi3op3rDmtxDShtXlQ1cbxYht1N67i0uRomUEI4AIKZCndBVjJR02F75QFHrkNLJOJY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813576; c=relaxed/simple; bh=ApIjPu6tr41pN88JrHKurRe18MYIB0RS0vVGxBjI5iI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Hofk+kC47w8cZ40idYU4mAann+Nj5GdqynuGfdyvocVceTwWzfvLbWN8lO6oXs0MCIvlC8vshVPTPjJkRRmJ1DTgHw/R9NDaJqpfGxWrgnHaHXVAcJdgm7qHpjKD33lOrhkO/YhO4l3iDDTKUQquMUTZm+Y3rogGMDfYEtWkTLw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y3U/VCcT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y3U/VCcT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7FBF7C4CEF7; Tue, 2 Sep 2025 11:46:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813576; bh=ApIjPu6tr41pN88JrHKurRe18MYIB0RS0vVGxBjI5iI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Y3U/VCcTyDJrfmX5N1I3ma4BNaMP6NnjIsplzE8qqNmwu2XCxAwPUL27++y15kkKf BmsUHdrWff0+p/dch1o3gGe+npIHUciHViJNy5RewFkc2eTGS73eT+87n5zjbvIgHL +d/B7S5l1MZcDikktnMIhVP+ZZp6Y4hPurHVL61m5+8pDKMhvsge7DALAiAvF3xK2R hBLIVSVdNSk1LVU8EuI/NB2BwDVEi1MCNj6EHElJ3Gcj8x0FhilbvHTC5mIfcYCzey OoVIK7Ua1fA02entp4HnJ4vr9YmZ2mJGEjIFto/pZJ13T/znl9czAGX9fYee9/n676 1bhOcFyHloFpw== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:29 +0100 Subject: [PATCH v8 26/29] KVM: arm64: Expose SME to nested guests Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-26-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=1234; i=broonie@kernel.org; h=from:subject:message-id; bh=ApIjPu6tr41pN88JrHKurRe18MYIB0RS0vVGxBjI5iI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiOkZba4yI6s4kTrmrdeDhrz3ok1FUW6bPBY q74Sa3+y/iJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYjgAKCRAk1otyXVSH 0B0eB/4yoHkWE2bYFaQkDOX1dyrxupmFWVwQPQqDQ5u01j8CCEG7HlTowkwPc+l3bqfDugrPdSs ZXSIAHLlcppfcijjwVmDKC6HpLNxG0f6rNy9DoxM6eHanWIjJaLs5yEqk5pmIjALIdPSxy5YnFL 7Owmj00+UDfilbqgmg1xdquA/jc0f9GICTfWllO8jkhzkQ9c8RgG9IKIGmV2InLCvYof3+ws3el ICGQ2kZL2svSg4O2ryDcK2wiottcvFbIIVqEGwrrARywl16wkp8mSHgNvJoETdt5ZfF2N615sry knnXAMcPzDGKO8TnyWmDB2E891gSf86fdlPcyRkiV7wt8SbY X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB With support for context switching SME state in place allow access to SME in nested guests. The SME floating point state is handled along with all the other floating point state, SME specific floating point exceptions are directed into the same handlers as other floating point exceptions with NV specific handling for the vector lengths already in place. TPIDR2_EL0 is context switched along with the other TPIDRs as part of the main guest register context switch. SME priority support is currently masked from all guests including nested ones. Signed-off-by: Mark Brown --- arch/arm64/kvm/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c index 153b3e11b115..804694f2dc9a 100644 --- a/arch/arm64/kvm/nested.c +++ b/arch/arm64/kvm/nested.c @@ -1458,9 +1458,10 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 va= l) break; =20 case SYS_ID_AA64PFR1_EL1: - /* Only support BTI, SSBS, CSV2_frac */ + /* Only support BTI, SSBS, SME, CSV2_frac */ val &=3D (ID_AA64PFR1_EL1_BT | ID_AA64PFR1_EL1_SSBS | + ID_AA64PFR1_EL1_SME | ID_AA64PFR1_EL1_CSV2_frac); break; =20 --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6BC02FD1CA; Tue, 2 Sep 2025 11:46:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813581; cv=none; b=pSfTwZQnOuX/8r3EdpItCVPHzc/IbpOGdQ171EqcowwVD1fLPWvkScOaqGIuSDBUTbhlC02UwHnnIc3RSKh6o2oE3cNhynjHZsE/s/SfFYuAV1nwpYVFPCi47Or9/MDlZ36MUxoJNcpMkIM3r6i3kehza0XY2eAQlbX+8wx3wzo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813581; c=relaxed/simple; bh=XqEN3TlOUqdTV/4QilKcAUW5tU01fUqiX4rm0wGB/6w=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BcTgrOq8ke08Ro312A0QCUel+1y5KZ/nc5rr3QveDfGetRdS1iVbFd/K6Bl58WZA766aF0tmXQ3ZjXw2o5I9niAKI6qWQGxQ+K+gepXgFJtIJMbjt51LZT4g7I0sZhW+56LEU4qc284/Vf3fnzZNwABYSVomAD1GV3g4jCYLC9Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BLlqnACv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BLlqnACv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A8D4AC4CEF5; Tue, 2 Sep 2025 11:46:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813580; bh=XqEN3TlOUqdTV/4QilKcAUW5tU01fUqiX4rm0wGB/6w=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=BLlqnACv0nQnvn2cWsv07Dga11GN4EQ6+gLHp5ptF+BoSqDAM5TNa1j/Fi+odFfud SQ8GnXPb7be5ULXntnMsYOhqMBHUVbUHYPjIBebFieDwGjw+6F38nh/mBHAxRYAV9o Uabw0W2yL1ZcN20O3LHOAu5p32iPSb8dx6hREhgUy7esLrKyolxDR54WRdpMoDDllc +65thI0BSUkQN2C4/d7A7KKRgmMoSFKIu3pTKxv0WDQqEORlLUCtjaabXOJzFfHgjS 2wPW3dAkpBJtq0c0luFXfaHQoOD61tKjuXV89HvTiZQ0XXTOdMJgU0oo/sLl7XyvTt qaCx2NxbwU5uQ== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:30 +0100 Subject: [PATCH v8 27/29] KVM: arm64: Provide interface for configuring and enabling SME for guests Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-27-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=15759; i=broonie@kernel.org; h=from:subject:message-id; bh=XqEN3TlOUqdTV/4QilKcAUW5tU01fUqiX4rm0wGB/6w=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiPCYDVKHG2va6qC5pqXiH0JjzhXyfyoigHD Ucnk2UoL2yJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYjwAKCRAk1otyXVSH 0JTbB/4tDdIDYpp5kYchKfb98Nlk6aR+j5uA4Y+2Jz9s/YdRtaoB9CGR733vi1eizO1RYMghe3G LSZdDIEFECogkT/L6eLB0TMEBMB4FtyZRl7ew4OfXT7USK6WBIMjZRtL5+UfTF0OAtwe7Mh8X/C mxTJlj+x7DWzuKVYsusa/0lsR6FhfUZBgAFtDi9FocrREYWnXPymJkHC+oMlGRAZ/ni/c6of9FI zIp2uYACIw+DVlMM6HoI77X9mPcTGBRyFojs4NxSwj7sb4qFrmOa0qZNjmuZqVwOuVrLtUKIqtR 9BlkA0S9h6PWSXQoHsPSG0SPhtjEh5XO0TVh34YXkG2A0WvR X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Since SME requires configuration of a vector length in order to know the size of both the streaming mode SVE state and ZA array we implement a capability for it and require that it be enabled and finalized before the SME specific state can be accessed, similarly to SVE. Due to the overlap with sizing the SVE state we finalise both SVE and SME with a single finalization, preventing any further changes to the SVE and SME configuration once KVM_ARM_VCPU_VEC (an alias for _VCPU_SVE) has been finalised. This is not a thing of great elegance but it ensures that we never have a state where one of SVE or SME is finalised and the other not, avoiding complexity. SME is supported for normal and protected guests. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 12 +++- arch/arm64/include/uapi/asm/kvm.h | 1 + arch/arm64/kvm/arm.c | 10 ++++ arch/arm64/kvm/hyp/nvhe/pkvm.c | 76 +++++++++++++++++++----- arch/arm64/kvm/hyp/nvhe/sys_regs.c | 6 ++ arch/arm64/kvm/reset.c | 116 +++++++++++++++++++++++++++++++--= ---- include/uapi/linux/kvm.h | 1 + 7 files changed, 189 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index fa646da80e3e..809cd1843386 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -39,7 +39,7 @@ =20 #define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS =20 -#define KVM_VCPU_MAX_FEATURES 9 +#define KVM_VCPU_MAX_FEATURES 10 #define KVM_VCPU_VALID_FEATURES (BIT(KVM_VCPU_MAX_FEATURES) - 1) =20 #define KVM_REQ_SLEEP \ @@ -81,6 +81,7 @@ extern unsigned int __ro_after_init kvm_host_max_vl[ARM64= _VEC_MAX]; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); =20 int __init kvm_arm_init_sve(void); +int __init kvm_arm_init_sme(void); =20 u32 __attribute_const__ kvm_target_cpu(void); void kvm_reset_vcpu(struct kvm_vcpu *vcpu); @@ -1146,7 +1147,14 @@ struct kvm_vcpu_arch { __size_ret; \ }) =20 -#define vcpu_sve_state_size(vcpu) sve_state_size_from_vl((vcpu)->arch.max_= vl[ARM64_VEC_SVE]) +#define vcpu_sve_state_size(vcpu) ({ \ + unsigned int __max_vl; \ + \ + __max_vl =3D max((vcpu)->arch.max_vl[ARM64_VEC_SVE], \ + (vcpu)->arch.max_vl[ARM64_VEC_SME]); \ + \ + sve_state_size_from_vl(__max_vl); \ +}) =20 #define vcpu_sme_state(vcpu) (kern_hyp_va((vcpu)->arch.sme_state)) =20 diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/as= m/kvm.h index cf75a830f17a..4fb0d7353d54 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -106,6 +106,7 @@ struct kvm_regs { #define KVM_ARM_VCPU_PTRAUTH_GENERIC 6 /* VCPU uses generic authentication= */ #define KVM_ARM_VCPU_HAS_EL2 7 /* Support nested virtualization */ #define KVM_ARM_VCPU_HAS_EL2_E2H0 8 /* Limit NV support to E2H RES0 */ +#define KVM_ARM_VCPU_SME 9 /* enable SME for this CPU */ =20 /* * An alias for _SVE since we finalize VL configuration for both SVE and S= ME diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 888f7c7abf54..ec74c6bd072f 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -392,6 +392,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long = ext) case KVM_CAP_ARM_SVE: r =3D system_supports_sve(); break; + case KVM_CAP_ARM_SME: + r =3D system_supports_sme(); + break; case KVM_CAP_ARM_PTRAUTH_ADDRESS: case KVM_CAP_ARM_PTRAUTH_GENERIC: r =3D kvm_has_full_ptr_auth(); @@ -1442,6 +1445,9 @@ static unsigned long system_supported_vcpu_features(v= oid) if (!system_supports_sve()) clear_bit(KVM_ARM_VCPU_SVE, &features); =20 + if (!system_supports_sme()) + clear_bit(KVM_ARM_VCPU_SME, &features); + if (!kvm_has_full_ptr_auth()) { clear_bit(KVM_ARM_VCPU_PTRAUTH_ADDRESS, &features); clear_bit(KVM_ARM_VCPU_PTRAUTH_GENERIC, &features); @@ -2849,6 +2855,10 @@ static __init int kvm_arm_init(void) if (err) return err; =20 + err =3D kvm_arm_init_sme(); + if (err) + return err; + err =3D kvm_arm_vmid_alloc_init(); if (err) { kvm_err("Failed to initialize VMID allocator.\n"); diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index fc9fb2e693f6..ce3e13acc25c 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -148,10 +148,6 @@ static int pkvm_check_pvm_cpu_features(struct kvm_vcpu= *vcpu) !kvm_has_feat(kvm, ID_AA64PFR0_EL1, AdvSIMD, IMP)) return -EINVAL; =20 - /* No SME support in KVM right now. Check to catch if it changes. */ - if (kvm_has_feat(kvm, ID_AA64PFR1_EL1, SME, IMP)) - return -EINVAL; - return 0; } =20 @@ -364,6 +360,11 @@ static void pkvm_init_features_from_host(struct pkvm_h= yp_vm *hyp_vm, const struc kvm->arch.flags |=3D host_arch_flags & BIT(KVM_ARCH_FLAG_GUEST_HAS_SVE); } =20 + if (kvm_pvm_ext_allowed(KVM_CAP_ARM_SME)) { + set_bit(KVM_ARM_VCPU_SME, allowed_features); + kvm->arch.flags |=3D host_arch_flags & BIT(KVM_ARCH_FLAG_GUEST_HAS_SME); + } + bitmap_and(kvm->arch.vcpu_features, host_kvm->arch.vcpu_features, allowed_features, KVM_VCPU_MAX_FEATURES); } @@ -386,6 +387,18 @@ static void unpin_host_sve_state(struct pkvm_hyp_vcpu = *hyp_vcpu) sve_state + vcpu_sve_state_size(&hyp_vcpu->vcpu)); } =20 +static void unpin_host_sme_state(struct pkvm_hyp_vcpu *hyp_vcpu) +{ + void *sme_state; + + if (!vcpu_has_feature(&hyp_vcpu->vcpu, KVM_ARM_VCPU_SME)) + return; + + sme_state =3D kern_hyp_va(hyp_vcpu->vcpu.arch.sme_state); + hyp_unpin_shared_mem(sme_state, + sme_state + vcpu_sme_state_size(&hyp_vcpu->vcpu)); +} + static void unpin_host_vcpus(struct pkvm_hyp_vcpu *hyp_vcpus[], unsigned int nr_vcpus) { @@ -399,6 +412,7 @@ static void unpin_host_vcpus(struct pkvm_hyp_vcpu *hyp_= vcpus[], =20 unpin_host_vcpu(hyp_vcpu->host_vcpu); unpin_host_sve_state(hyp_vcpu); + unpin_host_sme_state(hyp_vcpu); } } =20 @@ -413,23 +427,35 @@ static void init_pkvm_hyp_vm(struct kvm *host_kvm, st= ruct pkvm_hyp_vm *hyp_vm, pkvm_init_features_from_host(hyp_vm, host_kvm); } =20 -static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_v= cpu *host_vcpu) +static int pkvm_vcpu_init_vec(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_v= cpu *host_vcpu) { struct kvm_vcpu *vcpu =3D &hyp_vcpu->vcpu; - unsigned int sve_max_vl; - size_t sve_state_size; - void *sve_state; + unsigned int sve_max_vl, sme_max_vl; + size_t sve_state_size, sme_state_size; + void *sve_state, *sme_state; int ret =3D 0; =20 - if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) { + if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE) && + !vcpu_has_feature(vcpu, KVM_ARM_VCPU_SME)) { vcpu_clear_flag(vcpu, VCPU_VEC_FINALIZED); return 0; } =20 /* Limit guest vector length to the maximum supported by the host. */ - sve_max_vl =3D min(READ_ONCE(host_vcpu->arch.max_vl[ARM64_VEC_SVE]), - kvm_host_max_vl[ARM64_VEC_SVE]); - sve_state_size =3D sve_state_size_from_vl(sve_max_vl); + if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) + sve_max_vl =3D min(READ_ONCE(host_vcpu->arch.max_vl[ARM64_VEC_SVE]), + kvm_host_max_vl[ARM64_VEC_SVE]); + else + sve_max_vl =3D 0; + + if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SME)) + sme_max_vl =3D min(READ_ONCE(host_vcpu->arch.max_vl[ARM64_VEC_SME]), + kvm_host_max_vl[ARM64_VEC_SME]); + else + sme_max_vl =3D 0; + + /* We need SVE storage for the larger of normal or streaming mode */ + sve_state_size =3D sve_state_size_from_vl(max(sve_max_vl, sme_max_vl)); sve_state =3D kern_hyp_va(READ_ONCE(host_vcpu->arch.sve_state)); =20 if (!sve_state || !sve_state_size) { @@ -441,12 +467,36 @@ static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *h= yp_vcpu, struct kvm_vcpu *h if (ret) goto err; =20 + if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SME)) { + sme_state_size =3D sme_state_size_from_vl(sme_max_vl, + vcpu_has_sme2(vcpu)); + sme_state =3D kern_hyp_va(READ_ONCE(host_vcpu->arch.sme_state)); + + if (!sme_state || !sme_state_size) { + ret =3D -EINVAL; + goto err_sve_mapped; + } + + ret =3D hyp_pin_shared_mem(sme_state, sme_state + sme_state_size); + if (ret) + goto err_sve_mapped; + } else { + sme_state =3D 0; + } + vcpu->arch.sve_state =3D sve_state; vcpu->arch.max_vl[ARM64_VEC_SVE] =3D sve_max_vl; =20 + vcpu->arch.sme_state =3D sme_state; + vcpu->arch.max_vl[ARM64_VEC_SME] =3D sme_max_vl; + return 0; + +err_sve_mapped: + hyp_unpin_shared_mem(sve_state, sve_state + sve_state_size); err: clear_bit(KVM_ARM_VCPU_SVE, vcpu->kvm->arch.vcpu_features); + clear_bit(KVM_ARM_VCPU_SME, vcpu->kvm->arch.vcpu_features); return ret; } =20 @@ -476,7 +526,7 @@ static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp= _vcpu, if (ret) goto done; =20 - ret =3D pkvm_vcpu_init_sve(hyp_vcpu, host_vcpu); + ret =3D pkvm_vcpu_init_vec(hyp_vcpu, host_vcpu); done: if (ret) unpin_host_vcpu(host_vcpu); diff --git a/arch/arm64/kvm/hyp/nvhe/sys_regs.c b/arch/arm64/kvm/hyp/nvhe/s= ys_regs.c index 1ddd9ed3cbb3..eed177dfcb96 100644 --- a/arch/arm64/kvm/hyp/nvhe/sys_regs.c +++ b/arch/arm64/kvm/hyp/nvhe/sys_regs.c @@ -66,6 +66,11 @@ static bool vm_has_ptrauth(const struct kvm *kvm) kvm_vcpu_has_feature(kvm, KVM_ARM_VCPU_PTRAUTH_GENERIC); } =20 +static bool vm_has_sme(const struct kvm *kvm) +{ + return system_supports_sme() && kvm_vcpu_has_feature(kvm, KVM_ARM_VCPU_SM= E); +} + static bool vm_has_sve(const struct kvm *kvm) { return system_supports_sve() && kvm_vcpu_has_feature(kvm, KVM_ARM_VCPU_SV= E); @@ -102,6 +107,7 @@ static const struct pvm_ftr_bits pvmid_aa64pfr0[] =3D { }; =20 static const struct pvm_ftr_bits pvmid_aa64pfr1[] =3D { + MAX_FEAT_FUNC(ID_AA64PFR1_EL1, SME, SME2, vm_has_sme), MAX_FEAT(ID_AA64PFR1_EL1, BT, IMP), MAX_FEAT(ID_AA64PFR1_EL1, SSBS, SSBS2), MAX_FEAT_ENUM(ID_AA64PFR1_EL1, MTE_frac, NI), diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index a8684a1346ec..e6dc04267cbb 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -76,6 +76,34 @@ int __init kvm_arm_init_sve(void) return 0; } =20 +int __init kvm_arm_init_sme(void) +{ + if (system_supports_sme()) { + kvm_max_vl[ARM64_VEC_SME] =3D sme_max_virtualisable_vl(); + kvm_host_max_vl[ARM64_VEC_SME] =3D sme_max_vl(); + kvm_nvhe_sym(kvm_host_max_vl[ARM64_VEC_SME]) =3D kvm_host_max_vl[ARM64_V= EC_SME]; + + /* + * The get_sve_reg()/set_sve_reg() ioctl interface will need + * to be extended with multiple register slice support in + * order to support vector lengths greater than + * VL_ARCH_MAX: + */ + if (WARN_ON(kvm_max_vl[ARM64_VEC_SME] > VL_ARCH_MAX)) + kvm_max_vl[ARM64_VEC_SME] =3D VL_ARCH_MAX; + + /* + * Don't even try to make use of vector lengths that + * aren't available on all CPUs, for now: + */ + if (kvm_max_vl[ARM64_VEC_SME] < sme_max_vl()) + pr_warn("KVM: SME vector length for guests limited to %u bytes\n", + kvm_max_vl[ARM64_VEC_SME]); + } + + return 0; +} + static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) { vcpu->arch.max_vl[ARM64_VEC_SVE] =3D kvm_max_vl[ARM64_VEC_SVE]; @@ -88,42 +116,86 @@ static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) set_bit(KVM_ARCH_FLAG_GUEST_HAS_SVE, &vcpu->kvm->arch.flags); } =20 +static void kvm_vcpu_enable_sme(struct kvm_vcpu *vcpu) +{ + vcpu->arch.max_vl[ARM64_VEC_SME] =3D kvm_max_vl[ARM64_VEC_SME]; + + /* + * Userspace can still customize the vector lengths by writing + * KVM_REG_ARM64_SME_VLS. Allocation is deferred until + * kvm_arm_vcpu_finalize(), which freezes the configuration. + */ + set_bit(KVM_ARCH_FLAG_GUEST_HAS_SME, &vcpu->kvm->arch.flags); +} + /* - * Finalize vcpu's maximum SVE vector length, allocating - * vcpu->arch.sve_state as necessary. + * Finalize vcpu's maximum vector lengths, allocating + * vcpu->arch.sve_state and vcpu->arch.sme_state as necessary. */ static int kvm_vcpu_finalize_vec(struct kvm_vcpu *vcpu) { - void *buf; + void *sve_state, *sme_state; unsigned int vl; - size_t reg_sz; int ret; =20 - vl =3D vcpu->arch.max_vl[ARM64_VEC_SVE]; - /* * Responsibility for these properties is shared between * kvm_arm_init_sve(), kvm_vcpu_enable_sve() and * set_sve_vls(). Double-check here just to be sure: */ - if (WARN_ON(!sve_vl_valid(vl) || vl > sve_max_virtualisable_vl() || - vl > VL_ARCH_MAX)) - return -EIO; + if (vcpu_has_sve(vcpu)) { + vl =3D vcpu->arch.max_vl[ARM64_VEC_SVE]; + if (WARN_ON(!sve_vl_valid(vl) || + vl > sve_max_virtualisable_vl() || + vl > VL_ARCH_MAX)) + return -EIO; + } =20 - reg_sz =3D vcpu_sve_state_size(vcpu); - buf =3D kzalloc(reg_sz, GFP_KERNEL_ACCOUNT); - if (!buf) + /* Similarly for SME */ + if (vcpu_has_sme(vcpu)) { + vl =3D vcpu->arch.max_vl[ARM64_VEC_SME]; + if (WARN_ON(!sve_vl_valid(vl) || + vl > sme_max_virtualisable_vl() || + vl > VL_ARCH_MAX)) + return -EIO; + } + + sve_state =3D kzalloc(vcpu_sve_state_size(vcpu), GFP_KERNEL_ACCOUNT); + if (!sve_state) return -ENOMEM; =20 - ret =3D kvm_share_hyp(buf, buf + reg_sz); - if (ret) { - kfree(buf); - return ret; + ret =3D kvm_share_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); + if (ret) + goto err_sve_alloc; + + if (vcpu_has_sme(vcpu)) { + sme_state =3D kzalloc(vcpu_sme_state_size(vcpu), + GFP_KERNEL_ACCOUNT); + if (!sme_state) { + ret =3D -ENOMEM; + goto err_sve_map; + } + + ret =3D kvm_share_hyp(sme_state, + sme_state + vcpu_sme_state_size(vcpu)); + if (ret) + goto err_sme_alloc; + } else { + sme_state =3D NULL; } -=09 - vcpu->arch.sve_state =3D buf; + + vcpu->arch.sve_state =3D sve_state; + vcpu->arch.sme_state =3D sme_state; vcpu_set_flag(vcpu, VCPU_VEC_FINALIZED); return 0; + +err_sme_alloc: + kfree(sme_state); +err_sve_map: + kvm_unshare_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); +err_sve_alloc: + kfree(sve_state); + return ret; } =20 int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) @@ -153,12 +225,16 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu) void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu) { void *sve_state =3D vcpu->arch.sve_state; + void *sme_state =3D vcpu->arch.sme_state; =20 kvm_unshare_hyp(vcpu, vcpu + 1); if (sve_state) kvm_unshare_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); kfree(sve_state); free_page((unsigned long)vcpu->arch.ctxt.vncr_array); + if (sme_state) + kvm_unshare_hyp(sme_state, sme_state + vcpu_sme_state_size(vcpu)); + kfree(sme_state); kfree(vcpu->arch.vncr_tlb); kfree(vcpu->arch.ccsidr); } @@ -167,6 +243,8 @@ static void kvm_vcpu_reset_vec(struct kvm_vcpu *vcpu) { if (vcpu_has_sve(vcpu)) memset(vcpu->arch.sve_state, 0, vcpu_sve_state_size(vcpu)); + if (vcpu_has_sme(vcpu)) + memset(vcpu->arch.sme_state, 0, vcpu_sme_state_size(vcpu)); } =20 /** @@ -206,6 +284,8 @@ void kvm_reset_vcpu(struct kvm_vcpu *vcpu) if (!kvm_arm_vcpu_vec_finalized(vcpu)) { if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) kvm_vcpu_enable_sve(vcpu); + if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SME)) + kvm_vcpu_enable_sme(vcpu); } else { kvm_vcpu_reset_vec(vcpu); } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index f0f0d49d2544..8aacbc837b6e 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -962,6 +962,7 @@ struct kvm_enable_cap { #define KVM_CAP_ARM_EL2_E2H0 241 #define KVM_CAP_RISCV_MP_STATE_RESET 242 #define KVM_CAP_ARM_CACHEABLE_PFNMAP_SUPPORTED 243 +#define KVM_CAP_ARM_SME 244 =20 struct kvm_irq_routing_irqchip { __u32 irqchip; --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9AA3311C39; Tue, 2 Sep 2025 11:46:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813584; cv=none; b=hnLwdIwgdXujzo6gU4jEzSxhlDy3OKSB9PnzEO8xZ79r5dA6zPYF/xeEoaXRh8PpFHf3mFDY0MfcRBfPrJoDaBEC77cnBdXYyBqpAkoXkAr9GJe5PVf0OJrykKUsdX2g7H5inwOj+MzPR512la/ZYrSviHO34IJRap8eT9iDfzY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813584; c=relaxed/simple; bh=ZvTUw7NPQ0p4ee6PuzgOvjpUagMHBpEvC3cvgofAwS0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bmdFKt0ZK+BsNDjbEmmqZTFhG929hxVDUYjnPUXir63TY1zWuT92WRtB3AX64M0GPmhBjZiXjxs7y8Xv5FpL4xlXVJq8VXAs6wi2MADLiSlXLp5obEKjLDGdbBxkcJqs7BI0ZMgj59SihMbW7/qa1gqVHG09VcvL+Y6ojk8XdCQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bc4vHW5j; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bc4vHW5j" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D89B9C4CEF7; Tue, 2 Sep 2025 11:46:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813584; bh=ZvTUw7NPQ0p4ee6PuzgOvjpUagMHBpEvC3cvgofAwS0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=bc4vHW5j0aU/ug6ixQ7oHK3aJua06rmW47TykIiel63W16Bgpu/Nz3UjG7O3D+mKW 4S3KTeIVDTzSiFnmdW4lRV1eM5vNKOC0JO9LJmQNOJxyZgfCflIaT+d8ijeM+KSXVx KWTAZgMzk216ydJLEuq0Y0Cig3EhRZgTGmrZ0o1hfDul4TIncooiTq97n5GOrCvzPO RgohlY77yiPyp6spAJaxj7eGqUHqH89ZxkF6zZi5X+LOKe1vXAZ1TSZAOOER4+zvDV t7wTB9Q2BDciSM6r+xyfYfG4KgrcZ1Tf+Nu+Oha2kxP4u4PKnGQFpxRMFJ/G277pUc 2DBw2R4k0IVog== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:31 +0100 Subject: [PATCH v8 28/29] KVM: arm64: selftests: Add SME system registers to get-reg-list Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-28-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=3013; i=broonie@kernel.org; h=from:subject:message-id; bh=ZvTUw7NPQ0p4ee6PuzgOvjpUagMHBpEvC3cvgofAwS0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBottiP+BCS2rKN2diVP5zrkIThPt+KUFpUrWdiS H6AIXh56HqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCaLbYjwAKCRAk1otyXVSH 0CvVB/9qsmPU1KQh+8uEw3/lbl3fbYfi9MErtSdNtzyMMXbTaIqDlW6l+ODst140lnzuCkHERh2 oJbA+C+/peIm/+8m+uutO0tjlR7nBhhnrQoon8jaojMtb+PgSgJmJe+yTnAWbxjwScE3Z2sf85H udBrYF3KTFxZo+XuxcwnwciDYsC9Alep1/mz5sVZ/lrhXaIlaSYKhByM0L1+mhEAORzJ6o9muro ZzSIb73EJ9KeB32TzlRcLi9n5Mg14s7nXzEEMk3+aiwZ8sY6Xo/Pw8NGAzNOlurSxBGlSztfHIy 5q6+uISsTSpYJIq90iWXBobce9Ngb3+F8vgYCBKEx8RxTRaB X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME adds a number of new system registers, update get-reg-list to check for them based on the visibility of SME. Signed-off-by: Mark Brown --- tools/testing/selftests/kvm/arm64/get-reg-list.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/arm64/get-reg-list.c b/tools/testi= ng/selftests/kvm/arm64/get-reg-list.c index 011fad95dd02..54b371330f57 100644 --- a/tools/testing/selftests/kvm/arm64/get-reg-list.c +++ b/tools/testing/selftests/kvm/arm64/get-reg-list.c @@ -61,7 +61,13 @@ static struct feature_id_reg feat_id_regs[] =3D { REG_FEAT(HFGITR2_EL2, ID_AA64MMFR0_EL1, FGT, FGT2), REG_FEAT(HDFGRTR2_EL2, ID_AA64MMFR0_EL1, FGT, FGT2), REG_FEAT(HDFGWTR2_EL2, ID_AA64MMFR0_EL1, FGT, FGT2), - REG_FEAT(ZCR_EL2, ID_AA64PFR0_EL1, SVE, IMP), + REG_FEAT(SMCR_EL1, ID_AA64PFR1_EL1, SME, IMP), + REG_FEAT(SMCR_EL2, ID_AA64PFR1_EL1, SME, IMP), + REG_FEAT(SMIDR_EL1, ID_AA64PFR1_EL1, SME, IMP), + REG_FEAT(SMPRI_EL1, ID_AA64PFR1_EL1, SME, IMP), + REG_FEAT(SMPRIMAP_EL2, ID_AA64PFR1_EL1, SME, IMP), + REG_FEAT(TPIDR2_EL0, ID_AA64PFR1_EL1, SME, IMP), + REG_FEAT(SVCR, ID_AA64PFR1_EL1, SME, IMP), REG_FEAT(SCTLR2_EL1, ID_AA64MMFR3_EL1, SCTLRX, IMP), REG_FEAT(VDISR_EL2, ID_AA64PFR0_EL1, RAS, IMP), REG_FEAT(VSESR_EL2, ID_AA64PFR0_EL1, RAS, IMP), @@ -351,6 +357,7 @@ static __u64 base_regs[] =3D { ARM64_SYS_REG(3, 0, 0, 0, 0), /* MIDR_EL1 */ ARM64_SYS_REG(3, 0, 0, 0, 6), /* REVIDR_EL1 */ ARM64_SYS_REG(3, 1, 0, 0, 1), /* CLIDR_EL1 */ + ARM64_SYS_REG(3, 1, 0, 0, 6), /* SMIDR_EL1 */ ARM64_SYS_REG(3, 1, 0, 0, 7), /* AIDR_EL1 */ ARM64_SYS_REG(3, 3, 0, 0, 1), /* CTR_EL0 */ ARM64_SYS_REG(2, 0, 0, 0, 4), @@ -482,6 +489,8 @@ static __u64 base_regs[] =3D { ARM64_SYS_REG(3, 0, 1, 0, 1), /* ACTLR_EL1 */ ARM64_SYS_REG(3, 0, 1, 0, 2), /* CPACR_EL1 */ KVM_ARM64_SYS_REG(SYS_SCTLR2_EL1), + ARM64_SYS_REG(3, 0, 1, 2, 4), /* SMPRI_EL1 */ + ARM64_SYS_REG(3, 0, 1, 2, 6), /* SMCR_EL1 */ ARM64_SYS_REG(3, 0, 2, 0, 0), /* TTBR0_EL1 */ ARM64_SYS_REG(3, 0, 2, 0, 1), /* TTBR1_EL1 */ ARM64_SYS_REG(3, 0, 2, 0, 2), /* TCR_EL1 */ @@ -502,9 +511,11 @@ static __u64 base_regs[] =3D { ARM64_SYS_REG(3, 0, 13, 0, 4), /* TPIDR_EL1 */ ARM64_SYS_REG(3, 0, 14, 1, 0), /* CNTKCTL_EL1 */ ARM64_SYS_REG(3, 2, 0, 0, 0), /* CSSELR_EL1 */ + ARM64_SYS_REG(3, 3, 4, 2, 2), /* SVCR */ ARM64_SYS_REG(3, 3, 10, 2, 4), /* POR_EL0 */ ARM64_SYS_REG(3, 3, 13, 0, 2), /* TPIDR_EL0 */ ARM64_SYS_REG(3, 3, 13, 0, 3), /* TPIDRRO_EL0 */ + ARM64_SYS_REG(3, 3, 13, 0, 5), /* TPIDR2_EL0 */ ARM64_SYS_REG(3, 3, 14, 0, 1), /* CNTPCT_EL0 */ ARM64_SYS_REG(3, 3, 14, 2, 1), /* CNTP_CTL_EL0 */ ARM64_SYS_REG(3, 3, 14, 2, 2), /* CNTP_CVAL_EL0 */ @@ -713,6 +724,8 @@ static __u64 el2_regs[] =3D { SYS_REG(HFGITR_EL2), SYS_REG(HACR_EL2), SYS_REG(ZCR_EL2), + SYS_REG(SMPRIMAP_EL2), + SYS_REG(SMCR_EL2), SYS_REG(HCRX_EL2), SYS_REG(TTBR0_EL2), SYS_REG(TTBR1_EL2), --=20 2.39.5 From nobody Fri Oct 3 10:11:15 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E047E2FD7BF; Tue, 2 Sep 2025 11:46:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813589; cv=none; b=fcWVLDcSTq+w6nPBXhEduo8N1Q8UOWIa0TpglpsxBTwUPM0mEt+V6bhXvrYQTOvUdUJhGQ8F7h2XOqh53W2NTPNydf9YWPlBpwxsO6BnQKt+htd+h/FEhQTkHFM2i0ZvH7kXHRgpoIdnm9TG0zNZV5dwhLUWh24vikUfakTzj7g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756813589; c=relaxed/simple; bh=nitp/ehKgCus2GLbBQJBw9hgUMhrPQICaIiiL3f5Pj4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=miuNTPASiX3H/ROEHFMudgQ3xOoyP+QBFkFlRKQ2HlUB1rn8GuQXIDrGfrDpZY3P/+CKMp5fHjoyR+9s9R9+a9tOCSguyoqrmZxSTGErQjIyCBxYh/xEWEhSsLcL6p4P4Xj9A9Ol0uCoE+sMVcy7CLomradrtAJ88JSXH7WPR2I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mteyJTi4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mteyJTi4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F138C4CEED; Tue, 2 Sep 2025 11:46:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756813588; bh=nitp/ehKgCus2GLbBQJBw9hgUMhrPQICaIiiL3f5Pj4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=mteyJTi4KgOZSPIMQkhNh5JNgR82tE2sd3dR4ckNiBIjapz2PaGQ+qKZa+vpN4miR GitVMWXvlojCrBGCIFo/ug8TqdsgBLI5h7eai8ET8x3Jvy6IUFMHpu0+OAZbYqBh9M rok49FFSc8bsKoffqoRzTBPnsHWUabRdBP7f/b5uHV8smpwOYYy5PIIWyPpyW/r6xd gU5V7CC5QHbDs40v7m4SN5SlLhQLT4q3yBOtDY1pE2M9W7Ee+Q9VWDu536lh1uDfrZ 6vlDmxoezOat/sojpiyFVvWvAls2dyFejQW9szrQtz+DhFmBl9n9Y5+2UeBX7V/WKT OB1LN8wPHDE0w== From: Mark Brown Date: Tue, 02 Sep 2025 12:36:32 +0100 Subject: [PATCH v8 29/29] KVM: arm64: selftests: Add SME to set_id_regs test Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250902-kvm-arm64-sme-v8-29-2cb2199c656c@kernel.org> References: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> In-Reply-To: <20250902-kvm-arm64-sme-v8-0-2cb2199c656c@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Maydell , Eric Auger , Mark Brown X-Mailer: b4 0.15-dev-cff91 X-Developer-Signature: v=1; a=openpgp-sha256; l=3326; i=broonie@kernel.org; h=from:subject:message-id; bh=nitp/ehKgCus2GLbBQJBw9hgUMhrPQICaIiiL3f5Pj4=; b=owGbwMvMwMWocq27KDak/QLjabUkhoxtNyb4ca8+OKcvTa5ZVLXtgXt0W490XMG2T8cv/+9N+ xeg9jiwk9GYhYGRi0FWTJFl7bOMVenhElvnP5r/CmYQKxPIFAYuTgGYyLpDHAwN8y/HMatZMEzy 7prVXRXwQeeRbf1nnSP7RWdVfHnXLM45Y/OZbqNlq2bZ/tZ4cvH5duOcQvHPHsnrfD75dUhN3Pm wOfg+SzDXXon4Uz1KGU7RJ2xX3OyTe96317RJNVLs8W+tJIeDX9PCs3bPr/hzvmRC7sMNTn98tE rEfa7riblnzrOqO2sVWpT19fYxdhXZ93PLTU26uXeyTT63Yu2T1z6Od3TONpaFhXoHHLL29anWD tXUUq3IaprTIdR3Iuq3x97HHNPCm+MF1sz0SXxX1mXx1TLqy9rCF0XS3lEzZt3lNDojfP7ujny/ ln+GSUtnTjrSdNq+XtFWwEap5a2Fk9G3OJbDrLodsrM6AA== X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add coverage of the SME ID registers to set_id_regs, ID_AA64PFR1_EL1.SME becomes writable and we add ID_AA64SMFR_EL1 and it's subfields. Signed-off-by: Mark Brown --- tools/testing/selftests/kvm/arm64/set_id_regs.c | 27 +++++++++++++++++++++= +++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/arm64/set_id_regs.c b/tools/testin= g/selftests/kvm/arm64/set_id_regs.c index d3bf9204409c..f3f15145aa69 100644 --- a/tools/testing/selftests/kvm/arm64/set_id_regs.c +++ b/tools/testing/selftests/kvm/arm64/set_id_regs.c @@ -196,6 +196,28 @@ static const struct reg_ftr_bits ftr_id_aa64mmfr3_el1[= ] =3D { REG_FTR_END, }; =20 +static const struct reg_ftr_bits ftr_id_aa64smfr0_el1[] =3D { + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, FA64, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, LUTv2, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SMEver, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, I16I64, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F64F64, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, I16I32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, B16B16, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F16F16, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F8F16, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F8F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, I8I32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F16F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, B16F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, BI32I32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F32F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SF8FMA, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SF8DP4, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SF8DP2, 0), + REG_FTR_END, +}; + static const struct reg_ftr_bits ftr_id_aa64zfr0_el1[] =3D { REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ZFR0_EL1, F64MM, 0), REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ZFR0_EL1, F32MM, 0), @@ -227,6 +249,7 @@ static struct test_feature_reg test_regs[] =3D { TEST_REG(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1_el1), TEST_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2_el1), TEST_REG(SYS_ID_AA64MMFR3_EL1, ftr_id_aa64mmfr3_el1), + TEST_REG(SYS_ID_AA64SMFR0_EL1, ftr_id_aa64smfr0_el1), TEST_REG(SYS_ID_AA64ZFR0_EL1, ftr_id_aa64zfr0_el1), }; =20 @@ -243,6 +266,7 @@ static void guest_code(void) GUEST_REG_SYNC(SYS_ID_AA64MMFR0_EL1); GUEST_REG_SYNC(SYS_ID_AA64MMFR1_EL1); GUEST_REG_SYNC(SYS_ID_AA64MMFR2_EL1); + GUEST_REG_SYNC(SYS_ID_AA64SMFR0_EL1); GUEST_REG_SYNC(SYS_ID_AA64ZFR0_EL1); GUEST_REG_SYNC(SYS_CTR_EL0); GUEST_REG_SYNC(SYS_MIDR_EL1); @@ -784,7 +808,8 @@ int main(void) ARRAY_SIZE(ftr_id_aa64isar2_el1) + ARRAY_SIZE(ftr_id_aa64pfr0_el1) + ARRAY_SIZE(ftr_id_aa64pfr1_el1) + ARRAY_SIZE(ftr_id_aa64mmfr0_el1) + ARRAY_SIZE(ftr_id_aa64mmfr1_el1) + ARRAY_SIZE(ftr_id_aa64mmfr2_el1) + - ARRAY_SIZE(ftr_id_aa64mmfr3_el1) + ARRAY_SIZE(ftr_id_aa64zfr0_el1) - + ARRAY_SIZE(ftr_id_aa64mmfr3_el1) + ARRAY_SIZE(ftr_id_aa64smfr0_el1) + + ARRAY_SIZE(ftr_id_aa64zfr0_el1) - ARRAY_SIZE(test_regs) + 3 + MPAM_IDREG_TEST + MTE_IDREG_TEST; =20 ksft_set_plan(test_cnt); --=20 2.39.5