From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA4BCC61DA4 for ; Thu, 16 Feb 2023 16:01:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229598AbjBPQBj (ORCPT ); Thu, 16 Feb 2023 11:01:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229462AbjBPQBc (ORCPT ); Thu, 16 Feb 2023 11:01:32 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 68F8256487 for ; Thu, 16 Feb 2023 08:01:30 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D7CF3113E; Thu, 16 Feb 2023 08:02:12 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D83E83F881; Thu, 16 Feb 2023 08:01:26 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 01/10] KVM: arm64: initialize HCRX_EL2 Date: Thu, 16 Feb 2023 16:00:03 +0000 Message-Id: <20230216160012.272345-2-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" ARMv8.7/9.2 adds a new hypervisor configuration register HCRX_EL2. Initialize the register to a safe value (all fields 0), to be robust against firmware that has not initialized it. This is also needed to ensure that the register is reinitialized after kexec. In addition, move SMPME setup over to the new flags, as it would otherwise get overridden. It is safe to set the bit even if SME is not (uniformly) supported, as it will write to a RES0 bit (having no effect), and SME will be disabled by the cpufeature framework. (Similar to how e.g. the API bit is handled in HCR_HOST_NVHE_FLAGS.) Signed-off-by: Kristina Martsenko --- arch/arm64/include/asm/el2_setup.h | 8 -------- arch/arm64/include/asm/kvm_arm.h | 3 +++ arch/arm64/kernel/head.S | 7 +++++++ arch/arm64/kvm/hyp/nvhe/hyp-init.S | 6 ++++++ 4 files changed, 16 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el= 2_setup.h index e62785923ff6..699154229b15 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -284,14 +284,6 @@ cbz x1, .Lskip_sme_\@ =20 msr_s SYS_SMPRIMAP_EL2, xzr // Make all priorities equal - - mrs x1, id_aa64mmfr1_el1 // HCRX_EL2 present? - ubfx x1, x1, #ID_AA64MMFR1_EL1_HCX_SHIFT, #4 - cbz x1, .Lskip_sme_\@ - - mrs_s x1, SYS_HCRX_EL2 - orr x1, x1, #HCRX_EL2_SMPME_MASK // Enable priority mapping - msr_s SYS_HCRX_EL2, x1 .Lskip_sme_\@: .endm =20 diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_= arm.h index 5a4b2342d571..caa31f4ab1cd 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -9,6 +9,7 @@ =20 #include #include +#include #include =20 /* Hyp Configuration Register (HCR) bits */ @@ -92,6 +93,8 @@ #define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC) #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) =20 +#define HCRX_HOST_FLAGS (HCRX_EL2_SMPME) + /* TCR_EL2 Registers bits */ #define TCR_EL2_RES1 ((1U << 31) | (1 << 23)) #define TCR_EL2_TBI (1 << 20) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 212d93aca5e6..e06b34322339 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -572,6 +572,13 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) msr hcr_el2, x0 isb =20 + mrs x0, ID_AA64MMFR1_EL1 + ubfx x0, x0, #ID_AA64MMFR1_EL1_HCX_SHIFT, #4 + cbz x0, 3f + mov_q x1, HCRX_HOST_FLAGS + msr_s SYS_HCRX_EL2, x1 + isb +3: init_el2_state =20 /* Hypervisor stub */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/h= yp-init.S index a6d67c2bb5ae..01f854697c70 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -95,6 +95,12 @@ SYM_CODE_START_LOCAL(___kvm_hyp_init) ldr x1, [x0, #NVHE_INIT_HCR_EL2] msr hcr_el2, x1 =20 + mrs x1, ID_AA64MMFR1_EL1 + ubfx x1, x1, #ID_AA64MMFR1_EL1_HCX_SHIFT, #4 + cbz x1, 1f + mov_q x2, HCRX_HOST_FLAGS + msr_s SYS_HCRX_EL2, x2 +1: ldr x1, [x0, #NVHE_INIT_VTTBR] msr vttbr_el2, x1 =20 --=20 2.25.1 From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F9D0C61DA4 for ; Thu, 16 Feb 2023 16:01:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229611AbjBPQBn (ORCPT ); Thu, 16 Feb 2023 11:01:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229630AbjBPQBg (ORCPT ); Thu, 16 Feb 2023 11:01:36 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5CD5C5355B for ; Thu, 16 Feb 2023 08:01:34 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D68F1168F; Thu, 16 Feb 2023 08:02:16 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 14FAD3F881; Thu, 16 Feb 2023 08:01:30 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 02/10] arm64: cpufeature: detect FEAT_HCX Date: Thu, 16 Feb 2023 16:00:04 +0000 Message-Id: <20230216160012.272345-3-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Detect if the system has the new HCRX_EL2 register added in ARMv8.7/9.2, so that subsequent patches can check for its presence. Signed-off-by: Kristina Martsenko Reviewed-by: Catalin Marinas --- arch/arm64/kernel/cpufeature.c | 12 ++++++++++++ arch/arm64/tools/cpucaps | 1 + 2 files changed, 13 insertions(+) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 45a42cf2191c..1b0a71541381 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -357,6 +357,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = =3D { static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] =3D { ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_EL= 1_TIDCP1_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_EL1_= AFP_SHIFT, 4, 0), + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_EL1_H= CX_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_EL1_E= TS_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_EL1_T= WED_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_EL1_X= NX_SHIFT, 4, 0), @@ -2290,6 +2291,17 @@ static const struct arm64_cpu_capabilities arm64_fea= tures[] =3D { .type =3D ARM64_CPUCAP_SYSTEM_FEATURE, .matches =3D is_kvm_protected_mode, }, + { + .desc =3D "HCRX_EL2 register", + .capability =3D ARM64_HAS_HCX, + .type =3D ARM64_CPUCAP_SYSTEM_FEATURE, + .sys_reg =3D SYS_ID_AA64MMFR1_EL1, + .sign =3D FTR_UNSIGNED, + .field_pos =3D ID_AA64MMFR1_EL1_HCX_SHIFT, + .field_width =3D 4, + .min_field_value =3D ID_AA64MMFR1_EL1_HCX_IMP, + .matches =3D has_cpuid_feature, + }, #endif { .desc =3D "Kernel page table isolation (KPTI)", diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index 10dcfa13390a..bf32a1122c04 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -31,6 +31,7 @@ HAS_GENERIC_AUTH_IMP_DEF HAS_GIC_CPUIF_SYSREGS HAS_GIC_PRIO_MASKING HAS_GIC_PRIO_RELAXED_SYNC +HAS_HCX HAS_LDAPR HAS_LSE_ATOMICS HAS_NO_FPSIMD --=20 2.25.1 From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C705C636D7 for ; Thu, 16 Feb 2023 16:01:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229800AbjBPQBq (ORCPT ); Thu, 16 Feb 2023 11:01:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229672AbjBPQBl (ORCPT ); Thu, 16 Feb 2023 11:01:41 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 4A39454548 for ; Thu, 16 Feb 2023 08:01:38 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 127B91042; Thu, 16 Feb 2023 08:02:21 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 37A003F881; Thu, 16 Feb 2023 08:01:35 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 03/10] KVM: arm64: switch HCRX_EL2 between host and guest Date: Thu, 16 Feb 2023 16:00:05 +0000 Message-Id: <20230216160012.272345-4-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Switch the HCRX_EL2 register between host and guest configurations, in order to enable different features in the host and guest. Note that the guest flags are only set if all CPUs have HCRX_EL2. Asymmetric systems where only some CPUs have HCRX_EL2 are not supported and will result in guests running with the host flags set (and a "SANITY CHECK" warning printed for the host). After this change, SMPME is no longer set for guests, which should have no effect as SME is currently disabled for guests. Signed-off-by: Kristina Martsenko --- I wasn't sure what to do about asymmetric systems. It seems a bit fragile, maybe someone has a better idea? arch/arm64/include/asm/kvm_arm.h | 1 + arch/arm64/kvm/hyp/include/hyp/switch.h | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_= arm.h index caa31f4ab1cd..cd8dd307aaba 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -93,6 +93,7 @@ #define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC) #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) =20 +#define HCRX_GUEST_FLAGS 0 #define HCRX_HOST_FLAGS (HCRX_EL2_SMPME) =20 /* TCR_EL2 Registers bits */ diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/i= nclude/hyp/switch.h index 07d37ff88a3f..a1bf2d879db5 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -129,6 +129,9 @@ static inline void ___activate_traps(struct kvm_vcpu *v= cpu) =20 if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN) && (hcr & HCR_VSE)) write_sysreg_s(vcpu->arch.vsesr_el2, SYS_VSESR_EL2); + + if (cpus_have_final_cap(ARM64_HAS_HCX)) + write_sysreg_s(HCRX_GUEST_FLAGS, SYS_HCRX_EL2); } =20 static inline void ___deactivate_traps(struct kvm_vcpu *vcpu) @@ -143,6 +146,9 @@ static inline void ___deactivate_traps(struct kvm_vcpu = *vcpu) vcpu->arch.hcr_el2 &=3D ~HCR_VSE; vcpu->arch.hcr_el2 |=3D read_sysreg(hcr_el2) & HCR_VSE; } + + if (cpus_have_final_cap(ARM64_HAS_HCX)) + write_sysreg_s(HCRX_HOST_FLAGS, SYS_HCRX_EL2); } =20 static inline bool __populate_fault_info(struct kvm_vcpu *vcpu) --=20 2.25.1 From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88E31C61DA4 for ; Thu, 16 Feb 2023 16:01:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229947AbjBPQBz (ORCPT ); Thu, 16 Feb 2023 11:01:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229608AbjBPQBv (ORCPT ); Thu, 16 Feb 2023 11:01:51 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id D568D59B78 for ; Thu, 16 Feb 2023 08:01:42 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 432C5113E; Thu, 16 Feb 2023 08:02:25 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5D02B3F881; Thu, 16 Feb 2023 08:01:39 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 04/10] arm64: mops: document boot requirements for MOPS Date: Thu, 16 Feb 2023 16:00:06 +0000 Message-Id: <20230216160012.272345-5-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" FEAT_MOPS introduces new instructions, we require that these instructions not execute as UNDEFINED when we identify that the feature is supported. Additionally, the instructions may generate a Memory Copy or Memory Set exception, we require that the exception be taken to EL1 in case EL2 is not able to correctly handle it. Signed-off-by: Kristina Martsenko --- Documentation/arm64/booting.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/Documentation/arm64/booting.rst b/Documentation/arm64/booting.= rst index ffeccdd6bdac..f3913ee94c42 100644 --- a/Documentation/arm64/booting.rst +++ b/Documentation/arm64/booting.rst @@ -379,6 +379,14 @@ Before jumping into the kernel, the following conditio= ns must be met: =20 - SMCR_EL2.EZT0 (bit 30) must be initialised to 0b1. =20 + For CPUs with Memory Copy and Memory Set instructions (FEAT_MOPS): + + - If the kernel is entered at EL1 and EL2 is present: + + - HCRX_EL2.MSCEn (bit 11) must be initialised to 0b1. + + - HCRX_EL2.MCE2 (bit 10) must be initialised to 0b0. + The requirements described above for CPU mode, caches, MMUs, architected timers, coherency and system registers apply to all CPUs. All CPUs must enter the kernel in the same exception level. Where the values documented --=20 2.25.1 From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3572AC61DA4 for ; Thu, 16 Feb 2023 16:02:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229877AbjBPQB6 (ORCPT ); Thu, 16 Feb 2023 11:01:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59648 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229719AbjBPQBv (ORCPT ); Thu, 16 Feb 2023 11:01:51 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C45AD564A8 for ; Thu, 16 Feb 2023 08:01:46 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 41809168F; Thu, 16 Feb 2023 08:02:29 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7424F3F881; Thu, 16 Feb 2023 08:01:43 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 05/10] arm64: mops: don't disable host MOPS instructions from EL2 Date: Thu, 16 Feb 2023 16:00:07 +0000 Message-Id: <20230216160012.272345-6-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To allow nVHE host EL0 and EL1 to use FEAT_MOPS instructions, configure EL2 to not cause these instructions to be treated as UNDEFINED. A VHE host is unaffected by this control. Signed-off-by: Kristina Martsenko Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/kvm_arm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_= arm.h index cd8dd307aaba..f37d37957322 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -94,7 +94,7 @@ #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) =20 #define HCRX_GUEST_FLAGS 0 -#define HCRX_HOST_FLAGS (HCRX_EL2_SMPME) +#define HCRX_HOST_FLAGS (HCRX_EL2_SMPME | HCRX_EL2_MSCEn) =20 /* TCR_EL2 Registers bits */ #define TCR_EL2_RES1 ((1U << 31) | (1 << 23)) --=20 2.25.1 From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B21AC61DA4 for ; Thu, 16 Feb 2023 16:02:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230070AbjBPQCG (ORCPT ); Thu, 16 Feb 2023 11:02:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229658AbjBPQBy (ORCPT ); Thu, 16 Feb 2023 11:01:54 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9A4B9564B7 for ; Thu, 16 Feb 2023 08:01:50 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0DD0A1042; Thu, 16 Feb 2023 08:02:33 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6E66D3F881; Thu, 16 Feb 2023 08:01:47 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 06/10] KVM: arm64: hide MOPS from guests Date: Thu, 16 Feb 2023 16:00:08 +0000 Message-Id: <20230216160012.272345-7-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" As FEAT_MOPS is not supported in guests yet, hide it from the ID registers for guests. The MOPS instructions are UNDEFINED in guests as HCRX_EL2.MSCEn is not set in HCRX_GUEST_FLAGS, and will take an exception to EL1 if executed. Signed-off-by: Kristina Martsenko Acked-by: Catalin Marinas --- arch/arm64/kvm/sys_regs.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 0bc7df55916e..3456205c6b92 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1213,6 +1213,7 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu, s= truct sys_reg_desc const *r ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_GPA3)); if (!cpus_have_final_cap(ARM64_HAS_WFXT)) val &=3D ~ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_WFxT); + val &=3D ~ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_MOPS); break; case SYS_ID_AA64DFR0_EL1: /* Limit debug to ARMv8.0 */ --=20 2.25.1 From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CB43C61DA4 for ; Thu, 16 Feb 2023 16:02:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229490AbjBPQCR (ORCPT ); Thu, 16 Feb 2023 11:02:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60054 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229921AbjBPQCD (ORCPT ); Thu, 16 Feb 2023 11:02:03 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 700BF56EFE for ; Thu, 16 Feb 2023 08:01:54 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E7996113E; Thu, 16 Feb 2023 08:02:36 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 25CE73F881; Thu, 16 Feb 2023 08:01:51 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 07/10] arm64: mops: handle MOPS exceptions Date: Thu, 16 Feb 2023 16:00:09 +0000 Message-Id: <20230216160012.272345-8-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The memory copy/set instructions added as part of FEAT_MOPS can take an exception part-way through their execution and resume execution afterwards. If however the task is re-scheduled and execution resumes on a different CPU, then the CPU may take a new type of exception to indicate this. In this case the OS has to reset the registers and restart execution from the prologue instruction. The algorithm for doing this is provided as part of the Arm ARM. Add an exception handler for the new exception and wire it up for userspace tasks. Signed-off-by: Kristina Martsenko Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/esr.h | 11 +++++++- arch/arm64/include/asm/exception.h | 1 + arch/arm64/kernel/entry-common.c | 11 ++++++++ arch/arm64/kernel/traps.c | 45 ++++++++++++++++++++++++++++++ 4 files changed, 67 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index c9f15b9e3c71..96caaaee97a3 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -47,7 +47,7 @@ #define ESR_ELx_EC_DABT_LOW (0x24) #define ESR_ELx_EC_DABT_CUR (0x25) #define ESR_ELx_EC_SP_ALIGN (0x26) -/* Unallocated EC: 0x27 */ +#define ESR_ELx_EC_MOPS (0x27) #define ESR_ELx_EC_FP_EXC32 (0x28) /* Unallocated EC: 0x29 - 0x2B */ #define ESR_ELx_EC_FP_EXC64 (0x2C) @@ -352,6 +352,15 @@ #define ESR_ELx_SME_ISS_ZA_DISABLED 3 #define ESR_ELx_SME_ISS_ZT_DISABLED 4 =20 +/* ISS field definitions for MOPS exceptions */ +#define ESR_ELx_MOPS_ISS_MEM_INST (UL(1) << 24) +#define ESR_ELx_MOPS_ISS_FROM_EPILOGUE (UL(1) << 18) +#define ESR_ELx_MOPS_ISS_WRONG_OPTION (UL(1) << 17) +#define ESR_ELx_MOPS_ISS_OPTION_A (UL(1) << 16) +#define ESR_ELx_MOPS_ISS_DESTREG(esr) (((esr) & (UL(0x1f) << 10)) >> 10) +#define ESR_ELx_MOPS_ISS_SRCREG(esr) (((esr) & (UL(0x1f) << 5)) >> 5) +#define ESR_ELx_MOPS_ISS_SIZEREG(esr) (((esr) & (UL(0x1f) << 0)) >> 0) + #ifndef __ASSEMBLY__ #include =20 diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/ex= ception.h index 92963f98afec..5a6dc3643e9b 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -77,6 +77,7 @@ void do_el0_svc(struct pt_regs *regs); void do_el0_svc_compat(struct pt_regs *regs); void do_el0_fpac(struct pt_regs *regs, unsigned long esr); void do_el1_fpac(struct pt_regs *regs, unsigned long esr); +void do_el0_mops(struct pt_regs *regs, unsigned long esr); void do_serror(struct pt_regs *regs, unsigned long esr); void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags); =20 diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-com= mon.c index cce1167199e3..2ef3ab5d7555 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -611,6 +611,14 @@ static void noinstr el0_bti(struct pt_regs *regs) exit_to_user_mode(regs); } =20 +static void noinstr el0_mops(struct pt_regs *regs, unsigned long esr) +{ + enter_from_user_mode(regs); + local_daif_restore(DAIF_PROCCTX); + do_el0_mops(regs, esr); + exit_to_user_mode(regs); +} + static void noinstr el0_inv(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); @@ -688,6 +696,9 @@ asmlinkage void noinstr el0t_64_sync_handler(struct pt_= regs *regs) case ESR_ELx_EC_BTI: el0_bti(regs); break; + case ESR_ELx_EC_MOPS: + el0_mops(regs, esr); + break; case ESR_ELx_EC_BREAKPT_LOW: case ESR_ELx_EC_SOFTSTP_LOW: case ESR_ELx_EC_WATCHPT_LOW: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 0ccc063daccb..689188712909 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -507,6 +507,50 @@ void do_el1_fpac(struct pt_regs *regs, unsigned long e= sr) die("Oops - FPAC", regs, esr); } =20 +void do_el0_mops(struct pt_regs *regs, unsigned long esr) +{ + bool wrong_option =3D esr & ESR_ELx_MOPS_ISS_WRONG_OPTION; + bool option_a =3D esr & ESR_ELx_MOPS_ISS_OPTION_A; + int dstreg =3D ESR_ELx_MOPS_ISS_DESTREG(esr); + int srcreg =3D ESR_ELx_MOPS_ISS_SRCREG(esr); + int sizereg =3D ESR_ELx_MOPS_ISS_SIZEREG(esr); + unsigned long dst, src, size; + + dst =3D pt_regs_read_reg(regs, dstreg); + src =3D pt_regs_read_reg(regs, srcreg); + size =3D pt_regs_read_reg(regs, sizereg); + + /* + * Put the registers back in the original format suitable for a + * prologue instruction, using the generic return routine from the + * Arm ARM (DDI 0487I.a) rules CNTMJ and MWFQH. + */ + if (esr & ESR_ELx_MOPS_ISS_MEM_INST) { + if ((!option_a && wrong_option) || (option_a && !wrong_option)) { + pt_regs_write_reg(regs, dstreg, dst + size); + pt_regs_write_reg(regs, sizereg, -size); + } + } else { + if ((option_a && wrong_option) || (!option_a && !wrong_option)) { + if (regs->pstate & PSR_N_BIT) { + pt_regs_write_reg(regs, dstreg, dst - size); + pt_regs_write_reg(regs, srcreg, src - size); + } + } else { + if (size & BIT(63)) { + pt_regs_write_reg(regs, dstreg, dst + size); + pt_regs_write_reg(regs, srcreg, src + size); + pt_regs_write_reg(regs, sizereg, -size); + } + } + } + + if (esr & ESR_ELx_MOPS_ISS_FROM_EPILOGUE) + regs->pc -=3D 8; + else + regs->pc -=3D 4; +} + #define __user_cache_maint(insn, address, res) \ if (address >=3D TASK_SIZE_MAX) { \ res =3D -EFAULT; \ @@ -817,6 +861,7 @@ static const char *esr_class_str[] =3D { [ESR_ELx_EC_DABT_LOW] =3D "DABT (lower EL)", [ESR_ELx_EC_DABT_CUR] =3D "DABT (current EL)", [ESR_ELx_EC_SP_ALIGN] =3D "SP Alignment", + [ESR_ELx_EC_MOPS] =3D "MOPS", [ESR_ELx_EC_FP_EXC32] =3D "FP (AArch32)", [ESR_ELx_EC_FP_EXC64] =3D "FP (AArch64)", [ESR_ELx_EC_SERROR] =3D "SError", --=20 2.25.1 From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 719A7C636CC for ; Thu, 16 Feb 2023 16:02:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229653AbjBPQC0 (ORCPT ); Thu, 16 Feb 2023 11:02:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229820AbjBPQCO (ORCPT ); Thu, 16 Feb 2023 11:02:14 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B738585F5E for ; Thu, 16 Feb 2023 08:01:58 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3CBD21042; Thu, 16 Feb 2023 08:02:41 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3DA033F881; Thu, 16 Feb 2023 08:01:55 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 08/10] arm64: mops: handle single stepping after MOPS exception Date: Thu, 16 Feb 2023 16:00:10 +0000 Message-Id: <20230216160012.272345-9-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When a MOPS main or epilogue instruction is being executed, the task may get scheduled on a different CPU and restart execution from the prologue instruction. If the main or epilogue instruction is being single stepped then it makes sense to finish the step and take the step exception before starting to execute the next (prologue) instruction. So fast-forward the single step state machine when taking a MOPS exception. This means that if a main or epilogue instruction is single stepped with ptrace, the debugger will sometimes observe the PC moving back to the prologue instruction. (As already mentioned, this should be rare as it only happens when the task is scheduled to another CPU during the step.) This also ensures that perf breakpoints count prologue instructions consistently (i.e. every time they are executed), rather than skipping them when there also happens to be a breakpoint on a main or epilogue instruction. Signed-off-by: Kristina Martsenko Acked-by: Catalin Marinas --- arch/arm64/kernel/traps.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 689188712909..3dfc901a430b 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -549,6 +549,12 @@ void do_el0_mops(struct pt_regs *regs, unsigned long e= sr) regs->pc -=3D 8; else regs->pc -=3D 4; + + /* + * If single stepping then finish the step before executing the + * prologue instruction. + */ + user_fastforward_single_step(current); } =20 #define __user_cache_maint(insn, address, res) \ --=20 2.25.1 From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11209C636CC for ; Thu, 16 Feb 2023 16:02:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229898AbjBPQCb (ORCPT ); Thu, 16 Feb 2023 11:02:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229635AbjBPQCS (ORCPT ); Thu, 16 Feb 2023 11:02:18 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0681185F56 for ; Thu, 16 Feb 2023 08:02:02 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CCA3B168F; Thu, 16 Feb 2023 08:02:44 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6DC973F881; Thu, 16 Feb 2023 08:01:59 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 09/10] arm64: mops: detect and enable FEAT_MOPS Date: Thu, 16 Feb 2023 16:00:11 +0000 Message-Id: <20230216160012.272345-10-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The Arm v8.8/9.3 FEAT_MOPS feature provides new instructions that perform a memory copy or set. Wire up the cpufeature code to detect the presence of FEAT_MOPS and enable it. Signed-off-by: Kristina Martsenko Reviewed-by: Catalin Marinas --- Documentation/arm64/cpu-feature-registers.rst | 2 ++ Documentation/arm64/elf_hwcaps.rst | 3 +++ arch/arm64/include/asm/hwcap.h | 1 + arch/arm64/include/uapi/asm/hwcap.h | 1 + arch/arm64/kernel/cpufeature.c | 19 +++++++++++++++++++ arch/arm64/kernel/cpuinfo.c | 1 + arch/arm64/tools/cpucaps | 1 + 7 files changed, 28 insertions(+) diff --git a/Documentation/arm64/cpu-feature-registers.rst b/Documentation/= arm64/cpu-feature-registers.rst index c7adc7897df6..4e4625f2455f 100644 --- a/Documentation/arm64/cpu-feature-registers.rst +++ b/Documentation/arm64/cpu-feature-registers.rst @@ -288,6 +288,8 @@ infrastructure: +------------------------------+---------+---------+ | Name | bits | visible | +------------------------------+---------+---------+ + | MOPS | [19-16] | y | + +------------------------------+---------+---------+ | RPRES | [7-4] | y | +------------------------------+---------+---------+ | WFXT | [3-0] | y | diff --git a/Documentation/arm64/elf_hwcaps.rst b/Documentation/arm64/elf_h= wcaps.rst index 83e57e4d38e2..8f847d0dcf57 100644 --- a/Documentation/arm64/elf_hwcaps.rst +++ b/Documentation/arm64/elf_hwcaps.rst @@ -302,6 +302,9 @@ HWCAP2_SMEB16B16 HWCAP2_SMEF16F16 Functionality implied by ID_AA64SMFR0_EL1.F16F16 =3D=3D 0b1 =20 +HWCAP2_MOPS + Functionality implied by ID_AA64ISAR2_EL1.MOPS =3D=3D 0b0001. + 4. Unused AT_HWCAP bits ----------------------- =20 diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h index 5d45f19fda7f..692b1ec663b2 100644 --- a/arch/arm64/include/asm/hwcap.h +++ b/arch/arm64/include/asm/hwcap.h @@ -137,6 +137,7 @@ #define KERNEL_HWCAP_SME_BI32I32 __khwcap2_feature(SME_BI32I32) #define KERNEL_HWCAP_SME_B16B16 __khwcap2_feature(SME_B16B16) #define KERNEL_HWCAP_SME_F16F16 __khwcap2_feature(SME_F16F16) +#define KERNEL_HWCAP_MOPS __khwcap2_feature(MOPS) =20 /* * This yields a mask that user programs can use to figure out what diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/= asm/hwcap.h index 69a4fb749c65..a2cac4305b1e 100644 --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -102,5 +102,6 @@ #define HWCAP2_SME_BI32I32 (1UL << 40) #define HWCAP2_SME_B16B16 (1UL << 41) #define HWCAP2_SME_F16F16 (1UL << 42) +#define HWCAP2_MOPS (1UL << 43) =20 #endif /* _UAPI__ASM_HWCAP_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 1b0a71541381..70a08b2ddaa6 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -216,6 +216,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar2[] = =3D { ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_E= L1_CSSC_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_E= L1_RPRFM_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64ISAR2_EL1_= BC_SHIFT, 4, 0), + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_EL1_= MOPS_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH), FTR_STRICT, FTR_EXACT, ID_AA64ISAR2_EL1_APA3_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH), @@ -2158,6 +2159,11 @@ static void cpu_enable_dit(const struct arm64_cpu_ca= pabilities *__unused) set_pstate_dit(1); } =20 +static void cpu_enable_mops(const struct arm64_cpu_capabilities *__unused) +{ + sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_MSCEn); +} + /* Internal helper functions to match cpu capability type */ static bool cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap) @@ -2759,6 +2765,18 @@ static const struct arm64_cpu_capabilities arm64_fea= tures[] =3D { .matches =3D has_cpuid_feature, .cpu_enable =3D cpu_enable_dit, }, + { + .desc =3D "Memory Copy and Memory Set instructions", + .capability =3D ARM64_HAS_MOPS, + .type =3D ARM64_CPUCAP_SYSTEM_FEATURE, + .sys_reg =3D SYS_ID_AA64ISAR2_EL1, + .sign =3D FTR_UNSIGNED, + .field_pos =3D ID_AA64ISAR2_EL1_MOPS_SHIFT, + .field_width =3D 4, + .min_field_value =3D ID_AA64ISAR2_EL1_MOPS_IMP, + .matches =3D has_cpuid_feature, + .cpu_enable =3D cpu_enable_mops, + }, {}, }; =20 @@ -2891,6 +2909,7 @@ static const struct arm64_cpu_capabilities arm64_elf_= hwcaps[] =3D { HWCAP_CAP(ID_AA64ISAR2_EL1, RPRFM, IMP, CAP_HWCAP, KERNEL_HWCAP_RPRFM), HWCAP_CAP(ID_AA64ISAR2_EL1, RPRES, IMP, CAP_HWCAP, KERNEL_HWCAP_RPRES), HWCAP_CAP(ID_AA64ISAR2_EL1, WFxT, IMP, CAP_HWCAP, KERNEL_HWCAP_WFXT), + HWCAP_CAP(ID_AA64ISAR2_EL1, MOPS, IMP, CAP_HWCAP, KERNEL_HWCAP_MOPS), #ifdef CONFIG_ARM64_SME HWCAP_CAP(ID_AA64PFR1_EL1, SME, IMP, CAP_HWCAP, KERNEL_HWCAP_SME), HWCAP_CAP(ID_AA64SMFR0_EL1, FA64, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_FA64), diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c index eb4378c23b3c..076a124255d0 100644 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -125,6 +125,7 @@ static const char *const hwcap_str[] =3D { [KERNEL_HWCAP_SME_BI32I32] =3D "smebi32i32", [KERNEL_HWCAP_SME_B16B16] =3D "smeb16b16", [KERNEL_HWCAP_SME_F16F16] =3D "smef16f16", + [KERNEL_HWCAP_MOPS] =3D "mops", }; =20 #ifdef CONFIG_COMPAT diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index bf32a1122c04..5018da44def2 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -34,6 +34,7 @@ HAS_GIC_PRIO_RELAXED_SYNC HAS_HCX HAS_LDAPR HAS_LSE_ATOMICS +HAS_MOPS HAS_NO_FPSIMD HAS_NO_HW_PREFETCH HAS_PAN --=20 2.25.1 From nobody Thu Sep 11 18:29:02 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DF6EC636CC for ; Thu, 16 Feb 2023 16:02:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229842AbjBPQCk (ORCPT ); Thu, 16 Feb 2023 11:02:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229906AbjBPQCa (ORCPT ); Thu, 16 Feb 2023 11:02:30 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 851D95355B for ; Thu, 16 Feb 2023 08:02:08 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0D93F113E; Thu, 16 Feb 2023 08:02:50 -0800 (PST) Received: from e126864.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8AF763F881; Thu, 16 Feb 2023 08:02:04 -0800 (PST) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Mark Brown , Luis Machado , Vladimir Murzin , linux-kernel@vger.kernel.org Subject: [PATCH 10/10] arm64: mops: allow disabling MOPS from the kernel command line Date: Thu, 16 Feb 2023 16:00:12 +0000 Message-Id: <20230216160012.272345-11-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230216160012.272345-1-kristina.martsenko@arm.com> References: <20230216160012.272345-1-kristina.martsenko@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Make it possible to disable the MOPS extension at runtime using the kernel command line. This can be useful for testing or working around hardware issues. Signed-off-by: Kristina Martsenko Reviewed-by: Catalin Marinas --- Documentation/admin-guide/kernel-parameters.txt | 3 +++ arch/arm64/kernel/idreg-override.c | 2 ++ 2 files changed, 5 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index 6cfa6e3996cf..ee86fe17352d 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -408,6 +408,9 @@ arm64.nosme [ARM64] Unconditionally disable Scalable Matrix Extension support =20 + arm64.nomops [ARM64] Unconditionally disable Memory Copy and Memory + Set instructions support + ataflop=3D [HW,M68k] =20 atarimouse=3D [HW,MOUSE] Atari Mouse diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-o= verride.c index d833d78a7f31..d13ae90042cb 100644 --- a/arch/arm64/kernel/idreg-override.c +++ b/arch/arm64/kernel/idreg-override.c @@ -123,6 +123,7 @@ static const struct ftr_set_desc isar2 __initconst =3D { .fields =3D { FIELD("gpa3", ID_AA64ISAR2_EL1_GPA3_SHIFT, NULL), FIELD("apa3", ID_AA64ISAR2_EL1_APA3_SHIFT, NULL), + FIELD("mops", ID_AA64ISAR2_EL1_MOPS_SHIFT, NULL), {} }, }; @@ -174,6 +175,7 @@ static const struct { "id_aa64isar1.gpi=3D0 id_aa64isar1.gpa=3D0 " "id_aa64isar1.api=3D0 id_aa64isar1.apa=3D0 " "id_aa64isar2.gpa3=3D0 id_aa64isar2.apa3=3D0" }, + { "arm64.nomops", "id_aa64isar2.mops=3D0" }, { "arm64.nomte", "id_aa64pfr1.mte=3D0" }, { "nokaslr", "kaslr.disabled=3D1" }, }; --=20 2.25.1