From nobody Sun Apr 5 18:11:19 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 32E28330B10; Sat, 14 Mar 2026 17:51:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510707; cv=none; b=J2dQNv1Ge1t6FY0fInNkbX5xQJOMQdFXwJo3nOQRA34xJ3yc+cXmDyxb8HO8ELiv6ksUOUBZpE1Q0RR61u/7gEnyJbfFavaGFUroM70iRab5pRKL0L5gVFaY2fycK9whl6c/CPWnhjmVZcY//6JPXz5HJmaJYsl/NJhhXTRVJ98= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510707; c=relaxed/simple; bh=XESANQC/s6OALJdLs3NlY0zl7jnA3aP+X3rssV/Wfd0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pJluwfSQPTfk9+YiUkFvT/1ASv6BN4snxPFeLpRfB3qFlLYFOwSezJeboTMIZKnJTUnAeRzaqKL4RLD+Vf6/2KEpezVFCi6ubnN+xkYYXfZiacL4+xLJ8nDypyhoiapwwu/wMrZ6bO6FYXYNXudij719VXMeZwGnxLAAfAUhZTA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 44324153B; Sat, 14 Mar 2026 10:51:34 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.1.197.6]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E39093F73B; Sat, 14 Mar 2026 10:51:37 -0700 (PDT) From: Yeoreum Yun To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oupton@kernel.org, miko.lenczewski@arm.com, kevin.brodsky@arm.com, broonie@kernel.org, ardb@kernel.org, suzuki.poulose@arm.com, lpieralisi@kernel.org, joey.gouly@arm.com, yuzenghui@huawei.com, yeoreum.yun@arm.com Subject: [PATCH v17 1/8] arm64: cpufeature: add FEAT_LSUI Date: Sat, 14 Mar 2026 17:51:26 +0000 Message-Id: <20260314175133.1084528-2-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260314175133.1084528-1-yeoreum.yun@arm.com> References: <20260314175133.1084528-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since Armv9.6, FEAT_LSUI introduces load/store instructions that allow privileged code to access user memory without clearing the PSTATE.PAN bit. Add CPU feature detection for FEAT_LSUI and enable its use when FEAT_PAN is present so that removes the need for SW_PAN handling when using LSUI instructions. Signed-off-by: Yeoreum Yun --- arch/arm64/include/asm/cpucaps.h | 2 ++ arch/arm64/kernel/cpufeature.c | 10 ++++++++++ arch/arm64/tools/cpucaps | 1 + 3 files changed, 13 insertions(+) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpuc= aps.h index 177c691914f8..6e3da333442e 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -71,6 +71,8 @@ cpucap_is_possible(const unsigned int cap) return true; case ARM64_HAS_PMUV3: return IS_ENABLED(CONFIG_HW_PERF_EVENTS); + case ARM64_HAS_LSUI: + return IS_ENABLED(CONFIG_ARM64_LSUI); } =20 return true; diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index c31f8e17732a..5074ff32176f 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -281,6 +281,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar2[] = =3D { =20 static const struct arm64_ftr_bits ftr_id_aa64isar3[] =3D { ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR3_E= L1_FPRCVT_SHIFT, 4, 0), + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR3_EL= 1_LSUI_SHIFT, 4, ID_AA64ISAR3_EL1_LSUI_NI), ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR3_E= L1_LSFE_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR3_E= L1_FAMINMAX_SHIFT, 4, 0), ARM64_FTR_END, @@ -3169,6 +3170,15 @@ static const struct arm64_cpu_capabilities arm64_fea= tures[] =3D { .cpu_enable =3D cpu_enable_ls64_v, ARM64_CPUID_FIELDS(ID_AA64ISAR1_EL1, LS64, LS64_V) }, +#ifdef CONFIG_ARM64_LSUI + { + .desc =3D "Unprivileged Load Store Instructions (LSUI)", + .capability =3D ARM64_HAS_LSUI, + .type =3D ARM64_CPUCAP_SYSTEM_FEATURE, + .matches =3D has_cpuid_feature, + ARM64_CPUID_FIELDS(ID_AA64ISAR3_EL1, LSUI, IMP) + }, +#endif {}, }; =20 diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index 7261553b644b..b7286d977788 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -48,6 +48,7 @@ HAS_LPA2 HAS_LSE_ATOMICS HAS_LS64 HAS_LS64_V +HAS_LSUI HAS_MOPS HAS_NESTED_VIRT HAS_BBML2_NOABORT --=20 LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7} From nobody Sun Apr 5 18:11:19 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A0B842EC0A7; Sat, 14 Mar 2026 17:51:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510705; cv=none; b=eLL8f3eAcYwjTWe3X30JvavagqHwWQU/wu927CkghBDsfDsN6sHTPuWYyxppJ+O1tVporhJlZP4Tbw3QTmO7N0qgB+fEmXyEuhLDVjmXfrRetRvCWQB5Gnyv6qgJzxd3Mcuh2ATWlJXAcxBgs4declinGj1FFbvNl2/K2NGjnrk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510705; c=relaxed/simple; bh=4Gv8iwjh1zDT2SeKSMMFrQ3dnzhmmicacYc/sIOILow=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=n5dkLXKIGz9JvkA4TTcJawVTNYfKxKJ5se68lLqkCEedMOQKRQxjIk4faI02ZJreFzA18giyGK9Y23xeDc7yHLK5yCV1Tde4S9WSxjVy19s9hUL7O8B5Ker+6zOrVhNgjA/z3T5Sm9Y6fjAPWCPeNUjbDePtPu6w7gQOuleSl0U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 19F92175D; Sat, 14 Mar 2026 10:51:37 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.1.197.6]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B08E23F73B; Sat, 14 Mar 2026 10:51:40 -0700 (PDT) From: Yeoreum Yun To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oupton@kernel.org, miko.lenczewski@arm.com, kevin.brodsky@arm.com, broonie@kernel.org, ardb@kernel.org, suzuki.poulose@arm.com, lpieralisi@kernel.org, joey.gouly@arm.com, yuzenghui@huawei.com, yeoreum.yun@arm.com Subject: [PATCH v17 2/8] KVM: arm64: expose FEAT_LSUI to guest Date: Sat, 14 Mar 2026 17:51:27 +0000 Message-Id: <20260314175133.1084528-3-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260314175133.1084528-1-yeoreum.yun@arm.com> References: <20260314175133.1084528-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" expose FEAT_LSUI to guest. Signed-off-by: Yeoreum Yun Acked-by: Marc Zyngier Reviewed-by: Catalin Marinas --- arch/arm64/kvm/sys_regs.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 1b4cacb6e918..484f98eaf6d4 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1805,7 +1805,7 @@ static u64 __kvm_read_sanitised_id_reg(const struct k= vm_vcpu *vcpu, break; case SYS_ID_AA64ISAR3_EL1: val &=3D ID_AA64ISAR3_EL1_FPRCVT | ID_AA64ISAR3_EL1_LSFE | - ID_AA64ISAR3_EL1_FAMINMAX; + ID_AA64ISAR3_EL1_FAMINMAX | ID_AA64ISAR3_EL1_LSUI; break; case SYS_ID_AA64MMFR2_EL1: val &=3D ~ID_AA64MMFR2_EL1_CCIDX_MASK; @@ -3252,6 +3252,7 @@ static const struct sys_reg_desc sys_reg_descs[] =3D { ID_AA64ISAR2_EL1_GPA3)), ID_WRITABLE(ID_AA64ISAR3_EL1, (ID_AA64ISAR3_EL1_FPRCVT | ID_AA64ISAR3_EL1_LSFE | + ID_AA64ISAR3_EL1_LSUI | ID_AA64ISAR3_EL1_FAMINMAX)), ID_UNALLOCATED(6,4), ID_UNALLOCATED(6,5), --=20 LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7} From nobody Sun Apr 5 18:11:19 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6B7E0331215; Sat, 14 Mar 2026 17:51:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510709; cv=none; b=F8RuVplrdJPCI088eVPXzUNnKOanvhveyYnMT10C/RixUfFMVO/GtadJXlyMSUs6q2ugRSfDd9NokI7UXa3X6J6RnPiEuvanKqy3ZHb3YMe/BBJMyqzQ6psDCow0G5+C9OJQkmX9kqBmAX2KWvCnprVtyVnbEKlxCvSM3NyYxhc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510709; c=relaxed/simple; bh=vu1YTnurQsdl41tWBjnRNhm7kHGYenI4f+h1W0AsaGE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IpyYnGcxju3fcAmy6f557mTZV80dWPsjGVoWkm0x1rxMgNMdhMlx/XK4+ziEhxRSggVFx3I7fhP64qDXAp9q8WsNg4swXVWn2ghT4hXZ7Yr6NpWguMhJ/6gYomq7yLBR79+B9ws9qZAkJm7YYCtomc227KF7WnsAKfUINTw5SFc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E0DBF176A; Sat, 14 Mar 2026 10:51:39 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.1.197.6]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 85FB53F73B; Sat, 14 Mar 2026 10:51:43 -0700 (PDT) From: Yeoreum Yun To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oupton@kernel.org, miko.lenczewski@arm.com, kevin.brodsky@arm.com, broonie@kernel.org, ardb@kernel.org, suzuki.poulose@arm.com, lpieralisi@kernel.org, joey.gouly@arm.com, yuzenghui@huawei.com, yeoreum.yun@arm.com Subject: [PATCH v17 3/8] KVM: arm64: kselftest: set_id_regs: add test for FEAT_LSUI Date: Sat, 14 Mar 2026 17:51:28 +0000 Message-Id: <20260314175133.1084528-4-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260314175133.1084528-1-yeoreum.yun@arm.com> References: <20260314175133.1084528-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add test coverage for FEAT_LSUI. Signed-off-by: Yeoreum Yun Reviewed-by: Mark Brown --- tools/testing/selftests/kvm/arm64/set_id_regs.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/kvm/arm64/set_id_regs.c b/tools/testin= g/selftests/kvm/arm64/set_id_regs.c index 73de5be58bab..fa3478a6c914 100644 --- a/tools/testing/selftests/kvm/arm64/set_id_regs.c +++ b/tools/testing/selftests/kvm/arm64/set_id_regs.c @@ -124,6 +124,7 @@ static const struct reg_ftr_bits ftr_id_aa64isar2_el1[]= =3D { =20 static const struct reg_ftr_bits ftr_id_aa64isar3_el1[] =3D { REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ISAR3_EL1, FPRCVT, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ISAR3_EL1, LSUI, 0), REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ISAR3_EL1, LSFE, 0), REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ISAR3_EL1, FAMINMAX, 0), REG_FTR_END, --=20 LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7} From nobody Sun Apr 5 18:11:19 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 108812DEA62; Sat, 14 Mar 2026 17:51:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510710; cv=none; b=XA88jxvdd2EL7coz5ihyf9YK0CbrfQrk7VXRaGHAeBjfkUyAQMlVGlv0Ydqk5KX1OYYrYurFA0FTnqqyImyBonT/J3KEbQCiH5FXuq99iQfxy2Op9/THfxmCNNAEfcANJfXVvivo+3V0veTB2Yjdm6XaCF0fs2TJCvydtkNf1+c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510710; c=relaxed/simple; bh=u73gUnM5dAstqIoRDEbYJ+M3oUj6/3WQaFV3LyfVuuY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Fg+hcYG5kRFmou4YMZJdrwnXVUAkC3cn7TsI7lHr4eHqpk4cbuNM/eANiWrnR1gvUiyJ5/qYyHaJ4JK3wszs1p3fqNHmpySZizqe+sc4wO438gFbKmqxQnOKPpGnoIsw6b4UAMRmi7sbuijTUgM1a8gssi53m5+Ls58UDAP++TA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AFD9E152B; Sat, 14 Mar 2026 10:51:42 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.1.197.6]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5ACE43F73B; Sat, 14 Mar 2026 10:51:46 -0700 (PDT) From: Yeoreum Yun To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oupton@kernel.org, miko.lenczewski@arm.com, kevin.brodsky@arm.com, broonie@kernel.org, ardb@kernel.org, suzuki.poulose@arm.com, lpieralisi@kernel.org, joey.gouly@arm.com, yuzenghui@huawei.com, yeoreum.yun@arm.com Subject: [PATCH v17 4/8] arm64: futex: refactor futex atomic operation Date: Sat, 14 Mar 2026 17:51:29 +0000 Message-Id: <20260314175133.1084528-5-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260314175133.1084528-1-yeoreum.yun@arm.com> References: <20260314175133.1084528-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Refactor futex atomic operations using ll/sc method with clearing PSTATE.PAN to prepare to apply FEAT_LSUI on them. Signed-off-by: Yeoreum Yun --- arch/arm64/include/asm/futex.h | 156 +++++++++++++++++++++------------ 1 file changed, 98 insertions(+), 58 deletions(-) diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h index bc06691d2062..ba6a19de7823 100644 --- a/arch/arm64/include/asm/futex.h +++ b/arch/arm64/include/asm/futex.h @@ -7,73 +7,139 @@ =20 #include #include +#include =20 #include =20 #define FUTEX_MAX_LOOPS 128 /* What's the largest number you can think of?= */ =20 -#define __futex_atomic_op(insn, ret, oldval, uaddr, tmp, oparg) \ -do { \ +#define LLSC_FUTEX_ATOMIC_OP(op, insn) \ +static __always_inline int \ +__llsc_futex_atomic_##op(int oparg, u32 __user *uaddr, int *oval) \ +{ \ unsigned int loops =3D FUTEX_MAX_LOOPS; \ + int ret, oldval, newval; \ \ uaccess_enable_privileged(); \ - asm volatile( \ -" prfm pstl1strm, %2\n" \ -"1: ldxr %w1, %2\n" \ + asm volatile("// __llsc_futex_atomic_" #op "\n" \ +" prfm pstl1strm, %[uaddr]\n" \ +"1: ldxr %w[oldval], %[uaddr]\n" \ insn "\n" \ -"2: stlxr %w0, %w3, %2\n" \ -" cbz %w0, 3f\n" \ -" sub %w4, %w4, %w0\n" \ -" cbnz %w4, 1b\n" \ -" mov %w0, %w6\n" \ +"2: stlxr %w[ret], %w[newval], %[uaddr]\n" \ +" cbz %w[ret], 3f\n" \ +" sub %w[loops], %w[loops], %w[ret]\n" \ +" cbnz %w[loops], 1b\n" \ +" mov %w[ret], %w[err]\n" \ "3:\n" \ " dmb ish\n" \ - _ASM_EXTABLE_UACCESS_ERR(1b, 3b, %w0) \ - _ASM_EXTABLE_UACCESS_ERR(2b, 3b, %w0) \ - : "=3D&r" (ret), "=3D&r" (oldval), "+Q" (*uaddr), "=3D&r" (tmp), \ - "+r" (loops) \ - : "r" (oparg), "Ir" (-EAGAIN) \ + _ASM_EXTABLE_UACCESS_ERR(1b, 3b, %w[ret]) \ + _ASM_EXTABLE_UACCESS_ERR(2b, 3b, %w[ret]) \ + : [ret] "=3D&r" (ret), [oldval] "=3D&r" (oldval), \ + [uaddr] "+Q" (*uaddr), [newval] "=3D&r" (newval), \ + [loops] "+r" (loops) \ + : [oparg] "r" (oparg), [err] "Ir" (-EAGAIN) \ : "memory"); \ uaccess_disable_privileged(); \ -} while (0) + \ + if (!ret) \ + *oval =3D oldval; \ + \ + return ret; \ +} + +LLSC_FUTEX_ATOMIC_OP(add, "add %w[newval], %w[oldval], %w[oparg]") +LLSC_FUTEX_ATOMIC_OP(or, "orr %w[newval], %w[oldval], %w[oparg]") +LLSC_FUTEX_ATOMIC_OP(and, "and %w[newval], %w[oldval], %w[oparg]") +LLSC_FUTEX_ATOMIC_OP(eor, "eor %w[newval], %w[oldval], %w[oparg]") +LLSC_FUTEX_ATOMIC_OP(set, "mov %w[newval], %w[oparg]") + +static __always_inline int +__llsc_futex_cmpxchg(u32 __user *uaddr, u32 oldval, u32 newval, u32 *oval) +{ + int ret =3D 0; + unsigned int loops =3D FUTEX_MAX_LOOPS; + u32 val, tmp; + + uaccess_enable_privileged(); + asm volatile("//__llsc_futex_cmpxchg\n" +" prfm pstl1strm, %[uaddr]\n" +"1: ldxr %w[curval], %[uaddr]\n" +" eor %w[tmp], %w[curval], %w[oldval]\n" +" cbnz %w[tmp], 4f\n" +"2: stlxr %w[tmp], %w[newval], %[uaddr]\n" +" cbz %w[tmp], 3f\n" +" sub %w[loops], %w[loops], %w[tmp]\n" +" cbnz %w[loops], 1b\n" +" mov %w[ret], %w[err]\n" +"3:\n" +" dmb ish\n" +"4:\n" + _ASM_EXTABLE_UACCESS_ERR(1b, 4b, %w[ret]) + _ASM_EXTABLE_UACCESS_ERR(2b, 4b, %w[ret]) + : [ret] "+r" (ret), [curval] "=3D&r" (val), + [uaddr] "+Q" (*uaddr), [tmp] "=3D&r" (tmp), + [loops] "+r" (loops) + : [oldval] "r" (oldval), [newval] "r" (newval), + [err] "Ir" (-EAGAIN) + : "memory"); + uaccess_disable_privileged(); + + if (!ret) + *oval =3D val; + + return ret; +} + +#define FUTEX_ATOMIC_OP(op) \ +static __always_inline int \ +__futex_atomic_##op(int oparg, u32 __user *uaddr, int *oval) \ +{ \ + return __llsc_futex_atomic_##op(oparg, uaddr, oval); \ +} + +FUTEX_ATOMIC_OP(add) +FUTEX_ATOMIC_OP(or) +FUTEX_ATOMIC_OP(and) +FUTEX_ATOMIC_OP(eor) +FUTEX_ATOMIC_OP(set) + +static __always_inline int +__futex_cmpxchg(u32 __user *uaddr, u32 oldval, u32 newval, u32 *oval) +{ + return __llsc_futex_cmpxchg(uaddr, oldval, newval, oval); +} =20 static inline int arch_futex_atomic_op_inuser(int op, int oparg, int *oval, u32 __user *_uad= dr) { - int oldval =3D 0, ret, tmp; - u32 __user *uaddr =3D __uaccess_mask_ptr(_uaddr); + int ret; + u32 __user *uaddr; =20 if (!access_ok(_uaddr, sizeof(u32))) return -EFAULT; =20 + uaddr =3D __uaccess_mask_ptr(_uaddr); + switch (op) { case FUTEX_OP_SET: - __futex_atomic_op("mov %w3, %w5", - ret, oldval, uaddr, tmp, oparg); + ret =3D __futex_atomic_set(oparg, uaddr, oval); break; case FUTEX_OP_ADD: - __futex_atomic_op("add %w3, %w1, %w5", - ret, oldval, uaddr, tmp, oparg); + ret =3D __futex_atomic_add(oparg, uaddr, oval); break; case FUTEX_OP_OR: - __futex_atomic_op("orr %w3, %w1, %w5", - ret, oldval, uaddr, tmp, oparg); + ret =3D __futex_atomic_or(oparg, uaddr, oval); break; case FUTEX_OP_ANDN: - __futex_atomic_op("and %w3, %w1, %w5", - ret, oldval, uaddr, tmp, ~oparg); + ret =3D __futex_atomic_and(~oparg, uaddr, oval); break; case FUTEX_OP_XOR: - __futex_atomic_op("eor %w3, %w1, %w5", - ret, oldval, uaddr, tmp, oparg); + ret =3D __futex_atomic_eor(oparg, uaddr, oval); break; default: ret =3D -ENOSYS; } =20 - if (!ret) - *oval =3D oldval; - return ret; } =20 @@ -81,40 +147,14 @@ static inline int futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *_uaddr, u32 oldval, u32 newval) { - int ret =3D 0; - unsigned int loops =3D FUTEX_MAX_LOOPS; - u32 val, tmp; u32 __user *uaddr; =20 if (!access_ok(_uaddr, sizeof(u32))) return -EFAULT; =20 uaddr =3D __uaccess_mask_ptr(_uaddr); - uaccess_enable_privileged(); - asm volatile("// futex_atomic_cmpxchg_inatomic\n" -" prfm pstl1strm, %2\n" -"1: ldxr %w1, %2\n" -" sub %w3, %w1, %w5\n" -" cbnz %w3, 4f\n" -"2: stlxr %w3, %w6, %2\n" -" cbz %w3, 3f\n" -" sub %w4, %w4, %w3\n" -" cbnz %w4, 1b\n" -" mov %w0, %w7\n" -"3:\n" -" dmb ish\n" -"4:\n" - _ASM_EXTABLE_UACCESS_ERR(1b, 4b, %w0) - _ASM_EXTABLE_UACCESS_ERR(2b, 4b, %w0) - : "+r" (ret), "=3D&r" (val), "+Q" (*uaddr), "=3D&r" (tmp), "+r" (loops) - : "r" (oldval), "r" (newval), "Ir" (-EAGAIN) - : "memory"); - uaccess_disable_privileged(); =20 - if (!ret) - *uval =3D val; - - return ret; + return __futex_cmpxchg(uaddr, oldval, newval, uval); } =20 #endif /* __ASM_FUTEX_H */ --=20 LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7} From nobody Sun Apr 5 18:11:19 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1C39333ADBC; Sat, 14 Mar 2026 17:51:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510714; cv=none; b=RNJTJ6Bmd4DDFdhFeUZz6saeyzcN0vkf9LJsFFWULCLZdDZdEOsCCUUxdc5ylNWDEduBsgBZ6TA0ivJfdmxfGPAliyFvf5Jpi/Fzj3Rm7LCv+AHSz0XtMQ6NtRy1rRU54Cg0nzFCrx8Sv+AnUWcqxmABbnEGS8kUj6s5nm4gq20= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510714; c=relaxed/simple; bh=DSAN839k01DRYy+zRiXD2a+6Itk5HVOdc4RZlsWqD6U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=a7Va7Tf7fVG0S15VUdNWp1lZ+KRRwiLKZLGAFmfVHNhWWTgqWE1XT8ntkovr9EF+wV+I8f0E/oEBHWucw2aD2VdME/CklLdakR7H8RVmQNXBowH1iYkssnpMGgEEaQPBXH+RSW8RBpsuTjifwffAF1s6vrJ3fitF9XbPVdbNJNQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7B8FD153B; Sat, 14 Mar 2026 10:51:45 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.1.197.6]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 271AC3F73B; Sat, 14 Mar 2026 10:51:49 -0700 (PDT) From: Yeoreum Yun To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oupton@kernel.org, miko.lenczewski@arm.com, kevin.brodsky@arm.com, broonie@kernel.org, ardb@kernel.org, suzuki.poulose@arm.com, lpieralisi@kernel.org, joey.gouly@arm.com, yuzenghui@huawei.com, yeoreum.yun@arm.com Subject: [PATCH v17 5/8] arm64: futex: support futex with FEAT_LSUI Date: Sat, 14 Mar 2026 17:51:30 +0000 Message-Id: <20260314175133.1084528-6-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260314175133.1084528-1-yeoreum.yun@arm.com> References: <20260314175133.1084528-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Current futex atomic operations are implemented using LL/SC instructions while temporarily clearing PSTATE.PAN. Since Armv9.6, FEAT_LSUI provides load/store instructions for user memory access in the kernel as well as atomic operations, removing the need to clear PSTATE.PAN. With these instructions, some futex atomic operations no longer need to be implemented using an ldxr/stlxr pair. Instead, they can be performed using a single atomic instruction provided by FEAT_LSUI, without enabling MTE as required when using ldtr*/sttr* instructions. However, some futex atomic operations do not have a matching LSUI instruction, for example eor or word-sized cmpxchg. For such cases, use cas{al}t to implement the operation. FEAT_LSUI is introduced in Armv9.6, where FEAT_PAN is mandatory. However, this assumption may not always hold: - Some CPUs may advertise FEAT_LSUI but lack FEAT_PAN. - Virtualization or ID register overrides may expose invalid feature combinations. Therefore, instead of disabling FEAT_LSUI when FEAT_PAN is absent, wrap LSUI instructions with uaccess_ttbr0_enable()/disable() when ARM64_SW_TTBR0_PAN is enabled. Signed-off-by: Yeoreum Yun --- arch/arm64/include/asm/futex.h | 157 ++++++++++++++++++++++++++++++++- arch/arm64/include/asm/lsui.h | 27 ++++++ 2 files changed, 181 insertions(+), 3 deletions(-) create mode 100644 arch/arm64/include/asm/lsui.h diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h index ba6a19de7823..f2203a5e231c 100644 --- a/arch/arm64/include/asm/futex.h +++ b/arch/arm64/include/asm/futex.h @@ -7,9 +7,9 @@ =20 #include #include -#include =20 #include +#include =20 #define FUTEX_MAX_LOOPS 128 /* What's the largest number you can think of?= */ =20 @@ -90,11 +90,162 @@ __llsc_futex_cmpxchg(u32 __user *uaddr, u32 oldval, u3= 2 newval, u32 *oval) return ret; } =20 +#ifdef CONFIG_ARM64_LSUI + +/* + * Wrap LSUI instructions with uaccess_ttbr0_enable()/disable(), as + * PAN toggling is not required. + */ + +#define LSUI_FUTEX_ATOMIC_OP(op, asm_op) \ +static __always_inline int \ +__lsui_futex_atomic_##op(int oparg, u32 __user *uaddr, int *oval) \ +{ \ + int ret =3D 0; \ + int oldval; \ + \ + uaccess_ttbr0_enable(); \ + \ + asm volatile("// __lsui_futex_atomic_" #op "\n" \ + __LSUI_PREAMBLE \ +"1: " #asm_op "al %w[oparg], %w[oldval], %[uaddr]\n" \ +"2:\n" \ + _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w[ret]) \ + : [ret] "+r" (ret), [uaddr] "+Q" (*uaddr), \ + [oldval] "=3Dr" (oldval) \ + : [oparg] "r" (oparg) \ + : "memory"); \ + \ + uaccess_ttbr0_disable(); \ + \ + if (!ret) \ + *oval =3D oldval; \ + return ret; \ +} + +LSUI_FUTEX_ATOMIC_OP(add, ldtadd) +LSUI_FUTEX_ATOMIC_OP(or, ldtset) +LSUI_FUTEX_ATOMIC_OP(andnot, ldtclr) +LSUI_FUTEX_ATOMIC_OP(set, swpt) + +static __always_inline int +__lsui_cmpxchg64(u64 __user *uaddr, u64 *oldval, u64 newval) +{ + int ret =3D 0; + + uaccess_ttbr0_enable(); + + asm volatile("// __lsui_cmpxchg64\n" + __LSUI_PREAMBLE +"1: casalt %[oldval], %[newval], %[uaddr]\n" +"2:\n" + _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w[ret]) + : [ret] "+r" (ret), [uaddr] "+Q" (*uaddr), + [oldval] "+r" (*oldval) + : [newval] "r" (newval) + : "memory"); + + uaccess_ttbr0_disable(); + + return ret; +} + +static __always_inline int +__lsui_cmpxchg32(u32 __user *uaddr, u32 oldval, u32 newval, u32 *oval) +{ + u64 __user *uaddr64; + bool futex_pos, other_pos; + u32 other, orig_other; + union { + u32 futex[2]; + u64 raw; + } oval64, orig64, nval64; + + uaddr64 =3D (u64 __user *)PTR_ALIGN_DOWN(uaddr, sizeof(u64)); + futex_pos =3D !IS_ALIGNED((unsigned long)uaddr, sizeof(u64)); + other_pos =3D !futex_pos; + + oval64.futex[futex_pos] =3D oldval; + if (get_user(oval64.futex[other_pos], (u32 __user *)uaddr64 + other_pos)) + return -EFAULT; + + orig64.raw =3D oval64.raw; + + nval64.futex[futex_pos] =3D newval; + nval64.futex[other_pos] =3D oval64.futex[other_pos]; + + if (__lsui_cmpxchg64(uaddr64, &oval64.raw, nval64.raw)) + return -EFAULT; + + oldval =3D oval64.futex[futex_pos]; + other =3D oval64.futex[other_pos]; + orig_other =3D orig64.futex[other_pos]; + + if (other !=3D orig_other) + return -EAGAIN; + + *oval =3D oldval; + + return 0; +} + +static __always_inline int +__lsui_futex_atomic_and(int oparg, u32 __user *uaddr, int *oval) +{ + /* + * Undo the bitwise negation applied to the oparg passed from + * arch_futex_atomic_op_inuser() with FUTEX_OP_ANDN. + */ + return __lsui_futex_atomic_andnot(~oparg, uaddr, oval); +} + +static __always_inline int +__lsui_futex_atomic_eor(int oparg, u32 __user *uaddr, int *oval) +{ + u32 oldval, newval, val; + int ret, i; + + if (get_user(oldval, uaddr)) + return -EFAULT; + + /* + * there are no ldteor/stteor instructions... + */ + for (i =3D 0; i < FUTEX_MAX_LOOPS; i++) { + newval =3D oldval ^ oparg; + + ret =3D __lsui_cmpxchg32(uaddr, oldval, newval, &val); + switch (ret) { + case -EFAULT: + return ret; + case -EAGAIN: + continue; + } + + if (val =3D=3D oldval) { + *oval =3D val; + return 0; + } + + oldval =3D val; + } + + return -EAGAIN; +} + +static __always_inline int +__lsui_futex_cmpxchg(u32 __user *uaddr, u32 oldval, u32 newval, u32 *oval) +{ + return __lsui_cmpxchg32(uaddr, oldval, newval, oval); +} +#endif /* CONFIG_ARM64_LSUI */ + + #define FUTEX_ATOMIC_OP(op) \ static __always_inline int \ __futex_atomic_##op(int oparg, u32 __user *uaddr, int *oval) \ { \ - return __llsc_futex_atomic_##op(oparg, uaddr, oval); \ + return __lsui_llsc_body(futex_atomic_##op, oparg, uaddr, oval); \ } =20 FUTEX_ATOMIC_OP(add) @@ -106,7 +257,7 @@ FUTEX_ATOMIC_OP(set) static __always_inline int __futex_cmpxchg(u32 __user *uaddr, u32 oldval, u32 newval, u32 *oval) { - return __llsc_futex_cmpxchg(uaddr, oldval, newval, oval); + return __lsui_llsc_body(futex_cmpxchg, uaddr, oldval, newval, oval); } =20 static inline int diff --git a/arch/arm64/include/asm/lsui.h b/arch/arm64/include/asm/lsui.h new file mode 100644 index 000000000000..8f0d81953eb6 --- /dev/null +++ b/arch/arm64/include/asm/lsui.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_LSUI_H +#define __ASM_LSUI_H + +#include +#include +#include +#include +#include + +#define __LSUI_PREAMBLE ".arch_extension lsui\n" + +#ifdef CONFIG_ARM64_LSUI + +#define __lsui_llsc_body(op, ...) \ +({ \ + alternative_has_cap_unlikely(ARM64_HAS_LSUI) ? \ + __lsui_##op(__VA_ARGS__) : __llsc_##op(__VA_ARGS__); \ +}) + +#else /* CONFIG_ARM64_LSUI */ + +#define __lsui_llsc_body(op, ...) __llsc_##op(__VA_ARGS__) + +#endif /* CONFIG_ARM64_LSUI */ + +#endif /* __ASM_LSUI_H */ --=20 LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7} From nobody Sun Apr 5 18:11:19 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D629133F36B; Sat, 14 Mar 2026 17:51:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510715; cv=none; b=J2HtOMebGQO17VVwIi3JWI6YQTPoI+qwWW6zfPnXWkyiFpDLKfHVZR3wOS3roYHnYN3HigBHd9M7c5MCFvomwD61oX0AI5rMEawAOg3cQVVzjPhAwQZUY0D0AdCTYIl2KLXSXrbyzLZyJpwHL5Sb754+tkCd+hKiX9rA1h6WVU8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510715; c=relaxed/simple; bh=gC1ASY4XTWOU9NSvQyEf85WEp3yTpQCXJkzGjWaR5EM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dcsYAn8wY842+DqBKBnnOQcuh/mvDMWT/NSHEUf8UNB2xnHpthju+29Q4JNx0TEyH+p8fEDGKRDP4cjuX3t0/nDbkDKCQN/xByxF8ALg7bYlyf8OztxUanTcnhwjoxueW8WLyNKD2yNwX397peOiGVhJOv7BWN/v7Dpu1GKz0Fg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4413E175D; Sat, 14 Mar 2026 10:51:48 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.1.197.6]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E6DCE3F73B; Sat, 14 Mar 2026 10:51:51 -0700 (PDT) From: Yeoreum Yun To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oupton@kernel.org, miko.lenczewski@arm.com, kevin.brodsky@arm.com, broonie@kernel.org, ardb@kernel.org, suzuki.poulose@arm.com, lpieralisi@kernel.org, joey.gouly@arm.com, yuzenghui@huawei.com, yeoreum.yun@arm.com Subject: [PATCH v17 6/8] arm64: armv8_deprecated: disable swp emulation when FEAT_LSUI present Date: Sat, 14 Mar 2026 17:51:31 +0000 Message-Id: <20260314175133.1084528-7-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260314175133.1084528-1-yeoreum.yun@arm.com> References: <20260314175133.1084528-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The purpose of supporting LSUI is to eliminate PAN toggling. CPUs that support LSUI are unlikely to support a 32-bit runtime. Since environments that support both LSUI and a 32-bit runtimeare expected to be extremely rare, not to emulate the SWP instruction using LSUI instructions in order to remove PAN toggling, and instead simply disable SWP emulation. Signed-off-by: Yeoreum Yun --- arch/arm64/kernel/armv8_deprecated.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8= _deprecated.c index e737c6295ec7..049754f7da36 100644 --- a/arch/arm64/kernel/armv8_deprecated.c +++ b/arch/arm64/kernel/armv8_deprecated.c @@ -610,6 +610,22 @@ static int __init armv8_deprecated_init(void) } =20 #endif + +#ifdef CONFIG_SWP_EMULATION + /* + * The purpose of supporting LSUI is to eliminate PAN toggling. + * CPUs that support LSUI are unlikely to support a 32-bit runtime. + * Since environments that support both LSUI and a 32-bit runtime + * are expected to be extremely rare, we choose not to emulate + * the SWP instruction using LSUI instructions in order to remove PAN tog= gling, + * and instead simply disable SWP emulation. + */ + if (cpus_have_final_cap(ARM64_HAS_LSUI)) { + insn_swp.status =3D INSN_UNAVAILABLE; + pr_info("swp/swpb instruction emulation is not supported on this system\= n"); + } +#endif + for (int i =3D 0; i < ARRAY_SIZE(insn_emulations); i++) { struct insn_emulation *ie =3D insn_emulations[i]; =20 --=20 LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7} From nobody Sun Apr 5 18:11:19 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AE35B1DF25C; Sat, 14 Mar 2026 17:51:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510719; cv=none; b=u4f0UH4nVPz70b91caGzgIq5e6qYYm8CBZPQF7WzFWDOAeKUPcH1xiImRjiE9f+WXbA4ZzVT1CRjrmyG5Mtmi4R7Hi1jlbVozCrsaOV0Ldc9WzqzO3PBfpbhNyLNV9SzRZTRV3MSZgdDcpWbgqACnH2HrCywHDgw99ZCdTjT/Pc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510719; c=relaxed/simple; bh=I4J9rKMhApLJCCl52+Grp6+AKq8MbmNhvolUbJ5B1cs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BJ6PTWrj4AY5Sco6vYCHqqh1phfP2z2uR/M8ZNWotzbsaft9wdzI/TXaEJLP47x/DpL5SYbqobW6J69uvwc/ZdS7/XQhMVRTxWphiSeq2rXh8HmOFRE2zgRuLw0NyYU7eTXHPBx9rBpTZs7A43dCPz8v9DwvyMjBA6hCNV/tvI4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0F739152B; Sat, 14 Mar 2026 10:51:51 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.1.197.6]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B28AB3F73B; Sat, 14 Mar 2026 10:51:54 -0700 (PDT) From: Yeoreum Yun To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oupton@kernel.org, miko.lenczewski@arm.com, kevin.brodsky@arm.com, broonie@kernel.org, ardb@kernel.org, suzuki.poulose@arm.com, lpieralisi@kernel.org, joey.gouly@arm.com, yuzenghui@huawei.com, yeoreum.yun@arm.com Subject: [PATCH v17 7/8] KVM: arm64: use CAST instruction for swapping guest descriptor Date: Sat, 14 Mar 2026 17:51:32 +0000 Message-Id: <20260314175133.1084528-8-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260314175133.1084528-1-yeoreum.yun@arm.com> References: <20260314175133.1084528-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the CAST instruction to swap the guest descriptor when FEAT_LSUI is enabled, avoiding the need to clear the PAN bit. FEAT_LSUI is introduced in Armv9.6, where FEAT_PAN is mandatory. However, this assumption may not always hold: - Some CPUs may advertise FEAT_LSUI but lack FEAT_PAN. - Virtualization or ID register overrides may expose invalid feature combinations. Therefore, instead of disabling FEAT_LSUI when FEAT_PAN is absent, wrap LSUI instructions with uaccess_ttbr0_enable()/disable() when ARM64_SW_TTBR0_PAN is enabled. Signed-off-by: Yeoreum Yun Reviewed-by: Marc Zyngier --- arch/arm64/kvm/at.c | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c index 6588ea251ed7..1adf88a57328 100644 --- a/arch/arm64/kvm/at.c +++ b/arch/arm64/kvm/at.c @@ -9,6 +9,7 @@ #include #include #include +#include =20 static void fail_s1_walk(struct s1_walk_result *wr, u8 fst, bool s1ptw) { @@ -1681,6 +1682,35 @@ int __kvm_find_s1_desc_level(struct kvm_vcpu *vcpu, = u64 va, u64 ipa, int *level) } } =20 +static int __lsui_swap_desc(u64 __user *ptep, u64 old, u64 new) +{ + u64 tmp =3D old; + int ret =3D 0; + + /* + * Wrap LSUI instructions with uaccess_ttbr0_enable()/disable(), + * as PAN toggling is not required. + */ + uaccess_ttbr0_enable(); + + asm volatile(__LSUI_PREAMBLE + "1: cast %[old], %[new], %[addr]\n" + "2:\n" + _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w[ret]) + : [old] "+r" (old), [addr] "+Q" (*ptep), [ret] "+r" (ret) + : [new] "r" (new) + : "memory"); + + uaccess_ttbr0_disable(); + + if (ret) + return ret; + if (tmp !=3D old) + return -EAGAIN; + + return ret; +} + static int __lse_swap_desc(u64 __user *ptep, u64 old, u64 new) { u64 tmp =3D old; @@ -1756,7 +1786,9 @@ int __kvm_at_swap_desc(struct kvm *kvm, gpa_t ipa, u6= 4 old, u64 new) return -EPERM; =20 ptep =3D (u64 __user *)hva + offset; - if (cpus_have_final_cap(ARM64_HAS_LSE_ATOMICS)) + if (cpus_have_final_cap(ARM64_HAS_LSUI)) + r =3D __lsui_swap_desc(ptep, old, new); + else if (cpus_have_final_cap(ARM64_HAS_LSE_ATOMICS)) r =3D __lse_swap_desc(ptep, old, new); else r =3D __llsc_swap_desc(ptep, old, new); --=20 LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7} From nobody Sun Apr 5 18:11:19 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 52695347518; Sat, 14 Mar 2026 17:52:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510721; cv=none; b=pip7+K2DyOMVqlcvy1/NtirypLWBLIFDSI9mNlVYTaTUtmAd18tq6LL3wdxhpmo2zs6gr9PoBdyahILs32TZYFdKdXWPfiF3W/sS6V2WI2DhB/Jtj/buS3rrtYgA3JgAwj0AdL1cENofolY0DoMQ5Z7ofNYr5YHCMpXST7x4aLY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773510721; c=relaxed/simple; bh=iGDW5vhq3Q6kZFtorZ0MiHnTnFizA2hhJE+6ztz9Y9Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=l7rxCcAdifG9Az08ZTGSKA7oggcbAfQrQ6gKGhGbWahch/bslVj5V0pnRDZmek84Sf25jkEEScr8fl5N3159REFd8K5By13phSazotlSLucPsChrC7+XcpwX2C3xOcdbMAhn/o9GM+6siMYMjv1cn1tHTKBwR7K4I0aMqaX02pw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D2CA9152B; Sat, 14 Mar 2026 10:51:53 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.1.197.6]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7E58B3F73B; Sat, 14 Mar 2026 10:51:57 -0700 (PDT) From: Yeoreum Yun To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oupton@kernel.org, miko.lenczewski@arm.com, kevin.brodsky@arm.com, broonie@kernel.org, ardb@kernel.org, suzuki.poulose@arm.com, lpieralisi@kernel.org, joey.gouly@arm.com, yuzenghui@huawei.com, yeoreum.yun@arm.com Subject: [PATCH v17 8/8] arm64: Kconfig: add support for LSUI Date: Sat, 14 Mar 2026 17:51:33 +0000 Message-Id: <20260314175133.1084528-9-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260314175133.1084528-1-yeoreum.yun@arm.com> References: <20260314175133.1084528-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since Armv9.6, FEAT_LSUI supplies the load/store instructions for previleged level to access to access user memory without clearing PSTATE.PAN bit. Add Kconfig option entry for FEAT_LSUI. Signed-off-by: Yeoreum Yun Reviewed-by: Catalin Marinas --- arch/arm64/Kconfig | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 38dba5f7e4d2..890a1bedbf4a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2215,6 +2215,26 @@ config ARM64_GCS =20 endmenu # "ARMv9.4 architectural features" =20 +config AS_HAS_LSUI + def_bool $(as-instr,.arch_extension lsui) + help + Supported by LLVM 20+ and binutils 2.45+. + +menu "ARMv9.6 architectural features" + +config ARM64_LSUI + bool "Support Unprivileged Load Store Instructions (LSUI)" + default y + depends on AS_HAS_LSUI && !CPU_BIG_ENDIAN + help + The Unprivileged Load Store Instructions (LSUI) provides + variants load/store instructions that access user-space memory + from the kernel without clearing PSTATE.PAN bit. + + This feature is supported by LLVM 20+ and binutils 2.45+. + +endmenu # "ARMv9.6 architectural feature" + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y --=20 LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7}