From nobody Mon Oct 6 15:15:53 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4EF8E289812 for ; Mon, 21 Jul 2025 08:36:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753087002; cv=none; b=fAxeymucm71Wy54A/TawzxEuCv2wpZgeTe4rOrNA3fgYLTNMli0r+CLoO3GhFLRoTLfAmCokMKKJf9aXbLRVTRTN0xYMqW4ndADfiB7FXxqRxIk5nPFrm1AE7fgANt9rWqjq6H/0mi6zpVkmbhNYa6ZCCL0zc19IdLNIGG+u85I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753087002; c=relaxed/simple; bh=Xty6o4O/6jE/zrmvBD8pJ9Znos7innQaLeWnUhWql2A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JUVXHVui4CyQgSaCiWFxDTdKCZWJn3mLFF3Q6+W30s00Ec+ZJ9HvPA4ccI7oA2WDd/lifyhxmdeb8t239Rfg508usgw3aW9NUQQIGFd6imyPh5smGj1bBBmVcI/u9YVNak3yjepEjdJ9hVyJPkWuWSYqdEPs54nvLSVoYKC9N/4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BDDD7153B; Mon, 21 Jul 2025 01:36:34 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.1.197.6]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C881D3F66E; Mon, 21 Jul 2025 01:36:38 -0700 (PDT) From: Yeoreum Yun To: catalin.marinas@arm.com, will@kernel.org, broonie@kernel.org, oliver.upton@linux.dev, ardb@kernel.org, frederic@kernel.org, james.morse@arm.com, joey.gouly@arm.com, scott@os.amperecomputing.com, maz@kernel.org Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Yeoreum Yun Subject: [PATCH v4 4/7] arm64/futex: move futex atomic logic with clearing PAN bit Date: Mon, 21 Jul 2025 09:36:15 +0100 Message-Id: <20250721083618.2743569-5-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250721083618.2743569-1-yeoreum.yun@arm.com> References: <20250721083618.2743569-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move current futex atomic logics which uses ll/sc method with cleraing PSTATE.PAN to separate file (futex_ll_sc_u.h) so that former method will be used only when FEAT_LSUI isn't supported. Signed-off-by: Yeoreum Yun --- arch/arm64/include/asm/futex_ll_sc_u.h | 115 +++++++++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 arch/arm64/include/asm/futex_ll_sc_u.h diff --git a/arch/arm64/include/asm/futex_ll_sc_u.h b/arch/arm64/include/as= m/futex_ll_sc_u.h new file mode 100644 index 000000000000..6702ba66f1b2 --- /dev/null +++ b/arch/arm64/include/asm/futex_ll_sc_u.h @@ -0,0 +1,115 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2025 Arm Ltd. + */ +#ifndef __ASM_FUTEX_LL_SC_U_H +#define __ASM_FUTEX_LL_SC_U_H + +#include +#include + +#define FUTEX_ATOMIC_OP(op, asm_op) \ +static __always_inline int \ +__ll_sc_u_futex_atomic_##op(int oparg, u32 __user *uaddr, int *oval) \ +{ \ + unsigned int loops =3D LL_SC_MAX_LOOPS; \ + int ret, val, tmp; \ + \ + uaccess_enable_privileged(); \ + asm volatile("// __ll_sc_u_futex_atomic_" #op "\n" \ + " prfm pstl1strm, %2\n" \ + "1: ldxr %w1, %2\n" \ + " " #asm_op " %w3, %w1, %w5\n" \ + "2: stlxr %w0, %w3, %2\n" \ + " cbz %w0, 3f\n" \ + " sub %w4, %w4, %w0\n" \ + " cbnz %w4, 1b\n" \ + " mov %w0, %w6\n" \ + "3:\n" \ + " dmb ish\n" \ + _ASM_EXTABLE_UACCESS_ERR(1b, 3b, %w0) \ + _ASM_EXTABLE_UACCESS_ERR(2b, 3b, %w0) \ + : "=3D&r" (ret), "=3D&r" (val), "+Q" (*uaddr), "=3D&r" (tmp), \ + "+r" (loops) \ + : "r" (oparg), "Ir" (-EAGAIN) \ + : "memory"); \ + uaccess_disable_privileged(); \ + \ + if (!ret) \ + *oval =3D val; \ + \ + return ret; \ +} + +FUTEX_ATOMIC_OP(add, add) +FUTEX_ATOMIC_OP(or, orr) +FUTEX_ATOMIC_OP(and, and) +FUTEX_ATOMIC_OP(eor, eor) + +#undef FUTEX_ATOMIC_OP + +static __always_inline int +__ll_sc_u_futex_atomic_set(int oparg, u32 __user *uaddr, int *oval) +{ + unsigned int loops =3D LL_SC_MAX_LOOPS; + int ret, val; + + uaccess_enable_privileged(); + asm volatile("//__ll_sc_u_futex_xchg\n" + " prfm pstl1strm, %2\n" + "1: ldxr %w1, %2\n" + "2: stlxr %w0, %w4, %2\n" + " cbz %w3, 3f\n" + " sub %w3, %w3, %w0\n" + " cbnz %w3, 1b\n" + " mov %w0, %w5\n" + "3:\n" + " dmb ish\n" + _ASM_EXTABLE_UACCESS_ERR(1b, 3b, %w0) + _ASM_EXTABLE_UACCESS_ERR(2b, 3b, %w0) + : "=3D&r" (ret), "=3D&r" (val), "+Q" (*uaddr), "+r" (loops) + : "r" (oparg), "Ir" (-EAGAIN) + : "memory"); + uaccess_disable_privileged(); + + if (!ret) + *oval =3D val; + + return ret; +} + +static __always_inline int +__ll_sc_u_futex_cmpxchg(u32 __user *uaddr, u32 oldval, u32 newval, u32 *ov= al) +{ + int ret =3D 0; + unsigned int loops =3D LL_SC_MAX_LOOPS; + u32 val, tmp; + + uaccess_enable_privileged(); + asm volatile("//__ll_sc_u_futex_cmpxchg\n" + " prfm pstl1strm, %2\n" + "1: ldxr %w1, %2\n" + " eor %w3, %w1, %w5\n" + " cbnz %w3, 4f\n" + "2: stlxr %w3, %w6, %2\n" + " cbz %w3, 3f\n" + " sub %w4, %w4, %w3\n" + " cbnz %w4, 1b\n" + " mov %w0, %w7\n" + "3:\n" + " dmb ish\n" + "4:\n" + _ASM_EXTABLE_UACCESS_ERR(1b, 4b, %w0) + _ASM_EXTABLE_UACCESS_ERR(2b, 4b, %w0) + : "+r" (ret), "=3D&r" (val), "+Q" (*uaddr), "=3D&r" (tmp), "+r" (loops) + : "r" (oldval), "r" (newval), "Ir" (-EAGAIN) + : "memory"); + uaccess_disable_privileged(); + + if (!ret) + *oval =3D val; + + return ret; +} + +#endif /* __ASM_FUTEX_LL_SC_U_H */ --=20 LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7}