From nobody Fri Oct 3 20:48:04 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 28146245021; Mon, 25 Aug 2025 03:34:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092880; cv=none; b=LjFKBpNSsEJq1c4D5D2f3EQkZDS4rXejY2qV9iOnuzx6VqPCsILhBENRZ226TBGScZBWTPCeR8l+Nrn9xj1zZljcz2jqmG4nB/fSBwPo3UxEayuWx21DObNxBbGk2FvcUg6N3Xeh2Nh+fK027Orp2MHRC2JQ5MiW+ZrfGTjaifo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092880; c=relaxed/simple; bh=AbCmTU2WGCZBoY9NASa9L9cUCAS0wFLXwztVBLdMe88=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F284RP0F098i12GK5UB7TpJykOuAqU60L72AtR/0ch9SEw3/6Ty0CkGXvBXpN6ThCWSw3j0AUiI6l4RUmsY7xFIiE2iD4S9zPejjQgNDPuJX9llzmyOrM5A0MiG2fK7c2n/8ldYuwA0c8C1Gy/QLZHYEpaLybymC4mbzJ3i/rp4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5EC2A2934; Sun, 24 Aug 2025 20:34:22 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7FBB03F63F; Sun, 24 Aug 2025 20:34:29 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v7 1/7] arm64: probes: Break ret out from bl/blr Date: Sun, 24 Aug 2025 22:34:15 -0500 Message-ID: <20250825033421.463669-2-jeremy.linton@arm.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250825033421.463669-1-jeremy.linton@arm.com> References: <20250825033421.463669-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Prepare for GCS by breaking RET out into its own function, where it makes more sense to encapsulate the new behavior independent from the branch instructions. Signed-off-by: Jeremy Linton Reviewed-by: Catalin Marinas --- arch/arm64/kernel/probes/decode-insn.c | 7 ++++--- arch/arm64/kernel/probes/simulate-insn.c | 10 +++++++++- arch/arm64/kernel/probes/simulate-insn.h | 3 ++- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/probes/decode-insn.c b/arch/arm64/kernel/pro= bes/decode-insn.c index 6438bf62e753..4137cc5ef031 100644 --- a/arch/arm64/kernel/probes/decode-insn.c +++ b/arch/arm64/kernel/probes/decode-insn.c @@ -108,9 +108,10 @@ arm_probe_decode_insn(u32 insn, struct arch_probe_insn= *api) aarch64_insn_is_bl(insn)) { api->handler =3D simulate_b_bl; } else if (aarch64_insn_is_br(insn) || - aarch64_insn_is_blr(insn) || - aarch64_insn_is_ret(insn)) { - api->handler =3D simulate_br_blr_ret; + aarch64_insn_is_blr(insn)) { + api->handler =3D simulate_br_blr; + } else if (aarch64_insn_is_ret(insn)) { + api->handler =3D simulate_ret; } else { /* * Instruction cannot be stepped out-of-line and we don't diff --git a/arch/arm64/kernel/probes/simulate-insn.c b/arch/arm64/kernel/p= robes/simulate-insn.c index 4c6d2d712fbd..09a0b36122d0 100644 --- a/arch/arm64/kernel/probes/simulate-insn.c +++ b/arch/arm64/kernel/probes/simulate-insn.c @@ -126,7 +126,7 @@ simulate_b_cond(u32 opcode, long addr, struct pt_regs *= regs) } =20 void __kprobes -simulate_br_blr_ret(u32 opcode, long addr, struct pt_regs *regs) +simulate_br_blr(u32 opcode, long addr, struct pt_regs *regs) { int xn =3D (opcode >> 5) & 0x1f; =20 @@ -138,6 +138,14 @@ simulate_br_blr_ret(u32 opcode, long addr, struct pt_r= egs *regs) set_x_reg(regs, 30, addr + 4); } =20 +void __kprobes +simulate_ret(u32 opcode, long addr, struct pt_regs *regs) +{ + int xn =3D (opcode >> 5) & 0x1f; + + instruction_pointer_set(regs, get_x_reg(regs, xn)); +} + void __kprobes simulate_cbz_cbnz(u32 opcode, long addr, struct pt_regs *regs) { diff --git a/arch/arm64/kernel/probes/simulate-insn.h b/arch/arm64/kernel/p= robes/simulate-insn.h index efb2803ec943..9e772a292d56 100644 --- a/arch/arm64/kernel/probes/simulate-insn.h +++ b/arch/arm64/kernel/probes/simulate-insn.h @@ -11,7 +11,8 @@ void simulate_adr_adrp(u32 opcode, long addr, struct pt_regs *regs); void simulate_b_bl(u32 opcode, long addr, struct pt_regs *regs); void simulate_b_cond(u32 opcode, long addr, struct pt_regs *regs); -void simulate_br_blr_ret(u32 opcode, long addr, struct pt_regs *regs); +void simulate_br_blr(u32 opcode, long addr, struct pt_regs *regs); +void simulate_ret(u32 opcode, long addr, struct pt_regs *regs); void simulate_cbz_cbnz(u32 opcode, long addr, struct pt_regs *regs); void simulate_tbz_tbnz(u32 opcode, long addr, struct pt_regs *regs); void simulate_ldr_literal(u32 opcode, long addr, struct pt_regs *regs); --=20 2.50.1 From nobody Fri Oct 3 20:48:04 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 515EB23A58E; Mon, 25 Aug 2025 03:34:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092881; cv=none; b=d9Naqzv9xV8fguTNx6hPmJ+hQUerOSABhNqcYSbn2doG8hFELHU7xWsQpeOA5xETURRe2LP3ZUo0gm64FHNu3nZebu0LlvjWga9RRaSpcskwiXHr03we14haIAaxW+bxy3K1TSE4zORroXUv/lDv2H+AwyIiCm0WzqPLQ5SJkyc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092881; c=relaxed/simple; bh=G2jTfBPy5tqGKJ6OHbu0abTewPFDgcWMZyLkpvhl14g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HKuGaI6JEOtCyKghAMCCrvpdpeC28Gv4o2pKHlRoDBotAs0hqWWAtdcdE55bVwsADp7sCaQ7EafcINNbWcFJI/Msq86w469PKd/PFHo4/gZGdXjek3HK3HcQMobII8HWex9MzSPO/qC0PQtLLSBfHbJkRF6ceBJC1HfIavYAhuU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7B7822A6B; Sun, 24 Aug 2025 20:34:24 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9A1703F63F; Sun, 24 Aug 2025 20:34:31 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v7 2/7] arm64: uaccess: Move existing GCS accessors definitions to gcs.h Date: Sun, 24 Aug 2025 22:34:16 -0500 Message-ID: <20250825033421.463669-3-jeremy.linton@arm.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250825033421.463669-1-jeremy.linton@arm.com> References: <20250825033421.463669-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We are going to add some additional GCS access helpers to gcs.h in order to avoid some forward reference problems with uaccess. In preparation for that, lets move the existing gcssttr() and put_user_gcs() routines into gcs.h where it makes sense to keep all the accessors together. Further, the code which uses them already includes gcs.h and there is an existing CONFIG_ARM64_GCS check we can reuse. The GCSSTTR instruction description comment is corrected during the move. Signed-off-by: Jeremy Linton Reviewed-by: Catalin Marinas Reviewed-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 37 ++++++++++++++++++++++++++++- arch/arm64/include/asm/uaccess.h | 40 -------------------------------- 2 files changed, 36 insertions(+), 41 deletions(-) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index 5bc432234d3a..10c68d3e6e30 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -21,7 +21,7 @@ static inline void gcsstr(u64 *addr, u64 val) register u64 *_addr __asm__ ("x0") =3D addr; register long _val __asm__ ("x1") =3D val; =20 - /* GCSSTTR x1, x0 */ + /* GCSSTTR x1, [x0] */ asm volatile( ".inst 0xd91f1c01\n" : @@ -81,6 +81,41 @@ static inline int gcs_check_locked(struct task_struct *t= ask, return 0; } =20 +static inline int gcssttr(unsigned long __user *addr, unsigned long val) +{ + register unsigned long __user *_addr __asm__ ("x0") =3D addr; + register unsigned long _val __asm__ ("x1") =3D val; + int err =3D 0; + + /* GCSSTTR x1, [x0] */ + asm volatile( + "1: .inst 0xd91f1c01\n" + "2: \n" + _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0) + : "+r" (err) + : "rZ" (_val), "r" (_addr) + : "memory"); + + return err; +} + +static inline void put_user_gcs(unsigned long val, unsigned long __user *a= ddr, + int *err) +{ + int ret; + + if (!access_ok((char __user *)addr, sizeof(u64))) { + *err =3D -EFAULT; + return; + } + + uaccess_ttbr0_enable(); + ret =3D gcssttr(addr, val); + if (ret !=3D 0) + *err =3D ret; + uaccess_ttbr0_disable(); +} + #else =20 static inline bool task_gcs_el0_enabled(struct task_struct *task) diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uacc= ess.h index 5b91803201ef..1aa4ecb73429 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -502,44 +502,4 @@ static inline size_t probe_subpage_writeable(const cha= r __user *uaddr, =20 #endif /* CONFIG_ARCH_HAS_SUBPAGE_FAULTS */ =20 -#ifdef CONFIG_ARM64_GCS - -static inline int gcssttr(unsigned long __user *addr, unsigned long val) -{ - register unsigned long __user *_addr __asm__ ("x0") =3D addr; - register unsigned long _val __asm__ ("x1") =3D val; - int err =3D 0; - - /* GCSSTTR x1, x0 */ - asm volatile( - "1: .inst 0xd91f1c01\n" - "2: \n" - _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0) - : "+r" (err) - : "rZ" (_val), "r" (_addr) - : "memory"); - - return err; -} - -static inline void put_user_gcs(unsigned long val, unsigned long __user *a= ddr, - int *err) -{ - int ret; - - if (!access_ok((char __user *)addr, sizeof(u64))) { - *err =3D -EFAULT; - return; - } - - uaccess_ttbr0_enable(); - ret =3D gcssttr(addr, val); - if (ret !=3D 0) - *err =3D ret; - uaccess_ttbr0_disable(); -} - - -#endif /* CONFIG_ARM64_GCS */ - #endif /* __ASM_UACCESS_H */ --=20 2.50.1 From nobody Fri Oct 3 20:48:04 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8DA101C5496; Mon, 25 Aug 2025 03:34:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092878; cv=none; b=FUXARrhMe7EVmHL4Yt9UOHCKFdpWsQRi6h14nvYLF8LtlW5Ton0D/qVjBEe3RD1SpyIzD0CsCzLTB4XbtBgf7hwjFCPdG/IAEgDxI2dDjHJYjE+lvQROsic6zKJejVgAD1k98vtPkmJwVfoUvj2DmmEywXooLTCTzCl/D+pUx94= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092878; c=relaxed/simple; bh=HQ1D+tm6U/2Iz5bWZlC9enB6ulnn3ioXax8Z0NIkJmI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AsQGeKsRdzrg+CODyPhRkxBJvEyy10g9TbwxCEfZW24EafM2hnsm47sL6aCiNxvgPpwe2RH6lftJ8XrrY7bOK/zhlMr+zL3zkoRRWVUlHmzDsXxlYTOGEzBwvjpHBmMzEmE8UzYvRfgPk9B8thOBgtiOW9Y7Bug6NT4ZOSb24qk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 599A52A9A; Sun, 24 Aug 2025 20:34:26 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 711103F63F; Sun, 24 Aug 2025 20:34:33 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v7 3/7] arm64: uaccess: Add additional userspace GCS accessors Date: Sun, 24 Aug 2025 22:34:17 -0500 Message-ID: <20250825033421.463669-4-jeremy.linton@arm.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250825033421.463669-1-jeremy.linton@arm.com> References: <20250825033421.463669-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Uprobes need more advanced read, push, and pop userspace GCS functionality. Implement those features using the existing gcsstr() and copy_from_user(). Its important to note that GCS pages can be read by normal instructions, but the hardware validates that pages used by GCS specific operations, have a GCS privilege set. We aren't validating this in load_user_gcs because it requires stabilizing the VMA over the read which may fault. Signed-off-by: Jeremy Linton Reviewed-by: Catalin Marinas Reviewed-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 54 ++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index 10c68d3e6e30..97801543380e 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -116,6 +116,47 @@ static inline void put_user_gcs(unsigned long val, uns= igned long __user *addr, uaccess_ttbr0_disable(); } =20 +static inline void push_user_gcs(unsigned long val, int *err) +{ + u64 gcspr =3D read_sysreg_s(SYS_GCSPR_EL0); + + gcspr -=3D sizeof(u64); + put_user_gcs(val, (unsigned long __user *)gcspr, err); + if (!*err) + write_sysreg_s(gcspr, SYS_GCSPR_EL0); +} + +/* + * Unlike put/push_user_gcs() above, get/pop_user_gsc() doesn't + * validate the GCS permission is set on the page being read. This + * differs from how the hardware works when it consumes data stored at + * GCSPR. Callers should ensure this is acceptable. + */ +static inline u64 get_user_gcs(unsigned long __user *addr, int *err) +{ + unsigned long ret; + u64 load =3D 0; + + /* Ensure previous GCS operation are visible before we read the page */ + gcsb_dsync(); + ret =3D copy_from_user(&load, addr, sizeof(load)); + if (ret !=3D 0) + *err =3D ret; + return load; +} + +static inline u64 pop_user_gcs(int *err) +{ + u64 gcspr =3D read_sysreg_s(SYS_GCSPR_EL0); + u64 read_val; + + read_val =3D get_user_gcs((unsigned long __user *)gcspr, err); + if (!*err) + write_sysreg_s(gcspr + sizeof(u64), SYS_GCSPR_EL0); + + return read_val; +} + #else =20 static inline bool task_gcs_el0_enabled(struct task_struct *task) @@ -126,6 +167,10 @@ static inline bool task_gcs_el0_enabled(struct task_st= ruct *task) static inline void gcs_set_el0_mode(struct task_struct *task) { } static inline void gcs_free(struct task_struct *task) { } static inline void gcs_preserve_current_state(void) { } +static inline void put_user_gcs(unsigned long val, unsigned long __user *a= ddr, + int *err) { } +static inline void push_user_gcs(unsigned long val, int *err) { } + static inline unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, const struct kernel_clone_args *args) { @@ -136,6 +181,15 @@ static inline int gcs_check_locked(struct task_struct = *task, { return 0; } +static inline u64 get_user_gcs(unsigned long __user *addr, int *err) +{ + *err =3D -EFAULT; + return 0; +} +static inline u64 pop_user_gcs(int *err) +{ + return 0; +} =20 #endif =20 --=20 2.50.1 From nobody Fri Oct 3 20:48:04 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3DF3825A334; Mon, 25 Aug 2025 03:34:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092881; cv=none; b=SLMPZU7AyMwAwUUxWnwQeMfTnhLMM5KvHPKFJvwRI2QixjHQA8RMHuFiOS6UZma+5Xyus0+sLIsZj8A0fPkFO2fEOVqemPX5rBZ7IiGgfW2UK80W1+xBJ5gkm5ASFSC9MOxdkCTigqM/QA2+WxeBl7olUUGO8k4zH4nmqpdEhsE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092881; c=relaxed/simple; bh=p8rI+uwSux+oiul4MKcusHa0HDMta+a6vLzuD13hOYw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m0C9nAY9cJpfANQBvq7qjvjDvQPNPwxhMOhvY3O4zhBOOp7efK5f96HVi9W+iKT/y1VO2C7ri01Yk6/l1EBFU3t+JBVNlvAtEe65bSXVHQnbrVaevysjhWUFvh7ZHpR0J+JOIFtl3zpXD/MVOvdXtFL6qMV+eR6A2S5zsqBqpl0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 465532BC0; Sun, 24 Aug 2025 20:34:28 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 636663F63F; Sun, 24 Aug 2025 20:34:35 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v7 4/7] arm64: probes: Add GCS support to bl/blr/ret Date: Sun, 24 Aug 2025 22:34:18 -0500 Message-ID: <20250825033421.463669-5-jeremy.linton@arm.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250825033421.463669-1-jeremy.linton@arm.com> References: <20250825033421.463669-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The arm64 probe simulation doesn't currently have logic in place to deal with GCS and this results in core dumps if probes are inserted at control flow locations. Fix-up bl, blr and ret to manipulate the shadow stack as needed. While we manipulate and validate the shadow stack correctly, the hardware provides additional security by only allowing GCS operations against pages which are marked to support GCS. For writing there is gcssttr() which enforces this, but there isn't an equivalent for reading. This means that uprobe users should be aware that probing on control flow instructions which require reading the shadow stack (ex: ret) offers lower security guarantees than what is achieved without the uprobe active. Signed-off-by: Jeremy Linton Reviewed-by: Catalin Marinas --- arch/arm64/kernel/probes/simulate-insn.c | 44 +++++++++++++++++++----- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kernel/probes/simulate-insn.c b/arch/arm64/kernel/p= robes/simulate-insn.c index 09a0b36122d0..97ed4db75417 100644 --- a/arch/arm64/kernel/probes/simulate-insn.c +++ b/arch/arm64/kernel/probes/simulate-insn.c @@ -13,6 +13,7 @@ #include =20 #include "simulate-insn.h" +#include "asm/gcs.h" =20 #define bbl_displacement(insn) \ sign_extend32(((insn) & 0x3ffffff) << 2, 27) @@ -49,6 +50,21 @@ static inline u32 get_w_reg(struct pt_regs *regs, int re= g) return lower_32_bits(pt_regs_read_reg(regs, reg)); } =20 +static inline int update_lr(struct pt_regs *regs, long addr) +{ + int err =3D 0; + + if (user_mode(regs) && task_gcs_el0_enabled(current)) { + push_user_gcs(addr, &err); + if (err) { + force_sig(SIGSEGV); + return err; + } + } + procedure_link_pointer_set(regs, addr); + return err; +} + static bool __kprobes check_cbz(u32 opcode, struct pt_regs *regs) { int xn =3D opcode & 0x1f; @@ -107,9 +123,9 @@ simulate_b_bl(u32 opcode, long addr, struct pt_regs *re= gs) { int disp =3D bbl_displacement(opcode); =20 - /* Link register is x30 */ if (opcode & (1 << 31)) - set_x_reg(regs, 30, addr + 4); + if (update_lr(regs, addr + 4)) + return; =20 instruction_pointer_set(regs, addr + disp); } @@ -129,21 +145,31 @@ void __kprobes simulate_br_blr(u32 opcode, long addr, struct pt_regs *regs) { int xn =3D (opcode >> 5) & 0x1f; + int b_target =3D get_x_reg(regs, xn); =20 - /* update pc first in case we're doing a "blr lr" */ - instruction_pointer_set(regs, get_x_reg(regs, xn)); - - /* Link register is x30 */ if (((opcode >> 21) & 0x3) =3D=3D 1) - set_x_reg(regs, 30, addr + 4); + if (update_lr(regs, addr + 4)) + return; + + instruction_pointer_set(regs, b_target); } =20 void __kprobes simulate_ret(u32 opcode, long addr, struct pt_regs *regs) { + u64 ret_addr; + int err =3D 0; int xn =3D (opcode >> 5) & 0x1f; - - instruction_pointer_set(regs, get_x_reg(regs, xn)); + unsigned long r_target =3D get_x_reg(regs, xn); + + if (user_mode(regs) && task_gcs_el0_enabled(current)) { + ret_addr =3D pop_user_gcs(&err); + if (err || ret_addr !=3D r_target) { + force_sig(SIGSEGV); + return; + } + } + instruction_pointer_set(regs, r_target); } =20 void __kprobes --=20 2.50.1 From nobody Fri Oct 3 20:48:04 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3DE9425A2BB; Mon, 25 Aug 2025 03:34:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092880; cv=none; b=Xo+ZT1gOLEbIdn23Q1nc2INEuClb5xKb7QrnyI/509ku2CFPMoJF4V9+fQt1dEDPnRbe6qgp1cuIYS3FS75NvvBcY4isTRPRR21sKidd9AaS827FJxzUTXt8e24Wjr7I3p39O9HVG1kooCqPmJPlZRFGMwbiCXpzsKCGAANTMHM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092880; c=relaxed/simple; bh=Z+1IquETUuvnvgb9eqz1cuwQMXLohgMxMNidxvFnrZ0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MyVseF3VQzmI/LFs7pLsCpZpM+COU/oDmrCBdE4HhxeiJGRHaX/oeZndWveP2GsLrwxh4kB5nu2hfnhwxClX1SbNDRqTNrBJPePOWI7SVmUO1caRtD29hhyHeAryudraPFYKZXxbXuycpMHA3YkQlZDFv7zqzuIScmqbZ0DM2H0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3000915A1; Sun, 24 Aug 2025 20:34:30 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 463933F63F; Sun, 24 Aug 2025 20:34:37 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton , Steve Capper Subject: [PATCH v7 5/7] arm64: uprobes: Add GCS support to uretprobes Date: Sun, 24 Aug 2025 22:34:19 -0500 Message-ID: <20250825033421.463669-6-jeremy.linton@arm.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250825033421.463669-1-jeremy.linton@arm.com> References: <20250825033421.463669-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Ret probes work by changing the value in the link register at the probe location to return to the probe rather than the calling routine. Thus the GCS needs to be updated with this address as well. Since its possible to insert probes at locations where the current value of the LR doesn't match the GCS state this needs to be detected and handled in order to maintain the existing no-fault behavior. Co-developed-by: Steve Capper Signed-off-by: Steve Capper (updated to use new gcs accessors, and handle LR/GCS mismatches) Signed-off-by: Jeremy Linton Reviewed-by: Catalin Marinas --- arch/arm64/kernel/probes/uprobes.c | 33 ++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/arch/arm64/kernel/probes/uprobes.c b/arch/arm64/kernel/probes/= uprobes.c index 1f91fd2a8187..6b98503a0198 100644 --- a/arch/arm64/kernel/probes/uprobes.c +++ b/arch/arm64/kernel/probes/uprobes.c @@ -6,6 +6,7 @@ #include #include #include +#include =20 #include "decode-insn.h" =20 @@ -159,11 +160,43 @@ arch_uretprobe_hijack_return_addr(unsigned long tramp= oline_vaddr, struct pt_regs *regs) { unsigned long orig_ret_vaddr; + unsigned long gcs_ret_vaddr; + int err =3D 0; + u64 gcspr; =20 orig_ret_vaddr =3D procedure_link_pointer(regs); + + if (task_gcs_el0_enabled(current)) { + gcspr =3D read_sysreg_s(SYS_GCSPR_EL0); + gcs_ret_vaddr =3D get_user_gcs((unsigned long __user *)gcspr, &err); + if (err) { + force_sig(SIGSEGV); + goto out; + } + + /* + * If the LR and GCS return addr don't match, then some kind of PAC + * signing or control flow occurred since entering the probed function. + * Likely because the user is attempting to retprobe on an instruction + * that isn't a function boundary or inside a leaf function. Explicitly + * abort this retprobe because it will generate a GCS exception. + */ + if (gcs_ret_vaddr !=3D orig_ret_vaddr) { + orig_ret_vaddr =3D -1; + goto out; + } + + put_user_gcs(trampoline_vaddr, (unsigned long __user *)gcspr, &err); + if (err) { + force_sig(SIGSEGV); + goto out; + } + } + /* Replace the return addr with trampoline addr */ procedure_link_pointer_set(regs, trampoline_vaddr); =20 +out: return orig_ret_vaddr; } =20 --=20 2.50.1 From nobody Fri Oct 3 20:48:04 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EB956272E5A; Mon, 25 Aug 2025 03:34:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092882; cv=none; b=TczieE4ZEbsRhMWlB7lQ59wDMSAB0RwN8YFEhmPyrtZHvK//WoVz+w3TmlVjxvjdOhXOUe9ALjiOlJwDjHj4bwtwiUJgqwEzQOCyNmbU1Z7ZP5XRD0C7ZUwxuExL+xzPJ5ase56oqTVZMYRn2LE76hLLzB6sJHSnltEvQghY/mc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092882; c=relaxed/simple; bh=tLsK2O10wieJb1mRRo5CrgznO7vx9YT47u2gJ4bUwQs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pSePpr6KoQCLfEJdx8PivX0pWu9d6Yj4c3g87MMLfQL1CTKSN+oHTCsU+54r6xyIkzhT+D+mFMzEnNO4Aq3J5Df7c305aUWK1BJc+zc5gF7XoTlN9jLRUoBBU56fMGimod5zFyvC3Rr9Vhshk2fNSCX3yUfLMJb/8JPagM0qWuk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 062CF2934; Sun, 24 Aug 2025 20:34:32 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 28DA33F63F; Sun, 24 Aug 2025 20:34:39 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v7 6/7] arm64: Kconfig: Remove GCS restrictions on UPROBES Date: Sun, 24 Aug 2025 22:34:20 -0500 Message-ID: <20250825033421.463669-7-jeremy.linton@arm.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250825033421.463669-1-jeremy.linton@arm.com> References: <20250825033421.463669-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the uprobe paths have been made GCS compatible drop the Kconfig restriction. Signed-off-by: Jeremy Linton Acked-by: Catalin Marinas --- arch/arm64/Kconfig | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index e9bbfacc35a6..c61572bbe59b 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2225,7 +2225,6 @@ config ARM64_GCS default y select ARCH_HAS_USER_SHADOW_STACK select ARCH_USES_HIGH_VMA_FLAGS - depends on !UPROBES help Guarded Control Stack (GCS) provides support for a separate stack with restricted access which contains only return --=20 2.50.1 From nobody Fri Oct 3 20:48:04 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D3A1327C17E; Mon, 25 Aug 2025 03:34:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092885; cv=none; b=DVin9ZwKCt2VXGsEEbnPq96wO2aEUNoLODfdz9gQa/Y01LlmAE7XnXRiGkY0trrTZso/q5tvK+QN/bdIH5YRYVLlXTbN6xuYKwdtCJ/a0/DFt6Lg7CyWtw+54wGn+qctIUBRxKjwQ1Ufoq8R8FaCcpoc4xposkBt8FQ+vgSflDw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756092885; c=relaxed/simple; bh=qGKi7nFEQeTEbuUFTQQot3rZpaYZ++xqYf/CsvSj1kg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JV+DcMX1a6/LawxbuYQOjuX59tBnydRZfIzJw0KuklOgX5fJADUadDI8DxuOATDxTKQAsJqVHUtIM0Z7ep4ghqJb0h0K4AM6MjHa1Zx8/U6G+1pWna5/JLqNUbAklNyUitRFhzjqOGXmEUmrkYf5tXmWF1rGnChoOQtZXW3oLys= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E632A15A1; Sun, 24 Aug 2025 20:34:34 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0F7043F63F; Sun, 24 Aug 2025 20:34:42 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v7 7/7] uprobes: uprobe_warn should use passed task Date: Sun, 24 Aug 2025 22:34:21 -0500 Message-ID: <20250825033421.463669-8-jeremy.linton@arm.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250825033421.463669-1-jeremy.linton@arm.com> References: <20250825033421.463669-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" uprobe_warn() is passed a task structure, yet its using current. For the most part this shouldn't matter, but since a task structure is provided, lets use it. Fixes: 248d3a7b2f10 ("uprobes: Change uprobe_copy_process() to dup return_i= nstances") Signed-off-by: Jeremy Linton Reviewed-by: Catalin Marinas Acked-by: Oleg Nesterov Acked-by: Masami Hiramatsu (Google) --- kernel/events/uprobes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 7ca1940607bd..4b97d16f731c 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -121,7 +121,7 @@ struct xol_area { =20 static void uprobe_warn(struct task_struct *t, const char *msg) { - pr_warn("uprobe: %s:%d failed to %s\n", current->comm, current->pid, msg); + pr_warn("uprobe: %s:%d failed to %s\n", t->comm, t->pid, msg); } =20 /* --=20 2.50.1