From nobody Wed Jun 17 02:50:06 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C42DA3A545F for ; Tue, 21 Apr 2026 22:52:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811933; cv=none; b=UhOubNnYqT0Gv+TqF/1Tb+Orl+DRnGxudEgDxI9gRAuYnHfgQK2vMQhjepTCsLx/PSEWhnLQ1xM8m2ZGu+TIyEOwQOVyB7Nyvc5l+K+348ce90/BoL/yjULeHZ/D4aIo+pRu82ZbCwXnxE6iffaNwTHrJYU8bB0EqvGkqlOhIhg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811933; c=relaxed/simple; bh=zuU9O8aomlgvBvmbuscWrvAC/CK4ytWTqIMQMoBi5X0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=t5c0wZdbh5hmm0+sc2BWDtYIwOeI9numnHEaD6Nrmhuxktx7LKwDR29F0b2rYhlB37kO6h+vX8ZQq6av5FYBpLkaUqhfvF/9c+pBPTrzWSRqhtiaQ9JQKel+nZuP7MhWIESdhVzcG9XULt4Wnso6Ay9nWnUxWMHaZwrxSZJBz7A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=o+W3l2DB; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="o+W3l2DB" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2b4530a90fdso96155165ad.1 for ; Tue, 21 Apr 2026 15:52:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776811930; x=1777416730; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GXTAYy9nlQAlE+dRUkBUCLWHUqF3nKPP0PG5zltcaJc=; b=o+W3l2DBtm5Ek5CY6en8o6l3wNuJEd1vEqVLyM92XO9UXWIlouaHffB+LKQrDZ+sSw ZpT7h1IvlDuVIj2UOBc4IcyX4hl8OkMzfgBh0vF3C5hPBSfkJ8F+uGeG5GViuuS3nmaU EA6qoO+8DwIL2fE12kG7jkSjAdSCfEeqY6T5C1/E8EYpkFU6GT6otrB29zAyaZCi5BUz 1C2CFu2hKaLdXE0jDSemyEixZeWZXBu/LwH54KJOGco2TUdcZadAWY8Pibx5YmA0iEVT EuPyL/WovWA2MzlFa9tEvEcwbwrxeQOgGDFq6qUie3YSU6SJQsJEr9oEc1t3CmbA0er4 VW3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776811930; x=1777416730; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GXTAYy9nlQAlE+dRUkBUCLWHUqF3nKPP0PG5zltcaJc=; b=g8JJbzIuaxP8bKSsjt5fNSZQHWt7n7U0xmnCMYvelZIVQ4o+DQIdxP/O614qAUE7uY QCBdyzVsQsX6OTFbKbRbLo4Wwu9/qXIaOFwwDzrzOEsyLwEdyHExb26CAUDneTkd24iD AGRnvUqVzahPyTXMcDCu5wam9PUn217JmAJLqOKcqt/nOGPEH6OQlGKxNqzZRvVv+Jxt aempMu3VSrt4WcHI3GB9Kmt/Cn4UGZ7HpgUZ3MkCbbMnSO3NCBEhXntdR2J2mB+1jngS oUOHY1unFYkmiBcGGaFwRhNejcOGEB6L0TepNmBMlZaiWGiqGc63v8771BTh41vZUo/b p/SA== X-Forwarded-Encrypted: i=1; AFNElJ/E6t8DAFzciUs71tWIMUD8PKALTPLREWgVOIPbvUiXZ0s7AkghlfNL1MZT2LSR4FSFv4xr3r0D5h3aTjQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzfVGEZgNnd30goYt15U1RXpdAsTSt2qm+mZsfDMedFOQhkpV4+ CHO0JDmtpKV6TWKgh85qUwJlZrhYziinhctZVwAMWx1daAhXGTUVSAcy9s+/U54Nvc92m1IYy0U dlGVf1efHMhp3ejLt6GZ0yMatKw== X-Received: from pgcv22.prod.google.com ([2002:a05:6a02:5316:b0:c70:ab5b:1dbf]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:3d0c:b0:398:8f2a:5dd4 with SMTP id adf61e73a8af0-3a08d6eb6a3mr21877295637.8.1776811930035; Tue, 21 Apr 2026 15:52:10 -0700 (PDT) Date: Tue, 21 Apr 2026 22:51:53 +0000 In-Reply-To: <20260421225200.1198447-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260421225200.1198447-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.rc1.555.g9c883467ad-goog Message-ID: <20260421225200.1198447-2-dylanbhatch@google.com> Subject: [PATCH v4 1/8] sframe: Allow kernelspace sframe sections From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Generalize the sframe lookup code to support kernelspace sections. This is done by defining a SFRAME_LOOKUP option that can be activated separate from HAVE_UNWIND_USER_SFRAME, as there will be other client to this library than just userspace unwind. Sframe section location is now tracked in a separate sec_type field to determine whether user-access functions are necessary to read the sframe data. Relevant type delarations are moved and renamed to reflect the non-user sframe support. Signed-off-by: Dylan Hatch Reviewed-by: Jens Remus --- MAINTAINERS | 2 +- arch/Kconfig | 4 + .../{unwind_user_sframe.h =3D> unwind_sframe.h} | 6 +- arch/x86/include/asm/unwind_user.h | 12 +- include/linux/sframe.h | 48 ++-- include/linux/unwind_types.h | 46 +++ include/linux/unwind_user_types.h | 41 --- kernel/unwind/Makefile | 2 +- kernel/unwind/sframe.c | 270 ++++++++++++------ kernel/unwind/user.c | 41 +-- 10 files changed, 293 insertions(+), 179 deletions(-) rename arch/x86/include/asm/{unwind_user_sframe.h =3D> unwind_sframe.h} (5= 0%) create mode 100644 include/linux/unwind_types.h diff --git a/MAINTAINERS b/MAINTAINERS index 8c46465ee7a9..cfc7dec88da4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -27557,7 +27557,7 @@ F: Documentation/driver-api/uio-howto.rst F: drivers/uio/ F: include/linux/uio_driver.h =20 -USERSPACE STACK UNWINDING +STACK UNWINDING M: Josh Poimboeuf M: Steven Rostedt S: Maintained diff --git a/arch/Kconfig b/arch/Kconfig index f1ed8bc0806d..d7caf2e245ce 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -486,6 +486,9 @@ config AS_SFRAME3 def_bool $(as-instr,.cfi_startproc\n.cfi_endproc,-Wa$(comma)--gsframe-3) select AS_SFRAME =20 +config UNWIND_SFRAME_LOOKUP + bool + config UNWIND_USER bool =20 @@ -496,6 +499,7 @@ config HAVE_UNWIND_USER_FP config HAVE_UNWIND_USER_SFRAME bool select UNWIND_USER + select UNWIND_SFRAME_LOOKUP =20 config SFRAME_VALIDATION bool "Enable .sframe section debugging" diff --git a/arch/x86/include/asm/unwind_user_sframe.h b/arch/x86/include/a= sm/unwind_sframe.h similarity index 50% rename from arch/x86/include/asm/unwind_user_sframe.h rename to arch/x86/include/asm/unwind_sframe.h index d828ae1a4aac..44d42e6ffde4 100644 --- a/arch/x86/include/asm/unwind_user_sframe.h +++ b/arch/x86/include/asm/unwind_sframe.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#ifndef _ASM_X86_UNWIND_USER_SFRAME_H -#define _ASM_X86_UNWIND_USER_SFRAME_H +#ifndef _ASM_X86_UNWIND_SFRAME_H +#define _ASM_X86_UNWIND_SFRAME_H =20 #ifdef CONFIG_X86_64 =20 @@ -9,4 +9,4 @@ =20 #endif =20 -#endif /* _ASM_X86_UNWIND_USER_SFRAME_H */ +#endif /* _ASM_X86_UNWIND_SFRAME_H */ diff --git a/arch/x86/include/asm/unwind_user.h b/arch/x86/include/asm/unwi= nd_user.h index ae46906c3b39..8fdab3581b86 100644 --- a/arch/x86/include/asm/unwind_user.h +++ b/arch/x86/include/asm/unwind_user.h @@ -55,30 +55,30 @@ static inline int unwind_user_get_reg(unsigned long *va= l, unsigned int regnum) =20 #define ARCH_INIT_USER_FP_FRAME(ws) \ .cfa =3D { \ - .rule =3D UNWIND_USER_CFA_RULE_FP_OFFSET,\ + .rule =3D UNWIND_CFA_RULE_FP_OFFSET,\ .offset =3D 2*(ws), \ }, \ .ra =3D { \ - .rule =3D UNWIND_USER_RULE_CFA_OFFSET_DEREF,\ + .rule =3D UNWIND_RULE_CFA_OFFSET_DEREF,\ .offset =3D -1*(ws), \ }, \ .fp =3D { \ - .rule =3D UNWIND_USER_RULE_CFA_OFFSET_DEREF,\ + .rule =3D UNWIND_RULE_CFA_OFFSET_DEREF,\ .offset =3D -2*(ws), \ }, \ .outermost =3D false, =20 #define ARCH_INIT_USER_FP_ENTRY_FRAME(ws) \ .cfa =3D { \ - .rule =3D UNWIND_USER_CFA_RULE_SP_OFFSET,\ + .rule =3D UNWIND_CFA_RULE_SP_OFFSET,\ .offset =3D 1*(ws), \ }, \ .ra =3D { \ - .rule =3D UNWIND_USER_RULE_CFA_OFFSET_DEREF,\ + .rule =3D UNWIND_RULE_CFA_OFFSET_DEREF,\ .offset =3D -1*(ws), \ }, \ .fp =3D { \ - .rule =3D UNWIND_USER_RULE_RETAIN,\ + .rule =3D UNWIND_RULE_RETAIN,\ }, \ .outermost =3D false, =20 diff --git a/include/linux/sframe.h b/include/linux/sframe.h index b79c5ec09229..0cb2924367bc 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -3,37 +3,46 @@ #define _LINUX_SFRAME_H =20 #include +#include #include -#include =20 -#ifdef CONFIG_HAVE_UNWIND_USER_SFRAME +#ifdef CONFIG_UNWIND_SFRAME_LOOKUP + +enum sframe_sec_type { + SFRAME_KERNEL, + SFRAME_USER, +}; =20 struct sframe_section { - struct rcu_head rcu; + struct rcu_head rcu; #ifdef CONFIG_DYNAMIC_DEBUG - const char *filename; + const char *filename; #endif - unsigned long sframe_start; - unsigned long sframe_end; - unsigned long text_start; - unsigned long text_end; - - unsigned long fdes_start; - unsigned long fres_start; - unsigned long fres_end; - unsigned int num_fdes; - - signed char ra_off; - signed char fp_off; + enum sframe_sec_type sec_type; + unsigned long sframe_start; + unsigned long sframe_end; + unsigned long text_start; + unsigned long text_end; + + unsigned long fdes_start; + unsigned long fres_start; + unsigned long fres_end; + unsigned int num_fdes; + + signed char ra_off; + signed char fp_off; }; =20 +#endif /* CONFIG_UNWIND_SFRAME_LOOKUP */ + +#ifdef CONFIG_HAVE_UNWIND_USER_SFRAME + #define INIT_MM_SFRAME .sframe_mt =3D MTREE_INIT(sframe_mt, 0), extern void sframe_free_mm(struct mm_struct *mm); =20 extern int sframe_add_section(unsigned long sframe_start, unsigned long sf= rame_end, unsigned long text_start, unsigned long text_end); extern int sframe_remove_section(unsigned long sframe_addr); -extern int sframe_find(unsigned long ip, struct unwind_user_frame *frame); =20 static inline bool current_has_sframe(void) { @@ -42,6 +51,8 @@ static inline bool current_has_sframe(void) return mm && !mtree_empty(&mm->sframe_mt); } =20 +extern int sframe_find_user(unsigned long ip, struct unwind_frame *frame); + #else /* !CONFIG_HAVE_UNWIND_USER_SFRAME */ =20 #define INIT_MM_SFRAME @@ -52,9 +63,10 @@ static inline int sframe_add_section(unsigned long sfram= e_start, unsigned long s return -ENOSYS; } static inline int sframe_remove_section(unsigned long sframe_addr) { retur= n -ENOSYS; } -static inline int sframe_find(unsigned long ip, struct unwind_user_frame *= frame) { return -ENOSYS; } static inline bool current_has_sframe(void) { return false; } =20 +static inline int sframe_find_user(unsigned long ip, struct unwind_frame *= frame) { return -ENOSYS; } + #endif /* CONFIG_HAVE_UNWIND_USER_SFRAME */ =20 #endif /* _LINUX_SFRAME_H */ diff --git a/include/linux/unwind_types.h b/include/linux/unwind_types.h new file mode 100644 index 000000000000..08bcb0aa04aa --- /dev/null +++ b/include/linux/unwind_types.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_UNWIND_TYPES_H +#define _LINUX_UNWIND_TYPES_H + +#define UNWIND_RULE_DEREF BIT(31) + +enum unwind_cfa_rule { + UNWIND_CFA_RULE_SP_OFFSET, /* CFA =3D SP + offset */ + UNWIND_CFA_RULE_FP_OFFSET, /* CFA =3D FP + offset */ + UNWIND_CFA_RULE_REG_OFFSET, /* CFA =3D reg + offset */ + /* DEREF variants */ + UNWIND_CFA_RULE_REG_OFFSET_DEREF =3D /* CFA =3D *(reg + offset) */ + UNWIND_CFA_RULE_REG_OFFSET | UNWIND_RULE_DEREF, +}; + +struct unwind_cfa_rule_data { + enum unwind_cfa_rule rule; + s32 offset; + unsigned int regnum; +}; + +enum unwind_rule { + UNWIND_RULE_RETAIN, /* entity =3D entity */ + UNWIND_RULE_CFA_OFFSET, /* entity =3D CFA + offset */ + UNWIND_RULE_REG_OFFSET, /* entity =3D register + offset */ + /* DEREF variants */ + UNWIND_RULE_CFA_OFFSET_DEREF =3D /* entity =3D *(CFA + offset) */ + UNWIND_RULE_CFA_OFFSET | UNWIND_RULE_DEREF, + UNWIND_RULE_REG_OFFSET_DEREF =3D /* entity =3D *(register + offset) */ + UNWIND_RULE_REG_OFFSET | UNWIND_RULE_DEREF, +}; + +struct unwind_rule_data { + enum unwind_rule rule; + s32 offset; + unsigned int regnum; +}; + +struct unwind_frame { + struct unwind_cfa_rule_data cfa; + struct unwind_rule_data ra; + struct unwind_rule_data fp; + bool outermost; +}; + +#endif /* _LINUX_UNWIND_TYPES_H */ diff --git a/include/linux/unwind_user_types.h b/include/linux/unwind_user_= types.h index 059e5c76f2f3..646e5fb774db 100644 --- a/include/linux/unwind_user_types.h +++ b/include/linux/unwind_user_types.h @@ -27,47 +27,6 @@ struct unwind_stacktrace { unsigned long *entries; }; =20 -#define UNWIND_USER_RULE_DEREF BIT(31) - -enum unwind_user_cfa_rule { - UNWIND_USER_CFA_RULE_SP_OFFSET, /* CFA =3D SP + offset */ - UNWIND_USER_CFA_RULE_FP_OFFSET, /* CFA =3D FP + offset */ - UNWIND_USER_CFA_RULE_REG_OFFSET, /* CFA =3D reg + offset */ - /* DEREF variants */ - UNWIND_USER_CFA_RULE_REG_OFFSET_DEREF =3D /* CFA =3D *(reg + offset) */ - UNWIND_USER_CFA_RULE_REG_OFFSET | UNWIND_USER_RULE_DEREF, -}; - -struct unwind_user_cfa_rule_data { - enum unwind_user_cfa_rule rule; - s32 offset; - unsigned int regnum; -}; - -enum unwind_user_rule { - UNWIND_USER_RULE_RETAIN, /* entity =3D entity */ - UNWIND_USER_RULE_CFA_OFFSET, /* entity =3D CFA + offset */ - UNWIND_USER_RULE_REG_OFFSET, /* entity =3D register + offset */ - /* DEREF variants */ - UNWIND_USER_RULE_CFA_OFFSET_DEREF =3D /* entity =3D *(CFA + offset) */ - UNWIND_USER_RULE_CFA_OFFSET | UNWIND_USER_RULE_DEREF, - UNWIND_USER_RULE_REG_OFFSET_DEREF =3D /* entity =3D *(register + offset) = */ - UNWIND_USER_RULE_REG_OFFSET | UNWIND_USER_RULE_DEREF, -}; - -struct unwind_user_rule_data { - enum unwind_user_rule rule; - s32 offset; - unsigned int regnum; -}; - -struct unwind_user_frame { - struct unwind_user_cfa_rule_data cfa; - struct unwind_user_rule_data ra; - struct unwind_user_rule_data fp; - bool outermost; -}; - struct unwind_user_state { unsigned long ip; unsigned long sp; diff --git a/kernel/unwind/Makefile b/kernel/unwind/Makefile index 146038165865..c5f9f8124564 100644 --- a/kernel/unwind/Makefile +++ b/kernel/unwind/Makefile @@ -1,2 +1,2 @@ obj-$(CONFIG_UNWIND_USER) +=3D user.o deferred.o - obj-$(CONFIG_HAVE_UNWIND_USER_SFRAME) +=3D sframe.o + obj-$(CONFIG_UNWIND_SFRAME_LOOKUP) +=3D sframe.o diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index f24997e84e05..ae15f5c06939 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -12,8 +12,8 @@ #include #include #include -#include -#include +#include +#include =20 #include "sframe.h" #include "sframe_debug.h" @@ -44,8 +44,6 @@ struct sframe_fre_internal { unsigned char dw_size; }; =20 -DEFINE_STATIC_SRCU(sframe_srcu); - static __always_inline unsigned char fre_type_to_size(unsigned char fre_ty= pe) { if (fre_type > 2) @@ -60,6 +58,77 @@ static __always_inline unsigned char dataword_size_enum_= to_size(unsigned char da return 1 << dataword_size; } =20 +#ifdef CONFIG_HAVE_UNWIND_USER_SFRAME + +DEFINE_STATIC_SRCU(sframe_srcu); + +#define UNSAFE_USER_COPY(to, from, size, label) \ + unsafe_copy_from_user(to, (void __user *)from, size, label) + +#define UNSAFE_USER_GET(to, from, type, label) \ + unsafe_get_user(to, (type __user *)from, label) + +#else /* !CONFIG_HAVE_UNWIND_USER_SFRAME */ + +#define UNSAFE_USER_COPY(to, from, size, label) do { \ + (void)to; (void)from; (void)size; \ + goto label; \ +} while (0) + +#define UNSAFE_USER_GET(to, from, type, label) do { \ + (void)to; (void)from; \ + goto label; \ +} while (0) + +#endif /* !CONFIG_HAVE_UNWIND_USER_SFRAME */ + +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +#define KERNEL_COPY(to, from, size, label) memcpy(to, (void *)from, size) +#define KERNEL_GET(to, from, type, label) ({ (to) =3D *(type *)(from); }) + +#else /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + +#define KERNEL_COPY(to, from, size, label) do { \ + (void)(to); (void)(from); (void)size; \ + goto label; \ +} while (0) + +#define KERNEL_GET(to, from, type, label) do { \ + (void)(to); (void)(from); \ + goto label; \ +} while (0) + +#endif /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + +#define DATA_COPY(sec, to, from, size, label) \ +({ \ + switch (sec->sec_type) { \ + case SFRAME_KERNEL: \ + KERNEL_COPY(to, from, size, label); \ + break; \ + case SFRAME_USER: \ + UNSAFE_USER_COPY(to, from, size, label); \ + break; \ + default: \ + return -EFAULT; \ + } \ +}) + +#define DATA_GET(sec, to, from, type, label) \ +({ \ + switch (sec->sec_type) { \ + case SFRAME_KERNEL: \ + KERNEL_GET(to, from, type, label); \ + break; \ + case SFRAME_USER: \ + UNSAFE_USER_GET(to, from, type, label); \ + break; \ + default: \ + return -EFAULT; \ + } \ +}) + static __always_inline int __read_fde(struct sframe_section *sec, unsigned int fde_num, struct sframe_fde_internal *fde) @@ -69,8 +138,8 @@ static __always_inline int __read_fde(struct sframe_sect= ion *sec, struct sframe_fda_v3 _fda; =20 fde_addr =3D sec->fdes_start + (fde_num * sizeof(struct sframe_fde_v3)); - unsafe_copy_from_user(&_fde, (void __user *)fde_addr, - sizeof(struct sframe_fde_v3), Efault); + DATA_COPY(sec, &_fde, fde_addr, + sizeof(struct sframe_fde_v3), Efault); =20 func_addr =3D fde_addr + _fde.func_start_off; if (func_addr < sec->text_start || func_addr > sec->text_end) @@ -79,8 +148,8 @@ static __always_inline int __read_fde(struct sframe_sect= ion *sec, fda_addr =3D sec->fres_start + _fde.fres_off; if (fda_addr + sizeof(struct sframe_fda_v3) > sec->fres_end) return -EINVAL; - unsafe_copy_from_user(&_fda, (void __user *)fda_addr, - sizeof(struct sframe_fda_v3), Efault); + DATA_COPY(sec, &_fda, fda_addr, + sizeof(struct sframe_fda_v3), Efault); =20 fde->func_addr =3D func_addr; fde->func_size =3D _fde.func_size; @@ -102,21 +171,21 @@ static __always_inline int __find_fde(struct sframe_s= ection *sec, struct sframe_fde_internal *fde) { unsigned long func_addr_low =3D 0, func_addr_high =3D ULONG_MAX; - struct sframe_fde_v3 __user *first, *low, *high, *found =3D NULL; + struct sframe_fde_v3 *first, *low, *high, *found =3D NULL; int ret; =20 - first =3D (void __user *)sec->fdes_start; + first =3D (void *)sec->fdes_start; low =3D first; high =3D first + sec->num_fdes - 1; =20 while (low <=3D high) { - struct sframe_fde_v3 __user *mid; + struct sframe_fde_v3 *mid; s64 func_off; unsigned long func_addr; =20 mid =3D low + ((high - low) / 2); =20 - unsafe_get_user(func_off, (s64 __user *)mid, Efault); + DATA_GET(sec, func_off, mid, s64, Efault); func_addr =3D (unsigned long)mid + func_off; =20 if (ip >=3D func_addr) { @@ -154,47 +223,47 @@ static __always_inline int __find_fde(struct sframe_s= ection *sec, return -EFAULT; } =20 -#define ____UNSAFE_GET_USER_INC(to, from, type, label) \ +#define ____GET_INC(sec, to, from, type, label) \ ({ \ type __to; \ - unsafe_get_user(__to, (type __user *)from, label); \ + DATA_GET(sec, __to, from, type, label); \ from +=3D sizeof(__to); \ to =3D __to; \ }) =20 -#define __UNSAFE_GET_USER_INC(to, from, size, label, u_or_s) \ +#define __GET_INC(sec, to, from, size, label, u_or_s) \ ({ \ switch (size) { \ case 1: \ - ____UNSAFE_GET_USER_INC(to, from, u_or_s##8, label); \ + ____GET_INC(sec, to, from, u_or_s##8, label); \ break; \ case 2: \ - ____UNSAFE_GET_USER_INC(to, from, u_or_s##16, label); \ + ____GET_INC(sec, to, from, u_or_s##16, label); \ break; \ case 4: \ - ____UNSAFE_GET_USER_INC(to, from, u_or_s##32, label); \ + ____GET_INC(sec, to, from, u_or_s##32, label); \ break; \ default: \ return -EFAULT; \ } \ }) =20 -#define UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label) \ - __UNSAFE_GET_USER_INC(to, from, size, label, u) +#define GET_UNSIGNED_INC(sec, to, from, size, label) \ + __GET_INC(sec, to, from, size, label, u) =20 -#define UNSAFE_GET_USER_SIGNED_INC(to, from, size, label) \ - __UNSAFE_GET_USER_INC(to, from, size, label, s) +#define GET_SIGNED_INC(sec, to, from, size, label) \ + __GET_INC(sec, to, from, size, label, s) =20 -#define UNSAFE_GET_USER_INC(to, from, size, label) \ - _Generic(to, \ - u8 : UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label), \ - u16 : UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label), \ - u32 : UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label), \ - u64 : UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label), \ - s8 : UNSAFE_GET_USER_SIGNED_INC(to, from, size, label), \ - s16 : UNSAFE_GET_USER_SIGNED_INC(to, from, size, label), \ - s32 : UNSAFE_GET_USER_SIGNED_INC(to, from, size, label), \ - s64 : UNSAFE_GET_USER_SIGNED_INC(to, from, size, label)) +#define GET_INC(sec, to, from, size, label) \ + _Generic(to, \ + u8 : GET_UNSIGNED_INC(sec, to, from, size, label), \ + u16 : GET_UNSIGNED_INC(sec, to, from, size, label), \ + u32 : GET_UNSIGNED_INC(sec, to, from, size, label), \ + u64 : GET_UNSIGNED_INC(sec, to, from, size, label), \ + s8 : GET_SIGNED_INC(sec, to, from, size, label), \ + s16 : GET_SIGNED_INC(sec, to, from, size, label), \ + s32 : GET_SIGNED_INC(sec, to, from, size, label), \ + s64 : GET_SIGNED_INC(sec, to, from, size, label)) =20 static __always_inline int __read_regular_fre_datawords(struct sframe_section *sec, @@ -207,19 +276,19 @@ __read_regular_fre_datawords(struct sframe_section *s= ec, s32 cfa_off, ra_off, fp_off; unsigned int cfa_regnum; =20 - UNSAFE_GET_USER_INC(cfa_off, cur, dataword_size, Efault); + GET_INC(sec, cfa_off, cur, dataword_size, Efault); dataword_count--; =20 ra_off =3D sec->ra_off; if (!ra_off && dataword_count) { dataword_count--; - UNSAFE_GET_USER_INC(ra_off, cur, dataword_size, Efault); + GET_INC(sec, ra_off, cur, dataword_size, Efault); } =20 fp_off =3D sec->fp_off; if (!fp_off && dataword_count) { dataword_count--; - UNSAFE_GET_USER_INC(fp_off, cur, dataword_size, Efault); + GET_INC(sec, fp_off, cur, dataword_size, Efault); } =20 if (dataword_count) @@ -255,17 +324,17 @@ __read_flex_fde_fre_datawords(struct sframe_section *= sec, =20 if (dataword_count < 2) return -EFAULT; - UNSAFE_GET_USER_INC(cfa_ctl, cur, dataword_size, Efault); - UNSAFE_GET_USER_INC(cfa_off, cur, dataword_size, Efault); + GET_INC(sec, cfa_ctl, cur, dataword_size, Efault); + GET_INC(sec, cfa_off, cur, dataword_size, Efault); dataword_count -=3D 2; =20 ra_off =3D sec->ra_off; ra_ctl =3D ra_off ? 2 : 0; /* regnum=3D0, deref_p=3D(ra_off !=3D 0), reg_= p=3D0 */ if (dataword_count >=3D 2) { - UNSAFE_GET_USER_INC(ra_ctl, cur, dataword_size, Efault); + GET_INC(sec, ra_ctl, cur, dataword_size, Efault); dataword_count--; if (ra_ctl) { - UNSAFE_GET_USER_INC(ra_off, cur, dataword_size, Efault); + GET_INC(sec, ra_off, cur, dataword_size, Efault); dataword_count--; } else { /* Padding RA location info */ @@ -276,10 +345,10 @@ __read_flex_fde_fre_datawords(struct sframe_section *= sec, fp_off =3D sec->fp_off; fp_ctl =3D fp_off ? 2 : 0; /* regnum=3D0, deref_p=3D(fp_off !=3D 0), reg_= p=3D0 */ if (dataword_count >=3D 2) { - UNSAFE_GET_USER_INC(fp_ctl, cur, dataword_size, Efault); + GET_INC(sec, fp_ctl, cur, dataword_size, Efault); dataword_count--; if (fp_ctl) { - UNSAFE_GET_USER_INC(fp_off, cur, dataword_size, Efault); + GET_INC(sec, fp_off, cur, dataword_size, Efault); dataword_count--; } else { /* Padding FP location info */ @@ -353,11 +422,11 @@ static __always_inline int __read_fre(struct sframe_s= ection *sec, if (fre_addr + addr_size + 1 > sec->fres_end) return -EFAULT; =20 - UNSAFE_GET_USER_INC(ip_off, cur, addr_size, Efault); + GET_INC(sec, ip_off, cur, addr_size, Efault); if (fde_pctype =3D=3D SFRAME_FDE_PCTYPE_INC && ip_off > fde->func_size) return -EFAULT; =20 - UNSAFE_GET_USER_INC(info, cur, 1, Efault); + GET_INC(sec, info, cur, 1, Efault); dataword_count =3D SFRAME_V3_FRE_DATAWORD_COUNT(info); dataword_size =3D dataword_size_enum_to_size(SFRAME_V3_FRE_DATAWORD_SIZE= (info)); if (!dataword_size) @@ -380,7 +449,7 @@ static __always_inline int __read_fre(struct sframe_sec= tion *sec, } =20 static __always_inline int -sframe_init_cfa_rule_data(struct unwind_user_cfa_rule_data *cfa_rule_data, +sframe_init_cfa_rule_data(struct unwind_cfa_rule_data *cfa_rule_data, u32 ctlword, s32 offset) { bool deref_p =3D SFRAME_V3_FLEX_FDE_CTLWORD_DEREF_P(ctlword); @@ -391,13 +460,13 @@ sframe_init_cfa_rule_data(struct unwind_user_cfa_rule= _data *cfa_rule_data, =20 switch (regnum) { case SFRAME_REG_SP: - cfa_rule_data->rule =3D UNWIND_USER_CFA_RULE_SP_OFFSET; + cfa_rule_data->rule =3D UNWIND_CFA_RULE_SP_OFFSET; break; case SFRAME_REG_FP: - cfa_rule_data->rule =3D UNWIND_USER_CFA_RULE_FP_OFFSET; + cfa_rule_data->rule =3D UNWIND_CFA_RULE_FP_OFFSET; break; default: - cfa_rule_data->rule =3D UNWIND_USER_CFA_RULE_REG_OFFSET; + cfa_rule_data->rule =3D UNWIND_CFA_RULE_REG_OFFSET; cfa_rule_data->regnum =3D regnum; } } else { @@ -405,7 +474,7 @@ sframe_init_cfa_rule_data(struct unwind_user_cfa_rule_d= ata *cfa_rule_data, } =20 if (deref_p) - cfa_rule_data->rule |=3D UNWIND_USER_RULE_DEREF; + cfa_rule_data->rule |=3D UNWIND_RULE_DEREF; =20 cfa_rule_data->offset =3D offset; =20 @@ -413,27 +482,27 @@ sframe_init_cfa_rule_data(struct unwind_user_cfa_rule= _data *cfa_rule_data, } =20 static __always_inline void -sframe_init_rule_data(struct unwind_user_rule_data *rule_data, +sframe_init_rule_data(struct unwind_rule_data *rule_data, u32 ctlword, s32 offset) { bool deref_p =3D SFRAME_V3_FLEX_FDE_CTLWORD_DEREF_P(ctlword); bool reg_p =3D SFRAME_V3_FLEX_FDE_CTLWORD_REG_P(ctlword); =20 if (!ctlword && !offset) { - rule_data->rule =3D UNWIND_USER_RULE_RETAIN; + rule_data->rule =3D UNWIND_RULE_RETAIN; return; } if (reg_p) { unsigned int regnum =3D SFRAME_V3_FLEX_FDE_CTLWORD_REGNUM(ctlword); =20 - rule_data->rule =3D UNWIND_USER_RULE_REG_OFFSET; + rule_data->rule =3D UNWIND_RULE_REG_OFFSET; rule_data->regnum =3D regnum; } else { - rule_data->rule =3D UNWIND_USER_RULE_CFA_OFFSET; + rule_data->rule =3D UNWIND_RULE_CFA_OFFSET; } =20 if (deref_p) - rule_data->rule |=3D UNWIND_USER_RULE_DEREF; + rule_data->rule |=3D UNWIND_RULE_DEREF; =20 rule_data->offset =3D offset; } @@ -441,7 +510,7 @@ sframe_init_rule_data(struct unwind_user_rule_data *rul= e_data, static __always_inline int __find_fre(struct sframe_section *sec, struct sframe_fde_internal *fde, unsigned long ip, - struct unwind_user_frame *frame) + struct unwind_frame *frame) { unsigned char fde_pctype =3D SFRAME_V3_FDE_PCTYPE(fde->info); struct sframe_fre_internal *fre, *prev_fre =3D NULL; @@ -501,40 +570,18 @@ static __always_inline int __find_fre(struct sframe_s= ection *sec, return 0; } =20 -int sframe_find(unsigned long ip, struct unwind_user_frame *frame) +static __always_inline int __sframe_find(struct sframe_section *sec, + unsigned long ip, + struct unwind_frame *frame) { - struct mm_struct *mm =3D current->mm; - struct sframe_section *sec; struct sframe_fde_internal fde; int ret; =20 - if (!mm) - return -EINVAL; - - guard(srcu)(&sframe_srcu); - - sec =3D mtree_load(&mm->sframe_mt, ip); - if (!sec) - return -EINVAL; - - if (!user_read_access_begin((void __user *)sec->sframe_start, - sec->sframe_end - sec->sframe_start)) - return -EFAULT; - ret =3D __find_fde(sec, ip, &fde); if (ret) - goto end; - - ret =3D __find_fre(sec, &fde, ip, frame); -end: - user_read_access_end(); - - if (ret =3D=3D -EFAULT) { - dbg_sec("removing bad .sframe section\n"); - WARN_ON_ONCE(sframe_remove_section(sec->sframe_start)); - } + return ret; =20 - return ret; + return __find_fre(sec, &fde, ip, frame); } =20 #ifdef CONFIG_SFRAME_VALIDATION @@ -657,20 +704,23 @@ static int sframe_validate_section(struct sframe_sect= ion *sec) { return 0; } #endif /* !CONFIG_SFRAME_VALIDATION */ =20 =20 -static void free_section(struct sframe_section *sec) -{ - dbg_free(sec); - kfree(sec); -} - static int sframe_read_header(struct sframe_section *sec) { unsigned long header_end, fdes_start, fdes_end, fres_start, fres_end; struct sframe_header shdr; unsigned int num_fdes; =20 - if (copy_from_user(&shdr, (void __user *)sec->sframe_start, sizeof(shdr))= ) { - dbg_sec("header usercopy failed\n"); + switch (sec->sec_type) { + case SFRAME_USER: + if (copy_from_user(&shdr, (void __user *)sec->sframe_start, sizeof(shdr)= )) { + dbg_sec("header usercopy failed\n"); + return -EFAULT; + } + break; + case SFRAME_KERNEL: + shdr =3D *(struct sframe_header *)sec->sframe_start; + break; + default: return -EFAULT; } =20 @@ -717,6 +767,45 @@ static int sframe_read_header(struct sframe_section *s= ec) return 0; } =20 +#ifdef CONFIG_HAVE_UNWIND_USER_SFRAME + +int sframe_find_user(unsigned long ip, struct unwind_frame *frame) +{ + struct mm_struct *mm =3D current->mm; + struct sframe_section *sec; + int ret; + + if (!mm) + return -EINVAL; + + guard(srcu)(&sframe_srcu); + + sec =3D mtree_load(&mm->sframe_mt, ip); + if (!sec) + return -EINVAL; + + if (!user_read_access_begin((void __user *)sec->sframe_start, + sec->sframe_end - sec->sframe_start)) + return -EFAULT; + + ret =3D __sframe_find(sec, ip, frame); + + user_read_access_end(); + + if (ret =3D=3D -EFAULT) { + dbg_sec("removing bad .sframe section\n"); + WARN_ON_ONCE(sframe_remove_section(sec->sframe_start)); + } + + return ret; +} + +static void free_section(struct sframe_section *sec) +{ + dbg_free(sec); + kfree(sec); +} + int sframe_add_section(unsigned long sframe_start, unsigned long sframe_en= d, unsigned long text_start, unsigned long text_end) { @@ -753,6 +842,7 @@ int sframe_add_section(unsigned long sframe_start, unsi= gned long sframe_end, if (!sec) return -ENOMEM; =20 + sec->sec_type =3D SFRAME_USER; sec->sframe_start =3D sframe_start; sec->sframe_end =3D sframe_end; sec->text_start =3D text_start; @@ -838,3 +928,5 @@ void sframe_free_mm(struct mm_struct *mm) =20 mtree_destroy(&mm->sframe_mt); } + +#endif /* CONFIG_HAVE_UNWIND_USER_SFRAME */ diff --git a/kernel/unwind/user.c b/kernel/unwind/user.c index eb7d9489f671..9e57dd79559a 100644 --- a/kernel/unwind/user.c +++ b/kernel/unwind/user.c @@ -8,6 +8,7 @@ #include #include #include +#include =20 #define for_each_user_frame(state) \ for (unwind_user_start(state); !(state)->done; unwind_user_next(state)) @@ -28,7 +29,7 @@ get_user_word(unsigned long *word, unsigned long base, in= t off, unsigned int ws) } =20 static int unwind_user_next_common(struct unwind_user_state *state, - const struct unwind_user_frame *frame) + const struct unwind_frame *frame) { unsigned long cfa, fp, ra; =20 @@ -40,16 +41,16 @@ static int unwind_user_next_common(struct unwind_user_s= tate *state, =20 /* Get the Canonical Frame Address (CFA) */ switch (frame->cfa.rule) { - case UNWIND_USER_CFA_RULE_SP_OFFSET: + case UNWIND_CFA_RULE_SP_OFFSET: cfa =3D state->sp; break; - case UNWIND_USER_CFA_RULE_FP_OFFSET: + case UNWIND_CFA_RULE_FP_OFFSET: if (state->fp < state->sp) return -EINVAL; cfa =3D state->fp; break; - case UNWIND_USER_CFA_RULE_REG_OFFSET: - case UNWIND_USER_CFA_RULE_REG_OFFSET_DEREF: + case UNWIND_CFA_RULE_REG_OFFSET: + case UNWIND_CFA_RULE_REG_OFFSET_DEREF: if (!state->topmost || unwind_user_get_reg(&cfa, frame->cfa.regnum)) return -EINVAL; break; @@ -58,7 +59,7 @@ static int unwind_user_next_common(struct unwind_user_sta= te *state, return -EINVAL; } cfa +=3D frame->cfa.offset; - if (frame->cfa.rule & UNWIND_USER_RULE_DEREF && + if (frame->cfa.rule & UNWIND_RULE_DEREF && get_user_word(&cfa, cfa, 0, state->ws)) return -EINVAL; =20 @@ -76,16 +77,16 @@ static int unwind_user_next_common(struct unwind_user_s= tate *state, =20 /* Get the Return Address (RA) */ switch (frame->ra.rule) { - case UNWIND_USER_RULE_RETAIN: + case UNWIND_RULE_RETAIN: if (!state->topmost || unwind_user_get_ra_reg(&ra)) return -EINVAL; break; /* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */ - case UNWIND_USER_RULE_CFA_OFFSET_DEREF: + case UNWIND_RULE_CFA_OFFSET_DEREF: ra =3D cfa + frame->ra.offset; break; - case UNWIND_USER_RULE_REG_OFFSET: - case UNWIND_USER_RULE_REG_OFFSET_DEREF: + case UNWIND_RULE_REG_OFFSET: + case UNWIND_RULE_REG_OFFSET_DEREF: if (!state->topmost || unwind_user_get_reg(&ra, frame->ra.regnum)) return -EINVAL; ra +=3D frame->ra.offset; @@ -94,21 +95,21 @@ static int unwind_user_next_common(struct unwind_user_s= tate *state, WARN_ON_ONCE(1); return -EINVAL; } - if (frame->ra.rule & UNWIND_USER_RULE_DEREF && + if (frame->ra.rule & UNWIND_RULE_DEREF && get_user_word(&ra, ra, 0, state->ws)) return -EINVAL; =20 /* Get the Frame Pointer (FP) */ switch (frame->fp.rule) { - case UNWIND_USER_RULE_RETAIN: + case UNWIND_RULE_RETAIN: fp =3D state->fp; break; /* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */ - case UNWIND_USER_RULE_CFA_OFFSET_DEREF: + case UNWIND_RULE_CFA_OFFSET_DEREF: fp =3D cfa + frame->fp.offset; break; - case UNWIND_USER_RULE_REG_OFFSET: - case UNWIND_USER_RULE_REG_OFFSET_DEREF: + case UNWIND_RULE_REG_OFFSET: + case UNWIND_RULE_REG_OFFSET_DEREF: if (!state->topmost || unwind_user_get_reg(&fp, frame->fp.regnum)) return -EINVAL; fp +=3D frame->fp.offset; @@ -117,7 +118,7 @@ static int unwind_user_next_common(struct unwind_user_s= tate *state, WARN_ON_ONCE(1); return -EINVAL; } - if (frame->fp.rule & UNWIND_USER_RULE_DEREF && + if (frame->fp.rule & UNWIND_RULE_DEREF && get_user_word(&fp, fp, 0, state->ws)) return -EINVAL; =20 @@ -133,13 +134,13 @@ static int unwind_user_next_fp(struct unwind_user_sta= te *state) struct pt_regs *regs =3D task_pt_regs(current); =20 if (state->topmost && unwind_user_at_function_start(regs)) { - const struct unwind_user_frame fp_entry_frame =3D { + const struct unwind_frame fp_entry_frame =3D { ARCH_INIT_USER_FP_ENTRY_FRAME(state->ws) }; return unwind_user_next_common(state, &fp_entry_frame); } =20 - const struct unwind_user_frame fp_frame =3D { + const struct unwind_frame fp_frame =3D { ARCH_INIT_USER_FP_FRAME(state->ws) }; return unwind_user_next_common(state, &fp_frame); @@ -147,10 +148,10 @@ static int unwind_user_next_fp(struct unwind_user_sta= te *state) =20 static int unwind_user_next_sframe(struct unwind_user_state *state) { - struct unwind_user_frame frame; + struct unwind_frame frame; =20 /* sframe expects the frame to be local storage */ - if (sframe_find(state->ip, &frame)) + if (sframe_find_user(state->ip, &frame)) return -ENOENT; return unwind_user_next_common(state, &frame); } --=20 2.54.0.rc1.555.g9c883467ad-goog From nobody Wed Jun 17 02:50:06 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11D913A6EE6 for ; Tue, 21 Apr 2026 22:52:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811937; cv=none; b=Vlqo8XHyLVIX8bvSwy7v8ZeZN0ZY1BBUbh7Q+RVbB5PVfTXKfc3W6mJaVqKVI2PcmSfL+dyHKVbc5iOZ5SbJQIv95LM9lRaWn7wL4HujWwwcL3YwN9mS4bY+OMTMYa0d6VhnIkN0tvwFu7IETvJnKpQAdJcn08gECQfajU3GNLQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811937; c=relaxed/simple; bh=jS7x5BKiEgeUWjadfca406qjy01BOwaN0+ZxBvDMTQQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Todkod70UmtjHY6c62qbnLaQeOlgqydsDUnA32nKpsqK//FJpIapVSxDTNOO/9fWMvm5DDmLLLe7Dm71B2k458rIgo8k3XdeKDngWuZtkOvXjxFd7VsUpbjRqcm938HtloLtMW7kT8Qg0Lml/R2SSukXUasQEPgTrqvzBoXQXMQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YSnTmA09; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YSnTmA09" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2b249541063so42074565ad.3 for ; Tue, 21 Apr 2026 15:52:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776811933; x=1777416733; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ANGrhBV2LcMNZY9HEi2YWokunc4kr3qHq5bEiF7UAk8=; b=YSnTmA09JbIZ3jk6wlpNy+HaRwnNPYPun6PWTO3cOgctN8HLKQm1UttULUptNflZ3u RLolLDX5xPRnhN/eXS1sxH4BlOCPIe5+9M+/Zipn+/XuYYF54jNaqy1myXV4Qmwu0xCZ WwVgl0crLhgfk4+9ZO0LZYbKl5exka9rSVihVSTw4p7xe3OHJKO+8xKxP0P3SNWeBBpO 2Z2tD8iIwuSkqSGWLztNV0vuQUh5WMwX7Ug0OuoNWq0Xn+EfRY5LyU7lYoR2sbPomj09 drmxQOiqokl05OdjAHab2JHbamNiUXTiGGnCNspt1KasMMfY5ygv+Oh05W+4QsqEQmuU mvgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776811933; x=1777416733; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ANGrhBV2LcMNZY9HEi2YWokunc4kr3qHq5bEiF7UAk8=; b=Xx2Ay3sz1LUm/Z30bvm1u4IkYFAgFOD+T6BL8Jx+hwWa/9h7MsX/I0uwtqYeltbx1J 4gUnavKp0B5TlMwqPcIjEUXkDoa4Vsbng1SdMnzbq2dpBPPEWsMv8RwILZUDzbjSzjxh QzRyGXsZ2TFGTvnlxgxB/kXomfJe8mYslPG5iPcWf8kab5YTW3cdGeGUXrja+C6fHzae skx8dX6C3tRevhSCq53wA9fS2n7od0+BhFVBagKM/nBrJvrB1/l0Cnwi6nZVtZxmfy7e 9Uvf4Ci4UhvnpfQFe1t83Ak5eha/cYWVRkZ0jjeaqLGjJ1GNsK6ZF0g14XyLMP6WKIu+ c8eQ== X-Forwarded-Encrypted: i=1; AFNElJ8TYpjf7+ztHuZ0sZ4ZZhZjup9Pfr6ws0XlX86VXPtO+li1FlWBpQ2c3ujuHHv052YMeIgEnEWfE3CPits=@vger.kernel.org X-Gm-Message-State: AOJu0YzRpRUwgldGgySRySKwNLExDh1kegCwb88rEBem/dnzLeiVnKxb 0W7Cfk19H1nk9pkEOP+1bkQZB/EDiYMovI01kGQ+ymEHj60MJibfcr6rKwNmgJx181UMDg1RGZ0 RGiZAzCaplv4i70YaY/UExBBWkQ== X-Received: from plgb9.prod.google.com ([2002:a17:902:d509:b0:2b2:49fe:d6e9]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:3c43:b0:2ae:6259:5aff with SMTP id d9443c01a7336-2b5f9e77400mr207036455ad.6.1776811933130; Tue, 21 Apr 2026 15:52:13 -0700 (PDT) Date: Tue, 21 Apr 2026 22:51:54 +0000 In-Reply-To: <20260421225200.1198447-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260421225200.1198447-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.rc1.555.g9c883467ad-goog Message-ID: <20260421225200.1198447-3-dylanbhatch@google.com> Subject: [PATCH v4 2/8] arm64, unwind: build kernel with sframe V3 info From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Build with -Wa,--gsframe-3 flags to generate a .sframe section. This will be used for in-kernel reliable stacktrace in cases where the frame pointer alone is insufficient. Currently, the sframe format only supports arm64, x86_64 and s390x architectures. Signed-off-by: Weinan Liu Reviewed-by: Prasanna Kumar T S M Signed-off-by: Dylan Hatch Reviewed-by: Jens Remus --- MAINTAINERS | 1 + Makefile | 8 ++++++++ arch/Kconfig | 21 +++++++++++++++++++++ arch/arm64/Kconfig | 1 + arch/arm64/include/asm/unwind_sframe.h | 8 ++++++++ arch/arm64/kernel/vdso/Makefile | 2 +- include/asm-generic/sections.h | 4 ++++ include/asm-generic/vmlinux.lds.h | 15 +++++++++++++++ 8 files changed, 59 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/include/asm/unwind_sframe.h diff --git a/MAINTAINERS b/MAINTAINERS index cfc7dec88da4..a7d75f9cb5f4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -27561,6 +27561,7 @@ STACK UNWINDING M: Josh Poimboeuf M: Steven Rostedt S: Maintained +F: arch/*/include/asm/unwind_sframe.h F: include/linux/sframe.h F: include/linux/unwind*.h F: kernel/unwind/ diff --git a/Makefile b/Makefile index 2b15f0b4a0cb..6c94a5257679 100644 --- a/Makefile +++ b/Makefile @@ -1110,6 +1110,14 @@ endif # Ensure compilers do not transform certain loops into calls to wcslen() KBUILD_CFLAGS +=3D -fno-builtin-wcslen =20 +# build with sframe table +ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME +CC_FLAGS_SFRAME :=3D -Wa,--gsframe-3 +KBUILD_CFLAGS +=3D $(CC_FLAGS_SFRAME) +KBUILD_AFLAGS +=3D $(CC_FLAGS_SFRAME) +export CC_FLAGS_SFRAME +endif + # change __FILE__ to the relative path to the source directory ifdef building_out_of_srctree KBUILD_CPPFLAGS +=3D -fmacro-prefix-map=3D$(srcroot)/=3D diff --git a/arch/Kconfig b/arch/Kconfig index d7caf2e245ce..8d27b3249e7a 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -520,6 +520,27 @@ config SFRAME_VALIDATION =20 If unsure, say N. =20 +config ARCH_SUPPORTS_UNWIND_KERNEL_SFRAME + bool + help + An architecture can select this if it enables the SFrame (Simple + Frame) unwinder for unwinding kernel stack traces. It uses an unwind + table that is directly generated by the toolchain based on DWARF CFI + information. + +config HAVE_UNWIND_KERNEL_SFRAME + bool "Sframe unwinder" + depends on AS_SFRAME3 + depends on 64BIT + depends on ARCH_SUPPORTS_UNWIND_KERNEL_SFRAME + select UNWIND_SFRAME_LOOKUP + help + This option enables the SFrame (Simple Frame) unwinder for unwinding + kernel stack traces. It uses unwind an table that is directly + generated by the toolchain based on DWARF CFI information. In + practice, this can provide more reliable stacktrace results than + unwinding with frame pointers alone. + config HAVE_PERF_REGS bool help diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 38dba5f7e4d2..f7ae8eaaadc4 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -112,6 +112,7 @@ config ARM64 select ARCH_SUPPORTS_SCHED_SMT select ARCH_SUPPORTS_SCHED_CLUSTER select ARCH_SUPPORTS_SCHED_MC + select ARCH_SUPPORTS_UNWIND_KERNEL_SFRAME select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH select ARCH_WANT_COMPAT_IPC_PARSE_VERSION if COMPAT select ARCH_WANT_DEFAULT_BPF_JIT diff --git a/arch/arm64/include/asm/unwind_sframe.h b/arch/arm64/include/as= m/unwind_sframe.h new file mode 100644 index 000000000000..876412881196 --- /dev/null +++ b/arch/arm64/include/asm/unwind_sframe.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_ARM64_UNWIND_SFRAME_H +#define _ASM_ARM64_UNWIND_SFRAME_H + +#define SFRAME_REG_SP 31 +#define SFRAME_REG_FP 29 + +#endif /* _ASM_ARM64_UNWIND_SFRAME_H */ diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makef= ile index 7dec05dd33b7..c60ef921956f 100644 --- a/arch/arm64/kernel/vdso/Makefile +++ b/arch/arm64/kernel/vdso/Makefile @@ -38,7 +38,7 @@ ccflags-y +=3D -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO CC_FLAGS_REMOVE_VDSO :=3D $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) \ $(RANDSTRUCT_CFLAGS) $(KSTACK_ERASE_CFLAGS) \ $(GCC_PLUGINS_CFLAGS) \ - $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) \ + $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) $(CC_FLAGS_SFRAME) \ -Wmissing-prototypes -Wmissing-declarations =20 CC_FLAGS_ADD_VDSO :=3D -O2 -mcmodel=3Dtiny -fasynchronous-unwind-tables diff --git a/include/asm-generic/sections.h b/include/asm-generic/sections.h index 0755bc39b0d8..336d27011a58 100644 --- a/include/asm-generic/sections.h +++ b/include/asm-generic/sections.h @@ -31,6 +31,7 @@ * __irqentry_text_start, __irqentry_text_end * __softirqentry_text_start, __softirqentry_text_end * __start_opd, __end_opd + * __start_sframe, __end_sframe */ extern char _text[], _stext[], _etext[]; extern char _data[], _sdata[], _edata[]; @@ -53,6 +54,9 @@ extern char __ctors_start[], __ctors_end[]; /* Start and end of .opd section - used for function descriptors. */ extern char __start_opd[], __end_opd[]; =20 +/* Start and end of .sframe section - used for stack unwinding. */ +extern char __start_sframe[], __end_sframe[]; + /* Start and end of instrumentation protected text section */ extern char __noinstr_text_start[], __noinstr_text_end[]; =20 diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinu= x.lds.h index 1e1580febe4b..090da633db92 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -491,6 +491,8 @@ *(.rodata1) \ } \ \ + SFRAME \ + \ /* PCI quirks */ \ .pci_fixup : AT(ADDR(.pci_fixup) - LOAD_OFFSET) { \ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_early, _pci_fixups_early, __start= , __end) \ @@ -911,6 +913,19 @@ #define TRACEDATA #endif =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME +#define SFRAME \ + /* sframe */ \ + .sframe : AT(ADDR(.sframe) - LOAD_OFFSET) { \ + __start_sframe =3D .; \ + KEEP(*(.sframe)) \ + KEEP(*(.init.sframe)) \ + __end_sframe =3D .; \ + } +#else +#define SFRAME +#endif + #ifdef CONFIG_PRINTK_INDEX #define PRINTK_INDEX \ .printk_index : AT(ADDR(.printk_index) - LOAD_OFFSET) { \ --=20 2.54.0.rc1.555.g9c883467ad-goog From nobody Wed Jun 17 02:50:06 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23F193A6F06 for ; Tue, 21 Apr 2026 22:52:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811938; cv=none; b=TnwTmTL3oVNO9vy8MQlHpo66zXIbsx835RrLMeMlLPO8EKJxpsbQqiznn2paYJrhvSY23l+3rlXo96lxVrMVB21gE8KWnBZ4e/Z8I9dbHNjc6NqhUJFLX3dqQPu+6VqnfaBnJa3gw6lg7QxQMEVe+8V60GcnFrPNXbsgz4jlVpQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811938; c=relaxed/simple; bh=udffv3jNsveJNyWjHMP0HSXsJ23lIpfFHViHlEMIDZw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qABhuJGqDNga/Hr+ml+Cwuz1oCLWxJrT/Wt9IrMgWNwOBqRLYCvWgrmkqCWiPwrHlVoGBA0t3ak/UB4s33vbnjFSLDgfdn6vyo70WSDIGUeZ69QbqFxeViJsj79Ub9gPnnhBpQ8EYTugxuQT0pT/DFfnmSVif+VecExApnBq3uE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cxYHOJ4G; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cxYHOJ4G" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-358e425c261so5623228a91.3 for ; Tue, 21 Apr 2026 15:52:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776811936; x=1777416736; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=zNlZDyegmOFELN70B6bCpVLxtkFdBX1CWXSw170KDZw=; b=cxYHOJ4GQCzreXIoW6ThkcALOWw/RI0kOF1PrS/i5Uw8PUytIb1G2tBKEgNtZY31jx txGw1RywitrFtM7laUfnIUdeUzTswHBSPKjvUE+7da0uP+7qN7f8yMhqcM/oyZqmXy0K panydf9zqInBnlHgvWWSdQx2Ki9YdfhAmc9KeINastyt7SMKz29Hz1MU37kco5RnLloN 8Zn+gpzL00cGjisWqA4LZ6YIKLN38FHaWPSae5Qav5SZHkrfoRQAAluZV3Z+3dCityEs qSwyrch1bjFWUyh91BIKzuRREhlwOiFrNhKjglAUgk2X98MGx1jbe97XjsYaC9dyTLM1 rGeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776811936; x=1777416736; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zNlZDyegmOFELN70B6bCpVLxtkFdBX1CWXSw170KDZw=; b=Zc+z/Y8IjE+6BIjIqzjE2dRZIiRuzWswCpn7Ksc4FMhK0zD1jRcvnKPsekzL1lTyVT EA5fzVbG7sTRJlCOfU5Bp+aS5XkKZ50cgjjzjvZl4XnPM0AWLSDVzs3f7gtUfMV5riQD M8Y95zW3tLm2PwbuLwV9YsFJTg91l1gK/T8RlYBxwn6a+RlNPMc6WPyZK/0zAECgQ92p 278vFwBbHABluUxZHhW0B0c+xTA5jEm6xZ3dV1IauP4WoAlFxL1Hw3ShBADkescsRPNo woSMEbaikq8L8Lv4/zOdFjkQ3t8AMKmkGtY7zGzf2pkbL8V09RPcq5wA81P0RD4BuNgj pHcg== X-Forwarded-Encrypted: i=1; AFNElJ/Be7Ay/Z1y0kvYeq6M+ftN8j6r01lIwA95nRoSW5xl1iEptgGyC9bbHxrOEwTbgCoyXYag5PiIvjH/HIc=@vger.kernel.org X-Gm-Message-State: AOJu0YxNK6yZ5PZMRmqv/kRxImEMWqW4kSTTfn/3YPi9pCpmjTvCsUu/ 1+KzBHHN1mL+6drCZ/CCeTfTru+607HtDm+W1y8hY1uVQNfpS0uCIhR9sa9KfjLxJX5ZNupeSxX EFs+WqTKb0mE8LjMKPjSHtRHzzw== X-Received: from pjrx9.prod.google.com ([2002:a17:90a:bc89:b0:35d:9dbb:bda8]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3811:b0:35f:c729:de9f with SMTP id 98e67ed59e1d1-361404c2053mr20137572a91.27.1776811936241; Tue, 21 Apr 2026 15:52:16 -0700 (PDT) Date: Tue, 21 Apr 2026 22:51:55 +0000 In-Reply-To: <20260421225200.1198447-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260421225200.1198447-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.rc1.555.g9c883467ad-goog Message-ID: <20260421225200.1198447-4-dylanbhatch@google.com> Subject: [PATCH v4 3/8] arm64: entry: add unwind info for various kernel entries From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Weinan Liu DWARF CFI (Call Frame Information) specifies how to recover the return address and callee-saved registers at each PC in a given function. Compilers are able to generate the CFI annotations when they compile the code to assembly language. For handcrafted assembly, we need to annotate them by hand. Annotate minimal CFI to enable stacktracing using SFrame for kernel exception entries through el1*_64_*() paths and irq entries through call_on_irq_stack() Signed-off-by: Weinan Liu Signed-off-by: Dylan Hatch Suggested-by: Jens Remus Reviewed-by: Jens Remus --- arch/arm64/kernel/entry.S | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index f8018b5c1f9a..dc55b0b19cfa 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -30,6 +30,12 @@ #include #include =20 +/* + * Do not generate .eh_frame. Only generate .debug_frame and optionally + * .sframe (via assembler option --gsframe[-N]). + */ + .cfi_sections .debug_frame + .macro clear_gp_regs .irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,2= 5,26,27,28,29 mov x\n, xzr @@ -575,7 +581,16 @@ SYM_CODE_START_LOCAL(el\el\ht\()_\regsize\()_\label) .if \el =3D=3D 0 b ret_to_user .else + /* + * Minimal DWARF CFI for unwinding across the call above. + * Enable unwinding for el1*_64_*() path only. + */ + .cfi_startproc + .cfi_def_cfa_offset PT_REGS_SIZE + .cfi_offset 29, S_FP - PT_REGS_SIZE + .cfi_offset 30, S_LR - PT_REGS_SIZE b ret_to_kernel + .cfi_endproc .endif SYM_CODE_END(el\el\ht\()_\regsize\()_\label) .endm @@ -872,6 +887,7 @@ NOKPROBE(ret_from_fork) * Calls func(regs) using this CPU's irq stack and shadow irq stack. */ SYM_FUNC_START(call_on_irq_stack) + .cfi_startproc save_and_disable_daif x9 #ifdef CONFIG_SHADOW_CALL_STACK get_current_task x16 @@ -882,6 +898,9 @@ SYM_FUNC_START(call_on_irq_stack) /* Create a frame record to save our LR and SP (implicit in FP) */ stp x29, x30, [sp, #-16]! mov x29, sp + .cfi_def_cfa 29, 16 + .cfi_offset 29, -16 + .cfi_offset 30, -8 =20 ldr_this_cpu x16, irq_stack_ptr, x17 =20 @@ -897,9 +916,13 @@ SYM_FUNC_START(call_on_irq_stack) */ mov sp, x29 ldp x29, x30, [sp], #16 + .cfi_restore 29 + .cfi_restore 30 + .cfi_def_cfa 31, 0 scs_load_current restore_irq x9 ret + .cfi_endproc SYM_FUNC_END(call_on_irq_stack) NOKPROBE(call_on_irq_stack) =20 --=20 2.54.0.rc1.555.g9c883467ad-goog From nobody Wed Jun 17 02:50:06 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4DFE3A6F07 for ; Tue, 21 Apr 2026 22:52:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811942; cv=none; b=LjiwfzRAlVI908VFToKsowH2Nzr7Oa4RLpE3MmdOVnlEjEO44j6cuolpUWohzHQ8yyDpexAcxbHainnMdFIyjv5HGRl1rpCNWgLT6Zx4lXWs7BYqKQCkspq894bExBPCswyCy9cyFRDWKuKDDIf/Pw7nVX6rj54oKqozmd84R1Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811942; c=relaxed/simple; bh=HVaWq1dxFNQfepawJxR+vwfJ462W5or0AaOM9S8qC4g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tj8vuQNWyf18NwPTILrwQsHOa7rySr25u3dbJKJrh5GBKIOva/bbLu1wFd3k8p6/b7yw97tdG96jQJ4bizeS91wiHBEaIuSxekAHJ+xXnA6GVutwUh/N4U5m43QjakCXwQqB1d7XVTzAUQjFJA5cWptSkf8IAvK8auVw8nKA+JM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=c36Lf7T5; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="c36Lf7T5" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82f74f0e3c6so3097593b3a.0 for ; Tue, 21 Apr 2026 15:52:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776811940; x=1777416740; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=r/yLmRqvzN2aFF0ctBbudoPYYemVc0jOSsthBuuHiX4=; b=c36Lf7T5nK9liCQZTVHULZ54xQwFTg/QcKCha/Nqhjq6XcOWCzX8ASuLjTDNcaXVz8 FCaBtWxae+hNgm68T/sgqVSiuiz0b//T8BXkR1Lk4RuZlo4eJUNDRtr3wtMHR032XDL9 2/+lVX6hswUNgltJmQlnooR1EiiglgU8cesIasV+9QWweSljkInfk/BURnBlRpOLhH7t I/SikEXpQSjaUniI4NgobxxM1PGFlJkQF8SmO6lehv2aAzi2R2yJI5LQHvpu1rh5srKu C9H/HK4kFr2NqTL2PKP3YDogs8WG0PAM+hyflZ2A3VaTZST6KYHBd2Ir/9uBWEuZUi22 HBUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776811940; x=1777416740; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=r/yLmRqvzN2aFF0ctBbudoPYYemVc0jOSsthBuuHiX4=; b=i2OMz8Jj90gdUEDz0U9tPt/w6TmrqeY6zAIpgVs5mg1v00c19b0limNEjzNaG83Xzu as6f8WLmAsVWktAELadIbjZMje6eSIoUqNpx7VfBpyg7jdYFgJ9vrEJAEpekEo121FR6 ARq6Ya28i9sDJR3pa8cb4bPzx/MwAEWvDPpVnKJ29fh+uE4qKetaY/hQnItv20POyP33 lpN+lpbPiodQ5yUSmp4HbIrRxKn55XKEvzie0isCKU6Z7cHiRJWimBoo0xrfGtapB+Ww G73BP2FnVAn3b4OFYIKnHT16N192zLv5Y3ia8Hfuz1ABb7KxBws6CyvuvvHDpKeapmW9 adDQ== X-Forwarded-Encrypted: i=1; AFNElJ+GbtHLiASyc3R66aORfN9dt6P3FMvCKSd8znzxh8cFWLLtrW+hiSgGgCM6j+M/5jGsbI15Aamnqphp9mg=@vger.kernel.org X-Gm-Message-State: AOJu0Yz1yLBbSEdly6wy+XQsHTny8a+zuIfu5nfUAEku9HNC5dmnsOo+ EG+haPIcqTsVxv/ok79CRG6lRcre/xI6VdhVHp0blUqdnyRrumyrkq+RuZs6Pkx2nsjUiwJYe/0 kwOso/huqtF0q0n1yxipsiakndw== X-Received: from pfbde11.prod.google.com ([2002:a05:6a00:468b:b0:82f:3ec7:cdd]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:ae10:b0:82f:9300:cc44 with SMTP id d2e1a72fcca58-82f9300d8d3mr17127125b3a.8.1776811939880; Tue, 21 Apr 2026 15:52:19 -0700 (PDT) Date: Tue, 21 Apr 2026 22:51:56 +0000 In-Reply-To: <20260421225200.1198447-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260421225200.1198447-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.rc1.555.g9c883467ad-goog Message-ID: <20260421225200.1198447-5-dylanbhatch@google.com> Subject: [PATCH v4 4/8] sframe: Provide PC lookup for vmlinux .sframe section From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With SFRAME_UNWINDER, read in the .sframe section at boot. This provides unwind data as an alternative/supplement to frame pointer-based unwinding. Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- arch/arm64/kernel/setup.c | 2 ++ include/linux/sframe.h | 14 ++++++++++++++ kernel/unwind/sframe.c | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 52 insertions(+) diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index 23c05dc7a8f2..4a633bc7aefb 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -32,6 +32,7 @@ #include #include #include +#include =20 #include #include @@ -375,6 +376,7 @@ void __init __no_sanitize_address setup_arch(char **cmd= line_p) "This indicates a broken bootloader or old kernel\n", boot_args[1], boot_args[2], boot_args[3]); } + init_sframe_table(); } =20 static inline bool cpu_can_disable(unsigned int cpu) diff --git a/include/linux/sframe.h b/include/linux/sframe.h index 0cb2924367bc..5b7341b61a7c 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -69,4 +69,18 @@ static inline int sframe_find_user(unsigned long ip, str= uct unwind_frame *frame) =20 #endif /* CONFIG_HAVE_UNWIND_USER_SFRAME */ =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +void __init init_sframe_table(void); +void sframe_module_init(struct module *mod, void *sframe, size_t sframe_si= ze, + void *text, size_t text_size); + +extern int sframe_find_kernel(unsigned long ip, struct unwind_frame *frame= ); + +#else + +static inline void __init init_sframe_table(void) {} + +#endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + #endif /* _LINUX_SFRAME_H */ diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index ae15f5c06939..fb3b6b2d8677 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -14,10 +14,20 @@ #include #include #include +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME +#include +#endif =20 #include "sframe.h" #include "sframe_debug.h" =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +static bool sframe_init __ro_after_init; +static struct sframe_section kernel_sfsec __ro_after_init; + +#endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + struct sframe_fde_internal { unsigned long func_addr; u32 func_size; @@ -930,3 +940,29 @@ void sframe_free_mm(struct mm_struct *mm) } =20 #endif /* CONFIG_HAVE_UNWIND_USER_SFRAME */ + +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +int sframe_find_kernel(unsigned long ip, struct unwind_frame *frame) +{ + if (!frame || !sframe_init) + return -EINVAL; + + return __sframe_find(&kernel_sfsec, ip, frame); +} + +void __init init_sframe_table(void) +{ + kernel_sfsec.sec_type =3D SFRAME_KERNEL; + kernel_sfsec.sframe_start =3D (unsigned long)__start_sframe; + kernel_sfsec.sframe_end =3D (unsigned long)__end_sframe; + kernel_sfsec.text_start =3D (unsigned long)_stext; + kernel_sfsec.text_end =3D (unsigned long)_etext; + + if (WARN_ON(sframe_read_header(&kernel_sfsec))) + return; + + sframe_init =3D true; +} + +#endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ --=20 2.54.0.rc1.555.g9c883467ad-goog From nobody Wed Jun 17 02:50:06 2026 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A7DC3A6EE6 for ; Tue, 21 Apr 2026 22:52:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811948; cv=none; b=bEBNXg+DvSHt9OXjc7x6bzenFqKCd3sgmBNwMIj5lRK62HnaXp6XfYgC53T5+5RBGGHlaqKO6GYCJRCMnnjO+eFMOWhjblK/vhiPBytWUcJIOZwPMaocfOH7E8r4xKMmEBKfUSsg8bR/OQf828NERiK2nBqAkHfZv+QhdAXgBZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811948; c=relaxed/simple; bh=HcHbWf4But37GRlHHhJj2shggv7jUZ569KAsIFqpDcI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=K7bLOFCXCS1bvBJRLXMBNPM5Nf5PPShZz6b4ePAm2u9iy5g+Ralp77DRRM5QsLcnxsn80oAmZy4ns7DNNWiAkaZ+fpqbq3Z5I2+nk9KftoTkQ7hUqJ8oVLInlTY4k9kU7pZznF3owdOXdwQxKR0exg3QrjHsBVSFiC78hJ+ULxY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=s7XcmSRB; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="s7XcmSRB" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-8230d6d54a5so4092095b3a.1 for ; Tue, 21 Apr 2026 15:52:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776811943; x=1777416743; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=P5zH7mctwcDOEQHijpP6Dn53P2ryMqn8C5QEWnKsM/M=; b=s7XcmSRBQfgDCQGXlsAVvq9QJV4rTbdp4qr4VO+UpoXUt7JKU95py7OuChdzxjOYKK hld/CsPt3Bdv2mWP3JN5ermXrOV+Ec8q2hlPXRYRPFUT863vrEuMDfqsS/MMneR5ZKku MjLEBSA6WAF2kHG2LMgWyQ2RNzgQ4vVvlYj3dsnAXBznAzIsqzTLpPWuJvPORrcycCyO QNvZKbsO7+LaGcue30s1W1lIlTQw6NZEfeSli3GlW7SJ/WlNWhjIQadgXVuEWOQlcp65 JAPKEiPKs3Rso/CgXB/gvCMKVna9zEKOcioTzLGX0QFN/slvKrYkdCF9Yw+kgFTn0zDY 149g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776811943; x=1777416743; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=P5zH7mctwcDOEQHijpP6Dn53P2ryMqn8C5QEWnKsM/M=; b=sc/Wsc6SJfuEBqBRItvsI5VnqYpP+LZ/ESMoMnK0wD8uM6UJQ1D4V7J07JrEOYlx/6 fDrHLs5Tm1UrUa+TvaYTHkWNfJj3/9m5+249ewPMfEMt4GcYQ1RiOaFvI7fW6skIpYLE b+Ry4NEGJ807CzuJdKjTk3uTNJtbtFiThLermmnt9AbslAYIcFLJ+q2K+pXr1YbB0FcD Ahsx5sjjebNPNZSSajnFfVQx8qjS+0cafJwgus6aQaNQZ7dd7QSkfLCnsAHnnjiNHe6n TDa4OfkNrliOIRbld36H2ppfhtD8COmWEJzZ6qekC7XoVJNYgwJ6skhmaV5xi21Mjj3w Aw/Q== X-Forwarded-Encrypted: i=1; AFNElJ/XTQuHJe1FDNT/YFPv8TjaCUBmlNWM2m+4LqMUrgXgzTA2P7uRgziA17I/iAgs1AtdzdmE8HeploWHedE=@vger.kernel.org X-Gm-Message-State: AOJu0YwpsJw8YSDOl5UVIEMGMFJl1NDTttQgCmzBynVtqhDPKvGo/LEK 25E8j2AOA5zRISnLaZ2HK7iA9jPNlgmudDmL8Wz+H7hsydytPE44bbSAoX3w6OZ7XBlFQp1IDXB cA5FCjka1ekRTBKX+v4/Fg6Zedg== X-Received: from pfbjw25.prod.google.com ([2002:a05:6a00:9299:b0:82f:b709:3747]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:a247:b0:82c:1cd0:2f7e with SMTP id d2e1a72fcca58-82f8b553974mr15760811b3a.20.1776811942482; Tue, 21 Apr 2026 15:52:22 -0700 (PDT) Date: Tue, 21 Apr 2026 22:51:57 +0000 In-Reply-To: <20260421225200.1198447-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260421225200.1198447-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.rc1.555.g9c883467ad-goog Message-ID: <20260421225200.1198447-6-dylanbhatch@google.com> Subject: [PATCH v4 5/8] sframe: Allow unsorted FDEs From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The .sframe in kernel modules is built without SFRAME_F_FDE_SORTED set. In order to allow sframe PC lookup in modules, add a code path to handle unsorted FDE tables by doing a simple linear search. Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- include/linux/sframe.h | 1 + kernel/unwind/sframe.c | 45 +++++++++++++++++++++++++++++++++++++----- 2 files changed, 41 insertions(+), 5 deletions(-) diff --git a/include/linux/sframe.h b/include/linux/sframe.h index 5b7341b61a7c..8ae31ed36226 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -28,6 +28,7 @@ struct sframe_section { unsigned long fres_start; unsigned long fres_end; unsigned int num_fdes; + bool fdes_sorted; =20 signed char ra_off; signed char fp_off; diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index fb3b6b2d8677..243027244854 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -176,9 +176,35 @@ static __always_inline int __read_fde(struct sframe_se= ction *sec, return -EFAULT; } =20 -static __always_inline int __find_fde(struct sframe_section *sec, - unsigned long ip, - struct sframe_fde_internal *fde) +static __always_inline int __find_fde_unsorted(struct sframe_section *sec, + unsigned long ip, + struct sframe_fde_internal *fde) +{ + struct sframe_fde_v3 *cur, *start, *end; + + start =3D (struct sframe_fde_v3 *)sec->fdes_start; + end =3D start + sec->num_fdes; + + for (cur =3D start; cur < end; cur++) { + s64 func_off; + u32 func_size; + unsigned long func_addr; + + DATA_GET(sec, func_off, &cur->func_start_off, s64, Efault); + DATA_GET(sec, func_size, &cur->func_size, u32, Efault); + func_addr =3D (unsigned long)cur + func_off; + + if (ip >=3D func_addr && ip < func_addr + func_size) + return __read_fde(sec, cur - start, fde); + } + return -EINVAL; +Efault: + return -EFAULT; +} + +static __always_inline int __find_fde_sorted(struct sframe_section *sec, + unsigned long ip, + struct sframe_fde_internal *fde) { unsigned long func_addr_low =3D 0, func_addr_high =3D ULONG_MAX; struct sframe_fde_v3 *first, *low, *high, *found =3D NULL; @@ -233,6 +259,15 @@ static __always_inline int __find_fde(struct sframe_se= ction *sec, return -EFAULT; } =20 +static __always_inline int __find_fde(struct sframe_section *sec, + unsigned long ip, + struct sframe_fde_internal *fde) +{ + if (sec->fdes_sorted) + return __find_fde_sorted(sec, ip, fde); + return __find_fde_unsorted(sec, ip, fde); +} + #define ____GET_INC(sec, to, from, type, label) \ ({ \ type __to; \ @@ -657,7 +692,7 @@ static int sframe_validate_section(struct sframe_sectio= n *sec) return ret; =20 ip =3D fde.func_addr; - if (ip <=3D prev_ip) { + if (sec->fdes_sorted && ip <=3D prev_ip) { dbg_sec("fde %u not sorted\n", i); return -EFAULT; } @@ -736,7 +771,6 @@ static int sframe_read_header(struct sframe_section *se= c) =20 if (shdr.preamble.magic !=3D SFRAME_MAGIC || shdr.preamble.version !=3D SFRAME_VERSION_3 || - !(shdr.preamble.flags & SFRAME_F_FDE_SORTED) || !(shdr.preamble.flags & SFRAME_F_FDE_FUNC_START_PCREL) || shdr.auxhdr_len) { dbg_sec("bad/unsupported sframe header\n"); @@ -766,6 +800,7 @@ static int sframe_read_header(struct sframe_section *se= c) return -EINVAL; } =20 + sec->fdes_sorted =3D shdr.preamble.flags & SFRAME_F_FDE_SORTED; sec->num_fdes =3D num_fdes; sec->fdes_start =3D fdes_start; sec->fres_start =3D fres_start; --=20 2.54.0.rc1.555.g9c883467ad-goog From nobody Wed Jun 17 02:50:06 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E696C3A6F17 for ; Tue, 21 Apr 2026 22:52:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811948; cv=none; b=RzuTe+LXUE4OtSs2aoqUvYiNFgs0QBNKSM/yl6swFgviaAyprZHErLkD4y6GIA8IVKuYdYCwMabWHFm4beYT0S4wo7RpepcaKET1AdBaCSo9NNdv37n/wKWR2UkGYy0H0Z84sFp8d3pi5zS6KstPzsh0zqpz1Dn0roQa04b9iAM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811948; c=relaxed/simple; bh=U/oW1pM1+B2l4PEI4R6STohHvDoOjFzqqD4QLLeJcnE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=eChX+h5+tUKk65d6nuQiGYGAazdpxobAS+fCxlVwD7RZdupQ1RkIUeLwoLn90u321JY3ECk9Jh71CzpPEkmue+3eGTcIbpUQsMDJVayWKJP+FZhQAwsHQEPsZ44bqPYcTfsH/4werjFL1ZB952NXqb+TZhdkD3y1hGpKyfHxrxE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nKOpW3be; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nKOpW3be" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82f74f0e3c6so3097612b3a.0 for ; Tue, 21 Apr 2026 15:52:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776811945; x=1777416745; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=EFJJ7xjnagN752ApaKO1ffV+VO/ox5jo3HxnwGPNp0Q=; b=nKOpW3be3BClMnuJv43QqUsSkF5RWu0qTv0njVxcgQxDAXqe00mwZpRu0hIPvOZqu4 l7HeWoiEGX5+enRulpUIMjXHsNfNdIjk5eh0+t3jCaN1CMGfkUpXoX4/V9c3Sl4fXDPB YC78Sb6Q8ggD4fq9deYxwZsxo4pqjqGEip3rfOXVINHG3uAgUA8ILlpEgAKjSi4kkl27 RZ+vAHGFqPRwB/OTL9/h6xtj0lOVZG4B3mWzx8SiPTdvWM5xVTcuVRL2KqIxdgjq3i3G meTVgeITjL6V8GIdP+RDkIaYAsrEvgys4rB9jBhSyYNR/Y1ilxQ0gRFssjY2W2qgQPFe knOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776811945; x=1777416745; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=EFJJ7xjnagN752ApaKO1ffV+VO/ox5jo3HxnwGPNp0Q=; b=jfbSV2AqWU0fRNfyCOjRcaDJifoZytNsbKxlrtkJSFXK3+h0KyyfitM7oMGVKvBxww g2VI0oTiQiB0MGBCNhUmy/58UUPAUk0tCNRCj8M9KDYmkosjK3DQRhj7+cj7Nho86LLC FSYDVNe6jIcR69gCgC6QTxQEvdgmUTFSr3k3gKn5JoDe05zVdJ7DzjrVFL1iUj9xeoIi el5Rce6PfjRa/HpFSSXZdS+ZT9+taAsh60sIvEa9UW/LP62d8AWxuqZPTgLhracieFVj V98shu++wGt5cOYEUaQXn4chAkG85KoVHDa3/rif8/VLLW2WxoJLHU2mRyAbcrzsBvJZ uA4Q== X-Forwarded-Encrypted: i=1; AFNElJ9xJOKaNaBTmeYSeERBPiw3uB+YoV49wbmJYWgEpVdMqzzfRoLsSwlgh2qHFjPNXz4rajtk/Km890zOnaU=@vger.kernel.org X-Gm-Message-State: AOJu0YxVIhpN/LQO0jCDkwSfjAVyruix5tZG2mCxhHcxXXcFt/5FdM+e rnIgI6X/yaYXPtKC5tsJdrr5C/NY4qZ9z5/390fZ9UgH+5IJ/EaEKfB9MtfhV6o7j9M1d+M+7nC 4HS1Td6BJRl/aXVFivjZs80wT8A== X-Received: from pfbci14.prod.google.com ([2002:a05:6a00:28ce:b0:82f:7242:4cb9]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2d1f:b0:82c:e19d:cabd with SMTP id d2e1a72fcca58-82f8c7ee387mr20113311b3a.10.1776811945127; Tue, 21 Apr 2026 15:52:25 -0700 (PDT) Date: Tue, 21 Apr 2026 22:51:58 +0000 In-Reply-To: <20260421225200.1198447-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260421225200.1198447-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.rc1.555.g9c883467ad-goog Message-ID: <20260421225200.1198447-7-dylanbhatch@google.com> Subject: [PATCH v4 6/8] arm64/module, sframe: Add sframe support for modules From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add sframe table to mod_arch_specific and support sframe PC lookups when an .sframe section can be found on incoming modules. Signed-off-by: Weinan Liu Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- arch/arm64/include/asm/module.h | 6 +++++ arch/arm64/kernel/module.c | 8 +++++++ include/linux/sframe.h | 2 ++ kernel/unwind/sframe.c | 40 +++++++++++++++++++++++++++++++-- 4 files changed, 54 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/modul= e.h index fb9b88eebeb1..07f309c51eee 100644 --- a/arch/arm64/include/asm/module.h +++ b/arch/arm64/include/asm/module.h @@ -6,6 +6,7 @@ #define __ASM_MODULE_H =20 #include +#include =20 struct mod_plt_sec { int plt_shndx; @@ -17,6 +18,11 @@ struct mod_arch_specific { struct mod_plt_sec core; struct mod_plt_sec init; =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + struct sframe_section sframe_sec; + bool sframe_init; +#endif + /* for CONFIG_DYNAMIC_FTRACE */ struct plt_entry *ftrace_trampolines; struct plt_entry *init_ftrace_trampolines; diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 24adb581af0e..427f187e9531 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -18,6 +18,7 @@ #include #include #include +#include =20 #include #include @@ -515,5 +516,12 @@ int module_finalize(const Elf_Ehdr *hdr, } } =20 + s =3D find_section(hdr, sechdrs, ".sframe"); + if (s) { + struct module_memory *t =3D &me->mem[MOD_TEXT]; + + sframe_module_init(me, (void *)s->sh_addr, s->sh_size, + t->base, t->size); + } return module_init_ftrace_plt(hdr, sechdrs, me); } diff --git a/include/linux/sframe.h b/include/linux/sframe.h index 8ae31ed36226..27f5a66190af 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -81,6 +81,8 @@ extern int sframe_find_kernel(unsigned long ip, struct un= wind_frame *frame); #else =20 static inline void __init init_sframe_table(void) {} +static inline void sframe_module_init(struct module *mod, void *sframe, si= ze_t sframe_size, + void *text, size_t text_size) {} =20 #endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ =20 diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index 243027244854..20178e02f428 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -980,10 +980,27 @@ void sframe_free_mm(struct mm_struct *mm) =20 int sframe_find_kernel(unsigned long ip, struct unwind_frame *frame) { - if (!frame || !sframe_init) + struct sframe_section *sec; + + if (!frame) return -EINVAL; =20 - return __sframe_find(&kernel_sfsec, ip, frame); + if (is_ksym_addr(ip)) { + if (!sframe_init) + return -EINVAL; + + sec =3D &kernel_sfsec; + } else { + struct module *mod; + + mod =3D __module_address(ip); + if (!mod || !mod->arch.sframe_init) + return -EINVAL; + + sec =3D &mod->arch.sframe_sec; + } + + return __sframe_find(sec, ip, frame); } =20 void __init init_sframe_table(void) @@ -1000,4 +1017,23 @@ void __init init_sframe_table(void) sframe_init =3D true; } =20 +void sframe_module_init(struct module *mod, void *sframe, size_t sframe_si= ze, + void *text, size_t text_size) +{ + struct sframe_section sec; + + memset(&sec, 0, sizeof(sec)); + sec.sec_type =3D SFRAME_KERNEL; + sec.sframe_start =3D (unsigned long)sframe; + sec.sframe_end =3D (unsigned long)sframe + sframe_size; + sec.text_start =3D (unsigned long)text; + sec.text_end =3D (unsigned long)text + text_size; + + if (WARN_ON(sframe_read_header(&sec))) + return; + + mod->arch.sframe_sec =3D sec; + mod->arch.sframe_init =3D true; +} + #endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ --=20 2.54.0.rc1.555.g9c883467ad-goog From nobody Wed Jun 17 02:50:06 2026 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAED13A75B2 for ; Tue, 21 Apr 2026 22:52:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811951; cv=none; b=Oo5rdgU42NvHj4AQBdhUHNl/967aulVm62HL6ACcijJKnS/VLawSW7K9hKzo5wKn37pzOlLZUYVCgPexstkws1S57Rrq6EXIP3u2WLT6RWywewC4ZBLrpAmwbUutXAq4XtkppNAne4tK93YNKpqtI26mXNE8ccRDzPh/8GSb1dg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811951; c=relaxed/simple; bh=EJAc+0FQbTFEsl6Wy8lawnseH2UBpttzQ7ZGI4ry5HY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XdtqkaL+5LTOfKZgZpNebTNfF2ftC3kBhr6YkjfE4amqROfPn68ISFvUmYUd6s7QIKEDNzi/1pS6F/ffwxbW/Gckk6mjLQ0fkPWWaTuLqlCpxKkI+/ZxPSLugzmDeqmQj8KqVm3hbt6+BJS7bQ9WGIOO4T3/YGHZQlh9t8pvsio= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZPM7ywUu; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZPM7ywUu" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-82fbceb0181so1875099b3a.3 for ; Tue, 21 Apr 2026 15:52:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776811948; x=1777416748; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=nEd1rBfN1UN8u0n4o9rP4+aQumOHvzk1E9l2M+hyRdQ=; b=ZPM7ywUuHLJNnZ0bGID8a7NzM6a0LBS/M+Mhv0AbYI9M5D3vMIf+OkhiOZHCd9ufsi TLqEvP22nHUarBgF6oyYraz9T/JOlkP9JWBAJFAEIL5kDfMGMzcU/hxG64JDcfYG5kSg KDWq4AmQYP9FktptHEueNc8EOmR4ZmEfMwrJ7Q6yOd1znPcmiDXMtxypfUKl0atJCg99 31CWD1uFG7k5zOWmrzCl1/jvQTrPYzppdYyIGk+QX648ggCP5smz5G1P7nHfennwtGwN 6yU6+pONqiOUgkP2CWuR4VSA+1lmeySxZPdJVJ6kkDfJp0XKJNPi++rDSU2bpzNxHFgL rHfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776811948; x=1777416748; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nEd1rBfN1UN8u0n4o9rP4+aQumOHvzk1E9l2M+hyRdQ=; b=ZZMc+3TEl2x/dfolwHB0Mlus6dnUPKdtUaj8DJqQjKUXzvzpXDWznaBM0H37ZG6n6U LvgeH9oGQDl9XKi9uij9tLKSXbxNgiP3leaa2SRuKv7Sl8HqKVe/HnexZMPg/1U2Rv/O dKExxZZd2Yhf15vN64fR5djCdds2Q83/bGiv3lCfMvmN87HWhOanTH0zA9FfTDQEP//w G5UvtG6Q+fbOVeKHPbKVM8N5Xl3tr2rS1hs7UzUU333I/fQwzg86Lo+CAGr1/eY1Rnzb /L1R1rlTZo2l1v5dmmKDQGuRmYGia+vgHg9qvsjrYZXXLJotfAxPf/5JNgSnTjn2p94J NicQ== X-Forwarded-Encrypted: i=1; AFNElJ+RsIPzz2Zt9ajSXtt8b5c5+kTtrcWyoud3836MdzXJQpEeLNp50Nmz1yyGG1DZAoWLy1KuEKNPmBJBCL8=@vger.kernel.org X-Gm-Message-State: AOJu0Yz2IF0T0xFta362cPINWHcu1vrH33o8kOkeTV5CtBBuxO3SuLVO 2C2+meX+m6bjR3RUFCG3wNLDJnLdKDN5LDiWIp6zdWe4LZMGYHw1/my3tfxUA/e6SxOSE3RRso7 omdqeGUXHfEMyDrTCGZu7rmv7uA== X-Received: from pfoo23.prod.google.com ([2002:a05:6a00:1a17:b0:82f:aad4:3985]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:a90:b0:827:433c:ba7e with SMTP id d2e1a72fcca58-82f8c970703mr20257960b3a.41.1776811948135; Tue, 21 Apr 2026 15:52:28 -0700 (PDT) Date: Tue, 21 Apr 2026 22:51:59 +0000 In-Reply-To: <20260421225200.1198447-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260421225200.1198447-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.rc1.555.g9c883467ad-goog Message-ID: <20260421225200.1198447-8-dylanbhatch@google.com> Subject: [PATCH v4 7/8] sframe: Introduce in-kernel SFRAME_VALIDATION From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Generalize the __safe* helpers to support a non-user-access code path. This requires arch-specific function address validation. This is because arm64 vmlinux has an .rodata.text section which lies outside the bounds of the normal .text. It contains code that is never executed by the kernel mapping, but for which the toolchain nonetheless generates sframe data, and needs to be considered valid for a PC lookup. This arch-specific address validation logic is only necessary to support SFRAME_VALIDATION for the vmlinux .sframe, since these .rodata.text functions would never be encountered during normal unwinding. Signed-off-by: Dylan Hatch Suggested-by: Jens Remus Reviewed-by: Jens Remus --- arch/Kconfig | 2 +- arch/arm64/include/asm/sections.h | 1 + arch/arm64/include/asm/unwind_sframe.h | 21 +++++++++++++++++++++ arch/arm64/kernel/vmlinux.lds.S | 2 ++ include/linux/sframe.h | 2 ++ kernel/unwind/sframe.c | 25 +++++++++++++++++++++++-- 6 files changed, 50 insertions(+), 3 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 8d27b3249e7a..cd4849bb675c 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -503,7 +503,7 @@ config HAVE_UNWIND_USER_SFRAME =20 config SFRAME_VALIDATION bool "Enable .sframe section debugging" - depends on HAVE_UNWIND_USER_SFRAME + depends on SFRAME_LOOKUP depends on DYNAMIC_DEBUG help When adding an .sframe section for a task, validate the entire diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sec= tions.h index 51b0d594239e..5edb4304f661 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -23,6 +23,7 @@ extern char __irqentry_text_start[], __irqentry_text_end[= ]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; +extern char _srodatatext[], _erodatatext[]; =20 static inline size_t entry_tramp_text_size(void) { diff --git a/arch/arm64/include/asm/unwind_sframe.h b/arch/arm64/include/as= m/unwind_sframe.h index 876412881196..1e9d7b74c0c8 100644 --- a/arch/arm64/include/asm/unwind_sframe.h +++ b/arch/arm64/include/asm/unwind_sframe.h @@ -2,7 +2,28 @@ #ifndef _ASM_ARM64_UNWIND_SFRAME_H #define _ASM_ARM64_UNWIND_SFRAME_H =20 +#include +#include + #define SFRAME_REG_SP 31 #define SFRAME_REG_FP 29 =20 +static inline bool sframe_func_start_addr_valid(struct sframe_section *sec, + unsigned long func_addr) +{ + /* + * The .rodata.text section is outside the normal kernel .text, but the + * toolchain still generates sframe data for it. Allow sframe lookups + * for these functions, even though they are never executed from the + * kernel mapping. + */ + if (sec->sec_type =3D=3D SFRAME_KERNEL && sec =3D=3D &kernel_sfsec && + func_addr >=3D (unsigned long)_srodatatext && + func_addr < (unsigned long)_erodatatext) + return true; + + return (sec->text_start <=3D func_addr && func_addr < sec->text_end); +} +#define sframe_func_start_addr_valid sframe_func_start_addr_valid + #endif /* _ASM_ARM64_UNWIND_SFRAME_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index 2964aad0362e..8c2dae6e7a86 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -213,12 +213,14 @@ SECTIONS =20 /* code sections that are never executed via the kernel mapping */ .rodata.text : { + _srodatatext =3D .; TRAMP_TEXT HIBERNATE_TEXT KEXEC_TEXT IDMAP_TEXT . =3D ALIGN(PAGE_SIZE); } + _erodatatext =3D .; =20 idmap_pg_dir =3D .; . +=3D PAGE_SIZE; diff --git a/include/linux/sframe.h b/include/linux/sframe.h index 27f5a66190af..ac3aa9db7d91 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -34,6 +34,8 @@ struct sframe_section { signed char fp_off; }; =20 +extern struct sframe_section kernel_sfsec __ro_after_init; + #endif /* CONFIG_UNWIND_SFRAME_LOOKUP */ =20 #ifdef CONFIG_HAVE_UNWIND_USER_SFRAME diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index 20178e02f428..d76968547bad 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -21,10 +21,18 @@ #include "sframe.h" #include "sframe_debug.h" =20 +#ifndef sframe_func_start_addr_valid +static inline bool sframe_func_start_addr_valid(struct sframe_section *sec, + unsigned long func_addr) +{ + return (sec->text_start <=3D func_addr && func_addr < sec->text_end); +} +#endif + #ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME =20 static bool sframe_init __ro_after_init; -static struct sframe_section kernel_sfsec __ro_after_init; +struct sframe_section kernel_sfsec __ro_after_init; =20 #endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ =20 @@ -152,7 +160,7 @@ static __always_inline int __read_fde(struct sframe_sec= tion *sec, sizeof(struct sframe_fde_v3), Efault); =20 func_addr =3D fde_addr + _fde.func_start_off; - if (func_addr < sec->text_start || func_addr > sec->text_end) + if (!sframe_func_start_addr_valid(sec, func_addr)) return -EINVAL; =20 fda_addr =3D sec->fres_start + _fde.fres_off; @@ -636,6 +644,9 @@ static int safe_read_fde(struct sframe_section *sec, { int ret; =20 + if (sec->sec_type =3D=3D SFRAME_KERNEL) + return __read_fde(sec, fde_num, fde); + if (!user_read_access_begin((void __user *)sec->sframe_start, sec->sframe_end - sec->sframe_start)) return -EFAULT; @@ -651,6 +662,9 @@ static int safe_read_fre(struct sframe_section *sec, { int ret; =20 + if (sec->sec_type =3D=3D SFRAME_KERNEL) + return __read_fre(sec, fde, fre_addr, fre); + if (!user_read_access_begin((void __user *)sec->sframe_start, sec->sframe_end - sec->sframe_start)) return -EFAULT; @@ -665,6 +679,9 @@ static int safe_read_fre_datawords(struct sframe_sectio= n *sec, { int ret; =20 + if (sec->sec_type =3D=3D SFRAME_KERNEL) + return __read_fre_datawords(sec, fde, fre); + if (!user_read_access_begin((void __user *)sec->sframe_start, sec->sframe_end - sec->sframe_start)) return -EFAULT; @@ -1013,6 +1030,8 @@ void __init init_sframe_table(void) =20 if (WARN_ON(sframe_read_header(&kernel_sfsec))) return; + if (WARN_ON(sframe_validate_section(&kernel_sfsec))) + return; =20 sframe_init =3D true; } @@ -1031,6 +1050,8 @@ void sframe_module_init(struct module *mod, void *sfr= ame, size_t sframe_size, =20 if (WARN_ON(sframe_read_header(&sec))) return; + if (WARN_ON(sframe_validate_section(&sec))) + return; =20 mod->arch.sframe_sec =3D sec; mod->arch.sframe_init =3D true; --=20 2.54.0.rc1.555.g9c883467ad-goog From nobody Wed Jun 17 02:50:06 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3FE23A6F07 for ; Tue, 21 Apr 2026 22:52:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811956; cv=none; b=suQrh3vahgTqj1sPP6LdfZ6zB6L739iZUM9hQhg+NTq7Fu413emArGa4bNFzBwSndjGmtmgutCxyTyq34TiXAhhSZSHOtxNhUOcfI9QdL7y3ZWpT2EaHqyrsUNQME4tMvyH/lsZqLcYxqE3wdKD3URVSMJM0mOByGOhR48buxdw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776811956; c=relaxed/simple; bh=/5EA8V0AFaV5ZupztrpogGDPxDJ2/LH3J17oidvBwwM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=A62iReLNNWYkdEr/0cJG/1NN1yxLmBLhoyQrGQDyfjjuSMlGffmdxA1eL9elvFoXJlPdmgq3AyCPhoV3EfPGszxUUWOB+GoXtSxbH+XTTZSqWBmtd/xamIrR3cx2jzwfUom10/rM4ummS9mcFQajWdC4GoSsZfidF43YMy5ouwY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nwEMPuiA; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nwEMPuiA" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82f0e12d375so2831230b3a.1 for ; Tue, 21 Apr 2026 15:52:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776811951; x=1777416751; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Dpl0sEyknPwZ/fuYDcKDow4XXGPOWGVa/unDN584Nv0=; b=nwEMPuiA9XQYhriY3XAxuad8qt5lETmTikTTaPcM7IhnLWZRQtsMAXbqA6lw6nq3LN uooflvATqQ85dBut5C3j3vbtRwLWzcpf8H46YrX3j5qvizAnHW2KuaCLic79WxFrHdrv aADL0RgJOQjLRgsfQKOTu884KA1g3YXzZYeLFMoArWZoVXwqb4ppfQ+c9XGP+mvJxX8I w7cgCi71Q4AJvMsajKXh3qYfThV+ynodGj6UKp6CpJz/mfALm8DlcTZ2sUDiUlH9GGz1 MFhuwI3hcGBhIGUmJummO/3i4r+U+jiCGpdTeUMzxRADJ2EbYmpauBG2ljwGxa6ZrPUQ 55nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776811951; x=1777416751; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Dpl0sEyknPwZ/fuYDcKDow4XXGPOWGVa/unDN584Nv0=; b=snMn12ZA1TGZ0dk+2Btdm74JVR/ByekjK2XvWk/habZO/fYp+wMA+CipEMLjQySsu5 B0DUJcsA4vGyfxIcTdpmDD1/HN3xuDePWk3vsk2vk2wMDRm7C1FJ5NZ/VxC0KvhwikYV 1a8iyd1qNkLWVIK9o6i63amuZbOObHVNk4PqUS4NSUCoILrkHr12XRqvZzbUNsCk/pKu wzo/CgwJX6fIZ7A3K5xMxxArugaJLXLHDxz62omzPd8zXUFpga+xXHSyuXG+Brfs4FF6 bJu8Q8mdFclkcIXhqeI51mZnG7bRlRHo04YiqYJ/jg3vee6rgmerGHsjov1Z3fuOMBSR UDIQ== X-Forwarded-Encrypted: i=1; AFNElJ9Zp+S6MLKxMsgGQsc6qAEti6eA/7LkbDqjM00lHjR1rVOdVefg2tcazsVMvDoB/naFaVyeUWgc1gndaHM=@vger.kernel.org X-Gm-Message-State: AOJu0YxPnVc6qt7W2o1gD4EGrSCHQMzBIspmSXV/X2y9KM8nuMiUlNHk D1ez3hIfuw5+xmY5WWofJftVbIWefZTXWfYGJFUnL3gl+6e/JJgAjFEBI4sDZWt+s4A9FaHsXaI cXhIoFDEAtneOVrMEu7NtFaB+Hw== X-Received: from pfbmu27.prod.google.com ([2002:a05:6a00:6e9b:b0:82f:a396:2232]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:6988:b0:82f:aae5:c7a9 with SMTP id d2e1a72fcca58-82faae5d438mr8836750b3a.27.1776811951103; Tue, 21 Apr 2026 15:52:31 -0700 (PDT) Date: Tue, 21 Apr 2026 22:52:00 +0000 In-Reply-To: <20260421225200.1198447-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260421225200.1198447-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.rc1.555.g9c883467ad-goog Message-ID: <20260421225200.1198447-9-dylanbhatch@google.com> Subject: [PATCH v4 8/8] unwind: arm64: Use sframe to unwind interrupt frames From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add unwind_next_frame_sframe() function to unwind by sframe info if present. Use this method at exception boundaries, falling back to frame-pointer unwind only on failure. In such failure cases, the stacktrace is considered unreliable. During normal unwind, prefer frame pointer unwind (for better performance) with sframe as a backup. This change restores the LR behavior originally introduced in commit c2c6b27b5aa14fa2 ("arm64: stacktrace: unwind exception boundaries"), But later removed in commit 32ed1205682e ("arm64: stacktrace: Skip reporting LR at exception boundaries") This can be done because the sframe data can be used to determine whether the LR is current for the PC value recovered from pt_regs at the exception boundary. Signed-off-by: Weinan Liu Reviewed-by: Prasanna Kumar T S M Signed-off-by: Dylan Hatch Reviewed-by: Jens Remus --- arch/arm64/include/asm/stacktrace/common.h | 6 + arch/arm64/kernel/stacktrace.c | 246 +++++++++++++++++++-- 2 files changed, 232 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace/common.h b/arch/arm64/includ= e/asm/stacktrace/common.h index 821a8fdd31af..4df68181e1b5 100644 --- a/arch/arm64/include/asm/stacktrace/common.h +++ b/arch/arm64/include/asm/stacktrace/common.h @@ -21,6 +21,8 @@ struct stack_info { * * @fp: The fp value in the frame record (or the real fp) * @pc: The lr value in the frame record (or the real lr) + * @sp: The sp value at the call site of the current function. + * @unreliable: Stacktrace is unreliable. * * @stack: The stack currently being unwound. * @stacks: An array of stacks which can be unwound. @@ -29,7 +31,11 @@ struct stack_info { struct unwind_state { unsigned long fp; unsigned long pc; +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + unsigned long sp; +#endif =20 + bool unreliable; struct stack_info stack; struct stack_info *stacks; int nr_stacks; diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 3ebcf8c53fb0..c935323f393b 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -14,6 +14,7 @@ #include #include #include +#include =20 #include #include @@ -26,6 +27,7 @@ enum kunwind_source { KUNWIND_SOURCE_CALLER, KUNWIND_SOURCE_TASK, KUNWIND_SOURCE_REGS_PC, + KUNWIND_SOURCE_REGS_LR, }; =20 union unwind_flags { @@ -85,6 +87,9 @@ kunwind_init_from_regs(struct kunwind_state *state, state->regs =3D regs; state->common.fp =3D regs->regs[29]; state->common.pc =3D regs->pc; +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + state->common.sp =3D regs->sp; +#endif state->source =3D KUNWIND_SOURCE_REGS_PC; } =20 @@ -103,6 +108,9 @@ kunwind_init_from_caller(struct kunwind_state *state) =20 state->common.fp =3D (unsigned long)__builtin_frame_address(1); state->common.pc =3D (unsigned long)__builtin_return_address(0); +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + state->common.sp =3D (unsigned long)__builtin_frame_address(0); +#endif state->source =3D KUNWIND_SOURCE_CALLER; } =20 @@ -124,6 +132,9 @@ kunwind_init_from_task(struct kunwind_state *state, =20 state->common.fp =3D thread_saved_fp(task); state->common.pc =3D thread_saved_pc(task); +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + state->common.sp =3D thread_saved_sp(task); +#endif state->source =3D KUNWIND_SOURCE_TASK; } =20 @@ -181,7 +192,6 @@ int kunwind_next_regs_pc(struct kunwind_state *state) state->regs =3D regs; state->common.pc =3D regs->pc; state->common.fp =3D regs->regs[29]; - state->regs =3D NULL; state->source =3D KUNWIND_SOURCE_REGS_PC; return 0; } @@ -237,6 +247,9 @@ kunwind_next_frame_record(struct kunwind_state *state) =20 unwind_consume_stack(&state->common, info, fp, sizeof(*record)); =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + state->common.sp =3D state->common.fp; +#endif state->common.fp =3D new_fp; state->common.pc =3D new_pc; state->source =3D KUNWIND_SOURCE_FRAME; @@ -244,6 +257,176 @@ kunwind_next_frame_record(struct kunwind_state *state) return 0; } =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +static __always_inline struct stack_info * +get_word(struct unwind_state *state, unsigned long *word) +{ + unsigned long addr =3D *word; + struct stack_info *info; + + info =3D unwind_find_stack(state, addr, sizeof(addr)); + if (!info) + return info; + + *word =3D READ_ONCE(*(unsigned long *)addr); + + return info; +} + +static __always_inline int +get_consume_word(struct unwind_state *state, unsigned long *word) +{ + struct stack_info *info; + unsigned long addr =3D *word; + + info =3D get_word(state, word); + if (!info) + return -EINVAL; + + unwind_consume_stack(state, info, addr, sizeof(addr)); + return 0; +} + +/* + * Unwind to the next frame according to sframe. + */ +static __always_inline int +unwind_next_frame_sframe(struct kunwind_state *state) +{ + struct unwind_frame frame; + unsigned long cfa, fp, ra; + enum kunwind_source source =3D KUNWIND_SOURCE_FRAME; + struct pt_regs *regs =3D state->regs; + + int err; + + /* FP/SP alignment 8 bytes */ + if (state->common.fp & 0x7 || state->common.sp & 0x7) + return -EINVAL; + + /* + * Most/all outermost functions are not visible to sframe. So, check for + * a meta frame record if the sframe lookup fails. + */ + err =3D sframe_find_kernel(state->common.pc, &frame); + if (err) + return kunwind_next_frame_record_meta(state); + + if (frame.outermost) + return -ENOENT; + + /* Get the Canonical Frame Address (CFA) */ + switch (frame.cfa.rule) { + case UNWIND_CFA_RULE_SP_OFFSET: + cfa =3D state->common.sp; + break; + case UNWIND_CFA_RULE_FP_OFFSET: + if (state->common.fp < state->common.sp) + return -EINVAL; + cfa =3D state->common.fp; + break; + case UNWIND_CFA_RULE_REG_OFFSET: + case UNWIND_CFA_RULE_REG_OFFSET_DEREF: + /* regs only available in topmost/interrupt frame */ + if (!regs || frame.cfa.regnum > 30) + return -EINVAL; + cfa =3D regs->regs[frame.cfa.regnum]; + break; + default: + WARN_ON_ONCE(1); + return -EINVAL; + } + cfa +=3D frame.cfa.offset; + + /* + * CFA typically points to a higher address than RA or FP, so don't + * consume from the stack when we read it. + */ + if (frame.cfa.rule & UNWIND_RULE_DEREF && + !get_word(&state->common, &cfa)) + return -EINVAL; + + /* CFA alignment 8 bytes */ + if (cfa & 0x7) + return -EINVAL; + + /* Get the Return Address (RA) */ + switch (frame.ra.rule) { + case UNWIND_RULE_RETAIN: + /* regs only available in topmost/interrupt frame */ + if (!regs) + return -EINVAL; + ra =3D regs->regs[30]; + source =3D KUNWIND_SOURCE_REGS_LR; + break; + /* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */ + case UNWIND_RULE_CFA_OFFSET_DEREF: + ra =3D cfa + frame.ra.offset; + break; + case UNWIND_RULE_REG_OFFSET: + case UNWIND_RULE_REG_OFFSET_DEREF: + /* regs only available in topmost/interrupt frame */ + if (!regs) + return -EINVAL; + ra =3D regs->regs[frame.cfa.regnum]; + ra +=3D frame.ra.offset; + break; + default: + WARN_ON_ONCE(1); + return -EINVAL; + } + + /* Get the Frame Pointer (FP) */ + switch (frame.fp.rule) { + case UNWIND_RULE_RETAIN: + fp =3D state->common.fp; + break; + /* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */ + case UNWIND_RULE_CFA_OFFSET_DEREF: + fp =3D cfa + frame.fp.offset; + break; + case UNWIND_RULE_REG_OFFSET: + case UNWIND_RULE_REG_OFFSET_DEREF: + /* regs only available in topmost/interrupt frame */ + if (!regs) + return -EINVAL; + fp =3D regs->regs[frame.fp.regnum]; + fp +=3D frame.fp.offset; + break; + default: + WARN_ON_ONCE(1); + return -EINVAL; + } + + /* + * Consume RA and FP from the stack. The frame record puts FP at a lower + * address than RA, so we always read FP first. + */ + if (frame.fp.rule & UNWIND_RULE_DEREF && + !get_word(&state->common, &fp)) + return -EINVAL; + + if (frame.ra.rule & UNWIND_RULE_DEREF && + get_consume_word(&state->common, &ra)) + return -EINVAL; + + state->common.pc =3D ra; + state->common.sp =3D cfa; + state->common.fp =3D fp; + + state->source =3D source; + + return 0; +} + +#else /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + +static __always_inline int +unwind_next_frame_sframe(struct kunwind_state *state) { return -EINVAL; } + +#endif /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME*/ + /* * Unwind from one frame record (A) to the next frame record (B). * @@ -259,12 +442,25 @@ kunwind_next(struct kunwind_state *state) state->flags.all =3D 0; =20 switch (state->source) { + case KUNWIND_SOURCE_REGS_PC: + err =3D unwind_next_frame_sframe(state); + + if (err && err !=3D -ENOENT) { + /* Fallback to FP based unwinder */ + err =3D kunwind_next_frame_record(state); + state->common.unreliable =3D true; + } + state->regs =3D NULL; + break; case KUNWIND_SOURCE_FRAME: case KUNWIND_SOURCE_CALLER: case KUNWIND_SOURCE_TASK: - case KUNWIND_SOURCE_REGS_PC: + case KUNWIND_SOURCE_REGS_LR: err =3D kunwind_next_frame_record(state); + if (err && err !=3D -ENOENT) + err =3D unwind_next_frame_sframe(state); break; + default: err =3D -EINVAL; } @@ -350,6 +546,9 @@ kunwind_stack_walk(kunwind_consume_fn consume_state, .common =3D { .stacks =3D stacks, .nr_stacks =3D ARRAY_SIZE(stacks), +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + .sp =3D 0, +#endif }, }; =20 @@ -390,34 +589,40 @@ noinline noinstr void arch_stack_walk(stack_trace_con= sume_fn consume_entry, kunwind_stack_walk(arch_kunwind_consume_entry, &data, task, regs); } =20 +struct kunwind_reliable_consume_entry_data { + stack_trace_consume_fn consume_entry; + void *cookie; + bool unreliable; +}; + static __always_inline bool -arch_reliable_kunwind_consume_entry(const struct kunwind_state *state, voi= d *cookie) +arch_kunwind_reliable_consume_entry(const struct kunwind_state *state, voi= d *cookie) { - /* - * At an exception boundary we can reliably consume the saved PC. We do - * not know whether the LR was live when the exception was taken, and - * so we cannot perform the next unwind step reliably. - * - * All that matters is whether the *entire* unwind is reliable, so give - * up as soon as we hit an exception boundary. - */ - if (state->source =3D=3D KUNWIND_SOURCE_REGS_PC) - return false; + struct kunwind_reliable_consume_entry_data *data =3D cookie; =20 - return arch_kunwind_consume_entry(state, cookie); + if (state->common.unreliable) { + data->unreliable =3D true; + return false; + } + return data->consume_entry(data->cookie, state->common.pc); } =20 -noinline noinstr int arch_stack_walk_reliable(stack_trace_consume_fn consu= me_entry, - void *cookie, - struct task_struct *task) +noinline notrace int arch_stack_walk_reliable( + stack_trace_consume_fn consume_entry, + void *cookie, struct task_struct *task) { - struct kunwind_consume_entry_data data =3D { + struct kunwind_reliable_consume_entry_data data =3D { .consume_entry =3D consume_entry, .cookie =3D cookie, + .unreliable =3D false, }; =20 - return kunwind_stack_walk(arch_reliable_kunwind_consume_entry, &data, - task, NULL); + kunwind_stack_walk(arch_kunwind_reliable_consume_entry, &data, task, NULL= ); + + if (data.unreliable) + return -EINVAL; + + return 0; } =20 struct bpf_unwind_consume_entry_data { @@ -452,6 +657,7 @@ static const char *state_source_string(const struct kun= wind_state *state) case KUNWIND_SOURCE_CALLER: return "C"; case KUNWIND_SOURCE_TASK: return "T"; case KUNWIND_SOURCE_REGS_PC: return "P"; + case KUNWIND_SOURCE_REGS_LR: return "L"; default: return "U"; } } --=20 2.54.0.rc1.555.g9c883467ad-goog