From nobody Sat May 9 07:32:00 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8626A47AF69 for ; Tue, 28 Apr 2026 18:37:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401432; cv=none; b=k8ZXI/tNaRdHRErqhFKNTDAA+qiVyrLW5zmtHl700J7i0yj1jYRlB9Ccqw5mxGWzQLuDSg2h6tH8Etoi2eCjjoOVYxeX5I3axMvedIarXFcUM4qIbhZwKNruurQeRkcoLCuUU5ODgNKapFj4tsgk1UMR8BTmYsaM4dn4fKnIe/4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401432; c=relaxed/simple; bh=Lo85YR5j9U8ig7qvJ3SItDU29ffD3ql/EbDOKlz8kVY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mPYr9I7qZlhfVC6uXMaQ0+k3TGd7rKrOjlgqG2Zwcbh6ekMTDVBSGVOOlFASV/jOjBFTECXQW7mc03xrF+zyssRw19oaA2uWkOF5rS/o/Eh0diNil+a5/lGpFA/P4Zx0Cr6cDMJETCFwmk2LYa3B1qmmIdrSnv1KuHy79hrJCf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Cz3/+s3R; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Cz3/+s3R" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82fa2165c3eso7432545b3a.0 for ; Tue, 28 Apr 2026 11:37:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777401429; x=1778006229; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=cR5jdpb+bWj/e77nnnl7XPxPTyO+ySghN9I5TqEyziU=; b=Cz3/+s3RN4oQ1yCWcR9gso2lZ07Mg+LgYwayaSQYuCvnI1/ObvA+Y722dMVqlvM+XM +JaBzWvUSd2zTBg1fJ2ETBQlXx5arp6WKSN3gvy+1gDVMEJcAXEkrQ8hVmp7bmtw3U85 9Z7PPEpai4QSyujDLQXZDfBWRxcEh5cWLB25qdBwRst4gPrhyU67jXfblD5BoF/XGIeg OY9HIZFFYN8T/2n4K5sLhnaSNc2H88Z+rk9FqOkW0vMx/JoH36kBTrkHFqh+q4xbI9VD ocs6BFFOd70MfOnPsAKSobAv7CIrYUS6vJpZJyRaIm6hpO9oe0UpfECJ3k3NB6sp1HxU FJLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777401429; x=1778006229; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cR5jdpb+bWj/e77nnnl7XPxPTyO+ySghN9I5TqEyziU=; b=mOYOqDOQKv0/gNWapUfVsgxlDkziY95buyd3Cr7YOvgGJ0TIzLoX0wL6sMk6k+L1OT qOggZ4KtWT+c+4IWwQ/S91algoNeNAvEEqOlv+E763PJuLY8Jm8e/f7eu4vpVr2G3Jf2 548/gIj9R3K1xWE001LBIBtbs/uYp2kfA9Eb3/G9QqIwfNfYRjPlLA+tlLZaDt6XyODB vwnpgPJqdnaVhMebwOUCRqfjVFX+w3o8SuZLZnLpQtSeOLh1E0ZhNHX5MN2Y6IUkSThS GGHNNoSqCoAxBnB9KiSsaQAZdTFTLGM6ckrmwGyKL+EyLguLIXVgal84TofHSYPLI1d6 3IKg== X-Forwarded-Encrypted: i=1; AFNElJ9iE06sf/j3bIDvVp/zelCSFPLfprvSUlf2dZ9fXdjHihA6WorSJtgfyZS0WYTW+z3b6vYMOVbIqPueDio=@vger.kernel.org X-Gm-Message-State: AOJu0YyWrpSMiIhtbXl7Nv+H/XZKI6tL8dIHxkeFkSCr2vL5P7yVUviW mLRfYdRUEFOgxRTYN1cPScm/hzHDf3t6LrZCUhtWwcyix0cuxvPsm+EW7scLZtHWN3R7sK7+LH8 f24hTZ1cKuhpbq6RcmxgqsNkePw== X-Received: from pfnw7.prod.google.com ([2002:aa7:8587:0:b0:82f:5a4:a02c]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:4b0c:b0:82f:5726:be23 with SMTP id d2e1a72fcca58-834ddc94432mr4138635b3a.49.1777401428545; Tue, 28 Apr 2026 11:37:08 -0700 (PDT) Date: Tue, 28 Apr 2026 18:36:36 +0000 In-Reply-To: <20260428183643.3796063-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260428183643.3796063-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260428183643.3796063-2-dylanbhatch@google.com> Subject: [PATCH v5 1/8] sframe: Allow kernelspace sframe sections From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Generalize the sframe lookup code to support kernelspace sections. This is done by defining a SFRAME_LOOKUP option that can be activated separate from HAVE_UNWIND_USER_SFRAME, as there will be other client to this library than just userspace unwind. Sframe section location is now tracked in a separate sec_type field to determine whether user-access functions are necessary to read the sframe data. Relevant type delarations are moved and renamed to reflect the non-user sframe support. Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- MAINTAINERS | 2 +- arch/Kconfig | 4 + .../{unwind_user_sframe.h =3D> unwind_sframe.h} | 6 +- arch/x86/include/asm/unwind_user.h | 12 +- include/linux/sframe.h | 48 ++-- include/linux/unwind_types.h | 46 +++ include/linux/unwind_user_types.h | 41 --- kernel/unwind/Makefile | 2 +- kernel/unwind/sframe.c | 270 ++++++++++++------ kernel/unwind/user.c | 41 +-- 10 files changed, 293 insertions(+), 179 deletions(-) rename arch/x86/include/asm/{unwind_user_sframe.h =3D> unwind_sframe.h} (5= 0%) create mode 100644 include/linux/unwind_types.h diff --git a/MAINTAINERS b/MAINTAINERS index 8c46465ee7a9..cfc7dec88da4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -27557,7 +27557,7 @@ F: Documentation/driver-api/uio-howto.rst F: drivers/uio/ F: include/linux/uio_driver.h =20 -USERSPACE STACK UNWINDING +STACK UNWINDING M: Josh Poimboeuf M: Steven Rostedt S: Maintained diff --git a/arch/Kconfig b/arch/Kconfig index f1ed8bc0806d..d7caf2e245ce 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -486,6 +486,9 @@ config AS_SFRAME3 def_bool $(as-instr,.cfi_startproc\n.cfi_endproc,-Wa$(comma)--gsframe-3) select AS_SFRAME =20 +config UNWIND_SFRAME_LOOKUP + bool + config UNWIND_USER bool =20 @@ -496,6 +499,7 @@ config HAVE_UNWIND_USER_FP config HAVE_UNWIND_USER_SFRAME bool select UNWIND_USER + select UNWIND_SFRAME_LOOKUP =20 config SFRAME_VALIDATION bool "Enable .sframe section debugging" diff --git a/arch/x86/include/asm/unwind_user_sframe.h b/arch/x86/include/a= sm/unwind_sframe.h similarity index 50% rename from arch/x86/include/asm/unwind_user_sframe.h rename to arch/x86/include/asm/unwind_sframe.h index d828ae1a4aac..44d42e6ffde4 100644 --- a/arch/x86/include/asm/unwind_user_sframe.h +++ b/arch/x86/include/asm/unwind_sframe.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#ifndef _ASM_X86_UNWIND_USER_SFRAME_H -#define _ASM_X86_UNWIND_USER_SFRAME_H +#ifndef _ASM_X86_UNWIND_SFRAME_H +#define _ASM_X86_UNWIND_SFRAME_H =20 #ifdef CONFIG_X86_64 =20 @@ -9,4 +9,4 @@ =20 #endif =20 -#endif /* _ASM_X86_UNWIND_USER_SFRAME_H */ +#endif /* _ASM_X86_UNWIND_SFRAME_H */ diff --git a/arch/x86/include/asm/unwind_user.h b/arch/x86/include/asm/unwi= nd_user.h index ae46906c3b39..8fdab3581b86 100644 --- a/arch/x86/include/asm/unwind_user.h +++ b/arch/x86/include/asm/unwind_user.h @@ -55,30 +55,30 @@ static inline int unwind_user_get_reg(unsigned long *va= l, unsigned int regnum) =20 #define ARCH_INIT_USER_FP_FRAME(ws) \ .cfa =3D { \ - .rule =3D UNWIND_USER_CFA_RULE_FP_OFFSET,\ + .rule =3D UNWIND_CFA_RULE_FP_OFFSET,\ .offset =3D 2*(ws), \ }, \ .ra =3D { \ - .rule =3D UNWIND_USER_RULE_CFA_OFFSET_DEREF,\ + .rule =3D UNWIND_RULE_CFA_OFFSET_DEREF,\ .offset =3D -1*(ws), \ }, \ .fp =3D { \ - .rule =3D UNWIND_USER_RULE_CFA_OFFSET_DEREF,\ + .rule =3D UNWIND_RULE_CFA_OFFSET_DEREF,\ .offset =3D -2*(ws), \ }, \ .outermost =3D false, =20 #define ARCH_INIT_USER_FP_ENTRY_FRAME(ws) \ .cfa =3D { \ - .rule =3D UNWIND_USER_CFA_RULE_SP_OFFSET,\ + .rule =3D UNWIND_CFA_RULE_SP_OFFSET,\ .offset =3D 1*(ws), \ }, \ .ra =3D { \ - .rule =3D UNWIND_USER_RULE_CFA_OFFSET_DEREF,\ + .rule =3D UNWIND_RULE_CFA_OFFSET_DEREF,\ .offset =3D -1*(ws), \ }, \ .fp =3D { \ - .rule =3D UNWIND_USER_RULE_RETAIN,\ + .rule =3D UNWIND_RULE_RETAIN,\ }, \ .outermost =3D false, =20 diff --git a/include/linux/sframe.h b/include/linux/sframe.h index b79c5ec09229..0cb2924367bc 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -3,37 +3,46 @@ #define _LINUX_SFRAME_H =20 #include +#include #include -#include =20 -#ifdef CONFIG_HAVE_UNWIND_USER_SFRAME +#ifdef CONFIG_UNWIND_SFRAME_LOOKUP + +enum sframe_sec_type { + SFRAME_KERNEL, + SFRAME_USER, +}; =20 struct sframe_section { - struct rcu_head rcu; + struct rcu_head rcu; #ifdef CONFIG_DYNAMIC_DEBUG - const char *filename; + const char *filename; #endif - unsigned long sframe_start; - unsigned long sframe_end; - unsigned long text_start; - unsigned long text_end; - - unsigned long fdes_start; - unsigned long fres_start; - unsigned long fres_end; - unsigned int num_fdes; - - signed char ra_off; - signed char fp_off; + enum sframe_sec_type sec_type; + unsigned long sframe_start; + unsigned long sframe_end; + unsigned long text_start; + unsigned long text_end; + + unsigned long fdes_start; + unsigned long fres_start; + unsigned long fres_end; + unsigned int num_fdes; + + signed char ra_off; + signed char fp_off; }; =20 +#endif /* CONFIG_UNWIND_SFRAME_LOOKUP */ + +#ifdef CONFIG_HAVE_UNWIND_USER_SFRAME + #define INIT_MM_SFRAME .sframe_mt =3D MTREE_INIT(sframe_mt, 0), extern void sframe_free_mm(struct mm_struct *mm); =20 extern int sframe_add_section(unsigned long sframe_start, unsigned long sf= rame_end, unsigned long text_start, unsigned long text_end); extern int sframe_remove_section(unsigned long sframe_addr); -extern int sframe_find(unsigned long ip, struct unwind_user_frame *frame); =20 static inline bool current_has_sframe(void) { @@ -42,6 +51,8 @@ static inline bool current_has_sframe(void) return mm && !mtree_empty(&mm->sframe_mt); } =20 +extern int sframe_find_user(unsigned long ip, struct unwind_frame *frame); + #else /* !CONFIG_HAVE_UNWIND_USER_SFRAME */ =20 #define INIT_MM_SFRAME @@ -52,9 +63,10 @@ static inline int sframe_add_section(unsigned long sfram= e_start, unsigned long s return -ENOSYS; } static inline int sframe_remove_section(unsigned long sframe_addr) { retur= n -ENOSYS; } -static inline int sframe_find(unsigned long ip, struct unwind_user_frame *= frame) { return -ENOSYS; } static inline bool current_has_sframe(void) { return false; } =20 +static inline int sframe_find_user(unsigned long ip, struct unwind_frame *= frame) { return -ENOSYS; } + #endif /* CONFIG_HAVE_UNWIND_USER_SFRAME */ =20 #endif /* _LINUX_SFRAME_H */ diff --git a/include/linux/unwind_types.h b/include/linux/unwind_types.h new file mode 100644 index 000000000000..08bcb0aa04aa --- /dev/null +++ b/include/linux/unwind_types.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_UNWIND_TYPES_H +#define _LINUX_UNWIND_TYPES_H + +#define UNWIND_RULE_DEREF BIT(31) + +enum unwind_cfa_rule { + UNWIND_CFA_RULE_SP_OFFSET, /* CFA =3D SP + offset */ + UNWIND_CFA_RULE_FP_OFFSET, /* CFA =3D FP + offset */ + UNWIND_CFA_RULE_REG_OFFSET, /* CFA =3D reg + offset */ + /* DEREF variants */ + UNWIND_CFA_RULE_REG_OFFSET_DEREF =3D /* CFA =3D *(reg + offset) */ + UNWIND_CFA_RULE_REG_OFFSET | UNWIND_RULE_DEREF, +}; + +struct unwind_cfa_rule_data { + enum unwind_cfa_rule rule; + s32 offset; + unsigned int regnum; +}; + +enum unwind_rule { + UNWIND_RULE_RETAIN, /* entity =3D entity */ + UNWIND_RULE_CFA_OFFSET, /* entity =3D CFA + offset */ + UNWIND_RULE_REG_OFFSET, /* entity =3D register + offset */ + /* DEREF variants */ + UNWIND_RULE_CFA_OFFSET_DEREF =3D /* entity =3D *(CFA + offset) */ + UNWIND_RULE_CFA_OFFSET | UNWIND_RULE_DEREF, + UNWIND_RULE_REG_OFFSET_DEREF =3D /* entity =3D *(register + offset) */ + UNWIND_RULE_REG_OFFSET | UNWIND_RULE_DEREF, +}; + +struct unwind_rule_data { + enum unwind_rule rule; + s32 offset; + unsigned int regnum; +}; + +struct unwind_frame { + struct unwind_cfa_rule_data cfa; + struct unwind_rule_data ra; + struct unwind_rule_data fp; + bool outermost; +}; + +#endif /* _LINUX_UNWIND_TYPES_H */ diff --git a/include/linux/unwind_user_types.h b/include/linux/unwind_user_= types.h index 059e5c76f2f3..646e5fb774db 100644 --- a/include/linux/unwind_user_types.h +++ b/include/linux/unwind_user_types.h @@ -27,47 +27,6 @@ struct unwind_stacktrace { unsigned long *entries; }; =20 -#define UNWIND_USER_RULE_DEREF BIT(31) - -enum unwind_user_cfa_rule { - UNWIND_USER_CFA_RULE_SP_OFFSET, /* CFA =3D SP + offset */ - UNWIND_USER_CFA_RULE_FP_OFFSET, /* CFA =3D FP + offset */ - UNWIND_USER_CFA_RULE_REG_OFFSET, /* CFA =3D reg + offset */ - /* DEREF variants */ - UNWIND_USER_CFA_RULE_REG_OFFSET_DEREF =3D /* CFA =3D *(reg + offset) */ - UNWIND_USER_CFA_RULE_REG_OFFSET | UNWIND_USER_RULE_DEREF, -}; - -struct unwind_user_cfa_rule_data { - enum unwind_user_cfa_rule rule; - s32 offset; - unsigned int regnum; -}; - -enum unwind_user_rule { - UNWIND_USER_RULE_RETAIN, /* entity =3D entity */ - UNWIND_USER_RULE_CFA_OFFSET, /* entity =3D CFA + offset */ - UNWIND_USER_RULE_REG_OFFSET, /* entity =3D register + offset */ - /* DEREF variants */ - UNWIND_USER_RULE_CFA_OFFSET_DEREF =3D /* entity =3D *(CFA + offset) */ - UNWIND_USER_RULE_CFA_OFFSET | UNWIND_USER_RULE_DEREF, - UNWIND_USER_RULE_REG_OFFSET_DEREF =3D /* entity =3D *(register + offset) = */ - UNWIND_USER_RULE_REG_OFFSET | UNWIND_USER_RULE_DEREF, -}; - -struct unwind_user_rule_data { - enum unwind_user_rule rule; - s32 offset; - unsigned int regnum; -}; - -struct unwind_user_frame { - struct unwind_user_cfa_rule_data cfa; - struct unwind_user_rule_data ra; - struct unwind_user_rule_data fp; - bool outermost; -}; - struct unwind_user_state { unsigned long ip; unsigned long sp; diff --git a/kernel/unwind/Makefile b/kernel/unwind/Makefile index 146038165865..c5f9f8124564 100644 --- a/kernel/unwind/Makefile +++ b/kernel/unwind/Makefile @@ -1,2 +1,2 @@ obj-$(CONFIG_UNWIND_USER) +=3D user.o deferred.o - obj-$(CONFIG_HAVE_UNWIND_USER_SFRAME) +=3D sframe.o + obj-$(CONFIG_UNWIND_SFRAME_LOOKUP) +=3D sframe.o diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index f24997e84e05..89dd8c5a6a10 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -12,8 +12,8 @@ #include #include #include -#include -#include +#include +#include =20 #include "sframe.h" #include "sframe_debug.h" @@ -44,8 +44,6 @@ struct sframe_fre_internal { unsigned char dw_size; }; =20 -DEFINE_STATIC_SRCU(sframe_srcu); - static __always_inline unsigned char fre_type_to_size(unsigned char fre_ty= pe) { if (fre_type > 2) @@ -60,6 +58,77 @@ static __always_inline unsigned char dataword_size_enum_= to_size(unsigned char da return 1 << dataword_size; } =20 +#ifdef CONFIG_HAVE_UNWIND_USER_SFRAME + +DEFINE_STATIC_SRCU(sframe_srcu); + +#define UNSAFE_USER_COPY(to, from, size, label) \ + unsafe_copy_from_user(to, (void __user *)from, size, label) + +#define UNSAFE_USER_GET(to, from, type, label) \ + unsafe_get_user(to, (type __user *)from, label) + +#else /* !CONFIG_HAVE_UNWIND_USER_SFRAME */ + +#define UNSAFE_USER_COPY(to, from, size, label) do { \ + (void)to; (void)from; (void)size; \ + goto label; \ +} while (0) + +#define UNSAFE_USER_GET(to, from, type, label) do { \ + (void)to; (void)from; \ + goto label; \ +} while (0) + +#endif /* !CONFIG_HAVE_UNWIND_USER_SFRAME */ + +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +#define KERNEL_COPY(to, from, size, label) memcpy(to, (void *)from, size) +#define KERNEL_GET(to, from, type, label) ({ (to) =3D *(type *)(from); }) + +#else /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + +#define KERNEL_COPY(to, from, size, label) do { \ + (void)(to); (void)(from); (void)size; \ + goto label; \ +} while (0) + +#define KERNEL_GET(to, from, type, label) do { \ + (void)(to); (void)(from); \ + goto label; \ +} while (0) + +#endif /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + +#define DATA_COPY(sec, to, from, size, label) \ +({ \ + switch (sec->sec_type) { \ + case SFRAME_KERNEL: \ + KERNEL_COPY(to, from, size, label); \ + break; \ + case SFRAME_USER: \ + UNSAFE_USER_COPY(to, from, size, label); \ + break; \ + default: \ + goto label; \ + } \ +}) + +#define DATA_GET(sec, to, from, type, label) \ +({ \ + switch (sec->sec_type) { \ + case SFRAME_KERNEL: \ + KERNEL_GET(to, from, type, label); \ + break; \ + case SFRAME_USER: \ + UNSAFE_USER_GET(to, from, type, label); \ + break; \ + default: \ + goto label; \ + } \ +}) + static __always_inline int __read_fde(struct sframe_section *sec, unsigned int fde_num, struct sframe_fde_internal *fde) @@ -69,8 +138,8 @@ static __always_inline int __read_fde(struct sframe_sect= ion *sec, struct sframe_fda_v3 _fda; =20 fde_addr =3D sec->fdes_start + (fde_num * sizeof(struct sframe_fde_v3)); - unsafe_copy_from_user(&_fde, (void __user *)fde_addr, - sizeof(struct sframe_fde_v3), Efault); + DATA_COPY(sec, &_fde, fde_addr, + sizeof(struct sframe_fde_v3), Efault); =20 func_addr =3D fde_addr + _fde.func_start_off; if (func_addr < sec->text_start || func_addr > sec->text_end) @@ -79,8 +148,8 @@ static __always_inline int __read_fde(struct sframe_sect= ion *sec, fda_addr =3D sec->fres_start + _fde.fres_off; if (fda_addr + sizeof(struct sframe_fda_v3) > sec->fres_end) return -EINVAL; - unsafe_copy_from_user(&_fda, (void __user *)fda_addr, - sizeof(struct sframe_fda_v3), Efault); + DATA_COPY(sec, &_fda, fda_addr, + sizeof(struct sframe_fda_v3), Efault); =20 fde->func_addr =3D func_addr; fde->func_size =3D _fde.func_size; @@ -102,21 +171,21 @@ static __always_inline int __find_fde(struct sframe_s= ection *sec, struct sframe_fde_internal *fde) { unsigned long func_addr_low =3D 0, func_addr_high =3D ULONG_MAX; - struct sframe_fde_v3 __user *first, *low, *high, *found =3D NULL; + struct sframe_fde_v3 *first, *low, *high, *found =3D NULL; int ret; =20 - first =3D (void __user *)sec->fdes_start; + first =3D (void *)sec->fdes_start; low =3D first; high =3D first + sec->num_fdes - 1; =20 while (low <=3D high) { - struct sframe_fde_v3 __user *mid; + struct sframe_fde_v3 *mid; s64 func_off; unsigned long func_addr; =20 mid =3D low + ((high - low) / 2); =20 - unsafe_get_user(func_off, (s64 __user *)mid, Efault); + DATA_GET(sec, func_off, mid, s64, Efault); func_addr =3D (unsigned long)mid + func_off; =20 if (ip >=3D func_addr) { @@ -154,47 +223,47 @@ static __always_inline int __find_fde(struct sframe_s= ection *sec, return -EFAULT; } =20 -#define ____UNSAFE_GET_USER_INC(to, from, type, label) \ +#define ____GET_INC(sec, to, from, type, label) \ ({ \ type __to; \ - unsafe_get_user(__to, (type __user *)from, label); \ + DATA_GET(sec, __to, from, type, label); \ from +=3D sizeof(__to); \ to =3D __to; \ }) =20 -#define __UNSAFE_GET_USER_INC(to, from, size, label, u_or_s) \ +#define __GET_INC(sec, to, from, size, label, u_or_s) \ ({ \ switch (size) { \ case 1: \ - ____UNSAFE_GET_USER_INC(to, from, u_or_s##8, label); \ + ____GET_INC(sec, to, from, u_or_s##8, label); \ break; \ case 2: \ - ____UNSAFE_GET_USER_INC(to, from, u_or_s##16, label); \ + ____GET_INC(sec, to, from, u_or_s##16, label); \ break; \ case 4: \ - ____UNSAFE_GET_USER_INC(to, from, u_or_s##32, label); \ + ____GET_INC(sec, to, from, u_or_s##32, label); \ break; \ default: \ return -EFAULT; \ } \ }) =20 -#define UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label) \ - __UNSAFE_GET_USER_INC(to, from, size, label, u) +#define GET_UNSIGNED_INC(sec, to, from, size, label) \ + __GET_INC(sec, to, from, size, label, u) =20 -#define UNSAFE_GET_USER_SIGNED_INC(to, from, size, label) \ - __UNSAFE_GET_USER_INC(to, from, size, label, s) +#define GET_SIGNED_INC(sec, to, from, size, label) \ + __GET_INC(sec, to, from, size, label, s) =20 -#define UNSAFE_GET_USER_INC(to, from, size, label) \ - _Generic(to, \ - u8 : UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label), \ - u16 : UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label), \ - u32 : UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label), \ - u64 : UNSAFE_GET_USER_UNSIGNED_INC(to, from, size, label), \ - s8 : UNSAFE_GET_USER_SIGNED_INC(to, from, size, label), \ - s16 : UNSAFE_GET_USER_SIGNED_INC(to, from, size, label), \ - s32 : UNSAFE_GET_USER_SIGNED_INC(to, from, size, label), \ - s64 : UNSAFE_GET_USER_SIGNED_INC(to, from, size, label)) +#define GET_INC(sec, to, from, size, label) \ + _Generic(to, \ + u8 : GET_UNSIGNED_INC(sec, to, from, size, label), \ + u16 : GET_UNSIGNED_INC(sec, to, from, size, label), \ + u32 : GET_UNSIGNED_INC(sec, to, from, size, label), \ + u64 : GET_UNSIGNED_INC(sec, to, from, size, label), \ + s8 : GET_SIGNED_INC(sec, to, from, size, label), \ + s16 : GET_SIGNED_INC(sec, to, from, size, label), \ + s32 : GET_SIGNED_INC(sec, to, from, size, label), \ + s64 : GET_SIGNED_INC(sec, to, from, size, label)) =20 static __always_inline int __read_regular_fre_datawords(struct sframe_section *sec, @@ -207,19 +276,19 @@ __read_regular_fre_datawords(struct sframe_section *s= ec, s32 cfa_off, ra_off, fp_off; unsigned int cfa_regnum; =20 - UNSAFE_GET_USER_INC(cfa_off, cur, dataword_size, Efault); + GET_INC(sec, cfa_off, cur, dataword_size, Efault); dataword_count--; =20 ra_off =3D sec->ra_off; if (!ra_off && dataword_count) { dataword_count--; - UNSAFE_GET_USER_INC(ra_off, cur, dataword_size, Efault); + GET_INC(sec, ra_off, cur, dataword_size, Efault); } =20 fp_off =3D sec->fp_off; if (!fp_off && dataword_count) { dataword_count--; - UNSAFE_GET_USER_INC(fp_off, cur, dataword_size, Efault); + GET_INC(sec, fp_off, cur, dataword_size, Efault); } =20 if (dataword_count) @@ -255,17 +324,17 @@ __read_flex_fde_fre_datawords(struct sframe_section *= sec, =20 if (dataword_count < 2) return -EFAULT; - UNSAFE_GET_USER_INC(cfa_ctl, cur, dataword_size, Efault); - UNSAFE_GET_USER_INC(cfa_off, cur, dataword_size, Efault); + GET_INC(sec, cfa_ctl, cur, dataword_size, Efault); + GET_INC(sec, cfa_off, cur, dataword_size, Efault); dataword_count -=3D 2; =20 ra_off =3D sec->ra_off; ra_ctl =3D ra_off ? 2 : 0; /* regnum=3D0, deref_p=3D(ra_off !=3D 0), reg_= p=3D0 */ if (dataword_count >=3D 2) { - UNSAFE_GET_USER_INC(ra_ctl, cur, dataword_size, Efault); + GET_INC(sec, ra_ctl, cur, dataword_size, Efault); dataword_count--; if (ra_ctl) { - UNSAFE_GET_USER_INC(ra_off, cur, dataword_size, Efault); + GET_INC(sec, ra_off, cur, dataword_size, Efault); dataword_count--; } else { /* Padding RA location info */ @@ -276,10 +345,10 @@ __read_flex_fde_fre_datawords(struct sframe_section *= sec, fp_off =3D sec->fp_off; fp_ctl =3D fp_off ? 2 : 0; /* regnum=3D0, deref_p=3D(fp_off !=3D 0), reg_= p=3D0 */ if (dataword_count >=3D 2) { - UNSAFE_GET_USER_INC(fp_ctl, cur, dataword_size, Efault); + GET_INC(sec, fp_ctl, cur, dataword_size, Efault); dataword_count--; if (fp_ctl) { - UNSAFE_GET_USER_INC(fp_off, cur, dataword_size, Efault); + GET_INC(sec, fp_off, cur, dataword_size, Efault); dataword_count--; } else { /* Padding FP location info */ @@ -353,11 +422,11 @@ static __always_inline int __read_fre(struct sframe_s= ection *sec, if (fre_addr + addr_size + 1 > sec->fres_end) return -EFAULT; =20 - UNSAFE_GET_USER_INC(ip_off, cur, addr_size, Efault); + GET_INC(sec, ip_off, cur, addr_size, Efault); if (fde_pctype =3D=3D SFRAME_FDE_PCTYPE_INC && ip_off > fde->func_size) return -EFAULT; =20 - UNSAFE_GET_USER_INC(info, cur, 1, Efault); + GET_INC(sec, info, cur, 1, Efault); dataword_count =3D SFRAME_V3_FRE_DATAWORD_COUNT(info); dataword_size =3D dataword_size_enum_to_size(SFRAME_V3_FRE_DATAWORD_SIZE= (info)); if (!dataword_size) @@ -380,7 +449,7 @@ static __always_inline int __read_fre(struct sframe_sec= tion *sec, } =20 static __always_inline int -sframe_init_cfa_rule_data(struct unwind_user_cfa_rule_data *cfa_rule_data, +sframe_init_cfa_rule_data(struct unwind_cfa_rule_data *cfa_rule_data, u32 ctlword, s32 offset) { bool deref_p =3D SFRAME_V3_FLEX_FDE_CTLWORD_DEREF_P(ctlword); @@ -391,13 +460,13 @@ sframe_init_cfa_rule_data(struct unwind_user_cfa_rule= _data *cfa_rule_data, =20 switch (regnum) { case SFRAME_REG_SP: - cfa_rule_data->rule =3D UNWIND_USER_CFA_RULE_SP_OFFSET; + cfa_rule_data->rule =3D UNWIND_CFA_RULE_SP_OFFSET; break; case SFRAME_REG_FP: - cfa_rule_data->rule =3D UNWIND_USER_CFA_RULE_FP_OFFSET; + cfa_rule_data->rule =3D UNWIND_CFA_RULE_FP_OFFSET; break; default: - cfa_rule_data->rule =3D UNWIND_USER_CFA_RULE_REG_OFFSET; + cfa_rule_data->rule =3D UNWIND_CFA_RULE_REG_OFFSET; cfa_rule_data->regnum =3D regnum; } } else { @@ -405,7 +474,7 @@ sframe_init_cfa_rule_data(struct unwind_user_cfa_rule_d= ata *cfa_rule_data, } =20 if (deref_p) - cfa_rule_data->rule |=3D UNWIND_USER_RULE_DEREF; + cfa_rule_data->rule |=3D UNWIND_RULE_DEREF; =20 cfa_rule_data->offset =3D offset; =20 @@ -413,27 +482,27 @@ sframe_init_cfa_rule_data(struct unwind_user_cfa_rule= _data *cfa_rule_data, } =20 static __always_inline void -sframe_init_rule_data(struct unwind_user_rule_data *rule_data, +sframe_init_rule_data(struct unwind_rule_data *rule_data, u32 ctlword, s32 offset) { bool deref_p =3D SFRAME_V3_FLEX_FDE_CTLWORD_DEREF_P(ctlword); bool reg_p =3D SFRAME_V3_FLEX_FDE_CTLWORD_REG_P(ctlword); =20 if (!ctlword && !offset) { - rule_data->rule =3D UNWIND_USER_RULE_RETAIN; + rule_data->rule =3D UNWIND_RULE_RETAIN; return; } if (reg_p) { unsigned int regnum =3D SFRAME_V3_FLEX_FDE_CTLWORD_REGNUM(ctlword); =20 - rule_data->rule =3D UNWIND_USER_RULE_REG_OFFSET; + rule_data->rule =3D UNWIND_RULE_REG_OFFSET; rule_data->regnum =3D regnum; } else { - rule_data->rule =3D UNWIND_USER_RULE_CFA_OFFSET; + rule_data->rule =3D UNWIND_RULE_CFA_OFFSET; } =20 if (deref_p) - rule_data->rule |=3D UNWIND_USER_RULE_DEREF; + rule_data->rule |=3D UNWIND_RULE_DEREF; =20 rule_data->offset =3D offset; } @@ -441,7 +510,7 @@ sframe_init_rule_data(struct unwind_user_rule_data *rul= e_data, static __always_inline int __find_fre(struct sframe_section *sec, struct sframe_fde_internal *fde, unsigned long ip, - struct unwind_user_frame *frame) + struct unwind_frame *frame) { unsigned char fde_pctype =3D SFRAME_V3_FDE_PCTYPE(fde->info); struct sframe_fre_internal *fre, *prev_fre =3D NULL; @@ -501,40 +570,18 @@ static __always_inline int __find_fre(struct sframe_s= ection *sec, return 0; } =20 -int sframe_find(unsigned long ip, struct unwind_user_frame *frame) +static __always_inline int __sframe_find(struct sframe_section *sec, + unsigned long ip, + struct unwind_frame *frame) { - struct mm_struct *mm =3D current->mm; - struct sframe_section *sec; struct sframe_fde_internal fde; int ret; =20 - if (!mm) - return -EINVAL; - - guard(srcu)(&sframe_srcu); - - sec =3D mtree_load(&mm->sframe_mt, ip); - if (!sec) - return -EINVAL; - - if (!user_read_access_begin((void __user *)sec->sframe_start, - sec->sframe_end - sec->sframe_start)) - return -EFAULT; - ret =3D __find_fde(sec, ip, &fde); if (ret) - goto end; - - ret =3D __find_fre(sec, &fde, ip, frame); -end: - user_read_access_end(); - - if (ret =3D=3D -EFAULT) { - dbg_sec("removing bad .sframe section\n"); - WARN_ON_ONCE(sframe_remove_section(sec->sframe_start)); - } + return ret; =20 - return ret; + return __find_fre(sec, &fde, ip, frame); } =20 #ifdef CONFIG_SFRAME_VALIDATION @@ -657,20 +704,23 @@ static int sframe_validate_section(struct sframe_sect= ion *sec) { return 0; } #endif /* !CONFIG_SFRAME_VALIDATION */ =20 =20 -static void free_section(struct sframe_section *sec) -{ - dbg_free(sec); - kfree(sec); -} - static int sframe_read_header(struct sframe_section *sec) { unsigned long header_end, fdes_start, fdes_end, fres_start, fres_end; struct sframe_header shdr; unsigned int num_fdes; =20 - if (copy_from_user(&shdr, (void __user *)sec->sframe_start, sizeof(shdr))= ) { - dbg_sec("header usercopy failed\n"); + switch (sec->sec_type) { + case SFRAME_USER: + if (copy_from_user(&shdr, (void __user *)sec->sframe_start, sizeof(shdr)= )) { + dbg_sec("header usercopy failed\n"); + return -EFAULT; + } + break; + case SFRAME_KERNEL: + shdr =3D *(struct sframe_header *)sec->sframe_start; + break; + default: return -EFAULT; } =20 @@ -717,6 +767,45 @@ static int sframe_read_header(struct sframe_section *s= ec) return 0; } =20 +#ifdef CONFIG_HAVE_UNWIND_USER_SFRAME + +int sframe_find_user(unsigned long ip, struct unwind_frame *frame) +{ + struct mm_struct *mm =3D current->mm; + struct sframe_section *sec; + int ret; + + if (!mm) + return -EINVAL; + + guard(srcu)(&sframe_srcu); + + sec =3D mtree_load(&mm->sframe_mt, ip); + if (!sec) + return -EINVAL; + + if (!user_read_access_begin((void __user *)sec->sframe_start, + sec->sframe_end - sec->sframe_start)) + return -EFAULT; + + ret =3D __sframe_find(sec, ip, frame); + + user_read_access_end(); + + if (ret =3D=3D -EFAULT) { + dbg_sec("removing bad .sframe section\n"); + WARN_ON_ONCE(sframe_remove_section(sec->sframe_start)); + } + + return ret; +} + +static void free_section(struct sframe_section *sec) +{ + dbg_free(sec); + kfree(sec); +} + int sframe_add_section(unsigned long sframe_start, unsigned long sframe_en= d, unsigned long text_start, unsigned long text_end) { @@ -753,6 +842,7 @@ int sframe_add_section(unsigned long sframe_start, unsi= gned long sframe_end, if (!sec) return -ENOMEM; =20 + sec->sec_type =3D SFRAME_USER; sec->sframe_start =3D sframe_start; sec->sframe_end =3D sframe_end; sec->text_start =3D text_start; @@ -838,3 +928,5 @@ void sframe_free_mm(struct mm_struct *mm) =20 mtree_destroy(&mm->sframe_mt); } + +#endif /* CONFIG_HAVE_UNWIND_USER_SFRAME */ diff --git a/kernel/unwind/user.c b/kernel/unwind/user.c index eb7d9489f671..9e57dd79559a 100644 --- a/kernel/unwind/user.c +++ b/kernel/unwind/user.c @@ -8,6 +8,7 @@ #include #include #include +#include =20 #define for_each_user_frame(state) \ for (unwind_user_start(state); !(state)->done; unwind_user_next(state)) @@ -28,7 +29,7 @@ get_user_word(unsigned long *word, unsigned long base, in= t off, unsigned int ws) } =20 static int unwind_user_next_common(struct unwind_user_state *state, - const struct unwind_user_frame *frame) + const struct unwind_frame *frame) { unsigned long cfa, fp, ra; =20 @@ -40,16 +41,16 @@ static int unwind_user_next_common(struct unwind_user_s= tate *state, =20 /* Get the Canonical Frame Address (CFA) */ switch (frame->cfa.rule) { - case UNWIND_USER_CFA_RULE_SP_OFFSET: + case UNWIND_CFA_RULE_SP_OFFSET: cfa =3D state->sp; break; - case UNWIND_USER_CFA_RULE_FP_OFFSET: + case UNWIND_CFA_RULE_FP_OFFSET: if (state->fp < state->sp) return -EINVAL; cfa =3D state->fp; break; - case UNWIND_USER_CFA_RULE_REG_OFFSET: - case UNWIND_USER_CFA_RULE_REG_OFFSET_DEREF: + case UNWIND_CFA_RULE_REG_OFFSET: + case UNWIND_CFA_RULE_REG_OFFSET_DEREF: if (!state->topmost || unwind_user_get_reg(&cfa, frame->cfa.regnum)) return -EINVAL; break; @@ -58,7 +59,7 @@ static int unwind_user_next_common(struct unwind_user_sta= te *state, return -EINVAL; } cfa +=3D frame->cfa.offset; - if (frame->cfa.rule & UNWIND_USER_RULE_DEREF && + if (frame->cfa.rule & UNWIND_RULE_DEREF && get_user_word(&cfa, cfa, 0, state->ws)) return -EINVAL; =20 @@ -76,16 +77,16 @@ static int unwind_user_next_common(struct unwind_user_s= tate *state, =20 /* Get the Return Address (RA) */ switch (frame->ra.rule) { - case UNWIND_USER_RULE_RETAIN: + case UNWIND_RULE_RETAIN: if (!state->topmost || unwind_user_get_ra_reg(&ra)) return -EINVAL; break; /* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */ - case UNWIND_USER_RULE_CFA_OFFSET_DEREF: + case UNWIND_RULE_CFA_OFFSET_DEREF: ra =3D cfa + frame->ra.offset; break; - case UNWIND_USER_RULE_REG_OFFSET: - case UNWIND_USER_RULE_REG_OFFSET_DEREF: + case UNWIND_RULE_REG_OFFSET: + case UNWIND_RULE_REG_OFFSET_DEREF: if (!state->topmost || unwind_user_get_reg(&ra, frame->ra.regnum)) return -EINVAL; ra +=3D frame->ra.offset; @@ -94,21 +95,21 @@ static int unwind_user_next_common(struct unwind_user_s= tate *state, WARN_ON_ONCE(1); return -EINVAL; } - if (frame->ra.rule & UNWIND_USER_RULE_DEREF && + if (frame->ra.rule & UNWIND_RULE_DEREF && get_user_word(&ra, ra, 0, state->ws)) return -EINVAL; =20 /* Get the Frame Pointer (FP) */ switch (frame->fp.rule) { - case UNWIND_USER_RULE_RETAIN: + case UNWIND_RULE_RETAIN: fp =3D state->fp; break; /* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */ - case UNWIND_USER_RULE_CFA_OFFSET_DEREF: + case UNWIND_RULE_CFA_OFFSET_DEREF: fp =3D cfa + frame->fp.offset; break; - case UNWIND_USER_RULE_REG_OFFSET: - case UNWIND_USER_RULE_REG_OFFSET_DEREF: + case UNWIND_RULE_REG_OFFSET: + case UNWIND_RULE_REG_OFFSET_DEREF: if (!state->topmost || unwind_user_get_reg(&fp, frame->fp.regnum)) return -EINVAL; fp +=3D frame->fp.offset; @@ -117,7 +118,7 @@ static int unwind_user_next_common(struct unwind_user_s= tate *state, WARN_ON_ONCE(1); return -EINVAL; } - if (frame->fp.rule & UNWIND_USER_RULE_DEREF && + if (frame->fp.rule & UNWIND_RULE_DEREF && get_user_word(&fp, fp, 0, state->ws)) return -EINVAL; =20 @@ -133,13 +134,13 @@ static int unwind_user_next_fp(struct unwind_user_sta= te *state) struct pt_regs *regs =3D task_pt_regs(current); =20 if (state->topmost && unwind_user_at_function_start(regs)) { - const struct unwind_user_frame fp_entry_frame =3D { + const struct unwind_frame fp_entry_frame =3D { ARCH_INIT_USER_FP_ENTRY_FRAME(state->ws) }; return unwind_user_next_common(state, &fp_entry_frame); } =20 - const struct unwind_user_frame fp_frame =3D { + const struct unwind_frame fp_frame =3D { ARCH_INIT_USER_FP_FRAME(state->ws) }; return unwind_user_next_common(state, &fp_frame); @@ -147,10 +148,10 @@ static int unwind_user_next_fp(struct unwind_user_sta= te *state) =20 static int unwind_user_next_sframe(struct unwind_user_state *state) { - struct unwind_user_frame frame; + struct unwind_frame frame; =20 /* sframe expects the frame to be local storage */ - if (sframe_find(state->ip, &frame)) + if (sframe_find_user(state->ip, &frame)) return -ENOENT; return unwind_user_next_common(state, &frame); } --=20 2.54.0.545.g6539524ca2-goog From nobody Sat May 9 07:32:00 2026 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C2D647B410 for ; Tue, 28 Apr 2026 18:37:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401434; cv=none; b=BB5RBxuELvCv1nuLCUAICEmBz733omofFDP3FUQl7FvLV58isQd8Xp7D+ICJPvLeHKxQ43KBaOgn9l1awL/cCdBVZZfEBnZ090jjK2YfJwN0tj3R0s+sWIgwD6Ydx+BCnq4gv479LDw9pRZxU7B2wHcjhJn40B0mC/WT9Hofr/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401434; c=relaxed/simple; bh=0rU1L8BvPHQ2fDeUrdj36D6kTqUKvcHED5L5ySfdups=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kyR37ByRq+6DB83FoZetYwgah955hvYN2JLH+Zostd4bfPtGo1yipCgjCWa/yeYWmSPjQWRPK0LpyGdvn49vLZV8DGZck/RWBI+Kfz1oIa5MwyhfFyru4hht7hRdEDOy1fvqfqIJ6KxxNv4IER+EeELYT9aBKK8PtQ96Uv909qc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QFEt5Gsi; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QFEt5Gsi" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-82f6a5b4f88so15566590b3a.2 for ; Tue, 28 Apr 2026 11:37:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777401432; x=1778006232; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=MgLPe5eAKuVa5a2kQmP4Ur6cn6qfh4/sYx0yOH1k728=; b=QFEt5GsiNbeFzVyiFmYC6BeKifK1SBvlxCswW2GYvHFz8/NHexXvB8PFptbXuQc/lP 6Ir4OZL+hwh3E3VYKGSm2FCoZRcZNRr2+dFDmfdvisAQ2XQtuXh9xE1vg4V3sfOgqzhr osu28xxQE5MEd+ApYTuzjUeHW5rV7zvEk+vrijdJUWmPo6gtDNc7g/OtPTBBVZRdTY8T V3mypR3NAm8Y1xHR1ifDC+WmXmZ8ePngL4ofp0kvothTF/3A0lSiuzeBxlEeAfZEZgTx zJI9MplZqOtdB668KCbVMLZPCKTDC4r72JTUIaC2uq+EAfyrGZP4CpqqW5zp4PJgfgY/ cXAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777401432; x=1778006232; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MgLPe5eAKuVa5a2kQmP4Ur6cn6qfh4/sYx0yOH1k728=; b=MZeW0fK04KoMrqtJRElBnlv/k8iNgTOI6o3JHJau8iGwL/hp6GDObIl0VhOSCaAqjt jB7LbBePSV9MbFUcmq21P7B2wahCmiGrxqhAn802StI3msJFhXDLJwA3sf9ZZ7anPbw/ zpcrez5dpEP9E3Uwo2rvw0o4DhO1q7yRC3Onz7Epssw108tHx41w+Q21H5HgP6Mw1uyV 1QH19meotbxWPf+DQGPbJTbwnDBuIHT57Zq+fw/cjs8vd7pjXmnmW7F8hDVHECMS+Lr/ AtCR39/u0Tn1I5AtV6r/kblVxqJhirp5VcYAaefPFV3qRYoPl965j7hyx8KuzkIL8Hni tGzg== X-Forwarded-Encrypted: i=1; AFNElJ/7DxKSNq8HRg6a7Q0scF3liKhFYqSeUgYk56BAWgYeyje/dKqlwy+0ScxQv6Jww4DqnJMcTmGB77vDAB4=@vger.kernel.org X-Gm-Message-State: AOJu0YxWeKZCWQYNKwhw8zQEgEwaxjr6WCLkUxcl8mN/nHxw//y8n80E xpE83Bo1yViP6+6G4CBKJzHqLOfKTy0EWxqVKOzwtPZGVdzH/j78Rh4y4MJ62wjeVA0zJmaSJ8R eYwfxJ8sH+JOU7jjGwv2LVIKp3Q== X-Received: from pfnw4.prod.google.com ([2002:aa7:8584:0:b0:82f:473b:a2d5]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:340c:b0:82d:2086:de2a with SMTP id d2e1a72fcca58-834ddc643d1mr4609851b3a.35.1777401431392; Tue, 28 Apr 2026 11:37:11 -0700 (PDT) Date: Tue, 28 Apr 2026 18:36:37 +0000 In-Reply-To: <20260428183643.3796063-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260428183643.3796063-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260428183643.3796063-3-dylanbhatch@google.com> Subject: [PATCH v5 2/8] arm64, unwind: build kernel with sframe V3 info From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Build with -Wa,--gsframe-3 flags to generate a .sframe section. This will be used for in-kernel reliable stacktrace in cases where the frame pointer alone is insufficient. Currently, the sframe format only supports arm64, x86_64 and s390x architectures. Signed-off-by: Weinan Liu Reviewed-by: Prasanna Kumar T S M Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- MAINTAINERS | 1 + Makefile | 8 ++++++++ arch/Kconfig | 21 +++++++++++++++++++++ arch/arm64/Kconfig | 1 + arch/arm64/include/asm/unwind_sframe.h | 8 ++++++++ arch/arm64/kernel/vdso/Makefile | 2 +- include/asm-generic/sections.h | 4 ++++ include/asm-generic/vmlinux.lds.h | 15 +++++++++++++++ 8 files changed, 59 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/include/asm/unwind_sframe.h diff --git a/MAINTAINERS b/MAINTAINERS index cfc7dec88da4..a7d75f9cb5f4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -27561,6 +27561,7 @@ STACK UNWINDING M: Josh Poimboeuf M: Steven Rostedt S: Maintained +F: arch/*/include/asm/unwind_sframe.h F: include/linux/sframe.h F: include/linux/unwind*.h F: kernel/unwind/ diff --git a/Makefile b/Makefile index 2b15f0b4a0cb..6c94a5257679 100644 --- a/Makefile +++ b/Makefile @@ -1110,6 +1110,14 @@ endif # Ensure compilers do not transform certain loops into calls to wcslen() KBUILD_CFLAGS +=3D -fno-builtin-wcslen =20 +# build with sframe table +ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME +CC_FLAGS_SFRAME :=3D -Wa,--gsframe-3 +KBUILD_CFLAGS +=3D $(CC_FLAGS_SFRAME) +KBUILD_AFLAGS +=3D $(CC_FLAGS_SFRAME) +export CC_FLAGS_SFRAME +endif + # change __FILE__ to the relative path to the source directory ifdef building_out_of_srctree KBUILD_CPPFLAGS +=3D -fmacro-prefix-map=3D$(srcroot)/=3D diff --git a/arch/Kconfig b/arch/Kconfig index d7caf2e245ce..8d27b3249e7a 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -520,6 +520,27 @@ config SFRAME_VALIDATION =20 If unsure, say N. =20 +config ARCH_SUPPORTS_UNWIND_KERNEL_SFRAME + bool + help + An architecture can select this if it enables the SFrame (Simple + Frame) unwinder for unwinding kernel stack traces. It uses an unwind + table that is directly generated by the toolchain based on DWARF CFI + information. + +config HAVE_UNWIND_KERNEL_SFRAME + bool "Sframe unwinder" + depends on AS_SFRAME3 + depends on 64BIT + depends on ARCH_SUPPORTS_UNWIND_KERNEL_SFRAME + select UNWIND_SFRAME_LOOKUP + help + This option enables the SFrame (Simple Frame) unwinder for unwinding + kernel stack traces. It uses unwind an table that is directly + generated by the toolchain based on DWARF CFI information. In + practice, this can provide more reliable stacktrace results than + unwinding with frame pointers alone. + config HAVE_PERF_REGS bool help diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 38dba5f7e4d2..f7ae8eaaadc4 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -112,6 +112,7 @@ config ARM64 select ARCH_SUPPORTS_SCHED_SMT select ARCH_SUPPORTS_SCHED_CLUSTER select ARCH_SUPPORTS_SCHED_MC + select ARCH_SUPPORTS_UNWIND_KERNEL_SFRAME select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH select ARCH_WANT_COMPAT_IPC_PARSE_VERSION if COMPAT select ARCH_WANT_DEFAULT_BPF_JIT diff --git a/arch/arm64/include/asm/unwind_sframe.h b/arch/arm64/include/as= m/unwind_sframe.h new file mode 100644 index 000000000000..876412881196 --- /dev/null +++ b/arch/arm64/include/asm/unwind_sframe.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_ARM64_UNWIND_SFRAME_H +#define _ASM_ARM64_UNWIND_SFRAME_H + +#define SFRAME_REG_SP 31 +#define SFRAME_REG_FP 29 + +#endif /* _ASM_ARM64_UNWIND_SFRAME_H */ diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makef= ile index 7dec05dd33b7..c60ef921956f 100644 --- a/arch/arm64/kernel/vdso/Makefile +++ b/arch/arm64/kernel/vdso/Makefile @@ -38,7 +38,7 @@ ccflags-y +=3D -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO CC_FLAGS_REMOVE_VDSO :=3D $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) \ $(RANDSTRUCT_CFLAGS) $(KSTACK_ERASE_CFLAGS) \ $(GCC_PLUGINS_CFLAGS) \ - $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) \ + $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) $(CC_FLAGS_SFRAME) \ -Wmissing-prototypes -Wmissing-declarations =20 CC_FLAGS_ADD_VDSO :=3D -O2 -mcmodel=3Dtiny -fasynchronous-unwind-tables diff --git a/include/asm-generic/sections.h b/include/asm-generic/sections.h index 0755bc39b0d8..336d27011a58 100644 --- a/include/asm-generic/sections.h +++ b/include/asm-generic/sections.h @@ -31,6 +31,7 @@ * __irqentry_text_start, __irqentry_text_end * __softirqentry_text_start, __softirqentry_text_end * __start_opd, __end_opd + * __start_sframe, __end_sframe */ extern char _text[], _stext[], _etext[]; extern char _data[], _sdata[], _edata[]; @@ -53,6 +54,9 @@ extern char __ctors_start[], __ctors_end[]; /* Start and end of .opd section - used for function descriptors. */ extern char __start_opd[], __end_opd[]; =20 +/* Start and end of .sframe section - used for stack unwinding. */ +extern char __start_sframe[], __end_sframe[]; + /* Start and end of instrumentation protected text section */ extern char __noinstr_text_start[], __noinstr_text_end[]; =20 diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinu= x.lds.h index 1e1580febe4b..090da633db92 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -491,6 +491,8 @@ *(.rodata1) \ } \ \ + SFRAME \ + \ /* PCI quirks */ \ .pci_fixup : AT(ADDR(.pci_fixup) - LOAD_OFFSET) { \ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_early, _pci_fixups_early, __start= , __end) \ @@ -911,6 +913,19 @@ #define TRACEDATA #endif =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME +#define SFRAME \ + /* sframe */ \ + .sframe : AT(ADDR(.sframe) - LOAD_OFFSET) { \ + __start_sframe =3D .; \ + KEEP(*(.sframe)) \ + KEEP(*(.init.sframe)) \ + __end_sframe =3D .; \ + } +#else +#define SFRAME +#endif + #ifdef CONFIG_PRINTK_INDEX #define PRINTK_INDEX \ .printk_index : AT(ADDR(.printk_index) - LOAD_OFFSET) { \ --=20 2.54.0.545.g6539524ca2-goog From nobody Sat May 9 07:32:00 2026 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A83422DF717 for ; Tue, 28 Apr 2026 18:37:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401436; cv=none; b=rdasLMfzHBbqyvjlX9ev9WNn825IkS6R0hjfsk74OUj5VE2pmEGGCXUO992BSlA1wH/jTlw2GFCXS3hJ6f34CI1zGeYZGwxN9nxoHUPn2tpFBkRZiHaDCzxdzvzNBksQxQ8RTyFPqSSTk7stU4CXBvWNwjQWIQuIZ3n7su79NSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401436; c=relaxed/simple; bh=tefadcmVpwT/xUVJDO57dtYdiK3vwLPkO5vSZsjgUdk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Tkf1LC2Q0uIF6F3FvDADbZYAibzNy5IgcXuox5lcFO9ul5qMyhKkbvtmhruWO3WhKr2ofznOFoJWUPSeCg2xMitmfxYs2djzGIAZht/EfHcYhMiEIlLIUAkQasehSXteqseF4gDrvZ1O2PnogcQgbYxuW1wNRKWpwKBWTfMPSPc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rqQx3dAd; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rqQx3dAd" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-c76c6280732so13459810a12.0 for ; Tue, 28 Apr 2026 11:37:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777401435; x=1778006235; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rEpxSS5AT7sZwZhISlQ7XtXed3JthTvzY7OBT2Phvm0=; b=rqQx3dAd/ZXH5/17SVRgTl/JkHFXzU8byZNT00J97UJk+aU1evq09GDHXvI1yKYyN0 qTALV0aykEdUJcVtkkpWidN39NoE5KzOZvr6aePhQTS7chLF1axB5TVZfU92Foh1mQym 3KAIPenmQI1hO10E5CHsHvzvnsl5Wi8p6rTFigydrqH0yUrTL2a56QBteJSk35XvjkrD vbD4uQjVO1gPS+fep4iW0ea2S3IMERkAh2PE+DYbYisdsSYFvrpWyY+uY8C9G6tUcyle Zc27OpA7jwC3oUnSjtEqJP1/Vs8cjmp57M804JVrUNt140wWAIH293btd/8otWZ3b1VQ lBAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777401435; x=1778006235; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rEpxSS5AT7sZwZhISlQ7XtXed3JthTvzY7OBT2Phvm0=; b=lYb1jOJ0jDvVmg2Y4nlsMWUI9n/jxNbCKKj+kIH6MbHqCbmsgdBnCtLhQU//Qzcj7i 9myOlCsRJSehDYB0dE2YR3fVNCmgcBUsSLpA7hpTEoj9TZTh7BzCjZ7t3zoPe9pt/JDD +0E5oUvnTjOcrvMlF3j7fOeX70RGx3ngLV+0c3cSe5ekb8SOqQWAu/stI+rIQW2AXukE 4IcljmbLpJQ446LggyAB21esW3teAQy+CTs5Nd4chHVD34zkNN9xsIKRhHAVJgqXalxF lBl6nuMFd50lqi9HusQ9q/+WvFihnjnVUbtKQWig1pK7k1QYiaVbtwL0w9Daabt83xgU EknA== X-Forwarded-Encrypted: i=1; AFNElJ+PwmhlwuhmDYH/NYap6JH6FEx9G78LQxwJKiBCBxvsYaN/hCxhU7ZwGttqW8wxyaG08erF+PoLSFWuyJ4=@vger.kernel.org X-Gm-Message-State: AOJu0YwA26EXmMZl7tNLg4vnZnf3qZLDJQ+q0SeLXQ/1dbiXKpBuDtc8 8r2K5SCiteiCf5NgHC3AdrdDLXoen1bN8/RzFbA6ke7eomdS4KkmCScE4OyyEhp+vbqRyhzHdjJ Vs2lSpDrY29b+tmU2/psQW7a04g== X-Received: from pfuw1.prod.google.com ([2002:a05:6a00:14c1:b0:82f:a366:f295]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:6d89:b0:3a2:ebc1:4620 with SMTP id adf61e73a8af0-3a39c33cdeemr5030967637.38.1777401434645; Tue, 28 Apr 2026 11:37:14 -0700 (PDT) Date: Tue, 28 Apr 2026 18:36:38 +0000 In-Reply-To: <20260428183643.3796063-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260428183643.3796063-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260428183643.3796063-4-dylanbhatch@google.com> Subject: [PATCH v5 3/8] arm64: entry: add unwind info for various kernel entries From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Weinan Liu DWARF CFI (Call Frame Information) specifies how to recover the return address and callee-saved registers at each PC in a given function. Compilers are able to generate the CFI annotations when they compile the code to assembly language. For handcrafted assembly, we need to annotate them by hand. Annotate minimal CFI to enable stacktracing using SFrame for kernel exception entries through el1*_64_*() paths and irq entries through call_on_irq_stack() Signed-off-by: Weinan Liu Suggested-by: Jens Remus Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- arch/arm64/kernel/entry.S | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index f8018b5c1f9a..dc55b0b19cfa 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -30,6 +30,12 @@ #include #include =20 +/* + * Do not generate .eh_frame. Only generate .debug_frame and optionally + * .sframe (via assembler option --gsframe[-N]). + */ + .cfi_sections .debug_frame + .macro clear_gp_regs .irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,2= 5,26,27,28,29 mov x\n, xzr @@ -575,7 +581,16 @@ SYM_CODE_START_LOCAL(el\el\ht\()_\regsize\()_\label) .if \el =3D=3D 0 b ret_to_user .else + /* + * Minimal DWARF CFI for unwinding across the call above. + * Enable unwinding for el1*_64_*() path only. + */ + .cfi_startproc + .cfi_def_cfa_offset PT_REGS_SIZE + .cfi_offset 29, S_FP - PT_REGS_SIZE + .cfi_offset 30, S_LR - PT_REGS_SIZE b ret_to_kernel + .cfi_endproc .endif SYM_CODE_END(el\el\ht\()_\regsize\()_\label) .endm @@ -872,6 +887,7 @@ NOKPROBE(ret_from_fork) * Calls func(regs) using this CPU's irq stack and shadow irq stack. */ SYM_FUNC_START(call_on_irq_stack) + .cfi_startproc save_and_disable_daif x9 #ifdef CONFIG_SHADOW_CALL_STACK get_current_task x16 @@ -882,6 +898,9 @@ SYM_FUNC_START(call_on_irq_stack) /* Create a frame record to save our LR and SP (implicit in FP) */ stp x29, x30, [sp, #-16]! mov x29, sp + .cfi_def_cfa 29, 16 + .cfi_offset 29, -16 + .cfi_offset 30, -8 =20 ldr_this_cpu x16, irq_stack_ptr, x17 =20 @@ -897,9 +916,13 @@ SYM_FUNC_START(call_on_irq_stack) */ mov sp, x29 ldp x29, x30, [sp], #16 + .cfi_restore 29 + .cfi_restore 30 + .cfi_def_cfa 31, 0 scs_load_current restore_irq x9 ret + .cfi_endproc SYM_FUNC_END(call_on_irq_stack) NOKPROBE(call_on_irq_stack) =20 --=20 2.54.0.545.g6539524ca2-goog From nobody Sat May 9 07:32:00 2026 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEF7047B422 for ; Tue, 28 Apr 2026 18:37:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401441; cv=none; b=q56aXvGFz4V2WNyDZrPOLV88EyfQ3aWO8bI21T+Bp/V2mdiE644/FEnuDcyXsurEjMEA8RlH1K0zwcdHKAL5DiuqDfk037UtyAQ2Xe/wYcSlUIpO8fJdYD/MZALoFkN/q5dkbsmisBDeWu0E24Swawbr1k3oQoBxmZ1VfLhrs6U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401441; c=relaxed/simple; bh=57pPL7fXM2OfA7mEwfEVfLeqPvT/qphbCXDrvxnSDNE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uCGRetA2/57+bFeIvoXvUashVuGREqY1FMLtu/z+qGIAc3S8Bw6X6oZutAX4979MvlWraFrpv9WIx5L+TeWR3T+81PPMP7hr3N08A0bTWSiumUg0/gLnUGTk70mKDUv2xfjBT12vj46XF6p7gaJ4fC9+OusWR0omPnMQ+BoybfY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jTZeXEuC; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jTZeXEuC" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-c79943d2fbfso3602414a12.1 for ; Tue, 28 Apr 2026 11:37:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777401439; x=1778006239; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uAHjM+ToD9UfswYHel5T1N/OddHAZ2XMTwnzhPtkH7g=; b=jTZeXEuCbG9Pm2ikTqnhd0sLrkgpkhCQACujYB1rEOmNwRO8oMPrVWTHLC/7eW3thJ /aD/55xNoMuVtZGi8n5T9Zd18qV9tyhRNkY9TWtm8FfSbeVp0WAeDk4II7Kts+sI5HbZ 0sCymuBjA628Z5JwfetWoherPzqy1hWo1YYa7X3AWoPlEWvzocPozBRBHBqroppVXQ2a fwC6t1lNY38mPJacSTanr4pDAkYIcwqK5Aen+n4twOLgWxAhdW+WMRk1VV80RR3JY5C0 l1KqdMncOiV80HfKU32yfMvFKrs4uIumYfjgcBcfXI2DH60eHyxpCIp/XP/CAJOdllxZ adfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777401439; x=1778006239; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uAHjM+ToD9UfswYHel5T1N/OddHAZ2XMTwnzhPtkH7g=; b=NFdqgONaikN0uuRcjD1qiHzz/wp58HOVOikS9o3UOA2lL1MotwT9cBKUx09oRb0W9U vYaBbkjh91A07nxfgbl/GGKjaf4tyC5wk+Wb8SD41sZ4jWvQTUvH+VKPGiv+YigiG81c 81jgfnIjvX8FEmrTOUwKxiWVub0Vy0gZT2vpAjPgJmSoTcB6Lg5XGgdhtI5DInvFYBQ6 zKkrLi3/XwA68Rej+LoGfGfxerM/hcW91X7UTytsotky9XrMwBWXjrdOs3Ks8wJTrxK2 BSg6qhXbX+6xHfMUbNHSHAOyiA7ebkaSs3uN4wwTuVL685Vy5euOWgJQg2TlT5eO0S9k SKLg== X-Forwarded-Encrypted: i=1; AFNElJ/FFqX955BVPZUyPjNGJPgcjOYFwdpNJv7mTHzEfDjgrp9oMgCD/VAl0gRBIJW4yJsB/2SE0d3f2AZL1pE=@vger.kernel.org X-Gm-Message-State: AOJu0YyDGkJRR0CnjDuKYdTzAhJPM2ugO88HdA7eJBkADRHEQjjI1hvm CXZPVadlvm4z2APRo6udT9RuzM50WhaFquawhABGhQwDYUJI8k95v6H8LJ4sPt+T7GVE5UjGQlD SUhnv/pSswT07C9tnXg4n9GIhnw== X-Received: from pfblh13.prod.google.com ([2002:a05:6a00:710d:b0:82f:b5a0:b27]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1789:b0:82c:9c47:fefb with SMTP id d2e1a72fcca58-834ddad2852mr4478288b3a.20.1777401438937; Tue, 28 Apr 2026 11:37:18 -0700 (PDT) Date: Tue, 28 Apr 2026 18:36:39 +0000 In-Reply-To: <20260428183643.3796063-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260428183643.3796063-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260428183643.3796063-5-dylanbhatch@google.com> Subject: [PATCH v5 4/8] sframe: Provide PC lookup for vmlinux .sframe section From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With SFRAME_UNWINDER, read in the .sframe section at boot. This provides unwind data as an alternative/supplement to frame pointer-based unwinding. Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- arch/arm64/kernel/setup.c | 2 ++ include/linux/sframe.h | 14 ++++++++++++++ kernel/unwind/sframe.c | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 52 insertions(+) diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index 23c05dc7a8f2..4a633bc7aefb 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -32,6 +32,7 @@ #include #include #include +#include =20 #include #include @@ -375,6 +376,7 @@ void __init __no_sanitize_address setup_arch(char **cmd= line_p) "This indicates a broken bootloader or old kernel\n", boot_args[1], boot_args[2], boot_args[3]); } + init_sframe_table(); } =20 static inline bool cpu_can_disable(unsigned int cpu) diff --git a/include/linux/sframe.h b/include/linux/sframe.h index 0cb2924367bc..5b7341b61a7c 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -69,4 +69,18 @@ static inline int sframe_find_user(unsigned long ip, str= uct unwind_frame *frame) =20 #endif /* CONFIG_HAVE_UNWIND_USER_SFRAME */ =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +void __init init_sframe_table(void); +void sframe_module_init(struct module *mod, void *sframe, size_t sframe_si= ze, + void *text, size_t text_size); + +extern int sframe_find_kernel(unsigned long ip, struct unwind_frame *frame= ); + +#else + +static inline void __init init_sframe_table(void) {} + +#endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + #endif /* _LINUX_SFRAME_H */ diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index 89dd8c5a6a10..430bff9533ee 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -14,10 +14,20 @@ #include #include #include +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME +#include +#endif =20 #include "sframe.h" #include "sframe_debug.h" =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +static bool sframe_init __ro_after_init; +static struct sframe_section kernel_sfsec __ro_after_init; + +#endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + struct sframe_fde_internal { unsigned long func_addr; u32 func_size; @@ -930,3 +940,29 @@ void sframe_free_mm(struct mm_struct *mm) } =20 #endif /* CONFIG_HAVE_UNWIND_USER_SFRAME */ + +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +int sframe_find_kernel(unsigned long ip, struct unwind_frame *frame) +{ + if (!frame || !sframe_init) + return -EINVAL; + + return __sframe_find(&kernel_sfsec, ip, frame); +} + +void __init init_sframe_table(void) +{ + kernel_sfsec.sec_type =3D SFRAME_KERNEL; + kernel_sfsec.sframe_start =3D (unsigned long)__start_sframe; + kernel_sfsec.sframe_end =3D (unsigned long)__end_sframe; + kernel_sfsec.text_start =3D (unsigned long)_stext; + kernel_sfsec.text_end =3D (unsigned long)_etext; + + if (WARN_ON(sframe_read_header(&kernel_sfsec))) + return; + + sframe_init =3D true; +} + +#endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ --=20 2.54.0.545.g6539524ca2-goog From nobody Sat May 9 07:32:00 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DED7E47AF57 for ; Tue, 28 Apr 2026 18:37:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401445; cv=none; b=AHdb+3hhHXwTwQg+NljDCHOKgnseaAMgHJffY3M4xAfhYCvExd1QPSV8tQ8LBOXhX7oEBrq03BFTG1hC2SBFLut9V07UHEoRLQHTn1vMgBNuzKhpFIuNFlETHaq66tOJhlaV/5hOCxsUcPilnPTGWmn4/DPK+jOaj+zK/pl8vMI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401445; c=relaxed/simple; bh=zsbBPiSMTY+4JYf4sn3CPXmmxXgtYh5S3I+3ssE0e6M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cs1wY+aUfqyomPJnKHXz8RbNKyLx5F8bqUSKmaZd4I12+tDWrV0sD5hr750KmRylRbwVOOmq9/G7MZoQumH+58FCnKBTYOuGujM4Iu13vbxBr9yh595GNeU2HRTKVn48i8FJzMrZCqVjGsQ0GJrN8cYYjNz2OSBRIdMwVZ1XTGI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bWvTgcYK; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bWvTgcYK" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-35842aa350fso22479067a91.0 for ; Tue, 28 Apr 2026 11:37:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777401442; x=1778006242; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=HEk4CM6KkrmWUN7H1Rd0LwOd0ZnJZhF5RnXyjlQj1vc=; b=bWvTgcYKtxjAm4s8vmwxCVHLZfs+YYh47l5rhwi6Xs8d+9FnMsVUaocvRuR4m+HzmH GFRfltfwTIUffp7zKkY97sdeNeec9gs6zzTAn/U/q6DGc9l4nTRsaLYkeq4v/WAXAhNb K5NMV3jlgu8y/KaGCkk9oEA/AUK9ua/iA1e+z1qQHpItwsPCYdIwu5xVlhrAk3915hXR FmN0km7W/WtXlkpnlvVgJ0XC1YK9bEenZY2anUWw5DDRqwaBqKJZRswcjl6FQEZQ7mkq lbNpztfLGQCrDPdonnqMg6Oa+PntzjUoK922lAyM+Ap8CT0iaRcdtv0jpwTrSn+H4Lg5 2oSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777401442; x=1778006242; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=HEk4CM6KkrmWUN7H1Rd0LwOd0ZnJZhF5RnXyjlQj1vc=; b=R4kwqUJnHdix3AgU85xYcKM3656fFnpqahdZ3U8szODZIk3TSSgLWs+wtPGfttx2eY IR4rO6eKqbalXBWv3MdBDnZTRgOL6Q9VkUPz+bPOyU2I90VCqGZfcjSQyKCkdXx2iYD1 xqPY9HBS3vp9RPvaesPvdbAEefs23au3e6/MJjBsHInfN+lkwR84pGWXg/oIxuaZwA8X sOGNmWSaQhhBq+g8ZvX/S23xt8ZsJzYktq+RJ9yCilg2nQNNAWR/N3YfFxvMJdyY+3fe H9LUOsaqIguDnzaKKE9Y+xF6bkKF3Rqf2QK28AIrXXEY8joJlGsQWSKoJs9iCLj3KW6K kSUg== X-Forwarded-Encrypted: i=1; AFNElJ/LFMbd9WMawLzZaCcc1c7gyAJ2AtQujj1fyN9JslzY0GdQWQXViKrPiMqcIG/b69YM1gJCT605pIow8xk=@vger.kernel.org X-Gm-Message-State: AOJu0Ywegke8xRxT3EIrcOeGabOFlRfAcYSLoL2TC4LnoEhyF5EUuGLx 26LUsgvUPuRoPt/oYMTBgxcoBi7yG7eZmqn8NCU1tFRJNZ3nq0E6YIT3tTqsgGtaV/1KQhFjPUW 7QBw3TDfncAif4IPLfFIuDeR1iw== X-Received: from plbjl20.prod.google.com ([2002:a17:903:1354:b0:2b0:b22a:e6ef]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2ecd:b0:364:7467:53e5 with SMTP id 98e67ed59e1d1-36491ff60aamr4439314a91.2.1777401441910; Tue, 28 Apr 2026 11:37:21 -0700 (PDT) Date: Tue, 28 Apr 2026 18:36:40 +0000 In-Reply-To: <20260428183643.3796063-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260428183643.3796063-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260428183643.3796063-6-dylanbhatch@google.com> Subject: [PATCH v5 5/8] sframe: Allow unsorted FDEs From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The .sframe in kernel modules is built without SFRAME_F_FDE_SORTED set. In order to allow sframe PC lookup in modules, add a code path to handle unsorted FDE tables by doing a simple linear search. Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- include/linux/sframe.h | 1 + kernel/unwind/sframe.c | 45 +++++++++++++++++++++++++++++++++++++----- 2 files changed, 41 insertions(+), 5 deletions(-) diff --git a/include/linux/sframe.h b/include/linux/sframe.h index 5b7341b61a7c..8ae31ed36226 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -28,6 +28,7 @@ struct sframe_section { unsigned long fres_start; unsigned long fres_end; unsigned int num_fdes; + bool fdes_sorted; =20 signed char ra_off; signed char fp_off; diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index 430bff9533ee..dcf4deb378dc 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -176,9 +176,35 @@ static __always_inline int __read_fde(struct sframe_se= ction *sec, return -EFAULT; } =20 -static __always_inline int __find_fde(struct sframe_section *sec, - unsigned long ip, - struct sframe_fde_internal *fde) +static __always_inline int __find_fde_unsorted(struct sframe_section *sec, + unsigned long ip, + struct sframe_fde_internal *fde) +{ + struct sframe_fde_v3 *cur, *start, *end; + + start =3D (struct sframe_fde_v3 *)sec->fdes_start; + end =3D start + sec->num_fdes; + + for (cur =3D start; cur < end; cur++) { + s64 func_off; + u32 func_size; + unsigned long func_addr; + + DATA_GET(sec, func_off, &cur->func_start_off, s64, Efault); + DATA_GET(sec, func_size, &cur->func_size, u32, Efault); + func_addr =3D (unsigned long)cur + func_off; + + if (ip >=3D func_addr && ip < func_addr + func_size) + return __read_fde(sec, cur - start, fde); + } + return -EINVAL; +Efault: + return -EFAULT; +} + +static __always_inline int __find_fde_sorted(struct sframe_section *sec, + unsigned long ip, + struct sframe_fde_internal *fde) { unsigned long func_addr_low =3D 0, func_addr_high =3D ULONG_MAX; struct sframe_fde_v3 *first, *low, *high, *found =3D NULL; @@ -233,6 +259,15 @@ static __always_inline int __find_fde(struct sframe_se= ction *sec, return -EFAULT; } =20 +static __always_inline int __find_fde(struct sframe_section *sec, + unsigned long ip, + struct sframe_fde_internal *fde) +{ + if (sec->fdes_sorted) + return __find_fde_sorted(sec, ip, fde); + return __find_fde_unsorted(sec, ip, fde); +} + #define ____GET_INC(sec, to, from, type, label) \ ({ \ type __to; \ @@ -657,7 +692,7 @@ static int sframe_validate_section(struct sframe_sectio= n *sec) return ret; =20 ip =3D fde.func_addr; - if (ip <=3D prev_ip) { + if (sec->fdes_sorted && ip <=3D prev_ip) { dbg_sec("fde %u not sorted\n", i); return -EFAULT; } @@ -736,7 +771,6 @@ static int sframe_read_header(struct sframe_section *se= c) =20 if (shdr.preamble.magic !=3D SFRAME_MAGIC || shdr.preamble.version !=3D SFRAME_VERSION_3 || - !(shdr.preamble.flags & SFRAME_F_FDE_SORTED) || !(shdr.preamble.flags & SFRAME_F_FDE_FUNC_START_PCREL) || shdr.auxhdr_len) { dbg_sec("bad/unsupported sframe header\n"); @@ -766,6 +800,7 @@ static int sframe_read_header(struct sframe_section *se= c) return -EINVAL; } =20 + sec->fdes_sorted =3D shdr.preamble.flags & SFRAME_F_FDE_SORTED; sec->num_fdes =3D num_fdes; sec->fdes_start =3D fdes_start; sec->fres_start =3D fres_start; --=20 2.54.0.545.g6539524ca2-goog From nobody Sat May 9 07:32:00 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4791747B408 for ; Tue, 28 Apr 2026 18:37:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401447; cv=none; b=TiKLdj081SrFlBiFOk5pMIcWK0u2MjwAy3H1FU83k9labbVrf4Qs/7ZM6KLme8inPeQSi1exWcHaY086jbXYGVyMdjPtH8908XwFHsHkLB8tIsMT3deJ24pNZslpYmyKlYuAznwNpBl/3e8Xwc2Ol4f68HX3XoS0g4uxun9vnl4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401447; c=relaxed/simple; bh=VU/AUcZ1EaUk6V/EpDZo4mcmhiF2Ge8dGm5Jwl/Qplw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Pe4a/Le7cGcBvo0qHbkPfVwzO7KrWr+I25K7mZxGXQeISWSDxCjEilJWz5t/ukyiESsWuO+RH9YxuUoRotXoBCXKagfdjDqNa3LfsX32QWGOzOO7sGSvVPrE9VWI5oNN3QHjS08frtJiLPNIBu+VBhkDCcBHgoKXOVXROTLx+Qc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dBOUo4oH; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dBOUo4oH" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2b24af7ca99so160300055ad.1 for ; Tue, 28 Apr 2026 11:37:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777401446; x=1778006246; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=M7EyQfNtegfTWEfYy8kG9xX+fYrEWmDcicnmg1Ulors=; b=dBOUo4oHYE93I9Lvkkoi0jRWP3BrC2uBsU4+R3xwErIwl6rzl02duWiAfEaaoCVY01 bH7dQ616qBGVzMAJUElNLC0xxjtlzRXJEqMJHyVBpRG1qEKhm1tR4XrWdMQA1xcB9qF/ IFu01l1N/MoaJVSYvHKyoW422v4ZtuTA6Wfd1G+GHpOsltIX+xd2WqEHNH7/DPzt9cwC NgOiWWk54EWxe0kC9qrZZOK+J6pP1uc0vjHSa/HIsxl+4ihtfutPSyuzmCZgqlsKuHDr peKpe1ST94TvIUOhk7vESFjM0+lQqPUmX8hcv5P20TvRe4/2OoB7nv3NVcaeepgQC8Q6 QXwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777401446; x=1778006246; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=M7EyQfNtegfTWEfYy8kG9xX+fYrEWmDcicnmg1Ulors=; b=fLIYfsVQ4HcWdGRtOXEtqv7W1EwuF6jUVHTCvWo/Xli+A2usU60yi6Jnh2JmDKr5WT gLy2ksltTjp+8MlJWHBa5FJ+8c0g0lO1xBoCtqJpTvyERo1XGZCne14iO6WDUJSiIkf8 f3KUnuGjmPCaFaoKcGCkf/ugqLpXf/ciFXbkd1o/fFzHC5CwLP3zfIpJpc2XYHu51Wk/ UZxd6qgbR/zyA5Guq4Wzxe7zWIGJGAcAmqh3UdC39AZaYMONICykle/XmMI3jy4G0qhA +7YIZgrwXjENvipPD+/J7vI1PwUtH/KZl7YauXtJsCnZvBrzRX5q1R9zVHx+Kc7iFFhw guVA== X-Forwarded-Encrypted: i=1; AFNElJ8EMxlBuWDj5lcZIxgl7hoX2/OBuVs0HyNjySnXNCcC6dMYF6aU5pPU51qrhPwLQQPr0PuMrNiAlL1GOKo=@vger.kernel.org X-Gm-Message-State: AOJu0YxfVIuhK7Z37zXUaNcx9cU6RKWgQ4hLg/brXvZyVE4aF4Mlb/Yq ajjAev9HItoNhGUabNUofOpbUdaBoOra5pt0uBicuzi8/dIP+ax01sSL80y/lNCy11aH53RqAgV 3un2t6oVpCQSOgXxUZYAsaVo/HA== X-Received: from pgbdp11.prod.google.com ([2002:a05:6a02:f0b:b0:c6d:caaa:3364]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:7286:b0:3a2:d629:16a2 with SMTP id adf61e73a8af0-3a39bfc6a83mr4727109637.10.1777401445332; Tue, 28 Apr 2026 11:37:25 -0700 (PDT) Date: Tue, 28 Apr 2026 18:36:41 +0000 In-Reply-To: <20260428183643.3796063-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260428183643.3796063-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260428183643.3796063-7-dylanbhatch@google.com> Subject: [PATCH v5 6/8] arm64/module, sframe: Add sframe support for modules From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add sframe table to mod_arch_specific and support sframe PC lookups when an .sframe section can be found on incoming modules. Signed-off-by: Weinan Liu Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- arch/arm64/include/asm/module.h | 6 +++++ arch/arm64/kernel/module.c | 8 +++++++ include/linux/sframe.h | 2 ++ kernel/unwind/sframe.c | 40 +++++++++++++++++++++++++++++++-- 4 files changed, 54 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/modul= e.h index fb9b88eebeb1..07f309c51eee 100644 --- a/arch/arm64/include/asm/module.h +++ b/arch/arm64/include/asm/module.h @@ -6,6 +6,7 @@ #define __ASM_MODULE_H =20 #include +#include =20 struct mod_plt_sec { int plt_shndx; @@ -17,6 +18,11 @@ struct mod_arch_specific { struct mod_plt_sec core; struct mod_plt_sec init; =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + struct sframe_section sframe_sec; + bool sframe_init; +#endif + /* for CONFIG_DYNAMIC_FTRACE */ struct plt_entry *ftrace_trampolines; struct plt_entry *init_ftrace_trampolines; diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 24adb581af0e..427f187e9531 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -18,6 +18,7 @@ #include #include #include +#include =20 #include #include @@ -515,5 +516,12 @@ int module_finalize(const Elf_Ehdr *hdr, } } =20 + s =3D find_section(hdr, sechdrs, ".sframe"); + if (s) { + struct module_memory *t =3D &me->mem[MOD_TEXT]; + + sframe_module_init(me, (void *)s->sh_addr, s->sh_size, + t->base, t->size); + } return module_init_ftrace_plt(hdr, sechdrs, me); } diff --git a/include/linux/sframe.h b/include/linux/sframe.h index 8ae31ed36226..27f5a66190af 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -81,6 +81,8 @@ extern int sframe_find_kernel(unsigned long ip, struct un= wind_frame *frame); #else =20 static inline void __init init_sframe_table(void) {} +static inline void sframe_module_init(struct module *mod, void *sframe, si= ze_t sframe_size, + void *text, size_t text_size) {} =20 #endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ =20 diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index dcf4deb378dc..70001c8e586d 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -980,10 +980,27 @@ void sframe_free_mm(struct mm_struct *mm) =20 int sframe_find_kernel(unsigned long ip, struct unwind_frame *frame) { - if (!frame || !sframe_init) + struct sframe_section *sec; + + if (!frame) return -EINVAL; =20 - return __sframe_find(&kernel_sfsec, ip, frame); + if (is_ksym_addr(ip)) { + if (!sframe_init) + return -EINVAL; + + sec =3D &kernel_sfsec; + } else { + struct module *mod; + + mod =3D __module_address(ip); + if (!mod || !mod->arch.sframe_init) + return -EINVAL; + + sec =3D &mod->arch.sframe_sec; + } + + return __sframe_find(sec, ip, frame); } =20 void __init init_sframe_table(void) @@ -1000,4 +1017,23 @@ void __init init_sframe_table(void) sframe_init =3D true; } =20 +void sframe_module_init(struct module *mod, void *sframe, size_t sframe_si= ze, + void *text, size_t text_size) +{ + struct sframe_section sec; + + memset(&sec, 0, sizeof(sec)); + sec.sec_type =3D SFRAME_KERNEL; + sec.sframe_start =3D (unsigned long)sframe; + sec.sframe_end =3D (unsigned long)sframe + sframe_size; + sec.text_start =3D (unsigned long)text; + sec.text_end =3D (unsigned long)text + text_size; + + if (WARN_ON(sframe_read_header(&sec))) + return; + + mod->arch.sframe_sec =3D sec; + mod->arch.sframe_init =3D true; +} + #endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ --=20 2.54.0.545.g6539524ca2-goog From nobody Sat May 9 07:32:00 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D084147B439 for ; Tue, 28 Apr 2026 18:37:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401450; cv=none; b=WwZI+06ECDVKlCOXlZPqD/iAEGl+Wp5Lhsrbk6uQJMuvppoz8vnxCTHvgL1gE3oKW4LNxZIcKLfInCJBlcYIfv44FbegC0Sgwujt2mrYYm99Or40zSahZTm3akSATT++mOkCBL5bRT6V1GNlaYsGOCUicDXXC6o3R9as1yQgP+8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401450; c=relaxed/simple; bh=wm8fmz57HEth2J1hN6vgXStLpyz5KgZ/vbS889B0dtc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BAmK2R27V7u48aaFi7WTUru81F7v2eN6Tmb7eQxapEZwySDnvG7W1oh0D3rsbBxz+ZTZmFEcmvWXBR4TYs4ZIeSNPxJdBgEwR0ElsvA9aixdKzkL5DLUPoAFJdyaVPzGSwf0nMnGS+ixIUKxeMSGg96kFJH4DKZAk/MQnWM0u8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uRZkCkQg; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uRZkCkQg" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2b2ed279eedso82047535ad.0 for ; Tue, 28 Apr 2026 11:37:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777401448; x=1778006248; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Goh3F7XcpkTSr7iRpzcfIAl6DDNpb912M5nWLNqSnmg=; b=uRZkCkQgy1PKvqXpmzO4WYWxYJLJbamh9mliJPRtSNHvwF907HMdmgAXWVhvINHonG jYPPNcBRkqHuOJ2+zoKypMm+LA7rWzEpgG3dI0O2jMJnRkJ6oKykwYofV3fWzA7bmebN j0+f6hP9wdQx1GWEkKmWRk4ja7H1WV1rIvK84anqqdmt8z95XsnqZTBOg93wvOdUASwU +i+z8d92fjSoNUTNi1ETzgwOnTCx6yPOyffCtQgkx/FTzYXjHjoAV4SlwiDrrY7fLuc8 DZqF6uEZquvvETjiRANAIoWdVAtA7Ng2gE996+0RoZBk+M2ZiZTEQp0QRJoQQ33tWA/N 7T8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777401448; x=1778006248; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Goh3F7XcpkTSr7iRpzcfIAl6DDNpb912M5nWLNqSnmg=; b=f6zzQOTUaYV0dGDycpAOWNzCI/lR6jh2YepSV7POTPvPb7jMqEwPUiGx8tQHFh10dV SNP534Afwa60VrPCtlXCdpn2hasBIZ5BR5IZMAtfMrjqEfNfHv6bwTbnAkUSnUU35Y3T JSnDqU1F8gLjnvgx0PvkcIW/7cWf0MN6U9i1XUFeqlfVu2Xy5mg6BsTPZzfILpblDSn8 s6eCBb7SxKcnaEY+ju3UGv3SH1HxYxDfheTd+CW6BK1HPyWc0D74ool0OXtg4Xnstjo6 1paP46s46Fm5Qmv+gIoQsLTwLuuKVzTNd9RpdutEBMMM7OB3YcVE81kDJuUEinyNEwK5 5MsA== X-Forwarded-Encrypted: i=1; AFNElJ+2MYrq4pPOUH2zbiFbK87U6vLRNvZkbmbi5ibylQN0lOmolm+dPLPCENOROT13Xg/ePR8kvPfxQf4gT/c=@vger.kernel.org X-Gm-Message-State: AOJu0Yw7ahDsX5ueL8G5u8Fr2s1R//DsLj8gb6rY8DqmaZB1aycjnNun /SLdWszETYdDC2GkzBucfjhiwzAxHCx0k2y0lvRDe/lCHjOPvlgfvQWTdSDnlfS8tFgsjTURnsa TkpOqjG1lmddZvgBLS1+Pw8fMSg== X-Received: from plbkg3.prod.google.com ([2002:a17:903:603:b0:2b4:5fd0:a45d]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:3583:b0:2b0:59c4:e9dc with SMTP id d9443c01a7336-2b987407cfamr4803655ad.22.1777401447910; Tue, 28 Apr 2026 11:37:27 -0700 (PDT) Date: Tue, 28 Apr 2026 18:36:42 +0000 In-Reply-To: <20260428183643.3796063-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260428183643.3796063-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260428183643.3796063-8-dylanbhatch@google.com> Subject: [PATCH v5 7/8] sframe: Introduce in-kernel SFRAME_VALIDATION From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Generalize the __safe* helpers to support a non-user-access code path. This requires arch-specific function address validation. This is because arm64 vmlinux keeps .exit.text (normally discarded), and .rodata.text sections both of which lie outside the bounds of the normal .text. .rodata.text contains code that is never executed by the kernel mapping, but for which the toolchain nonetheless generates sframe data, and needs to be considered valid for a PC lookup. Additionally .init.text lies outside .text for all arches and must be accounted for as well. Suggested-by: Jens Remus Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- arch/Kconfig | 2 +- arch/arm64/include/asm/sections.h | 1 + arch/arm64/include/asm/unwind_sframe.h | 47 ++++++++++++++++++++++++++ arch/arm64/kernel/vmlinux.lds.S | 2 ++ include/linux/sframe.h | 2 ++ kernel/unwind/sframe.c | 25 ++++++++++++-- 6 files changed, 76 insertions(+), 3 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 8d27b3249e7a..a528f5b23647 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -503,7 +503,7 @@ config HAVE_UNWIND_USER_SFRAME =20 config SFRAME_VALIDATION bool "Enable .sframe section debugging" - depends on HAVE_UNWIND_USER_SFRAME + depends on UNWIND_SFRAME_LOOKUP depends on DYNAMIC_DEBUG help When adding an .sframe section for a task, validate the entire diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sec= tions.h index 51b0d594239e..5edb4304f661 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -23,6 +23,7 @@ extern char __irqentry_text_start[], __irqentry_text_end[= ]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; +extern char _srodatatext[], _erodatatext[]; =20 static inline size_t entry_tramp_text_size(void) { diff --git a/arch/arm64/include/asm/unwind_sframe.h b/arch/arm64/include/as= m/unwind_sframe.h index 876412881196..66ebe5f38bd0 100644 --- a/arch/arm64/include/asm/unwind_sframe.h +++ b/arch/arm64/include/asm/unwind_sframe.h @@ -2,7 +2,54 @@ #ifndef _ASM_ARM64_UNWIND_SFRAME_H #define _ASM_ARM64_UNWIND_SFRAME_H =20 +#include +#include +#include + #define SFRAME_REG_SP 31 #define SFRAME_REG_FP 29 =20 +static inline bool sframe_func_start_addr_valid(struct sframe_section *sec, + unsigned long func_addr) +{ + /* Common case for unwinding */ + if (sec->text_start <=3D func_addr && func_addr < sec->text_end) + return true; + + if (sec->sec_type !=3D SFRAME_KERNEL) + return false; + + /* + * Account for vmlinux and module code outside the normal .text section. + * The toolchain still generates sframe data for these functions, so + * sframe lookups on them should be allowed. + */ + if (sec =3D=3D &kernel_sfsec) { + if (is_kernel_inittext(func_addr)) + return true; + + /* .exit.text is retained in vmlinux on arm64. */ + if (func_addr >=3D (unsigned long)__exittext_begin && + func_addr < (unsigned long)__exittext_end) + return true; + + + /* + * .rodata.text is never executed from the kernel mapping, but + * still has sframe data + */ + if (func_addr >=3D (unsigned long)_srodatatext && + func_addr < (unsigned long)_erodatatext) + return true; + } else { + struct module *mod =3D container_of(sec, struct module, + arch.sframe_sec); + if (within_module_mem_type(func_addr, mod, MOD_INIT_TEXT)) + return true; + } + + return false; +} +#define sframe_func_start_addr_valid sframe_func_start_addr_valid + #endif /* _ASM_ARM64_UNWIND_SFRAME_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index 2964aad0362e..8c2dae6e7a86 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -213,12 +213,14 @@ SECTIONS =20 /* code sections that are never executed via the kernel mapping */ .rodata.text : { + _srodatatext =3D .; TRAMP_TEXT HIBERNATE_TEXT KEXEC_TEXT IDMAP_TEXT . =3D ALIGN(PAGE_SIZE); } + _erodatatext =3D .; =20 idmap_pg_dir =3D .; . +=3D PAGE_SIZE; diff --git a/include/linux/sframe.h b/include/linux/sframe.h index 27f5a66190af..ac3aa9db7d91 100644 --- a/include/linux/sframe.h +++ b/include/linux/sframe.h @@ -34,6 +34,8 @@ struct sframe_section { signed char fp_off; }; =20 +extern struct sframe_section kernel_sfsec __ro_after_init; + #endif /* CONFIG_UNWIND_SFRAME_LOOKUP */ =20 #ifdef CONFIG_HAVE_UNWIND_USER_SFRAME diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c index 70001c8e586d..99c2a39c51ce 100644 --- a/kernel/unwind/sframe.c +++ b/kernel/unwind/sframe.c @@ -21,10 +21,18 @@ #include "sframe.h" #include "sframe_debug.h" =20 +#ifndef sframe_func_start_addr_valid +static inline bool sframe_func_start_addr_valid(struct sframe_section *sec, + unsigned long func_addr) +{ + return (sec->text_start <=3D func_addr && func_addr < sec->text_end); +} +#endif + #ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME =20 static bool sframe_init __ro_after_init; -static struct sframe_section kernel_sfsec __ro_after_init; +struct sframe_section kernel_sfsec __ro_after_init; =20 #endif /* CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ =20 @@ -152,7 +160,7 @@ static __always_inline int __read_fde(struct sframe_sec= tion *sec, sizeof(struct sframe_fde_v3), Efault); =20 func_addr =3D fde_addr + _fde.func_start_off; - if (func_addr < sec->text_start || func_addr > sec->text_end) + if (!sframe_func_start_addr_valid(sec, func_addr)) return -EINVAL; =20 fda_addr =3D sec->fres_start + _fde.fres_off; @@ -636,6 +644,9 @@ static int safe_read_fde(struct sframe_section *sec, { int ret; =20 + if (sec->sec_type =3D=3D SFRAME_KERNEL) + return __read_fde(sec, fde_num, fde); + if (!user_read_access_begin((void __user *)sec->sframe_start, sec->sframe_end - sec->sframe_start)) return -EFAULT; @@ -651,6 +662,9 @@ static int safe_read_fre(struct sframe_section *sec, { int ret; =20 + if (sec->sec_type =3D=3D SFRAME_KERNEL) + return __read_fre(sec, fde, fre_addr, fre); + if (!user_read_access_begin((void __user *)sec->sframe_start, sec->sframe_end - sec->sframe_start)) return -EFAULT; @@ -665,6 +679,9 @@ static int safe_read_fre_datawords(struct sframe_sectio= n *sec, { int ret; =20 + if (sec->sec_type =3D=3D SFRAME_KERNEL) + return __read_fre_datawords(sec, fde, fre); + if (!user_read_access_begin((void __user *)sec->sframe_start, sec->sframe_end - sec->sframe_start)) return -EFAULT; @@ -1013,6 +1030,8 @@ void __init init_sframe_table(void) =20 if (WARN_ON(sframe_read_header(&kernel_sfsec))) return; + if (WARN_ON(sframe_validate_section(&kernel_sfsec))) + return; =20 sframe_init =3D true; } @@ -1031,6 +1050,8 @@ void sframe_module_init(struct module *mod, void *sfr= ame, size_t sframe_size, =20 if (WARN_ON(sframe_read_header(&sec))) return; + if (WARN_ON(sframe_validate_section(&sec))) + return; =20 mod->arch.sframe_sec =3D sec; mod->arch.sframe_init =3D true; --=20 2.54.0.545.g6539524ca2-goog From nobody Sat May 9 07:32:00 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A18647B409 for ; Tue, 28 Apr 2026 18:37:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401455; cv=none; b=sttxumjAN27M1TmB98npWxRttGk/eheBzsNrBK8/cz0GuY8Iw7BljtikFDReQ2qRg493N2zWJcAsuVKbMZ3yfny33TkykfMdCvKiSJS6uebO7vv6D7kI0nDV+pLgUEJ7GPcq4UOPg6FzImrtb3DVS1jGyyNI06IhRYTVCZO0lDY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777401455; c=relaxed/simple; bh=eUcBXFOTvR8ZdL6IqA6TLXYsvIHF4JVdur74bowB8GA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QLrh6imcPolMirh6qHHfl3BQwJox1wgxIpq+cst/qklPK20A6Gh7pIHcKCLGZpzqB1tAqiuLVtVK9T/xQ9AAB88e21Ygn6PH4CkJLriSmlwYlgTL/aHBQkWmp5EzRHClBK+IrbJBItSgJvUWzWxEq1jFwVPUD8mT0y/Url1xm8U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=m9oYEckb; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="m9oYEckb" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82f6b984b3aso6591519b3a.3 for ; Tue, 28 Apr 2026 11:37:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777401451; x=1778006251; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Cq4/UBTJbBSIBMYrEdgu8vO+sExHQ1BWSN/SoA97xbg=; b=m9oYEckb9cPaumnJOoFOqpscQsTNLt+5LVP8BWhPv9Kt5BhLc0ut6J1MqfCS0uJVx8 sHQxxfrSD/3fjGZwlMOlfc0E0iJppqRA3mG84leUm5Bwx3znvYLuww8l5yfa43N6DuyU PiEt+t/xZWiA1tDRnqTGQ6XTuX2elRtrYfPI6uEvNJE7XzS2nk0mLiWjhUmrw+fFBBZ3 59Bdycu8jnuJiv5bID2z6Zny1Wo6gwQLwBMMgCDjaXhQVYMMTyQkt2flYMpYE1ycPE30 ysHAaM3BOJGZ4Ppx7vjJr3h+SJVpG/PN+59hNvJ6aZRD/QqNL1s+vIMyc8dl8EXMw7OG QdVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777401451; x=1778006251; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Cq4/UBTJbBSIBMYrEdgu8vO+sExHQ1BWSN/SoA97xbg=; b=jq4LePBTRK5/VkkdJnIEjBQjJ+DBolNvC6vwk1hwz8bTtPDIPp6CX49TYoPFAZpMgt MIrYsghTKt7CLkbZw7scF/lJkwo5DOUeqx5lgEE1Ui6pTmbgo9S6xk3mCcStjbh1DGgz +lKnicW7zo/ymHNN0HYelDBgnVRnWQ0/6wg/bHnq/h1Icos7JlHNR6zBiYF0Ir2+Xoaw YFI9HtUn/kZzTekI6wl0+1cngRPgaTt2LFuP/MQmqrgt7Ck/HhqmMZUzUq45Yh1Cmlko 3g3qJbEEAxCqy/icrMr/b5dNaNHM4OxcxxotUK0OoYeiN1ZbNYgjD+NO60tpFMFFl80Y 0YPQ== X-Forwarded-Encrypted: i=1; AFNElJ8eAzJ5YWbnYmgaKl6d9KkULezSo2W8xb3duZnaPGBhfqQAoRpfyVUPaJYCf8E0GKe7O+ONy+kAnbMSU1M=@vger.kernel.org X-Gm-Message-State: AOJu0Yz8DO5FMfAG39ejCZPS1N7SUll0H2d91PnS83wxNaJTl19TlUg5 dJUeAortKRWAWXrIpJAEGWQYSR3u4EjXBx99PxmcFltvh4mBAgvTU/Xd3EpQzMPdbciGjzSAkMs EU0EVf/kdPYRr1SpUWGlpIB+Iog== X-Received: from pfbif1.prod.google.com ([2002:a05:6a00:8b01:b0:82f:a396:2229]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:4286:b0:82f:7182:693d with SMTP id d2e1a72fcca58-834ddbefcc3mr4672076b3a.30.1777401450692; Tue, 28 Apr 2026 11:37:30 -0700 (PDT) Date: Tue, 28 Apr 2026 18:36:43 +0000 In-Reply-To: <20260428183643.3796063-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260428183643.3796063-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260428183643.3796063-9-dylanbhatch@google.com> Subject: [PATCH v5 8/8] unwind: arm64: Use sframe to unwind interrupt frames From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Jens Remus Cc: Dylan Hatch , Mark Rutland , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add unwind_next_frame_sframe() function to unwind by sframe info if present. Use this method at exception boundaries, falling back to frame-pointer unwind only on failure. In such failure cases, the stacktrace is considered unreliable. During normal unwind, prefer frame pointer unwind (for better performance) with sframe as a backup. This change restores the LR behavior originally introduced in commit c2c6b27b5aa14fa2 ("arm64: stacktrace: unwind exception boundaries"), But later removed in commit 32ed1205682e ("arm64: stacktrace: Skip reporting LR at exception boundaries") This can be done because the sframe data can be used to determine whether the LR is current for the PC value recovered from pt_regs at the exception boundary. Signed-off-by: Weinan Liu Reviewed-by: Prasanna Kumar T S M Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- arch/arm64/include/asm/stacktrace/common.h | 6 + arch/arm64/kernel/stacktrace.c | 246 +++++++++++++++++++-- 2 files changed, 232 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace/common.h b/arch/arm64/includ= e/asm/stacktrace/common.h index 821a8fdd31af..4df68181e1b5 100644 --- a/arch/arm64/include/asm/stacktrace/common.h +++ b/arch/arm64/include/asm/stacktrace/common.h @@ -21,6 +21,8 @@ struct stack_info { * * @fp: The fp value in the frame record (or the real fp) * @pc: The lr value in the frame record (or the real lr) + * @sp: The sp value at the call site of the current function. + * @unreliable: Stacktrace is unreliable. * * @stack: The stack currently being unwound. * @stacks: An array of stacks which can be unwound. @@ -29,7 +31,11 @@ struct stack_info { struct unwind_state { unsigned long fp; unsigned long pc; +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + unsigned long sp; +#endif =20 + bool unreliable; struct stack_info stack; struct stack_info *stacks; int nr_stacks; diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 3ebcf8c53fb0..c935323f393b 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -14,6 +14,7 @@ #include #include #include +#include =20 #include #include @@ -26,6 +27,7 @@ enum kunwind_source { KUNWIND_SOURCE_CALLER, KUNWIND_SOURCE_TASK, KUNWIND_SOURCE_REGS_PC, + KUNWIND_SOURCE_REGS_LR, }; =20 union unwind_flags { @@ -85,6 +87,9 @@ kunwind_init_from_regs(struct kunwind_state *state, state->regs =3D regs; state->common.fp =3D regs->regs[29]; state->common.pc =3D regs->pc; +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + state->common.sp =3D regs->sp; +#endif state->source =3D KUNWIND_SOURCE_REGS_PC; } =20 @@ -103,6 +108,9 @@ kunwind_init_from_caller(struct kunwind_state *state) =20 state->common.fp =3D (unsigned long)__builtin_frame_address(1); state->common.pc =3D (unsigned long)__builtin_return_address(0); +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + state->common.sp =3D (unsigned long)__builtin_frame_address(0); +#endif state->source =3D KUNWIND_SOURCE_CALLER; } =20 @@ -124,6 +132,9 @@ kunwind_init_from_task(struct kunwind_state *state, =20 state->common.fp =3D thread_saved_fp(task); state->common.pc =3D thread_saved_pc(task); +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + state->common.sp =3D thread_saved_sp(task); +#endif state->source =3D KUNWIND_SOURCE_TASK; } =20 @@ -181,7 +192,6 @@ int kunwind_next_regs_pc(struct kunwind_state *state) state->regs =3D regs; state->common.pc =3D regs->pc; state->common.fp =3D regs->regs[29]; - state->regs =3D NULL; state->source =3D KUNWIND_SOURCE_REGS_PC; return 0; } @@ -237,6 +247,9 @@ kunwind_next_frame_record(struct kunwind_state *state) =20 unwind_consume_stack(&state->common, info, fp, sizeof(*record)); =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + state->common.sp =3D state->common.fp; +#endif state->common.fp =3D new_fp; state->common.pc =3D new_pc; state->source =3D KUNWIND_SOURCE_FRAME; @@ -244,6 +257,176 @@ kunwind_next_frame_record(struct kunwind_state *state) return 0; } =20 +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +static __always_inline struct stack_info * +get_word(struct unwind_state *state, unsigned long *word) +{ + unsigned long addr =3D *word; + struct stack_info *info; + + info =3D unwind_find_stack(state, addr, sizeof(addr)); + if (!info) + return info; + + *word =3D READ_ONCE(*(unsigned long *)addr); + + return info; +} + +static __always_inline int +get_consume_word(struct unwind_state *state, unsigned long *word) +{ + struct stack_info *info; + unsigned long addr =3D *word; + + info =3D get_word(state, word); + if (!info) + return -EINVAL; + + unwind_consume_stack(state, info, addr, sizeof(addr)); + return 0; +} + +/* + * Unwind to the next frame according to sframe. + */ +static __always_inline int +unwind_next_frame_sframe(struct kunwind_state *state) +{ + struct unwind_frame frame; + unsigned long cfa, fp, ra; + enum kunwind_source source =3D KUNWIND_SOURCE_FRAME; + struct pt_regs *regs =3D state->regs; + + int err; + + /* FP/SP alignment 8 bytes */ + if (state->common.fp & 0x7 || state->common.sp & 0x7) + return -EINVAL; + + /* + * Most/all outermost functions are not visible to sframe. So, check for + * a meta frame record if the sframe lookup fails. + */ + err =3D sframe_find_kernel(state->common.pc, &frame); + if (err) + return kunwind_next_frame_record_meta(state); + + if (frame.outermost) + return -ENOENT; + + /* Get the Canonical Frame Address (CFA) */ + switch (frame.cfa.rule) { + case UNWIND_CFA_RULE_SP_OFFSET: + cfa =3D state->common.sp; + break; + case UNWIND_CFA_RULE_FP_OFFSET: + if (state->common.fp < state->common.sp) + return -EINVAL; + cfa =3D state->common.fp; + break; + case UNWIND_CFA_RULE_REG_OFFSET: + case UNWIND_CFA_RULE_REG_OFFSET_DEREF: + /* regs only available in topmost/interrupt frame */ + if (!regs || frame.cfa.regnum > 30) + return -EINVAL; + cfa =3D regs->regs[frame.cfa.regnum]; + break; + default: + WARN_ON_ONCE(1); + return -EINVAL; + } + cfa +=3D frame.cfa.offset; + + /* + * CFA typically points to a higher address than RA or FP, so don't + * consume from the stack when we read it. + */ + if (frame.cfa.rule & UNWIND_RULE_DEREF && + !get_word(&state->common, &cfa)) + return -EINVAL; + + /* CFA alignment 8 bytes */ + if (cfa & 0x7) + return -EINVAL; + + /* Get the Return Address (RA) */ + switch (frame.ra.rule) { + case UNWIND_RULE_RETAIN: + /* regs only available in topmost/interrupt frame */ + if (!regs) + return -EINVAL; + ra =3D regs->regs[30]; + source =3D KUNWIND_SOURCE_REGS_LR; + break; + /* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */ + case UNWIND_RULE_CFA_OFFSET_DEREF: + ra =3D cfa + frame.ra.offset; + break; + case UNWIND_RULE_REG_OFFSET: + case UNWIND_RULE_REG_OFFSET_DEREF: + /* regs only available in topmost/interrupt frame */ + if (!regs) + return -EINVAL; + ra =3D regs->regs[frame.cfa.regnum]; + ra +=3D frame.ra.offset; + break; + default: + WARN_ON_ONCE(1); + return -EINVAL; + } + + /* Get the Frame Pointer (FP) */ + switch (frame.fp.rule) { + case UNWIND_RULE_RETAIN: + fp =3D state->common.fp; + break; + /* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */ + case UNWIND_RULE_CFA_OFFSET_DEREF: + fp =3D cfa + frame.fp.offset; + break; + case UNWIND_RULE_REG_OFFSET: + case UNWIND_RULE_REG_OFFSET_DEREF: + /* regs only available in topmost/interrupt frame */ + if (!regs) + return -EINVAL; + fp =3D regs->regs[frame.fp.regnum]; + fp +=3D frame.fp.offset; + break; + default: + WARN_ON_ONCE(1); + return -EINVAL; + } + + /* + * Consume RA and FP from the stack. The frame record puts FP at a lower + * address than RA, so we always read FP first. + */ + if (frame.fp.rule & UNWIND_RULE_DEREF && + !get_word(&state->common, &fp)) + return -EINVAL; + + if (frame.ra.rule & UNWIND_RULE_DEREF && + get_consume_word(&state->common, &ra)) + return -EINVAL; + + state->common.pc =3D ra; + state->common.sp =3D cfa; + state->common.fp =3D fp; + + state->source =3D source; + + return 0; +} + +#else /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + +static __always_inline int +unwind_next_frame_sframe(struct kunwind_state *state) { return -EINVAL; } + +#endif /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME*/ + /* * Unwind from one frame record (A) to the next frame record (B). * @@ -259,12 +442,25 @@ kunwind_next(struct kunwind_state *state) state->flags.all =3D 0; =20 switch (state->source) { + case KUNWIND_SOURCE_REGS_PC: + err =3D unwind_next_frame_sframe(state); + + if (err && err !=3D -ENOENT) { + /* Fallback to FP based unwinder */ + err =3D kunwind_next_frame_record(state); + state->common.unreliable =3D true; + } + state->regs =3D NULL; + break; case KUNWIND_SOURCE_FRAME: case KUNWIND_SOURCE_CALLER: case KUNWIND_SOURCE_TASK: - case KUNWIND_SOURCE_REGS_PC: + case KUNWIND_SOURCE_REGS_LR: err =3D kunwind_next_frame_record(state); + if (err && err !=3D -ENOENT) + err =3D unwind_next_frame_sframe(state); break; + default: err =3D -EINVAL; } @@ -350,6 +546,9 @@ kunwind_stack_walk(kunwind_consume_fn consume_state, .common =3D { .stacks =3D stacks, .nr_stacks =3D ARRAY_SIZE(stacks), +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + .sp =3D 0, +#endif }, }; =20 @@ -390,34 +589,40 @@ noinline noinstr void arch_stack_walk(stack_trace_con= sume_fn consume_entry, kunwind_stack_walk(arch_kunwind_consume_entry, &data, task, regs); } =20 +struct kunwind_reliable_consume_entry_data { + stack_trace_consume_fn consume_entry; + void *cookie; + bool unreliable; +}; + static __always_inline bool -arch_reliable_kunwind_consume_entry(const struct kunwind_state *state, voi= d *cookie) +arch_kunwind_reliable_consume_entry(const struct kunwind_state *state, voi= d *cookie) { - /* - * At an exception boundary we can reliably consume the saved PC. We do - * not know whether the LR was live when the exception was taken, and - * so we cannot perform the next unwind step reliably. - * - * All that matters is whether the *entire* unwind is reliable, so give - * up as soon as we hit an exception boundary. - */ - if (state->source =3D=3D KUNWIND_SOURCE_REGS_PC) - return false; + struct kunwind_reliable_consume_entry_data *data =3D cookie; =20 - return arch_kunwind_consume_entry(state, cookie); + if (state->common.unreliable) { + data->unreliable =3D true; + return false; + } + return data->consume_entry(data->cookie, state->common.pc); } =20 -noinline noinstr int arch_stack_walk_reliable(stack_trace_consume_fn consu= me_entry, - void *cookie, - struct task_struct *task) +noinline notrace int arch_stack_walk_reliable( + stack_trace_consume_fn consume_entry, + void *cookie, struct task_struct *task) { - struct kunwind_consume_entry_data data =3D { + struct kunwind_reliable_consume_entry_data data =3D { .consume_entry =3D consume_entry, .cookie =3D cookie, + .unreliable =3D false, }; =20 - return kunwind_stack_walk(arch_reliable_kunwind_consume_entry, &data, - task, NULL); + kunwind_stack_walk(arch_kunwind_reliable_consume_entry, &data, task, NULL= ); + + if (data.unreliable) + return -EINVAL; + + return 0; } =20 struct bpf_unwind_consume_entry_data { @@ -452,6 +657,7 @@ static const char *state_source_string(const struct kun= wind_state *state) case KUNWIND_SOURCE_CALLER: return "C"; case KUNWIND_SOURCE_TASK: return "T"; case KUNWIND_SOURCE_REGS_PC: return "P"; + case KUNWIND_SOURCE_REGS_LR: return "L"; default: return "U"; } } --=20 2.54.0.545.g6539524ca2-goog