From nobody Fri Oct 3 05:25:20 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5389A2EC569 for ; Thu, 4 Sep 2025 22:39:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025557; cv=none; b=ngblvgM5LqTpnTDxkrZvXnFUy5w1tD0DVq1Pd8pB24TZmHRFPhphi5Ob8f7Yrh3QhIOjc1Y7LXIK0U6sDB9R26o/Oym0PrOOWw+Vporu0Hwzb1UNBlzoWowTVG/n0FPReluWwRW1RLUnabQR879RxjCgDf29TLkeQNz+YBoNWac= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025557; c=relaxed/simple; bh=wpukPNGidBsZHZNBLMBXDHyt6toluvhOIdHF3dHictg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pROE07/YRkFEJk4N2v/ILj2GoCMolWC+D47odI6eoDq3Z11l6xTcUt2e0u9Em8PfyiNBYg5p2OIeL39gdA6j4441dNyUUb0528q5x6KMNiSrjnszXSv5e0mKXMtefVKVO+StJjbb4QzOQinaYm8p3dwFw625hMg7b+k/68nPWZ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hZQ3Onqm; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hZQ3Onqm" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-329ee88fb47so2390364a91.2 for ; Thu, 04 Sep 2025 15:39:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1757025555; x=1757630355; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dtLRBEibPgJC5DKIIcXUNvr0Jg7QinxVii68wLCzhSQ=; b=hZQ3Onqmi4lLIpKpNRqcRCVknF2dTIeIZLqF+mXwiP3lD9C86SXGo/qxgybR/CQaVd OVUj7FTSC24yLc0iOLBcYrGu2rojA8as0vFpUskKXoLOrtJDyMKm8Rsa9XGJtdwE8Y/Y Uji7rKIv+vz0qgRkmAnyGAuuFatagBCqlxJl4bEPvbTsLLSlrhHZTaA43PH33zbbMvpf /bo7il7PtHKgE5jG/VhNtUMkdEIjuVjBmInRfXaFCFMx00tTVme+ImfAU/o5ASy5gpF+ k9ZRCQWQ2HKNArie+CHDJgsJ308zZBN7RHFUYOuDuuLXie5MmR0JP4Q1vSvZPYAff84I AAGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757025555; x=1757630355; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dtLRBEibPgJC5DKIIcXUNvr0Jg7QinxVii68wLCzhSQ=; b=FuX6O5lNU5qRnOZr6PQtI3RdP8Gv0wp2sOrpXszyWVMX+sO1O077hYrDG55rq/kr62 k0ATA2n7NtGqjaTlplzD4YR9IkZARbNOoG/v4Q0F4LJxXXQLj5phnlB/RsN0QyybwXNL N3qjOZncS5ew2mSSlPbjPQvGJqOWhAS9AlSRcceS7K5zNRRhei8qXi3So+rJvKC5Dqp/ 0qX+mgIq1Mc30tIlu/Bh7d94oBojbLoHkCq91dcE0On24WBl5py8FnjSh+dhBeZt7s0w tbqsGCh0WdfaEdrqOdpwvZ1GzjqURZvVz9wabi6htBkqRHqJvZeawXm4d7/f+4SgH29E jLYg== X-Forwarded-Encrypted: i=1; AJvYcCV1N8j35NMX4xy8F4Dnw3T8sqekHkGX8E/2bkpaZUhkPEGq6flCdYyB4AVk3iM4x1SOwv7QD0aEdkoEGuU=@vger.kernel.org X-Gm-Message-State: AOJu0YzLbEPhafMbYsI5ky9l6ZkJd8zY0wuo/P96jW2DMdJD/LegSmLD 2WpLeY/T1HJGW8SBOPAqsfXKeD//WdFc0fnFmdN0MRJknkoL5K1DuGO28pczKfoVD2gIkF7RGWN UJfljSqo8p8VxHc2CI12KdUSVWQ== X-Google-Smtp-Source: AGHT+IGXLPNA47OzrNlwR+6Ds93TCeSbcveB/7SH+UNLQts5VBH9tcTynj2fUKqKY9GmrReRRyDIXA2CAA/pnv5LGQ== X-Received: from pjl11.prod.google.com ([2002:a17:90b:2f8b:b0:329:d09b:a3f2]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3905:b0:32b:8b8d:c2af with SMTP id 98e67ed59e1d1-32b8b8dc8e4mr6542460a91.9.1757025555612; Thu, 04 Sep 2025 15:39:15 -0700 (PDT) Date: Thu, 4 Sep 2025 22:38:45 +0000 In-Reply-To: <20250904223850.884188-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250904223850.884188-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.51.0.355.g5224444f11-goog Message-ID: <20250904223850.884188-2-dylanbhatch@google.com> Subject: [PATCH v2 1/6] unwind: build kernel with sframe info From: Dylan Hatch To: Josh Poimboeuf , Steven Rostedt , Indu Bhagat , Peter Zijlstra , Will Deacon , Catalin Marinas , Jiri Kosina Cc: Dylan Hatch , Roman Gushchin , Weinan Liu , Mark Rutland , Ian Rogers , linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, joe.lawrence@redhat.com, Puranjay Mohan , Song Liu , Prasanna Kumar T S M Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the -Wa,--gsframe flags to build the code, so GAS will generate a new .sframe section for the stack trace information. Currently, the sframe format only supports arm64 and x86_64 architectures. Add this configuration on arm64 to enable sframe unwinder in the future. Signed-off-by: Weinan Liu Signed-off-by: Dylan Hatch Reviewed-by: Prasanna Kumar T S M --- Makefile | 8 ++++++++ arch/Kconfig | 6 ++++++ arch/arm64/Kconfig.debug | 10 ++++++++++ arch/arm64/kernel/vdso/Makefile | 2 +- include/asm-generic/vmlinux.lds.h | 15 +++++++++++++++ 5 files changed, 40 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index b9c661913250..09972c71a3e8 100644 --- a/Makefile +++ b/Makefile @@ -1078,6 +1078,14 @@ endif # Ensure compilers do not transform certain loops into calls to wcslen() KBUILD_CFLAGS +=3D -fno-builtin-wcslen =20 +# build with sframe table +ifdef CONFIG_SFRAME_UNWIND_TABLE +CC_FLAGS_SFRAME :=3D -Wa,--gsframe +KBUILD_CFLAGS +=3D $(CC_FLAGS_SFRAME) +KBUILD_AFLAGS +=3D $(CC_FLAGS_SFRAME) +export CC_FLAGS_SFRAME +endif + # change __FILE__ to the relative path to the source directory ifdef building_out_of_srctree KBUILD_CPPFLAGS +=3D $(call cc-option,-fmacro-prefix-map=3D$(srcroot)/=3D) diff --git a/arch/Kconfig b/arch/Kconfig index d1b4ffd6e085..4362d2f49d91 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1782,4 +1782,10 @@ config ARCH_WANTS_PRE_LINK_VMLINUX config ARCH_HAS_CPU_ATTACK_VECTORS bool =20 +config AS_SFRAME + def_bool $(as-instr,.cfi_sections .sframe\n.cfi_startproc\n.cfi_endproc) + +config SFRAME_UNWIND_TABLE + bool + endmenu diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug index 265c4461031f..d64bf58457de 100644 --- a/arch/arm64/Kconfig.debug +++ b/arch/arm64/Kconfig.debug @@ -20,4 +20,14 @@ config ARM64_RELOC_TEST depends on m tristate "Relocation testing module" =20 +config SFRAME_UNWINDER + bool "Sframe unwinder" + depends on AS_SFRAME + depends on 64BIT + select SFRAME_UNWIND_TABLE + help + This option enables the sframe (Simple Frame) unwinder for unwinding + kernel stack traces. It uses unwind table that is directly generated + by toolchain based on DWARF CFI information. + source "drivers/hwtracing/coresight/Kconfig" diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makef= ile index 7dec05dd33b7..c60ef921956f 100644 --- a/arch/arm64/kernel/vdso/Makefile +++ b/arch/arm64/kernel/vdso/Makefile @@ -38,7 +38,7 @@ ccflags-y +=3D -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO CC_FLAGS_REMOVE_VDSO :=3D $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) \ $(RANDSTRUCT_CFLAGS) $(KSTACK_ERASE_CFLAGS) \ $(GCC_PLUGINS_CFLAGS) \ - $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) \ + $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) $(CC_FLAGS_SFRAME) \ -Wmissing-prototypes -Wmissing-declarations =20 CC_FLAGS_ADD_VDSO :=3D -O2 -mcmodel=3Dtiny -fasynchronous-unwind-tables diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinu= x.lds.h index ae2d2359b79e..4f486080e4fb 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -473,6 +473,8 @@ defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPELL= ER_CLANG) *(.rodata1) \ } \ \ + SFRAME \ + \ /* PCI quirks */ \ .pci_fixup : AT(ADDR(.pci_fixup) - LOAD_OFFSET) { \ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_early, _pci_fixups_early, __start= , __end) \ @@ -891,6 +893,19 @@ defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPEL= LER_CLANG) #define TRACEDATA #endif =20 +#ifdef CONFIG_SFRAME_UNWIND_TABLE +#define SFRAME \ + /* sframe */ \ + .sframe : AT(ADDR(.sframe) - LOAD_OFFSET) { \ + __start_sframe_header =3D .; \ + KEEP(*(.sframe)) \ + KEEP(*(.init.sframe)) \ + __stop_sframe_header =3D .; \ + } +#else +#define SFRAME +#endif + #ifdef CONFIG_PRINTK_INDEX #define PRINTK_INDEX \ .printk_index : AT(ADDR(.printk_index) - LOAD_OFFSET) { \ --=20 2.51.0.355.g5224444f11-goog From nobody Fri Oct 3 05:25:20 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEBFB2ECD11 for ; Thu, 4 Sep 2025 22:39:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025560; cv=none; b=IOkrvYym1lTL3YF9e4vPbm+mlGIvFQbln/X8NiQYIkp/Ohu3WriQBzH4vYo0CwT3hVMTuUZJtey5ZSk5zRkcH8DagMLseTCv41VLO+s5CSPi/cNdxLcT5ROFxJxLKHElTGs0jTh0Fhaj2F6qz0wDp9M+Biy+RjduqkU7H5U6ujU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025560; c=relaxed/simple; bh=xnlkf0BdWeK6D1DOdzra4zgNbxjdYbPxVOi1fszhAWM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=n2R34RAj9v8anH3MLHVZmx/Q9di85DCTu7+onqQXHO9bZbHXED6v7MGTOsmZFR9EcThlttozIWmA9m/5hwEAAUw+lnbSVyx7+SF06c/Fx7nA+0Fd6PEUqlJLvRFQq3bNCUAhjKnC32VLFTelU0ESO0Mp/cr3CqjjxGruBzIW19o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LNCqNpGR; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LNCqNpGR" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-3278bb34a68so1288729a91.0 for ; Thu, 04 Sep 2025 15:39:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1757025558; x=1757630358; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=EwoyDNtF8aplk2UKsRnTrhjyV0ajLd9EcSIYMNuhcao=; b=LNCqNpGRwGf1vbqhjoJkFtCERBxk7xFzYWKOlaegL/Sde4SngB3unOFLsdSq8V2pCs KQLFD8/pSKDP4hyo3otwvTK45vN2cWEvqDJAxx00JW9fcAga2vpR8XUGYn3Nyt6FBKyi FVORTIzYR9oGz5MzKDubORIRugTrQO2eEvkftZcRuftjT2uqOYBaKwyzFy6nWPntnnkz 4RNJrbYfOJiEX8x1B3hJJLoK/Rtel2ZF1acWgdmib92UW2J3LAsCJI0hZOHB9MLFxyo5 +6tWuQ0SgO80aE1hSxZRakoP6z4KOPBrfsZiURVcAWfTMua7RmP7pecGJLl/ripwtHD5 Z34Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757025558; x=1757630358; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=EwoyDNtF8aplk2UKsRnTrhjyV0ajLd9EcSIYMNuhcao=; b=jBA8ksGfLsZnWWhaJu1wlKTqd0f4JCD1QPrq2iycUArb3+aNl5g2/fb+xwV+QQF7Bz k8k4Klwg0W1sJVxZDpPbsnzltTPj0F+UQIURb2K8wCCfE+oLzMywiVH1HDvuZ9xL1tfz 7bqQvAV16R3FqJPfc5D51cq60/kNHMkxSYUCeVq+fx4wVi/yvPTn6YQJRJgxdE72biyw UcPHUhmLRzaWMC77qT0REA/KqdDg+CgL8NmAw1RzKzCHzhhBxwoFfdN1LwSc/fpQ8J1X i9fykd+SD3blyCmZfr1Wl3Odcw9jnlGuWehQ8+LOYK1nDx44mr14XmaTg/uLcLWTWNE0 ZTNQ== X-Forwarded-Encrypted: i=1; AJvYcCU3uawjzH9TFHO1w4LSYpTnOeUwP4g08xdqia8nmIroEaBKX9XKNoAXpnZDWhiwFMvx07+Lb1rqIgCQHlA=@vger.kernel.org X-Gm-Message-State: AOJu0Yyn7pCpFPSTGXiodbW/kyu1UFCNHnxw/12rXCTkB5sgQ5JVA6Gk TEgGGgglMILfTaQB1soels7Uf4Cvg4CJLc1PABHMGA4APq0k9tJOFPxF/jmkXKhsUHk24DOkpRo BUi3rr4dOOBE0C48YRN/NzSPZjw== X-Google-Smtp-Source: AGHT+IG7LIyWvYU3YC2gAf7gR5JcEes7dNZngeNd+zYZ0BktekgBZQfRfAdsbGLuA2OZMC4Xg5mySNnpvS3fkIoA9Q== X-Received: from pjyd12.prod.google.com ([2002:a17:90a:dfcc:b0:31f:b2f:aeed]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2802:b0:32b:9506:1773 with SMTP id 98e67ed59e1d1-32b950618b8mr5636335a91.33.1757025558182; Thu, 04 Sep 2025 15:39:18 -0700 (PDT) Date: Thu, 4 Sep 2025 22:38:46 +0000 In-Reply-To: <20250904223850.884188-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250904223850.884188-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.51.0.355.g5224444f11-goog Message-ID: <20250904223850.884188-3-dylanbhatch@google.com> Subject: [PATCH v2 2/6] arm64: entry: add unwind info for various kernel entries From: Dylan Hatch To: Josh Poimboeuf , Steven Rostedt , Indu Bhagat , Peter Zijlstra , Will Deacon , Catalin Marinas , Jiri Kosina Cc: Dylan Hatch , Roman Gushchin , Weinan Liu , Mark Rutland , Ian Rogers , linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, joe.lawrence@redhat.com, Puranjay Mohan , Song Liu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Weinan Liu DWARF CFI (Call Frame Information) specifies how to recover the return address and callee-saved registers at each PC in a given function. Compilers are able to generate the CFI annotations when they compile the code to assembly language. For handcrafted assembly, we need to annotate them by hand. Annotate CFI unwind info for assembly for interrupt and exception handlers. Signed-off-by: Weinan Liu Signed-off-by: Dylan Hatch --- arch/arm64/kernel/entry.S | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index f8018b5c1f9a..3148ede8c2c6 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -575,7 +575,12 @@ SYM_CODE_START_LOCAL(el\el\ht\()_\regsize\()_\label) .if \el =3D=3D 0 b ret_to_user .else + .cfi_startproc + .cfi_def_cfa_offset PT_REGS_SIZE + .cfi_offset 29, S_FP - PT_REGS_SIZE + .cfi_offset 30, S_LR - PT_REGS_SIZE b ret_to_kernel + .cfi_endproc .endif SYM_CODE_END(el\el\ht\()_\regsize\()_\label) .endm @@ -889,6 +894,10 @@ SYM_FUNC_START(call_on_irq_stack) add sp, x16, #IRQ_STACK_SIZE restore_irq x9 blr x1 + .cfi_startproc + .cfi_def_cfa 29, 16 + .cfi_offset 29, -16 + .cfi_offset 30, -8 =20 save_and_disable_daif x9 /* @@ -900,6 +909,7 @@ SYM_FUNC_START(call_on_irq_stack) scs_load_current restore_irq x9 ret + .cfi_endproc SYM_FUNC_END(call_on_irq_stack) NOKPROBE(call_on_irq_stack) =20 --=20 2.51.0.355.g5224444f11-goog From nobody Fri Oct 3 05:25:20 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C12BE2ED87C for ; Thu, 4 Sep 2025 22:39:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025563; cv=none; b=Bk6MvnCmQIEy4TwE/wZxZacbHYyvR4L7MFCnMWpavnpE9m+chOU+Bn3+F9r5wSC+xt+kXQfeZMBorgZdTReX0WnV4snEJ7rs5fL22oZQasrLy6+ZD7FJvfn5SgayXLqP0dJbvrT7e6sydeOsAEQAkDLMMEpWfQ+O0gvvWZSQYNU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025563; c=relaxed/simple; bh=NovLOGBdyArFTxtU+57EVnKh/sjgYEL5oQ0KRAhMNk8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ohDvH8lakgLc4vZduj7Jnx6ttxMZJZ8YnNQllo7w0kSqgfoZdTsQ0E6lvHx74HRBXv3hIesrB7W3pYhzKb9AlcUtbEy75YfdiqMgrbek21rvJiH699x8w2gEhQLYva+AG3J5ZGvGLi8e3D/7VoA0v2e9k2oVl+lqa6FlUi8XQ/o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3+mP/jxQ; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3+mP/jxQ" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-32bbe7c5b87so285172a91.3 for ; Thu, 04 Sep 2025 15:39:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1757025561; x=1757630361; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2m08r/ZQjtbTIB1gTP1Sld7JP5u72ORtVv9DJEb0tSQ=; b=3+mP/jxQ0ziCoP9Ai1chQHA6UMMd5teHT6Uqo12P1qQPcKbx2JHSORrMDmISmM3D27 7I7ar/02FXtPV7gLQyH6ADtcxco6Rgqocku51fmsRr0H+FLYVUl+KZdU2DkCL4zJc7KS MBhW9NAPi7leJl5fTywmfDSRqA/2EzcLUIyKQX/jxbVMjeAUNh05F3dcilyXm56hwjZv /5r1ewuEe95hrHBG6Fa5WaFKzBAVyrV7C69wFVmtPXvBDq+UJI2M7FBKKCXDngqueWMk 2manxpxiOtbqkdy3Hb+enQvA3oVp29cnEfONL1Ni26m88HzlOvF4HDHtq4cTqByYRNTg 8WAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757025561; x=1757630361; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2m08r/ZQjtbTIB1gTP1Sld7JP5u72ORtVv9DJEb0tSQ=; b=LoHd9arN2TzId5l3uvfhnHwduP9J50c8CTCHUEXPlJtM/TrTMsK5aLrpMoQkSsVNxk vsgdkhcByeGc4QKn0QBF8PPRgoUHp4VbuFrfPUSgwafqhi0N5VRFySmxOw6hK1NdCyVf +vvyqW2SeixUPvZGEwHnrsicejOJG7uebg5qPYPcmXcUUoAtRk1MOd7f5CJK/j0AbAG1 UhvFtnJDhLUXjLwPyGuJNQiAeTKCdkf2SdyGe5p8bf8GW7Q2kiNqUfRKK/XqoKPcpggm uaHvgRrRR4VXKf6wOaQ9XKigfyV/HmNXBPPomkh5KIEf0zLC5qRUmfhHBb0oQsz70ZzO sLPg== X-Forwarded-Encrypted: i=1; AJvYcCVeCyadu8jDnvfz1DYHJSMEv7T3bZVgY44/lA0F9oFJ0+xm1Ar7siTYn4BEtFPAm4tqplcCxCfPD3h+uY4=@vger.kernel.org X-Gm-Message-State: AOJu0YzMXqBr/LSbudItSRhmFcfVurwKGlYBKzg1xeWUvBi/TxhEExCf Hpfz9KbTsBaYfaYP6nGTRWf3ouyjHImDk4LsgwnQto8f9wllHoBLkWTarJBcKhI9fVArw8iM7Er dDbsoKJ31zSsv0R6BfiyEUL0Ptw== X-Google-Smtp-Source: AGHT+IEydTUhEuAQ3A1H8zKyclA89jCNB3VUkBozS1oBCNaZG2tV6zRAiEaoyka9mctrU/XAtHk0Qslbb13/nP1pWg== X-Received: from pjbsx14.prod.google.com ([2002:a17:90b:2cce:b0:32b:682f:e6ca]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1d03:b0:320:fda8:fabe with SMTP id 98e67ed59e1d1-328156ba3b0mr23030730a91.22.1757025561033; Thu, 04 Sep 2025 15:39:21 -0700 (PDT) Date: Thu, 4 Sep 2025 22:38:47 +0000 In-Reply-To: <20250904223850.884188-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250904223850.884188-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.51.0.355.g5224444f11-goog Message-ID: <20250904223850.884188-4-dylanbhatch@google.com> Subject: [PATCH v2 3/6] unwind: add sframe v2 header From: Dylan Hatch To: Josh Poimboeuf , Steven Rostedt , Indu Bhagat , Peter Zijlstra , Will Deacon , Catalin Marinas , Jiri Kosina Cc: Dylan Hatch , Roman Gushchin , Weinan Liu , Mark Rutland , Ian Rogers , linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, joe.lawrence@redhat.com, Puranjay Mohan , Song Liu , Prasanna Kumar T S M Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add sframe header so that we know how to access the sframe section generated by compilers. This is the sframe header file borrowed from the patchset [1] Josh Poimboeuf according to sframe v2 spec [2]. [1]: https://lore.kernel.org/all/f27e8463783febfa0dabb0432a3dd6be8ad98412.1= 737511963.git.jpoimboe@kernel.org/ [2]: https://sourceware.org/binutils/docs/sframe-spec.html Signed-off-by: Weinan Liu Signed-off-by: Dylan Hatch Reviewed-by: Prasanna Kumar T S M --- kernel/sframe.h | 75 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 kernel/sframe.h diff --git a/kernel/sframe.h b/kernel/sframe.h new file mode 100644 index 000000000000..e9045f980fee --- /dev/null +++ b/kernel/sframe.h @@ -0,0 +1,75 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * From https://www.sourceware.org/binutils/docs/sframe-spec.html + */ +#ifndef _SFRAME_H +#define _SFRAME_H + +#include + +#define SFRAME_VERSION_1 1 +#define SFRAME_VERSION_2 2 +#define SFRAME_MAGIC 0xdee2 + +#define SFRAME_F_FDE_SORTED 0x1 +#define SFRAME_F_FRAME_POINTER 0x2 + +#define SFRAME_ABI_AARCH64_ENDIAN_BIG 1 +#define SFRAME_ABI_AARCH64_ENDIAN_LITTLE 2 +#define SFRAME_ABI_AMD64_ENDIAN_LITTLE 3 + +#define SFRAME_FRE_TYPE_ADDR1 0 +#define SFRAME_FRE_TYPE_ADDR2 1 +#define SFRAME_FRE_TYPE_ADDR4 2 + +#define SFRAME_FDE_TYPE_PCINC 0 +#define SFRAME_FDE_TYPE_PCMASK 1 + +struct sframe_preamble { + u16 magic; + u8 version; + u8 flags; +} __packed; + +struct sframe_header { + struct sframe_preamble preamble; + u8 abi_arch; + s8 cfa_fixed_fp_offset; + s8 cfa_fixed_ra_offset; + u8 auxhdr_len; + u32 num_fdes; + u32 num_fres; + u32 fre_len; + u32 fdes_off; + u32 fres_off; +} __packed; + +#define SFRAME_HEADER_SIZE(header) \ + ((sizeof(struct sframe_header) + (header).auxhdr_len)) + +#define SFRAME_AARCH64_PAUTH_KEY_A 0 +#define SFRAME_AARCH64_PAUTH_KEY_B 1 + +struct sframe_fde { + s32 start_addr; + u32 func_size; + u32 fres_off; + u32 fres_num; + u8 info; + u8 rep_size; + u16 padding; +} __packed; + +#define SFRAME_FUNC_FRE_TYPE(data) (data & 0xf) +#define SFRAME_FUNC_FDE_TYPE(data) ((data >> 4) & 0x1) +#define SFRAME_FUNC_PAUTH_KEY(data) ((data >> 5) & 0x1) + +#define SFRAME_BASE_REG_FP 0 +#define SFRAME_BASE_REG_SP 1 + +#define SFRAME_FRE_CFA_BASE_REG_ID(data) (data & 0x1) +#define SFRAME_FRE_OFFSET_COUNT(data) ((data >> 1) & 0xf) +#define SFRAME_FRE_OFFSET_SIZE(data) ((data >> 5) & 0x3) +#define SFRAME_FRE_MANGLED_RA_P(data) ((data >> 7) & 0x1) + +#endif /* _SFRAME_H */ --=20 2.51.0.355.g5224444f11-goog From nobody Fri Oct 3 05:25:20 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7E6E2F069B for ; Thu, 4 Sep 2025 22:39:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025567; cv=none; b=nMS0uwOkAV+JJ1Ky8bYBL98HwadAqMwo65kzRfVwwqsx++B+S5QCw2gojRlAFe01lAhIfgxpWmGOFzM4v3o3T6ZYNw/s720Jhcj2BtPiokulSFiqhALTT11BI7WEJq3NS23ZTKZMMrfGj0D7lYJkRFF+xmM56Ks2MHeMpmQp6D8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025567; c=relaxed/simple; bh=IeZxR7TyG4FPSEisgcdIPCKln5kfppzISNwk9hjNvvc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=L7x0oGJ38qlb7u+LAJJeRjtPpPwQIwev6WHNOCYALbq5R4X3VlCcq0556IXkNxnBWxeJFHjMWMN0DYkdRGNnsdjt2QbX/MxD+4M2sTcpyldVgifpe2IkanMsL8AkwGYozKHTCZ8tuGefHfdNZGMfP5Q3r6wI+TklbX0O6g37UJE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iNa5Z+CD; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iNa5Z+CD" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-329e55e686dso1866800a91.3 for ; Thu, 04 Sep 2025 15:39:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1757025565; x=1757630365; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Y2/9lQLIV+FHCh92ZbMoaFXnxUMAJdtb0VfSyDhZjLY=; b=iNa5Z+CDXRv5vn22XlDzI1RWCI9OtL8QlxoqJ0O6PFRsuxyKVRcEleoAKmJQQa4QaO VdP4fQtrmMANathpVLDmWK7NdSdH6CkiZJ3zOr9fDM0pg+XtkbtQgiG7z7MDA+Zl9JC/ fpY6Kij1QI1ecIqdYt1DmRA1Eb4p0O7wi1PVzc9N1lCjl1R3op/iYj6Rbs6lC3Fl1k86 CN5jtXDOXvJMX4DKrHmYvAoWF982l8mhwzkhDz5A+4RaAnj78qfTIL314mOXNCZEfvgw mi2euverO7Rm1JEXUxtVf6hSUzdTNLm22x8w3uxSXI3oW+homyZfr1Eif+EiXU+5Wd2d E7+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757025565; x=1757630365; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Y2/9lQLIV+FHCh92ZbMoaFXnxUMAJdtb0VfSyDhZjLY=; b=jUs1UkVvlm+oXHZ4y6HgFkjM0DaHszrjGGUE4nRgMS2OdKqPcX2N+e/abRrtnPFDdh JPJ09AsqJL63mJFTpbT20RtpR+lXaKAN7k+H6uVZx+YEHU9qcRTYOxoAgWZIe7dc5hMK JTWezczH8EEMcNyvVQilX+zz+p1eL5FfyeVhg3y7G7DSbTKcnxgD+K3qxcHNOpw+B1R+ 7wZSSRWKbJiKl+vaiaCx/9PN8N9pV23Of3dy7mAzNkI7nVLp5VvUYduRJnmYcdgF1R1g WJYCR4CNAv2latx9dr+JnDjxd0uD2JsXvjtje3sht5q9GGhM7545gUEgqvZkFHDhP2JX dxPQ== X-Forwarded-Encrypted: i=1; AJvYcCXG6RPmhPCBd7ouKGaUJ+uyU4nH58/kk7QttitzoKa2EQwNoS2YrHWY0FlbBzbu9WGDhjBxv+XuwDXZauU=@vger.kernel.org X-Gm-Message-State: AOJu0YyVvxteHabOnGUtALly9MTxuBebK4EFYPQt9ARdIfkIVaPnTwlR 9nbYJXpxo0wXXfsJ6w5MWtg8JrxAFq7AN1egZSImLI8TT9DHRYObh22FjFUyymoEKITHgup0Vvf Rc+yX0YZ7I7ny29dCBkBooBfk7g== X-Google-Smtp-Source: AGHT+IHy96MyTB0pkWlUUJ6E8LnjrTTKObtRIYDMqttTIQgktOfu9B5oK7HG5r5CV4MNuHJ8A85VZ/GzHRPhLbfOwQ== X-Received: from pjbpx16.prod.google.com ([2002:a17:90b:2710:b0:32a:a943:aa8a]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:288b:b0:32b:a307:23dc with SMTP id 98e67ed59e1d1-32ba30726e7mr3995474a91.24.1757025565433; Thu, 04 Sep 2025 15:39:25 -0700 (PDT) Date: Thu, 4 Sep 2025 22:38:48 +0000 In-Reply-To: <20250904223850.884188-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250904223850.884188-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.51.0.355.g5224444f11-goog Message-ID: <20250904223850.884188-5-dylanbhatch@google.com> Subject: [PATCH v2 4/6] unwind: Implement generic sframe unwinder library From: Dylan Hatch To: Josh Poimboeuf , Steven Rostedt , Indu Bhagat , Peter Zijlstra , Will Deacon , Catalin Marinas , Jiri Kosina Cc: Dylan Hatch , Roman Gushchin , Weinan Liu , Mark Rutland , Ian Rogers , linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, joe.lawrence@redhat.com, Puranjay Mohan , Song Liu , Prasanna Kumar T S M Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Weinan Liu This change introduces a kernel space unwinder using sframe table for architectures without ORC unwinder support. The implementation is adapted from Josh's userspace sframe unwinder proposal[1] according to the sframe v2 spec[2]. [1] https://lore.kernel.org/lkml/42c0a99236af65c09c8182e260af7bcf5aa1e158.1= 730150953.git.jpoimboe@kernel.org/ [2] https://sourceware.org/binutils/docs/sframe-spec.html Signed-off-by: Weinan Liu Signed-off-by: Dylan Hatch Reviewed-by: Prasanna Kumar T S M --- include/linux/sframe_lookup.h | 43 ++++++++ kernel/Makefile | 1 + kernel/sframe_lookup.c | 196 ++++++++++++++++++++++++++++++++++ 3 files changed, 240 insertions(+) create mode 100644 include/linux/sframe_lookup.h create mode 100644 kernel/sframe_lookup.c diff --git a/include/linux/sframe_lookup.h b/include/linux/sframe_lookup.h new file mode 100644 index 000000000000..1c26cf1f38d4 --- /dev/null +++ b/include/linux/sframe_lookup.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_SFRAME_LOOKUP_H +#define _LINUX_SFRAME_LOOKUP_H + +/** + * struct sframe_ip_entry - sframe unwind info for given ip + * @cfa_offset: Offset for the Canonical Frame Address(CFA) from Frame + * Pointer(FP) or Stack Pointer(SP) + * @ra_offset: Offset for the Return Address from CFA. + * @fp_offset: Offset for the Frame Pointer (FP) from CFA. + * @use_fp: Use FP to get next CFA or not + */ +struct sframe_ip_entry { + int32_t cfa_offset; + int32_t ra_offset; + int32_t fp_offset; + bool use_fp; +}; + +/** + * struct sframe_table - sframe struct of a table + * @sfhdr_p: Pointer to sframe header + * @fde_p: Pointer to the first of sframe frame description entry(FDE). + * @fre_p: Pointer to the first of sframe frame row entry(FRE). + */ +struct sframe_table { + struct sframe_header *sfhdr_p; + struct sframe_fde *fde_p; + char *fre_p; +}; + +#ifdef CONFIG_SFRAME_UNWINDER +void init_sframe_table(void); +int sframe_find_pc(unsigned long pc, struct sframe_ip_entry *entry); +#else +static inline void init_sframe_table(void) {} +static inline int sframe_find_pc(unsigned long pc, struct sframe_ip_entry = *entry) +{ + return -EINVAL; +} +#endif + +#endif /* _LINUX_SFRAME_LOOKUP_H */ diff --git a/kernel/Makefile b/kernel/Makefile index c60623448235..17e9cfe09dc0 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -138,6 +138,7 @@ obj-$(CONFIG_WATCH_QUEUE) +=3D watch_queue.o =20 obj-$(CONFIG_RESOURCE_KUNIT_TEST) +=3D resource_kunit.o obj-$(CONFIG_SYSCTL_KUNIT_TEST) +=3D sysctl-test.o +obj-$(CONFIG_SFRAME_UNWINDER) +=3D sframe_lookup.o =20 CFLAGS_kstack_erase.o +=3D $(DISABLE_KSTACK_ERASE) CFLAGS_kstack_erase.o +=3D $(call cc-option,-mgeneral-regs-only) diff --git a/kernel/sframe_lookup.c b/kernel/sframe_lookup.c new file mode 100644 index 000000000000..51cd24a75956 --- /dev/null +++ b/kernel/sframe_lookup.c @@ -0,0 +1,196 @@ +// SPDX-License-Identifier: GPL-2.0-only +#define pr_fmt(fmt) "sframe: " fmt + +#include +#include +#include +#include +#include "sframe.h" + +extern char __start_sframe_header[]; +extern char __stop_sframe_header[]; + +static bool sframe_init __ro_after_init; +static struct sframe_table sftbl; + +#define SFRAME_READ_TYPE(in, out, type) \ +({ \ + type __tmp; \ + memcpy(&__tmp, in, sizeof(__tmp)); \ + in +=3D sizeof(__tmp); \ + out =3D __tmp; \ +}) + +#define SFRAME_READ_ROW_ADDR(in_addr, out_addr, type) \ +({ \ + switch (type) { \ + case SFRAME_FRE_TYPE_ADDR1: \ + SFRAME_READ_TYPE(in_addr, out_addr, u8); \ + break; \ + case SFRAME_FRE_TYPE_ADDR2: \ + SFRAME_READ_TYPE(in_addr, out_addr, u16); \ + break; \ + case SFRAME_FRE_TYPE_ADDR4: \ + SFRAME_READ_TYPE(in_addr, out_addr, u32); \ + break; \ + default: \ + break; \ + } \ +}) + +#define SFRAME_READ_ROW_OFFSETS(in_addr, out_addr, size) \ +({ \ + switch (size) { \ + case 1: \ + SFRAME_READ_TYPE(in_addr, out_addr, s8); \ + break; \ + case 2: \ + SFRAME_READ_TYPE(in_addr, out_addr, s16); \ + break; \ + case 4: \ + SFRAME_READ_TYPE(in_addr, out_addr, s32); \ + break; \ + default: \ + break; \ + } \ +}) + +static struct sframe_fde *find_fde(const struct sframe_table *tbl, unsigne= d long pc) +{ + int l, r, m, f; + int32_t ip; + struct sframe_fde *fdep; + + if (!tbl || !tbl->sfhdr_p || !tbl->fde_p) + return NULL; + + ip =3D (pc - (unsigned long)tbl->sfhdr_p); + + /* Do a binary range search to find the rightmost FDE start_addr < ip */ + l =3D m =3D f =3D 0; + r =3D tbl->sfhdr_p->num_fdes; + while (l < r) { + m =3D l + ((r - l) / 2); + fdep =3D tbl->fde_p + m; + if (fdep->start_addr > ip) + r =3D m; + else + l =3D m + 1; + } + /* use l - 1 because l will be the first item fdep->start_addr > ip */ + f =3D l - 1; + if (f >=3D tbl->sfhdr_p->num_fdes || f < 0) + return NULL; + fdep =3D tbl->fde_p + f; + if (ip < fdep->start_addr || ip > fdep->start_addr + fdep->func_size) + return NULL; + + return fdep; +} + +static int find_fre(const struct sframe_table *tbl, unsigned long pc, + const struct sframe_fde *fdep, struct sframe_ip_entry *entry) +{ + int i, offset_size, offset_count; + char *fres, *offsets_loc; + int32_t ip_off; + uint32_t next_row_ip_off; + uint8_t fre_info, fde_type =3D SFRAME_FUNC_FDE_TYPE(fdep->info), + fre_type =3D SFRAME_FUNC_FRE_TYPE(fdep->info); + + fres =3D tbl->fre_p + fdep->fres_off; + + /* Whether PCs in the FREs should be treated as masks or not */ + if (fde_type =3D=3D SFRAME_FDE_TYPE_PCMASK) + ip_off =3D pc % fdep->rep_size; + else + ip_off =3D (int32_t)(pc - (unsigned long)tbl->sfhdr_p) - fdep->start_add= r; + + if (ip_off < 0 || ip_off > fdep->func_size) + return -EINVAL; + + /* + * FRE structure starts by address of the entry with variants length. Use + * two pointers to track current head(fres) and the address of last + * offset(offsets_loc) + */ + for (i =3D 0; i < fdep->fres_num; i++) { + SFRAME_READ_ROW_ADDR(fres, next_row_ip_off, fre_type); + if (ip_off < next_row_ip_off) + break; + SFRAME_READ_TYPE(fres, fre_info, u8); + offsets_loc =3D fres; + /* + * jump to the start of next fre + * fres +=3D fre_offets_cnt*offset_size + */ + fres +=3D SFRAME_FRE_OFFSET_COUNT(fre_info) << SFRAME_FRE_OFFSET_SIZE(fr= e_info); + } + + offset_size =3D 1 << SFRAME_FRE_OFFSET_SIZE(fre_info); + offset_count =3D SFRAME_FRE_OFFSET_COUNT(fre_info); + + if (offset_count > 0) { + SFRAME_READ_ROW_OFFSETS(offsets_loc, entry->cfa_offset, offset_size); + offset_count--; + } + if (offset_count > 0 && !entry->ra_offset) { + SFRAME_READ_ROW_OFFSETS(offsets_loc, entry->ra_offset, offset_size); + offset_count--; + } + if (offset_count > 0 && !entry->fp_offset) { + SFRAME_READ_ROW_OFFSETS(offsets_loc, entry->fp_offset, offset_size); + offset_count--; + } + if (offset_count) + return -EINVAL; + + entry->use_fp =3D SFRAME_FRE_CFA_BASE_REG_ID(fre_info) =3D=3D SFRAME_BASE= _REG_FP; + + return 0; +} + +int sframe_find_pc(unsigned long pc, struct sframe_ip_entry *entry) +{ + struct sframe_fde *fdep; + struct sframe_table *sftbl_p =3D &sftbl; + int err; + + if (!sframe_init) + return -EINVAL; + + memset(entry, 0, sizeof(*entry)); + entry->ra_offset =3D sftbl_p->sfhdr_p->cfa_fixed_ra_offset; + entry->fp_offset =3D sftbl_p->sfhdr_p->cfa_fixed_fp_offset; + + fdep =3D find_fde(sftbl_p, pc); + if (!fdep) + return -EINVAL; + err =3D find_fre(sftbl_p, pc, fdep, entry); + if (err) + return err; + + return 0; +} + +void __init init_sframe_table(void) +{ + size_t sframe_size =3D (void *)__stop_sframe_header - (void *)__start_sfr= ame_header; + void *sframe_buf =3D __start_sframe_header; + + if (sframe_size <=3D 0) + return; + sftbl.sfhdr_p =3D sframe_buf; + if (!sftbl.sfhdr_p || sftbl.sfhdr_p->preamble.magic !=3D SFRAME_MAGIC || + sftbl.sfhdr_p->preamble.version !=3D SFRAME_VERSION_2 || + !(sftbl.sfhdr_p->preamble.flags & SFRAME_F_FDE_SORTED)) { + pr_warn("WARNING: Unable to read sframe header. Disabling unwinder.\n"); + return; + } + + sftbl.fde_p =3D (struct sframe_fde *)(__start_sframe_header + SFRAME_HEAD= ER_SIZE(*sftbl.sfhdr_p) + + sftbl.sfhdr_p->fdes_off); + sftbl.fre_p =3D __start_sframe_header + SFRAME_HEADER_SIZE(*sftbl.sfhdr_p) + + sftbl.sfhdr_p->fres_off; + sframe_init =3D true; +} --=20 2.51.0.355.g5224444f11-goog From nobody Fri Oct 3 05:25:20 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F5042F0C6C for ; Thu, 4 Sep 2025 22:39:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025572; cv=none; b=NAbtdJg1lmR1Hygba9ZWXZ5eRTlb57RWPynjQUOOF7clOCg4imDsg91nZJX6fu8rY+i0QmzIejV7qg/pdrRMouczqPUXpgJD8E9Cv2EPbOp8HuBi3DQAS5L9kqYh5oZ9+rUGU8l4/KH1lQVSjUjNJgUcr3iUt+gT2gJMBL76ZUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025572; c=relaxed/simple; bh=Py6ERm9u4XbjFAnqCPl9ivc1rlJYdEkiBn2dS6BUjHQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XgFAhVjdoqBLarZ9/5VqR2fyBRLcQt1eoUCvu5sS0EIV78XWLZv+TebYSNBNYEqhdn2K+B7KXXPyKsZr+8hymqs3maIhVDWvYGGgQTuWr64WHhDYZ3Ua/8inj0cd5FAe2cPvErZCe5S9CZDUgTH4TzMWPIYfszdhoIaDdfwdSYQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ksg6emtH; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ksg6emtH" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-772642f9fa3so2466130b3a.1 for ; Thu, 04 Sep 2025 15:39:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1757025570; x=1757630370; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6spkF2Mtmky9u9zJohv2so8g+gmgPw5xY2VZfo9rxBs=; b=ksg6emtHKDGZHt4AWKH5+VJvPShUmKuue5R3n3v5p/s7hmgReLr4yjp3W0M4JKALN5 A22Zrb/GFO194uCAZFc7AwfOv9C4xKJ/TuaSnZc8nQ3z0iL0wVd1TRTG/nWwDKNQM0GV 04ijDf9Or4lqHadn4fjksN/76DkQUIQIBQasKc0CQt3GRmIkDWAOfK7oHOQcbuK0ksK6 Bv90dJw/7gQoNFP3eMb55m/Prdl7eftI0e77U2tqOh1kDT9Qk/02TIHElI05k/beGmv6 5HEiS9QqI1Wr+8bFeitaTeAT1kE33s+KPfmcDDOiuBo9RPNWprxMa0gB/PlhHx0/rB17 QcJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757025570; x=1757630370; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6spkF2Mtmky9u9zJohv2so8g+gmgPw5xY2VZfo9rxBs=; b=TyamCS79Bm5myn1odZsOswih//p5IIr4fmbpdKSRRLF6E1r7s22Qqk3EMar/oFvmHr i4pM3uWSeP4nJhvFCHmLDvRriURYU6m2hrbQZykl5NLGpZT5j9OkJSOEFuqjqogF1lCo IQ+XOrGH93Sjsww3gzgGOU1FhTctY9ZbB6nC12p6Y7hzOUpv+Ajvjr07441pzLkaenEJ um55oclHYvb/dzKplgV+cYDIHSg9LKIEDX2ufWi1lL4+M5bpslZhkwhx8TN8gcTbrJ4N pd3iPWClRbbcxtKNdgcf8QxRNhTFRgcx5Gk566pdCGKEhw6l8DWAp8byzHQFq+IrDc5H nS8w== X-Forwarded-Encrypted: i=1; AJvYcCWV9rpIkCD9xCLeMI1fd6XMOTgpclI1RT5Xzeno9JBabntNhno3TecaKGejsvRPcOJmTI6ds7h+kDtzyj0=@vger.kernel.org X-Gm-Message-State: AOJu0YwEIF/nXM1s/aUPVmUnpk9Bai0FgVi7BOBrwatzz9MAfhWr5PUa 5Oa8PY9lcbAIWQTH8nqsECarRVHkS9lDZJJ7qU1s+cem5mOE7qL1vgE6BLOBbaR0HK/aB1Q4ePc Mptb4/BYU7ordv1PvnfWgiMhWPg== X-Google-Smtp-Source: AGHT+IE4cVZe/Ko33sMePIr0pO/SLi1MRU5bNT+p9QofRSNLSjKXtZiV5Q8NrVHWoHaTXOqxsAmwXjjIdyqhxxJI8g== X-Received: from pgbfy15.prod.google.com ([2002:a05:6a02:2a8f:b0:b4f:9acb:771e]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:9997:b0:24c:c33e:8ddc with SMTP id adf61e73a8af0-24e873686a5mr1550457637.28.1757025569652; Thu, 04 Sep 2025 15:39:29 -0700 (PDT) Date: Thu, 4 Sep 2025 22:38:49 +0000 In-Reply-To: <20250904223850.884188-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250904223850.884188-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.51.0.355.g5224444f11-goog Message-ID: <20250904223850.884188-6-dylanbhatch@google.com> Subject: [PATCH v2 5/6] arm64/module, unwind: Add sframe support for modules. From: Dylan Hatch To: Josh Poimboeuf , Steven Rostedt , Indu Bhagat , Peter Zijlstra , Will Deacon , Catalin Marinas , Jiri Kosina Cc: Dylan Hatch , Roman Gushchin , Weinan Liu , Mark Rutland , Ian Rogers , linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, joe.lawrence@redhat.com, Puranjay Mohan , Song Liu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Weinan Liu Add sframe table to mod_arch_specific and support sframe unwind when .sframe section can be found on incoming modules. Signed-off-by: Dylan Hatch Signed-off-by: Weinan Liu --- arch/arm64/include/asm/module.h | 6 ++++++ arch/arm64/kernel/module.c | 5 +++++ include/linux/sframe_lookup.h | 2 ++ kernel/sframe_lookup.c | 38 ++++++++++++++++++++++++++++++++- 4 files changed, 50 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/modul= e.h index 79550b22ba19..e3781fcdc620 100644 --- a/arch/arm64/include/asm/module.h +++ b/arch/arm64/include/asm/module.h @@ -6,6 +6,7 @@ #define __ASM_MODULE_H =20 #include +#include =20 struct mod_plt_sec { int plt_shndx; @@ -17,6 +18,11 @@ struct mod_arch_specific { struct mod_plt_sec core; struct mod_plt_sec init; =20 +#ifdef CONFIG_SFRAME_UNWINDER + struct sframe_table sftbl; + bool sframe_init; +#endif + /* for CONFIG_DYNAMIC_FTRACE */ struct plt_entry *ftrace_trampolines; }; diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 40148d2725ce..d0adeb4cf63d 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -18,6 +18,7 @@ #include #include #include +#include =20 #include #include @@ -491,5 +492,9 @@ int module_finalize(const Elf_Ehdr *hdr, } } =20 + s =3D find_section(hdr, sechdrs, ".sframe"); + if (s) + sframe_module_init(me, (void *)s->sh_addr, s->sh_size); + return module_init_ftrace_plt(hdr, sechdrs, me); } diff --git a/include/linux/sframe_lookup.h b/include/linux/sframe_lookup.h index 1c26cf1f38d4..f84c1f41a421 100644 --- a/include/linux/sframe_lookup.h +++ b/include/linux/sframe_lookup.h @@ -31,9 +31,11 @@ struct sframe_table { =20 #ifdef CONFIG_SFRAME_UNWINDER void init_sframe_table(void); +void sframe_module_init(struct module *mod, void *_sframe, size_t _sframe_= size); int sframe_find_pc(unsigned long pc, struct sframe_ip_entry *entry); #else static inline void init_sframe_table(void) {} +static inline void sframe_module_init(struct module *mod, void *_sframe, s= ize_t _sframe_size) {} static inline int sframe_find_pc(unsigned long pc, struct sframe_ip_entry = *entry) { return -EINVAL; diff --git a/kernel/sframe_lookup.c b/kernel/sframe_lookup.c index 51cd24a75956..c87a94f01891 100644 --- a/kernel/sframe_lookup.c +++ b/kernel/sframe_lookup.c @@ -156,10 +156,20 @@ int sframe_find_pc(unsigned long pc, struct sframe_ip= _entry *entry) struct sframe_table *sftbl_p =3D &sftbl; int err; =20 - if (!sframe_init) + if (!entry || !sframe_init) return -EINVAL; =20 memset(entry, 0, sizeof(*entry)); + + if (!is_ksym_addr(pc)) { + struct module *mod; + + mod =3D __module_address(pc); + if (!mod || !mod->arch.sframe_init) + return -EINVAL; + sftbl_p =3D &mod->arch.sftbl; + } + entry->ra_offset =3D sftbl_p->sfhdr_p->cfa_fixed_ra_offset; entry->fp_offset =3D sftbl_p->sfhdr_p->cfa_fixed_fp_offset; =20 @@ -194,3 +204,29 @@ void __init init_sframe_table(void) + sftbl.sfhdr_p->fres_off; sframe_init =3D true; } + +void sframe_module_init(struct module *mod, void *_sframe, size_t _sframe_= size) +{ + size_t sframe_size =3D _sframe_size; + void *sframe_buf =3D _sframe; + struct sframe_table _sftbl; + + + if (sframe_size <=3D 0) + return; + _sftbl.sfhdr_p =3D sframe_buf; + if (!_sftbl.sfhdr_p || _sftbl.sfhdr_p->preamble.magic !=3D SFRAME_MAGIC || + _sftbl.sfhdr_p->preamble.version !=3D SFRAME_VERSION_2 || + !(_sftbl.sfhdr_p->preamble.flags & SFRAME_F_FDE_SORTED)) { + pr_warn("WARNING: Unable to read sframe header. Disabling unwinder.\n"); + return; + } + + _sftbl.fde_p =3D (struct sframe_fde *)(sframe_buf + SFRAME_HEADER_SIZE(*_= sftbl.sfhdr_p) + + _sftbl.sfhdr_p->fdes_off); + _sftbl.fre_p =3D sframe_buf + SFRAME_HEADER_SIZE(*_sftbl.sfhdr_p) + + _sftbl.sfhdr_p->fres_off; + + mod->arch.sftbl =3D _sftbl; + mod->arch.sframe_init =3D true; +} --=20 2.51.0.355.g5224444f11-goog From nobody Fri Oct 3 05:25:20 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C81122ECE83 for ; Thu, 4 Sep 2025 22:39:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025579; cv=none; b=Sj1uoZBV5bwHQUf8QQC9jBNy0+XI8vYyXXTxbiOUZyUt/1CPpc4XO3ZTqdDoCZYfLg9HEc2eJI053wouvh8Z6z1WF3CNqe5FP+n7kR+o8x/ro82p2GUHScprtx5u/1SS0pfOXiKU8NpCLZezOefuoEgiNCMFsbzSvDkA76zEE/c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757025579; c=relaxed/simple; bh=dlLs+yi7RwXEFoGkLBmy8K3WE1BY0qc9j744kGghnWs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jVjOXCXfjwcDs6ygI/OCmbAlCmszK8hVNW06vmp0PmsE1WW2WhWzPNRrMCgSixm5q/ARjmwUIAQCHOUbcKjXLfLNfNLRpz/1nfJCnYjOK1DImh35PjPER00uVxaJUFtL+Y3fNgWVjlArOOMCpQqaGbHmr4fkbw7tMs/oLyA2C0M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iJBhliaz; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iJBhliaz" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-24458274406so28057745ad.3 for ; Thu, 04 Sep 2025 15:39:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1757025576; x=1757630376; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0NQT71Rd3hFVreB2uwDzx4dZZ1nQuKfGGKuQOi/O3l8=; b=iJBhliaznHDR/NuiId7sWKJ6WkgbRZAgCRhMivhm6r2jkRWugg3+RAGUbQPA5+xmRh kkVlkOmsItRApf6el4SMs41avh27Y8MncbON+Xa8J+bfCaEh1ynL5bmlRfnI8MC1ul9s 1OeKFKkWS7Dl9uuxrIb6nWcu3Nat1kiGq5wh1mnK4o5VN1lmdSzwZJlHM1XIll2ul9mT b6HMFZGWNWlanqZQDq6qUiSHqRcXHW83IIuAq/xCVfNS48LzcdfaRdCJ20/Df+Op+Ko0 nfd2it4LMynCExS1ajhkE2b7CkCNeRSWqnIojzUg0q8/IifnlakM6mZYzZWyO9p9fGE7 aPTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757025576; x=1757630376; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0NQT71Rd3hFVreB2uwDzx4dZZ1nQuKfGGKuQOi/O3l8=; b=DZ03vpMH+lVrzzBxESe1w7/fddp9CZ02TR3s0r73vQP0sJLcBUZ4bRUdNxAs/LP98+ 8YBHt/hsHfkmQqf9q+Q/czAkqq/9CztzRViEz5ZcZBdpA4bKmWg4aDpLqv2tNAY3RdfT 4YSkVSix37XYgueFmDVKQXcbHktOHTWgJpBDTa2qNJXvDAd6EfsvARUfiVeDvdgwD4Af ulr48sTTGKTf11DTE8bx/bxk8HCC0UOFMwUAQKOgeutM0hH5zFaLwIyn+8j85H87TXft 4TyzskCGDciVs7oojIk8RuTGsRttvIQJhOSpnpiphDlNNELBb2tkuDskLj0PCHG5aKb9 ozwg== X-Forwarded-Encrypted: i=1; AJvYcCUQuvzTwrRrlLyhFr3Dp7mMouTlgjEWf5L8yf9S0nV9D/nvfgILAuOBhpIRVp0YDZzn2gbjZGrHNos0a9s=@vger.kernel.org X-Gm-Message-State: AOJu0YyhJ0hLMlO5QnBiNSrVVAyJnkSI0u4ZjyMLqO/HifzrJoBYaEUu 73C+OikOC9gMit8+cWwMzNP+ii8LsiP9+QKP+HLJDp9LbuANXtA0Ss0v2Xhv8X2kJ0dk/dWOtMy x0+YySwMXCBgpIHywOvONl/cLrA== X-Google-Smtp-Source: AGHT+IGU9/lQfrcWOX/dpTT9E6qyc3rvM+V4d9jDHZ4cKgBH8shTzfPR7bsY0ynubJ92rvKh6T2ZEGmb0PlBrTok+w== X-Received: from pjbsy7.prod.google.com ([2002:a17:90b:2d07:b0:327:b430:11ad]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e5cf:b0:242:8a45:a959 with SMTP id d9443c01a7336-24944ae5ff7mr273875345ad.47.1757025576197; Thu, 04 Sep 2025 15:39:36 -0700 (PDT) Date: Thu, 4 Sep 2025 22:38:50 +0000 In-Reply-To: <20250904223850.884188-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250904223850.884188-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.51.0.355.g5224444f11-goog Message-ID: <20250904223850.884188-7-dylanbhatch@google.com> Subject: [PATCH v2 6/6] unwind: arm64: Add reliable stacktrace with sframe unwinder. From: Dylan Hatch To: Josh Poimboeuf , Steven Rostedt , Indu Bhagat , Peter Zijlstra , Will Deacon , Catalin Marinas , Jiri Kosina Cc: Dylan Hatch , Roman Gushchin , Weinan Liu , Mark Rutland , Ian Rogers , linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, joe.lawrence@redhat.com, Puranjay Mohan , Song Liu , Prasanna Kumar T S M Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Weinan Liu Add unwind_next_frame_sframe() function to unwind by sframe info. Built with GNU Binutils 2.42 to verify that this sframe unwinder can backtrace correctly on arm64. To support livepatch, we need to add arch_stack_walk_reliable to support reliable stacktrace according to https://docs.kernel.org/livepatch/reliable-stacktrace.html#requirements report stacktrace is not reliable if we are not able to unwind the stack by sframe unwinder and fallback to FP based unwinder Signed-off-by: Weinan Liu Signed-off-by: Dylan Hatch Reviewed-by: Prasanna Kumar T S M --- arch/arm64/include/asm/stacktrace/common.h | 6 ++ arch/arm64/kernel/setup.c | 2 + arch/arm64/kernel/stacktrace.c | 102 +++++++++++++++++++++ 3 files changed, 110 insertions(+) diff --git a/arch/arm64/include/asm/stacktrace/common.h b/arch/arm64/includ= e/asm/stacktrace/common.h index 821a8fdd31af..26449cd402db 100644 --- a/arch/arm64/include/asm/stacktrace/common.h +++ b/arch/arm64/include/asm/stacktrace/common.h @@ -25,6 +25,8 @@ struct stack_info { * @stack: The stack currently being unwound. * @stacks: An array of stacks which can be unwound. * @nr_stacks: The number of stacks in @stacks. + * @cfa: The sp value at the call site of the current function. + * @unreliable: Stacktrace is unreliable. */ struct unwind_state { unsigned long fp; @@ -33,6 +35,10 @@ struct unwind_state { struct stack_info stack; struct stack_info *stacks; int nr_stacks; +#ifdef CONFIG_SFRAME_UNWINDER + unsigned long cfa; + bool unreliable; +#endif }; =20 static inline struct stack_info stackinfo_get_unknown(void) diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index 77c7926a4df6..ac1da45da532 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -32,6 +32,7 @@ #include #include #include +#include =20 #include #include @@ -375,6 +376,7 @@ void __init __no_sanitize_address setup_arch(char **cmd= line_p) "This indicates a broken bootloader or old kernel\n", boot_args[1], boot_args[2], boot_args[3]); } + init_sframe_table(); } =20 static inline bool cpu_can_disable(unsigned int cpu) diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 3ebcf8c53fb0..72e78024d05e 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -14,6 +14,7 @@ #include #include #include +#include =20 #include #include @@ -244,6 +245,53 @@ kunwind_next_frame_record(struct kunwind_state *state) return 0; } =20 +#ifdef CONFIG_SFRAME_UNWINDER +/* + * Unwind to the next frame according to sframe. + */ +static __always_inline int +unwind_next_frame_sframe(struct unwind_state *state) +{ + unsigned long fp =3D state->fp, ip =3D state->pc; + unsigned long base_reg, cfa; + unsigned long pc_addr, fp_addr; + struct sframe_ip_entry entry; + struct stack_info *info; + struct frame_record *record =3D (struct frame_record *)fp; + + int err; + + /* frame record alignment 8 bytes */ + if (fp & 0x7) + return -EINVAL; + + info =3D unwind_find_stack(state, fp, sizeof(*record)); + if (!info) + return -EINVAL; + + err =3D sframe_find_pc(ip, &entry); + if (err) + return -EINVAL; + + unwind_consume_stack(state, info, fp, sizeof(*record)); + + base_reg =3D entry.use_fp ? fp : state->cfa; + + /* Set up the initial CFA using fp based info if CFA is not set */ + if (!state->cfa) + cfa =3D fp - entry.fp_offset; + else + cfa =3D base_reg + entry.cfa_offset; + fp_addr =3D cfa + entry.fp_offset; + pc_addr =3D cfa + entry.ra_offset; + state->cfa =3D cfa; + state->fp =3D READ_ONCE(*(unsigned long *)(fp_addr)); + state->pc =3D READ_ONCE(*(unsigned long *)(pc_addr)); + + return 0; +} +#endif + /* * Unwind from one frame record (A) to the next frame record (B). * @@ -263,7 +311,20 @@ kunwind_next(struct kunwind_state *state) case KUNWIND_SOURCE_CALLER: case KUNWIND_SOURCE_TASK: case KUNWIND_SOURCE_REGS_PC: +#ifdef CONFIG_SFRAME_UNWINDER + if (!state->common.unreliable) + err =3D unwind_next_frame_sframe(&state->common); + + /* Fallback to FP based unwinder */ + if (err || state->common.unreliable) { err =3D kunwind_next_frame_record(state); + /* Mark its stacktrace result as unreliable if it is unwindable via FP */ + if (!err) + state->common.unreliable =3D true; + } +#else + err =3D kunwind_next_frame_record(state); +#endif break; default: err =3D -EINVAL; @@ -350,6 +411,9 @@ kunwind_stack_walk(kunwind_consume_fn consume_state, .common =3D { .stacks =3D stacks, .nr_stacks =3D ARRAY_SIZE(stacks), +#ifdef CONFIG_SFRAME_UNWINDER + .cfa =3D 0, +#endif }, }; =20 @@ -390,6 +454,43 @@ noinline noinstr void arch_stack_walk(stack_trace_cons= ume_fn consume_entry, kunwind_stack_walk(arch_kunwind_consume_entry, &data, task, regs); } =20 +#ifdef CONFIG_SFRAME_UNWINDER +struct kunwind_reliable_consume_entry_data { + stack_trace_consume_fn consume_entry; + void *cookie; + bool unreliable; +}; + +static __always_inline bool +arch_kunwind_reliable_consume_entry(const struct kunwind_state *state, voi= d *cookie) +{ + struct kunwind_reliable_consume_entry_data *data =3D cookie; + + if (state->common.unreliable) { + data->unreliable =3D true; + return false; + } + return data->consume_entry(data->cookie, state->common.pc); +} + +noinline notrace int arch_stack_walk_reliable( + stack_trace_consume_fn consume_entry, + void *cookie, struct task_struct *task) +{ + struct kunwind_reliable_consume_entry_data data =3D { + .consume_entry =3D consume_entry, + .cookie =3D cookie, + .unreliable =3D false, + }; + + kunwind_stack_walk(arch_kunwind_reliable_consume_entry, &data, task, NULL= ); + + if (data.unreliable) + return -EINVAL; + + return 0; +} +#else static __always_inline bool arch_reliable_kunwind_consume_entry(const struct kunwind_state *state, voi= d *cookie) { @@ -419,6 +520,7 @@ noinline noinstr int arch_stack_walk_reliable(stack_tra= ce_consume_fn consume_ent return kunwind_stack_walk(arch_reliable_kunwind_consume_entry, &data, task, NULL); } +#endif =20 struct bpf_unwind_consume_entry_data { bool (*consume_entry)(void *cookie, u64 ip, u64 sp, u64 fp); --=20 2.51.0.355.g5224444f11-goog