From nobody Mon Apr 6 14:12:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56B88288517; Mon, 6 Apr 2026 10:24:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471069; cv=none; b=eklIMfcBcaanFauzzuYAQuQUZT1Onzr5M/ZI1+SQkYDNN+/a0S1fyJ8iUYrrtwgWUb46lD5JzaQfGD1PKV84fROyvR+Pfv3qEigHCDchnFchksvZ20ifcordosmQqZ0wOj7PuBkkevWbCzcPDwqwhM5CpkzG91A089vKDqofB44= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471069; c=relaxed/simple; bh=3p7mpbdJlDzSYIjIy8yoDOnpM0yjrBJsi7RvJETMZi0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MIl/qvzlCcYMkpmVvzOUOi7XcRXWz1nPjF8HXWGNl+wSB68BMjVVhEP35kOrOQkis49YyTVOAHLLGjPMw0FjQF0v0mbkojRtGJbg21B4R/L+5OFQhhZm+A4faBVWcRKw0fd/dI5/q5QqwYZl5JwiwvmbCiEKE9FE90k/aEvFZmQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NtLUWxun; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NtLUWxun" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CBB51C4CEF7; Mon, 6 Apr 2026 10:24:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775471069; bh=3p7mpbdJlDzSYIjIy8yoDOnpM0yjrBJsi7RvJETMZi0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NtLUWxuniPXcaI00IAqVboiyTc1ujUUQ7gHuSd+CIZAmlR7RxlutHDB2g2NzGWBZt UFOeslMCuqSFnf+JQm//IKIZKxB0/s6uXhZSwPEEmG/g6g9WZz54pI7eCfrGZPXziA a4NKw5X26Pqjq44rMTK4LN0HTBrDVgFKngwC8lp/KwcfcGS/m2fGsWArI1UtlQrhab pG/NMEIDVQe0l11/xofsrtx3f1O2J7w+o4k6LCTIob8ZsUNydJ0jFfeBnbDt5P+l0+ VaQtKmg845j1KYXrTdS6/Bu8B6fFpl5ezlC7TttBp6OesUF+2SSqv6q7DRuPA5Z4ez HUxjgzfZFtj0Q== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Catalin Marinas , Will Deacon Cc: Masami Hiramatsu , Mathieu Desnoyers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Ian Rogers , linux-arm-kernel@lists.infradead.org Subject: [PATCH v16 1/5] ring-buffer: Flush and stop persistent ring buffer on panic Date: Mon, 6 Apr 2026 19:24:26 +0900 Message-ID: <177547106614.259641.13534691124205921635.stgit@mhiramat.tok.corp.google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog In-Reply-To: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> References: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) On real hardware, panic and machine reboot may not flush hardware cache to memory. This means the persistent ring buffer, which relies on a coherent state of memory, may not have its events written to the buffer and they may be lost. Moreover, there may be inconsistency with the counters which are used for validation of the integrity of the persistent ring buffer which may cause all data to be discarded. To avoid this issue, stop recording of the ring buffer on panic and flush the cache of the ring buffer's memory. Fixes: e645535a954a ("tracing: Add option to use memmapped memory for trace= boot instance") Cc: stable@vger.kernel.org Signed-off-by: Masami Hiramatsu (Google) Acked-by: Catalin Marinas --- Changes in v13: - Fix a rebase conflict. Changes in v11: - Do nothing by default since flush_cache_vmap() does nothing on x86 but it can cause deadlock on some architectures via on_each_cpu() because other CPUs will be stoppped when panic notifier is called. Changes in v9: - Fix typo of & to &&. - Fix typo of "Generic" Changes in v6: - Introduce asm/ring_buffer.h for arch_ring_buffer_flush_range(). - Use flush_cache_vmap() instead of flush_cache_all(). Changes in v5: - Use ring_buffer_record_off() instead of ring_buffer_record_disable(). - Use flush_cache_all() to ensure flush all cache. Changes in v3: - update patch description. --- arch/alpha/include/asm/Kbuild | 1 + arch/arc/include/asm/Kbuild | 1 + arch/arm/include/asm/Kbuild | 1 + arch/arm64/include/asm/ring_buffer.h | 10 ++++++++++ arch/csky/include/asm/Kbuild | 1 + arch/hexagon/include/asm/Kbuild | 1 + arch/loongarch/include/asm/Kbuild | 1 + arch/m68k/include/asm/Kbuild | 1 + arch/microblaze/include/asm/Kbuild | 1 + arch/mips/include/asm/Kbuild | 1 + arch/nios2/include/asm/Kbuild | 1 + arch/openrisc/include/asm/Kbuild | 1 + arch/parisc/include/asm/Kbuild | 1 + arch/powerpc/include/asm/Kbuild | 1 + arch/riscv/include/asm/Kbuild | 1 + arch/s390/include/asm/Kbuild | 1 + arch/sh/include/asm/Kbuild | 1 + arch/sparc/include/asm/Kbuild | 1 + arch/um/include/asm/Kbuild | 1 + arch/x86/include/asm/Kbuild | 1 + arch/xtensa/include/asm/Kbuild | 1 + include/asm-generic/ring_buffer.h | 13 +++++++++++++ kernel/trace/ring_buffer.c | 22 ++++++++++++++++++++++ 23 files changed, 65 insertions(+) create mode 100644 arch/arm64/include/asm/ring_buffer.h create mode 100644 include/asm-generic/ring_buffer.h diff --git a/arch/alpha/include/asm/Kbuild b/arch/alpha/include/asm/Kbuild index 483965c5a4de..b154b4e3dfa8 100644 --- a/arch/alpha/include/asm/Kbuild +++ b/arch/alpha/include/asm/Kbuild @@ -5,4 +5,5 @@ generic-y +=3D agp.h generic-y +=3D asm-offsets.h generic-y +=3D kvm_para.h generic-y +=3D mcs_spinlock.h +generic-y +=3D ring_buffer.h generic-y +=3D text-patching.h diff --git a/arch/arc/include/asm/Kbuild b/arch/arc/include/asm/Kbuild index 4c69522e0328..483caacc6988 100644 --- a/arch/arc/include/asm/Kbuild +++ b/arch/arc/include/asm/Kbuild @@ -5,5 +5,6 @@ generic-y +=3D extable.h generic-y +=3D kvm_para.h generic-y +=3D mcs_spinlock.h generic-y +=3D parport.h +generic-y +=3D ring_buffer.h generic-y +=3D user.h generic-y +=3D text-patching.h diff --git a/arch/arm/include/asm/Kbuild b/arch/arm/include/asm/Kbuild index 03657ff8fbe3..decad5f2c826 100644 --- a/arch/arm/include/asm/Kbuild +++ b/arch/arm/include/asm/Kbuild @@ -3,6 +3,7 @@ generic-y +=3D early_ioremap.h generic-y +=3D extable.h generic-y +=3D flat.h generic-y +=3D parport.h +generic-y +=3D ring_buffer.h =20 generated-y +=3D mach-types.h generated-y +=3D unistd-nr.h diff --git a/arch/arm64/include/asm/ring_buffer.h b/arch/arm64/include/asm/= ring_buffer.h new file mode 100644 index 000000000000..62316c406888 --- /dev/null +++ b/arch/arm64/include/asm/ring_buffer.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _ASM_ARM64_RING_BUFFER_H +#define _ASM_ARM64_RING_BUFFER_H + +#include + +/* Flush D-cache on persistent ring buffer */ +#define arch_ring_buffer_flush_range(start, end) dcache_clean_pop(start, e= nd) + +#endif /* _ASM_ARM64_RING_BUFFER_H */ diff --git a/arch/csky/include/asm/Kbuild b/arch/csky/include/asm/Kbuild index 3a5c7f6e5aac..7dca0c6cdc84 100644 --- a/arch/csky/include/asm/Kbuild +++ b/arch/csky/include/asm/Kbuild @@ -9,6 +9,7 @@ generic-y +=3D qrwlock.h generic-y +=3D qrwlock_types.h generic-y +=3D qspinlock.h generic-y +=3D parport.h +generic-y +=3D ring_buffer.h generic-y +=3D user.h generic-y +=3D vmlinux.lds.h generic-y +=3D text-patching.h diff --git a/arch/hexagon/include/asm/Kbuild b/arch/hexagon/include/asm/Kbu= ild index 1efa1e993d4b..0f887d4238ed 100644 --- a/arch/hexagon/include/asm/Kbuild +++ b/arch/hexagon/include/asm/Kbuild @@ -5,4 +5,5 @@ generic-y +=3D extable.h generic-y +=3D iomap.h generic-y +=3D kvm_para.h generic-y +=3D mcs_spinlock.h +generic-y +=3D ring_buffer.h generic-y +=3D text-patching.h diff --git a/arch/loongarch/include/asm/Kbuild b/arch/loongarch/include/asm= /Kbuild index 9034b583a88a..7e92957baf6a 100644 --- a/arch/loongarch/include/asm/Kbuild +++ b/arch/loongarch/include/asm/Kbuild @@ -10,5 +10,6 @@ generic-y +=3D qrwlock.h generic-y +=3D user.h generic-y +=3D ioctl.h generic-y +=3D mmzone.h +generic-y +=3D ring_buffer.h generic-y +=3D statfs.h generic-y +=3D text-patching.h diff --git a/arch/m68k/include/asm/Kbuild b/arch/m68k/include/asm/Kbuild index b282e0dd8dc1..62543bf305ff 100644 --- a/arch/m68k/include/asm/Kbuild +++ b/arch/m68k/include/asm/Kbuild @@ -3,5 +3,6 @@ generated-y +=3D syscall_table.h generic-y +=3D extable.h generic-y +=3D kvm_para.h generic-y +=3D mcs_spinlock.h +generic-y +=3D ring_buffer.h generic-y +=3D spinlock.h generic-y +=3D text-patching.h diff --git a/arch/microblaze/include/asm/Kbuild b/arch/microblaze/include/a= sm/Kbuild index 7178f990e8b3..0030309b47ad 100644 --- a/arch/microblaze/include/asm/Kbuild +++ b/arch/microblaze/include/asm/Kbuild @@ -5,6 +5,7 @@ generic-y +=3D extable.h generic-y +=3D kvm_para.h generic-y +=3D mcs_spinlock.h generic-y +=3D parport.h +generic-y +=3D ring_buffer.h generic-y +=3D syscalls.h generic-y +=3D tlb.h generic-y +=3D user.h diff --git a/arch/mips/include/asm/Kbuild b/arch/mips/include/asm/Kbuild index 684569b2ecd6..9771c3d85074 100644 --- a/arch/mips/include/asm/Kbuild +++ b/arch/mips/include/asm/Kbuild @@ -12,5 +12,6 @@ generic-y +=3D mcs_spinlock.h generic-y +=3D parport.h generic-y +=3D qrwlock.h generic-y +=3D qspinlock.h +generic-y +=3D ring_buffer.h generic-y +=3D user.h generic-y +=3D text-patching.h diff --git a/arch/nios2/include/asm/Kbuild b/arch/nios2/include/asm/Kbuild index 28004301c236..0a2530964413 100644 --- a/arch/nios2/include/asm/Kbuild +++ b/arch/nios2/include/asm/Kbuild @@ -5,6 +5,7 @@ generic-y +=3D cmpxchg.h generic-y +=3D extable.h generic-y +=3D kvm_para.h generic-y +=3D mcs_spinlock.h +generic-y +=3D ring_buffer.h generic-y +=3D spinlock.h generic-y +=3D user.h generic-y +=3D text-patching.h diff --git a/arch/openrisc/include/asm/Kbuild b/arch/openrisc/include/asm/K= build index cef49d60d74c..8aa34621702d 100644 --- a/arch/openrisc/include/asm/Kbuild +++ b/arch/openrisc/include/asm/Kbuild @@ -8,4 +8,5 @@ generic-y +=3D spinlock_types.h generic-y +=3D spinlock.h generic-y +=3D qrwlock_types.h generic-y +=3D qrwlock.h +generic-y +=3D ring_buffer.h generic-y +=3D user.h diff --git a/arch/parisc/include/asm/Kbuild b/arch/parisc/include/asm/Kbuild index 4fb596d94c89..d48d158f7241 100644 --- a/arch/parisc/include/asm/Kbuild +++ b/arch/parisc/include/asm/Kbuild @@ -4,4 +4,5 @@ generated-y +=3D syscall_table_64.h generic-y +=3D agp.h generic-y +=3D kvm_para.h generic-y +=3D mcs_spinlock.h +generic-y +=3D ring_buffer.h generic-y +=3D user.h diff --git a/arch/powerpc/include/asm/Kbuild b/arch/powerpc/include/asm/Kbu= ild index 2e23533b67e3..805b5aeebb6f 100644 --- a/arch/powerpc/include/asm/Kbuild +++ b/arch/powerpc/include/asm/Kbuild @@ -5,4 +5,5 @@ generated-y +=3D syscall_table_spu.h generic-y +=3D agp.h generic-y +=3D mcs_spinlock.h generic-y +=3D qrwlock.h +generic-y +=3D ring_buffer.h generic-y +=3D early_ioremap.h diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index bd5fc9403295..7721b63642f4 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -14,5 +14,6 @@ generic-y +=3D ticket_spinlock.h generic-y +=3D qrwlock.h generic-y +=3D qrwlock_types.h generic-y +=3D qspinlock.h +generic-y +=3D ring_buffer.h generic-y +=3D user.h generic-y +=3D vmlinux.lds.h diff --git a/arch/s390/include/asm/Kbuild b/arch/s390/include/asm/Kbuild index 80bad7de7a04..0c1fc47c3ba0 100644 --- a/arch/s390/include/asm/Kbuild +++ b/arch/s390/include/asm/Kbuild @@ -7,3 +7,4 @@ generated-y +=3D unistd_nr.h generic-y +=3D asm-offsets.h generic-y +=3D mcs_spinlock.h generic-y +=3D mmzone.h +generic-y +=3D ring_buffer.h diff --git a/arch/sh/include/asm/Kbuild b/arch/sh/include/asm/Kbuild index 4d3f10ed8275..f0403d3ee8ab 100644 --- a/arch/sh/include/asm/Kbuild +++ b/arch/sh/include/asm/Kbuild @@ -3,4 +3,5 @@ generated-y +=3D syscall_table.h generic-y +=3D kvm_para.h generic-y +=3D mcs_spinlock.h generic-y +=3D parport.h +generic-y +=3D ring_buffer.h generic-y +=3D text-patching.h diff --git a/arch/sparc/include/asm/Kbuild b/arch/sparc/include/asm/Kbuild index 17ee8a273aa6..49c6bb326b75 100644 --- a/arch/sparc/include/asm/Kbuild +++ b/arch/sparc/include/asm/Kbuild @@ -4,4 +4,5 @@ generated-y +=3D syscall_table_64.h generic-y +=3D agp.h generic-y +=3D kvm_para.h generic-y +=3D mcs_spinlock.h +generic-y +=3D ring_buffer.h generic-y +=3D text-patching.h diff --git a/arch/um/include/asm/Kbuild b/arch/um/include/asm/Kbuild index 1b9b82bbe322..2a1629ba8140 100644 --- a/arch/um/include/asm/Kbuild +++ b/arch/um/include/asm/Kbuild @@ -17,6 +17,7 @@ generic-y +=3D module.lds.h generic-y +=3D parport.h generic-y +=3D percpu.h generic-y +=3D preempt.h +generic-y +=3D ring_buffer.h generic-y +=3D runtime-const.h generic-y +=3D softirq_stack.h generic-y +=3D switch_to.h diff --git a/arch/x86/include/asm/Kbuild b/arch/x86/include/asm/Kbuild index 4566000e15c4..078fd2c0d69d 100644 --- a/arch/x86/include/asm/Kbuild +++ b/arch/x86/include/asm/Kbuild @@ -14,3 +14,4 @@ generic-y +=3D early_ioremap.h generic-y +=3D fprobe.h generic-y +=3D mcs_spinlock.h generic-y +=3D mmzone.h +generic-y +=3D ring_buffer.h diff --git a/arch/xtensa/include/asm/Kbuild b/arch/xtensa/include/asm/Kbuild index 13fe45dea296..e57af619263a 100644 --- a/arch/xtensa/include/asm/Kbuild +++ b/arch/xtensa/include/asm/Kbuild @@ -6,5 +6,6 @@ generic-y +=3D mcs_spinlock.h generic-y +=3D parport.h generic-y +=3D qrwlock.h generic-y +=3D qspinlock.h +generic-y +=3D ring_buffer.h generic-y +=3D user.h generic-y +=3D text-patching.h diff --git a/include/asm-generic/ring_buffer.h b/include/asm-generic/ring_b= uffer.h new file mode 100644 index 000000000000..201d2aee1005 --- /dev/null +++ b/include/asm-generic/ring_buffer.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Generic arch dependent ring_buffer macros. + */ +#ifndef __ASM_GENERIC_RING_BUFFER_H__ +#define __ASM_GENERIC_RING_BUFFER_H__ + +#include + +/* Flush cache on ring buffer range if needed. Do nothing by default. */ +#define arch_ring_buffer_flush_range(start, end) do { } while (0) + +#endif /* __ASM_GENERIC_RING_BUFFER_H__ */ diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 2caa5d3d0ae9..4d5817286791 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include #include @@ -31,6 +32,7 @@ #include #include =20 +#include #include #include #include @@ -559,6 +561,7 @@ struct trace_buffer { =20 unsigned long range_addr_start; unsigned long range_addr_end; + struct notifier_block flush_nb; =20 struct ring_buffer_meta *meta; =20 @@ -2520,6 +2523,16 @@ static void rb_free_cpu_buffer(struct ring_buffer_pe= r_cpu *cpu_buffer) kfree(cpu_buffer); } =20 +/* Stop recording on a persistent buffer and flush cache if needed. */ +static int rb_flush_buffer_cb(struct notifier_block *nb, unsigned long eve= nt, void *data) +{ + struct trace_buffer *buffer =3D container_of(nb, struct trace_buffer, flu= sh_nb); + + ring_buffer_record_off(buffer); + arch_ring_buffer_flush_range(buffer->range_addr_start, buffer->range_addr= _end); + return NOTIFY_DONE; +} + static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flag= s, int order, unsigned long start, unsigned long end, @@ -2650,6 +2663,12 @@ static struct trace_buffer *alloc_buffer(unsigned lo= ng size, unsigned flags, =20 mutex_init(&buffer->mutex); =20 + /* Persistent ring buffer needs to flush cache before reboot. */ + if (start && end) { + buffer->flush_nb.notifier_call =3D rb_flush_buffer_cb; + atomic_notifier_chain_register(&panic_notifier_list, &buffer->flush_nb); + } + return_ptr(buffer); =20 fail_free_buffers: @@ -2748,6 +2767,9 @@ ring_buffer_free(struct trace_buffer *buffer) { int cpu; =20 + if (buffer->range_addr_start && buffer->range_addr_end) + atomic_notifier_chain_unregister(&panic_notifier_list, &buffer->flush_nb= ); + cpuhp_state_remove_instance(CPUHP_TRACE_RB_PREPARE, &buffer->node); =20 irq_work_sync(&buffer->irq_work.work); From nobody Mon Apr 6 14:12:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61915327C00; Mon, 6 Apr 2026 10:24:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471077; cv=none; b=eq9YqE5tDRt+K36uqXR1ppJIhFAxriC6ow/HdJp++nHAbnr7Iz2O6WszXUXjC/sp4nziZNQM8Xl9Y38Nj106nC5gfDiOQ0XNgkKVim778mnB8gQUKFsG29WtdOVVB4VUWg/vYGQya84XDc+IBhumn1D/ingw6lJL8zW30WA1DF4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471077; c=relaxed/simple; bh=7n5MYeapY896oPrdhOd97GzM5OgAj9c57Ec4UzMRsps=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=c8xEFuWn+H7cI2soy3xpoJRGery5gwUbTuP0nlKIKgeutsXo8s48ccxC4bpE0b8wAKW8psEeEwYwyeRupotaAcgdSQgbhQ3mX0ZL/dV8TgKOr+ibTRtRgokOGyr33nocUccmMeUFHCSd3HY4CoUp9mjZywZzguoBxlu5Q9ZfhDc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nNOAb9Uy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nNOAb9Uy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB3EEC4CEF7; Mon, 6 Apr 2026 10:24:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775471077; bh=7n5MYeapY896oPrdhOd97GzM5OgAj9c57Ec4UzMRsps=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nNOAb9UyMUyeKgQwzL3FLwy9PdcKn190XPc9230hVDtJxYyxYu3L++pSj2uu/0ozz 3dCKoDwCOH1UBprveuXncikkDyU/nxz84CVf6UaUoEYv2jEAjQLPEWvio3bmq82v19 nk2eBIwXSiYDdiNCNDrsIt6ItAVurCcmYV+4NNYHeqaQBoAI9nHxcVzN6pBzXzvlJ9 UOTwxbk2H0f+xP8hAYAOfWUtBb6sexfmQBjV7kesOFSPgY7f+4QjfGSX7OOmfPO0e8 KLpzRwZ9SOzQXiQNbgysQvep4IMTKD3hWPl7G/iO7447I9hWfNZ4dUsVDfLgB0c3KG hLAAIscZfQweg== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Catalin Marinas , Will Deacon Cc: Masami Hiramatsu , Mathieu Desnoyers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Ian Rogers , linux-arm-kernel@lists.infradead.org Subject: [PATCH v16 2/5] ring-buffer: Skip invalid sub-buffers when validating persistent ring buffer Date: Mon, 6 Apr 2026 19:24:34 +0900 Message-ID: <177547107421.259641.8573640291559263589.stgit@mhiramat.tok.corp.google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog In-Reply-To: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> References: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Skip invalid sub-buffers when validating the persistent ring buffer instead of discarding the entire ring buffer. Only skipped buffers are invalidated (cleared). If the cache data in memory fails to be synchronized during a reboot, the persistent ring buffer may become partially corrupted, but other sub-buffers may still contain readable event data. Only discard the subbuffers that are found to be corrupted. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v15: - Skip reader_page loop check on persistent ring buffer because there can be contiguous empty(invalidated) pages. - Do not show discarded page number information if it is 0. Changes in v11: - Fix a typo. Changes in v9: - Add meta->subbuf_size check. - Fix a typo. - Handle invalid reader_page case. Changes in v8: - Add comment in rb_valudate_buffer() - Clear the RB_MISSED_* flags in rb_valudate_buffer() instead of skipping subbuf. - Remove unused subbuf local variable from rb_cpu_meta_valid(). Changes in v7: - Combined with Handling RB_MISSED_* flags patch, focus on validation at = boot. - Remove checking subbuffer data when validating metadata, because it sho= uld be done later. - Do not mark the discarded sub buffer page but just reset it. Changes in v6: - Show invalid page detection message once per CPU. Changes in v5: - Instead of showing errors for each page, just show the number of discarded pages at last. Changes in v3: - Record missed data event on commit. --- kernel/trace/ring_buffer.c | 109 ++++++++++++++++++++++++++--------------= ---- 1 file changed, 65 insertions(+), 44 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 4d5817286791..0c284094f7d0 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -370,6 +370,12 @@ static __always_inline unsigned int rb_page_commit(str= uct buffer_page *bpage) return local_read(&bpage->page->commit); } =20 +/* Size is determined by what has been committed */ +static __always_inline unsigned int rb_page_size(struct buffer_page *bpage) +{ + return rb_page_commit(bpage) & ~RB_MISSED_MASK; +} + static void free_buffer_page(struct buffer_page *bpage) { /* Range pages are not to be freed */ @@ -1762,7 +1768,6 @@ static bool rb_cpu_meta_valid(struct ring_buffer_cpu_= meta *meta, int cpu, unsigned long *subbuf_mask) { int subbuf_size =3D PAGE_SIZE; - struct buffer_data_page *subbuf; unsigned long buffers_start; unsigned long buffers_end; int i; @@ -1770,6 +1775,11 @@ static bool rb_cpu_meta_valid(struct ring_buffer_cpu= _meta *meta, int cpu, if (!subbuf_mask) return false; =20 + if (meta->subbuf_size !=3D PAGE_SIZE) { + pr_info("Ring buffer boot meta [%d] invalid subbuf_size\n", cpu); + return false; + } + buffers_start =3D meta->first_buffer; buffers_end =3D meta->first_buffer + (subbuf_size * meta->nr_subbufs); =20 @@ -1786,11 +1796,12 @@ static bool rb_cpu_meta_valid(struct ring_buffer_cp= u_meta *meta, int cpu, return false; } =20 - subbuf =3D rb_subbufs_from_meta(meta); - bitmap_clear(subbuf_mask, 0, meta->nr_subbufs); =20 - /* Is the meta buffers and the subbufs themselves have correct data? */ + /* + * Ensure the meta::buffers array has correct data. The data in each subb= ufs + * are checked later in rb_meta_validate_events(). + */ for (i =3D 0; i < meta->nr_subbufs; i++) { if (meta->buffers[i] < 0 || meta->buffers[i] >=3D meta->nr_subbufs) { @@ -1798,18 +1809,12 @@ static bool rb_cpu_meta_valid(struct ring_buffer_cp= u_meta *meta, int cpu, return false; } =20 - if ((unsigned)local_read(&subbuf->commit) > subbuf_size) { - pr_info("Ring buffer boot meta [%d] buffer invalid commit\n", cpu); - return false; - } - if (test_bit(meta->buffers[i], subbuf_mask)) { pr_info("Ring buffer boot meta [%d] array has duplicates\n", cpu); return false; } =20 set_bit(meta->buffers[i], subbuf_mask); - subbuf =3D (void *)subbuf + subbuf_size; } =20 return true; @@ -1873,13 +1878,22 @@ static int rb_read_data_buffer(struct buffer_data_p= age *dpage, int tail, int cpu return events; } =20 -static int rb_validate_buffer(struct buffer_data_page *dpage, int cpu) +static int rb_validate_buffer(struct buffer_data_page *dpage, int cpu, + struct ring_buffer_cpu_meta *meta) { unsigned long long ts; + unsigned long tail; u64 delta; - int tail; =20 - tail =3D local_read(&dpage->commit); + /* + * When a sub-buffer is recovered from a read, the commit value may + * have RB_MISSED_* bits set, as these bits are reset on reuse. + * Even after clearing these bits, a commit value greater than the + * subbuf_size is considered invalid. + */ + tail =3D local_read(&dpage->commit) & ~RB_MISSED_MASK; + if (tail > meta->subbuf_size) + return -1; return rb_read_data_buffer(dpage, tail, cpu, &ts, &delta); } =20 @@ -1890,6 +1904,7 @@ static void rb_meta_validate_events(struct ring_buffe= r_per_cpu *cpu_buffer) struct buffer_page *head_page, *orig_head; unsigned long entry_bytes =3D 0; unsigned long entries =3D 0; + int discarded =3D 0; int ret; u64 ts; int i; @@ -1900,14 +1915,19 @@ static void rb_meta_validate_events(struct ring_buf= fer_per_cpu *cpu_buffer) orig_head =3D head_page =3D cpu_buffer->head_page; =20 /* Do the reader page first */ - ret =3D rb_validate_buffer(cpu_buffer->reader_page->page, cpu_buffer->cpu= ); + ret =3D rb_validate_buffer(cpu_buffer->reader_page->page, cpu_buffer->cpu= , meta); if (ret < 0) { - pr_info("Ring buffer reader page is invalid\n"); - goto invalid; + pr_info("Ring buffer meta [%d] invalid reader page detected\n", + cpu_buffer->cpu); + discarded++; + /* Instead of discard whole ring buffer, discard only this sub-buffer. */ + local_set(&cpu_buffer->reader_page->entries, 0); + local_set(&cpu_buffer->reader_page->page->commit, 0); + } else { + entries +=3D ret; + entry_bytes +=3D rb_page_size(cpu_buffer->reader_page); + local_set(&cpu_buffer->reader_page->entries, ret); } - entries +=3D ret; - entry_bytes +=3D local_read(&cpu_buffer->reader_page->page->commit); - local_set(&cpu_buffer->reader_page->entries, ret); =20 ts =3D head_page->page->time_stamp; =20 @@ -1935,7 +1955,7 @@ static void rb_meta_validate_events(struct ring_buffe= r_per_cpu *cpu_buffer) break; =20 /* Stop rewind if the page is invalid. */ - ret =3D rb_validate_buffer(head_page->page, cpu_buffer->cpu); + ret =3D rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta); if (ret < 0) break; =20 @@ -2014,21 +2034,24 @@ static void rb_meta_validate_events(struct ring_buf= fer_per_cpu *cpu_buffer) if (head_page =3D=3D cpu_buffer->reader_page) continue; =20 - ret =3D rb_validate_buffer(head_page->page, cpu_buffer->cpu); + ret =3D rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta); if (ret < 0) { - pr_info("Ring buffer meta [%d] invalid buffer page\n", - cpu_buffer->cpu); - goto invalid; - } - - /* If the buffer has content, update pages_touched */ - if (ret) - local_inc(&cpu_buffer->pages_touched); - - entries +=3D ret; - entry_bytes +=3D local_read(&head_page->page->commit); - local_set(&head_page->entries, ret); + if (!discarded) + pr_info("Ring buffer meta [%d] invalid buffer page detected\n", + cpu_buffer->cpu); + discarded++; + /* Instead of discard whole ring buffer, discard only this sub-buffer. = */ + local_set(&head_page->entries, 0); + local_set(&head_page->page->commit, 0); + } else { + /* If the buffer has content, update pages_touched */ + if (ret) + local_inc(&cpu_buffer->pages_touched); =20 + entries +=3D ret; + entry_bytes +=3D rb_page_size(head_page); + local_set(&head_page->entries, ret); + } if (head_page =3D=3D cpu_buffer->commit_page) break; } @@ -2042,7 +2065,10 @@ static void rb_meta_validate_events(struct ring_buff= er_per_cpu *cpu_buffer) local_set(&cpu_buffer->entries, entries); local_set(&cpu_buffer->entries_bytes, entry_bytes); =20 - pr_info("Ring buffer meta [%d] is from previous boot!\n", cpu_buffer->cpu= ); + pr_info("Ring buffer meta [%d] is from previous boot!", cpu_buffer->cpu); + if (discarded) + pr_cont(" (%d pages discarded)", discarded); + pr_cont("\n"); return; =20 invalid: @@ -3329,12 +3355,6 @@ rb_iter_head_event(struct ring_buffer_iter *iter) return NULL; } =20 -/* Size is determined by what has been committed */ -static __always_inline unsigned rb_page_size(struct buffer_page *bpage) -{ - return rb_page_commit(bpage) & ~RB_MISSED_MASK; -} - static __always_inline unsigned rb_commit_index(struct ring_buffer_per_cpu *cpu_buffer) { @@ -5647,11 +5667,12 @@ __rb_get_reader_page(struct ring_buffer_per_cpu *cp= u_buffer) again: /* * This should normally only loop twice. But because the - * start of the reader inserts an empty page, it causes - * a case where we will loop three times. There should be no - * reason to loop four times (that I know of). + * start of the reader inserts an empty page, it causes a + * case where we will loop three times. There should be no + * reason to loop four times unless the ring buffer is a + * recovered persistent ring buffer. */ - if (RB_WARN_ON(cpu_buffer, ++nr_loops > 3)) { + if (RB_WARN_ON(cpu_buffer, ++nr_loops > 3 && !cpu_buffer->ring_meta)) { reader =3D NULL; goto out; } From nobody Mon Apr 6 14:12:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 874132E6116; Mon, 6 Apr 2026 10:24:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471085; cv=none; b=CS/5wfmL7Xu93nVdmPmI2vBc8bEQ75EiCYVfv9yQRM7Z3nb45JNW8uxW+4Z+3J4KBDTYy4w9RatIBlsFvWrrbkqkIGTO/h4vDabhCC5myPA9zsx++i1+Fyk3fJc5J36Sg4wa6VdusbpJ/l1pvLgwTd64sAccW2lQYaWrQIYyuL0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471085; c=relaxed/simple; bh=IiX3iAIS0Tud3lHL61RRs72AmWZT7+Pt71cqRKLfPd8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hfjj4LJJKUCRKq4LGpiRhl0FHJac8D6pOgMKoWngh1NySSeAz0j4PYS4vREHdw6YqOVF5KbG5iEo/WZEtRDzZPA/bdtR/JuoZEQMCh7g9nT1AQrMzF26cfIPlLRwKPyXk72l0OzyZiMEEuLOtXBkHcChuot/KoFoT+sW6/g1vkc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fAQwZ+Nj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fAQwZ+Nj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF6F4C2BCB0; Mon, 6 Apr 2026 10:24:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775471085; bh=IiX3iAIS0Tud3lHL61RRs72AmWZT7+Pt71cqRKLfPd8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fAQwZ+Nj7rABoCnI6LeXzjUECnpc2cUoMog+F0vIVKrkR+KzNG8gjeiyUky816xjc Rn4jl2h+2TW0827t5fEmkN2w7tHpF/BUfoRwXcZf0+anIH1m5vSLMR2XbDFK1bM/fg axreEg9cUAfGcTK6tLXDAPphLjJXwsbDFVce1Qja165dVaHK+P+QDZQ4xztA0rqI9a ys6nVfjXdXeJz6Ghr5TS8kF2gOYvTRw9lyQNmraHclZzIu9Jok8KMxhMXGN84PiLsU CPQruPMEZmjPa+DBL8wPqAO6BzuLCtPsX9JahXePQ9oRshYT3L04O3jKyNBtw54jfP MwFdfMaF6t4CQ== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Catalin Marinas , Will Deacon Cc: Masami Hiramatsu , Mathieu Desnoyers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Ian Rogers , linux-arm-kernel@lists.infradead.org Subject: [PATCH v16 3/5] ring-buffer: Skip invalid sub-buffers when rewinding persistent ring buffer Date: Mon, 6 Apr 2026 19:24:42 +0900 Message-ID: <177547108231.259641.7993975668235545903.stgit@mhiramat.tok.corp.google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog In-Reply-To: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> References: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Skip invalid sub-buffers when rewinding the persistent ring buffer instead of stopping the rewinding the ring buffer. The skipped buffers are cleared. To ensure the rewinding stops at the unused page, this also clears buffer_data_page::time_stamp when tracing resets the buffer. This allows us to identify unused pages and empty pages. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v12: - Fix build error. Changes in v11: - Reset timestamp when the buffer is invalid. - When rewinding, skip subbuf page if timestamp is wrong and check timestamp after validating buffer data page. Changes in v10: - Newly added. --- kernel/trace/ring_buffer.c | 76 +++++++++++++++++++++++++---------------= ---- 1 file changed, 43 insertions(+), 33 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 0c284094f7d0..518a05df6ef7 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -363,6 +363,7 @@ struct buffer_page { static void rb_init_page(struct buffer_data_page *bpage) { local_set(&bpage->commit, 0); + bpage->time_stamp =3D 0; } =20 static __always_inline unsigned int rb_page_commit(struct buffer_page *bpa= ge) @@ -1878,12 +1879,14 @@ static int rb_read_data_buffer(struct buffer_data_p= age *dpage, int tail, int cpu return events; } =20 -static int rb_validate_buffer(struct buffer_data_page *dpage, int cpu, +static int rb_validate_buffer(struct buffer_page *bpage, int cpu, struct ring_buffer_cpu_meta *meta) { + struct buffer_data_page *dpage =3D bpage->page; unsigned long long ts; unsigned long tail; u64 delta; + int ret =3D -1; =20 /* * When a sub-buffer is recovered from a read, the commit value may @@ -1892,9 +1895,17 @@ static int rb_validate_buffer(struct buffer_data_pag= e *dpage, int cpu, * subbuf_size is considered invalid. */ tail =3D local_read(&dpage->commit) & ~RB_MISSED_MASK; - if (tail > meta->subbuf_size) - return -1; - return rb_read_data_buffer(dpage, tail, cpu, &ts, &delta); + if (tail <=3D meta->subbuf_size) + ret =3D rb_read_data_buffer(dpage, tail, cpu, &ts, &delta); + + if (ret < 0) { + local_set(&bpage->entries, 0); + local_set(&bpage->page->commit, 0); + } else { + local_set(&bpage->entries, ret); + } + + return ret; } =20 /* If the meta data has been validated, now validate the events */ @@ -1915,18 +1926,14 @@ static void rb_meta_validate_events(struct ring_buf= fer_per_cpu *cpu_buffer) orig_head =3D head_page =3D cpu_buffer->head_page; =20 /* Do the reader page first */ - ret =3D rb_validate_buffer(cpu_buffer->reader_page->page, cpu_buffer->cpu= , meta); + ret =3D rb_validate_buffer(cpu_buffer->reader_page, cpu_buffer->cpu, meta= ); if (ret < 0) { pr_info("Ring buffer meta [%d] invalid reader page detected\n", cpu_buffer->cpu); discarded++; - /* Instead of discard whole ring buffer, discard only this sub-buffer. */ - local_set(&cpu_buffer->reader_page->entries, 0); - local_set(&cpu_buffer->reader_page->page->commit, 0); } else { entries +=3D ret; entry_bytes +=3D rb_page_size(cpu_buffer->reader_page); - local_set(&cpu_buffer->reader_page->entries, ret); } =20 ts =3D head_page->page->time_stamp; @@ -1945,26 +1952,33 @@ static void rb_meta_validate_events(struct ring_buf= fer_per_cpu *cpu_buffer) if (head_page =3D=3D cpu_buffer->tail_page) break; =20 - /* Ensure the page has older data than head. */ - if (ts < head_page->page->time_stamp) - break; - - ts =3D head_page->page->time_stamp; - /* Ensure the page has correct timestamp and some data. */ - if (!ts || rb_page_commit(head_page) =3D=3D 0) - break; - - /* Stop rewind if the page is invalid. */ - ret =3D rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta); - if (ret < 0) + /* Rewind until unused page (no timestamp, no commit). */ + if (!head_page->page->time_stamp && rb_page_commit(head_page) =3D=3D 0) break; =20 - /* Recover the number of entries and update stats. */ - local_set(&head_page->entries, ret); - if (ret) - local_inc(&cpu_buffer->pages_touched); - entries +=3D ret; - entry_bytes +=3D rb_page_commit(head_page); + /* + * Skip if the page is invalid, or its timestamp is newer than the + * previous valid page. + */ + ret =3D rb_validate_buffer(head_page, cpu_buffer->cpu, meta); + if (ret >=3D 0 && ts < head_page->page->time_stamp) { + local_set(&head_page->entries, 0); + local_set(&head_page->page->commit, 0); + head_page->page->time_stamp =3D ts; + ret =3D -1; + } + if (ret < 0) { + if (!discarded) + pr_info("Ring buffer meta [%d] invalid buffer page detected\n", + cpu_buffer->cpu); + discarded++; + } else { + entries +=3D ret; + entry_bytes +=3D rb_page_size(head_page); + if (ret > 0) + local_inc(&cpu_buffer->pages_touched); + ts =3D head_page->page->time_stamp; + } } if (i) pr_info("Ring buffer [%d] rewound %d pages\n", cpu_buffer->cpu, i); @@ -2034,15 +2048,12 @@ static void rb_meta_validate_events(struct ring_buf= fer_per_cpu *cpu_buffer) if (head_page =3D=3D cpu_buffer->reader_page) continue; =20 - ret =3D rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta); + ret =3D rb_validate_buffer(head_page, cpu_buffer->cpu, meta); if (ret < 0) { if (!discarded) pr_info("Ring buffer meta [%d] invalid buffer page detected\n", cpu_buffer->cpu); discarded++; - /* Instead of discard whole ring buffer, discard only this sub-buffer. = */ - local_set(&head_page->entries, 0); - local_set(&head_page->page->commit, 0); } else { /* If the buffer has content, update pages_touched */ if (ret) @@ -2050,7 +2061,6 @@ static void rb_meta_validate_events(struct ring_buffe= r_per_cpu *cpu_buffer) =20 entries +=3D ret; entry_bytes +=3D rb_page_size(head_page); - local_set(&head_page->entries, ret); } if (head_page =3D=3D cpu_buffer->commit_page) break; @@ -2083,7 +2093,7 @@ static void rb_meta_validate_events(struct ring_buffe= r_per_cpu *cpu_buffer) /* Reset all the subbuffers */ for (i =3D 0; i < meta->nr_subbufs - 1; i++, rb_inc_page(&head_page)) { local_set(&head_page->entries, 0); - local_set(&head_page->page->commit, 0); + rb_init_page(head_page->page); } } From nobody Mon Apr 6 14:12:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63ABD322B7D; Mon, 6 Apr 2026 10:24:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471093; cv=none; b=K1wZv1JMFgo2fCRMtygeMszm45kauB/KALEnX/Dt7n541+4UHYml1wFMkpaGac9i48yVjO0qd2LsnR601n6kQ5Z5bdw99EEVDRXT3PePX29EZbVJVkC0+CFhknvs9rzXCEvtctRYHu36yWH4g1CSpeS7hKrI0G2G5LukFyYbr+o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471093; c=relaxed/simple; bh=uwmZ+5HNv6rXMmJbE2yt2zDuVwlMOrqPvHcFQxOh690=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=JJUiC96Q7LoTrhP2wOj48z8zCIEVUYICuLfTmOeCKBkJABhIAy/W4dY1JWFUEpyVFpgclRr+xHBIHRI7v0EYti4vvECD6F8crABOPOcB5cIJskIGirA5WPUGuwtM31UdHQVKV9BY3bEQtCbqOLmlq6T3NhYvyRunkx2gM3srbD8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VWzpYJO4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VWzpYJO4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1362EC4CEF7; Mon, 6 Apr 2026 10:24:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775471093; bh=uwmZ+5HNv6rXMmJbE2yt2zDuVwlMOrqPvHcFQxOh690=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VWzpYJO4yzaSmcF8AMElKPIrom1+8uNXFCfGH5WRIN+4HtJXkoLm4/kKu4U8XnahP l38DgLnd6mwqvSaRm5a6vSkHc0RAFALzEmHHoDBcvktOACyik7+meySabzDN5IjOZ2 pts0Oatp0Jl8Y6RELeiybh4ZE0mwIm/fUuU4bFGYnkRA4O94BF9c3fUnVnGMksZqC/ KL3/WW2K8yIuGV9WUXGr1Xige+97IlKU7p7XC1uebD7grESAWsYI7hS/jWfxvwO7Jv mBg1YsGBKGdlBrH/WdT7wvjADKebpZrcwpXAv2dh8K6E78SgRP0Z+KMwLor9nyWu+K 0eMu2eH4zJL9g== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Catalin Marinas , Will Deacon Cc: Masami Hiramatsu , Mathieu Desnoyers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Ian Rogers , linux-arm-kernel@lists.infradead.org Subject: [PATCH v16 4/5] ring-buffer: Add persistent ring buffer invalid-page inject test Date: Mon, 6 Apr 2026 19:24:50 +0900 Message-ID: <177547109039.259641.4525585553763534144.stgit@mhiramat.tok.corp.google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog In-Reply-To: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> References: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Add a self-corrupting test for the persistent ring buffer. This will inject an erroneous value to some sub-buffer pages (where the index is even or multiples of 5) in the persistent ring buffer when the kernel panics, and checks whether the number of detected invalid pages and the total entry_bytes are the same as the recorded values after reboot. This ensures that the kernel can correctly recover a partially corrupted persistent ring buffer after a reboot or panic. The test only runs on the persistent ring buffer whose name is "ptracingtest". The user has to fill it with events before a kernel panic. To run the test, enable CONFIG_RING_BUFFER_PERSISTENT_INJECT and add the following kernel cmdline: reserve_mem=3D20M:2M:trace trace_instance=3Dptracingtest^traceoff@trace panic=3D1 Run the following commands after the 1st boot: cd /sys/kernel/tracing/instances/ptracingtest echo 1 > tracing_on echo 1 > events/enable sleep 3 echo c > /proc/sysrq-trigger After panic message, the kernel will reboot and run the verification on the persistent ring buffer, e.g. Ring buffer meta [2] invalid buffer page detected Ring buffer meta [2] is from previous boot! (318 pages discarded) Ring buffer testing [2] invalid pages: PASSED (318/318) Ring buffer testing [2] entry_bytes: PASSED (1300476/1300476) Signed-off-by: Masami Hiramatsu (Google) --- Changes in v16: - Update description and comments according to review comments. Changes in v15: - Use pr_warn() for test result. - Inject errors on the page index is multiples of 5 so that this can reproduce contiguous empty pages. Changes in v14: - Rename config to CONFIG_RING_BUFFER_PERSISTENT_INJECT. - Clear meta->nr_invalid/entry_bytes after testing. - Add test commands in config comment. Changes in v10: - Add entry_bytes test. - Do not compile test code if CONFIG_RING_BUFFER_PERSISTENT_SELFTEST=3Dn. Changes in v9: - Test also reader pages. --- include/linux/ring_buffer.h | 1 + kernel/trace/Kconfig | 34 ++++++++++++++++++++ kernel/trace/ring_buffer.c | 74 +++++++++++++++++++++++++++++++++++++++= ++++ kernel/trace/trace.c | 4 ++ 4 files changed, 113 insertions(+) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index 994f52b34344..0670742b2d60 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -238,6 +238,7 @@ int ring_buffer_subbuf_size_get(struct trace_buffer *bu= ffer); =20 enum ring_buffer_flags { RB_FL_OVERWRITE =3D 1 << 0, + RB_FL_TESTING =3D 1 << 1, }; =20 #ifdef CONFIG_RING_BUFFER diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index e130da35808f..084f34dc6c9f 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -1202,6 +1202,40 @@ config RING_BUFFER_VALIDATE_TIME_DELTAS Only say Y if you understand what this does, and you still want it enabled. Otherwise say N =20 +config RING_BUFFER_PERSISTENT_INJECT + bool "Enable persistent ring buffer error injection test" + depends on RING_BUFFER + help + This option will have the kernel check if the persistent ring + buffer is named "ptracingtest". and if so, it will corrupt some + of its pages on a kernel panic. This is used to test if the + persistent ring buffer can recover from some of its sub-buffers + being corrupted. + To use this, boot a kernel with a "ptracingtest" persistent + ring buffer, e.g. + + reserve_mem=3D20M:2M:trace trace_instance=3Dptracingtest@trace panic= =3D1 + + And after the 1st boot, run the following commands: + + cd /sys/kernel/tracing/instances/ptracingtest + echo 1 > events/enable + echo 1 > tracing_on + sleep 3 + echo c > /proc/sysrq-trigger + + After the panic message, the kernel will reboot and will show + the test results in the console output. + + Note that events for the test ring buffer needs to be enabled + prior to crashing the kernel so that the ring buffer has content + that the test will corrupt. + As the test will corrupt events in the "ptracingtest" persistent + ring buffer, it should not be used for any other purpose other + than this test. + + If unsure, say N + config MMIOTRACE_TEST tristate "Test module for mmiotrace" depends on MMIOTRACE && m diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 518a05df6ef7..e56fe9dcc7d7 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -64,6 +64,10 @@ struct ring_buffer_cpu_meta { unsigned long commit_buffer; __u32 subbuf_size; __u32 nr_subbufs; +#ifdef CONFIG_RING_BUFFER_PERSISTENT_INJECT + __u32 nr_invalid; + __u32 entry_bytes; +#endif int buffers[]; }; =20 @@ -2079,6 +2083,21 @@ static void rb_meta_validate_events(struct ring_buff= er_per_cpu *cpu_buffer) if (discarded) pr_cont(" (%d pages discarded)", discarded); pr_cont("\n"); + +#ifdef CONFIG_RING_BUFFER_PERSISTENT_INJECT + if (meta->nr_invalid) + pr_warn("Ring buffer testing [%d] invalid pages: %s (%d/%d)\n", + cpu_buffer->cpu, + (discarded =3D=3D meta->nr_invalid) ? "PASSED" : "FAILED", + discarded, meta->nr_invalid); + if (meta->entry_bytes) + pr_warn("Ring buffer testing [%d] entry_bytes: %s (%ld/%ld)\n", + cpu_buffer->cpu, + (entry_bytes =3D=3D meta->entry_bytes) ? "PASSED" : "FAILED", + (long)entry_bytes, (long)meta->entry_bytes); + meta->nr_invalid =3D 0; + meta->entry_bytes =3D 0; +#endif return; =20 invalid: @@ -2559,12 +2578,67 @@ static void rb_free_cpu_buffer(struct ring_buffer_p= er_cpu *cpu_buffer) kfree(cpu_buffer); } =20 +#ifdef CONFIG_RING_BUFFER_PERSISTENT_INJECT +static void rb_test_inject_invalid_pages(struct trace_buffer *buffer) +{ + struct ring_buffer_per_cpu *cpu_buffer; + struct ring_buffer_cpu_meta *meta; + struct buffer_data_page *dpage; + u32 entry_bytes =3D 0; + unsigned long ptr; + int subbuf_size; + int invalid =3D 0; + int cpu; + int i; + + if (!(buffer->flags & RB_FL_TESTING)) + return; + + guard(preempt)(); + cpu =3D smp_processor_id(); + + cpu_buffer =3D buffer->buffers[cpu]; + meta =3D cpu_buffer->ring_meta; + ptr =3D (unsigned long)rb_subbufs_from_meta(meta); + subbuf_size =3D meta->subbuf_size; + + for (i =3D 0; i < meta->nr_subbufs; i++) { + int idx =3D meta->buffers[i]; + + dpage =3D (void *)(ptr + idx * subbuf_size); + /* Skip unused pages */ + if (!local_read(&dpage->commit)) + continue; + + /* + * Invalidate even pages or multiples of 5. This will cause 3 + * contiguous invalidated(empty) pages. + */ + if (!(i & 0x1) || !(i % 5)) { + local_add(subbuf_size + 1, &dpage->commit); + invalid++; + } else { + /* Count total commit bytes. */ + entry_bytes +=3D local_read(&dpage->commit); + } + } + + pr_info("Inject invalidated %d pages on CPU%d, total size: %ld\n", + invalid, cpu, (long)entry_bytes); + meta->nr_invalid =3D invalid; + meta->entry_bytes =3D entry_bytes; +} +#else /* !CONFIG_RING_BUFFER_PERSISTENT_INJECT */ +#define rb_test_inject_invalid_pages(buffer) do { } while (0) +#endif + /* Stop recording on a persistent buffer and flush cache if needed. */ static int rb_flush_buffer_cb(struct notifier_block *nb, unsigned long eve= nt, void *data) { struct trace_buffer *buffer =3D container_of(nb, struct trace_buffer, flu= sh_nb); =20 ring_buffer_record_off(buffer); + rb_test_inject_invalid_pages(buffer); arch_ring_buffer_flush_range(buffer->range_addr_start, buffer->range_addr= _end); return NOTIFY_DONE; } diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index e9455d46ec16..96101d276d13 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -9436,6 +9436,8 @@ static void setup_trace_scratch(struct trace_array *t= r, memset(tscratch, 0, size); } =20 +#define TRACE_TEST_PTRACING_NAME "ptracingtest" + static int allocate_trace_buffer(struct trace_array *tr, struct array_buffer *buf, un= signed long size) { @@ -9448,6 +9450,8 @@ allocate_trace_buffer(struct trace_array *tr, struct = array_buffer *buf, unsigned buf->tr =3D tr; =20 if (tr->range_addr_start && tr->range_addr_size) { + if (!strcmp(tr->name, TRACE_TEST_PTRACING_NAME)) + rb_flags |=3D RB_FL_TESTING; /* Add scratch buffer to handle 128 modules */ buf->buffer =3D ring_buffer_alloc_range(size, rb_flags, 0, tr->range_addr_start, From nobody Mon Apr 6 14:12:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE559323417; Mon, 6 Apr 2026 10:25:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471101; cv=none; b=WKnj1ygfLEHAlwGrIc/wPH0rha/8gbEwt7Ed3bdOn6sa2KNmCvlKhAdcQzVeTtvcsD7JPQGifi+JAPOCjo8sFRaP3TrtP5/hVNw5JVrB7kDL6cW1fykXfZ05xo5ie6RakpLNR7t5ASAN7AY//gkzWydqMqvrVU4pjLYo7NgQlIU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775471101; c=relaxed/simple; bh=04xzV7B6C0nmMB8kee/matN9EpL075hjg5Q3i9FX6OQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hKr6zczrA/lIg5gR6w8HcXWDk3T1Oatj2eSpZnMQHQhDU9uIsN/mxXLZukcJys7bn0jV6Lgcp6dTQJVusBT5i2BxyjCRAfM/ZhbRyG+TTt8mqBDJLaI/kJxhF7mIy6HKyMCka8PhUmB3axHcQrjH5kP9VBewYgaD6U33Hn0/A9g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=E1IclHoM; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="E1IclHoM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 31C97C4CEF7; Mon, 6 Apr 2026 10:25:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775471101; bh=04xzV7B6C0nmMB8kee/matN9EpL075hjg5Q3i9FX6OQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E1IclHoMlFg67kG65rpRpLwRfbHVTs5ZkORk+pxRgxrLE6xWaZaEK9qubaRpMj65P f/OGyw/MgKXaG/NFtYfRFiXZSwjDWcYVsIG/W30FztV9pX+BujkcJgacqOKk9NFeJ7 VFegw/3cISJgzNviEx5LSCxie7aeaQyyJeOr0mue+XQ2Mpc6AXovhmQYHS9uuy88Oc an5emKPHvsbwKKBTayqWiBQJ74vzoQwMwrNOWtU/2knEHFuy7sA4/siCeR8IT2iNPS myVuTBER+U4H3MW5DsJsI639seQ5ueHXEbCJn+KQnooRbEu+itgjW/wsCYUop3L54D iqAc6CTjWzMNQ== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Catalin Marinas , Will Deacon Cc: Masami Hiramatsu , Mathieu Desnoyers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Ian Rogers , linux-arm-kernel@lists.infradead.org Subject: [PATCH v16 5/5] ring-buffer: Show commit numbers in buffer_meta file Date: Mon, 6 Apr 2026 19:24:58 +0900 Message-ID: <177547109851.259641.2033773321493831101.stgit@mhiramat.tok.corp.google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog In-Reply-To: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> References: <177547105523.259641.14385891517704197263.stgit@mhiramat.tok.corp.google.com> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) In addition to the index number, show the commit numbers of each data page in the per_cpu buffer_meta file. This is useful for understanding the current status of the persistent ring buffer. (Note that this file is shown only for persistent ring buffer and its backup instance) Signed-off-by: Masami Hiramatsu (Google) --- Changes in v16: - update description. --- kernel/trace/ring_buffer.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index e56fe9dcc7d7..4bf83b7805da 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -2209,6 +2209,7 @@ static int rbm_show(struct seq_file *m, void *v) struct ring_buffer_per_cpu *cpu_buffer =3D m->private; struct ring_buffer_cpu_meta *meta =3D cpu_buffer->ring_meta; unsigned long val =3D (unsigned long)v; + struct buffer_data_page *dpage; =20 if (val =3D=3D 1) { seq_printf(m, "head_buffer: %d\n", @@ -2221,7 +2222,9 @@ static int rbm_show(struct seq_file *m, void *v) } =20 val -=3D 2; - seq_printf(m, "buffer[%ld]: %d\n", val, meta->buffers[val]); + dpage =3D rb_range_buffer(cpu_buffer, val); + seq_printf(m, "buffer[%ld]: %d (commit: %ld)\n", + val, meta->buffers[val], local_read(&dpage->commit)); =20 return 0; }