From nobody Sun Jun 14 17:31:44 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0C36296BBC for ; Sat, 4 Apr 2026 02:25:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269533; cv=none; b=hpaS6STlbzTr0kNg4DBWeNU7QJfLIPyS/iz8KKsLo6iHGgFjaqXAwh3d6vWiFeGfnyPd6fcOWzKqmzkRES9d0sH3nVXNYn7FPQ5GsdiehKUnU5b+nAJYh6fMH6AtvzsaQXzEsaRDY9msRnp6K+Fhyo6PcBJWTowxwZYppDZzE3E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269533; c=relaxed/simple; bh=f693D/SPUG6YUzBtB3hmnFTfWKBWezOeD+zZ1c2CXF0=; h=Message-ID:Date:From:To:Cc:Subject:References:MIME-Version: Content-Type; b=d8N/5xQUY0seU1f9zXC54MLnwfL5NvuG8plNt2D5BJ00yMFMJnOal473hsYdLiH7oavV1tIB11ziQ+NguWrz+1zEiAoTNQMCF87nvGDVTo4yxmCnM3L0pSzCKZ4IOV0SO/y7qZFI1hW9vfUrnuxJ4e/PembUZauuIMeAamb1Zdk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BkM7myjS; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BkM7myjS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5B13BC19423; Sat, 4 Apr 2026 02:25:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775269533; bh=f693D/SPUG6YUzBtB3hmnFTfWKBWezOeD+zZ1c2CXF0=; h=Date:From:To:Cc:Subject:References:From; b=BkM7myjSJVYssLF7AOpvkHMfCvOCZDGbgFGOoM7MI5Gc3DRRJgQFiDBUkhlKbsjsc wlhYrcNpLyjyh9KZywzw3e6U+2RWqy6SvhNAy1NRiW0m3XNmVAp+ClwaEycIofG86l LIIuCDZDiNlpYv/QYfZZdPzi0tsEL9+kLR5FmfydLtiOe8DmDihCnoSpf+cbqst8f8 hqqZ3nBY9ZfaFUEANu4BRgo1HGr9Y//fjdTOhKC0jXctpZBGw0dVYp/NJ7PohGHB7W 7ytjtMy6ibPIJXnzwlYAkdzDM+/EJ6Wss/X5unpK3Zp5v1NN0r6dDJedGelKKJLACC Zs9r1RnVK4tXg== Received: from rostedt by gandalf with local (Exim 4.99.1) (envelope-from ) id 1w8qid-00000001642-2se5; Fri, 03 Apr 2026 22:26:39 -0400 Message-ID: <20260404022639.546384239@kernel.org> User-Agent: quilt/0.69 Date: Fri, 03 Apr 2026 22:26:18 -0400 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton Subject: [for-next][PATCH 1/6] ring-buffer: Show what clock function is used on timestamp errors References: <20260404022617.436859059@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Steven Rostedt The testing for tracing was triggering a timestamp count issue that was always off by one. This has been happening for some time but has never been reported by anyone else. It was finally discovered to be an issue with the "uptime" (jiffies) clock that happened to be traced and the internal recursion caused the discrepancy. This would have been much easier to solve if the clock function being used was displayed when the error was detected. Add the clock function to the error output. Cc: Mathieu Desnoyers Link: https://patch.msgid.link/20260323202212.479bb288@gandalf.local.home Acked-by: Masami Hiramatsu (Google) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index d6bebb782efc..8b6c39bba56d 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -4506,18 +4506,20 @@ static void check_buffer(struct ring_buffer_per_cpu= *cpu_buffer, ret =3D rb_read_data_buffer(bpage, tail, cpu_buffer->cpu, &ts, &delta); if (ret < 0) { if (delta < ts) { - buffer_warn_return("[CPU: %d]ABSOLUTE TIME WENT BACKWARDS: last ts: %ll= d absolute ts: %lld\n", - cpu_buffer->cpu, ts, delta); + buffer_warn_return("[CPU: %d]ABSOLUTE TIME WENT BACKWARDS: last ts: %ll= d absolute ts: %lld clock:%pS\n", + cpu_buffer->cpu, ts, delta, + cpu_buffer->buffer->clock); goto out; } } if ((full && ts > info->ts) || (!full && ts + info->delta !=3D info->ts)) { - buffer_warn_return("[CPU: %d]TIME DOES NOT MATCH expected:%lld actual:%l= ld delta:%lld before:%lld after:%lld%s context:%s\n", + buffer_warn_return("[CPU: %d]TIME DOES NOT MATCH expected:%lld actual:%l= ld delta:%lld before:%lld after:%lld%s context:%s\ntrace clock:%pS", cpu_buffer->cpu, ts + info->delta, info->ts, info->delta, info->before, info->after, - full ? " (full)" : "", show_interrupt_level()); + full ? " (full)" : "", show_interrupt_level(), + cpu_buffer->buffer->clock); } out: atomic_dec(this_cpu_ptr(&checking)); --=20 2.51.0 From nobody Sun Jun 14 17:31:44 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0CA4296BCB for ; Sat, 4 Apr 2026 02:25:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269533; cv=none; b=B6/cV9SGQnKm94rSaT04ZN+m5aFnIXCU+pZYULuQR41+wRedsZyKNwNSKhCdp3d97r1X+5aYGRSioaJyLTVPydwrxSHDdXg1twNrUUqKWXjMNDspj/YGuP3/fr8vMuWu2ME7J4aI/XAgJ9AJFB52H4MNfAXqB5MNMMsJU5xQ6EM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269533; c=relaxed/simple; bh=M5yBLEkfX8xBSUWEh0U5haPjD0m0eKWo0usQWtS3kCU=; h=Message-ID:Date:From:To:Cc:Subject:References:MIME-Version: Content-Type; b=nPt3z13U/ju+2pMxuQzd4ozR/Na+CZepPnc71MNlvj1YmUG073k3YLoVJ2NMW16YJycnP2aSnN3iIPC2eHLa31G+42f2vxh/ts61NVBq0uhRfZyna30XWee0U/4ZTit6fE3YH9Y6OpSg0cdIkDCJrmTjw2SEquSDUtsHEb4oXCs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=r/SWHVVK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="r/SWHVVK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7261FC2BCAF; Sat, 4 Apr 2026 02:25:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775269533; bh=M5yBLEkfX8xBSUWEh0U5haPjD0m0eKWo0usQWtS3kCU=; h=Date:From:To:Cc:Subject:References:From; b=r/SWHVVKfxo5kC4zojaUaaNnwf3P42iPVANg6/dWyGQQjG04j3zTkrq+9RL6yJuuF eGCDXOVrLSSRzzy3+iwCqd/MCkQfoGTWyB4M89+hmgG11pmWd9QEktdwD+Ee9URCpY n52pUhZZ1Q+YoPph2M8YX6/RNNuv/It2t7x9R/MB0vtxVGIMTXuBbjum0CuFm5iurI emP0FvFL0BZVwJShe3ZigxVLj7ifqVss9V1pgCi0FKj+i+cUaEnf6A2xBGvbmpQCNY vwq8AQ+d3orYVDwyPKWpgy1LLwH5HOo2XG1a0vfEoq1lchzPI4nf5jv0GfWLQn7A2v iBclX518TnBPA== Received: from rostedt by gandalf with local (Exim 4.99.1) (envelope-from ) id 1w8qid-0000000164X-3Yok; Fri, 03 Apr 2026 22:26:39 -0400 Message-ID: <20260404022639.705588294@kernel.org> User-Agent: quilt/0.69 Date: Fri, 03 Apr 2026 22:26:19 -0400 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Vincent Donnefort Subject: [for-next][PATCH 2/6] ring-buffer: Enforce read ordering of trace_buffer cpumask and buffers References: <20260404022617.436859059@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vincent Donnefort On CPU hotplug, if it is the first time a trace_buffer sees a CPU, a ring_buffer_per_cpu will be allocated and its corresponding bit toggled in the cpumask. Many readers check this cpumask to know if they can safely read the ring_buffer_per_cpu but they are doing so without memory ordering and may observe the cpumask bit set while having NULL buffer pointer. Enforce the memory read ordering by sending an IPI to all online CPUs. The hotplug path is a slow-path anyway and it saves us from adding read barriers in numerous call sites. Link: https://patch.msgid.link/20260401053659.3458961-1-vdonnefort@google.c= om Signed-off-by: Vincent Donnefort Suggested-by: Steven Rostedt (Google) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 8b6c39bba56d..2caa5d3d0ae9 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -7722,6 +7722,12 @@ int ring_buffer_map_get_reader(struct trace_buffer *= buffer, int cpu) return 0; } =20 +static void rb_cpu_sync(void *data) +{ + /* Not really needed, but documents what is happening */ + smp_rmb(); +} + /* * We only allocate new buffers, never free them if the CPU goes down. * If we were to free the buffer, then the user would lose any trace that = was in @@ -7760,7 +7766,18 @@ int trace_rb_cpu_prepare(unsigned int cpu, struct hl= ist_node *node) cpu); return -ENOMEM; } - smp_wmb(); + + /* + * Ensure trace_buffer readers observe the newly allocated + * ring_buffer_per_cpu before they check the cpumask. Instead of using a + * read barrier for all readers, send an IPI. + */ + if (unlikely(system_state =3D=3D SYSTEM_RUNNING)) { + on_each_cpu(rb_cpu_sync, NULL, 1); + /* Not really needed, but documents what is happening */ + smp_wmb(); + } + cpumask_set_cpu(cpu, buffer->cpumask); return 0; } --=20 2.51.0 From nobody Sun Jun 14 17:31:44 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1D37256C61 for ; Sat, 4 Apr 2026 02:25:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269533; cv=none; b=WtTJjGhreH97pnXiMtvyyWW26hKTD8tAY/mv30BeVQrtfNI8RiVtMjuQ1MKPuqIr3eA98rzWyIPr0PFQgzplI/B4OrGtJKB1Ppj/CvKVH99l1ebSqbsbDf2g7rxGW/MKQ/SAt04Ydm1tDK3pjR/fYiEdmUZs2Deko+UCKNDT8qE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269533; c=relaxed/simple; bh=fH6E3tFlmY0yoWkt/Z2s7drRdOiIQFdR0WHXFwe9h8A=; h=Message-ID:Date:From:To:Cc:Subject:References:MIME-Version: Content-Type; b=rRfHEVibTA1EC73Asp3Pt4eeZGY4zyn7MqIPUMz3WEvg5dGqm2Ys5PQ9LXwdyNr3mnPCS8Kjwxpp8Jl3riQx30Ci31tXJoj5r6dgiF7v++XiTD4GMBEhFHhS06KOF2ENe0SS6U1pyoayA1mf/aiQohEJoAsGowh2tMO0SZsH1pU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ku08bsL5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ku08bsL5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A5E1DC2BCB2; Sat, 4 Apr 2026 02:25:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775269533; bh=fH6E3tFlmY0yoWkt/Z2s7drRdOiIQFdR0WHXFwe9h8A=; h=Date:From:To:Cc:Subject:References:From; b=Ku08bsL5kNEaeKo9f14rz+ZvlNTbPFhH+LHO8NPxbzFv2hCBczNEYZBrkzUn5kt+k p+rgBN6vlPl0akRGIB6U+j+nogJUEl781YeRHUPDGh3mRql72ndhxLVtb906WfZOYN 5M4qKrJTlSvB3rFHpLJ+Guex2/mN4ejzwAvHKzfth6WCh91tpKDgyal5NcTPl1MLpJ ItkznWYXOKRhsh7Or/2VGdu/uM8ifYorF+auSZAqQxCGn7plbjUZZ1PBSjg4eL0iz9 WYL0TqDJIzGGdU+NtcCTKuRpWa5ZjIaQDlCxmOUBo9FVDaxRlixQWiKc5A9lNftTLN EuXYiqvvSIH7Q== Received: from rostedt by gandalf with local (Exim 4.99.1) (envelope-from ) id 1w8qie-00000001651-01Sw; Fri, 03 Apr 2026 22:26:40 -0400 Message-ID: <20260404022639.867891192@kernel.org> User-Agent: quilt/0.69 Date: Fri, 03 Apr 2026 22:26:20 -0400 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton Subject: [for-next][PATCH 3/6] tracing: Make the backup instance non-reusable References: <20260404022617.436859059@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Masami Hiramatsu (Google)" Since there is no reason to reuse the backup instance, make it readonly (but erasable). Note that only backup instances are readonly, because other trace instances will be empty unless it is writable. Only backup instances have copy entries from the original. With this change, most of the trace control files are removed from the backup instance, including eventfs enable/filter etc. # find /sys/kernel/tracing/instances/backup/events/ | wc -l 4093 # find /sys/kernel/tracing/instances/boot_map/events/ | wc -l 9573 Cc: Mathieu Desnoyers Link: https://patch.msgid.link/177502546939.1311542.1826814401724828930.stg= it@mhiramat.tok.corp.google.com Signed-off-by: Masami Hiramatsu (Google) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 81 +++++++++++++++++++++++-------------- kernel/trace/trace.h | 7 ++++ kernel/trace/trace_boot.c | 5 ++- kernel/trace/trace_events.c | 76 +++++++++++++++++++--------------- 4 files changed, 104 insertions(+), 65 deletions(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 4189ec9df6a5..c8e43675ff76 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -4022,6 +4022,11 @@ int tracing_open_generic_tr(struct inode *inode, str= uct file *filp) if (ret) return ret; =20 + if ((filp->f_mode & FMODE_WRITE) && trace_array_is_readonly(tr)) { + trace_array_put(tr); + return -EACCES; + } + filp->private_data =3D inode->i_private; =20 return 0; @@ -7097,6 +7102,11 @@ static int tracing_clock_open(struct inode *inode, s= truct file *file) if (ret) return ret; =20 + if ((file->f_mode & FMODE_WRITE) && trace_array_is_readonly(tr)) { + trace_array_put(tr); + return -EACCES; + } + ret =3D single_open(file, tracing_clock_show, inode->i_private); if (ret < 0) trace_array_put(tr); @@ -9866,17 +9876,22 @@ static __init void create_trace_instances(struct de= ntry *d_tracer) static void init_tracer_tracefs(struct trace_array *tr, struct dentry *d_tracer) { + umode_t writable_mode =3D TRACE_MODE_WRITE; int cpu; =20 + if (trace_array_is_readonly(tr)) + writable_mode =3D TRACE_MODE_READ; + trace_create_file("available_tracers", TRACE_MODE_READ, d_tracer, - tr, &show_traces_fops); + tr, &show_traces_fops); =20 - trace_create_file("current_tracer", TRACE_MODE_WRITE, d_tracer, - tr, &set_tracer_fops); + trace_create_file("current_tracer", writable_mode, d_tracer, + tr, &set_tracer_fops); =20 - trace_create_file("tracing_cpumask", TRACE_MODE_WRITE, d_tracer, + trace_create_file("tracing_cpumask", writable_mode, d_tracer, tr, &tracing_cpumask_fops); =20 + /* Options are used for changing print-format even for readonly instance.= */ trace_create_file("trace_options", TRACE_MODE_WRITE, d_tracer, tr, &tracing_iter_fops); =20 @@ -9886,12 +9901,36 @@ init_tracer_tracefs(struct trace_array *tr, struct = dentry *d_tracer) trace_create_file("trace_pipe", TRACE_MODE_READ, d_tracer, tr, &tracing_pipe_fops); =20 - trace_create_file("buffer_size_kb", TRACE_MODE_WRITE, d_tracer, + trace_create_file("buffer_size_kb", writable_mode, d_tracer, tr, &tracing_entries_fops); =20 trace_create_file("buffer_total_size_kb", TRACE_MODE_READ, d_tracer, tr, &tracing_total_entries_fops); =20 + trace_create_file("trace_clock", writable_mode, d_tracer, tr, + &trace_clock_fops); + + trace_create_file("timestamp_mode", TRACE_MODE_READ, d_tracer, tr, + &trace_time_stamp_mode_fops); + + tr->buffer_percent =3D 50; + + trace_create_file("buffer_subbuf_size_kb", writable_mode, d_tracer, + tr, &buffer_subbuf_size_fops); + + create_trace_options_dir(tr); + + if (tr->range_addr_start) + trace_create_file("last_boot_info", TRACE_MODE_READ, d_tracer, + tr, &last_boot_fops); + + for_each_tracing_cpu(cpu) + tracing_init_tracefs_percpu(tr, cpu); + + /* Read-only instance has above files only. */ + if (trace_array_is_readonly(tr)) + return; + trace_create_file("free_buffer", 0200, d_tracer, tr, &tracing_free_buffer_fops); =20 @@ -9903,49 +9942,29 @@ init_tracer_tracefs(struct trace_array *tr, struct = dentry *d_tracer) trace_create_file("trace_marker_raw", 0220, d_tracer, tr, &tracing_mark_raw_fops); =20 - trace_create_file("trace_clock", TRACE_MODE_WRITE, d_tracer, tr, - &trace_clock_fops); - - trace_create_file("tracing_on", TRACE_MODE_WRITE, d_tracer, - tr, &rb_simple_fops); - - trace_create_file("timestamp_mode", TRACE_MODE_READ, d_tracer, tr, - &trace_time_stamp_mode_fops); - - tr->buffer_percent =3D 50; - trace_create_file("buffer_percent", TRACE_MODE_WRITE, d_tracer, - tr, &buffer_percent_fops); - - trace_create_file("buffer_subbuf_size_kb", TRACE_MODE_WRITE, d_tracer, - tr, &buffer_subbuf_size_fops); + tr, &buffer_percent_fops); =20 trace_create_file("syscall_user_buf_size", TRACE_MODE_WRITE, d_tracer, - tr, &tracing_syscall_buf_fops); + tr, &tracing_syscall_buf_fops); =20 - create_trace_options_dir(tr); + trace_create_file("tracing_on", TRACE_MODE_WRITE, d_tracer, + tr, &rb_simple_fops); =20 trace_create_maxlat_file(tr, d_tracer); =20 if (ftrace_create_function_files(tr, d_tracer)) MEM_FAIL(1, "Could not allocate function filter files"); =20 - if (tr->range_addr_start) { - trace_create_file("last_boot_info", TRACE_MODE_READ, d_tracer, - tr, &last_boot_fops); #ifdef CONFIG_TRACER_SNAPSHOT - } else { + if (!tr->range_addr_start) trace_create_file("snapshot", TRACE_MODE_WRITE, d_tracer, tr, &snapshot_fops); #endif - } =20 trace_create_file("error_log", TRACE_MODE_WRITE, d_tracer, tr, &tracing_err_log_fops); =20 - for_each_tracing_cpu(cpu) - tracing_init_tracefs_percpu(tr, cpu); - ftrace_init_tracefs(tr, d_tracer); } =20 @@ -10772,7 +10791,7 @@ __init static void enable_instances(void) * Backup buffers can be freed but need vfree(). */ if (backup) - tr->flags |=3D TRACE_ARRAY_FL_VMALLOC; + tr->flags |=3D TRACE_ARRAY_FL_VMALLOC | TRACE_ARRAY_FL_RDONLY; =20 if (start || backup) { tr->flags |=3D TRACE_ARRAY_FL_BOOT | TRACE_ARRAY_FL_LAST_BOOT; diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index fbbd0b2ee76f..366baa43dfab 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -462,6 +462,7 @@ enum { TRACE_ARRAY_FL_MOD_INIT =3D BIT(3), TRACE_ARRAY_FL_MEMMAP =3D BIT(4), TRACE_ARRAY_FL_VMALLOC =3D BIT(5), + TRACE_ARRAY_FL_RDONLY =3D BIT(6), }; =20 #ifdef CONFIG_MODULES @@ -491,6 +492,12 @@ extern unsigned long trace_adjust_address(struct trace= _array *tr, unsigned long =20 extern struct trace_array *printk_trace; =20 +static inline bool trace_array_is_readonly(struct trace_array *tr) +{ + /* backup instance is read only. */ + return tr->flags & TRACE_ARRAY_FL_RDONLY; +} + /* * The global tracer (top) should be the first trace array added, * but we check the flag anyway. diff --git a/kernel/trace/trace_boot.c b/kernel/trace/trace_boot.c index dbe29b4c6a7a..2ca2541c8a58 100644 --- a/kernel/trace/trace_boot.c +++ b/kernel/trace/trace_boot.c @@ -61,7 +61,8 @@ trace_boot_set_instance_options(struct trace_array *tr, s= truct xbc_node *node) v =3D memparse(p, NULL); if (v < PAGE_SIZE) pr_err("Buffer size is too small: %s\n", p); - if (tracing_resize_ring_buffer(tr, v, RING_BUFFER_ALL_CPUS) < 0) + if (trace_array_is_readonly(tr) || + tracing_resize_ring_buffer(tr, v, RING_BUFFER_ALL_CPUS) < 0) pr_err("Failed to resize trace buffer to %s\n", p); } =20 @@ -597,7 +598,7 @@ trace_boot_enable_tracer(struct trace_array *tr, struct= xbc_node *node) =20 p =3D xbc_node_find_value(node, "tracer", NULL); if (p && *p !=3D '\0') { - if (tracing_set_tracer(tr, p) < 0) + if (trace_array_is_readonly(tr) || tracing_set_tracer(tr, p) < 0) pr_err("Failed to set given tracer: %s\n", p); } =20 diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index 249d1cba72c0..aa422dc80ae8 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c @@ -1401,6 +1401,9 @@ static int __ftrace_set_clr_event(struct trace_array = *tr, const char *match, { int ret; =20 + if (trace_array_is_readonly(tr)) + return -EACCES; + mutex_lock(&event_mutex); ret =3D __ftrace_set_clr_event_nolock(tr, match, sub, event, set, mod); mutex_unlock(&event_mutex); @@ -2973,8 +2976,8 @@ event_subsystem_dir(struct trace_array *tr, const cha= r *name, } else __get_system(system); =20 - /* ftrace only has directories no files */ - if (strcmp(name, "ftrace") =3D=3D 0) + /* ftrace only has directories no files, readonly instance too. */ + if (strcmp(name, "ftrace") =3D=3D 0 || trace_array_is_readonly(tr)) nr_entries =3D 0; else nr_entries =3D ARRAY_SIZE(system_entries); @@ -3139,28 +3142,30 @@ event_create_dir(struct eventfs_inode *parent, stru= ct trace_event_file *file) int ret; static struct eventfs_entry event_entries[] =3D { { - .name =3D "enable", + .name =3D "format", .callback =3D event_callback, - .release =3D event_release, }, +#ifdef CONFIG_PERF_EVENTS { - .name =3D "filter", + .name =3D "id", .callback =3D event_callback, }, +#endif +#define NR_RO_EVENT_ENTRIES (1 + IS_ENABLED(CONFIG_PERF_EVENTS)) +/* Readonly files must be above this line and counted by NR_RO_EVENT_ENTRI= ES. */ { - .name =3D "trigger", + .name =3D "enable", .callback =3D event_callback, + .release =3D event_release, }, { - .name =3D "format", + .name =3D "filter", .callback =3D event_callback, }, -#ifdef CONFIG_PERF_EVENTS { - .name =3D "id", + .name =3D "trigger", .callback =3D event_callback, }, -#endif #ifdef CONFIG_HIST_TRIGGERS { .name =3D "hist", @@ -3193,7 +3198,10 @@ event_create_dir(struct eventfs_inode *parent, struc= t trace_event_file *file) if (!e_events) return -ENOMEM; =20 - nr_entries =3D ARRAY_SIZE(event_entries); + if (trace_array_is_readonly(tr)) + nr_entries =3D NR_RO_EVENT_ENTRIES; + else + nr_entries =3D ARRAY_SIZE(event_entries); =20 name =3D trace_event_name(call); ei =3D eventfs_create_dir(name, e_events, event_entries, nr_entries, file= ); @@ -4536,31 +4544,44 @@ create_event_toplevel_files(struct dentry *parent, = struct trace_array *tr) int nr_entries; static struct eventfs_entry events_entries[] =3D { { - .name =3D "enable", + .name =3D "header_page", .callback =3D events_callback, }, { - .name =3D "header_page", + .name =3D "header_event", .callback =3D events_callback, }, +#define NR_RO_TOP_ENTRIES 2 +/* Readonly files must be above this line and counted by NR_RO_TOP_ENTRIES= . */ { - .name =3D "header_event", + .name =3D "enable", .callback =3D events_callback, }, }; =20 - entry =3D trace_create_file("set_event", TRACE_MODE_WRITE, parent, - tr, &ftrace_set_event_fops); - if (!entry) - return -ENOMEM; + if (!trace_array_is_readonly(tr)) { + entry =3D trace_create_file("set_event", TRACE_MODE_WRITE, parent, + tr, &ftrace_set_event_fops); + if (!entry) + return -ENOMEM; =20 - trace_create_file("show_event_filters", TRACE_MODE_READ, parent, tr, - &ftrace_show_event_filters_fops); + /* There are not as crucial, just warn if they are not created */ + trace_create_file("show_event_filters", TRACE_MODE_READ, parent, tr, + &ftrace_show_event_filters_fops); =20 - trace_create_file("show_event_triggers", TRACE_MODE_READ, parent, tr, - &ftrace_show_event_triggers_fops); + trace_create_file("show_event_triggers", TRACE_MODE_READ, parent, tr, + &ftrace_show_event_triggers_fops); =20 - nr_entries =3D ARRAY_SIZE(events_entries); + trace_create_file("set_event_pid", TRACE_MODE_WRITE, parent, + tr, &ftrace_set_event_pid_fops); + + trace_create_file("set_event_notrace_pid", + TRACE_MODE_WRITE, parent, tr, + &ftrace_set_event_notrace_pid_fops); + nr_entries =3D ARRAY_SIZE(events_entries); + } else { + nr_entries =3D NR_RO_TOP_ENTRIES; + } =20 e_events =3D eventfs_create_events_dir("events", parent, events_entries, nr_entries, tr); @@ -4569,15 +4590,6 @@ create_event_toplevel_files(struct dentry *parent, s= truct trace_array *tr) return -ENOMEM; } =20 - /* There are not as crucial, just warn if they are not created */ - - trace_create_file("set_event_pid", TRACE_MODE_WRITE, parent, - tr, &ftrace_set_event_pid_fops); - - trace_create_file("set_event_notrace_pid", - TRACE_MODE_WRITE, parent, tr, - &ftrace_set_event_notrace_pid_fops); - tr->event_dir =3D e_events; =20 return 0; --=20 2.51.0 From nobody Sun Jun 14 17:31:44 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE21329E0F7 for ; Sat, 4 Apr 2026 02:25:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269534; cv=none; b=CTAC7cP0aVR4Mpz5EJvAilrHkO+gSc9E/5x30l84Opf9w3zuJ1zM0UxOe7GlkMTzHREPU0+pT5rWDZ+kaIo1jvWhqh6dd5CQUJC8jPIXESUAIokRk8eGlBC75i6qCq0uLgyYSWE9gjx+O0Bc+ZFP72hw4M2LNL9uY+DN2DNuFF8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269534; c=relaxed/simple; bh=GlAOu9D4ccKEqv8CeX256rPP3vz/3BcjHxOsiFna/pM=; h=Message-ID:Date:From:To:Cc:Subject:References:MIME-Version: Content-Type; b=rxApf20SOS/e/mVedM4YTPHwKztqsMDjzeCZSv+x5VpP5nbt+01V4rfYvM3RzaBzpetoPghCuOIfaoJxW7cLM8NnUIkKVInaoPKIQN/de3u5vROmGMPDCrAQvzNLd6IGEvuQkcjs439gi+yHUwHc0x6CGqkwu7DqqAi/7MLt9ZA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cpPdqfOg; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cpPdqfOg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE172C2BCB5; Sat, 4 Apr 2026 02:25:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775269533; bh=GlAOu9D4ccKEqv8CeX256rPP3vz/3BcjHxOsiFna/pM=; h=Date:From:To:Cc:Subject:References:From; b=cpPdqfOgmiYzWGHGwJwrGYCz1eBoiGLOn+lTil4b+t8L0EkMmTfh5ugUCfYSdj5J9 yJ+E0YCnq6Q6khZOtMy8uXeVhnK6q2IUxiGDzcX0+nTYLHXYqYXowP3f8UtrjPHgF+ G2YVoc1B+miBiIlFgHE/WHhb2hBfotS+0FsF+6P0UfDuBdtyfPf5ZtKrynV36LfDdM 11hQT7YS18R4Nf5CYf1/LCWqB+H3b3XouWccbZneaPbalaDmZgb4vcpsomdYhQo1PS U4tU2ERFqXHE0LCr+glmrs+at0QSXLNSjDj8Br1wEn7EW6E79ynK0wu7ImWmQVOJ1Q 6V1cHWMP4XMqw== Received: from rostedt by gandalf with local (Exim 4.99.1) (envelope-from ) id 1w8qie-0000000165V-0gK2; Fri, 03 Apr 2026 22:26:40 -0400 Message-ID: <20260404022640.022996869@kernel.org> User-Agent: quilt/0.69 Date: Fri, 03 Apr 2026 22:26:21 -0400 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton Subject: [for-next][PATCH 4/6] tracing: Remove the backup instance automatically after read References: <20260404022617.436859059@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Masami Hiramatsu (Google)" Since the backup instance is readonly, after reading all data via pipe, no data is left on the instance. Thus it can be removed safely after closing all files. This also removes it if user resets the ring buffer manually via 'trace' file. Cc: Mathieu Desnoyers Link: https://patch.msgid.link/177502547711.1311542.12572973358010839400.st= git@mhiramat.tok.corp.google.com Signed-off-by: Masami Hiramatsu (Google) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 79 ++++++++++++++++++++++++++++++++++++++++---- kernel/trace/trace.h | 6 ++++ 2 files changed, 79 insertions(+), 6 deletions(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index c8e43675ff76..267671b33dd3 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -578,8 +578,59 @@ void trace_set_ring_buffer_expanded(struct trace_array= *tr) tr->ring_buffer_expanded =3D true; } =20 +static void trace_array_autoremove(struct work_struct *work) +{ + struct trace_array *tr =3D container_of(work, struct trace_array, autorem= ove_work); + + trace_array_destroy(tr); +} + +static struct workqueue_struct *autoremove_wq; + +static void trace_array_kick_autoremove(struct trace_array *tr) +{ + if (autoremove_wq) + queue_work(autoremove_wq, &tr->autoremove_work); +} + +static void trace_array_cancel_autoremove(struct trace_array *tr) +{ + /* + * Since this can be called inside trace_array_autoremove(), + * it has to avoid deadlock of the workqueue. + */ + if (work_pending(&tr->autoremove_work)) + cancel_work_sync(&tr->autoremove_work); +} + +static void trace_array_init_autoremove(struct trace_array *tr) +{ + INIT_WORK(&tr->autoremove_work, trace_array_autoremove); +} + +static void trace_array_start_autoremove(void) +{ + if (autoremove_wq) + return; + + autoremove_wq =3D alloc_workqueue("tr_autoremove_wq", + WQ_UNBOUND | WQ_HIGHPRI, 0); + if (!autoremove_wq) + pr_warn("Unable to allocate tr_autoremove_wq. autoremove disabled.\n"); +} + LIST_HEAD(ftrace_trace_arrays); =20 +static int __trace_array_get(struct trace_array *this_tr) +{ + /* When free_on_close is set, this is not available anymore. */ + if (autoremove_wq && this_tr->free_on_close) + return -ENODEV; + + this_tr->ref++; + return 0; +} + int trace_array_get(struct trace_array *this_tr) { struct trace_array *tr; @@ -587,8 +638,7 @@ int trace_array_get(struct trace_array *this_tr) guard(mutex)(&trace_types_lock); list_for_each_entry(tr, &ftrace_trace_arrays, list) { if (tr =3D=3D this_tr) { - tr->ref++; - return 0; + return __trace_array_get(tr); } } =20 @@ -599,6 +649,12 @@ static void __trace_array_put(struct trace_array *this= _tr) { WARN_ON(!this_tr->ref); this_tr->ref--; + /* + * When free_on_close is set, prepare removing the array + * when the last reference is released. + */ + if (this_tr->ref =3D=3D 1 && this_tr->free_on_close) + trace_array_kick_autoremove(this_tr); } =20 /** @@ -5467,6 +5523,10 @@ static void update_last_data(struct trace_array *tr) /* Only if the buffer has previous boot data clear and update it. */ tr->flags &=3D ~TRACE_ARRAY_FL_LAST_BOOT; =20 + /* If this is a backup instance, mark it for autoremove. */ + if (tr->flags & TRACE_ARRAY_FL_VMALLOC) + tr->free_on_close =3D true; + /* Reset the module list and reload them */ if (tr->scratch) { struct trace_scratch *tscratch =3D tr->scratch; @@ -9537,8 +9597,8 @@ struct trace_array *trace_array_find_get(const char *= instance) =20 guard(mutex)(&trace_types_lock); tr =3D trace_array_find(instance); - if (tr) - tr->ref++; + if (tr && __trace_array_get(tr) < 0) + tr =3D NULL; =20 return tr; } @@ -9635,6 +9695,8 @@ trace_array_create_systems(const char *name, const ch= ar *systems, if (ftrace_allocate_ftrace_ops(tr) < 0) goto out_free_tr; =20 + trace_array_init_autoremove(tr); + ftrace_init_trace_array(tr); =20 init_trace_flags_index(tr); @@ -9745,7 +9807,9 @@ struct trace_array *trace_array_get_by_name(const cha= r *name, const char *system =20 list_for_each_entry(tr, &ftrace_trace_arrays, list) { if (tr->name && strcmp(tr->name, name) =3D=3D 0) { - tr->ref++; + /* if this fails, @tr is going to be removed. */ + if (__trace_array_get(tr) < 0) + tr =3D NULL; return tr; } } @@ -9784,6 +9848,7 @@ static int __remove_instance(struct trace_array *tr) set_tracer_flag(tr, 1ULL << i, 0); } =20 + trace_array_cancel_autoremove(tr); tracing_set_nop(tr); clear_ftrace_function_probes(tr); event_trace_del_tracer(tr); @@ -10790,8 +10855,10 @@ __init static void enable_instances(void) /* * Backup buffers can be freed but need vfree(). */ - if (backup) + if (backup) { tr->flags |=3D TRACE_ARRAY_FL_VMALLOC | TRACE_ARRAY_FL_RDONLY; + trace_array_start_autoremove(); + } =20 if (start || backup) { tr->flags |=3D TRACE_ARRAY_FL_BOOT | TRACE_ARRAY_FL_LAST_BOOT; diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index 366baa43dfab..b001fbba0881 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -453,6 +453,12 @@ struct trace_array { * we do not waste memory on systems that are not using tracing. */ bool ring_buffer_expanded; + /* + * If the ring buffer is a read only backup instance, it will be + * removed after dumping all data via pipe, because no readable data. + */ + bool free_on_close; + struct work_struct autoremove_work; }; =20 enum { --=20 2.51.0 From nobody Sun Jun 14 17:31:44 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B4C52BE639 for ; Sat, 4 Apr 2026 02:25:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269534; cv=none; b=aU9V0QsdjOff/3sOXMhaKw84+XlhZGv4W6/X/RDRpD/4JW2zIN1PtJR9fOegIKSUp8v5G/bkvekMTVwZLOQJ5uzJzEZN5IcgKo5YW+yBl72GjAQDWXg/hX8dI5bJnt1AJlWIKeE0PY6IYz97hYSGQZvGy1nfaFSiMx0EDxL5bo4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269534; c=relaxed/simple; bh=UTz0hADeyOa52+IenGEb2Z21TkD+5Ro8LZ15cJFEcfU=; h=Message-ID:Date:From:To:Cc:Subject:References:MIME-Version: Content-Type; b=lcfHx6ECNROF8CZIHeqFc2nVz0pSyWT1hmdblgmZdVg+7PZ6fxKGK9gdhxJyNQj1NeaKh2pTqM4B2SBesOSSOc5Vpj0yEfUhUYJCfHLRt7EOK0Eu4ug30RaA3Z+piEsaC2E3RY2vMOo/7O6x00tfTht0o+LFpZYR9HXZf+293ec= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UrcOy2jB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UrcOy2jB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F1A49C19421; Sat, 4 Apr 2026 02:25:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775269534; bh=UTz0hADeyOa52+IenGEb2Z21TkD+5Ro8LZ15cJFEcfU=; h=Date:From:To:Cc:Subject:References:From; b=UrcOy2jBiwjDXUybtQtFUkAA3sf0ev6gD7kD4fpXAWCngJobfNqZvlPA0RPQdJA8G hBWmGtggngcuA2gtjhUBoIO7t4N5surSkYDIaQaIrEq+9T6vouEoh6I6FLOkc3WZFE NU50OREGZ7fZ6mTZE8fORHEpWO7FmMDPHg2+7o6FZKAL7ER8tv8Qp9LYQY5aini/MN UAEKVYjQ3EVtXILHWS0fR1bwP2ghPP0ubNZxY/ccSQlTTB1JTzWERUKp/lM8kyMNoR Fm9ysqlsJHK0B3F+5cHgSYJOsMR06CalVWQA00+UpirULkpy+AzW8lHadLabJ/hcus DkAGBc35nrUPQ== Received: from rostedt by gandalf with local (Exim 4.99.1) (envelope-from ) id 1w8qie-0000000165z-1GFH; Fri, 03 Apr 2026 22:26:40 -0400 Message-ID: <20260404022640.180611561@kernel.org> User-Agent: quilt/0.69 Date: Fri, 03 Apr 2026 22:26:22 -0400 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton Subject: [for-next][PATCH 5/6] tracing/Documentation: Add a section about backup instance References: <20260404022617.436859059@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Masami Hiramatsu (Google)" Add a section about backup instance to the debugging.rst. Cc: Mathieu Desnoyers Link: https://patch.msgid.link/177502548479.1311542.7062269603547001007.stg= it@mhiramat.tok.corp.google.com Signed-off-by: Masami Hiramatsu (Google) Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/debugging.rst | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/Documentation/trace/debugging.rst b/Documentation/trace/debugg= ing.rst index 4d88c346fc38..bca1710d92bf 100644 --- a/Documentation/trace/debugging.rst +++ b/Documentation/trace/debugging.rst @@ -159,3 +159,22 @@ If setting it from the kernel command line, it is reco= mmended to also disable tracing with the "traceoff" flag, and enable tracing after boot up. Otherwise the trace from the most recent boot will be mixed with the trace from the previous boot, and may make it confusing to read. + +Using a backup instance for keeping previous boot data +------------------------------------------------------ + +It is also possible to record trace data at system boot time by specifying +events with the persistent ring buffer, but in this case the data before t= he +reboot will be lost before it can be read. This problem can be solved by a +backup instance. From the kernel command line:: + + reserve_mem=3D12M:4096:trace trace_instance=3Dboot_map@trace,sched,irq t= race_instance=3Dbackup=3Dboot_map + +On boot up, the previous data in the "boot_map" is copied to the "backup" +instance, and the "sched:*" and "irq:*" events for the current boot are tr= aced +in the "boot_map". Thus the user can read the previous boot data from the = "backup" +instance without stopping the trace. + +Note that this "backup" instance is readonly, and will be removed automati= cally +if you clear the trace data or read out all trace data from the "trace_pip= e" +or the "trace_pipe_raw" files. --=20 2.51.0 From nobody Sun Jun 14 17:31:44 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E5BD2BFC8F for ; Sat, 4 Apr 2026 02:25:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269534; cv=none; b=HqAcXKVM9ESCU7Omc5AvzzVlwRR8dJe9P8UXM9DwHtGYOxNctqdp9uPwzHiDkiqrRj15YL/ku+XMHEbY8ZFr6kiWa6diVFstWO+KQOmu7V3D99ofwbj9VamF3SL+X/hqDHnYc8gHt4QqEKDb+923S8uKWwYY5p07t2U2IResJO0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775269534; c=relaxed/simple; bh=nUowXFSnfTSR/DpjEObO4xQTh/eJ3gqq1fj8RBEb3wQ=; h=Message-ID:Date:From:To:Cc:Subject:References:MIME-Version: Content-Type; b=N1ZhjWt8Wg+T7l5INyNGhW3MmhoCCsBs383/QTq4lukj9llUvnjGmqUneNsStlhdxs3n2sMRLt6iNT1Ag6IolSIHCUwCnG5eQYrhqSeJsk+f/iYplR9nuwQgmbkCWs3AI+yLJWslWzAKM2pbTEd/sRvPLOQNJB9uPgyjBHMC8HY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=eTPeRcga; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="eTPeRcga" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0EA43C19423; Sat, 4 Apr 2026 02:25:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775269534; bh=nUowXFSnfTSR/DpjEObO4xQTh/eJ3gqq1fj8RBEb3wQ=; h=Date:From:To:Cc:Subject:References:From; b=eTPeRcgaO0UKcN9z7fdB2eJNRD+ScXfV3HC6QG7F/13WEjLwUvfagGoCbQkl96g72 tEKt+lpdhe56RkrDr3PZoQjie7F+/JVTpcRSJJ2lKrjeNrd6L9x8uR7k73wljn7thq mmvod8oqpASWEXRVXLIFTpzLjblPoQfMBj6yOheK6wZenKNR/CranP+tp+7Zl8gkb+ Bwo2EopjXwGiMLOeVcI0pOpLB0Hmj0RF0nKz7lQonfmfk0H1u0sDrAwXTsSrohzZhb 4tnUlk2dBA1uh9PyrndztQd/WOuRAeinS2hraKa+LPj/1Au4ZpkKLS8nUcnBM4EAa1 1xGWRmtBwNLwQ== Received: from rostedt by gandalf with local (Exim 4.99.1) (envelope-from ) id 1w8qie-0000000166T-1v8V; Fri, 03 Apr 2026 22:26:40 -0400 Message-ID: <20260404022640.317560976@kernel.org> User-Agent: quilt/0.69 Date: Fri, 03 Apr 2026 22:26:23 -0400 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , John Stultz Subject: [for-next][PATCH 6/6] tracing: Allow backup to save persistent ring buffer before it starts References: <20260404022617.436859059@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Steven Rostedt When the persistent ring buffer was first introduced, it did not make sense to start tracing for it on the kernel command line. That's because if there was a crash, the start of events would invalidate the events from the previous boot that had the crash. But now that there's a "backup" instance that can take a snapshot of the persistent ring buffer when boot starts, it is possible to have the persistent ring buffer start events at boot up and not lose the old events. Update the code where the boot events start after all boot time instances are created. This will allow the backup instance to copy the persistent ring buffer from the previous boot, and allow the persistent ring buffer to start tracing new events for the current boot. reserve_mem=3D100M:12M:trace trace_instance=3Dboot_mapped^@trace,sched tr= ace_instance=3Dbackup=3Dboot_mapped The above will create a boot_mapped persistent ring buffer and enabled the scheduler events. If there's a crash, a "backup" instance will be created holding the events of the persistent ring buffer from the previous boot, while the persistent ring buffer will once again start tracing scheduler events of the current boot. Now the user doesn't have to remember to start the persistent ring buffer. It will always have the events started at each boot. Cc: Masami Hiramatsu Cc: Mathieu Desnoyers Cc: John Stultz Link: https://patch.msgid.link/20260331163924.6ccb3896@gandalf.local.home Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 22 ++++++++++++++++++++++ kernel/trace/trace.h | 5 ++++- 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 267671b33dd3..e9455d46ec16 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -10865,9 +10865,31 @@ __init static void enable_instances(void) tr->range_name =3D no_free_ptr(rname); } =20 + /* + * Save the events to start and enabled them after all boot instances + * have been created. + */ + tr->boot_events =3D curr_str; + } + + /* Enable the events after all boot instances have been created */ + list_for_each_entry(tr, &ftrace_trace_arrays, list) { + + if (!tr->boot_events || !(*tr->boot_events)) { + tr->boot_events =3D NULL; + continue; + } + + curr_str =3D tr->boot_events; + + /* Clear the instance if this is a persistent buffer */ + if (tr->flags & TRACE_ARRAY_FL_LAST_BOOT) + update_last_data(tr); + while ((tok =3D strsep(&curr_str, ","))) { early_enable_events(tr, tok, true); } + tr->boot_events =3D NULL; } } =20 diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index b001fbba0881..e68f9c2027eb 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -405,7 +405,10 @@ struct trace_array { unsigned char trace_flags_index[TRACE_FLAGS_MAX_SIZE]; unsigned int flags; raw_spinlock_t start_lock; - const char *system_names; + union { + const char *system_names; + char *boot_events; + }; struct list_head err_log; struct dentry *dir; struct dentry *options; --=20 2.51.0