From nobody Fri Dec 19 07:31:48 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EDD49195B04 for ; Fri, 14 Jun 2024 16:30:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718382649; cv=none; b=NQX5BKL/O5eda2XTrcxsCRjUA45H0pG+gXlZw3k3P8jT0ksP0jSylcaeSG2p1tw8NZ8ChmzbWsPLy5yPdmjR3YsuWUng7dkQ6nOsK1U0E6lYndNbwmw9dQFVfdfmlt7+MGXUG48hFdVzvTvxOpt+hs06Wj1ZQEyFwZsYyWB6Tuk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718382649; c=relaxed/simple; bh=ffDMNay5V4HCQNkyqfXkxcR32rvFhwWt7bVdmaqOd6M=; h=Message-ID:Date:From:To:Cc:Subject:References:MIME-Version: Content-Type; b=Ji3/NfSGw0c/AkTnPEqyARFywYqVlgVfKETPTgessXor4CLF764kl+lg8qIn4UIVVLorkpuoXFtsK91LNOPKFpnZ5RGXEC0JS5DRjW8xly++WCH4GdYvA2JOa48I2PX4lI+wwQvFhO/gCoDGhaJ/Uz3uei8hJ6VUww0NcAqnlgE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6854EC4DDEA; Fri, 14 Jun 2024 16:30:48 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1sI9pX-00000001rsP-1u2O; Fri, 14 Jun 2024 12:31:11 -0400 Message-ID: <20240614163111.310579196@goodmis.org> User-Agent: quilt/0.68 Date: Fri, 14 Jun 2024 12:30:22 -0400 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Vincent Donnefort , Joel Fernandes , Daniel Bristot de Oliveira , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Vineeth Pillai , Youssef Esmat , Beau Belgrave , Alexander Graf , Baoquan He , Borislav Petkov , "Paul E. McKenney" , David Howells , Mike Rapoport , Dave Hansen , Tony Luck , Guenter Roeck , Ross Zwisler , Kees Cook Subject: [for-next][PATCH 07/13] ring-buffer: Validate boot range memory events References: <20240614163015.456541709@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Steven Rostedt (Google)" Make sure all the events in each of the sub-buffers that were mapped in a memory region are valid. This moves the code that walks the buffers for time-stamp validation out of the CONFIG_RING_BUFFER_VALIDATE_TIME_DELTAS ifdef block and is used to validate the content. Only the ring buffer event meta data and time stamps are checked and not the data load. This also has a second purpose. The buffer_page structure that points to the data sub-buffers has accounting that keeps track of the number of events that are on the sub-buffer. This updates that counter as well. That counter is used in reading the buffer and knowing if the ring buffer is empty or not. Link: https://lkml.kernel.org/r/20240612232026.172503570@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Vincent Donnefort Cc: Joel Fernandes Cc: Daniel Bristot de Oliveira Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Vineeth Pillai Cc: Youssef Esmat Cc: Beau Belgrave Cc: Alexander Graf Cc: Baoquan He Cc: Borislav Petkov Cc: "Paul E. McKenney" Cc: David Howells Cc: Mike Rapoport Cc: Dave Hansen Cc: Tony Luck Cc: Guenter Roeck Cc: Ross Zwisler Cc: Kees Cook Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 190 +++++++++++++++++++++++++++++-------- 1 file changed, 152 insertions(+), 38 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 588bc057bad7..804dfbdeef84 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -1675,10 +1675,152 @@ static bool rb_meta_valid(struct ring_buffer_meta = *meta, int cpu, subbuf =3D (void *)subbuf + subbuf_size; } =20 - pr_info("Ring buffer meta is from previous boot!\n"); return true; } =20 +static int rb_meta_subbuf_idx(struct ring_buffer_meta *meta, void *subbuf); + +static int rb_read_data_buffer(struct buffer_data_page *dpage, int tail, i= nt cpu, + unsigned long long *timestamp, u64 *delta_ptr) +{ + struct ring_buffer_event *event; + u64 ts, delta; + int events =3D 0; + int e; + + *delta_ptr =3D 0; + *timestamp =3D 0; + + ts =3D dpage->time_stamp; + + for (e =3D 0; e < tail; e +=3D rb_event_length(event)) { + + event =3D (struct ring_buffer_event *)(dpage->data + e); + + switch (event->type_len) { + + case RINGBUF_TYPE_TIME_EXTEND: + delta =3D rb_event_time_stamp(event); + ts +=3D delta; + break; + + case RINGBUF_TYPE_TIME_STAMP: + delta =3D rb_event_time_stamp(event); + delta =3D rb_fix_abs_ts(delta, ts); + if (delta < ts) { + *delta_ptr =3D delta; + *timestamp =3D ts; + return -1; + } + ts =3D delta; + break; + + case RINGBUF_TYPE_PADDING: + if (event->time_delta =3D=3D 1) + break; + fallthrough; + case RINGBUF_TYPE_DATA: + events++; + ts +=3D event->time_delta; + break; + + default: + return -1; + } + } + *timestamp =3D ts; + return events; +} + +static int rb_validate_buffer(struct buffer_data_page *dpage, int cpu) +{ + unsigned long long ts; + u64 delta; + int tail; + + tail =3D local_read(&dpage->commit); + return rb_read_data_buffer(dpage, tail, cpu, &ts, &delta); +} + +/* If the meta data has been validated, now validate the events */ +static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer) +{ + struct ring_buffer_meta *meta =3D cpu_buffer->ring_meta; + struct buffer_page *head_page; + unsigned long entry_bytes =3D 0; + unsigned long entries =3D 0; + int ret; + int i; + + if (!meta || !meta->head_buffer) + return; + + /* Do the reader page first */ + ret =3D rb_validate_buffer(cpu_buffer->reader_page->page, cpu_buffer->cpu= ); + if (ret < 0) { + pr_info("Ring buffer reader page is invalid\n"); + goto invalid; + } + entries +=3D ret; + entry_bytes +=3D local_read(&cpu_buffer->reader_page->page->commit); + local_set(&cpu_buffer->reader_page->entries, ret); + + head_page =3D cpu_buffer->head_page; + + /* If both the head and commit are on the reader_page then we are done. */ + if (head_page =3D=3D cpu_buffer->reader_page && + head_page =3D=3D cpu_buffer->commit_page) + goto done; + + /* Iterate until finding the commit page */ + for (i =3D 0; i < meta->nr_subbufs + 1; i++, rb_inc_page(&head_page)) { + + /* Reader page has already been done */ + if (head_page =3D=3D cpu_buffer->reader_page) + continue; + + ret =3D rb_validate_buffer(head_page->page, cpu_buffer->cpu); + if (ret < 0) { + pr_info("Ring buffer meta [%d] invalid buffer page\n", + cpu_buffer->cpu); + goto invalid; + } + entries +=3D ret; + entry_bytes +=3D local_read(&head_page->page->commit); + local_set(&cpu_buffer->head_page->entries, ret); + + if (head_page =3D=3D cpu_buffer->commit_page) + break; + } + + if (head_page !=3D cpu_buffer->commit_page) { + pr_info("Ring buffer meta [%d] commit page not found\n", + cpu_buffer->cpu); + goto invalid; + } + done: + local_set(&cpu_buffer->entries, entries); + local_set(&cpu_buffer->entries_bytes, entry_bytes); + + pr_info("Ring buffer meta [%d] is from previous boot!\n", cpu_buffer->cpu= ); + return; + + invalid: + /* The content of the buffers are invalid, reset the meta data */ + meta->head_buffer =3D 0; + meta->commit_buffer =3D 0; + + /* Reset the reader page */ + local_set(&cpu_buffer->reader_page->entries, 0); + local_set(&cpu_buffer->reader_page->page->commit, 0); + + /* Reset all the subbuffers */ + for (i =3D 0; i < meta->nr_subbufs - 1; i++, rb_inc_page(&head_page)) { + local_set(&head_page->entries, 0); + local_set(&head_page->page->commit, 0); + } +} + static void rb_range_meta_init(struct trace_buffer *buffer, int nr_pages) { struct ring_buffer_meta *meta; @@ -1757,8 +1899,6 @@ static void *rbm_next(struct seq_file *m, void *v, lo= ff_t *pos) return rbm_start(m, pos); } =20 -static int rb_meta_subbuf_idx(struct ring_buffer_meta *meta, void *subbuf); - static int rbm_show(struct seq_file *m, void *v) { struct ring_buffer_per_cpu *cpu_buffer =3D m->private; @@ -2011,6 +2151,8 @@ rb_allocate_cpu_buffer(struct trace_buffer *buffer, l= ong nr_pages, int cpu) if (ret < 0) goto fail_free_reader; =20 + rb_meta_validate_events(cpu_buffer); + /* If the boot meta was valid then this has already been updated */ meta =3D cpu_buffer->ring_meta; if (!meta || !meta->head_buffer || @@ -3955,11 +4097,10 @@ static void check_buffer(struct ring_buffer_per_cpu= *cpu_buffer, struct rb_event_info *info, unsigned long tail) { - struct ring_buffer_event *event; struct buffer_data_page *bpage; u64 ts, delta; bool full =3D false; - int e; + int ret; =20 bpage =3D info->tail_page->page; =20 @@ -3985,39 +4126,12 @@ static void check_buffer(struct ring_buffer_per_cpu= *cpu_buffer, if (atomic_inc_return(this_cpu_ptr(&checking)) !=3D 1) goto out; =20 - ts =3D bpage->time_stamp; - - for (e =3D 0; e < tail; e +=3D rb_event_length(event)) { - - event =3D (struct ring_buffer_event *)(bpage->data + e); - - switch (event->type_len) { - - case RINGBUF_TYPE_TIME_EXTEND: - delta =3D rb_event_time_stamp(event); - ts +=3D delta; - break; - - case RINGBUF_TYPE_TIME_STAMP: - delta =3D rb_event_time_stamp(event); - delta =3D rb_fix_abs_ts(delta, ts); - if (delta < ts) { - buffer_warn_return("[CPU: %d]ABSOLUTE TIME WENT BACKWARDS: last ts: %l= ld absolute ts: %lld\n", - cpu_buffer->cpu, ts, delta); - } - ts =3D delta; - break; - - case RINGBUF_TYPE_PADDING: - if (event->time_delta =3D=3D 1) - break; - fallthrough; - case RINGBUF_TYPE_DATA: - ts +=3D event->time_delta; - break; - - default: - RB_WARN_ON(cpu_buffer, 1); + ret =3D rb_read_data_buffer(bpage, tail, cpu_buffer->cpu, &ts, &delta); + if (ret < 0) { + if (delta < ts) { + buffer_warn_return("[CPU: %d]ABSOLUTE TIME WENT BACKWARDS: last ts: %ll= d absolute ts: %lld\n", + cpu_buffer->cpu, ts, delta); + goto out; } } if ((full && ts > info->ts) || --=20 2.43.0