From nobody Mon Apr 6 15:42:22 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B72EA3BA232; Thu, 19 Mar 2026 09:12:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773911564; cv=none; b=W39hymuKMojNBhYJP0GsxwXpRescDFjuGTVP2axDtK/VmtcPgOBEKUcecuU1XQz2PTsPW9wrDdlPSEnJiQl7MFhtXbK/vowS7bigjlZ8IgqjfyDXL9jJByphNU+gH2ppTifoAY9siwrIIcnf1P1bIo4+Qf69ELyuDXW2Rl8L1tI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773911564; c=relaxed/simple; bh=KYJEoFi/bHWkYAzYeWUC5mBd7mk1SQnlrKLaY+qZvJA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MuYZyvNoprkpjP+A4Bil0X3Sr7CYXsVYcANkUncJUQ2qAmZJxjNt5hB1AP1UKys4M1/uzn5//pWvkW2soQI9V9rFm0JozfzDRCe9meVHXA1oSnK7svUB+FaUpFNJseEItyNOH9hj9G2oRUTHmRwuo2kxyqinBFuJviw+SDRZ9ls= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=H3KViRpL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="H3KViRpL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C3CADC19424; Thu, 19 Mar 2026 09:12:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1773911564; bh=KYJEoFi/bHWkYAzYeWUC5mBd7mk1SQnlrKLaY+qZvJA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H3KViRpL2eje/LzWV9690MoqxreRlGWJik4QsSdTyePmnLw7djS0FtLQb80T4qAcj fRqjITeRF4DBbJCd6vJp3B6fW+9SGWw6JHeLExtJgtCRxG/F6LdR+X2tgewSIXNZU0 3YSS88o8Stz7KRZVLpLJgYf7lozRIshtm8fz0r+X6aa+axTG5pqpdrUUJvihEqkQje k04MNgCZ/3ioxvgTB638fDruPs5erpKE6emSMVM67N7Jwjwl3Wff1wutudoAE75rU7 5B9wS6oXab6ABomDFU6H+a9t57dRp55+rLHEoQl3fNmVYon8y7UNpJhq2dd1V6oAYJ yyLUPoS5XMvYQ== From: "Masami Hiramatsu (Google)" To: Steven Rostedt Cc: Masami Hiramatsu , Mathieu Desnoyers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Ian Rogers Subject: [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding persistent ring buffer Date: Thu, 19 Mar 2026 18:12:42 +0900 Message-ID: <177391156211.193994.7531495945584650297.stgit@mhiramat.tok.corp.google.com> X-Mailer: git-send-email 2.53.0.851.ga537e3e6e9-goog In-Reply-To: <177391152793.193994.8986943289250629418.stgit@mhiramat.tok.corp.google.com> References: <177391152793.193994.8986943289250629418.stgit@mhiramat.tok.corp.google.com> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Skip invalid sub-buffers when rewinding the persistent ring buffer instead of stopping the rewinding the ring buffer. The skipped buffers are cleared. To ensure the rewinding stops at the unused page, this also clears buffer_data_page::time_stamp when tracing resets the buffer. This allows us to identify unused pages and empty pages. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v11: - Reset timestamp when the buffer is invalid. - When rewinding, skip subbuf page if timestamp is wrong and check timestamp after validating buffer data page. Changes in v10: - Newly added. --- 0 files changed diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 67826021867b..b436d2982c5e 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -363,6 +363,7 @@ struct buffer_page { static void rb_init_page(struct buffer_data_page *bpage) { local_set(&bpage->commit, 0); + bpage->time_stamp =3D 0; } =20 static __always_inline unsigned int rb_page_commit(struct buffer_page *bpa= ge) @@ -1878,12 +1879,14 @@ static int rb_read_data_buffer(struct buffer_data_p= age *dpage, int tail, int cpu return events; } =20 -static int rb_validate_buffer(struct buffer_data_page *dpage, int cpu, +static int rb_validate_buffer(struct buffer_page *bpage, int cpu, struct ring_buffer_cpu_meta *meta) { + struct buffer_data_page *dpage =3D bpage->page; unsigned long long ts; unsigned long tail; u64 delta; + int ret =3D -1; =20 /* * When a sub-buffer is recovered from a read, the commit value may @@ -1892,9 +1895,17 @@ static int rb_validate_buffer(struct buffer_data_pag= e *dpage, int cpu, * subbuf_size is considered invalid. */ tail =3D local_read(&dpage->commit) & ~RB_MISSED_MASK; - if (tail > meta->subbuf_size) - return -1; - return rb_read_data_buffer(dpage, tail, cpu, &ts, &delta); + if (tail <=3D meta->subbuf_size) + ret =3D rb_read_data_buffer(dpage, tail, cpu, &ts, &delta); + + if (ret < 0) { + local_set(&bpage->entries, 0); + local_set(&bpage->page->commit, 0); + } else { + local_set(&bpage->entries, ret); + } + + return ret; } =20 /* If the meta data has been validated, now validate the events */ @@ -1915,18 +1926,14 @@ static void rb_meta_validate_events(struct ring_buf= fer_per_cpu *cpu_buffer) orig_head =3D head_page =3D cpu_buffer->head_page; =20 /* Do the reader page first */ - ret =3D rb_validate_buffer(cpu_buffer->reader_page->page, cpu_buffer->cpu= , meta); + ret =3D rb_validate_buffer(cpu_buffer->reader_page, cpu_buffer->cpu, meta= ); if (ret < 0) { pr_info("Ring buffer meta [%d] invalid reader page detected\n", cpu_buffer->cpu); discarded++; - /* Instead of discard whole ring buffer, discard only this sub-buffer. */ - local_set(&cpu_buffer->reader_page->entries, 0); - local_set(&cpu_buffer->reader_page->page->commit, 0); } else { entries +=3D ret; entry_bytes +=3D rb_page_size(cpu_buffer->reader_page); - local_set(&cpu_buffer->reader_page->entries, ret); } =20 ts =3D head_page->page->time_stamp; @@ -1945,26 +1952,33 @@ static void rb_meta_validate_events(struct ring_buf= fer_per_cpu *cpu_buffer) if (head_page =3D=3D cpu_buffer->tail_page) break; =20 - /* Ensure the page has older data than head. */ - if (ts < head_page->page->time_stamp) + /* Rewind until unused page (no timestamp, no commit). */ + if (!head_page->page->time_stamp && rb_page_commit(head_page) =3D=3D 0) break; =20 - ts =3D head_page->page->time_stamp; - /* Ensure the page has correct timestamp and some data. */ - if (!ts || rb_page_commit(head_page) =3D=3D 0) - break; - - /* Stop rewind if the page is invalid. */ - ret =3D rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta); - if (ret < 0) - break; - - /* Recover the number of entries and update stats. */ - local_set(&head_page->entries, ret); - if (ret) - local_inc(&cpu_buffer->pages_touched); - entries +=3D ret; - entry_bytes +=3D rb_page_commit(head_page); + /* + * Skip if the page is invalid, or its timestamp is newer than the + * previous valid page. + */ + ret =3D rb_validate_buffer(head_page, cpu_buffer->cpu, meta); + if (ret >=3D 0 && ts < head_page->page->time_stamp) { + local_set(&bpage->entries, 0); + local_set(&bpage->page->commit, 0); + head_page->page->time_stamp =3D ts; + ret =3D -1; + } + if (ret < 0) { + if (!discarded) + pr_info("Ring buffer meta [%d] invalid buffer page detected\n", + cpu_buffer->cpu); + discarded++; + } else { + entries +=3D ret; + entry_bytes +=3D rb_page_size(head_page); + if (ret > 0) + local_inc(&cpu_buffer->pages_touched); + ts =3D head_page->page->time_stamp; + } } if (i) pr_info("Ring buffer [%d] rewound %d pages\n", cpu_buffer->cpu, i); @@ -2034,15 +2048,12 @@ static void rb_meta_validate_events(struct ring_buf= fer_per_cpu *cpu_buffer) if (head_page =3D=3D cpu_buffer->reader_page) continue; =20 - ret =3D rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta); + ret =3D rb_validate_buffer(head_page, cpu_buffer->cpu, meta); if (ret < 0) { if (!discarded) pr_info("Ring buffer meta [%d] invalid buffer page detected\n", cpu_buffer->cpu); discarded++; - /* Instead of discard whole ring buffer, discard only this sub-buffer. = */ - local_set(&head_page->entries, 0); - local_set(&head_page->page->commit, 0); } else { /* If the buffer has content, update pages_touched */ if (ret) @@ -2050,7 +2061,6 @@ static void rb_meta_validate_events(struct ring_buffe= r_per_cpu *cpu_buffer) =20 entries +=3D ret; entry_bytes +=3D rb_page_size(head_page); - local_set(&head_page->entries, ret); } if (head_page =3D=3D cpu_buffer->commit_page) break; @@ -2081,7 +2091,7 @@ static void rb_meta_validate_events(struct ring_buffe= r_per_cpu *cpu_buffer) /* Reset all the subbuffers */ for (i =3D 0; i < meta->nr_subbufs - 1; i++, rb_inc_page(&head_page)) { local_set(&head_page->entries, 0); - local_set(&head_page->page->commit, 0); + rb_init_page(head_page->page); } }