[PATCH v3 4/4] ceph: assert writeback loop invariants

Sam Edwards posted 4 patches 1 week, 5 days ago
[PATCH v3 4/4] ceph: assert writeback loop invariants
Posted by Sam Edwards 1 week, 5 days ago
If `locked_pages` is zero, the page array must not be allocated:
ceph_process_folio_batch() uses `locked_pages` to decide when to
allocate `pages`, and redundant allocations trigger
ceph_allocate_page_array()'s BUG_ON(), resulting in a worker oops (and
writeback stall) or even a kernel panic. Consequently, the main loop in
ceph_writepages_start() assumes that the lifetime of `pages` is confined
to a single iteration.

This expectation is currently not clear enough, as evidenced by two
recent patches which fix oopses caused by `pages` persisting into
the next loop iteration:
- "ceph: do not propagate page array emplacement errors as batch errors"
- "ceph: free page array when ceph_submit_write() fails"

Use an explicit BUG_ON() at the top of the loop to assert the loop's
preexisting expectation that `pages` is cleaned up by the previous
iteration. Because this is closely tied to `locked_pages`, also make it
the previous iteration's responsibility to guarantee its reset, and
verify with a second new BUG_ON() instead of handling (and masking)
failures to do so.

This patch does not change invariants, behavior, or failure modes.
The added BUG_ON() lines catch conditions that would already trigger oops,
but do so earlier for easier debugging and programmer clarity.

Signed-off-by: Sam Edwards <CFSworks@gmail.com>
---
 fs/ceph/addr.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index cdf11288d6b7..4e392fc70d33 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -1663,7 +1663,9 @@ static int ceph_writepages_start(struct address_space *mapping,
 		tag_pages_for_writeback(mapping, ceph_wbc.index, ceph_wbc.end);
 
 	while (!has_writeback_done(&ceph_wbc)) {
-		ceph_wbc.locked_pages = 0;
+		BUG_ON(ceph_wbc.locked_pages);
+		BUG_ON(ceph_wbc.pages);
+
 		ceph_wbc.max_pages = ceph_wbc.wsize >> PAGE_SHIFT;
 
 get_more_pages:
-- 
2.52.0
Re: [PATCH v3 4/4] ceph: assert writeback loop invariants
Posted by Viacheslav Dubeyko 1 week, 4 days ago
On Sun, 2026-01-25 at 18:30 -0800, Sam Edwards wrote:
> If `locked_pages` is zero, the page array must not be allocated:
> ceph_process_folio_batch() uses `locked_pages` to decide when to
> allocate `pages`, and redundant allocations trigger
> ceph_allocate_page_array()'s BUG_ON(), resulting in a worker oops (and
> writeback stall) or even a kernel panic. Consequently, the main loop in
> ceph_writepages_start() assumes that the lifetime of `pages` is confined
> to a single iteration.
> 
> This expectation is currently not clear enough, as evidenced by two
> recent patches which fix oopses caused by `pages` persisting into
> the next loop iteration:
> - "ceph: do not propagate page array emplacement errors as batch errors"
> - "ceph: free page array when ceph_submit_write() fails"
> 
> Use an explicit BUG_ON() at the top of the loop to assert the loop's
> preexisting expectation that `pages` is cleaned up by the previous
> iteration. Because this is closely tied to `locked_pages`, also make it
> the previous iteration's responsibility to guarantee its reset, and
> verify with a second new BUG_ON() instead of handling (and masking)
> failures to do so.
> 
> This patch does not change invariants, behavior, or failure modes.
> The added BUG_ON() lines catch conditions that would already trigger oops,
> but do so earlier for easier debugging and programmer clarity.
> 
> Signed-off-by: Sam Edwards <CFSworks@gmail.com>
> ---
>  fs/ceph/addr.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
> index cdf11288d6b7..4e392fc70d33 100644
> --- a/fs/ceph/addr.c
> +++ b/fs/ceph/addr.c
> @@ -1663,7 +1663,9 @@ static int ceph_writepages_start(struct address_space *mapping,
>  		tag_pages_for_writeback(mapping, ceph_wbc.index, ceph_wbc.end);
>  
>  	while (!has_writeback_done(&ceph_wbc)) {
> -		ceph_wbc.locked_pages = 0;
> +		BUG_ON(ceph_wbc.locked_pages);
> +		BUG_ON(ceph_wbc.pages);

My complains are still the same. I would like not have BUG_ON() here.

Thanks,
Slava.

> +
>  		ceph_wbc.max_pages = ceph_wbc.wsize >> PAGE_SHIFT;
>  
>  get_more_pages:
Re: [PATCH v3 4/4] ceph: assert writeback loop invariants
Posted by Sam Edwards 1 week, 2 days ago
On Mon, Jan 26, 2026 at 2:54 PM Viacheslav Dubeyko
<Slava.Dubeyko@ibm.com> wrote:
>
> On Sun, 2026-01-25 at 18:30 -0800, Sam Edwards wrote:
> > If `locked_pages` is zero, the page array must not be allocated:
> > ceph_process_folio_batch() uses `locked_pages` to decide when to
> > allocate `pages`, and redundant allocations trigger
> > ceph_allocate_page_array()'s BUG_ON(), resulting in a worker oops (and
> > writeback stall) or even a kernel panic. Consequently, the main loop in
> > ceph_writepages_start() assumes that the lifetime of `pages` is confined
> > to a single iteration.
> >
> > This expectation is currently not clear enough, as evidenced by two
> > recent patches which fix oopses caused by `pages` persisting into
> > the next loop iteration:
> > - "ceph: do not propagate page array emplacement errors as batch errors"
> > - "ceph: free page array when ceph_submit_write() fails"
> >
> > Use an explicit BUG_ON() at the top of the loop to assert the loop's
> > preexisting expectation that `pages` is cleaned up by the previous
> > iteration. Because this is closely tied to `locked_pages`, also make it
> > the previous iteration's responsibility to guarantee its reset, and
> > verify with a second new BUG_ON() instead of handling (and masking)
> > failures to do so.
> >
> > This patch does not change invariants, behavior, or failure modes.
> > The added BUG_ON() lines catch conditions that would already trigger oops,
> > but do so earlier for easier debugging and programmer clarity.
> >
> > Signed-off-by: Sam Edwards <CFSworks@gmail.com>
> > ---
> >  fs/ceph/addr.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
> > index cdf11288d6b7..4e392fc70d33 100644
> > --- a/fs/ceph/addr.c
> > +++ b/fs/ceph/addr.c
> > @@ -1663,7 +1663,9 @@ static int ceph_writepages_start(struct address_space *mapping,
> >               tag_pages_for_writeback(mapping, ceph_wbc.index, ceph_wbc.end);
> >
> >       while (!has_writeback_done(&ceph_wbc)) {
> > -             ceph_wbc.locked_pages = 0;
> > +             BUG_ON(ceph_wbc.locked_pages);
> > +             BUG_ON(ceph_wbc.pages);
>
> My complains are still the same. I would like not have BUG_ON() here.

Hey Slava,

I understand your preference, but as discussed before, the BUG_ON() is
defensive programming to catch an invariant violation earlier than the
existing BUG_ON() in ceph_allocate_page_array(). If the invariant is
broken, we'll oops anyway; this just makes the oops happen sooner and
easier to debug, and reduces cognitive load for future programmers
trying to understand the loop invariants.

Regards,
Sam

>
> Thanks,
> Slava.
>
> > +
> >               ceph_wbc.max_pages = ceph_wbc.wsize >> PAGE_SHIFT;
> >
> >  get_more_pages: