slub: keep empty main sheaf as spare in __pcs_replace_empty_main()

[PATCH v2] slub: keep empty main sheaf as spare in __pcs_replace_empty_main()

Posted by Hao Li 1 week, 1 day ago

From: Hao Li <haolee.swjtu@gmail.com>

When __pcs_replace_empty_main() fails to obtain a full sheaf directly
from the barn, it may either:

  - Refill an empty sheaf obtained via barn_get_empty_sheaf(), or
  - Allocate a brand new full sheaf via alloc_full_sheaf().

After reacquiring the per-CPU lock, if pcs->main is still empty and
pcs->spare is NULL, the current code donates the empty main sheaf to
the barn via barn_put_empty_sheaf() and installs the full sheaf as
pcs->main, leaving pcs->spare unpopulated.

Instead, keep the existing empty main sheaf locally as the spare:

  pcs->spare = pcs->main;
  pcs->main = full;

This populates pcs->spare earlier, which can reduce future barn traffic.

Suggested-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Hao Li <haolee.swjtu@gmail.com>
---

The Gmail account(haoli.tcs) I used to send v1 of the patch has been
restricted from sending emails for unknown reasons, so I'm sending v2
from this address instead. Thanks.

 mm/slub.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/mm/slub.c b/mm/slub.c
index a0b905c2a557..a3e73ebb0cc8 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -5077,6 +5077,11 @@ __pcs_replace_empty_main(struct kmem_cache *s, struct slub_percpu_sheaves *pcs,
 	 */
 
 	if (pcs->main->size == 0) {
+		if (!pcs->spare) {
+			pcs->spare = pcs->main;
+			pcs->main = full;
+			return pcs;
+		}
 		barn_put_empty_sheaf(barn, pcs->main);
 		pcs->main = full;
 		return pcs;
-- 
2.50.1

Re: [PATCH v2] slub: keep empty main sheaf as spare in __pcs_replace_empty_main()

Posted by Vlastimil Babka 2 days, 10 hours ago

On 12/10/25 01:26, Hao Li wrote:
> From: Hao Li <haolee.swjtu@gmail.com>
> 
> When __pcs_replace_empty_main() fails to obtain a full sheaf directly
> from the barn, it may either:
> 
>   - Refill an empty sheaf obtained via barn_get_empty_sheaf(), or
>   - Allocate a brand new full sheaf via alloc_full_sheaf().
> 
> After reacquiring the per-CPU lock, if pcs->main is still empty and
> pcs->spare is NULL, the current code donates the empty main sheaf to
> the barn via barn_put_empty_sheaf() and installs the full sheaf as
> pcs->main, leaving pcs->spare unpopulated.
> 
> Instead, keep the existing empty main sheaf locally as the spare:
> 
>   pcs->spare = pcs->main;
>   pcs->main = full;
> 
> This populates pcs->spare earlier, which can reduce future barn traffic.
> 
> Suggested-by: Vlastimil Babka <vbabka@suse.cz>
> Signed-off-by: Hao Li <haolee.swjtu@gmail.com>
> ---
> 
> The Gmail account(haoli.tcs) I used to send v1 of the patch has been
> restricted from sending emails for unknown reasons, so I'm sending v2
> from this address instead. Thanks.
> 
>  mm/slub.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index a0b905c2a557..a3e73ebb0cc8 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -5077,6 +5077,11 @@ __pcs_replace_empty_main(struct kmem_cache *s, struct slub_percpu_sheaves *pcs,
>  	 */
>  
>  	if (pcs->main->size == 0) {
> +		if (!pcs->spare) {
> +			pcs->spare = pcs->main;
> +			pcs->main = full;
> +			return pcs;
> +		}
>  		barn_put_empty_sheaf(barn, pcs->main);
>  		pcs->main = full;
>  		return pcs;

Thanks, LGTM. We can make it smaller though. Adding to slab/for-next
adjusted like this:

diff --git a/mm/slub.c b/mm/slub.c
index f21b2f0c6f5a..ad71f01571f0 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -5052,7 +5052,11 @@ __pcs_replace_empty_main(struct kmem_cache *s, struct slub_percpu_sheaves *pcs,
         */
 
        if (pcs->main->size == 0) {
-               barn_put_empty_sheaf(barn, pcs->main);
+               if (!pcs->spare) {
+                       pcs->spare = pcs->main;
+               } else {
+                       barn_put_empty_sheaf(barn, pcs->main);
+               }
                pcs->main = full;
                return pcs;
        }

Re: [PATCH v2] slub: keep empty main sheaf as spare in __pcs_replace_empty_main()

Posted by Hao Lee 1 day, 22 hours ago

On Mon, Dec 15, 2025 at 10:30 PM Vlastimil Babka <vbabka@suse.cz> wrote:
>
> On 12/10/25 01:26, Hao Li wrote:
> > From: Hao Li <haolee.swjtu@gmail.com>
> >
> > When __pcs_replace_empty_main() fails to obtain a full sheaf directly
> > from the barn, it may either:
> >
> >   - Refill an empty sheaf obtained via barn_get_empty_sheaf(), or
> >   - Allocate a brand new full sheaf via alloc_full_sheaf().
> >
> > After reacquiring the per-CPU lock, if pcs->main is still empty and
> > pcs->spare is NULL, the current code donates the empty main sheaf to
> > the barn via barn_put_empty_sheaf() and installs the full sheaf as
> > pcs->main, leaving pcs->spare unpopulated.
> >
> > Instead, keep the existing empty main sheaf locally as the spare:
> >
> >   pcs->spare = pcs->main;
> >   pcs->main = full;
> >
> > This populates pcs->spare earlier, which can reduce future barn traffic.
> >
> > Suggested-by: Vlastimil Babka <vbabka@suse.cz>
> > Signed-off-by: Hao Li <haolee.swjtu@gmail.com>
> > ---
> >
> > The Gmail account(haoli.tcs) I used to send v1 of the patch has been
> > restricted from sending emails for unknown reasons, so I'm sending v2
> > from this address instead. Thanks.
> >
> >  mm/slub.c | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/mm/slub.c b/mm/slub.c
> > index a0b905c2a557..a3e73ebb0cc8 100644
> > --- a/mm/slub.c
> > +++ b/mm/slub.c
> > @@ -5077,6 +5077,11 @@ __pcs_replace_empty_main(struct kmem_cache *s, struct slub_percpu_sheaves *pcs,
> >        */
> >
> >       if (pcs->main->size == 0) {
> > +             if (!pcs->spare) {
> > +                     pcs->spare = pcs->main;
> > +                     pcs->main = full;
> > +                     return pcs;
> > +             }
> >               barn_put_empty_sheaf(barn, pcs->main);
> >               pcs->main = full;
> >               return pcs;
>
> Thanks, LGTM. We can make it smaller though. Adding to slab/for-next
> adjusted like this:

Thanks!

>
> diff --git a/mm/slub.c b/mm/slub.c
> index f21b2f0c6f5a..ad71f01571f0 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -5052,7 +5052,11 @@ __pcs_replace_empty_main(struct kmem_cache *s, struct slub_percpu_sheaves *pcs,
>          */
>
>         if (pcs->main->size == 0) {
> -               barn_put_empty_sheaf(barn, pcs->main);
> +               if (!pcs->spare) {
> +                       pcs->spare = pcs->main;
> +               } else {
> +                       barn_put_empty_sheaf(barn, pcs->main);
> +               }

Nice simplification. Thanks!

>                 pcs->main = full;
>                 return pcs;
>         }
>
>
>