We need to leave lazy MMU mode before unlocking.

Fixes: b2f557a21bc8 ("mm/madvise: add cond_resched() in madvise_cold_or_pageout_pte_range()")
Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org>
---
 mm/madvise.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/madvise.c b/mm/madvise.c
index 0f222d464254..127f0c7b69ac 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -451,6 +451,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
 		if (++batch_count == SWAP_CLUSTER_MAX) {
 			batch_count = 0;
 			if (need_resched()) {
+				arch_leave_lazy_mmu_mode();
 				pte_unmap_unlock(start_pte, ptl);
 				cond_resched();
 				goto restart;
-- 
2.43.0.429.g432eaa2c6b-goog

On Fri, 26 Jan 2024 12:25:48 +0900 Sergey Senozhatsky <senozhatsky@chromium.org> wrote:

> We need to leave lazy MMU mode before unlocking.

What might be the userspace-visible effects of this?

> Fixes: b2f557a21bc8 ("mm/madvise: add cond_resched() in madvise_cold_or_pageout_pte_range()"
> Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org>

I'll add a cc:stable.

> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -451,6 +451,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
>  		if (++batch_count == SWAP_CLUSTER_MAX) {
>  			batch_count = 0;
>  			if (need_resched()) {
> +				arch_leave_lazy_mmu_mode();
>  				pte_unmap_unlock(start_pte, ptl);
>  				cond_resched();
>  				goto restart;

On (24/01/25 22:53), Andrew Morton wrote:
> Date: Thu, 25 Jan 2024 22:53:36 -0800
> From: Andrew Morton <akpm@linux-foundation.org>
> To: Sergey Senozhatsky <senozhatsky@chromium.org>
> Cc: Jiexun Wang <wangjiexun@tinylab.org>, linux-kernel@vger.kernel.org,
>  linux-mm@kvack.org
> Subject: Re: [PATCH] mm/madvise: don't forget to leave lazy MMU mode in
>  madvise_cold_or_pageout_pte_range()
> Message-Id: <20240125225336.6a444c01d9d9812a23a6890b@linux-foundation.org>
> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu)
> 
> On Fri, 26 Jan 2024 12:25:48 +0900 Sergey Senozhatsky <senozhatsky@chromium.org> wrote:
> 
> > We need to leave lazy MMU mode before unlocking.

It depends on the arch, as far as I understand it.
We can enter lazy MMU mode (on each goto restart) more times than leave
it, and, for isntance, on powerpc that means that we can preempt_disable()
more times than preempt_enable(). That's how enter/leave lazy MMU mode is
implemented there:

static inline void arch_enter_lazy_mmu_mode(void)
{
        struct ppc64_tlb_batch *batch;

        if (radix_enabled())
                return;
        /*
         * apply_to_page_range can call us this preempt enabled when
         * operating on kernel page tables.
         */
        preempt_disable();
        batch = this_cpu_ptr(&ppc64_tlb_batch);
        batch->active = 1;
}

static inline void arch_leave_lazy_mmu_mode(void)
{
        struct ppc64_tlb_batch *batch;

        if (radix_enabled())
                return;
        batch = this_cpu_ptr(&ppc64_tlb_batch);

        if (batch->index)
                __flush_tlb_pending(batch);
        batch->active = 0;
        preempt_enable();
}

> What might be the userspace-visible effects of this?
> 
> > Fixes: b2f557a21bc8 ("mm/madvise: add cond_resched() in madvise_cold_or_pageout_pte_range()"
> > Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org>
> 
> I'll add a cc:stable.

Thanks.