Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents
where architecturally supported.
This is only available with !CONFIG_HIGHMEM because the intent is to
use architecture support to clear contiguous extents in a single
operation (ex. via FEAT_MOPS on arm64, string instructions on x86)
which excludes any possibility of interspersing kmap()/kunmap().
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
mm/Kconfig | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/mm/Kconfig b/mm/Kconfig
index 781be3240e21..a74a5e02de28 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -910,6 +910,15 @@ config NO_PAGE_MAPCOUNT
endif # TRANSPARENT_HUGEPAGE
+config CLEAR_PAGE_EXTENT
+ def_bool y
+ depends on !HIGHMEM && ARCH_HAS_CLEAR_PAGES
+ depends on TRANSPARENT_HUGEPAGE || HUGETLBFS
+
+ help
+ Use architectural support for clear_pages() to zero page-extents.
+ This is likely to be faster than zeroing page-at-a-time.
+
# simple helper to make the code a bit easier to read
config PAGE_MAPCOUNT
def_bool !NO_PAGE_MAPCOUNT
--
2.43.5
On 10.07.25 02:59, Ankur Arora wrote: > Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents > where architecturally supported. > > This is only available with !CONFIG_HIGHMEM because the intent is to > use architecture support to clear contiguous extents in a single > operation (ex. via FEAT_MOPS on arm64, string instructions on x86) > which excludes any possibility of interspersing kmap()/kunmap(). > > Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> > --- Staring at the next patch, I think this can easily be squashed into the next patch where you add actual MM core support. -- Cheers, David / dhildenb
David Hildenbrand <david@redhat.com> writes: > On 10.07.25 02:59, Ankur Arora wrote: >> Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents >> where architecturally supported. >> This is only available with !CONFIG_HIGHMEM because the intent is to >> use architecture support to clear contiguous extents in a single >> operation (ex. via FEAT_MOPS on arm64, string instructions on x86) >> which excludes any possibility of interspersing kmap()/kunmap(). >> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> >> --- > > Staring at the next patch, I think this can easily be squashed into the next > patch where you add actual MM core support. I wanted to do this in a separate patch to explicitly document what the responsibility of the interface provided by the architecture is. That said, the commit message didn't actually do a good job of doing that :D. Copying the more detailed commit message from my reply to Andrew, one important part of the clear_pages() is that it be interruptible because clear_pages_resched() implicitly depends on it. > This is only enabled with !CONFIG_HIGHMEM because the intent is > to use architecture support to clear contiguous extents in a > single interruptible operation (ex. via FEAT_MOPS on arm64, > string instructions on x86). > Given that we might be zeroing the whole extent with a single > instruction, this excludes any possibility of constructing > intermediate HIGHMEM maps. Do you think it is best documented in the next patch in a comment instead? -- ankur
On 11.07.25 19:32, Ankur Arora wrote: > > David Hildenbrand <david@redhat.com> writes: > >> On 10.07.25 02:59, Ankur Arora wrote: >>> Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents >>> where architecturally supported. >>> This is only available with !CONFIG_HIGHMEM because the intent is to >>> use architecture support to clear contiguous extents in a single >>> operation (ex. via FEAT_MOPS on arm64, string instructions on x86) >>> which excludes any possibility of interspersing kmap()/kunmap(). >>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> >>> --- >> >> Staring at the next patch, I think this can easily be squashed into the next >> patch where you add actual MM core support. > > I wanted to do this in a separate patch to explicitly document what the > responsibility of the interface provided by the architecture is. > That said, the commit message didn't actually do a good job of doing > that :D. > > Copying the more detailed commit message from my reply to Andrew, > one important part of the clear_pages() is that it be interruptible > because clear_pages_resched() implicitly depends on it. > >> This is only enabled with !CONFIG_HIGHMEM because the intent is >> to use architecture support to clear contiguous extents in a >> single interruptible operation (ex. via FEAT_MOPS on arm64, >> string instructions on x86). > >> Given that we might be zeroing the whole extent with a single >> instruction, this excludes any possibility of constructing >> intermediate HIGHMEM maps. > > Do you think it is best documented in the next patch in a comment > instead? I would just add + document it as part of the next patch. Looking at the bigger picture now, you introduce ARCH_HAS_CLEAR_PAGES To say whether an architecture provides clear_pages(). Now we want to conditionally use that to optimize folio_zero_user(). Remind me, why do we want to glue this to THP / HUGETLBFS only? I would assume that the code footprint is rather small, and the systems out there that are compiled with ARCH_HAS_CLEAR_PAGES but without THP / HUGETLBFS are rather ... rare (mostly 32BIT x86 only). -- Cheers, David / dhildenb
[ Added Zi Yan. ]
David Hildenbrand <david@redhat.com> writes:
> On 11.07.25 19:32, Ankur Arora wrote:
>> David Hildenbrand <david@redhat.com> writes:
>>
>>> On 10.07.25 02:59, Ankur Arora wrote:
>>>> Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents
>>>> where architecturally supported.
>>>> This is only available with !CONFIG_HIGHMEM because the intent is to
>>>> use architecture support to clear contiguous extents in a single
>>>> operation (ex. via FEAT_MOPS on arm64, string instructions on x86)
>>>> which excludes any possibility of interspersing kmap()/kunmap().
>>>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>>>> ---
>>>
>>> Staring at the next patch, I think this can easily be squashed into the next
>>> patch where you add actual MM core support.
>> I wanted to do this in a separate patch to explicitly document what the
>> responsibility of the interface provided by the architecture is.
>> That said, the commit message didn't actually do a good job of doing
>> that :D.
>> Copying the more detailed commit message from my reply to Andrew,
>> one important part of the clear_pages() is that it be interruptible
>> because clear_pages_resched() implicitly depends on it.
>>
>>> This is only enabled with !CONFIG_HIGHMEM because the intent is
>>> to use architecture support to clear contiguous extents in a
>>> single interruptible operation (ex. via FEAT_MOPS on arm64,
>>> string instructions on x86).
>>
>>> Given that we might be zeroing the whole extent with a single
>>> instruction, this excludes any possibility of constructing
>>> intermediate HIGHMEM maps.
>> Do you think it is best documented in the next patch in a comment
>> instead?
>
> I would just add + document it as part of the next patch.
>
> Looking at the bigger picture now, you introduce
>
> ARCH_HAS_CLEAR_PAGES
>
> To say whether an architecture provides clear_pages().
>
> Now we want to conditionally use that to optimize folio_zero_user().
>
> Remind me, why do we want to glue this to THP / HUGETLBFS only? I would assume
> that the code footprint is rather small, and the systems out there that are
> compiled with ARCH_HAS_CLEAR_PAGES but without THP / HUGETLBFS are rather ...
> rare (mostly 32BIT x86 only).
I thought about this some more and there are a few other interfaces that
end up clearing pages:
> clear_highpage()
> clear_highpage_kasan_tagged()
> tag_clear_highpage()
In this set, there are many loops of the form:
for (i = 0; i < n; i++)
clear_highpage();
At least some of these (including kernel_init_pages()) could be migrated
to variations on a clear_highpages() which could be:
static inline void clear_highpages(struct page *page, u32 num_pages)
{
if (!IS_ENABLED(CONFIG_HIGHMEM))
clear_pages_resched(page, num_pages);
else
for (i = 0; i < num_pages; ++i)
clear_highpage(page + i);
}
(clear_pages_resched() should be safe to be used from here because
everybody using this should be in a schedulable context.)
(The kernel_init_pages() was also suggested by Zi Yan in a review of v3 [1].)
> clear_user_highpage()
Only users folio_zero_user(), __collapse_huge_page_copy() and
userfaultd.
> clear_user_page()
Not many users apart from the highmem interface.
> clear_page()
Not many users apart from the highmem interface.
I'm happy to do this work, just not sure how to stage it. In particular I
would like to avoid a series which tries to address all of the cases.
Maybe it makes sense to handle just add the clear_highpages() variants,
folio_zero_user() handling and some of the obvious users of
clear_highpages() for v6?
Thanks
Ankur
[1] https://lore.kernel.org/lkml/AC2C5344-E655-45BB-B90B-D63C4AC8F2F6@nvidia.com/
On 14.07.25 22:35, Ankur Arora wrote:
>
> [ Added Zi Yan. ]
>
> David Hildenbrand <david@redhat.com> writes:
>
>> On 11.07.25 19:32, Ankur Arora wrote:
>>> David Hildenbrand <david@redhat.com> writes:
>>>
>>>> On 10.07.25 02:59, Ankur Arora wrote:
>>>>> Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents
>>>>> where architecturally supported.
>>>>> This is only available with !CONFIG_HIGHMEM because the intent is to
>>>>> use architecture support to clear contiguous extents in a single
>>>>> operation (ex. via FEAT_MOPS on arm64, string instructions on x86)
>>>>> which excludes any possibility of interspersing kmap()/kunmap().
>>>>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>>>>> ---
>>>>
>>>> Staring at the next patch, I think this can easily be squashed into the next
>>>> patch where you add actual MM core support.
>>> I wanted to do this in a separate patch to explicitly document what the
>>> responsibility of the interface provided by the architecture is.
>>> That said, the commit message didn't actually do a good job of doing
>>> that :D.
>>> Copying the more detailed commit message from my reply to Andrew,
>>> one important part of the clear_pages() is that it be interruptible
>>> because clear_pages_resched() implicitly depends on it.
>>>
>>>> This is only enabled with !CONFIG_HIGHMEM because the intent is
>>>> to use architecture support to clear contiguous extents in a
>>>> single interruptible operation (ex. via FEAT_MOPS on arm64,
>>>> string instructions on x86).
>>>
>>>> Given that we might be zeroing the whole extent with a single
>>>> instruction, this excludes any possibility of constructing
>>>> intermediate HIGHMEM maps.
>>> Do you think it is best documented in the next patch in a comment
>>> instead?
>>
>> I would just add + document it as part of the next patch.
>>
>> Looking at the bigger picture now, you introduce
>>
>> ARCH_HAS_CLEAR_PAGES
>>
>> To say whether an architecture provides clear_pages().
>>
>> Now we want to conditionally use that to optimize folio_zero_user().
>>
>> Remind me, why do we want to glue this to THP / HUGETLBFS only? I would assume
>> that the code footprint is rather small, and the systems out there that are
>> compiled with ARCH_HAS_CLEAR_PAGES but without THP / HUGETLBFS are rather ...
>> rare (mostly 32BIT x86 only).
Agreed.
>
> I thought about this some more and there are a few other interfaces that
> end up clearing pages:
>
>> clear_highpage()
>> clear_highpage_kasan_tagged()
>> tag_clear_highpage()
>
> In this set, there are many loops of the form:
>
> for (i = 0; i < n; i++)
> clear_highpage();
>
> At least some of these (including kernel_init_pages()) could be migrated
> to variations on a clear_highpages() which could be:
>
> static inline void clear_highpages(struct page *page, u32 num_pages)
> {
> if (!IS_ENABLED(CONFIG_HIGHMEM))
> clear_pages_resched(page, num_pages);
> else
> for (i = 0; i < num_pages; ++i)
> clear_highpage(page + i);
> }
>
> (clear_pages_resched() should be safe to be used from here because
> everybody using this should be in a schedulable context.)
>
> (The kernel_init_pages() was also suggested by Zi Yan in a review of v3 [1].)
>
>> clear_user_highpage()
>
> Only users folio_zero_user(), __collapse_huge_page_copy() and
> userfaultd.
>
>> clear_user_page()
> Not many users apart from the highmem interface.
>
>> clear_page()
>
> Not many users apart from the highmem interface.
>
> I'm happy to do this work, just not sure how to stage it. In particular I
> would like to avoid a series which tries to address all of the cases.
>
> Maybe it makes sense to handle just add the clear_highpages() variants,
> folio_zero_user() handling and some of the obvious users of
> clear_highpages() for v6?
Yes, no need for excessive handling.
What I was getting at was: could we get rid of the kconfig option and
simply glue it to the availability of clear_pages() in a reasonable way.
--
Cheers,
David / dhildenb
David Hildenbrand <david@redhat.com> writes: > On 11.07.25 19:32, Ankur Arora wrote: >> David Hildenbrand <david@redhat.com> writes: >> >>> On 10.07.25 02:59, Ankur Arora wrote: >>>> Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents >>>> where architecturally supported. >>>> This is only available with !CONFIG_HIGHMEM because the intent is to >>>> use architecture support to clear contiguous extents in a single >>>> operation (ex. via FEAT_MOPS on arm64, string instructions on x86) >>>> which excludes any possibility of interspersing kmap()/kunmap(). >>>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> >>>> --- >>> >>> Staring at the next patch, I think this can easily be squashed into the next >>> patch where you add actual MM core support. >> I wanted to do this in a separate patch to explicitly document what the >> responsibility of the interface provided by the architecture is. >> That said, the commit message didn't actually do a good job of doing >> that :D. >> Copying the more detailed commit message from my reply to Andrew, >> one important part of the clear_pages() is that it be interruptible >> because clear_pages_resched() implicitly depends on it. >> >>> This is only enabled with !CONFIG_HIGHMEM because the intent is >>> to use architecture support to clear contiguous extents in a >>> single interruptible operation (ex. via FEAT_MOPS on arm64, >>> string instructions on x86). >> >>> Given that we might be zeroing the whole extent with a single >>> instruction, this excludes any possibility of constructing >>> intermediate HIGHMEM maps. >> Do you think it is best documented in the next patch in a comment >> instead? > > I would just add + document it as part of the next patch. > > Looking at the bigger picture now, you introduce > > ARCH_HAS_CLEAR_PAGES > > To say whether an architecture provides clear_pages(). > > Now we want to conditionally use that to optimize folio_zero_user(). > > Remind me, why do we want to glue this to THP / HUGETLBFS only? I would assume > that the code footprint is rather small, and the systems out there that are > compiled with ARCH_HAS_CLEAR_PAGES but without THP / HUGETLBFS are rather ... > rare (mostly 32BIT x86 only). A failure of imagination, I guess. I couldn't think of a way anyone would have to operate on contiguous pages without also having THP or HUGETLBFS. And, for the cases where we want to operate on kernel ranges, we could directly use clear_pages(). (Though that needs some work to add a generic wrapper.) -- ankur
On 10.07.25 02:59, Ankur Arora wrote: > Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents > where architecturally supported. > > This is only available with !CONFIG_HIGHMEM because the intent is to > use architecture support to clear contiguous extents in a single > operation (ex. via FEAT_MOPS on arm64, string instructions on x86) > which excludes any possibility of interspersing kmap()/kunmap(). > > Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> > --- > mm/Kconfig | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/mm/Kconfig b/mm/Kconfig > index 781be3240e21..a74a5e02de28 100644 > --- a/mm/Kconfig > +++ b/mm/Kconfig > @@ -910,6 +910,15 @@ config NO_PAGE_MAPCOUNT > > endif # TRANSPARENT_HUGEPAGE > > +config CLEAR_PAGE_EXTENT > + def_bool y > + depends on !HIGHMEM && ARCH_HAS_CLEAR_PAGES > + depends on TRANSPARENT_HUGEPAGE || HUGETLBFS > + > + help > + Use architectural support for clear_pages() to zero page-extents. > + This is likely to be faster than zeroing page-at-a-time. > + IIRC, adding a help text will make this option be configurable by the user, no? This seems like something we can just keep internal, right? -- Cheers, David / dhildenb
David Hildenbrand <david@redhat.com> writes: > On 10.07.25 02:59, Ankur Arora wrote: >> Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents >> where architecturally supported. >> This is only available with !CONFIG_HIGHMEM because the intent is to >> use architecture support to clear contiguous extents in a single >> operation (ex. via FEAT_MOPS on arm64, string instructions on x86) >> which excludes any possibility of interspersing kmap()/kunmap(). >> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> >> --- >> mm/Kconfig | 9 +++++++++ >> 1 file changed, 9 insertions(+) >> diff --git a/mm/Kconfig b/mm/Kconfig >> index 781be3240e21..a74a5e02de28 100644 >> --- a/mm/Kconfig >> +++ b/mm/Kconfig >> @@ -910,6 +910,15 @@ config NO_PAGE_MAPCOUNT >> endif # TRANSPARENT_HUGEPAGE >> +config CLEAR_PAGE_EXTENT >> + def_bool y >> + depends on !HIGHMEM && ARCH_HAS_CLEAR_PAGES >> + depends on TRANSPARENT_HUGEPAGE || HUGETLBFS >> + >> + help >> + Use architectural support for clear_pages() to zero page-extents. >> + This is likely to be faster than zeroing page-at-a-time. >> + > > IIRC, adding a help text will make this option be configurable by the user, no? I tried changing the value in the generated .config by hand and that reverted back to the computed value. So, I think this isn't configurable by the user. From a quick look at the kbuild documentation, for this to be user selectable it needs a prompt of some kind. That said, the help text doesn't add anything here. Will remove. > This seems like something we can just keep internal, right? Agreed. Thanks for the review! -- ankur
On 11.07.25 19:25, Ankur Arora wrote: > > David Hildenbrand <david@redhat.com> writes: > >> On 10.07.25 02:59, Ankur Arora wrote: >>> Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents >>> where architecturally supported. >>> This is only available with !CONFIG_HIGHMEM because the intent is to >>> use architecture support to clear contiguous extents in a single >>> operation (ex. via FEAT_MOPS on arm64, string instructions on x86) >>> which excludes any possibility of interspersing kmap()/kunmap(). >>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> >>> --- >>> mm/Kconfig | 9 +++++++++ >>> 1 file changed, 9 insertions(+) >>> diff --git a/mm/Kconfig b/mm/Kconfig >>> index 781be3240e21..a74a5e02de28 100644 >>> --- a/mm/Kconfig >>> +++ b/mm/Kconfig >>> @@ -910,6 +910,15 @@ config NO_PAGE_MAPCOUNT >>> endif # TRANSPARENT_HUGEPAGE >>> +config CLEAR_PAGE_EXTENT >>> + def_bool y >>> + depends on !HIGHMEM && ARCH_HAS_CLEAR_PAGES >>> + depends on TRANSPARENT_HUGEPAGE || HUGETLBFS >>> + >>> + help >>> + Use architectural support for clear_pages() to zero page-extents. >>> + This is likely to be faster than zeroing page-at-a-time. >>> + >> >> IIRC, adding a help text will make this option be configurable by the user, no? > > I tried changing the value in the generated .config by hand and that > reverted back to the computed value. So, I think this isn't configurable > by the user. Yes, I misremembered and the help text confused me. So yes, the help text in that case (internal entries) is uncommon (no need to guide the user when there are no options :) ), but feel free to keep it for documentation purposes. (e.g., HAVE_64BIT_ALIGNED_ACCESS does that) Maybe we should call it similarly HAVE_... to make it clearer that there are really no options. But I'm bad at #kconfig, so whatever you prefer. -- Cheers, David / dhildenb
David Hildenbrand <david@redhat.com> writes: > On 11.07.25 19:25, Ankur Arora wrote: >> David Hildenbrand <david@redhat.com> writes: >> >>> On 10.07.25 02:59, Ankur Arora wrote: >>>> Add CONFIG_CLEAR_PAGE_EXTENT to allow clearing of page-extents >>>> where architecturally supported. >>>> This is only available with !CONFIG_HIGHMEM because the intent is to >>>> use architecture support to clear contiguous extents in a single >>>> operation (ex. via FEAT_MOPS on arm64, string instructions on x86) >>>> which excludes any possibility of interspersing kmap()/kunmap(). >>>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> >>>> --- >>>> mm/Kconfig | 9 +++++++++ >>>> 1 file changed, 9 insertions(+) >>>> diff --git a/mm/Kconfig b/mm/Kconfig >>>> index 781be3240e21..a74a5e02de28 100644 >>>> --- a/mm/Kconfig >>>> +++ b/mm/Kconfig >>>> @@ -910,6 +910,15 @@ config NO_PAGE_MAPCOUNT >>>> endif # TRANSPARENT_HUGEPAGE >>>> +config CLEAR_PAGE_EXTENT >>>> + def_bool y >>>> + depends on !HIGHMEM && ARCH_HAS_CLEAR_PAGES >>>> + depends on TRANSPARENT_HUGEPAGE || HUGETLBFS >>>> + >>>> + help >>>> + Use architectural support for clear_pages() to zero page-extents. >>>> + This is likely to be faster than zeroing page-at-a-time. >>>> + >>> >>> IIRC, adding a help text will make this option be configurable by the user, no? >> I tried changing the value in the generated .config by hand and that >> reverted back to the computed value. So, I think this isn't configurable >> by the user. > > Yes, I misremembered and the help text confused me. > > So yes, the help text in that case (internal entries) is uncommon (no need to > guide the user when there are no options :) ), but feel free to keep it for > documentation purposes. > > (e.g., HAVE_64BIT_ALIGNED_ACCESS does that) > > Maybe we should call it similarly HAVE_... to make it clearer that there are > really no options. Actually a prefix is a good idea. Btw, the natural follow-up (to me anyway) to this series would be to also do this for the copy path. From some quick tests on x86, the performance improvement was similar. So, maybe it could just be something like: USE_CONTIG_PAGES. But that needs a verb somewhere. > But I'm bad at #kconfig, so whatever you prefer. Not much better, as you can see. -- ankur
On Wed, 9 Jul 2025 17:59:24 -0700 Ankur Arora <ankur.a.arora@oracle.com> wrote: > This is only available with !CONFIG_HIGHMEM because the intent is to > use architecture support to clear contiguous extents in a single > operation (ex. via FEAT_MOPS on arm64, string instructions on x86) > which excludes any possibility of interspersing kmap()/kunmap(). I'm feeling dumb. What does the use of a string instruction have to do with kmap/kunmap?
Andrew Morton <akpm@linux-foundation.org> writes: > On Wed, 9 Jul 2025 17:59:24 -0700 Ankur Arora <ankur.a.arora@oracle.com> wrote: > >> This is only available with !CONFIG_HIGHMEM because the intent is to >> use architecture support to clear contiguous extents in a single >> operation (ex. via FEAT_MOPS on arm64, string instructions on x86) >> which excludes any possibility of interspersing kmap()/kunmap(). > > I'm feeling dumb. What does the use of a string instruction have to do > with kmap/kunmap? Sorry, that might have been unnecessarily inscrutable. How about something like this: This is only enabled with !CONFIG_HIGHMEM because the intent is to use architecture support to clear contiguous extents in a single interruptible operation (ex. via FEAT_MOPS on arm64, string instructions on x86). Given that we might be zeroing the whole extent with a single instruction, this excludes any possibility of constructing intermediate HIGHMEM maps. -- ankur
© 2016 - 2025 Red Hat, Inc.