1. Patchew
  2. linux
  3. View series

[PATCH] kselftest: mm: fix potential failure for merged VMA in guard-regions

Yeoreum Yun posted 1 patch 2 weeks ago
There is a newer version of this series
tools/testing/selftests/mm/guard-regions.c | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
[PATCH] kselftest: mm: fix potential failure for merged VMA in guard-regions
Posted by Yeoreum Yun 2 weeks ago 
check_vmflag_guard() uses /proc/self/smaps to retrieve the VMA flags,
but this can fail if the mapping is merged with an adjacent VMA.

To avoid this potential failure, first allocate a temporary region with
extra pages at both ends, unmap it, and then map the test region within
the temporary address range, leaving an unmapped page on each side to
prevent VMA merging.

Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
---
 tools/testing/selftests/mm/guard-regions.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/mm/guard-regions.c b/tools/testing/selftests/mm/guard-regions.c
index 5c8ec3ca75d7..a28a57d34e97 100644
--- a/tools/testing/selftests/mm/guard-regions.c
+++ b/tools/testing/selftests/mm/guard-regions.c
@@ -2257,8 +2257,18 @@ TEST_F(guard_regions, smaps)
 	char *ptr, *ptr2;
 	int i;
 
-	/* Map a region. */
-	ptr = mmap_(self, variant, NULL, 10 * page_size, PROT_READ | PROT_WRITE, 0, 0);
+	/* Try to Map a region. */
+	ptr = mmap_(self, variant, NULL, 12 * page_size, PROT_READ | PROT_WRITE, 0, 0);
+	ASSERT_NE(ptr, MAP_FAILED);
+	ASSERT_EQ(munmap(ptr, 12 * page_size), 0);
+
+	/*
+	 * Map a region for the test. Since the preceding temporary mapping
+	 * succeeded, this mapping should also succeed without merging with
+	 * adjacent VMAs.
+	 */
+	ptr = mmap_(self, variant, ptr + page_size, 10 * page_size,
+		    PROT_READ | PROT_WRITE, MAP_FIXED, 0);
 	ASSERT_NE(ptr, MAP_FAILED);
 
 	/* We shouldn't yet see a guard flag. */
-- 
LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7}
Re: [PATCH] kselftest: mm: fix potential failure for merged VMA in guard-regions
Posted by Lorenzo Stoakes (ARM) 2 weeks ago 
On Fri, Sep 11, 2026 at 01:35:34PM +0100, Yeoreum Yun wrote:
> check_vmflag_guard() uses /proc/self/smaps to retrieve the VMA flags,
> but this can fail if the mapping is merged with an adjacent VMA.
>
> To avoid this potential failure, first allocate a temporary region with
> extra pages at both ends, unmap it, and then map the test region within
> the temporary address range, leaving an unmapped page on each side to
> prevent VMA merging.
>
> Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
> ---
>  tools/testing/selftests/mm/guard-regions.c | 14 ++++++++++++--
>  1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/guard-regions.c b/tools/testing/selftests/mm/guard-regions.c
> index 5c8ec3ca75d7..a28a57d34e97 100644
> --- a/tools/testing/selftests/mm/guard-regions.c
> +++ b/tools/testing/selftests/mm/guard-regions.c
> @@ -2257,8 +2257,18 @@ TEST_F(guard_regions, smaps)
>  	char *ptr, *ptr2;
>  	int i;
>
> -	/* Map a region. */
> -	ptr = mmap_(self, variant, NULL, 10 * page_size, PROT_READ | PROT_WRITE, 0, 0);
> +	/* Try to Map a region. */

Map -> map

> +	ptr = mmap_(self, variant, NULL, 12 * page_size, PROT_READ | PROT_WRITE, 0, 0);

Should be PROT_NONE otherwise it'll merge with the below.

> +	ASSERT_NE(ptr, MAP_FAILED);
> +	ASSERT_EQ(munmap(ptr, 12 * page_size), 0);
> +
> +	/*
> +	 * Map a region for the test. Since the preceding temporary mapping
> +	 * succeeded, this mapping should also succeed without merging with
> +	 * adjacent VMAs.
> +	 */
> +	ptr = mmap_(self, variant, ptr + page_size, 10 * page_size,
> +		    PROT_READ | PROT_WRITE, MAP_FIXED, 0);
>  	ASSERT_NE(ptr, MAP_FAILED);
>
>  	/* We shouldn't yet see a guard flag. */
> --
> LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7}
>

--
Cheers, Lorenzo
Re: [PATCH] kselftest: mm: fix potential failure for merged VMA in guard-regions
Posted by Yeoreum Yun 1 week, 6 days ago 
On Fri, Sep 11, 2026 at 01:59:31PM +0100, Lorenzo Stoakes (ARM) wrote:
> On Fri, Sep 11, 2026 at 01:35:34PM +0100, Yeoreum Yun wrote:
> > check_vmflag_guard() uses /proc/self/smaps to retrieve the VMA flags,
> > but this can fail if the mapping is merged with an adjacent VMA.
> >
> > To avoid this potential failure, first allocate a temporary region with
> > extra pages at both ends, unmap it, and then map the test region within
> > the temporary address range, leaving an unmapped page on each side to
> > prevent VMA merging.
> >
> > Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
> > ---
> >  tools/testing/selftests/mm/guard-regions.c | 14 ++++++++++++--
> >  1 file changed, 12 insertions(+), 2 deletions(-)
> >
> > diff --git a/tools/testing/selftests/mm/guard-regions.c b/tools/testing/selftests/mm/guard-regions.c
> > index 5c8ec3ca75d7..a28a57d34e97 100644
> > --- a/tools/testing/selftests/mm/guard-regions.c
> > +++ b/tools/testing/selftests/mm/guard-regions.c
> > @@ -2257,8 +2257,18 @@ TEST_F(guard_regions, smaps)
> >  	char *ptr, *ptr2;
> >  	int i;
> >
> > -	/* Map a region. */
> > -	ptr = mmap_(self, variant, NULL, 10 * page_size, PROT_READ | PROT_WRITE, 0, 0);
> > +	/* Try to Map a region. */
> 
> Map -> map
> 
> > +	ptr = mmap_(self, variant, NULL, 12 * page_size, PROT_READ | PROT_WRITE, 0, 0);
> 
> Should be PROT_NONE otherwise it'll merge with the below.

It doesn't matter. since this memory is unmapped and then second map
at ptr + page_size.

IOW, though the first one is merged, it unammped and then 
the second is allocated at ptr + page_size, it wouldn't be merged:

  after unmap:
   [existing VMA][ 12 pages ][existing VMA]

  second:
   [exiting VMA] [hole (page)] [ 10 pages (for test)] [hole (page)] [existing VMA]

Am I missing something?

-- 
Sincerely,
Yeoreum Yun
Re: [PATCH] kselftest: mm: fix potential failure for merged VMA in guard-regions
Posted by Lorenzo Stoakes (ARM) 1 week, 6 days ago 
On Fri, Sep 11, 2026 at 02:34:37PM +0100, Yeoreum Yun wrote:
> On Fri, Sep 11, 2026 at 01:59:31PM +0100, Lorenzo Stoakes (ARM) wrote:
> > On Fri, Sep 11, 2026 at 01:35:34PM +0100, Yeoreum Yun wrote:
> > > check_vmflag_guard() uses /proc/self/smaps to retrieve the VMA flags,
> > > but this can fail if the mapping is merged with an adjacent VMA.
> > >
> > > To avoid this potential failure, first allocate a temporary region with
> > > extra pages at both ends, unmap it, and then map the test region within
> > > the temporary address range, leaving an unmapped page on each side to
> > > prevent VMA merging.
> > >
> > > Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
> > > ---
> > >  tools/testing/selftests/mm/guard-regions.c | 14 ++++++++++++--
> > >  1 file changed, 12 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/tools/testing/selftests/mm/guard-regions.c b/tools/testing/selftests/mm/guard-regions.c
> > > index 5c8ec3ca75d7..a28a57d34e97 100644
> > > --- a/tools/testing/selftests/mm/guard-regions.c
> > > +++ b/tools/testing/selftests/mm/guard-regions.c
> > > @@ -2257,8 +2257,18 @@ TEST_F(guard_regions, smaps)
> > >  	char *ptr, *ptr2;
> > >  	int i;
> > >
> > > -	/* Map a region. */
> > > -	ptr = mmap_(self, variant, NULL, 10 * page_size, PROT_READ | PROT_WRITE, 0, 0);
> > > +	/* Try to Map a region. */
> >
> > Map -> map
> >
> > > +	ptr = mmap_(self, variant, NULL, 12 * page_size, PROT_READ | PROT_WRITE, 0, 0);
> >
> > Should be PROT_NONE otherwise it'll merge with the below.
>
> It doesn't matter. since this memory is unmapped and then second map
> at ptr + page_size.
>
> IOW, though the first one is merged, it unammped and then
> the second is allocated at ptr + page_size, it wouldn't be merged:
>
>   after unmap:
>    [existing VMA][ 12 pages ][existing VMA]
>
>   second:
>    [exiting VMA] [hole (page)] [ 10 pages (for test)] [hole (page)] [existing VMA]
>
> Am I missing something?

Yeah, unmapped (unfaulted) VMAs can be merged with mapped (faulted) VMAs.

In general it's also better to be explicit by specifying distinct attributes
anyway to spell out clearly that the VMAs are intended to perform that task.

>
> --
> Sincerely,
> Yeoreum Yun

--
Cheers, Lorenzo
Re: [PATCH] kselftest: mm: fix potential failure for merged VMA in guard-regions
Posted by Lorenzo Stoakes (ARM) 1 week, 6 days ago 
On Fri, Sep 11, 2026 at 02:51:09PM +0100, Lorenzo Stoakes (ARM) wrote:
> On Fri, Sep 11, 2026 at 02:34:37PM +0100, Yeoreum Yun wrote:
> > On Fri, Sep 11, 2026 at 01:59:31PM +0100, Lorenzo Stoakes (ARM) wrote:
> > > On Fri, Sep 11, 2026 at 01:35:34PM +0100, Yeoreum Yun wrote:
> > > > check_vmflag_guard() uses /proc/self/smaps to retrieve the VMA flags,
> > > > but this can fail if the mapping is merged with an adjacent VMA.
> > > >
> > > > To avoid this potential failure, first allocate a temporary region with
> > > > extra pages at both ends, unmap it, and then map the test region within
> > > > the temporary address range, leaving an unmapped page on each side to
> > > > prevent VMA merging.
> > > >
> > > > Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
> > > > ---
> > > >  tools/testing/selftests/mm/guard-regions.c | 14 ++++++++++++--
> > > >  1 file changed, 12 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/tools/testing/selftests/mm/guard-regions.c b/tools/testing/selftests/mm/guard-regions.c
> > > > index 5c8ec3ca75d7..a28a57d34e97 100644
> > > > --- a/tools/testing/selftests/mm/guard-regions.c
> > > > +++ b/tools/testing/selftests/mm/guard-regions.c
> > > > @@ -2257,8 +2257,18 @@ TEST_F(guard_regions, smaps)
> > > >  	char *ptr, *ptr2;
> > > >  	int i;
> > > >
> > > > -	/* Map a region. */
> > > > -	ptr = mmap_(self, variant, NULL, 10 * page_size, PROT_READ | PROT_WRITE, 0, 0);
> > > > +	/* Try to Map a region. */
> > >
> > > Map -> map
> > >
> > > > +	ptr = mmap_(self, variant, NULL, 12 * page_size, PROT_READ | PROT_WRITE, 0, 0);
> > >
> > > Should be PROT_NONE otherwise it'll merge with the below.
> >
> > It doesn't matter. since this memory is unmapped and then second map
> > at ptr + page_size.
> >
> > IOW, though the first one is merged, it unammped and then
> > the second is allocated at ptr + page_size, it wouldn't be merged:
> >
> >   after unmap:
> >    [existing VMA][ 12 pages ][existing VMA]
> >
> >   second:
> >    [exiting VMA] [hole (page)] [ 10 pages (for test)] [hole (page)] [existing VMA]
> >
> > Am I missing something?
>
> Yeah, unmapped (unfaulted) VMAs can be merged with mapped (faulted) VMAs.
>
> In general it's also better to be explicit by specifying distinct attributes
> anyway to spell out clearly that the VMAs are intended to perform that task.

Oops, I missed that you immediately unmapped the VMA too :)

In that case it's fine but I'd still prefer it PROT_NONE to clearly single it
out as a placeholder.

Also a comment above it like:

	/* Map then unmap placeholder to avoid adjacent merges */

>
> >
> > --
> > Sincerely,
> > Yeoreum Yun
>
> --
> Cheers, Lorenzo

--
Cheers, Lorenzo
Re: [PATCH] kselftest: mm: fix potential failure for merged VMA in guard-regions
Posted by Yeoreum Yun 1 week, 6 days ago 
> On Fri, Sep 11, 2026 at 02:51:09PM +0100, Lorenzo Stoakes (ARM) wrote:
> > On Fri, Sep 11, 2026 at 02:34:37PM +0100, Yeoreum Yun wrote:
> > > On Fri, Sep 11, 2026 at 01:59:31PM +0100, Lorenzo Stoakes (ARM) wrote:
> > > > On Fri, Sep 11, 2026 at 01:35:34PM +0100, Yeoreum Yun wrote:
> > > > > check_vmflag_guard() uses /proc/self/smaps to retrieve the VMA flags,
> > > > > but this can fail if the mapping is merged with an adjacent VMA.
> > > > >
> > > > > To avoid this potential failure, first allocate a temporary region with
> > > > > extra pages at both ends, unmap it, and then map the test region within
> > > > > the temporary address range, leaving an unmapped page on each side to
> > > > > prevent VMA merging.
> > > > >
> > > > > Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
> > > > > ---
> > > > >  tools/testing/selftests/mm/guard-regions.c | 14 ++++++++++++--
> > > > >  1 file changed, 12 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/tools/testing/selftests/mm/guard-regions.c b/tools/testing/selftests/mm/guard-regions.c
> > > > > index 5c8ec3ca75d7..a28a57d34e97 100644
> > > > > --- a/tools/testing/selftests/mm/guard-regions.c
> > > > > +++ b/tools/testing/selftests/mm/guard-regions.c
> > > > > @@ -2257,8 +2257,18 @@ TEST_F(guard_regions, smaps)
> > > > >  	char *ptr, *ptr2;
> > > > >  	int i;
> > > > >
> > > > > -	/* Map a region. */
> > > > > -	ptr = mmap_(self, variant, NULL, 10 * page_size, PROT_READ | PROT_WRITE, 0, 0);
> > > > > +	/* Try to Map a region. */
> > > >
> > > > Map -> map
> > > >
> > > > > +	ptr = mmap_(self, variant, NULL, 12 * page_size, PROT_READ | PROT_WRITE, 0, 0);
> > > >
> > > > Should be PROT_NONE otherwise it'll merge with the below.
> > >
> > > It doesn't matter. since this memory is unmapped and then second map
> > > at ptr + page_size.
> > >
> > > IOW, though the first one is merged, it unammped and then
> > > the second is allocated at ptr + page_size, it wouldn't be merged:
> > >
> > >   after unmap:
> > >    [existing VMA][ 12 pages ][existing VMA]
> > >
> > >   second:
> > >    [exiting VMA] [hole (page)] [ 10 pages (for test)] [hole (page)] [existing VMA]
> > >
> > > Am I missing something?
> >
> > Yeah, unmapped (unfaulted) VMAs can be merged with mapped (faulted) VMAs.
> >
> > In general it's also better to be explicit by specifying distinct attributes
> > anyway to spell out clearly that the VMAs are intended to perform that task.
> 
> Oops, I missed that you immediately unmapped the VMA too :)
> 
> In that case it's fine but I'd still prefer it PROT_NONE to clearly single it
> out as a placeholder.
> 
> Also a comment above it like:
> 
> 	/* Map then unmap placeholder to avoid adjacent merges */

Might the ASSERT() made you miss the unmap :).

I'll post soon again.

Thanks!

-- 
Sincerely,
Yeoreum Yun

Patchew 2.1-devel-ea86937

© 2016 - 2026 Red Hat, Inc.