selftests/bpf: Page out as late a possible in file_reader

[PATCH] selftests/bpf: Page out as late a possible in file_reader

Posted by Jerome Marchand 1 month, 3 weeks ago

The file_reader/on_open_expect_fault fails consistently on my system.
It expects a page fault on first dynptr read of some range the exe
file of the current process because it has paged out that page range
earlier. However a lot can happen to that range (which depending on
the actual memory layout could contain text section, data section,
sections )related to dynamic linking...) between the moment it was
paged out and the moment the bpf program expected to hit a pagefault
actually run.

A bit of instrumentation with mincore() shows that pages from that
range were accessed several times before the program is run. In
particular the call of file_reader__load() seems to fault all the
range in.

Move the call to madvise(MADV_PAGEOUT) to just before attaching the
program to minimize the risk of having those page pulled back in from
under our feet.

Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
---
 .../selftests/bpf/prog_tests/file_reader.c    | 22 +++++++++----------
 1 file changed, 10 insertions(+), 12 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/file_reader.c b/tools/testing/selftests/bpf/prog_tests/file_reader.c
index 5cde32b35da44..48aae7ea0e4bb 100644
--- a/tools/testing/selftests/bpf/prog_tests/file_reader.c
+++ b/tools/testing/selftests/bpf/prog_tests/file_reader.c
@@ -10,6 +10,7 @@
 
 const char *user_ptr = "hello world";
 char file_contents[256000];
+void *addr;
 
 void *get_executable_base_addr(void)
 {
@@ -26,8 +27,7 @@ void *get_executable_base_addr(void)
 static int initialize_file_contents(void)
 {
 	int fd, page_sz = sysconf(_SC_PAGESIZE);
-	ssize_t n = 0, cur, off;
-	void *addr;
+	ssize_t n = 0, cur;
 
 	fd = open("/proc/self/exe", O_RDONLY);
 	if (!ASSERT_OK_FD(fd, "Open /proc/self/exe\n"))
@@ -52,16 +52,6 @@ static int initialize_file_contents(void)
 	/* page-align base file address */
 	addr = (void *)((unsigned long)addr & ~(page_sz - 1));
 
-	/*
-	 * Page out range 0..512K, use 0..256K for positive tests and
-	 * 256K..512K for negative tests expecting page faults
-	 */
-	for (off = 0; off < sizeof(file_contents) * 2; off += page_sz) {
-		if (!ASSERT_OK(madvise(addr + off, page_sz, MADV_PAGEOUT),
-			       "madvise pageout"))
-			return errno;
-	}
-
 	return 0;
 }
 
@@ -90,6 +80,14 @@ static void run_test(const char *prog_name)
 	if (!ASSERT_OK(err, "file_reader__load"))
 		goto cleanup;
 
+	/*
+	 * Page out range 0..512K, use 0..256K for positive tests and
+	 * 256K..512K for negative tests expecting page faults
+	 */
+	if (!ASSERT_OK(madvise(addr, sizeof(file_contents) * 2, MADV_PAGEOUT),
+		       "madvise pageout"))
+		goto cleanup;
+
 	err = file_reader__attach(skel);
 	if (!ASSERT_OK(err, "file_reader__attach"))
 		goto cleanup;
-- 
2.53.0

Re: [PATCH] selftests/bpf: Page out as late a possible in file_reader

Posted by Jiri Olsa 1 month, 3 weeks ago

On Mon, Apr 20, 2026 at 03:46:37PM +0200, Jerome Marchand wrote:
> The file_reader/on_open_expect_fault fails consistently on my system.
> It expects a page fault on first dynptr read of some range the exe
> file of the current process because it has paged out that page range
> earlier. However a lot can happen to that range (which depending on
> the actual memory layout could contain text section, data section,
> sections )related to dynamic linking...) between the moment it was
> paged out and the moment the bpf program expected to hit a pagefault
> actually run.
> 
> A bit of instrumentation with mincore() shows that pages from that
> range were accessed several times before the program is run. In
> particular the call of file_reader__load() seems to fault all the
> range in.
> 
> Move the call to madvise(MADV_PAGEOUT) to just before attaching the
> program to minimize the risk of having those page pulled back in from
> under our feet.
> 
> Signed-off-by: Jerome Marchand <jmarchan@redhat.com>

nit typo in the subject a -> as

Acked-by: Jiri Olsa <jolsa@kernel.org>

jirka


> ---
>  .../selftests/bpf/prog_tests/file_reader.c    | 22 +++++++++----------
>  1 file changed, 10 insertions(+), 12 deletions(-)
> 
> diff --git a/tools/testing/selftests/bpf/prog_tests/file_reader.c b/tools/testing/selftests/bpf/prog_tests/file_reader.c
> index 5cde32b35da44..48aae7ea0e4bb 100644
> --- a/tools/testing/selftests/bpf/prog_tests/file_reader.c
> +++ b/tools/testing/selftests/bpf/prog_tests/file_reader.c
> @@ -10,6 +10,7 @@
>  
>  const char *user_ptr = "hello world";
>  char file_contents[256000];
> +void *addr;
>  
>  void *get_executable_base_addr(void)
>  {
> @@ -26,8 +27,7 @@ void *get_executable_base_addr(void)
>  static int initialize_file_contents(void)
>  {
>  	int fd, page_sz = sysconf(_SC_PAGESIZE);
> -	ssize_t n = 0, cur, off;
> -	void *addr;
> +	ssize_t n = 0, cur;
>  
>  	fd = open("/proc/self/exe", O_RDONLY);
>  	if (!ASSERT_OK_FD(fd, "Open /proc/self/exe\n"))
> @@ -52,16 +52,6 @@ static int initialize_file_contents(void)
>  	/* page-align base file address */
>  	addr = (void *)((unsigned long)addr & ~(page_sz - 1));
>  
> -	/*
> -	 * Page out range 0..512K, use 0..256K for positive tests and
> -	 * 256K..512K for negative tests expecting page faults
> -	 */
> -	for (off = 0; off < sizeof(file_contents) * 2; off += page_sz) {
> -		if (!ASSERT_OK(madvise(addr + off, page_sz, MADV_PAGEOUT),
> -			       "madvise pageout"))
> -			return errno;
> -	}
> -
>  	return 0;
>  }
>  
> @@ -90,6 +80,14 @@ static void run_test(const char *prog_name)
>  	if (!ASSERT_OK(err, "file_reader__load"))
>  		goto cleanup;
>  
> +	/*
> +	 * Page out range 0..512K, use 0..256K for positive tests and
> +	 * 256K..512K for negative tests expecting page faults
> +	 */
> +	if (!ASSERT_OK(madvise(addr, sizeof(file_contents) * 2, MADV_PAGEOUT),
> +		       "madvise pageout"))
> +		goto cleanup;
> +
>  	err = file_reader__attach(skel);
>  	if (!ASSERT_OK(err, "file_reader__attach"))
>  		goto cleanup;
> -- 
> 2.53.0
>

Re: [PATCH] selftests/bpf: Page out as late a possible in file_reader

Posted by Mykyta Yatsenko 1 month, 3 weeks ago

On 4/20/26 2:46 PM, Jerome Marchand wrote:
> The file_reader/on_open_expect_fault fails consistently on my system.
> It expects a page fault on first dynptr read of some range the exe
> file of the current process because it has paged out that page range
> earlier. However a lot can happen to that range (which depending on
> the actual memory layout could contain text section, data section,
> sections )related to dynamic linking...) between the moment it was
> paged out and the moment the bpf program expected to hit a pagefault
> actually run.
> 
> A bit of instrumentation with mincore() shows that pages from that
> range were accessed several times before the program is run. In
> particular the call of file_reader__load() seems to fault all the
> range in.
> 
> Move the call to madvise(MADV_PAGEOUT) to just before attaching the
> program to minimize the risk of having those page pulled back in from
> under our feet.
> 
> Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
> ---

Thank you for the patch, the change looks good. Does it fail 
consistently on 4K page size?

Acked-by: Mykyta Yatsenko <yatsenko@meta.com>

>   .../selftests/bpf/prog_tests/file_reader.c    | 22 +++++++++----------
>   1 file changed, 10 insertions(+), 12 deletions(-)
> 
> diff --git a/tools/testing/selftests/bpf/prog_tests/file_reader.c b/tools/testing/selftests/bpf/prog_tests/file_reader.c
> index 5cde32b35da44..48aae7ea0e4bb 100644
> --- a/tools/testing/selftests/bpf/prog_tests/file_reader.c
> +++ b/tools/testing/selftests/bpf/prog_tests/file_reader.c
> @@ -10,6 +10,7 @@
>   
>   const char *user_ptr = "hello world";
>   char file_contents[256000];
> +void *addr;
>   
>   void *get_executable_base_addr(void)
>   {
> @@ -26,8 +27,7 @@ void *get_executable_base_addr(void)
>   static int initialize_file_contents(void)
>   {
>   	int fd, page_sz = sysconf(_SC_PAGESIZE);
> -	ssize_t n = 0, cur, off;
> -	void *addr;
> +	ssize_t n = 0, cur;
>   
>   	fd = open("/proc/self/exe", O_RDONLY);
>   	if (!ASSERT_OK_FD(fd, "Open /proc/self/exe\n"))
> @@ -52,16 +52,6 @@ static int initialize_file_contents(void)
>   	/* page-align base file address */
>   	addr = (void *)((unsigned long)addr & ~(page_sz - 1));
>   
> -	/*
> -	 * Page out range 0..512K, use 0..256K for positive tests and
> -	 * 256K..512K for negative tests expecting page faults
> -	 */
> -	for (off = 0; off < sizeof(file_contents) * 2; off += page_sz) {
> -		if (!ASSERT_OK(madvise(addr + off, page_sz, MADV_PAGEOUT),
> -			       "madvise pageout"))
> -			return errno;
> -	}
> -
>   	return 0;
>   }
>   
> @@ -90,6 +80,14 @@ static void run_test(const char *prog_name)
>   	if (!ASSERT_OK(err, "file_reader__load"))
>   		goto cleanup;
>   
> +	/*
> +	 * Page out range 0..512K, use 0..256K for positive tests and
> +	 * 256K..512K for negative tests expecting page faults
> +	 */
> +	if (!ASSERT_OK(madvise(addr, sizeof(file_contents) * 2, MADV_PAGEOUT),
> +		       "madvise pageout"))
> +		goto cleanup;
> +
>   	err = file_reader__attach(skel);
>   	if (!ASSERT_OK(err, "file_reader__attach"))
>   		goto cleanup;

Re: [PATCH] selftests/bpf: Page out as late a possible in file_reader

Posted by Jerome Marchand 1 month, 3 weeks ago

On 4/20/26 6:52 PM, Mykyta Yatsenko wrote:
> On 4/20/26 2:46 PM, Jerome Marchand wrote:
>> The file_reader/on_open_expect_fault fails consistently on my system.
>> It expects a page fault on first dynptr read of some range the exe
>> file of the current process because it has paged out that page range
>> earlier. However a lot can happen to that range (which depending on
>> the actual memory layout could contain text section, data section,
>> sections )related to dynamic linking...) between the moment it was
>> paged out and the moment the bpf program expected to hit a pagefault
>> actually run.
>>
>> A bit of instrumentation with mincore() shows that pages from that
>> range were accessed several times before the program is run. In
>> particular the call of file_reader__load() seems to fault all the
>> range in.
>>
>> Move the call to madvise(MADV_PAGEOUT) to just before attaching the
>> program to minimize the risk of having those page pulled back in from
>> under our feet.
>>
>> Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
>> ---
> 
> Thank you for the patch, the change looks good. Does it fail 
> consistently on 4K page size?

It did when I ran the test manually. On our automated testing system, it 
failed intermittently.

Regards,
Jerome

> 
> Acked-by: Mykyta Yatsenko <yatsenko@meta.com>
> 
>>   .../selftests/bpf/prog_tests/file_reader.c    | 22 +++++++++----------
>>   1 file changed, 10 insertions(+), 12 deletions(-)
>>
>> diff --git a/tools/testing/selftests/bpf/prog_tests/file_reader.c b/ 
>> tools/testing/selftests/bpf/prog_tests/file_reader.c
>> index 5cde32b35da44..48aae7ea0e4bb 100644
>> --- a/tools/testing/selftests/bpf/prog_tests/file_reader.c
>> +++ b/tools/testing/selftests/bpf/prog_tests/file_reader.c
>> @@ -10,6 +10,7 @@
>>   const char *user_ptr = "hello world";
>>   char file_contents[256000];
>> +void *addr;
>>   void *get_executable_base_addr(void)
>>   {
>> @@ -26,8 +27,7 @@ void *get_executable_base_addr(void)
>>   static int initialize_file_contents(void)
>>   {
>>       int fd, page_sz = sysconf(_SC_PAGESIZE);
>> -    ssize_t n = 0, cur, off;
>> -    void *addr;
>> +    ssize_t n = 0, cur;
>>       fd = open("/proc/self/exe", O_RDONLY);
>>       if (!ASSERT_OK_FD(fd, "Open /proc/self/exe\n"))
>> @@ -52,16 +52,6 @@ static int initialize_file_contents(void)
>>       /* page-align base file address */
>>       addr = (void *)((unsigned long)addr & ~(page_sz - 1));
>> -    /*
>> -     * Page out range 0..512K, use 0..256K for positive tests and
>> -     * 256K..512K for negative tests expecting page faults
>> -     */
>> -    for (off = 0; off < sizeof(file_contents) * 2; off += page_sz) {
>> -        if (!ASSERT_OK(madvise(addr + off, page_sz, MADV_PAGEOUT),
>> -                   "madvise pageout"))
>> -            return errno;
>> -    }
>> -
>>       return 0;
>>   }
>> @@ -90,6 +80,14 @@ static void run_test(const char *prog_name)
>>       if (!ASSERT_OK(err, "file_reader__load"))
>>           goto cleanup;
>> +    /*
>> +     * Page out range 0..512K, use 0..256K for positive tests and
>> +     * 256K..512K for negative tests expecting page faults
>> +     */
>> +    if (!ASSERT_OK(madvise(addr, sizeof(file_contents) * 2, 
>> MADV_PAGEOUT),
>> +               "madvise pageout"))
>> +        goto cleanup;
>> +
>>       err = file_reader__attach(skel);
>>       if (!ASSERT_OK(err, "file_reader__attach"))
>>           goto cleanup;
>

Re: [PATCH] selftests/bpf: Page out as late a possible in file_reader

Posted by Shung-Hsi Yu 2 weeks ago

On Tue, Apr 21, 2026 at 07:59:08AM +0200, Jerome Marchand wrote:
> On 4/20/26 6:52 PM, Mykyta Yatsenko wrote:
> > On 4/20/26 2:46 PM, Jerome Marchand wrote:
> > > The file_reader/on_open_expect_fault fails consistently on my system.
> > > It expects a page fault on first dynptr read of some range the exe
> > > file of the current process because it has paged out that page range
> > > earlier. However a lot can happen to that range (which depending on
> > > the actual memory layout could contain text section, data section,
> > > sections )related to dynamic linking...) between the moment it was
> > > paged out and the moment the bpf program expected to hit a pagefault
> > > actually run.
> > > 
> > > A bit of instrumentation with mincore() shows that pages from that
> > > range were accessed several times before the program is run. In
> > > particular the call of file_reader__load() seems to fault all the
> > > range in.
> > > 
> > > Move the call to madvise(MADV_PAGEOUT) to just before attaching the
> > > program to minimize the risk of having those page pulled back in from
> > > under our feet.
> > > 
> > > Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
> > > ---
> > 
> > Thank you for the patch, the change looks good. Does it fail
> > consistently on 4K page size?
> 
> It did when I ran the test manually. On our automated testing system, it
> failed intermittently.

On my fork of BPF CI the test was failing[1] consistently, even when
this commit is applied. However I wasn't able to reproduce locally on my
laptop when I ran manually.

Any hints regarding where to look and how to reproduce?

Thanks,
Shung-Hsi

1: https://github.com/shunghsiyu/libbpf/actions/runs/26807198259/job/79027625188

...

Re: [PATCH] selftests/bpf: Page out as late a possible in file_reader

Posted by Jerome Marchand 2 weeks ago

On 02/06/2026 11:26, Shung-Hsi Yu wrote:
> On Tue, Apr 21, 2026 at 07:59:08AM +0200, Jerome Marchand wrote:
>> On 4/20/26 6:52 PM, Mykyta Yatsenko wrote:
>>> On 4/20/26 2:46 PM, Jerome Marchand wrote:
>>>> The file_reader/on_open_expect_fault fails consistently on my system.
>>>> It expects a page fault on first dynptr read of some range the exe
>>>> file of the current process because it has paged out that page range
>>>> earlier. However a lot can happen to that range (which depending on
>>>> the actual memory layout could contain text section, data section,
>>>> sections )related to dynamic linking...) between the moment it was
>>>> paged out and the moment the bpf program expected to hit a pagefault
>>>> actually run.
>>>>
>>>> A bit of instrumentation with mincore() shows that pages from that
>>>> range were accessed several times before the program is run. In
>>>> particular the call of file_reader__load() seems to fault all the
>>>> range in.
>>>>
>>>> Move the call to madvise(MADV_PAGEOUT) to just before attaching the
>>>> program to minimize the risk of having those page pulled back in from
>>>> under our feet.
>>>>
>>>> Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
>>>> ---
>>>
>>> Thank you for the patch, the change looks good. Does it fail
>>> consistently on 4K page size?
>>
>> It did when I ran the test manually. On our automated testing system, it
>> failed intermittently.
> 
> On my fork of BPF CI the test was failing[1] consistently, even when
> this commit is applied. However I wasn't able to reproduce locally on my
> laptop when I ran manually.
> 
> Any hints regarding where to look and how to reproduce?

To investigate the failure I witnessed I instrumented the test with some 
calls to mincore(). You can do that to see whether madvise(PAGEOUT) 
didn't successfully paged out those pages, or if they were paged back in 
by the MM subsystem. I'm afraid that this test isn't very robust: the MM 
subsystem is the one that ultimately decides which pages are in or out 
and it might not work in the way that this test assumes.

Thanks,
Jerome

> 
> Thanks,
> Shung-Hsi
> 
> 1: https://github.com/shunghsiyu/libbpf/actions/runs/26807198259/job/79027625188
> 
> ...
>

Re: [PATCH] selftests/bpf: Page out as late a possible in file_reader

Posted by Mykyta Yatsenko 2 weeks ago

On 6/2/26 10:51 AM, Jerome Marchand wrote:
> On 02/06/2026 11:26, Shung-Hsi Yu wrote:
>> On Tue, Apr 21, 2026 at 07:59:08AM +0200, Jerome Marchand wrote:
>>> On 4/20/26 6:52 PM, Mykyta Yatsenko wrote:
>>>> On 4/20/26 2:46 PM, Jerome Marchand wrote:
>>>>> The file_reader/on_open_expect_fault fails consistently on my system.
>>>>> It expects a page fault on first dynptr read of some range the exe
>>>>> file of the current process because it has paged out that page range
>>>>> earlier. However a lot can happen to that range (which depending on
>>>>> the actual memory layout could contain text section, data section,
>>>>> sections )related to dynamic linking...) between the moment it was
>>>>> paged out and the moment the bpf program expected to hit a pagefault
>>>>> actually run.
>>>>>
>>>>> A bit of instrumentation with mincore() shows that pages from that
>>>>> range were accessed several times before the program is run. In
>>>>> particular the call of file_reader__load() seems to fault all the
>>>>> range in.
>>>>>
>>>>> Move the call to madvise(MADV_PAGEOUT) to just before attaching the
>>>>> program to minimize the risk of having those page pulled back in from
>>>>> under our feet.
>>>>>
>>>>> Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
>>>>> ---
>>>>
>>>> Thank you for the patch, the change looks good. Does it fail
>>>> consistently on 4K page size?
>>>
>>> It did when I ran the test manually. On our automated testing system, it
>>> failed intermittently.
>>
>> On my fork of BPF CI the test was failing[1] consistently, even when
>> this commit is applied. However I wasn't able to reproduce locally on my
>> laptop when I ran manually.
>>
>> Any hints regarding where to look and how to reproduce?
> 
> To investigate the failure I witnessed I instrumented the test with some calls to mincore(). You can do that to see whether madvise(PAGEOUT) didn't successfully paged out those pages, or if they were paged back in by the MM subsystem. I'm afraid that this test isn't very robust: the MM subsystem is the one that ultimately decides which pages are in or out and it might not work in the way that this test assumes.
> 

thanks for the reports, this test is indeed flaky.
I'll take a look if it's possible to fix it, or we
should just delete it.

> Thanks,
> Jerome
> 
>>
>> Thanks,
>> Shung-Hsi
>>
>> 1: https://github.com/shunghsiyu/libbpf/actions/runs/26807198259/job/79027625188
>>
>> ...
>>
>