selftests/mm: Skip tests dependent on a binary not built

[RFC] selftests/mm: Skip tests dependent on a binary not built

Posted by Khaled Elnaggar 6 months, 3 weeks ago

Hello.

Running the mm selftests from the kernel's root directory
on an x86_64 debian machine using:

  make defconfig
  sudo make kselftest TARGETS=mm

the tests run normally till we reach one which stalls
for 180 seconds and times out according to the following logs:

```
 -----------------------------------------------
 running ./charge_reserved_hugetlb.sh -cgroup-v2
 -----------------------------------------------
 CLEANUP DONE
 CLEANUP DONE
  
 Test normal case.
 private=, populate=, method=0, reserve=
 nr hugepages = 10
 writing cgroup limit: 20971520
 writing reseravation limit: 20971520

 Starting:
 hugetlb_usage=0
 reserved_usage=0
 expect_failure is 0
 Putting task in cgroup 'hugetlb_cgroup_test'
 Method is 0
 >>> write_hugetlb_memory.sh: line 22: ./write_to_hugetlbfs: No such file or directory <<<
 Waiting for hugetlb memory reservation to reach size 10485760.
 0
 Waiting for hugetlb memory reservation to reach size 10485760.
 0
 ...
 Waiting for hugetlb memory reservation to reach size 10485760.
 0
 Waiting for hugetlb memory reservation to reach size 10485760.
 0

 not ok 1 selftests: mm: run_vmtests.sh # TIMEOUT 180 seconds
 make[3]: Leaving directory '/linux/tools/testing/selftests/mm'
```

Logs show that the executable "write_to_hugetlbfs" is missing, causing
the test to hang waiting for hugepage reservations.

The executable not found means it was not built by the Make system.
It is mentioned in Makefile:136-142, and only built if ARCH is 64-bit

```
   ifneq (,$(filter $(ARCH),arm64 mips64 parisc64 powerpc riscv64 s390x sparc64 x86_64 s390))
   TEST_GEN_FILES += va_high_addr_switch
   ifneq ($(ARCH),riscv64)
   TEST_GEN_FILES += virtual_address_range
   endif
   TEST_GEN_FILES += write_to_hugetlbfs
   endif
```

So, for some reason, the top-level Makefile provides ARCH as x86.

My proposed solution is similar to existing virtual_address_range check
that is to check for the binary, and if it is not found, skip these 2
test cases: charge_reserved_hugetlb.sh and hugetlb_reparenting_test.sh
since they directly and indirectly depend on write_to_hugetlbfs binary.

This is just a workaround, the root issue of different ARCH detection
when running tests from the kernel root directory should still be
addressed. I am not sure how to approach it and open for your suggestions.

Note that this issue does not happen when ran from selftests/mm using
something like

  sudo make -C tools/testing/selftests/mm

because then mm/Makefile's ARCH detection runs correctly (x86_64)

Kindly review and share your thoughts.

Signed-off-by: Khaled Elnaggar <khaledelnaggarlinux@gmail.com>
---
 tools/testing/selftests/mm/run_vmtests.sh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index dddd1dd8af14..cdbcfdb62f8a 100755
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -375,8 +375,13 @@ CATEGORY="process_mrelease" run_test ./mrelease_test
 CATEGORY="mremap" run_test ./mremap_test

 CATEGORY="hugetlb" run_test ./thuge-gen
+
+# the following depend on write_to_hugetlbfs binary
+if [ -x ./write_to_hugetlbfs ]; then
 CATEGORY="hugetlb" run_test ./charge_reserved_hugetlb.sh -cgroup-v2
 CATEGORY="hugetlb" run_test ./hugetlb_reparenting_test.sh -cgroup-v2
+fi
+
 if $RUN_DESTRUCTIVE; then
 nr_hugepages_tmp=$(cat /proc/sys/vm/nr_hugepages)
 enable_soft_offline=$(cat /proc/sys/vm/enable_soft_offline)
--
2.47.2

Re: [RFC] selftests/mm: Skip tests dependent on a binary not built

Posted by Shuah Khan 6 months, 3 weeks ago

On 5/23/25 12:43, Khaled Elnaggar wrote:
> Hello.
> 

The above doesn't belong in change log. Refer to a few change logs
in the repo to learn  how to write them.

You can also check the kernel documentation - this change log
is way too long. It doesn't clearly state what is being fixed.

In this case, "write_to_hugetlbfs" is missing because the right
config isn't enabled. Test stalls waiting for write_to_hugetlbfs
to run.

You are fixing this by checking if write_to_hugetlbfs exixts
from /mm/run_vmtests.sh

Summarize this in short description so people can understand
the change.

thanks,
-- Shuah

Re: [RFC] selftests/mm: Skip tests dependent on a binary not built

Posted by Khaled Elnaggar 6 months, 3 weeks ago

On 24/05/2025 12:09 am, Shuah Khan wrote:
> On 5/23/25 12:43, Khaled Elnaggar wrote:
>> Hello.
>>
> 
> The above doesn't belong in change log. Refer to a few change logs
> in the repo to learn  how to write them.
> 
> You can also check the kernel documentation - this change log
> is way too long. It doesn't clearly state what is being fixed.
> 
> In this case, "write_to_hugetlbfs" is missing because the right
> config isn't enabled. Test stalls waiting for write_to_hugetlbfs
> to run.
> 
> You are fixing this by checking if write_to_hugetlbfs exixts
> from /mm/run_vmtests.sh
> 
> Summarize this in short description so people can understand
> the change.
> 
> thanks,
> -- Shuah

Oh, I mistakenly thought since RFC does not get merged into source tree, 
it could have a relaxed tone, hence the much text.

Appreciate your feedback, I will rewrite the patch considering the
points mentioned. Thank you.

[PATCH RESEND] selftests/mm/run_vmtests.sh: skip hugevm tests if write_to_hugetlbfs is missing

Posted by Khaled Elnaggar 6 months, 2 weeks ago

The hugevm tests 'charge_reserved_hugetlb.sh' and 'hugetlb_reparenting_test.sh'
depend on the 'write_to_hugetlbfs' binary to simulate writes to hugetlbfs
while checking reservations asynchronously in the background.

When this binary is missing (e.g., excluded from the build), these tests hang
for up to 180 seconds. During this time, run_vmtests.sh is eventually killed
due to timeout, aborting all subsequent tests.

This patch skips these tests if the binary is not found, preventing delays
and ensuring that the test suite runs to completion.

Signed-off-by: Khaled Elnaggar <khaledelnaggarlinux@gmail.com>
---
 tools/testing/selftests/mm/run_vmtests.sh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index dddd1dd8af14..0b55395ee2cb 100755
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -375,8 +375,13 @@ CATEGORY="process_mrelease" run_test ./mrelease_test
 CATEGORY="mremap" run_test ./mremap_test

 CATEGORY="hugetlb" run_test ./thuge-gen
+
+# the following tests depend on write_to_hugetlbfs binary
+if [ -x ./write_to_hugetlbfs ]; then
 CATEGORY="hugetlb" run_test ./charge_reserved_hugetlb.sh -cgroup-v2
 CATEGORY="hugetlb" run_test ./hugetlb_reparenting_test.sh -cgroup-v2
+fi
+
 if $RUN_DESTRUCTIVE; then
 nr_hugepages_tmp=$(cat /proc/sys/vm/nr_hugepages)
 enable_soft_offline=$(cat /proc/sys/vm/enable_soft_offline)
--
2.47.2

Re: [PATCH RESEND] selftests/mm/run_vmtests.sh: skip hugevm tests if write_to_hugetlbfs is missing

Posted by Andrew Morton 6 months, 2 weeks ago

On Tue,  3 Jun 2025 02:22:32 +0300 Khaled Elnaggar <khaledelnaggarlinux@gmail.com> wrote:

> The hugevm tests 'charge_reserved_hugetlb.sh' and 'hugetlb_reparenting_test.sh'
> depend on the 'write_to_hugetlbfs' binary to simulate writes to hugetlbfs
> while checking reservations asynchronously in the background.
> 
> When this binary is missing (e.g., excluded from the build), these tests hang
> for up to 180 seconds. During this time, run_vmtests.sh is eventually killed
> due to timeout, aborting all subsequent tests.
> 
> This patch skips these tests if the binary is not found, preventing delays
> and ensuring that the test suite runs to completion.

OK, but why is write_to_hugetlbfs missing?  If we're in a situation
where we _could_ run it then we _should_ run it!  The user wants to
test stuff so we should test as much as we can.

So I'm thinking that it would be preferable to make sure the dang thing
is there?

Re: [PATCH RESEND] selftests/mm/run_vmtests.sh: skip hugevm tests if write_to_hugetlbfs is missing

Posted by Khaled Elnaggar 6 months, 1 week ago

On 03/06/2025 5:12 am, Andrew Morton wrote:
> On Tue,  3 Jun 2025 02:22:32 +0300 Khaled Elnaggar <khaledelnaggarlinux@gmail.com> wrote:
> 
>> The hugevm tests 'charge_reserved_hugetlb.sh' and 'hugetlb_reparenting_test.sh'
>> depend on the 'write_to_hugetlbfs' binary to simulate writes to hugetlbfs
>> while checking reservations asynchronously in the background.
>>
>> When this binary is missing (e.g., excluded from the build), these tests hang
>> for up to 180 seconds. During this time, run_vmtests.sh is eventually killed
>> due to timeout, aborting all subsequent tests.
>>
>> This patch skips these tests if the binary is not found, preventing delays
>> and ensuring that the test suite runs to completion.
> 
> OK, but why is write_to_hugetlbfs missing?  If we're in a situation
> where we _could_ run it then we _should_ run it!  The user wants to
> test stuff so we should test as much as we can.
> So I'm thinking that it would be preferable to make sure the dang thing
> is there?
> 

Totally agree, let me try to explain how I understand the issue.

The write_to_hugetlbfs binary is built from selftests/mm/Makefile,
lines 136–142. It is only compiled if ARCH matches one of the explicitly
listed 64-bit architectures:

```
   ifneq (,$(filter $(ARCH),arm64 mips64 parisc64 powerpc riscv64 s390x sparc64 x86_64 s390))
   TEST_GEN_FILES += va_high_addr_switch
   ifneq ($(ARCH),riscv64)
   TEST_GEN_FILES += virtual_address_range
   endif
   TEST_GEN_FILES += write_to_hugetlbfs
   endif
```

However, when the MM selftests are run from the kernel’s top-level Makefile,
(root directory for example:

  make defconfig
  sudo make kselftest TARGETS=mm

ARCH is resolved as x86, even on an x86_64 machine (Debian in my case),
ARCH=x86 is the reason why the binary gets excluded from the build system.

As far as I understand, the top-level Makefile normalizes both
i.86 and x86_64 to x86 for ARCH variable:

Makfile: lines: 383,403
        383:include $(srctree)/scripts/subarch.include
        403:ARCH                ?= $(SUBARCH)

scripts/subarch.include: line: 7
        7:SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \

This mapping probably makes sense for selecting the correct arch/ directory
(since we have arch/x86, not arch/x86_64) for top-level Makefile.

But the mm selftests Makefile expects ARCH to differentiate between x86 and x86_64
to decide whether to build certain binaries.
As a result, the 64-bit-only binaries like write_to_hugetlbfs are skipped
during build, yet still expected at runtime (by run_vmtests.sh).

That's why this whole issue of the missing executable does not happen when
ran from selftests/mm using something like:

  sudo make -C tools/testing/selftests/mm

Because then selftests/mm/Makefile arch detection rus as intended.

You're right — the proper fix is to improve how we propagate architecture
information from the top-level Makefile to selftests.
But since that's a larger change (and possibly beyond what I can safely
attempt at this point), this patch is just a targeted workaround to
avoid test stalls when the binary is missing.

I hope I haven't gone completely wrong with this analysis, happy to improve
or revise it further if needed.