tools/testing/selftests/mm/khugepaged.c | 321 ++++++++++-------------- 1 file changed, 132 insertions(+), 189 deletions(-)
Hi all,
This patchset removes READ_ONLY_THP_FOR_FS Kconfig and enables creating
file-backed THPs for FSes with large folio support (the supported orders
need to include PMD_ORDER) by default, including for writable files. It
is an in-place replacement of V5 in mm-new. It affects Mike Rapoport's
"make MM selftests more CI friendly", since "selftests/mm: khugepaged:
use kselftest framework" needs to be updated. I updated it and put it at
the end of this cover letter.
Before the patchset, the status of creating read-only THPs is below:
| PF | MADV_COLLAPSE | khugepaged |
|-----------|---------------|------------|
large folio FSes only | ✓ | x | x |
READ_ONLY_THP_FOR_FS only | x | ✓ | ✓ |
both | ✓ | ✓ | ✓ |
where READ_ONLY_THP_FOR_FS implies no large folio FSes.
Now without READ_ONLY_THP_FOR_FS:
| PF | MADV_COLLAPSE | khugepaged |
|-----------|---------------|------------|
large folio FSes (read-only fd) | ✓ | ✓ | ✓ |
large folio FSes (read-write fd) | ✓ | ✓ | ✓* |
no large folio FSes | x | x | x |
* khugepaged only collapses clean folios from writable files. Userspace
must flush dirty folios explicitly before khugepaged can collapse them.
MADV_COLLAPSE handles the flush automatically via its writeback-and-retry
path. Collapsing writable MAP_PRIVATE pagecache folios is still not
supported, since PMD THP CoW only faults in at PTE level to avoid long
CoW latency, and file_backed_vma_is_retractable() prevents it.
This means no-large-folio FSes need to add large folio support (the
supported orders need to include PMD_ORDER), so that they can leverage
file THP creation.
To prevent breaking file THP support for large folio FSes,
1. first 4 patches enable the support, so that without READ_ONLY_THP_FOR_FS,
file THP still works for large folio FSes,
2. Patch 5 removes READ_ONLY_THP_FOR_FS Kconfig,
3. patches 6-12 remove code related to READ_ONLY_THP_FOR_FS,
4. patches 13-14 enable clean pagecache folio collapse for writable files.
NOTE: collapsing writable MAP_PRIVATE pagecache folios is not supported,
since:
1. PMD THP CoW only faults in at PTE level to avoid long CoW latency,
2. the first check, due to 1, in file_backed_vma_is_retractable() prevents it.
Overview
===
1. collapse_file() checks for to-be-collapsed folio dirtiness after they
are locked and unmapped to make sure no new write happens. Before,
mapping->nr_thps and inode->i_writecount were used to cause read-only
THP truncation before a fd becomes writable.
2. hugepage_enabled() is true for anon, shmem, and file-backed cases
if the global khugepaged control is on, otherwise, khugepaged for
file-backed case is turned off and anon and shmem depend on per-size
control knobs.
3. collapse_file() from mm/khugepaged.c, instead of checking
CONFIG_READ_ONLY_THP_FOR_FS, makes sure the mapping_max_folio_order()
of struct address_space of the file is at least PMD_ORDER.
4. file_thp_enabled() checks mapping_max_folio_order() instead of
CONFIG_READ_ONLY_THP_FOR_FS and no longer checks if the file is opened
read-only. The dirty folio check after try_to_unmap() (Change 1)
handles writable files correctly.
5. truncate_inode_partial_folio() calls folio_split() directly instead
of the removed try_folio_split_to_order(), since large folios can
only show up on a FS with large folio support.
6. nr_thps is removed from struct address_space, since it is no longer
needed to drop all read-only THPs from a FS without large folio
support when the fd becomes writable. Its related filemap_nr_thps*()
are removed too.
7. folio_check_splittable() no longer checks READ_ONLY_THP_FOR_FS.
8. collapse_file() only calls filemap_flush() for read-only files.
Blindly flushing dirty folios from writable files would cause
undesirable system-wide writeback; userspace is expected to flush
explicitly, or use MADV_COLLAPSE which handles it via its retry path.
9. Updated comments and selftests in various places.
Changelog
===
From V5[6]:
1. added mapping_min_folio_order(mapping) <= PMD_ORDER check to
mapping_pmd_folio_support() in Patch 1 to correctly handle
filesystems whose minimum folio order exceeds PMD_ORDER. Also
improved the kernel-doc comment per David's suggestions.
2. cleaned up Patch 11 per David's review: use const for open_opt and
mmap_prot, remove mmap_opt (use MAP_SHARED for both read-only and
read-write mappings), inline file_fault_common() into separate
file_fault_read() and file_fault_write() functions, fix "read only"
typo to "read-only", update usage message to "with PMD-sized large
folio support". Also fixed run_vmtests.sh to use elif test_selected
thp for the SKIP case to avoid spurious [SKIP] output per Nico's
report.
3. revised stale comment in Patch 13: removed "There won't be new dirty
pages" and updated "khugepaged only works on read-only fd" to reflect
that writable files are now supported; merged the comment blocks per
David's suggestion.
From V4[5]:
1. fixed Patch 1's compilation error in !CONFIG_TRANSPARENT_HUGEPAGE
2. changed Patch 3 to no longer enable collapse for read-write fd but only
allowe read-only fd.
3. added two new patches to enable clean pagecache folio collapse for
writable files:
- Patch 13: remove inode_is_open_for_write() from file_thp_enabled()
so that khugepaged and MADV_COLLAPSE can process writable files.
filemap_flush() in collapse_file() is now conditionalized on the file
being read-only, to avoid repeatedly writing back dirty folios from
writable files.
- Patch 14: add read_write_file_read_ops and read_write_file_write_ops
to the khugepaged selftest to cover the new writable-file collapse paths.
From V3[4]:
1. added a TODO comment in patch 1 noting that the is_shmem exception in
the VM_WARN_ON_ONCE() check can be removed once shmem always calls
mapping_set_large_folios() on its mapping. Used VM_WARN_ON_ONCE() in
mapping_pmd_thp_support() instead.
2. fixed the dirty folio bail-out path in patch 2: add xas_unlock_irq()
and folio_putback_lru() before the goto, which were missing and would
have left the XA lock held and the LRU isolation ref leaked.
3. renamed hugepage_pmd_enabled() to hugepage_enabled() to reflect it
controls khugepaged for all transparent hugepage types.
4. reverted the comment in hugepage_enabled() in patch 4 to the original;
only removed the phrase "when configured in," which referred to
CONFIG_READ_ONLY_THP_FOR_FS.
5. fixed commit message in patch 6: the dirty folio check is added after
try_to_unmap() in collapse_file(), not after try_to_unmap_flush().
From V2[3]:
1. removed unnecessary check in collapse_scan_file().
2. removed inode_is_open_for_write() check in file_thp_enabled().
3. changed hugepage_enabled() to return true if khugepaged global
control is on instead of false. cleaned up anon and shmem code in the
function.
4. moved folio dirtiness check after try_to_unmap() but before
try_to_unmap_flush(), since that is sufficient to prevent new writes.
5. reordered patch 4 and 5, so that khugepaged behavior does not change
after READ_ONLY_THP_FOR_FS is removed.
6. added read-write file test in khugepaged selftest.
7. removed the read-only file restriction from guard-region selftest.
From V1[2]:
1. removed inode_is_open_for_write() check in collapse_file(), since the
added folio dirtiness check after try_to_unmap_flush() should be
sufficient to prevent writes to candidate folios.
2. removed READ_ONLY_THP_FOR_FS check in hugepage_enabled(), please
see Patch 5 and item 2 in the overview for more details.
3. moved the patch removing READ_ONLY_THP_FOR_FS Kconfig after enabling
khugepaged and MADV_COLLAPSE to create read-only THPs.
4. added mapping_pmd_thp_support() helper function.
5. used VM_WARN_ON_ONCE() in collapse_file() for mapping eligibility check
and address alignment check instead of if + return error code. Always
allow shmem, since MADV_COLLAPSE ignore shmem huge config.
6. added mapping eligibility check in collapse_scan_file().
7. removed trailing ; for folio_split() in the !CONFIG_TRANSPARENT_HUGEPAGE.
8. simplified code in folio_check_splittable() after removing
READ_ONLY_THP_FOR_FS code.
9. clarified that read-only THP works for FSes with PMD THP support by
default.
From RFC[1]:
1. instead of removing READ_ONLY_THP_FOR_FS function entirely, turn it
on by default for all FSes with large folio support and the supported
orders includes PMD_ORDER.
Suggestions and comments are welcome.
Link: https://lore.kernel.org/all/20260323190644.1714379-1-ziy@nvidia.com/ [1]
Link: https://lore.kernel.org/all/20260327014255.2058916-1-ziy@nvidia.com/ [2]
Link: https://lore.kernel.org/all/20260413192030.3275825-1-ziy@nvidia.com/ [3]
Link: https://lore.kernel.org/all/20260418024429.4055056-1-ziy@nvidia.com/ [4]
Link: https://lore.kernel.org/all/20260424024915.28758-1-ziy@nvidia.com/ [5]
Link: https://lore.kernel.org/all/20260429152924.727124-1-ziy@nvidia.com/ [6]
For Andrew to update "selftests/mm: khugepaged: use kselftest framework"
from Mike Rapoport's "make MM selftests more CI friendly" series.
===
From 29f1e70373419e304ba7a69bc78fb43ba40ebfed Mon Sep 17 00:00:00 2001
From: "Mike Rapoport (Microsoft)" <rppt@kernel.org>
Date: Mon, 11 May 2026 19:27:58 +0300
Subject: [PATCH] selftests/mm: khugepaged: use kselftest framework
Convert khugepaged tests to use kselftest framework for reporting and
tracking successful and failing runs.
The conversion is mostly about replacing printf()/perror() + exit() pairs
with their ksft_ counterparts.
The nice colored success and failure indications are left intact.
Replace the progress report in collapse_compound_extreme() with a single
ksft_print_msg() to avoid headache with formatting and make the test
output more concise.
Link: https://lore.kernel.org/20260511162840.375890-15-rppt@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Tested-by: Luiz Capitulino <luizcap@redhat.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Donet Tom <donettom@linux.ibm.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Li Wang <li.wang@linux.dev>
Cc: Sarthak Sharma <sarthak.sharma@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
tools/testing/selftests/mm/khugepaged.c | 321 ++++++++++--------------
1 file changed, 132 insertions(+), 189 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
index 7f61bfa455e96..a2a3a52031031 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -86,17 +86,19 @@ static int exit_status;
static void success(const char *msg)
{
printf(" \e[32m%s\e[0m\n", msg);
+ exit_status = KSFT_PASS;
}
static void fail(const char *msg)
{
printf(" \e[31m%s\e[0m\n", msg);
- exit_status++;
+ exit_status = KSFT_FAIL;
}
static void skip(const char *msg)
{
printf(" \e[33m%s\e[0m\n", msg);
+ exit_status = KSFT_SKIP;
}
static void restore_settings_atexit(void)
@@ -104,22 +106,24 @@ static void restore_settings_atexit(void)
if (skip_settings_restore)
return;
- printf("Restore THP and khugepaged settings...");
+ ksft_print_msg("Restore THP and khugepaged settings...");
thp_restore_settings();
success("OK");
skip_settings_restore = true;
+ ksft_print_cnts();
+ exit(exit_status);
}
static void restore_settings(int sig)
{
/* exit() will invoke the restore_settings_atexit handler. */
- exit(sig ? EXIT_FAILURE : exit_status);
+ exit(sig ? KSFT_FAIL : exit_status);
}
static void save_settings(void)
{
- printf("Save THP and khugepaged settings...");
+ ksft_print_msg("Save THP and khugepaged settings...");
if ((read_only_file_ops || read_write_file_read_ops ||
read_write_file_write_ops) &&
finfo.type == VMA_FILE)
@@ -145,19 +149,13 @@ static void get_finfo(const char *dir)
finfo.dir = dir;
stat(finfo.dir, &path_stat);
- if (!S_ISDIR(path_stat.st_mode)) {
- printf("%s: Not a directory (%s)\n", __func__, finfo.dir);
- exit(EXIT_FAILURE);
- }
+ if (!S_ISDIR(path_stat.st_mode))
+ ksft_exit_fail_msg("%s: Not a directory (%s)\n", __func__, finfo.dir);
if (snprintf(finfo.path, sizeof(finfo.path), "%s/" TEST_FILE,
- finfo.dir) >= sizeof(finfo.path)) {
- printf("%s: Pathname is too long\n", __func__);
- exit(EXIT_FAILURE);
- }
- if (statfs(finfo.dir, &fs)) {
- perror("statfs()");
- exit(EXIT_FAILURE);
- }
+ finfo.dir) >= sizeof(finfo.path))
+ ksft_exit_fail_msg("%s: Pathname is too long\n", __func__);
+ if (statfs(finfo.dir, &fs))
+ ksft_exit_fail_perror("statfs()");
finfo.type = fs.f_type == TMPFS_MAGIC ? VMA_SHMEM : VMA_FILE;
if (finfo.type == VMA_SHMEM)
return;
@@ -165,40 +163,30 @@ static void get_finfo(const char *dir)
/* Find owning device's queue/read_ahead_kb control */
if (snprintf(path, sizeof(path), "/sys/dev/block/%d:%d/uevent",
major(path_stat.st_dev), minor(path_stat.st_dev))
- >= sizeof(path)) {
- printf("%s: Pathname is too long\n", __func__);
- exit(EXIT_FAILURE);
- }
- if (read_file(path, buf, sizeof(buf)) < 0) {
- perror("read_file(read_num)");
- exit(EXIT_FAILURE);
- }
+ >= sizeof(path))
+ ksft_exit_fail_msg("%s: Pathname is too long\n", __func__);
+ if (read_file(path, buf, sizeof(buf)) < 0)
+ ksft_exit_fail_perror("read_file(read_num)");
if (strstr(buf, "DEVTYPE=disk")) {
/* Found it */
if (snprintf(finfo.dev_queue_read_ahead_path,
sizeof(finfo.dev_queue_read_ahead_path),
"/sys/dev/block/%d:%d/queue/read_ahead_kb",
major(path_stat.st_dev), minor(path_stat.st_dev))
- >= sizeof(finfo.dev_queue_read_ahead_path)) {
- printf("%s: Pathname is too long\n", __func__);
- exit(EXIT_FAILURE);
- }
+ >= sizeof(finfo.dev_queue_read_ahead_path))
+ ksft_exit_fail_msg("%s: Pathname is too long\n", __func__);
return;
}
- if (!strstr(buf, "DEVTYPE=partition")) {
- printf("%s: Unknown device type: %s\n", __func__, path);
- exit(EXIT_FAILURE);
- }
+ if (!strstr(buf, "DEVTYPE=partition"))
+ ksft_exit_fail_msg("%s: Unknown device type: %s\n", __func__, path);
/*
* Partition of block device - need to find actual device.
* Using naming convention that devnameN is partition of
* device devname.
*/
str = strstr(buf, "DEVNAME=");
- if (!str) {
- printf("%s: Could not read: %s", __func__, path);
- exit(EXIT_FAILURE);
- }
+ if (!str)
+ ksft_exit_fail_msg("%s: Could not read: %s", __func__, path);
str += 8;
end = str;
while (*end) {
@@ -207,16 +195,13 @@ static void get_finfo(const char *dir)
if (snprintf(finfo.dev_queue_read_ahead_path,
sizeof(finfo.dev_queue_read_ahead_path),
"/sys/block/%s/queue/read_ahead_kb",
- str) >= sizeof(finfo.dev_queue_read_ahead_path)) {
- printf("%s: Pathname is too long\n", __func__);
- exit(EXIT_FAILURE);
- }
+ str) >= sizeof(finfo.dev_queue_read_ahead_path))
+ ksft_exit_fail_msg("%s: Pathname is too long\n", __func__);
return;
}
++end;
}
- printf("%s: Could not read: %s\n", __func__, path);
- exit(EXIT_FAILURE);
+ ksft_exit_fail_msg("%s: Could not read: %s\n", __func__, path);
}
static bool check_swap(void *addr, unsigned long size)
@@ -229,26 +214,19 @@ static bool check_swap(void *addr, unsigned long size)
ret = snprintf(addr_pattern, MAX_LINE_LENGTH, "%08lx-",
(unsigned long) addr);
- if (ret >= MAX_LINE_LENGTH) {
- printf("%s: Pattern is too long\n", __func__);
- exit(EXIT_FAILURE);
- }
-
+ if (ret >= MAX_LINE_LENGTH)
+ ksft_exit_fail_msg("%s: Pattern is too long\n", __func__);
fp = fopen(PID_SMAPS, "r");
- if (!fp) {
- printf("%s: Failed to open file %s\n", __func__, PID_SMAPS);
- exit(EXIT_FAILURE);
- }
+ if (!fp)
+ ksft_exit_fail_msg("%s: Failed to open file %s\n", __func__, PID_SMAPS);
if (!check_for_pattern(fp, addr_pattern, buffer, sizeof(buffer)))
goto err_out;
ret = snprintf(addr_pattern, MAX_LINE_LENGTH, "Swap:%19ld kB",
size >> 10);
- if (ret >= MAX_LINE_LENGTH) {
- printf("%s: Pattern is too long\n", __func__);
- exit(EXIT_FAILURE);
- }
+ if (ret >= MAX_LINE_LENGTH)
+ ksft_exit_fail_msg("%s: Pattern is too long\n", __func__);
/*
* Fetch the Swap: in the same block and check whether it got
* the expected number of hugeepages next.
@@ -271,10 +249,8 @@ static void *alloc_mapping(int nr)
p = mmap(BASE_ADDR, nr * hpage_pmd_size, PROT_READ | PROT_WRITE,
MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
- if (p != BASE_ADDR) {
- printf("Failed to allocate VMA at %p\n", BASE_ADDR);
- exit(EXIT_FAILURE);
- }
+ if (p != BASE_ADDR)
+ ksft_exit_fail_msg("Failed to allocate VMA at %p\n", BASE_ADDR);
return p;
}
@@ -324,19 +300,13 @@ static void *alloc_hpage(struct mem_ops *ops)
* khugepaged on low-load system (like a test machine), which
* would cause MADV_COLLAPSE to fail with EAGAIN.
*/
- printf("Allocate huge page...");
- if (madvise_collapse_retry(p, hpage_pmd_size)) {
- perror("madvise(MADV_COLLAPSE)");
- exit(EXIT_FAILURE);
- }
- if (!ops->check_huge(p, 1)) {
- perror("madvise(MADV_COLLAPSE)");
- exit(EXIT_FAILURE);
- }
- if (madvise(p, hpage_pmd_size, MADV_HUGEPAGE)) {
- perror("madvise(MADV_HUGEPAGE)");
- exit(EXIT_FAILURE);
- }
+ ksft_print_msg("Allocate huge page...");
+ if (madvise_collapse_retry(p, hpage_pmd_size))
+ ksft_exit_fail_perror("madvise(MADV_COLLAPSE)");
+ if (!ops->check_huge(p, 1))
+ ksft_exit_fail_perror("madvise(MADV_COLLAPSE)");
+ if (madvise(p, hpage_pmd_size, MADV_HUGEPAGE))
+ ksft_exit_fail_perror("madvise(MADV_HUGEPAGE)");
success("OK");
return p;
}
@@ -346,11 +316,9 @@ static void validate_memory(int *p, unsigned long start, unsigned long end)
int i;
for (i = start / page_size; i < end / page_size; i++) {
- if (p[i * page_size / sizeof(*p)] != i + 0xdead0000) {
- printf("Page %d is corrupted: %#x\n",
- i, p[i * page_size / sizeof(*p)]);
- exit(EXIT_FAILURE);
- }
+ if (p[i * page_size / sizeof(*p)] != i + 0xdead0000)
+ ksft_exit_fail_msg("Page %d is corrupted: %#x\n",
+ i, p[i * page_size / sizeof(*p)]);
}
}
@@ -383,14 +351,12 @@ static void *file_setup_area_common(int nr_hpages, enum file_setup_ops setup)
unsigned long size;
unlink(finfo.path); /* Cleanup from previous failed tests */
- printf("Creating %s for collapse%s...", finfo.path,
- finfo.type == VMA_SHMEM ? " (tmpfs)" : "");
+ ksft_print_msg("Creating %s for collapse%s...", finfo.path,
+ finfo.type == VMA_SHMEM ? " (tmpfs)" : "");
fd = open(finfo.path, O_CREAT | O_RDWR | O_TRUNC | O_EXCL,
777);
- if (fd < 0) {
- perror("open()");
- exit(EXIT_FAILURE);
- }
+ if (fd < 0)
+ ksft_exit_fail_perror("open()");
size = nr_hpages * hpage_pmd_size;
if (ftruncate(fd, size)) {
@@ -411,22 +377,17 @@ static void *file_setup_area_common(int nr_hpages, enum file_setup_ops setup)
close(fd);
munmap(p, size);
success("OK");
-
- printf("Opening %s %s for collapse...", finfo.path,
+ ksft_print_msg("Opening %s %s for collapse...", finfo.path,
setup == FILE_SETUP_READ_ONLY_FS ? "read-only" :
setup == FILE_SETUP_READ_WRITE_FS_READ_DATA ?
"read-write (read)" :
"read-write (write)");
finfo.fd = open(finfo.path, open_opt, 777);
- if (finfo.fd < 0) {
- perror("open()");
- exit(EXIT_FAILURE);
- }
+ if (finfo.fd < 0)
+ ksft_exit_fail_perror("open()");
p = mmap(BASE_ADDR, size, mmap_prot, MAP_SHARED, finfo.fd, 0);
- if (p == MAP_FAILED || p != BASE_ADDR) {
- perror("mmap()");
- exit(EXIT_FAILURE);
- }
+ if (p == MAP_FAILED || p != BASE_ADDR)
+ ksft_exit_fail_perror("mmap()");
/* Drop page cache */
write_file("/proc/sys/vm/drop_caches", "3", 2);
@@ -458,10 +419,8 @@ static void file_cleanup_area(void *p, unsigned long size)
static void file_fault_read(void *p, unsigned long start, unsigned long end)
{
- if (madvise(((char *)p) + start, end - start, MADV_POPULATE_READ)) {
- perror("madvise(MADV_POPULATE_READ)");
- exit(EXIT_FAILURE);
- }
+ if (madvise(((char *)p) + start, end - start, MADV_POPULATE_READ))
+ ksft_exit_fail_perror("madvise(MADV_POPULATE_READ)");
}
static void file_fault_read_and_flush(void *p, unsigned long start, unsigned long end)
@@ -476,10 +435,8 @@ static void file_fault_read_and_flush(void *p, unsigned long start, unsigned lon
static void file_fault_write(void *p, unsigned long start, unsigned long end)
{
- if (madvise(((char *)p) + start, end - start, MADV_POPULATE_WRITE)) {
- perror("madvise(MADV_POPULATE_WRITE)");
- exit(EXIT_FAILURE);
- }
+ if (madvise(((char *)p) + start, end - start, MADV_POPULATE_WRITE))
+ ksft_exit_fail_perror("madvise(MADV_POPULATE_WRITE)");
}
static bool file_check_huge(void *addr, int nr_hpages)
@@ -501,20 +458,14 @@ static void *shmem_setup_area(int nr_hpages)
unsigned long size = nr_hpages * hpage_pmd_size;
finfo.fd = memfd_create("khugepaged-selftest-collapse-shmem", 0);
- if (finfo.fd < 0) {
- perror("memfd_create()");
- exit(EXIT_FAILURE);
- }
- if (ftruncate(finfo.fd, size)) {
- perror("ftruncate()");
- exit(EXIT_FAILURE);
- }
+ if (finfo.fd < 0)
+ ksft_exit_fail_perror("memfd_create()");
+ if (ftruncate(finfo.fd, size))
+ ksft_exit_fail_perror("ftruncate()");
p = mmap(BASE_ADDR, size, PROT_READ | PROT_WRITE, MAP_SHARED, finfo.fd,
0);
- if (p != BASE_ADDR) {
- perror("mmap()");
- exit(EXIT_FAILURE);
- }
+ if (p != BASE_ADDR)
+ ksft_exit_fail_perror("mmap()");
return p;
}
@@ -588,7 +539,7 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
int ret;
struct thp_settings settings = *thp_current_settings();
- printf("%s...", msg);
+ ksft_print_msg("%s...", msg);
/*
* read&write file collapse succeeds for MADV_COLLAPSE because dirty
@@ -621,10 +572,8 @@ static void madvise_collapse(const char *msg, char *p, int nr_hpages,
struct mem_ops *ops, bool expect)
{
/* Sanity check */
- if (!ops->check_huge(p, 0)) {
- printf("Unexpected huge page\n");
- exit(EXIT_FAILURE);
- }
+ if (!ops->check_huge(p, 0))
+ ksft_exit_fail_msg("Unexpected huge page\n");
__madvise_collapse(msg, p, nr_hpages, ops, expect);
}
@@ -636,17 +585,15 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,
int timeout = 6; /* 3 seconds */
/* Sanity check */
- if (!ops->check_huge(p, 0)) {
- printf("Unexpected huge page\n");
- exit(EXIT_FAILURE);
- }
+ if (!ops->check_huge(p, 0))
+ ksft_exit_fail_msg("Unexpected huge page\n");
madvise(p, nr_hpages * hpage_pmd_size, MADV_HUGEPAGE);
/* Wait until the second full_scan completed */
full_scans = thp_read_num("khugepaged/full_scans") + 2;
- printf("%s...", msg);
+ ksft_print_msg("%s...", msg);
while (timeout--) {
if (ops->check_huge(p, nr_hpages))
break;
@@ -713,7 +660,7 @@ static void alloc_at_fault(void)
p = alloc_mapping(1);
*p = 1;
- printf("Allocate huge page on fault...");
+ ksft_print_msg("Allocate huge page on fault...");
if (check_huge_anon(p, 1, hpage_pmd_size))
success("OK");
else
@@ -722,12 +669,14 @@ static void alloc_at_fault(void)
thp_pop_settings();
madvise(p, page_size, MADV_DONTNEED);
- printf("Split huge PMD on MADV_DONTNEED...");
+ ksft_print_msg("Split huge PMD on MADV_DONTNEED...");
if (check_huge_anon(p, 0, hpage_pmd_size))
success("OK");
else
fail("Fail");
munmap(p, hpage_pmd_size);
+
+ ksft_test_result_report(exit_status, "allocate on fault and split\n");
}
static void collapse_full(struct collapse_context *c, struct mem_ops *ops)
@@ -742,6 +691,8 @@ static void collapse_full(struct collapse_context *c, struct mem_ops *ops)
ops, true);
validate_memory(p, 0, size);
ops->cleanup_area(p, size);
+
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_empty(struct collapse_context *c, struct mem_ops *ops)
@@ -751,6 +702,7 @@ static void collapse_empty(struct collapse_context *c, struct mem_ops *ops)
p = ops->setup_area(1);
c->collapse("Do not collapse empty PTE table", p, 1, ops, false);
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_single_pte_entry(struct collapse_context *c, struct mem_ops *ops)
@@ -762,6 +714,7 @@ static void collapse_single_pte_entry(struct collapse_context *c, struct mem_ops
c->collapse("Collapse PTE table with single PTE entry present", p,
1, ops, true);
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_max_ptes_none(struct collapse_context *c, struct mem_ops *ops)
@@ -801,6 +754,7 @@ static void collapse_max_ptes_none(struct collapse_context *c, struct mem_ops *o
skip:
ops->cleanup_area(p, hpage_pmd_size);
thp_pop_settings();
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_swapin_single_pte(struct collapse_context *c, struct mem_ops *ops)
@@ -810,11 +764,9 @@ static void collapse_swapin_single_pte(struct collapse_context *c, struct mem_op
p = ops->setup_area(1);
ops->fault(p, 0, hpage_pmd_size);
- printf("Swapout one page...");
- if (madvise(p, page_size, MADV_PAGEOUT)) {
- perror("madvise(MADV_PAGEOUT)");
- exit(EXIT_FAILURE);
- }
+ ksft_print_msg("Swapout one page...");
+ if (madvise(p, page_size, MADV_PAGEOUT))
+ ksft_exit_fail_perror("madvise(MADV_PAGEOUT)");
if (check_swap(p, page_size)) {
success("OK");
} else {
@@ -827,6 +779,7 @@ static void collapse_swapin_single_pte(struct collapse_context *c, struct mem_op
validate_memory(p, 0, hpage_pmd_size);
out:
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *ops)
@@ -837,11 +790,9 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *o
p = ops->setup_area(1);
ops->fault(p, 0, hpage_pmd_size);
- printf("Swapout %d of %d pages...", max_ptes_swap + 1, hpage_pmd_nr);
- if (madvise(p, (max_ptes_swap + 1) * page_size, MADV_PAGEOUT)) {
- perror("madvise(MADV_PAGEOUT)");
- exit(EXIT_FAILURE);
- }
+ ksft_print_msg("Swapout %d of %d pages...", max_ptes_swap + 1, hpage_pmd_nr);
+ if (madvise(p, (max_ptes_swap + 1) * page_size, MADV_PAGEOUT))
+ ksft_exit_fail_perror("madvise(MADV_PAGEOUT)");
if (check_swap(p, (max_ptes_swap + 1) * page_size)) {
success("OK");
} else {
@@ -855,12 +806,10 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *o
if (c->enforce_pte_scan_limits) {
ops->fault(p, 0, hpage_pmd_size);
- printf("Swapout %d of %d pages...", max_ptes_swap,
+ ksft_print_msg("Swapout %d of %d pages...", max_ptes_swap,
hpage_pmd_nr);
- if (madvise(p, max_ptes_swap * page_size, MADV_PAGEOUT)) {
- perror("madvise(MADV_PAGEOUT)");
- exit(EXIT_FAILURE);
- }
+ if (madvise(p, max_ptes_swap * page_size, MADV_PAGEOUT))
+ ksft_exit_fail_perror("madvise(MADV_PAGEOUT)");
if (check_swap(p, max_ptes_swap * page_size)) {
success("OK");
} else {
@@ -874,6 +823,7 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *o
}
out:
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_single_pte_entry_compound(struct collapse_context *c, struct mem_ops *ops)
@@ -890,7 +840,7 @@ static void collapse_single_pte_entry_compound(struct collapse_context *c, struc
}
madvise(p, hpage_pmd_size, MADV_NOHUGEPAGE);
- printf("Split huge page leaving single PTE mapping compound page...");
+ ksft_print_msg("Split huge page leaving single PTE mapping compound page...");
madvise(p + page_size, hpage_pmd_size - page_size, MADV_DONTNEED);
if (ops->check_huge(p, 0))
success("OK");
@@ -902,6 +852,7 @@ static void collapse_single_pte_entry_compound(struct collapse_context *c, struc
validate_memory(p, 0, page_size);
skip:
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_full_of_compound(struct collapse_context *c, struct mem_ops *ops)
@@ -909,7 +860,7 @@ static void collapse_full_of_compound(struct collapse_context *c, struct mem_ops
void *p;
p = alloc_hpage(ops);
- printf("Split huge page leaving single PTE page table full of compound pages...");
+ ksft_print_msg("Split huge page leaving single PTE page table full of compound pages...");
madvise(p, page_size, MADV_NOHUGEPAGE);
madvise(p, hpage_pmd_size, MADV_NOHUGEPAGE);
if (ops->check_huge(p, 0))
@@ -921,6 +872,7 @@ static void collapse_full_of_compound(struct collapse_context *c, struct mem_ops
true);
validate_memory(p, 0, hpage_pmd_size);
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_compound_extreme(struct collapse_context *c, struct mem_ops *ops)
@@ -929,16 +881,12 @@ static void collapse_compound_extreme(struct collapse_context *c, struct mem_ops
int i;
p = ops->setup_area(1);
+ ksft_print_msg("Construct PTE page table full of different PTE-mapped compound pages\n");
for (i = 0; i < hpage_pmd_nr; i++) {
- printf("\rConstruct PTE page table full of different PTE-mapped compound pages %3d/%d...",
- i + 1, hpage_pmd_nr);
-
madvise(BASE_ADDR, hpage_pmd_size, MADV_HUGEPAGE);
ops->fault(BASE_ADDR, 0, hpage_pmd_size);
- if (!ops->check_huge(BASE_ADDR, 1)) {
- printf("Failed to allocate huge page\n");
- exit(EXIT_FAILURE);
- }
+ if (!ops->check_huge(BASE_ADDR, 1))
+ ksft_exit_fail_msg("Failed to allocate huge page\n");
madvise(BASE_ADDR, hpage_pmd_size, MADV_NOHUGEPAGE);
p = mremap(BASE_ADDR - i * page_size,
@@ -946,20 +894,16 @@ static void collapse_compound_extreme(struct collapse_context *c, struct mem_ops
(i + 1) * page_size,
MREMAP_MAYMOVE | MREMAP_FIXED,
BASE_ADDR + 2 * hpage_pmd_size);
- if (p == MAP_FAILED) {
- perror("mremap+unmap");
- exit(EXIT_FAILURE);
- }
+ if (p == MAP_FAILED)
+ ksft_exit_fail_perror("mremap+unmap");
p = mremap(BASE_ADDR + 2 * hpage_pmd_size,
(i + 1) * page_size,
(i + 1) * page_size + hpage_pmd_size,
MREMAP_MAYMOVE | MREMAP_FIXED,
BASE_ADDR - (i + 1) * page_size);
- if (p == MAP_FAILED) {
- perror("mremap+alloc");
- exit(EXIT_FAILURE);
- }
+ if (p == MAP_FAILED)
+ ksft_exit_fail_perror("mremap+alloc");
}
ops->cleanup_area(BASE_ADDR, hpage_pmd_size);
@@ -974,6 +918,7 @@ static void collapse_compound_extreme(struct collapse_context *c, struct mem_ops
validate_memory(p, 0, hpage_pmd_size);
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_fork(struct collapse_context *c, struct mem_ops *ops)
@@ -983,18 +928,17 @@ static void collapse_fork(struct collapse_context *c, struct mem_ops *ops)
p = ops->setup_area(1);
- printf("Allocate small page...");
+ ksft_print_msg("Allocate small page...");
ops->fault(p, 0, page_size);
if (ops->check_huge(p, 0))
success("OK");
else
fail("Fail");
- printf("Share small page over fork()...");
+ ksft_print_msg("Share small page over fork()...");
if (!fork()) {
/* Do not touch settings on child exit */
skip_settings_restore = true;
- exit_status = 0;
if (ops->check_huge(p, 0))
success("OK");
@@ -1011,15 +955,16 @@ static void collapse_fork(struct collapse_context *c, struct mem_ops *ops)
}
wait(&wstatus);
- exit_status += WEXITSTATUS(wstatus);
+ exit_status = WEXITSTATUS(wstatus);
- printf("Check if parent still has small page...");
+ ksft_print_msg("Check if parent still has small page...");
if (ops->check_huge(p, 0))
success("OK");
else
fail("Fail");
validate_memory(p, 0, page_size);
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_fork_compound(struct collapse_context *c, struct mem_ops *ops)
@@ -1028,18 +973,17 @@ static void collapse_fork_compound(struct collapse_context *c, struct mem_ops *o
void *p;
p = alloc_hpage(ops);
- printf("Share huge page over fork()...");
+ ksft_print_msg("Share huge page over fork()...");
if (!fork()) {
/* Do not touch settings on child exit */
skip_settings_restore = true;
- exit_status = 0;
if (ops->check_huge(p, 1))
success("OK");
else
fail("Fail");
- printf("Split huge page PMD in child process...");
+ ksft_print_msg("Split huge page PMD in child process...");
madvise(p, page_size, MADV_NOHUGEPAGE);
madvise(p, hpage_pmd_size, MADV_NOHUGEPAGE);
if (ops->check_huge(p, 0))
@@ -1060,15 +1004,16 @@ static void collapse_fork_compound(struct collapse_context *c, struct mem_ops *o
}
wait(&wstatus);
- exit_status += WEXITSTATUS(wstatus);
+ exit_status = WEXITSTATUS(wstatus);
- printf("Check if parent still has huge page...");
+ ksft_print_msg("Check if parent still has huge page...");
if (ops->check_huge(p, 1))
success("OK");
else
fail("Fail");
validate_memory(p, 0, hpage_pmd_size);
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_ops *ops)
@@ -1078,18 +1023,17 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_ops
void *p;
p = alloc_hpage(ops);
- printf("Share huge page over fork()...");
+ ksft_print_msg("Share huge page over fork()...");
if (!fork()) {
/* Do not touch settings on child exit */
skip_settings_restore = true;
- exit_status = 0;
if (ops->check_huge(p, 1))
success("OK");
else
fail("Fail");
- printf("Trigger CoW on page %d of %d...",
+ ksft_print_msg("Trigger CoW on page %d of %d...",
hpage_pmd_nr - max_ptes_shared - 1, hpage_pmd_nr);
ops->fault(p, 0, (hpage_pmd_nr - max_ptes_shared - 1) * page_size);
if (ops->check_huge(p, 0))
@@ -1101,7 +1045,7 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_ops
1, ops, !c->enforce_pte_scan_limits);
if (c->enforce_pte_scan_limits) {
- printf("Trigger CoW on page %d of %d...",
+ ksft_print_msg("Trigger CoW on page %d of %d...",
hpage_pmd_nr - max_ptes_shared, hpage_pmd_nr);
ops->fault(p, 0, (hpage_pmd_nr - max_ptes_shared) *
page_size);
@@ -1120,15 +1064,16 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_ops
}
wait(&wstatus);
- exit_status += WEXITSTATUS(wstatus);
+ exit_status = WEXITSTATUS(wstatus);
- printf("Check if parent still has huge page...");
+ ksft_print_msg("Check if parent still has huge page...");
if (ops->check_huge(p, 1))
success("OK");
else
fail("Fail");
validate_memory(p, 0, hpage_pmd_size);
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void madvise_collapse_existing_thps(struct collapse_context *c,
@@ -1145,6 +1090,7 @@ static void madvise_collapse_existing_thps(struct collapse_context *c,
__madvise_collapse("Re-collapse PMD-mapped hugepage", p, 1, ops, true);
validate_memory(p, 0, hpage_pmd_size);
ops->cleanup_area(p, hpage_pmd_size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
/*
@@ -1172,6 +1118,7 @@ static void madvise_retracted_page_tables(struct collapse_context *c,
true);
validate_memory(p, 0, size);
ops->cleanup_area(p, size);
+ ksft_test_result_report(exit_status, "%s\n", __func__);
}
static void usage(void)
@@ -1280,10 +1227,8 @@ static int nr_test_cases;
#define TEST(t, c, o) do { \
if (c && o) { \
- if (nr_test_cases >= MAX_TEST_CASES) { \
- printf("MAX_TEST_CASES is too small\n"); \
- exit(EXIT_FAILURE); \
- } \
+ if (nr_test_cases >= MAX_TEST_CASES) \
+ ksft_exit_fail_msg("MAX_TEST_CASES is too small\n"); \
test_cases[nr_test_cases++] = (struct test_case){ \
.ctx = c, \
.ops = o, \
@@ -1316,10 +1261,10 @@ int main(int argc, char **argv)
.read_ahead_kb = 0,
};
- if (!thp_is_enabled()) {
- printf("Transparent Hugepages not available\n");
- return KSFT_SKIP;
- }
+ ksft_print_header();
+
+ if (!thp_is_enabled())
+ ksft_exit_skip("Transparent Hugepages not available\n");
parse_test_type(argc, argv);
@@ -1327,10 +1272,8 @@ int main(int argc, char **argv)
page_size = getpagesize();
hpage_pmd_size = read_pmd_pagesize();
- if (!hpage_pmd_size) {
- printf("Reading PMD pagesize failed");
- exit(EXIT_FAILURE);
- }
+ if (!hpage_pmd_size)
+ ksft_exit_fail_msg("Reading PMD pagesize failed\n");
hpage_pmd_nr = hpage_pmd_size / page_size;
hpage_pmd_order = __builtin_ctz(hpage_pmd_nr);
@@ -1346,8 +1289,6 @@ int main(int argc, char **argv)
save_settings();
thp_push_settings(&default_settings);
- alloc_at_fault();
-
TEST(collapse_full, khugepaged_context, anon_ops);
TEST(collapse_full, khugepaged_context, read_only_file_ops);
TEST(collapse_full, khugepaged_context, read_write_file_read_ops);
@@ -1425,11 +1366,13 @@ int main(int argc, char **argv)
TEST(madvise_retracted_page_tables, madvise_context, read_write_file_read_ops);
TEST(madvise_retracted_page_tables, madvise_context, shmem_ops);
- exit_status = KSFT_PASS;
+ ksft_set_plan(nr_test_cases + 1);
+
+ alloc_at_fault();
for (int i = 0; i < nr_test_cases; i++) {
struct test_case *t = &test_cases[i];
- printf("\nRun test: %s (%s:%s)\n", t->desc, t->ctx->name, t->ops->name);
+ ksft_print_msg("\nRun test: %s (%s:%s)\n", t->desc, t->ctx->name, t->ops->name);
t->fn(t->ctx, t->ops);
}
--
2.53.0
Zi Yan (14):
mm/khugepaged: remove READ_ONLY_THP_FOR_FS check
mm/khugepaged: add folio dirty check after try_to_unmap()
mm/huge_memory: remove READ_ONLY_THP_FOR_FS from file_thp_enabled()
mm/khugepaged: remove READ_ONLY_THP_FOR_FS check in hugepage_enabled()
mm: remove READ_ONLY_THP_FOR_FS Kconfig option
mm: fs: remove filemap_nr_thps*() functions and their users
fs: remove nr_thps from struct address_space
mm/huge_memory: remove folio split check for READ_ONLY_THP_FOR_FS
mm/truncate: use folio_split() in truncate_inode_partial_folio()
fs/btrfs: remove a comment referring to READ_ONLY_THP_FOR_FS
selftests/mm: remove READ_ONLY_THP_FOR_FS in khugepaged
selftests/mm: remove READ_ONLY_THP_FOR_FS code from guard-regions
mm/khugepaged: enable clean pagecache folio collapse for writable
files
selftests/mm: add writable-file collapse tests for khugepaged
fs/btrfs/defrag.c | 3 -
fs/inode.c | 3 -
fs/open.c | 27 ---
include/linux/fs.h | 5 -
include/linux/huge_mm.h | 25 +--
include/linux/pagemap.h | 50 +++---
include/linux/shmem_fs.h | 2 +-
mm/Kconfig | 11 --
mm/filemap.c | 1 -
mm/huge_memory.c | 39 +----
mm/khugepaged.c | 107 ++++++------
mm/truncate.c | 8 +-
tools/testing/selftests/mm/guard-regions.c | 18 +-
tools/testing/selftests/mm/khugepaged.c | 184 ++++++++++++++++-----
tools/testing/selftests/mm/run_vmtests.sh | 12 +-
15 files changed, 254 insertions(+), 241 deletions(-)
--
2.53.0
On Sun, 17 May 2026 09:54:02 -0400 Zi Yan <ziy@nvidia.com> wrote:
> Hi all,
>
> This patchset removes READ_ONLY_THP_FOR_FS Kconfig and enables creating
> file-backed THPs for FSes with large folio support (the supported orders
> need to include PMD_ORDER) by default, including for writable files.
Cool. Sashiko wasn't able to apply this (presumably because of Mike's
CI-friendly series). I take it that the AI review from v5
(https://sashiko.dev/#/patchset/20260429152924.727124-1-ziy@nvidia.com)
was considered?
Also, please check that the below were considered:
https://lore.kernel.org/e9e61132-902a-445f-9c4c-4d405d164e70@kernel.org
https://lore.kernel.org/22831162-abe7-4498-9e81-7f5aa3526d00@kernel.org
https://lore.kernel.org/959238dd-2493-4d9c-ac35-6d04460a8239@kernel.org
https://lore.kernel.org/1895A67C-BB1F-49EA-ADC3-AA4F51A6ED57@nvidia.com
https://lore.kernel.org/20260508074643.55548-1-lance.yang@linux.dev
https://lore.kernel.org/b8a3c3eb-f241-40fe-9121-4ae5a1097807@kernel.org
> is an in-place replacement of V5 in mm-new. It affects Mike Rapoport's
> "make MM selftests more CI friendly", since "selftests/mm: khugepaged:
> use kselftest framework" needs to be updated. I updated it and put it at
> the end of this cover letter.
Helpful, thanks. It was a little complicated because your email client
messes with whitespace (it always has!), but I figured it out.
> Changelog
> ===
> >From V5[6]:
> 1. added mapping_min_folio_order(mapping) <= PMD_ORDER check to
> mapping_pmd_folio_support() in Patch 1 to correctly handle
> filesystems whose minimum folio order exceeds PMD_ORDER. Also
> improved the kernel-doc comment per David's suggestions.
>
> 2. cleaned up Patch 11 per David's review: use const for open_opt and
> mmap_prot, remove mmap_opt (use MAP_SHARED for both read-only and
> read-write mappings), inline file_fault_common() into separate
> file_fault_read() and file_fault_write() functions, fix "read only"
> typo to "read-only", update usage message to "with PMD-sized large
> folio support". Also fixed run_vmtests.sh to use elif test_selected
> thp for the SKIP case to avoid spurious [SKIP] output per Nico's
> report.
>
> 3. revised stale comment in Patch 13: removed "There won't be new dirty
> pages" and updated "khugepaged only works on read-only fd" to reflect
> that writable files are now supported; merged the comment blocks per
> David's suggestion.
>
Here's how v6 altered mm.git:
include/linux/pagemap.h | 12 +++----
mm/khugepaged.c | 18 ++++-------
tools/testing/selftests/mm/khugepaged.c | 35 ++++++++--------------
3 files changed, 26 insertions(+), 39 deletions(-)
--- a/include/linux/pagemap.h~b
+++ a/include/linux/pagemap.h
@@ -514,15 +514,15 @@ static inline bool mapping_large_folio_s
}
/**
- * mapping_pmd_folio_support() - Check if a mapping support PMD-sized folio
+ * mapping_pmd_folio_support() - Check if a mapping supports PMD-sized folio
* @mapping: The address_space
*
- * Some file supports large folio but does not support as large as PMD order.
- * If a PMD-sized pagecache folio is attempted to be created on a filesystem,
- * this check needs to be performed first.
+ * While some mappings support large folios, they might not support PMD-sized
+ * folios. This function checks whether a mapping supports PMD-sized folios.
+ * For example, khugepaged needs this information before attempting to
+ * collapsing THPs.
*
- * Return: true - PMD-sized folio is supported, false - PMD-sized folio is not
- * supported.
+ * Return: True if PMD-sized folios are supported, otherwise false.
*/
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
static inline bool mapping_pmd_folio_support(const struct address_space *mapping)
--- a/mm/khugepaged.c~b
+++ a/mm/khugepaged.c
@@ -2342,23 +2342,19 @@ static enum scan_result collapse_file(st
} else if (folio_test_dirty(folio)) {
/*
* This page is dirty because it hasn't
- * been flushed since first write. There
- * won't be new dirty pages.
+ * been flushed since first write.
*
- * Trigger async flush here and hope the
- * writeback is done when khugepaged
- * revisits this page.
+ * Trigger async flush for read-only files and
+ * hope the writeback is done when khugepaged
+ * revisits this page. Writable files can have
+ * their folios dirty at any time; blindly
+ * flushing them would cause undesirable
+ * system-wide writeback.
*
* This is a one-off situation. We are not
* forcing writeback in loop.
*/
xas_unlock_irq(&xas);
- /*
- * Only flush for read-only files. Writable
- * files can have their folios dirty at any
- * time; blindly flushing them would cause
- * undesirable system-wide writeback.
- */
if (!inode_is_open_for_write(mapping->host))
filemap_flush(mapping);
result = SCAN_PAGE_DIRTY_OR_WRITEBACK;
--- a/tools/testing/selftests/mm/khugepaged.c~b
+++ a/tools/testing/selftests/mm/khugepaged.c
@@ -376,12 +376,11 @@ static bool anon_check_huge(void *addr,
static void *file_setup_area_common(int nr_hpages, enum file_setup_ops setup)
{
+ const int open_opt = setup == FILE_SETUP_READ_ONLY_FS ? O_RDONLY : O_RDWR;
+ const int mmap_prot = setup == FILE_SETUP_READ_ONLY_FS ? PROT_READ : (PROT_READ | PROT_WRITE);
int fd;
void *p;
unsigned long size;
- int open_opt = setup == FILE_SETUP_READ_ONLY_FS ? O_RDONLY : O_RDWR;
- int mmap_prot = setup == FILE_SETUP_READ_ONLY_FS ? PROT_READ : (PROT_READ | PROT_WRITE);
- int mmap_opt = setup == FILE_SETUP_READ_ONLY_FS ? MAP_PRIVATE : MAP_SHARED;
unlink(finfo.path); /* Cleanup from previous failed tests */
printf("Creating %s for collapse%s...", finfo.path,
@@ -414,7 +413,7 @@ static void *file_setup_area_common(int
success("OK");
printf("Opening %s %s for collapse...", finfo.path,
- setup == FILE_SETUP_READ_ONLY_FS ? "read only" :
+ setup == FILE_SETUP_READ_ONLY_FS ? "read-only" :
setup == FILE_SETUP_READ_WRITE_FS_READ_DATA ?
"read-write (read)" :
"read-write (write)");
@@ -423,8 +422,7 @@ static void *file_setup_area_common(int
perror("open()");
exit(EXIT_FAILURE);
}
- p = mmap(BASE_ADDR, size, mmap_prot,
- mmap_opt, finfo.fd, 0);
+ p = mmap(BASE_ADDR, size, mmap_prot, MAP_SHARED, finfo.fd, 0);
if (p == MAP_FAILED || p != BASE_ADDR) {
perror("mmap()");
exit(EXIT_FAILURE);
@@ -458,27 +456,17 @@ static void file_cleanup_area(void *p, u
unlink(finfo.path);
}
-static void file_fault_common(void *p, unsigned long start, unsigned long end,
- int madv_ops)
+static void file_fault_read(void *p, unsigned long start, unsigned long end)
{
- if (madvise(((char *)p) + start, end - start, madv_ops)) {
- if (madv_ops == MADV_POPULATE_READ)
- perror("madvise(MADV_POPULATE_READ");
- else if (madv_ops == MADV_POPULATE_WRITE)
- perror("madvise(MADV_POPULATE_WRITE");
+ if (madvise(((char *)p) + start, end - start, MADV_POPULATE_READ)) {
+ perror("madvise(MADV_POPULATE_READ)");
exit(EXIT_FAILURE);
}
}
-static void file_fault_read(void *p, unsigned long start, unsigned long end)
-{
- file_fault_common(p, start, end, MADV_POPULATE_READ);
-}
-
static void file_fault_read_and_flush(void *p, unsigned long start, unsigned long end)
{
- file_fault_common(p, start, end, MADV_POPULATE_READ);
-
+ file_fault_read(p, start, end);
/*
* make folio clean, since dirty folios from read&write file are
* rejected and not flushed
@@ -488,7 +476,10 @@ static void file_fault_read_and_flush(vo
static void file_fault_write(void *p, unsigned long start, unsigned long end)
{
- file_fault_common(p, start, end, MADV_POPULATE_WRITE);
+ if (madvise(((char *)p) + start, end - start, MADV_POPULATE_WRITE)) {
+ perror("madvise(MADV_POPULATE_WRITE)");
+ exit(EXIT_FAILURE);
+ }
}
static bool file_check_huge(void *addr, int nr_hpages)
@@ -1191,7 +1182,7 @@ static void usage(void)
fprintf(stderr, "\t<mem_type>\t: [all|anon|file|shmem]\n");
fprintf(stderr, "\n\t\"file,all\" mem_type requires [dir] argument\n");
fprintf(stderr, "\n\t\"file,all\" mem_type requires a file system\n");
- fprintf(stderr, "\twith large folio support (order >= PMD order)\n");
+ fprintf(stderr, "\twith PMD-sized large folio support\n");
fprintf(stderr, "\n\tif [dir] is a (sub)directory of a tmpfs mount, tmpfs must be\n");
fprintf(stderr, "\tmounted with huge=advise option for khugepaged tests to work\n");
fprintf(stderr, "\n\tSupported Options:\n");
_
On 19 May 2026, at 6:21, Andrew Morton wrote: > On Sun, 17 May 2026 09:54:02 -0400 Zi Yan <ziy@nvidia.com> wrote: > >> Hi all, >> >> This patchset removes READ_ONLY_THP_FOR_FS Kconfig and enables creating >> file-backed THPs for FSes with large folio support (the supported orders >> need to include PMD_ORDER) by default, including for writable files. > > Cool. Sashiko wasn't able to apply this (presumably because of Mike's > CI-friendly series). I take it that the AI review from v5 > (https://sashiko.dev/#/patchset/20260429152924.727124-1-ziy@nvidia.com) > was considered? > > Also, please check that the below were considered: > > https://lore.kernel.org/e9e61132-902a-445f-9c4c-4d405d164e70@kernel.org > https://lore.kernel.org/22831162-abe7-4498-9e81-7f5aa3526d00@kernel.org > https://lore.kernel.org/959238dd-2493-4d9c-ac35-6d04460a8239@kernel.org > https://lore.kernel.org/1895A67C-BB1F-49EA-ADC3-AA4F51A6ED57@nvidia.com > > https://lore.kernel.org/20260508074643.55548-1-lance.yang@linux.dev > https://lore.kernel.org/b8a3c3eb-f241-40fe-9121-4ae5a1097807@kernel.org > I addressed all of them, except the second one and I will reply to the second one soon. > >> is an in-place replacement of V5 in mm-new. It affects Mike Rapoport's >> "make MM selftests more CI friendly", since "selftests/mm: khugepaged: >> use kselftest framework" needs to be updated. I updated it and put it at >> the end of this cover letter. > > Helpful, thanks. It was a little complicated because your email client > messes with whitespace (it always has!), but I figured it out. You mean tabs become whitespace? And does all my patches have the whitespace issue or the one I put at the bottom? I am trying to figure out the issue. BTW, All patches are sent via git send-email using mstmp and I copied the bottom one from git format-patch. I wonder which part went wrong. Thanks. Best Regards, Yan, Zi
On Tue, 19 May 2026 07:39:07 +0800 Zi Yan <ziy@nvidia.com> wrote: > On 19 May 2026, at 6:21, Andrew Morton wrote: > > > On Sun, 17 May 2026 09:54:02 -0400 Zi Yan <ziy@nvidia.com> wrote: > > > >> Hi all, > >> > >> This patchset removes READ_ONLY_THP_FOR_FS Kconfig and enables creating > >> file-backed THPs for FSes with large folio support (the supported orders > >> need to include PMD_ORDER) by default, including for writable files. > > > > Cool. Sashiko wasn't able to apply this (presumably because of Mike's > > CI-friendly series). I take it that the AI review from v5 > > (https://sashiko.dev/#/patchset/20260429152924.727124-1-ziy@nvidia.com) > > was considered? > > > > Also, please check that the below were considered: > > > > https://lore.kernel.org/e9e61132-902a-445f-9c4c-4d405d164e70@kernel.org > > https://lore.kernel.org/22831162-abe7-4498-9e81-7f5aa3526d00@kernel.org > > https://lore.kernel.org/959238dd-2493-4d9c-ac35-6d04460a8239@kernel.org > > https://lore.kernel.org/1895A67C-BB1F-49EA-ADC3-AA4F51A6ED57@nvidia.com > > > > https://lore.kernel.org/20260508074643.55548-1-lance.yang@linux.dev > > https://lore.kernel.org/b8a3c3eb-f241-40fe-9121-4ae5a1097807@kernel.org > > > > I addressed all of them, except the second one and I will reply to the > second one soon. Great. > > > >> is an in-place replacement of V5 in mm-new. It affects Mike Rapoport's > >> "make MM selftests more CI friendly", since "selftests/mm: khugepaged: > >> use kselftest framework" needs to be updated. I updated it and put it at > >> the end of this cover letter. > > > > Helpful, thanks. It was a little complicated because your email client > > messes with whitespace (it always has!), but I figured it out. > > You mean tabs become whitespace? And does all my patches have the whitespace > issue or the one I put at the bottom? I am trying to figure out the issue. > BTW, All patches are sent via git send-email using mstmp and > I copied the bottom one from git format-patch. I wonder which part went > wrong. Your regular patches are OK - but when you informally paste a patch into the message body the tabs turn into whitespace. You aren't the only one ;) I usually don't comment...
On 19 May 2026, at 8:45, Andrew Morton wrote: > On Tue, 19 May 2026 07:39:07 +0800 Zi Yan <ziy@nvidia.com> wrote: > >> On 19 May 2026, at 6:21, Andrew Morton wrote: >> >>> On Sun, 17 May 2026 09:54:02 -0400 Zi Yan <ziy@nvidia.com> wrote: >>> >>>> Hi all, >>>> >>>> This patchset removes READ_ONLY_THP_FOR_FS Kconfig and enables creating >>>> file-backed THPs for FSes with large folio support (the supported orders >>>> need to include PMD_ORDER) by default, including for writable files. >>> >>> Cool. Sashiko wasn't able to apply this (presumably because of Mike's >>> CI-friendly series). I take it that the AI review from v5 >>> (https://sashiko.dev/#/patchset/20260429152924.727124-1-ziy@nvidia.com) >>> was considered? >>> >>> Also, please check that the below were considered: >>> >>> https://lore.kernel.org/e9e61132-902a-445f-9c4c-4d405d164e70@kernel.org >>> https://lore.kernel.org/22831162-abe7-4498-9e81-7f5aa3526d00@kernel.org >>> https://lore.kernel.org/959238dd-2493-4d9c-ac35-6d04460a8239@kernel.org >>> https://lore.kernel.org/1895A67C-BB1F-49EA-ADC3-AA4F51A6ED57@nvidia.com >>> >>> https://lore.kernel.org/20260508074643.55548-1-lance.yang@linux.dev >>> https://lore.kernel.org/b8a3c3eb-f241-40fe-9121-4ae5a1097807@kernel.org >>> >> >> I addressed all of them, except the second one and I will reply to the >> second one soon. > > Great. > >>> >>>> is an in-place replacement of V5 in mm-new. It affects Mike Rapoport's >>>> "make MM selftests more CI friendly", since "selftests/mm: khugepaged: >>>> use kselftest framework" needs to be updated. I updated it and put it at >>>> the end of this cover letter. >>> >>> Helpful, thanks. It was a little complicated because your email client >>> messes with whitespace (it always has!), but I figured it out. >> >> You mean tabs become whitespace? And does all my patches have the whitespace >> issue or the one I put at the bottom? I am trying to figure out the issue. >> BTW, All patches are sent via git send-email using mstmp and >> I copied the bottom one from git format-patch. I wonder which part went >> wrong. > > Your regular patches are OK - but when you informally paste a patch > into the message body the tabs turn into whitespace. I did that before, since I was copying diffs directly from a GUI terminal that converts tabs to whitespaces. Since you mentioned it last time, I always do git format-patch and copy the content from the patch file and it should work. For the patch I pasted in this cover letter, I copied it from the patch file directly using neovim, but somehow tabs from surrounding code are converted to whitespaces (tabs from +/- are not). I noticed it when I checked my original cover letter just now. So my email client does not cause the issue. Anyway, I will be more careful next time. Sorry for the trouble. > > You aren't the only one ;) I usually don't comment... Thank you for reminding me of the issue. Best Regards, Yan, Zi
collapse_file() requires FSes supporting large folio with at least
PMD_ORDER, so replace the READ_ONLY_THP_FOR_FS check with that.
MADV_COLLAPSE ignores shmem huge config, so exclude the check for shmem.
While at it, replace VM_BUG_ON with VM_WARN_ON_ONCE.
Add a helper function mapping_pmd_folio_support() for FSes supporting
large folio with at least PMD_ORDER.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Nico Pache <npache@redhat.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
include/linux/pagemap.h | 27 +++++++++++++++++++++++++++
mm/khugepaged.c | 10 ++++++++--
2 files changed, 35 insertions(+), 2 deletions(-)
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 1f50991b43e3b..308d846531d03 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -513,6 +513,33 @@ static inline bool mapping_large_folio_support(const struct address_space *mappi
return mapping_max_folio_order(mapping) > 0;
}
+/**
+ * mapping_pmd_folio_support() - Check if a mapping supports PMD-sized folio
+ * @mapping: The address_space
+ *
+ * While some mappings support large folios, they might not support PMD-sized
+ * folios. This function checks whether a mapping supports PMD-sized folios.
+ * For example, khugepaged needs this information before attempting to
+ * collapsing THPs.
+ *
+ * Return: True if PMD-sized folios are supported, otherwise false.
+ */
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+static inline bool mapping_pmd_folio_support(const struct address_space *mapping)
+{
+ /* AS_FOLIO_ORDER is only reasonable for pagecache folios */
+ VM_WARN_ON_ONCE((unsigned long)mapping & FOLIO_MAPPING_ANON);
+
+ return mapping_min_folio_order(mapping) <= PMD_ORDER &&
+ mapping_max_folio_order(mapping) >= PMD_ORDER;
+}
+#else
+static inline bool mapping_pmd_folio_support(const struct address_space *mapping)
+{
+ return false;
+}
+#endif
+
/* Return the maximum folio size for this pagecache mapping, in bytes. */
static inline size_t mapping_max_folio_size(const struct address_space *mapping)
{
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 5ba298d420b74..afd61168d915c 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -2243,8 +2243,14 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
int nr_none = 0;
bool is_shmem = shmem_file(file);
- VM_BUG_ON(!IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && !is_shmem);
- VM_BUG_ON(start & (HPAGE_PMD_NR - 1));
+ /*
+ * MADV_COLLAPSE ignores shmem huge config, so do not check shmem
+ *
+ * TODO: once shmem always calls mapping_set_large_folios() on its
+ * mapping, the shmem check can be removed.
+ */
+ VM_WARN_ON_ONCE(!is_shmem && !mapping_pmd_folio_support(mapping));
+ VM_WARN_ON_ONCE(start & (HPAGE_PMD_NR - 1));
result = alloc_charge_folio(&new_folio, mm, cc, HPAGE_PMD_ORDER);
if (result != SCAN_SUCCEED)
--
2.53.0This check ensures the correctness of read-only PMD folio collapse after
it is enabled for all FSes supporting PMD pagecache folios and replaces
READ_ONLY_THP_FOR_FS.
READ_ONLY_THP_FOR_FS only supports read-only fd and uses mapping->nr_thps
and inode->i_writecount to prevent any write to read-only to-be-collapsed
folios. In upcoming commits, READ_ONLY_THP_FOR_FS will be removed and the
aforementioned mechanism will go away too. To ensure khugepaged functions
as expected after the changes, skip if any folio is dirty after
try_to_unmap(), since a dirty folio at that point means this read-only
folio can get writes between try_to_unmap() and try_to_unmap_flush() via
cached TLB entries and khugepaged does not support writable pagecache
folio collapse yet.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Nico Pache <npache@redhat.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
mm/khugepaged.c | 28 ++++++++++++++++++++++++----
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index afd61168d915c..2ebbfbd260ec4 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -2335,8 +2335,7 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
}
} else if (folio_test_dirty(folio)) {
/*
- * khugepaged only works on read-only fd,
- * so this page is dirty because it hasn't
+ * This page is dirty because it hasn't
* been flushed since first write. There
* won't be new dirty pages.
*
@@ -2394,8 +2393,8 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
if (!is_shmem && (folio_test_dirty(folio) ||
folio_test_writeback(folio))) {
/*
- * khugepaged only works on read-only fd, so this
- * folio is dirty because it hasn't been flushed
+ * khugepaged only works on clean file-backed folios,
+ * so this folio is dirty because it hasn't been flushed
* since first write.
*/
result = SCAN_PAGE_DIRTY_OR_WRITEBACK;
@@ -2439,6 +2438,27 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
goto out_unlock;
}
+ /*
+ * At this point, the folio is locked and unmapped. If the PTE
+ * was dirty, try_to_unmap() has transferred the dirty bit to
+ * the folio and we must not collapse it into a clean
+ * file-backed folio.
+ *
+ * If the folio is clean here, no one can write it until we
+ * drop the folio lock. A write through a stale TLB entry came
+ * from a clean PTE and must fault because the PTE has been
+ * cleared; the fault path has to take the folio lock before
+ * installing a writable mapping. Buffered write paths also
+ * have to take the folio lock before modifying file contents
+ * without a mapping, typically via write_begin_get_folio().
+ */
+ if (!is_shmem && folio_test_dirty(folio)) {
+ result = SCAN_PAGE_DIRTY_OR_WRITEBACK;
+ xas_unlock_irq(&xas);
+ folio_putback_lru(folio);
+ goto out_unlock;
+ }
+
/*
* Accumulate the folios that are being collapsed.
*/
--
2.53.0Replace it with a check on the max folio order of the file's address space
mapping, making sure PMD folio is supported. Keep the inode
open-for-write check, since even if collapse_file() now makes sure all
to-be-collapsed folios are clean and the created PMD file THP can be
handled by FSes properly, the filemap_flush() could perform undesirable
write back.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Nico Pache <npache@redhat.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
mm/huge_memory.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 3e9eabc74c6c3..ccd623b9501b5 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -86,9 +86,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
{
struct inode *inode;
- if (!IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS))
- return false;
-
if (!vma->vm_file)
return false;
@@ -97,6 +94,9 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
if (IS_ANON_FILE(inode))
return false;
+ if (!mapping_pmd_folio_support(vma->vm_file->f_mapping))
+ return false;
+
return !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
}
--
2.53.0Remove the READ_ONLY_THP_FOR_FS gate and khugepaged for file-backed
pmd-sized hugepages are enabled by the global transparent hugepage
control. khugepaged can still be enabled by per-size control for anon and
shmem when the global control is off.
Add shmem_hpage_pmd_enabled() stub for !CONFIG_SHMEM to remove
IS_ENABLED(SHMEM) in hugepage_enabled().
Clean up hugepage_enabled() by moving anon code to anon_hpage_enabled().
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Nico Pache <npache@redhat.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
include/linux/shmem_fs.h | 2 +-
mm/khugepaged.c | 26 ++++++++++++++++----------
2 files changed, 17 insertions(+), 11 deletions(-)
diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h
index 93a0ba872ebe0..acb8dd961b45c 100644
--- a/include/linux/shmem_fs.h
+++ b/include/linux/shmem_fs.h
@@ -127,7 +127,7 @@ int shmem_writeout(struct folio *folio, struct swap_iocb **plug,
void shmem_truncate_range(struct inode *inode, loff_t start, uoff_t end);
int shmem_unuse(unsigned int type);
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && defined(CONFIG_SHMEM)
unsigned long shmem_allowable_huge_orders(struct inode *inode,
struct vm_area_struct *vma, pgoff_t index,
loff_t write_end, bool shmem_huge_force);
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 2ebbfbd260ec4..edb5c3656c168 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -522,26 +522,32 @@ static inline int collapse_test_exit_or_disable(struct mm_struct *mm)
mm_flags_test(MMF_DISABLE_THP_COMPLETELY, mm);
}
+static inline bool anon_hpage_enabled(void)
+{
+ if (READ_ONCE(huge_anon_orders_always))
+ return true;
+ if (READ_ONCE(huge_anon_orders_madvise))
+ return true;
+ if (READ_ONCE(huge_anon_orders_inherit) &&
+ hugepage_global_enabled())
+ return true;
+ return false;
+}
+
static bool hugepage_enabled(void)
{
/*
* We cover the anon, shmem and the file-backed case here; file-backed
- * hugepages, when configured in, are determined by the global control.
+ * hugepages are determined by the global control.
* Anon hugepages are determined by its per-size mTHP control.
* Shmem pmd-sized hugepages are also determined by its pmd-size control,
* except when the global shmem_huge is set to SHMEM_HUGE_DENY.
*/
- if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) &&
- hugepage_global_enabled())
- return true;
- if (READ_ONCE(huge_anon_orders_always))
+ if (hugepage_global_enabled())
return true;
- if (READ_ONCE(huge_anon_orders_madvise))
- return true;
- if (READ_ONCE(huge_anon_orders_inherit) &&
- hugepage_global_enabled())
+ if (anon_hpage_enabled())
return true;
- if (IS_ENABLED(CONFIG_SHMEM) && shmem_hpage_pmd_enabled())
+ if (shmem_hpage_pmd_enabled())
return true;
return false;
}
--
2.53.0After removing READ_ONLY_THP_FOR_FS check in file_thp_enabled(),
khugepaged and MADV_COLLAPSE can run on FSes with PMD THP pagecache
support even without READ_ONLY_THP_FOR_FS enabled. Remove the Kconfig
first so that no one can use READ_ONLY_THP_FOR_FS as upcoming commits
remove mapping->nr_thps, which its safe guard mechanism relies on.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Nico Pache <npache@redhat.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
mm/Kconfig | 11 -----------
1 file changed, 11 deletions(-)
diff --git a/mm/Kconfig b/mm/Kconfig
index e221fa1dc54d0..27dc5b0139ba6 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -936,17 +936,6 @@ config THP_SWAP
For selection by architectures with reasonable THP sizes.
-config READ_ONLY_THP_FOR_FS
- bool "Read-only THP for filesystems (EXPERIMENTAL)"
- depends on TRANSPARENT_HUGEPAGE
-
- help
- Allow khugepaged to put read-only file-backed pages in THP.
-
- This is marked experimental because it is a new feature. Write
- support of file THPs will be developed in the next few release
- cycles.
-
config NO_PAGE_MAPCOUNT
bool "No per-page mapcount (EXPERIMENTAL)"
help
--
2.53.0They are used by READ_ONLY_THP_FOR_FS to handle writes to FSes without
large folio support, so that read-only THPs created in these FSes are not
seen by the FSes when the underlying fd becomes writable. Now read-only
PMD THPs only appear in a FS with large folio support and the supported
orders include PMD_ORDER.
READ_ONLY_THP_FOR_FS was using mapping->nr_thps, inode->i_writecount, and
smp_mb() to prevent writes to a read-only THP and collapsing writable
folios into a THP. In collapse_file(), mapping->nr_thps is increased,
then smp_mb(), and if inode->i_writecount > 0, collapse is stopped, while
do_dentry_open() first increases inode->i_writecount, then a full memory
fence, and if mapping->nr_thps > 0, all read-only THPs are truncated.
Now this mechanism can be removed along with READ_ONLY_THP_FOR_FS code,
since a dirty folio check has been added after try_to_unmap() in
collapse_file() to prevent dirty folios from being collapsed as clean.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
fs/open.c | 27 ---------------------------
include/linux/pagemap.h | 29 -----------------------------
mm/filemap.c | 1 -
mm/huge_memory.c | 1 -
mm/khugepaged.c | 28 ----------------------------
5 files changed, 86 deletions(-)
diff --git a/fs/open.c b/fs/open.c
index 681d405bc61eb..c321b80027f13 100644
--- a/fs/open.c
+++ b/fs/open.c
@@ -968,33 +968,6 @@ static int do_dentry_open(struct file *f,
if ((f->f_flags & O_DIRECT) && !(f->f_mode & FMODE_CAN_ODIRECT))
return -EINVAL;
- /*
- * XXX: Huge page cache doesn't support writing yet. Drop all page
- * cache for this file before processing writes.
- */
- if (f->f_mode & FMODE_WRITE) {
- /*
- * Depends on full fence from get_write_access() to synchronize
- * against collapse_file() regarding i_writecount and nr_thps
- * updates. Ensures subsequent insertion of THPs into the page
- * cache will fail.
- */
- if (filemap_nr_thps(inode->i_mapping)) {
- struct address_space *mapping = inode->i_mapping;
-
- filemap_invalidate_lock(inode->i_mapping);
- /*
- * unmap_mapping_range just need to be called once
- * here, because the private pages is not need to be
- * unmapped mapping (e.g. data segment of dynamic
- * shared libraries here).
- */
- unmap_mapping_range(mapping, 0, 0, 0);
- truncate_inode_pages(mapping, 0);
- filemap_invalidate_unlock(inode->i_mapping);
- }
- }
-
return 0;
cleanup_all:
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 308d846531d03..627771e82eb16 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -546,35 +546,6 @@ static inline size_t mapping_max_folio_size(const struct address_space *mapping)
return PAGE_SIZE << mapping_max_folio_order(mapping);
}
-static inline int filemap_nr_thps(const struct address_space *mapping)
-{
-#ifdef CONFIG_READ_ONLY_THP_FOR_FS
- return atomic_read(&mapping->nr_thps);
-#else
- return 0;
-#endif
-}
-
-static inline void filemap_nr_thps_inc(struct address_space *mapping)
-{
-#ifdef CONFIG_READ_ONLY_THP_FOR_FS
- if (!mapping_large_folio_support(mapping))
- atomic_inc(&mapping->nr_thps);
-#else
- WARN_ON_ONCE(mapping_large_folio_support(mapping) == 0);
-#endif
-}
-
-static inline void filemap_nr_thps_dec(struct address_space *mapping)
-{
-#ifdef CONFIG_READ_ONLY_THP_FOR_FS
- if (!mapping_large_folio_support(mapping))
- atomic_dec(&mapping->nr_thps);
-#else
- WARN_ON_ONCE(mapping_large_folio_support(mapping) == 0);
-#endif
-}
-
struct address_space *folio_mapping(const struct folio *folio);
/**
diff --git a/mm/filemap.c b/mm/filemap.c
index ab34cab2416a4..9a5e23fa6a238 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -189,7 +189,6 @@ static void filemap_unaccount_folio(struct address_space *mapping,
lruvec_stat_mod_folio(folio, NR_SHMEM_THPS, -nr);
} else if (folio_test_pmd_mappable(folio)) {
lruvec_stat_mod_folio(folio, NR_FILE_THPS, -nr);
- filemap_nr_thps_dec(mapping);
}
if (test_bit(AS_KERNEL_FILE, &folio->mapping->flags))
mod_node_page_state(folio_pgdat(folio),
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index ccd623b9501b5..ed49e5d40e8f8 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -3951,7 +3951,6 @@ static int __folio_freeze_and_split_unmapped(struct folio *folio, unsigned int n
} else {
lruvec_stat_mod_folio(folio,
NR_FILE_THPS, -nr);
- filemap_nr_thps_dec(mapping);
}
}
}
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index edb5c3656c168..c743ec41a7b8b 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -2477,21 +2477,6 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
goto xa_unlocked;
}
- if (!is_shmem) {
- filemap_nr_thps_inc(mapping);
- /*
- * Paired with the fence in do_dentry_open() -> get_write_access()
- * to ensure i_writecount is up to date and the update to nr_thps
- * is visible. Ensures the page cache will be truncated if the
- * file is opened writable.
- */
- smp_mb();
- if (inode_is_open_for_write(mapping->host)) {
- result = SCAN_FAIL;
- filemap_nr_thps_dec(mapping);
- }
- }
-
xa_locked:
xas_unlock_irq(&xas);
xa_unlocked:
@@ -2669,19 +2654,6 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
folio_putback_lru(folio);
folio_put(folio);
}
- /*
- * Undo the updates of filemap_nr_thps_inc for non-SHMEM
- * file only. This undo is not needed unless failure is
- * due to SCAN_COPY_MC.
- */
- if (!is_shmem && result == SCAN_COPY_MC) {
- filemap_nr_thps_dec(mapping);
- /*
- * Paired with the fence in do_dentry_open() -> get_write_access()
- * to ensure the update to nr_thps is visible.
- */
- smp_mb();
- }
new_folio->mapping = NULL;
--
2.53.0filemap_nr_thps*() are removed, the related field, address_space->nr_thps,
is no longer needed. Remove it. This shrinks struct address_space by 8
bytes on 64-bit systems which may increase the number of inodes we can
cache.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Nico Pache <npache@redhat.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
fs/inode.c | 3 ---
include/linux/fs.h | 5 -----
2 files changed, 8 deletions(-)
diff --git a/fs/inode.c b/fs/inode.c
index 6a3cbc7dcd28c..d8a6d6266c3c3 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -279,9 +279,6 @@ int inode_init_always_gfp(struct super_block *sb, struct inode *inode, gfp_t gfp
mapping->flags = 0;
mapping->wb_err = 0;
atomic_set(&mapping->i_mmap_writable, 0);
-#ifdef CONFIG_READ_ONLY_THP_FOR_FS
- atomic_set(&mapping->nr_thps, 0);
-#endif
mapping_set_gfp_mask(mapping, GFP_HIGHUSER_MOVABLE);
mapping->writeback_index = 0;
init_rwsem(&mapping->invalidate_lock);
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 11559c513dfbb..bb9cc4f7207c1 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -460,7 +460,6 @@ struct mapping_metadata_bhs {
* memory mappings.
* @gfp_mask: Memory allocation flags to use for allocating pages.
* @i_mmap_writable: Number of VM_SHARED, VM_MAYWRITE mappings.
- * @nr_thps: Number of THPs in the pagecache (non-shmem only).
* @i_mmap: Tree of private and shared mappings.
* @i_mmap_rwsem: Protects @i_mmap and @i_mmap_writable.
* @nrpages: Number of page entries, protected by the i_pages lock.
@@ -476,10 +475,6 @@ struct address_space {
struct rw_semaphore invalidate_lock;
gfp_t gfp_mask;
atomic_t i_mmap_writable;
-#ifdef CONFIG_READ_ONLY_THP_FOR_FS
- /* number of thp, only for non-shmem files */
- atomic_t nr_thps;
-#endif
struct rb_root_cached i_mmap;
unsigned long nrpages;
pgoff_t writeback_index;
--
2.53.0Without READ_ONLY_THP_FOR_FS, large file-backed folios cannot be created
by a FS without large folio support. The check is no longer needed.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
mm/huge_memory.c | 30 +++---------------------------
1 file changed, 3 insertions(+), 27 deletions(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index ed49e5d40e8f8..d055f53be8502 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -3846,33 +3846,9 @@ int folio_check_splittable(struct folio *folio, unsigned int new_order,
if (!folio->mapping && !folio_test_anon(folio))
return -EBUSY;
- if (folio_test_anon(folio)) {
- /* order-1 is not supported for anonymous THP. */
- if (new_order == 1)
- return -EINVAL;
- } else if (split_type == SPLIT_TYPE_NON_UNIFORM || new_order) {
- if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) &&
- !mapping_large_folio_support(folio->mapping)) {
- /*
- * We can always split a folio down to a single page
- * (new_order == 0) uniformly.
- *
- * For any other scenario
- * a) uniform split targeting a large folio
- * (new_order > 0)
- * b) any non-uniform split
- * we must confirm that the file system supports large
- * folios.
- *
- * Note that we might still have THPs in such
- * mappings, which is created from khugepaged when
- * CONFIG_READ_ONLY_THP_FOR_FS is enabled. But in that
- * case, the mapping does not actually support large
- * folios properly.
- */
- return -EINVAL;
- }
- }
+ /* order-1 is not supported for anonymous THP. */
+ if (folio_test_anon(folio) && new_order == 1)
+ return -EINVAL;
/*
* swapcache folio could only be split to order 0
--
2.53.0After READ_ONLY_THP_FOR_FS is removed, FS either supports large folio or
not. folio_split() can be used on a FS with large folio support without
worrying about getting a THP on a FS without large folio support.
When READ_ONLY_THP_FOR_FS was present, a PMD large pagecache folio can
appear in a FS without large folio support after khugepaged or
madvise(MADV_COLLAPSE) creates it. During truncate_inode_partial_folio(),
such a PMD large pagecache folio is split and if the FS does not support
large folio, it needs to be split to order-0 ones and could not be split
non uniformly to ones with various orders. try_folio_split_to_order() was
added to handle this situation by checking folio_check_splittable(...,
SPLIT_TYPE_NON_UNIFORM) to detect if the large folio is created due to
READ_ONLY_THP_FOR_FS and the FS does not support large folio. Now
READ_ONLY_THP_FOR_FS is removed, all large pagecache folios are created
with FSes supporting large folio, this function is no longer needed and
all large pagecache folios can be split non uniformly.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
include/linux/huge_mm.h | 25 ++-----------------------
mm/truncate.c | 8 ++++----
2 files changed, 6 insertions(+), 27 deletions(-)
diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index 48496f09909be..127f9e1e7604c 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -394,27 +394,6 @@ static inline int split_huge_page_to_order(struct page *page, unsigned int new_o
return split_huge_page_to_list_to_order(page, NULL, new_order);
}
-/**
- * try_folio_split_to_order() - try to split a @folio at @page to @new_order
- * using non uniform split.
- * @folio: folio to be split
- * @page: split to @new_order at the given page
- * @new_order: the target split order
- *
- * Try to split a @folio at @page using non uniform split to @new_order, if
- * non uniform split is not supported, fall back to uniform split. After-split
- * folios are put back to LRU list. Use min_order_for_split() to get the lower
- * bound of @new_order.
- *
- * Return: 0 - split is successful, otherwise split failed.
- */
-static inline int try_folio_split_to_order(struct folio *folio,
- struct page *page, unsigned int new_order)
-{
- if (folio_check_splittable(folio, new_order, SPLIT_TYPE_NON_UNIFORM))
- return split_huge_page_to_order(&folio->page, new_order);
- return folio_split(folio, new_order, page, NULL);
-}
static inline int split_huge_page(struct page *page)
{
return split_huge_page_to_list_to_order(page, NULL, 0);
@@ -647,8 +626,8 @@ static inline int split_folio_to_list(struct folio *folio, struct list_head *lis
return -EINVAL;
}
-static inline int try_folio_split_to_order(struct folio *folio,
- struct page *page, unsigned int new_order)
+static inline int folio_split(struct folio *folio, unsigned int new_order,
+ struct page *page, struct list_head *list)
{
VM_WARN_ON_ONCE_FOLIO(1, folio);
return -EINVAL;
diff --git a/mm/truncate.c b/mm/truncate.c
index 12cc89f89afcf..b58ba940be474 100644
--- a/mm/truncate.c
+++ b/mm/truncate.c
@@ -177,7 +177,7 @@ int truncate_inode_folio(struct address_space *mapping, struct folio *folio)
return 0;
}
-static int try_folio_split_or_unmap(struct folio *folio, struct page *split_at,
+static int folio_split_or_unmap(struct folio *folio, struct page *split_at,
unsigned long min_order)
{
enum ttu_flags ttu_flags =
@@ -186,7 +186,7 @@ static int try_folio_split_or_unmap(struct folio *folio, struct page *split_at,
TTU_IGNORE_MLOCK;
int ret;
- ret = try_folio_split_to_order(folio, split_at, min_order);
+ ret = folio_split(folio, min_order, split_at, NULL);
/*
* If the split fails, unmap the folio, so it will be refaulted
@@ -252,7 +252,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
min_order = mapping_min_folio_order(folio->mapping);
split_at = folio_page(folio, PAGE_ALIGN_DOWN(offset) / PAGE_SIZE);
- if (!try_folio_split_or_unmap(folio, split_at, min_order)) {
+ if (!folio_split_or_unmap(folio, split_at, min_order)) {
/*
* try to split at offset + length to make sure folios within
* the range can be dropped, especially to avoid memory waste
@@ -279,7 +279,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
/* make sure folio2 is large and does not change its mapping */
if (folio_test_large(folio2) &&
folio2->mapping == folio->mapping)
- try_folio_split_or_unmap(folio2, split_at2, min_order);
+ folio_split_or_unmap(folio2, split_at2, min_order);
folio_unlock(folio2);
out:
--
2.53.0READ_ONLY_THP_FOR_FS is no longer present, remove related comment.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: David Sterba <dsterba@suse.com>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
fs/btrfs/defrag.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/fs/btrfs/defrag.c b/fs/btrfs/defrag.c
index 7e2db5d3a4d4c..a8d49d9ca981c 100644
--- a/fs/btrfs/defrag.c
+++ b/fs/btrfs/defrag.c
@@ -860,9 +860,6 @@ static struct folio *defrag_prepare_one_folio(struct btrfs_inode *inode, pgoff_t
return folio;
/*
- * Since we can defragment files opened read-only, we can encounter
- * transparent huge pages here (see CONFIG_READ_ONLY_THP_FOR_FS).
- *
* The IO for such large folios is not fully tested, thus return
* an error to reject such folios unless it's an experimental build.
*
--
2.53.0Change the requirement to a file system with large folio support and the
supported order needs to include PMD_ORDER.
Also add tests of opening a file with read write permission and populating
folios with writes. Reuse the XFS image from split_huge_page_test.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
tools/testing/selftests/mm/khugepaged.c | 125 +++++++++++++++-------
tools/testing/selftests/mm/run_vmtests.sh | 12 ++-
2 files changed, 96 insertions(+), 41 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
index c8393ca52cab7..7ee6dfe93b3d9 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -49,7 +49,8 @@ struct mem_ops {
const char *name;
};
-static struct mem_ops *file_ops;
+static struct mem_ops *read_only_file_ops;
+static struct mem_ops *read_write_file_ops;
static struct mem_ops *anon_ops;
static struct mem_ops *shmem_ops;
@@ -112,7 +113,8 @@ static void restore_settings(int sig)
static void save_settings(void)
{
printf("Save THP and khugepaged settings...");
- if (file_ops && finfo.type == VMA_FILE)
+ if ((read_only_file_ops || read_write_file_ops) &&
+ finfo.type == VMA_FILE)
thp_set_read_ahead_path(finfo.dev_queue_read_ahead_path);
thp_save_settings();
@@ -364,8 +366,10 @@ static bool anon_check_huge(void *addr, int nr_hpages)
return check_huge_anon(addr, nr_hpages, hpage_pmd_size);
}
-static void *file_setup_area(int nr_hpages)
+static void *file_setup_area_common(int nr_hpages, bool read_only)
{
+ const int open_opt = read_only ? O_RDONLY : O_RDWR;
+ const int mmap_prot = read_only ? PROT_READ : (PROT_READ | PROT_WRITE);
int fd;
void *p;
unsigned long size;
@@ -400,14 +404,14 @@ static void *file_setup_area(int nr_hpages)
munmap(p, size);
success("OK");
- printf("Opening %s read only for collapse...", finfo.path);
- finfo.fd = open(finfo.path, O_RDONLY, 777);
+ printf("Opening %s %s for collapse...", finfo.path,
+ read_only ? "read-only" : "read-write");
+ finfo.fd = open(finfo.path, open_opt, 777);
if (finfo.fd < 0) {
perror("open()");
exit(EXIT_FAILURE);
}
- p = mmap(BASE_ADDR, size, PROT_READ,
- MAP_PRIVATE, finfo.fd, 0);
+ p = mmap(BASE_ADDR, size, mmap_prot, MAP_SHARED, finfo.fd, 0);
if (p == MAP_FAILED || p != BASE_ADDR) {
perror("mmap()");
exit(EXIT_FAILURE);
@@ -419,6 +423,16 @@ static void *file_setup_area(int nr_hpages)
return p;
}
+static void *file_setup_read_only_area(int nr_hpages)
+{
+ return file_setup_area_common(nr_hpages, /* read_only= */ true);
+}
+
+static void *file_setup_read_write_area(int nr_hpages)
+{
+ return file_setup_area_common(nr_hpages, /* read_only= */ false);
+}
+
static void file_cleanup_area(void *p, unsigned long size)
{
munmap(p, size);
@@ -426,10 +440,18 @@ static void file_cleanup_area(void *p, unsigned long size)
unlink(finfo.path);
}
-static void file_fault(void *p, unsigned long start, unsigned long end)
+static void file_fault_read(void *p, unsigned long start, unsigned long end)
{
if (madvise(((char *)p) + start, end - start, MADV_POPULATE_READ)) {
- perror("madvise(MADV_POPULATE_READ");
+ perror("madvise(MADV_POPULATE_READ)");
+ exit(EXIT_FAILURE);
+ }
+}
+
+static void file_fault_write(void *p, unsigned long start, unsigned long end)
+{
+ if (madvise(((char *)p) + start, end - start, MADV_POPULATE_WRITE)) {
+ perror("madvise(MADV_POPULATE_WRITE)");
exit(EXIT_FAILURE);
}
}
@@ -489,10 +511,18 @@ static struct mem_ops __anon_ops = {
.name = "anon",
};
-static struct mem_ops __file_ops = {
- .setup_area = &file_setup_area,
+static struct mem_ops __read_only_file_ops = {
+ .setup_area = &file_setup_read_only_area,
.cleanup_area = &file_cleanup_area,
- .fault = &file_fault,
+ .fault = &file_fault_read,
+ .check_huge = &file_check_huge,
+ .name = "file",
+};
+
+static struct mem_ops __read_write_file_ops = {
+ .setup_area = &file_setup_read_write_area,
+ .cleanup_area = &file_cleanup_area,
+ .fault = &file_fault_write,
.check_huge = &file_check_huge,
.name = "file",
};
@@ -505,6 +535,17 @@ static struct mem_ops __shmem_ops = {
.name = "shmem",
};
+static bool is_tmpfs(struct mem_ops *ops)
+{
+ return (ops == &__read_only_file_ops || ops == &__read_write_file_ops) &&
+ finfo.type == VMA_SHMEM;
+}
+
+static bool is_anon(struct mem_ops *ops)
+{
+ return ops == &__anon_ops;
+}
+
static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
struct mem_ops *ops, bool expect)
{
@@ -513,6 +554,10 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
printf("%s...", msg);
+ /* read&write file collapse always fail */
+ if (!is_tmpfs(ops) && ops == &__read_write_file_ops)
+ expect = false;
+
/*
* Prevent khugepaged interference and tests that MADV_COLLAPSE
* ignores /sys/kernel/mm/transparent_hugepage/enabled
@@ -579,6 +624,10 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,
static void khugepaged_collapse(const char *msg, char *p, int nr_hpages,
struct mem_ops *ops, bool expect)
{
+ /* read&write file collapse always fail */
+ if (!is_tmpfs(ops) && ops == &__read_write_file_ops)
+ expect = false;
+
if (wait_for_scan(msg, p, nr_hpages, ops)) {
if (expect)
fail("Timeout");
@@ -613,16 +662,6 @@ static struct collapse_context __madvise_context = {
.name = "madvise",
};
-static bool is_tmpfs(struct mem_ops *ops)
-{
- return ops == &__file_ops && finfo.type == VMA_SHMEM;
-}
-
-static bool is_anon(struct mem_ops *ops)
-{
- return ops == &__anon_ops;
-}
-
static void alloc_at_fault(void)
{
struct thp_settings settings = *thp_current_settings();
@@ -1098,8 +1137,8 @@ static void usage(void)
fprintf(stderr, "\t<context>\t: [all|khugepaged|madvise]\n");
fprintf(stderr, "\t<mem_type>\t: [all|anon|file|shmem]\n");
fprintf(stderr, "\n\t\"file,all\" mem_type requires [dir] argument\n");
- fprintf(stderr, "\n\t\"file,all\" mem_type requires kernel built with\n");
- fprintf(stderr, "\tCONFIG_READ_ONLY_THP_FOR_FS=y\n");
+ fprintf(stderr, "\n\t\"file,all\" mem_type requires a file system\n");
+ fprintf(stderr, "\twith PMD-sized large folio support\n");
fprintf(stderr, "\n\tif [dir] is a (sub)directory of a tmpfs mount, tmpfs must be\n");
fprintf(stderr, "\tmounted with huge=advise option for khugepaged tests to work\n");
fprintf(stderr, "\n\tSupported Options:\n");
@@ -1155,20 +1194,22 @@ static void parse_test_type(int argc, char **argv)
usage();
if (!strcmp(buf, "all")) {
- file_ops = &__file_ops;
+ read_only_file_ops = &__read_only_file_ops;
+ read_write_file_ops = &__read_write_file_ops;
anon_ops = &__anon_ops;
shmem_ops = &__shmem_ops;
} else if (!strcmp(buf, "anon")) {
anon_ops = &__anon_ops;
} else if (!strcmp(buf, "file")) {
- file_ops = &__file_ops;
+ read_only_file_ops = &__read_only_file_ops;
+ read_write_file_ops = &__read_write_file_ops;
} else if (!strcmp(buf, "shmem")) {
shmem_ops = &__shmem_ops;
} else {
usage();
}
- if (!file_ops)
+ if (!read_only_file_ops && !read_write_file_ops)
return;
if (argc != 2)
@@ -1240,37 +1281,43 @@ int main(int argc, char **argv)
} while (0)
TEST(collapse_full, khugepaged_context, anon_ops);
- TEST(collapse_full, khugepaged_context, file_ops);
+ TEST(collapse_full, khugepaged_context, read_only_file_ops);
+ TEST(collapse_full, khugepaged_context, read_write_file_ops);
TEST(collapse_full, khugepaged_context, shmem_ops);
TEST(collapse_full, madvise_context, anon_ops);
- TEST(collapse_full, madvise_context, file_ops);
+ TEST(collapse_full, madvise_context, read_only_file_ops);
+ TEST(collapse_full, madvise_context, read_write_file_ops);
TEST(collapse_full, madvise_context, shmem_ops);
TEST(collapse_empty, khugepaged_context, anon_ops);
TEST(collapse_empty, madvise_context, anon_ops);
TEST(collapse_single_pte_entry, khugepaged_context, anon_ops);
- TEST(collapse_single_pte_entry, khugepaged_context, file_ops);
+ TEST(collapse_single_pte_entry, khugepaged_context, read_only_file_ops);
+ TEST(collapse_single_pte_entry, khugepaged_context, read_write_file_ops);
TEST(collapse_single_pte_entry, khugepaged_context, shmem_ops);
TEST(collapse_single_pte_entry, madvise_context, anon_ops);
- TEST(collapse_single_pte_entry, madvise_context, file_ops);
+ TEST(collapse_single_pte_entry, madvise_context, read_only_file_ops);
+ TEST(collapse_single_pte_entry, madvise_context, read_write_file_ops);
TEST(collapse_single_pte_entry, madvise_context, shmem_ops);
TEST(collapse_max_ptes_none, khugepaged_context, anon_ops);
- TEST(collapse_max_ptes_none, khugepaged_context, file_ops);
+ TEST(collapse_max_ptes_none, khugepaged_context, read_only_file_ops);
+ TEST(collapse_max_ptes_none, khugepaged_context, read_write_file_ops);
TEST(collapse_max_ptes_none, madvise_context, anon_ops);
- TEST(collapse_max_ptes_none, madvise_context, file_ops);
+ TEST(collapse_max_ptes_none, madvise_context, read_only_file_ops);
+ TEST(collapse_max_ptes_none, madvise_context, read_write_file_ops);
TEST(collapse_single_pte_entry_compound, khugepaged_context, anon_ops);
- TEST(collapse_single_pte_entry_compound, khugepaged_context, file_ops);
+ TEST(collapse_single_pte_entry_compound, khugepaged_context, read_only_file_ops);
TEST(collapse_single_pte_entry_compound, madvise_context, anon_ops);
- TEST(collapse_single_pte_entry_compound, madvise_context, file_ops);
+ TEST(collapse_single_pte_entry_compound, madvise_context, read_only_file_ops);
TEST(collapse_full_of_compound, khugepaged_context, anon_ops);
- TEST(collapse_full_of_compound, khugepaged_context, file_ops);
+ TEST(collapse_full_of_compound, khugepaged_context, read_only_file_ops);
TEST(collapse_full_of_compound, khugepaged_context, shmem_ops);
TEST(collapse_full_of_compound, madvise_context, anon_ops);
- TEST(collapse_full_of_compound, madvise_context, file_ops);
+ TEST(collapse_full_of_compound, madvise_context, read_only_file_ops);
TEST(collapse_full_of_compound, madvise_context, shmem_ops);
TEST(collapse_compound_extreme, khugepaged_context, anon_ops);
@@ -1292,10 +1339,10 @@ int main(int argc, char **argv)
TEST(collapse_max_ptes_shared, madvise_context, anon_ops);
TEST(madvise_collapse_existing_thps, madvise_context, anon_ops);
- TEST(madvise_collapse_existing_thps, madvise_context, file_ops);
+ TEST(madvise_collapse_existing_thps, madvise_context, read_only_file_ops);
TEST(madvise_collapse_existing_thps, madvise_context, shmem_ops);
- TEST(madvise_retracted_page_tables, madvise_context, file_ops);
+ TEST(madvise_retracted_page_tables, madvise_context, read_only_file_ops);
TEST(madvise_retracted_page_tables, madvise_context, shmem_ops);
restore_settings(0);
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index 3b61677fe9840..b73921b2cac02 100755
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -490,8 +490,6 @@ CATEGORY="thp" run_test ./khugepaged all:shmem
CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
-CATEGORY="thp" run_test ./transhuge-stress -d 20
-
# Try to create XFS if not provided
if [ -z "${SPLIT_HUGE_PAGE_TEST_XFS_PATH}" ]; then
if [ "${HAVE_HUGEPAGES}" = "1" ]; then
@@ -508,6 +506,14 @@ if [ -z "${SPLIT_HUGE_PAGE_TEST_XFS_PATH}" ]; then
fi
fi
+if [ -n "${SPLIT_HUGE_PAGE_TEST_XFS_PATH}" ]; then
+CATEGORY="thp" run_test ./khugepaged all:file ${SPLIT_HUGE_PAGE_TEST_XFS_PATH}
+elif test_selected thp; then
+ count_total=$(( count_total + 1 ))
+ count_skip=$(( count_skip + 1 ))
+ echo "[SKIP] ./khugepaged all:file" | tap_prefix
+fi
+
CATEGORY="thp" run_test ./split_huge_page_test ${SPLIT_HUGE_PAGE_TEST_XFS_PATH}
if [ -n "${MOUNTED_XFS}" ]; then
@@ -516,6 +522,8 @@ if [ -n "${MOUNTED_XFS}" ]; then
rm -f ${XFS_IMG}
fi
+CATEGORY="thp" run_test ./transhuge-stress -d 20
+
CATEGORY="thp" run_test ./folio_split_race_test
CATEGORY="migration" run_test ./migration
--
2.53.0Any file system with large folio support and the supported orders include
PMD_ORDER can be used. There is no need to open a file with read-only.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
tools/testing/selftests/mm/guard-regions.c | 18 ++++--------------
1 file changed, 4 insertions(+), 14 deletions(-)
diff --git a/tools/testing/selftests/mm/guard-regions.c b/tools/testing/selftests/mm/guard-regions.c
index 48e8b1539be3a..1176398919537 100644
--- a/tools/testing/selftests/mm/guard-regions.c
+++ b/tools/testing/selftests/mm/guard-regions.c
@@ -2203,17 +2203,6 @@ TEST_F(guard_regions, collapse)
if (variant->backing != ANON_BACKED)
ASSERT_EQ(ftruncate(self->fd, size), 0);
- /*
- * We must close and re-open local-file backed as read-only for
- * CONFIG_READ_ONLY_THP_FOR_FS to work.
- */
- if (variant->backing == LOCAL_FILE_BACKED) {
- ASSERT_EQ(close(self->fd), 0);
-
- self->fd = open(self->path, O_RDONLY);
- ASSERT_GE(self->fd, 0);
- }
-
ptr = mmap_(self, variant, NULL, size, PROT_READ, 0, 0);
ASSERT_NE(ptr, MAP_FAILED);
@@ -2237,9 +2226,10 @@ TEST_F(guard_regions, collapse)
/*
* Now collapse the entire region. This should fail in all cases.
*
- * The madvise() call will also fail if CONFIG_READ_ONLY_THP_FOR_FS is
- * not set for the local file case, but we can't differentiate whether
- * this occurred or if the collapse was rightly rejected.
+ * The madvise() call will also fail if the file system does not support
+ * large folio or the supported orders do not include PMD_ORDER for the
+ * local file case, but we can't differentiate whether this occurred or
+ * if the collapse was rightly rejected.
*/
EXPECT_NE(madvise(ptr, size, MADV_COLLAPSE), 0);
--
2.53.0collapse_file() is capable of collapsing pagecache folios from writable
files to PMD folios. Now enable clean pagecache folio collapse in
addition to read-only pagecache folio collapse by removing the
inode_is_open_for_write() from file_thp_enabled() and only performing
filemap_flush() if the file is read-only.
This means userspace needs to explicitly flush the content of pagecache
folios before khugepaged can collapse the folios, or use
madvise(MADV_COLLAPSE), which does the flush in the retry. The reason is
that blindly enabling dirty pagecache folio from writable files collapse
makes khugepaged flush these folios all the time. It is undesirable to
cause system level pagecache flushes.
To properly support dirty pagecache folio collapse, filemap_flush() needs
to be avoided. Potentially, merging associated buffer instead of dropping
it with filemap_release_folio() might be needed.
NOTE: this breaks khugepaged selftests for writable file pagecache
collapse, which is set to fail all the time. The next commit fixes it.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
mm/huge_memory.c | 2 +-
mm/khugepaged.c | 15 +++++++++------
2 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index d055f53be8502..c565b2a651e06 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -97,7 +97,7 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
if (!mapping_pmd_folio_support(vma->vm_file->f_mapping))
return false;
- return !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
+ return S_ISREG(inode->i_mode);
}
/* If returns true, we are unable to access the VMA's folios. */
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index c743ec41a7b8b..395c40c24dbc5 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -2342,18 +2342,21 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
} else if (folio_test_dirty(folio)) {
/*
* This page is dirty because it hasn't
- * been flushed since first write. There
- * won't be new dirty pages.
+ * been flushed since first write.
*
- * Trigger async flush here and hope the
- * writeback is done when khugepaged
- * revisits this page.
+ * Trigger async flush for read-only files and
+ * hope the writeback is done when khugepaged
+ * revisits this page. Writable files can have
+ * their folios dirty at any time; blindly
+ * flushing them would cause undesirable
+ * system-wide writeback.
*
* This is a one-off situation. We are not
* forcing writeback in loop.
*/
xas_unlock_irq(&xas);
- filemap_flush(mapping);
+ if (!inode_is_open_for_write(mapping->host))
+ filemap_flush(mapping);
result = SCAN_PAGE_DIRTY_OR_WRITEBACK;
goto xa_unlocked;
} else if (folio_test_writeback(folio)) {
--
2.53.0collapse_file() now supports collapsing clean pagecache folios from
writable files, so add corresponding tests.
Note that madvise(MADV_COLLAPSE) works for dirty pagecache folios from
writable files, because collapse_single_pmd() triggers a synchronous
writeback when first attempt of collapse_file() fails. That writeback
makes dirty folios clean and the retry of collapse_file() succeeds.
Signed-off-by: Zi Yan <ziy@nvidia.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: David Sterba <dsterba@suse.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
tools/testing/selftests/mm/khugepaged.c | 111 ++++++++++++++++++------
1 file changed, 85 insertions(+), 26 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
index 7ee6dfe93b3d9..9ce0a11b461df 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -41,6 +41,12 @@ enum vma_type {
VMA_SHMEM,
};
+enum file_setup_ops {
+ FILE_SETUP_READ_ONLY_FS,
+ FILE_SETUP_READ_WRITE_FS_READ_DATA,
+ FILE_SETUP_READ_WRITE_FS_WRITE_DATA,
+};
+
struct mem_ops {
void *(*setup_area)(int nr_hpages);
void (*cleanup_area)(void *p, unsigned long size);
@@ -50,7 +56,8 @@ struct mem_ops {
};
static struct mem_ops *read_only_file_ops;
-static struct mem_ops *read_write_file_ops;
+static struct mem_ops *read_write_file_read_ops;
+static struct mem_ops *read_write_file_write_ops;
static struct mem_ops *anon_ops;
static struct mem_ops *shmem_ops;
@@ -113,7 +120,8 @@ static void restore_settings(int sig)
static void save_settings(void)
{
printf("Save THP and khugepaged settings...");
- if ((read_only_file_ops || read_write_file_ops) &&
+ if ((read_only_file_ops || read_write_file_read_ops ||
+ read_write_file_write_ops) &&
finfo.type == VMA_FILE)
thp_set_read_ahead_path(finfo.dev_queue_read_ahead_path);
thp_save_settings();
@@ -366,10 +374,10 @@ static bool anon_check_huge(void *addr, int nr_hpages)
return check_huge_anon(addr, nr_hpages, hpage_pmd_size);
}
-static void *file_setup_area_common(int nr_hpages, bool read_only)
+static void *file_setup_area_common(int nr_hpages, enum file_setup_ops setup)
{
- const int open_opt = read_only ? O_RDONLY : O_RDWR;
- const int mmap_prot = read_only ? PROT_READ : (PROT_READ | PROT_WRITE);
+ const int open_opt = setup == FILE_SETUP_READ_ONLY_FS ? O_RDONLY : O_RDWR;
+ const int mmap_prot = setup == FILE_SETUP_READ_ONLY_FS ? PROT_READ : (PROT_READ | PROT_WRITE);
int fd;
void *p;
unsigned long size;
@@ -405,7 +413,10 @@ static void *file_setup_area_common(int nr_hpages, bool read_only)
success("OK");
printf("Opening %s %s for collapse...", finfo.path,
- read_only ? "read-only" : "read-write");
+ setup == FILE_SETUP_READ_ONLY_FS ? "read-only" :
+ setup == FILE_SETUP_READ_WRITE_FS_READ_DATA ?
+ "read-write (read)" :
+ "read-write (write)");
finfo.fd = open(finfo.path, open_opt, 777);
if (finfo.fd < 0) {
perror("open()");
@@ -425,12 +436,17 @@ static void *file_setup_area_common(int nr_hpages, bool read_only)
static void *file_setup_read_only_area(int nr_hpages)
{
- return file_setup_area_common(nr_hpages, /* read_only= */ true);
+ return file_setup_area_common(nr_hpages, FILE_SETUP_READ_ONLY_FS);
+}
+
+static void *file_setup_read_write_fs_read_area(int nr_hpages)
+{
+ return file_setup_area_common(nr_hpages, FILE_SETUP_READ_WRITE_FS_READ_DATA);
}
-static void *file_setup_read_write_area(int nr_hpages)
+static void *file_setup_read_write_fs_write_area(int nr_hpages)
{
- return file_setup_area_common(nr_hpages, /* read_only= */ false);
+ return file_setup_area_common(nr_hpages, FILE_SETUP_READ_WRITE_FS_WRITE_DATA);
}
static void file_cleanup_area(void *p, unsigned long size)
@@ -448,6 +464,16 @@ static void file_fault_read(void *p, unsigned long start, unsigned long end)
}
}
+static void file_fault_read_and_flush(void *p, unsigned long start, unsigned long end)
+{
+ file_fault_read(p, start, end);
+ /*
+ * make folio clean, since dirty folios from read&write file are
+ * rejected and not flushed
+ */
+ msync((char *)p + start, end - start, MS_SYNC);
+}
+
static void file_fault_write(void *p, unsigned long start, unsigned long end)
{
if (madvise(((char *)p) + start, end - start, MADV_POPULATE_WRITE)) {
@@ -519,8 +545,16 @@ static struct mem_ops __read_only_file_ops = {
.name = "file",
};
-static struct mem_ops __read_write_file_ops = {
- .setup_area = &file_setup_read_write_area,
+static struct mem_ops __read_write_file_read_ops = {
+ .setup_area = &file_setup_read_write_fs_read_area,
+ .cleanup_area = &file_cleanup_area,
+ .fault = &file_fault_read_and_flush,
+ .check_huge = &file_check_huge,
+ .name = "file",
+};
+
+static struct mem_ops __read_write_file_write_ops = {
+ .setup_area = &file_setup_read_write_fs_write_area,
.cleanup_area = &file_cleanup_area,
.fault = &file_fault_write,
.check_huge = &file_check_huge,
@@ -537,7 +571,9 @@ static struct mem_ops __shmem_ops = {
static bool is_tmpfs(struct mem_ops *ops)
{
- return (ops == &__read_only_file_ops || ops == &__read_write_file_ops) &&
+ return (ops == &__read_only_file_ops ||
+ ops == &__read_write_file_read_ops ||
+ ops == &__read_write_file_write_ops) &&
finfo.type == VMA_SHMEM;
}
@@ -554,9 +590,11 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
printf("%s...", msg);
- /* read&write file collapse always fail */
- if (!is_tmpfs(ops) && ops == &__read_write_file_ops)
- expect = false;
+ /*
+ * read&write file collapse succeeds for MADV_COLLAPSE because dirty
+ * folios are written back after collapse fails for dirty folios and
+ * another collapse is attempted.
+ */
/*
* Prevent khugepaged interference and tests that MADV_COLLAPSE
@@ -624,8 +662,11 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,
static void khugepaged_collapse(const char *msg, char *p, int nr_hpages,
struct mem_ops *ops, bool expect)
{
- /* read&write file collapse always fail */
- if (!is_tmpfs(ops) && ops == &__read_write_file_ops)
+ /*
+ * read&write file collapse fails since khugepaged does not flush
+ * the target dirty folios
+ */
+ if (!is_tmpfs(ops) && ops == &__read_write_file_write_ops)
expect = false;
if (wait_for_scan(msg, p, nr_hpages, ops)) {
@@ -748,6 +789,9 @@ static void collapse_max_ptes_none(struct collapse_context *c, struct mem_ops *o
validate_memory(p, 0, (hpage_pmd_nr - max_ptes_none - fault_nr_pages) * page_size);
if (c->enforce_pte_scan_limits) {
+ ops->cleanup_area(p, hpage_pmd_size);
+ p = ops->setup_area(1);
+
ops->fault(p, 0, (hpage_pmd_nr - max_ptes_none) * page_size);
c->collapse("Collapse with max_ptes_none PTEs empty", p, 1, ops,
true);
@@ -1195,21 +1239,24 @@ static void parse_test_type(int argc, char **argv)
if (!strcmp(buf, "all")) {
read_only_file_ops = &__read_only_file_ops;
- read_write_file_ops = &__read_write_file_ops;
+ read_write_file_read_ops = &__read_write_file_read_ops;
+ read_write_file_write_ops = &__read_write_file_write_ops;
anon_ops = &__anon_ops;
shmem_ops = &__shmem_ops;
} else if (!strcmp(buf, "anon")) {
anon_ops = &__anon_ops;
} else if (!strcmp(buf, "file")) {
read_only_file_ops = &__read_only_file_ops;
- read_write_file_ops = &__read_write_file_ops;
+ read_write_file_read_ops = &__read_write_file_read_ops;
+ read_write_file_write_ops = &__read_write_file_write_ops;
} else if (!strcmp(buf, "shmem")) {
shmem_ops = &__shmem_ops;
} else {
usage();
}
- if (!read_only_file_ops && !read_write_file_ops)
+ if (!read_only_file_ops && !read_write_file_read_ops &&
+ !read_write_file_write_ops)
return;
if (argc != 2)
@@ -1282,11 +1329,13 @@ int main(int argc, char **argv)
TEST(collapse_full, khugepaged_context, anon_ops);
TEST(collapse_full, khugepaged_context, read_only_file_ops);
- TEST(collapse_full, khugepaged_context, read_write_file_ops);
+ TEST(collapse_full, khugepaged_context, read_write_file_read_ops);
+ TEST(collapse_full, khugepaged_context, read_write_file_write_ops);
TEST(collapse_full, khugepaged_context, shmem_ops);
TEST(collapse_full, madvise_context, anon_ops);
TEST(collapse_full, madvise_context, read_only_file_ops);
- TEST(collapse_full, madvise_context, read_write_file_ops);
+ TEST(collapse_full, madvise_context, read_write_file_read_ops);
+ TEST(collapse_full, madvise_context, read_write_file_write_ops);
TEST(collapse_full, madvise_context, shmem_ops);
TEST(collapse_empty, khugepaged_context, anon_ops);
@@ -1294,30 +1343,38 @@ int main(int argc, char **argv)
TEST(collapse_single_pte_entry, khugepaged_context, anon_ops);
TEST(collapse_single_pte_entry, khugepaged_context, read_only_file_ops);
- TEST(collapse_single_pte_entry, khugepaged_context, read_write_file_ops);
+ TEST(collapse_single_pte_entry, khugepaged_context, read_write_file_read_ops);
+ TEST(collapse_single_pte_entry, khugepaged_context, read_write_file_write_ops);
TEST(collapse_single_pte_entry, khugepaged_context, shmem_ops);
TEST(collapse_single_pte_entry, madvise_context, anon_ops);
TEST(collapse_single_pte_entry, madvise_context, read_only_file_ops);
- TEST(collapse_single_pte_entry, madvise_context, read_write_file_ops);
+ TEST(collapse_single_pte_entry, madvise_context, read_write_file_read_ops);
+ TEST(collapse_single_pte_entry, madvise_context, read_write_file_write_ops);
TEST(collapse_single_pte_entry, madvise_context, shmem_ops);
TEST(collapse_max_ptes_none, khugepaged_context, anon_ops);
TEST(collapse_max_ptes_none, khugepaged_context, read_only_file_ops);
- TEST(collapse_max_ptes_none, khugepaged_context, read_write_file_ops);
+ TEST(collapse_max_ptes_none, khugepaged_context, read_write_file_read_ops);
+ TEST(collapse_max_ptes_none, khugepaged_context, read_write_file_write_ops);
TEST(collapse_max_ptes_none, madvise_context, anon_ops);
TEST(collapse_max_ptes_none, madvise_context, read_only_file_ops);
- TEST(collapse_max_ptes_none, madvise_context, read_write_file_ops);
+ TEST(collapse_max_ptes_none, madvise_context, read_write_file_read_ops);
+ TEST(collapse_max_ptes_none, madvise_context, read_write_file_write_ops);
TEST(collapse_single_pte_entry_compound, khugepaged_context, anon_ops);
TEST(collapse_single_pte_entry_compound, khugepaged_context, read_only_file_ops);
+ TEST(collapse_single_pte_entry_compound, khugepaged_context, read_write_file_read_ops);
TEST(collapse_single_pte_entry_compound, madvise_context, anon_ops);
TEST(collapse_single_pte_entry_compound, madvise_context, read_only_file_ops);
+ TEST(collapse_single_pte_entry_compound, madvise_context, read_write_file_read_ops);
TEST(collapse_full_of_compound, khugepaged_context, anon_ops);
TEST(collapse_full_of_compound, khugepaged_context, read_only_file_ops);
+ TEST(collapse_full_of_compound, khugepaged_context, read_write_file_read_ops);
TEST(collapse_full_of_compound, khugepaged_context, shmem_ops);
TEST(collapse_full_of_compound, madvise_context, anon_ops);
TEST(collapse_full_of_compound, madvise_context, read_only_file_ops);
+ TEST(collapse_full_of_compound, madvise_context, read_write_file_read_ops);
TEST(collapse_full_of_compound, madvise_context, shmem_ops);
TEST(collapse_compound_extreme, khugepaged_context, anon_ops);
@@ -1340,9 +1397,11 @@ int main(int argc, char **argv)
TEST(madvise_collapse_existing_thps, madvise_context, anon_ops);
TEST(madvise_collapse_existing_thps, madvise_context, read_only_file_ops);
+ TEST(madvise_collapse_existing_thps, madvise_context, read_write_file_read_ops);
TEST(madvise_collapse_existing_thps, madvise_context, shmem_ops);
TEST(madvise_retracted_page_tables, madvise_context, read_only_file_ops);
+ TEST(madvise_retracted_page_tables, madvise_context, read_write_file_read_ops);
TEST(madvise_retracted_page_tables, madvise_context, shmem_ops);
restore_settings(0);
--
2.53.0© 2016 - 2026 Red Hat, Inc.