v1:
https://lore.kernel.org/xen-devel/20260727150615.1373200-1-kevin.lampis@citrix.com/
Previous discussion:
RFC: unmap_page_range optimisation (avoiding emulation faults during VM migration)
https://lore.kernel.org/xen-devel/16133EFF-88FF-467F-B78F-E96EB148C3A5@citrix.com/
This series adds a new MMU_PT_UPDATE_SWAP sub-op to the mmu_update hypercall
which Linux can use from a new pte_get_and_clear pv-op to significantly improve
the performance of unmap_page_range.
A microbenchmark[1] which allocates a large number of pages and then clears
them shows a performance increase from 1100ms to 640ms using the
new pte_get_and_clear pv-op.
Further profiling the microbenchmark with bpftrace[2] shows that Linux calls
unmap_page_range() 23 times with an average execution time of 48ms
dropping to 28ms when using the new operation.
Changes in v2:
- Add a sub-op to mmu_update instead of new hypercall
- Add Linux patch
- Other review comments
[1] microbenchmark
#include <err.h>
#include <sys/mman.h>
#include <time.h>
#include <stdint.h>
#include <stdio.h>
#include <unistd.h>
static uint64_t nsec(void)
{
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (uint64_t)ts.tv_sec * 1e9 + ts.tv_nsec;
}
int main(int argc, char **argv)
{
const size_t len = 1024UL * 1024 * 1024 * 4;
const long pagesz = sysconf(_SC_PAGESIZE);
char *p = mmap(NULL, len,
PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0);
if ( p == MAP_FAILED )
err(1, "mmap");
/* Fault every page in */
for (size_t i = 0; i < len; i += pagesz)
p[i] = 1;
uint64_t start = nsec();
if ( madvise(p, len, MADV_DONTNEED) )
err(1, "madvise");
uint64_t end = nsec();
printf("MADV_DONTNEED on %zu MB took %.3f ms\n",
len / 1024 / 1024,
(end - start) / 1e6);
munmap(p, len);
return 0;
}
[2] bpftrace
bpftrace -e '
kprobe:unmap_page_range
/comm == "a.out"/
{
@start[tid] = nsecs;
}
kretprobe:unmap_page_range
/@start[tid] && comm == "a.out"/
{
$delta = nsecs - @start[tid];
@count = count();
@total = sum($delta);
delete(@start[tid]);
}
interval:s:10
{
printf("avg call time = %d ns (%d calls)\n",
(@total / @count), (uint64)@count);
exit();
}'
Jan Beulich (1):
x86: make UPDATE_ENTRY() allow for multiple operation flags
Kevin Lampis (5):
x86: Remove return value from UPDATE_ENTRY
x86: extend update_intpte() to support atomic get-and-update
x86: extend mod_l1_entry() to optionally return the old PTE value
x86: extend do_mmu_update() to support returning the old PTE value
x86: New feature flag XENFEAT_mmu_pt_update_swap.
xen: Add new Xen pv-op pte_get_and_clear.
xen/arch/x86/mm.c | 161 +++++++++++++++++++-------------
xen/arch/x86/pv/grant_table.c | 53 +++++------
xen/arch/x86/pv/mm.h | 27 +++---
xen/arch/x86/pv/ro-page-fault.c | 3 +-
xen/common/kernel.c | 3 +-
xen/include/public/features.h | 3 +
xen/include/public/xen.h | 1 +
7 files changed, 140 insertions(+), 111 deletions(-)
--
2.52.0