Private Memory NUMA Nodes

[PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Gregory Price 4 days, 5 hours ago

This series introduces the concept of "Private Memory Nodes", which
are opted into two basic functionalities by default:
  - page allocation (mm/page_alloc.c)
  - OOM killing

All other features of mm/ are opted out of managing these NUMA
nodes and its memory.  Then we add capability bits to allow node
owners to opt back into those services (if supported).

NUMA-hosted memory is presently a most-privilege system (any memory on
any node, except ZONE_DEVICE, is 100% fungible and accessible). We
then slap restrictions on top of it (cpuset, mempolicy, ZONE type,
page/folio flags, etc).

The goal here is to flip that dynamic, isolate by default and then
opt-in to specific services that the device says is safe.

Isolation at the NUMA/Zonelist layer provides a powerful mechanism for
memory hosted on accelerators - re-use of the kernel mm/ code.

 - Accelerators (GPUs) can use demotion, numactl, and reclaim.
 - Special memory devices (Compressed RAM) with special access controls
   (promote-on-write) can have generic services written for them.
 - Network devices with large memory regions intended for ring buffers
   can use the buddy and standard networking stack.
 - Slow, disaggregated memory pools which aren't suitable as general
   purpose memory get cleaner interfaces (no need to re-write the buddy
   in userland, can use migration interface, etc).
 - Per-workload dedicated memory nodes (disaggregated VM memory)

And more use cases I have collected over the past few years.

Not included here is a dax-extension [1] that exposes all the internal
bits as userland controls for testing - along with a pile of selftests
that prove correctness.

Changes Since V4
================
 - Massive reduction in complexity.
     - no ops struct
     - no callback functions
     - no __GFP_PRIVATE
     - no __GFP_THISNODE requirement
     - no task flags (no PF_MEMALLOC_* in the alloc path)
 - isolation via a dedicated zonelist:
     - private nodes are omitted from FALLBACK/NOFALLBACK
     - added ZONELIST_PRIVATE(_NOFALLBACK)
 - ALLOC_ZONELIST_PRIVATE alloc_flag
     - on top of Brendan Jackman's recent mm/page_alloc.h work [3]
     - Zonelist selection rides the allocator's alloc_flags
 - rename OPS -> CAPS (capabilities)
 - split base functionality (isolation) from opt-ins (CAPS)
     - first half of series can be merged without CAPS
 - dropped compressed ram example from series
     - will submit separately if this moves forward
 - Added KVM as first primary in-tree user (mempolicy / CAP_USER_NUMA)
 - fully functional dax-kmem extension and huge suite of selftests
   located at my github, to be discussed separately [1]

Patch Layout
============
The series is broken into two sections:

1) N_MEMORY_PRIVATE Introduction.
   Introduce the node state.
   Opt those nodes out of mm/ services.

2) NODE_PRIVATE_CAP_* features
   A set of mm/ service opt-in flags that augment private
   nodes to make them more useful (i.e. reclaim = overcommit).

   NODE_PRIVATE_CAP_LTPIN for private node folio pinning
   NODE_PRIVATE_CAP_NUMA_BALANCING for private-node NUMA balancing
   NODE_PRIVATE_CAP_DEMOTION for private-node tiering demotion
   NODE_PRIVATE_CAP_HOTUNPLUG for opted-in private nodes
   NODE_PRIVATE_CAP_USER_NUMA for userland numa controls
   NODE_PRIVATE_CAP_RECLAIM for opted-in private node reclaim

My hope is to merge at least #1 pulled ahead while #2 is debated.

Allocation Isolation
====================
page_alloc presently controls whether a node's memory can be allocated
on a given call by 4 things (in order of authority)

  1) ZONELIST membership
     If a node is not in the walked zonelist, it's unreachable.

  2) __GFP_THISNODE
     If this flag is set and the only node in the ZONELIST is the
     singular preferred node (or the local node, for -1)

  3) cpuset.mems membership
     cpuset trims any node in its allowed list

  4) mempolicy nodemask
     the allocator will skip any node not in the nodemask.

Except for #1 (Zonelist membership) there are all kinds of weird corner
conditions in which 2-4 can be completely ignored (interrupt context,
empty set because cpuset doesn't intersect mempolicy, shared vma, ...)

But ZONELIST membership is *absolute*.  If a zone is not in the
zonelist being walked, IT CANNOT BE ALLOCATED FROM. PERIOD.

The existing zonelists are constructed like so:
  ZONELIST_FALLBACK    : All N_MEMORY nodes
  ZONELIST_NOFALLBACK  : A singleton N_MEMORY node

Private node isolation is implemented via ZONELIST isolation:
  ZONELIST_PRIVATE            : The private node + N_MEMORY
  ZONELIST_PRIVATE_NOFALLBACK : The private node alone.

(mirrors exactly the fallback/nofallback for __GFP_THISNODE)

Private nodes:
  1) Never appear in any ZONELIST_FALLBACK
  2) Have an empty ZONELIST_NOFALLBACK
  3) Only appear in their own ZONELIST_PRIVATE(_NOFALLBACK)

1 & 2 mean all existing in-tree callers to page_alloc can NEVER
accidentally allocate from a private node.

An allocation must explicitly ask via a zonelist and a nodemask.

  alloc_flags |= ALLOC_ZONELIST_PRIVATE; /* use ZONELIST_PRIVATE */
  __alloc_pages(..., nodemask);     /* with the private node set */

The page allocator keeps all its original interfaces which only
ever touch the default zonelists - avoiding churn.

Making it accessible via Mempolicy: MPOL_F_PRIVATE and page_alloc
=================================================================
The vast majority of the kernel will never need to know about
ZONELIST_PRIVATE, because we add MPOL_F_PRIVATE to mempolicy.

When a mempolicy has MPOL_F_PRIVATE, the alloc_mpol() interfaces
do the zonelist selection for the source of the allocation.

That really is the whole explanation of the mechanism:

  alloc_flags = mpol_alloc_flags(pol);
  page = __alloc_frozen_pages_noprof(..., alloc_flags);

On mm-new this rides the allocator's existing alloc_flags plumbing:
ALLOC_ZONELIST_PRIVATE is just another alloc_flag, so no new
parameter, enum, or alloc_context change is required.

For modules that want to implement their own special handling, they
get the _private variants for the page allocator.  This lets modules
re-use the buddy instead of rewriting it.

   - alloc_pages_node_private_noprof()
   - folio_alloc_node_private_noprof()

This keeps ALLOC_ flags mm/ internal (these functions add the flags).

Isolating private node folios from kernel services
==================================================
We implement filter points in mm/ to prevent operations on
private node memory.  Where possible, we even re-use existing
filter points from ZONE_DEVICE.

Most filter points are one or two lines of code:

Combining ZONE_DEVICE and N_MEMORY_PRIVATE opt-out spots:
  -     if (folio_is_zone_device(folio))
  +     if (unlikely(folio_is_private_managed(folio)))

Disabling a service:
  +  if (!node_is_private(nid)) {
  +      kswapd_run(nid);
  +      kcompactd_run(nid);
  +  }

Disallowing a uapi interaction:
  +  if (node_state(nid, N_MEMORY_PRIVATE))
  +      return -EINVAL;

In the second half of the series, we replace blanket N_MEMORY_PRIVATE
filters with NODE_PRIVATE_CAP_* filters to opt those nodes into that
interaction if CAP is set.

We abstract this with a nice clean interface to make it really clear
what is happening (nodes have features!)

  -  if (node_state(pgdat->node_id, N_MEMORY_PRIVATE))
  +  if (!node_allows_reclaim(pgdat->node_id))

NODE_PRIVATE_CAP_* features
===========================
This series of commits opts private nodes into various mm/ services.

Capabilities:
  NODE_PRIVATE_CAP_RECLAIM       - direct and kswapd reclaim
  NODE_PRIVATE_CAP_USER_NUMA     - userland numa controls
  NODE_PRIVATE_CAP_DEMOTION      - node is a demotion target
  NODE_PRIVATE_CAP_HOTUNPLUG     - hotunplug may migrate
  NODE_PRIVATE_CAP_NUMA_BALANCING - NUMAB may target node folios
  NODE_PRIVATE_CAP_LTPIN         - Longterm pin operates normally

Some opt-in support is more intensive than others, so these features
are broken out in a way that we can defer them as future work streams.

NODE_PRIVATE_CAP_RECLAIM:
  Enabling reclaim for these nodes is actually surprisingly trivial.

  Without CAP_RECLAIM, when an allocation failure occurs, the system
  will not attempt to swap the memory - and instead will OOM (typically
  whatever task is using the most memory on *that* private node).

  This capability consists of:
    1) enabling kswapd and kcompactd for that node at hotplug time.
    2) formalizing opt-out hooks to node_allows_reclaim() opt-in hooks.
    3) Sets normal watermarks for these nodes.
    4) Allow madvise operations on that node (PAGEOUT).
    5) A small tweak to how LRU decides which zones to visit.

NODE_PRIVATE_CAP_USER_NUMA
  Enables the following userland interfaces to accept the node:
    mbind()
    set_mempolicy()
    set_mempolicy_home_node()
    move_pages()
    migrate_pages()

  example:
     buf = mmap(..., MAP_ANON);
     mbind(buf, ..., {private_node});
     buf[0] = 0xDEADBEEF; /* Page faults onto the private node */

  Later - the KVM example shows how in-kernel mempolicies can
  also be bound by CAP_USER_NUMA.

  Otherwise, that's it - it's just a mempolicy with MPOL_F_PRIVATE.

NODE_PRIVATE_CAP_HOTUNPLUG
  This is simple: allow hotunplug to migrate this nodes folios.

  Some devices may not be able to tolerate unexpected migrations,
  so we prevent hotunplug from engaging in migration by default.

  Some devices may have an mmu_notifier in their driver that can
  manage the migration and subsequent refault.

  CAP_HOTUNPLUG allows memory_hotplug.c to migrate normally.

NODE_PRIVATE_CAP_DEMOTION
  This adds the private node as a valid demotion target, and allows
  reclaim to demote memory from a private node to a demotion target.

  Requires:  NODE_PRIVATE_CAP_RECLAIM

NODE_PRIVATE_CAP_NUMA_BALANCING
  This enables numa balancing to inject prot_none on private node
  folio mappings and promote them when faults are taken.

NODE_PRIVATE_CAP_LTPIN
  This allows GUP Longterm Pinning to operate normally.

  Normally, longterm pinning determines folio eligibility based
  on its ZONE_* membership (among other things).

  ZONE_NORMAL is eligible, while ZONE_MOVABLE folios require
  migration to ZONE_NORMAL before pinning.

  Neither operation is preferable by default on a private node,
  so the base behavior of FOLL_LONGTERM is to FAIL.

  This capability allows longterm pinning to operate normally
  based on the ZONE membership.  Private node memory may be
  hotplugged as either ZONE_NORMAL or ZONE_MOVABLE.

In-tree User: KVM
=================
Dave Jiang proposed [2] dax-backed guest_memfd() memory as a way of
enabling disaggregated memory pools to host dedicated KVM memory.

With private nodes, this is trivial (with a bit of basic plumbing):

static int kvm_gmem_bind_node(struct inode *inode, int node)
{
...
  /* Bind to a private node - gated on CAP_USER_NUMA */
  pol = mpol_bind_node(node);
  if (IS_ERR(pol))
    return PTR_ERR(pol);

  /* Set the shared policy */
  err = mpol_set_shared_policy_range(&GMEM_I(inode)->policy, ..., pol);
...
}

KVM doesn't even need to know about private nodes at all, all it
does is ask mempolicy whether the requested node is a valid bind.

mm/ component testing with dax driver
=====================================
The dax driver extensions[1] implements a simple interface to create
a private node from a dax device created by any source.

I left the dax driver extensions out of this feature set because
it locks in the CAP_ bits before anyone has input.  It's there
primarily for testing at this point.

The simplest way to get a dax device is with the memmap= boot arg.
   e.g.: "memmap=0x40000000!0x140000000"

The dax driver extension has the following sysfs entries:
  dax0.0/private         - set the node to private
  dax0.0/dax_file        - make /dev/dax0.0 mmap'able in kmem mode
  dax0.0/adistance       - dictate memory_tierN membership
  dax0.0/reclaim         - CAP_RECLAIM
  dax0.0/demotion        - CAP_DEMOTION
  dax0.0/user_numa       - CAP_USER_NUMA
  dax0.0/hotunplug       - CAP_HOTUNPLUG
  dax0.0/numa_balancing  - CAP_NUMA_BALANCING
  dax0.0/ltpin           - CAP_LTPIN

Now consider the following...

Single node reclaim + mbind support:
  echo 1 > dax0.0/private
  echo 1 > dax0.0/reclaim
  echo 1 > dax0.0/user_numa
  echo online_movable > dax0.0/state

Test program:
   /* node1: 1GB Private Memory Node, 4GB swap */
   buf = mmap(NULL, TWO_GB, PROT_READ | PROT_WRITE,
              MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
   sys_mbind(p, len, MPOL_BIND, mask, MAXNODE, MPOL_MF_STRICT);
   memset(buf, 0xff, TWO_GB);

We can *guarantee* the ONLY reclaiming tasks are exactly:
  - kswapdN (in theory we can even make this optional!)
  - the memset task faulting pages in

This also means these are the only tasks capable of becoming
locked up and OOMing (as long as it is not under pressure).

The rest of the system remains entirely functional for debugging.

It now becomes possible to micro-benchmark and A/B test reclaim
changes with different scenarios (number of tasks, amount of
memory, watermark targets, etc), because we have hard controls
over exactly which tasks can access that node memory and how.

If we broke CAP_RECLAIM into subflags:
  - CAP_RECLAIM_KSWAPD
  - CAP_RECLAIM_DIRECT
  - CAP_COMPACTION_KCOMPACTD
  - CAP_COMPACTION_DIRECT

We can actually test the efficacy of each of these mechanisms in
isolation to each other - something that is strictly impossible today.

Bonus Configuration: HBM device memory tiering
==============================================
echo 1 > dax0.0/private       # make the node private
echo 0 > dax0.0/adistance     # highest tier
echo 1 > dax0.0/reclaim       # reclaim active
echo 1 > dax0.0/demotion      # may demote from the node
echo 1 > dax0.0/user_numa     # mbind()
echo online_movable > dax0.0/state
echo 1 > numa/demotion_enabled

This is an HBM device which is treated as the top-tier in the
system but for which memory can only enter via explicit mbind().

It can be overcommitted because it can be reclaimed (demotions
go to CPU DRAM, and reclaim can swap from it).

If the HBM is managed by an accelerator (GPU), the mmu_notifier
allows it to know when reclaim is moving memory out to do
device-mmu invalidation prior to migration.

Prereqs, base commit, references
================================
akpm/mm-new - for Brendan Jackman's mm/page_alloc.h work[3]

Prereqs (all already in akpm/mm-new; listed for out-of-tree application):

page_alloc.h split + alloc_flags plumbing this series rides on:
commit e81fae43cd69 ("mm: split out internal page_alloc.h")
commit b4ff3b6d0a1d ("mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG")

dax atomic whole-device hotplug (used by the dax extension [1]):
commit d7aa81b9a919 ("mm/memory: add memory_block_aligned_range() helper")
commit 3b2f402a1754 ("dax/kmem: add sysfs interface for atomic whole-device hotplug")

[1] https://github.com/gourryinverse/linux/tree/scratch/gourry/managed_nodes/dax_private-mm-new
[2] https://lore.kernel.org/all/20260423170219.281618-1-dave.jiang@intel.com/
[3] https://lore.kernel.org/all/20260702-alloc-trylock-v4-0-0af8ff387e80@google.com/

base-commit: c872b70f5d6c742ad34b8e838c92af81c8920b3e

Gregory Price (36):
  mm: refactor find_next_best_node to find_next_best_node_in
  mm/page_alloc: refactor build_node_zonelist() out of build_zonelists()
  mm/page_alloc: let the bulk and folio allocators carry alloc_flags
  numa: introduce N_MEMORY_PRIVATE
  mm: add ZONELIST_PRIVATE(_NOFALLBACK) for N_MEMORY_PRIVATE nodes.
  cpuset: exclude private nodes from cpuset.mems (default-open)
  mm/memory_hotplug: disallow migration-driven private node hotunplug
  mm/mempolicy: skip private node folios when queueing for migration
  mm/migrate: disallow userland driven migration for private nodes
  mm/madvise: disallow madvise operations on private node folios
  mm/compaction: disallow compaction on private nodes
  mm/page_alloc: clear private node watermarks and system reserves
  mm/mempolicy: disallow NUMA Balancing prot_none on private nodes
  mm/damon: skip private node memory in DAMON migration and pageout
  mm/ksm: skip KSM for managed-memory folios
  mm/khugepaged: skip private node folios when trying to collapse.
  mm/vmscan: disallow reclaim of private node memory
  mm/gup: disallow longterm pin of private node folios
  proc: include N_MEMORY_PRIVATE nodes in numa_maps output
  mm/memcontrol: account private-node memory in per-node stats
  proc/kcore: include private-node RAM in the kcore RAM map
  mm/mempolicy: add MPOL_F_PRIVATE and zonelist selection
  mm/mempolicy: apply policy at the kernel zone for private-node binds
  mm/mempolicy: add in-kernel MPOL_BIND interfaces for drivers/services
  mm/memory_hotplug: support N_MEMORY_PRIVATE node hotplug
  mm: add NODE_PRIVATE_CAP_RECLAIM for opted-in private node reclaim
  mm: add NODE_PRIVATE_CAP_USER_NUMA for userland numa controls
  mm: add NODE_PRIVATE_CAP_HOTUNPLUG for opted-in private nodes
  mm: add NODE_PRIVATE_CAP_DEMOTION for private-node tiering demotion
  mm: add NODE_PRIVATE_CAP_NUMA_BALANCING for private-node NUMA
    balancing
  mm: add NODE_PRIVATE_CAP_LTPIN for private node folio pinning
  mm/khugepaged: base private node collapse eligiblity on actor/cap bits
  Documentation/mm: describe private (N_MEMORY_PRIVATE) memory nodes
  mm/mempolicy: add mpol_set_shared_policy_range()
  KVM: guest_memfd: bind backing memory to a NUMA node at creation
  KVM: selftests: add a guest_memfd FLAG_BIND_NODE test

 Documentation/ABI/stable/sysfs-devices-node   |  10 +
 Documentation/mm/index.rst                    |   1 +
 Documentation/mm/numa_private_nodes.rst       | 160 ++++++++++
 drivers/base/node.c                           | 118 +++++++
 drivers/dax/kmem.c                            |   2 +-
 fs/proc/kcore.c                               |   5 +-
 fs/proc/task_mmu.c                            |  10 +-
 include/linux/gfp.h                           |  22 +-
 include/linux/kvm_host.h                      |   3 +
 include/linux/memory_hotplug.h                |   5 +-
 include/linux/mempolicy.h                     |  16 +
 include/linux/mmzone.h                        |  26 +-
 include/linux/node_private.h                  | 262 ++++++++++++++++
 include/linux/nodemask.h                      |   7 +-
 include/uapi/linux/kvm.h                      |   5 +-
 include/uapi/linux/mempolicy.h                |   1 +
 kernel/cgroup/cpuset.c                        |  26 +-
 mm/compaction.c                               |  13 +
 mm/damon/paddr.c                              |   9 +
 mm/gup.c                                      |  28 +-
 mm/huge_memory.c                              |   5 +
 mm/internal.h                                 |  97 +++++-
 mm/khugepaged.c                               |  18 +-
 mm/ksm.c                                      |   8 +-
 mm/madvise.c                                  |   8 +-
 mm/memcontrol-v1.c                            |   8 +-
 mm/memcontrol.c                               |  13 +-
 mm/memory-tiers.c                             |  40 ++-
 mm/memory_hotplug.c                           | 119 +++++++-
 mm/mempolicy.c                                | 288 +++++++++++++++---
 mm/migrate.c                                  |  19 +-
 mm/mm_init.c                                  |   2 +-
 mm/page_alloc.c                               | 189 +++++++++---
 mm/page_alloc.h                               |  34 +++
 mm/vmscan.c                                   |  57 +++-
 tools/testing/selftests/kvm/Makefile.kvm      |   2 +
 .../kvm/guest_memfd_bind_node_test.c          | 213 +++++++++++++
 virt/kvm/guest_memfd.c                        |  39 ++-
 38 files changed, 1728 insertions(+), 160 deletions(-)
 create mode 100644 Documentation/mm/numa_private_nodes.rst
 create mode 100644 include/linux/node_private.h
 create mode 100644 tools/testing/selftests/kvm/guest_memfd_bind_node_test.c

-- 
2.53.0-Meta

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Richard Cheng 2 days, 11 hours ago

On Mon, Jul 20, 2026 at 03:33:54PM +0800, Gregory Price wrote:
> This series introduces the concept of "Private Memory Nodes", which
> are opted into two basic functionalities by default:
>   - page allocation (mm/page_alloc.c)
>   - OOM killing
> 
> All other features of mm/ are opted out of managing these NUMA
> nodes and its memory.  Then we add capability bits to allow node
> owners to opt back into those services (if supported).
> 
> NUMA-hosted memory is presently a most-privilege system (any memory on
> any node, except ZONE_DEVICE, is 100% fungible and accessible). We
> then slap restrictions on top of it (cpuset, mempolicy, ZONE type,
> page/folio flags, etc).
>

Hi Gregory,

I applied your series on mm-next and give a quick review,
not thoroughly, still I have some questions below regarding the design.
 
> The goal here is to flip that dynamic, isolate by default and then
> opt-in to specific services that the device says is safe.
> 
> Isolation at the NUMA/Zonelist layer provides a powerful mechanism for
> memory hosted on accelerators - re-use of the kernel mm/ code.
> 
>  - Accelerators (GPUs) can use demotion, numactl, and reclaim.
>  - Special memory devices (Compressed RAM) with special access controls
>    (promote-on-write) can have generic services written for them.
>  - Network devices with large memory regions intended for ring buffers
>    can use the buddy and standard networking stack.
>  - Slow, disaggregated memory pools which aren't suitable as general
>    purpose memory get cleaner interfaces (no need to re-write the buddy
>    in userland, can use migration interface, etc).
>  - Per-workload dedicated memory nodes (disaggregated VM memory)
> 

For accelerator part, take CXL Type-2 as example, it has its own protocol
CXL.cache, CXL.mem which is the rule they need to obey during memory transaction,
Adding more rules for them since they are NUMA node confused me, I wonder the reason ?

Because NUMA node, at least for me, representing topology/locality rather than
something with ownership or capability or rules.


> And more use cases I have collected over the past few years.
> 
> Not included here is a dax-extension [1] that exposes all the internal
> bits as userland controls for testing - along with a pile of selftests
> that prove correctness.
> 
> Changes Since V4
> ================
>  - Massive reduction in complexity.
>      - no ops struct
>      - no callback functions
>      - no __GFP_PRIVATE
>      - no __GFP_THISNODE requirement
>      - no task flags (no PF_MEMALLOC_* in the alloc path)
>  - isolation via a dedicated zonelist:
>      - private nodes are omitted from FALLBACK/NOFALLBACK
>      - added ZONELIST_PRIVATE(_NOFALLBACK)

I saw the reply in patch 5, so in fact there's not only one
dedicate zonelist, but numerous ?
I raise the question because zonelist was supposed to be a
global, unbypasssable thing in MM design, but now what you
are trying to do is to seperate the whole global list into several
parts ?

Can you explain why in current design you don't consider to support
something like ZONELIST_PRIVATE[n]={0, .. ,n-1} ? that's my imagination
of what a global zonelist should look like, no matter private or non.


>  - ALLOC_ZONELIST_PRIVATE alloc_flag
>      - on top of Brendan Jackman's recent mm/page_alloc.h work [3]
>      - Zonelist selection rides the allocator's alloc_flags
>  - rename OPS -> CAPS (capabilities)
>  - split base functionality (isolation) from opt-ins (CAPS)
>      - first half of series can be merged without CAPS
>  - dropped compressed ram example from series
>      - will submit separately if this moves forward
>  - Added KVM as first primary in-tree user (mempolicy / CAP_USER_NUMA)
>  - fully functional dax-kmem extension and huge suite of selftests
>    located at my github, to be discussed separately [1]
> 
> Patch Layout
> ============
> The series is broken into two sections:
> 
> 1) N_MEMORY_PRIVATE Introduction.
>    Introduce the node state.
>    Opt those nodes out of mm/ services.
> 
> 2) NODE_PRIVATE_CAP_* features
>    A set of mm/ service opt-in flags that augment private
>    nodes to make them more useful (i.e. reclaim = overcommit).
> 
>    NODE_PRIVATE_CAP_LTPIN for private node folio pinning
>    NODE_PRIVATE_CAP_NUMA_BALANCING for private-node NUMA balancing
>    NODE_PRIVATE_CAP_DEMOTION for private-node tiering demotion
>    NODE_PRIVATE_CAP_HOTUNPLUG for opted-in private nodes
>    NODE_PRIVATE_CAP_USER_NUMA for userland numa controls
>    NODE_PRIVATE_CAP_RECLAIM for opted-in private node reclaim
> 
> My hope is to merge at least #1 pulled ahead while #2 is debated.
> 
> Allocation Isolation
> ====================
> page_alloc presently controls whether a node's memory can be allocated
> on a given call by 4 things (in order of authority)
> 
>   1) ZONELIST membership
>      If a node is not in the walked zonelist, it's unreachable.
>

So a device gets hotplugged in the system will get a dedicated zonelist here ?
And make sure it obeys the device's own protocol if it has one ?

>   2) __GFP_THISNODE
>      If this flag is set and the only node in the ZONELIST is the
>      singular preferred node (or the local node, for -1)
> 
>   3) cpuset.mems membership
>      cpuset trims any node in its allowed list
> 

Sounds sane to me.

>   4) mempolicy nodemask
>      the allocator will skip any node not in the nodemask.
> 
> Except for #1 (Zonelist membership) there are all kinds of weird corner
> conditions in which 2-4 can be completely ignored (interrupt context,
> empty set because cpuset doesn't intersect mempolicy, shared vma, ...)
> 
> But ZONELIST membership is *absolute*.  If a zone is not in the
> zonelist being walked, IT CANNOT BE ALLOCATED FROM. PERIOD.
> 
> The existing zonelists are constructed like so:
>   ZONELIST_FALLBACK    : All N_MEMORY nodes
>   ZONELIST_NOFALLBACK  : A singleton N_MEMORY node
> 
> Private node isolation is implemented via ZONELIST isolation:
>   ZONELIST_PRIVATE            : The private node + N_MEMORY
>   ZONELIST_PRIVATE_NOFALLBACK : The private node alone.
> 
> (mirrors exactly the fallback/nofallback for __GFP_THISNODE)
> 
> Private nodes:
>   1) Never appear in any ZONELIST_FALLBACK
>   2) Have an empty ZONELIST_NOFALLBACK
>   3) Only appear in their own ZONELIST_PRIVATE(_NOFALLBACK)
> 
> 1 & 2 mean all existing in-tree callers to page_alloc can NEVER
> accidentally allocate from a private node.
> 
> An allocation must explicitly ask via a zonelist and a nodemask.
> 
>   alloc_flags |= ALLOC_ZONELIST_PRIVATE; /* use ZONELIST_PRIVATE */
>   __alloc_pages(..., nodemask);     /* with the private node set */
> 

As I stated above, NUMA concept was quite naive at first glance for my
limited knowledge.

This is quite alot to add for NUMA node concept, I'll want to see
more explanation in v6 and learn from it, thanks.

> The page allocator keeps all its original interfaces which only
> ever touch the default zonelists - avoiding churn.
> 
> Making it accessible via Mempolicy: MPOL_F_PRIVATE and page_alloc
> =================================================================
> The vast majority of the kernel will never need to know about
> ZONELIST_PRIVATE, because we add MPOL_F_PRIVATE to mempolicy.
> 
> When a mempolicy has MPOL_F_PRIVATE, the alloc_mpol() interfaces
> do the zonelist selection for the source of the allocation.
> 
> That really is the whole explanation of the mechanism:
> 
>   alloc_flags = mpol_alloc_flags(pol);
>   page = __alloc_frozen_pages_noprof(..., alloc_flags);
> 
> On mm-new this rides the allocator's existing alloc_flags plumbing:
> ALLOC_ZONELIST_PRIVATE is just another alloc_flag, so no new
> parameter, enum, or alloc_context change is required.
> 
> For modules that want to implement their own special handling, they
> get the _private variants for the page allocator.  This lets modules
> re-use the buddy instead of rewriting it.
> 
>    - alloc_pages_node_private_noprof()
>    - folio_alloc_node_private_noprof()
> 
> This keeps ALLOC_ flags mm/ internal (these functions add the flags).
> 
> Isolating private node folios from kernel services
> ==================================================
> We implement filter points in mm/ to prevent operations on
> private node memory.  Where possible, we even re-use existing
> filter points from ZONE_DEVICE.
> 
> Most filter points are one or two lines of code:
> 
> Combining ZONE_DEVICE and N_MEMORY_PRIVATE opt-out spots:
>   -     if (folio_is_zone_device(folio))
>   +     if (unlikely(folio_is_private_managed(folio)))
> 
> Disabling a service:
>   +  if (!node_is_private(nid)) {
>   +      kswapd_run(nid);
>   +      kcompactd_run(nid);
>   +  }
> 
> Disallowing a uapi interaction:
>   +  if (node_state(nid, N_MEMORY_PRIVATE))
>   +      return -EINVAL;
> 

I'm not sure of why do we re-implement more filter and basically
doing the same thing ? Any unavoidable scenario ?

re-using the exisintg filter would be nice if that's possible.

> In the second half of the series, we replace blanket N_MEMORY_PRIVATE
> filters with NODE_PRIVATE_CAP_* filters to opt those nodes into that
> interaction if CAP is set.
> 
> We abstract this with a nice clean interface to make it really clear
> what is happening (nodes have features!)
> 
>   -  if (node_state(pgdat->node_id, N_MEMORY_PRIVATE))
>   +  if (!node_allows_reclaim(pgdat->node_id))
> 
> NODE_PRIVATE_CAP_* features
> ===========================
> This series of commits opts private nodes into various mm/ services.
> 
> Capabilities:
>   NODE_PRIVATE_CAP_RECLAIM       - direct and kswapd reclaim
>   NODE_PRIVATE_CAP_USER_NUMA     - userland numa controls
>   NODE_PRIVATE_CAP_DEMOTION      - node is a demotion target
>   NODE_PRIVATE_CAP_HOTUNPLUG     - hotunplug may migrate
>   NODE_PRIVATE_CAP_NUMA_BALANCING - NUMAB may target node folios
>   NODE_PRIVATE_CAP_LTPIN         - Longterm pin operates normally
> 
> Some opt-in support is more intensive than others, so these features
> are broken out in a way that we can defer them as future work streams.
> 
> NODE_PRIVATE_CAP_RECLAIM:
>   Enabling reclaim for these nodes is actually surprisingly trivial.
> 
>   Without CAP_RECLAIM, when an allocation failure occurs, the system
>   will not attempt to swap the memory - and instead will OOM (typically
>   whatever task is using the most memory on *that* private node).
> 
>   This capability consists of:
>     1) enabling kswapd and kcompactd for that node at hotplug time.
>     2) formalizing opt-out hooks to node_allows_reclaim() opt-in hooks.
>     3) Sets normal watermarks for these nodes.
>     4) Allow madvise operations on that node (PAGEOUT).
>     5) A small tweak to how LRU decides which zones to visit.
> 
> NODE_PRIVATE_CAP_USER_NUMA
>   Enables the following userland interfaces to accept the node:
>     mbind()
>     set_mempolicy()
>     set_mempolicy_home_node()
>     move_pages()
>     migrate_pages()
> 
>   example:
>      buf = mmap(..., MAP_ANON);
>      mbind(buf, ..., {private_node});
>      buf[0] = 0xDEADBEEF; /* Page faults onto the private node */
> 
>   Later - the KVM example shows how in-kernel mempolicies can
>   also be bound by CAP_USER_NUMA.
> 
>   Otherwise, that's it - it's just a mempolicy with MPOL_F_PRIVATE.
> 
> NODE_PRIVATE_CAP_HOTUNPLUG
>   This is simple: allow hotunplug to migrate this nodes folios.
> 
>   Some devices may not be able to tolerate unexpected migrations,
>   so we prevent hotunplug from engaging in migration by default.
> 
>   Some devices may have an mmu_notifier in their driver that can
>   manage the migration and subsequent refault.
> 
>   CAP_HOTUNPLUG allows memory_hotplug.c to migrate normally.
> 
> NODE_PRIVATE_CAP_DEMOTION
>   This adds the private node as a valid demotion target, and allows
>   reclaim to demote memory from a private node to a demotion target.
> 
>   Requires:  NODE_PRIVATE_CAP_RECLAIM
> 
> NODE_PRIVATE_CAP_NUMA_BALANCING
>   This enables numa balancing to inject prot_none on private node
>   folio mappings and promote them when faults are taken.
> 
> NODE_PRIVATE_CAP_LTPIN
>   This allows GUP Longterm Pinning to operate normally.
> 
>   Normally, longterm pinning determines folio eligibility based
>   on its ZONE_* membership (among other things).
> 
>   ZONE_NORMAL is eligible, while ZONE_MOVABLE folios require
>   migration to ZONE_NORMAL before pinning.
> 
>   Neither operation is preferable by default on a private node,
>   so the base behavior of FOLL_LONGTERM is to FAIL.
> 
>   This capability allows longterm pinning to operate normally
>   based on the ZONE membership.  Private node memory may be
>   hotplugged as either ZONE_NORMAL or ZONE_MOVABLE.
> 
> In-tree User: KVM
> =================
> Dave Jiang proposed [2] dax-backed guest_memfd() memory as a way of
> enabling disaggregated memory pools to host dedicated KVM memory.
> 
> With private nodes, this is trivial (with a bit of basic plumbing):
> 
> static int kvm_gmem_bind_node(struct inode *inode, int node)
> {
> ...
>   /* Bind to a private node - gated on CAP_USER_NUMA */
>   pol = mpol_bind_node(node);
>   if (IS_ERR(pol))
>     return PTR_ERR(pol);
> 
>   /* Set the shared policy */
>   err = mpol_set_shared_policy_range(&GMEM_I(inode)->policy, ..., pol);
> ...
> }
> 
> KVM doesn't even need to know about private nodes at all, all it
> does is ask mempolicy whether the requested node is a valid bind.
> 
> mm/ component testing with dax driver
> =====================================
> The dax driver extensions[1] implements a simple interface to create
> a private node from a dax device created by any source.
> 
> I left the dax driver extensions out of this feature set because
> it locks in the CAP_ bits before anyone has input.  It's there
> primarily for testing at this point.
> 
> The simplest way to get a dax device is with the memmap= boot arg.
>    e.g.: "memmap=0x40000000!0x140000000"
> 
> The dax driver extension has the following sysfs entries:
>   dax0.0/private         - set the node to private
>   dax0.0/dax_file        - make /dev/dax0.0 mmap'able in kmem mode
>   dax0.0/adistance       - dictate memory_tierN membership
>   dax0.0/reclaim         - CAP_RECLAIM
>   dax0.0/demotion        - CAP_DEMOTION
>   dax0.0/user_numa       - CAP_USER_NUMA
>   dax0.0/hotunplug       - CAP_HOTUNPLUG
>   dax0.0/numa_balancing  - CAP_NUMA_BALANCING
>   dax0.0/ltpin           - CAP_LTPIN
> 
> Now consider the following...
> 
> Single node reclaim + mbind support:
>   echo 1 > dax0.0/private
>   echo 1 > dax0.0/reclaim
>   echo 1 > dax0.0/user_numa
>   echo online_movable > dax0.0/state
> 
> Test program:
>    /* node1: 1GB Private Memory Node, 4GB swap */
>    buf = mmap(NULL, TWO_GB, PROT_READ | PROT_WRITE,
>               MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>    sys_mbind(p, len, MPOL_BIND, mask, MAXNODE, MPOL_MF_STRICT);
>    memset(buf, 0xff, TWO_GB);
> 
> We can *guarantee* the ONLY reclaiming tasks are exactly:
>   - kswapdN (in theory we can even make this optional!)
>   - the memset task faulting pages in
> 
> This also means these are the only tasks capable of becoming
> locked up and OOMing (as long as it is not under pressure).
> 
> The rest of the system remains entirely functional for debugging.
> 
> It now becomes possible to micro-benchmark and A/B test reclaim
> changes with different scenarios (number of tasks, amount of
> memory, watermark targets, etc), because we have hard controls
> over exactly which tasks can access that node memory and how.
> 
> If we broke CAP_RECLAIM into subflags:
>   - CAP_RECLAIM_KSWAPD
>   - CAP_RECLAIM_DIRECT
>   - CAP_COMPACTION_KCOMPACTD
>   - CAP_COMPACTION_DIRECT
> 
> We can actually test the efficacy of each of these mechanisms in
> isolation to each other - something that is strictly impossible today.
> 
> Bonus Configuration: HBM device memory tiering
> ==============================================
> echo 1 > dax0.0/private       # make the node private
> echo 0 > dax0.0/adistance     # highest tier
> echo 1 > dax0.0/reclaim       # reclaim active
> echo 1 > dax0.0/demotion      # may demote from the node
> echo 1 > dax0.0/user_numa     # mbind()
> echo online_movable > dax0.0/state
> echo 1 > numa/demotion_enabled
> 
> This is an HBM device which is treated as the top-tier in the
> system but for which memory can only enter via explicit mbind().
> 
> It can be overcommitted because it can be reclaimed (demotions
> go to CPU DRAM, and reclaim can swap from it).
> 
> If the HBM is managed by an accelerator (GPU), the mmu_notifier
> allows it to know when reclaim is moving memory out to do
> device-mmu invalidation prior to migration.
> 
> Prereqs, base commit, references
> ================================
> akpm/mm-new - for Brendan Jackman's mm/page_alloc.h work[3]
> 

Still thanks for the work, I learn alot from your work as well, thanks.

Best regards,
Richard Cheng.



> Prereqs (all already in akpm/mm-new; listed for out-of-tree application):
> 
> page_alloc.h split + alloc_flags plumbing this series rides on:
> commit e81fae43cd69 ("mm: split out internal page_alloc.h")
> commit b4ff3b6d0a1d ("mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG")
> 
> dax atomic whole-device hotplug (used by the dax extension [1]):
> commit d7aa81b9a919 ("mm/memory: add memory_block_aligned_range() helper")
> commit 3b2f402a1754 ("dax/kmem: add sysfs interface for atomic whole-device hotplug")
> 
> [1] https://github.com/gourryinverse/linux/tree/scratch/gourry/managed_nodes/dax_private-mm-new
> [2] https://lore.kernel.org/all/20260423170219.281618-1-dave.jiang@intel.com/
> [3] https://lore.kernel.org/all/20260702-alloc-trylock-v4-0-0af8ff387e80@google.com/
> 
> base-commit: c872b70f5d6c742ad34b8e838c92af81c8920b3e
> 
> Gregory Price (36):
>   mm: refactor find_next_best_node to find_next_best_node_in
>   mm/page_alloc: refactor build_node_zonelist() out of build_zonelists()
>   mm/page_alloc: let the bulk and folio allocators carry alloc_flags
>   numa: introduce N_MEMORY_PRIVATE
>   mm: add ZONELIST_PRIVATE(_NOFALLBACK) for N_MEMORY_PRIVATE nodes.
>   cpuset: exclude private nodes from cpuset.mems (default-open)
>   mm/memory_hotplug: disallow migration-driven private node hotunplug
>   mm/mempolicy: skip private node folios when queueing for migration
>   mm/migrate: disallow userland driven migration for private nodes
>   mm/madvise: disallow madvise operations on private node folios
>   mm/compaction: disallow compaction on private nodes
>   mm/page_alloc: clear private node watermarks and system reserves
>   mm/mempolicy: disallow NUMA Balancing prot_none on private nodes
>   mm/damon: skip private node memory in DAMON migration and pageout
>   mm/ksm: skip KSM for managed-memory folios
>   mm/khugepaged: skip private node folios when trying to collapse.
>   mm/vmscan: disallow reclaim of private node memory
>   mm/gup: disallow longterm pin of private node folios
>   proc: include N_MEMORY_PRIVATE nodes in numa_maps output
>   mm/memcontrol: account private-node memory in per-node stats
>   proc/kcore: include private-node RAM in the kcore RAM map
>   mm/mempolicy: add MPOL_F_PRIVATE and zonelist selection
>   mm/mempolicy: apply policy at the kernel zone for private-node binds
>   mm/mempolicy: add in-kernel MPOL_BIND interfaces for drivers/services
>   mm/memory_hotplug: support N_MEMORY_PRIVATE node hotplug
>   mm: add NODE_PRIVATE_CAP_RECLAIM for opted-in private node reclaim
>   mm: add NODE_PRIVATE_CAP_USER_NUMA for userland numa controls
>   mm: add NODE_PRIVATE_CAP_HOTUNPLUG for opted-in private nodes
>   mm: add NODE_PRIVATE_CAP_DEMOTION for private-node tiering demotion
>   mm: add NODE_PRIVATE_CAP_NUMA_BALANCING for private-node NUMA
>     balancing
>   mm: add NODE_PRIVATE_CAP_LTPIN for private node folio pinning
>   mm/khugepaged: base private node collapse eligiblity on actor/cap bits
>   Documentation/mm: describe private (N_MEMORY_PRIVATE) memory nodes
>   mm/mempolicy: add mpol_set_shared_policy_range()
>   KVM: guest_memfd: bind backing memory to a NUMA node at creation
>   KVM: selftests: add a guest_memfd FLAG_BIND_NODE test
> 
>  Documentation/ABI/stable/sysfs-devices-node   |  10 +
>  Documentation/mm/index.rst                    |   1 +
>  Documentation/mm/numa_private_nodes.rst       | 160 ++++++++++
>  drivers/base/node.c                           | 118 +++++++
>  drivers/dax/kmem.c                            |   2 +-
>  fs/proc/kcore.c                               |   5 +-
>  fs/proc/task_mmu.c                            |  10 +-
>  include/linux/gfp.h                           |  22 +-
>  include/linux/kvm_host.h                      |   3 +
>  include/linux/memory_hotplug.h                |   5 +-
>  include/linux/mempolicy.h                     |  16 +
>  include/linux/mmzone.h                        |  26 +-
>  include/linux/node_private.h                  | 262 ++++++++++++++++
>  include/linux/nodemask.h                      |   7 +-
>  include/uapi/linux/kvm.h                      |   5 +-
>  include/uapi/linux/mempolicy.h                |   1 +
>  kernel/cgroup/cpuset.c                        |  26 +-
>  mm/compaction.c                               |  13 +
>  mm/damon/paddr.c                              |   9 +
>  mm/gup.c                                      |  28 +-
>  mm/huge_memory.c                              |   5 +
>  mm/internal.h                                 |  97 +++++-
>  mm/khugepaged.c                               |  18 +-
>  mm/ksm.c                                      |   8 +-
>  mm/madvise.c                                  |   8 +-
>  mm/memcontrol-v1.c                            |   8 +-
>  mm/memcontrol.c                               |  13 +-
>  mm/memory-tiers.c                             |  40 ++-
>  mm/memory_hotplug.c                           | 119 +++++++-
>  mm/mempolicy.c                                | 288 +++++++++++++++---
>  mm/migrate.c                                  |  19 +-
>  mm/mm_init.c                                  |   2 +-
>  mm/page_alloc.c                               | 189 +++++++++---
>  mm/page_alloc.h                               |  34 +++
>  mm/vmscan.c                                   |  57 +++-
>  tools/testing/selftests/kvm/Makefile.kvm      |   2 +
>  .../kvm/guest_memfd_bind_node_test.c          | 213 +++++++++++++
>  virt/kvm/guest_memfd.c                        |  39 ++-
>  38 files changed, 1728 insertions(+), 160 deletions(-)
>  create mode 100644 Documentation/mm/numa_private_nodes.rst
>  create mode 100644 include/linux/node_private.h
>  create mode 100644 tools/testing/selftests/kvm/guest_memfd_bind_node_test.c
> 
> -- 
> 2.53.0-Meta
> 
>

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Gregory Price 2 days, 9 hours ago

On Wed, Jul 22, 2026 at 10:20:00PM +0800, Richard Cheng wrote:
> On Mon, Jul 20, 2026 at 03:33:54PM +0800, Gregory Price wrote:
> Hi Gregory,
> 
> I applied your series on mm-next and give a quick review,
> not thoroughly, still I have some questions below regarding the design.
>  

Hi Richard,

Thank you for the read.

Just a heads up, this was based on mm-new because of some recent work
on Brendan's page_alloc.h cleanup, but if you managed to get it applied
then maybe that's made its way forward already.

> > The goal here is to flip that dynamic, isolate by default and then
> > opt-in to specific services that the device says is safe.
> > 
> > Isolation at the NUMA/Zonelist layer provides a powerful mechanism for
> > memory hosted on accelerators - re-use of the kernel mm/ code.
> > 
> >  - Accelerators (GPUs) can use demotion, numactl, and reclaim.
> >  - Special memory devices (Compressed RAM) with special access controls
> >    (promote-on-write) can have generic services written for them.
> >  - Network devices with large memory regions intended for ring buffers
> >    can use the buddy and standard networking stack.
> >  - Slow, disaggregated memory pools which aren't suitable as general
> >    purpose memory get cleaner interfaces (no need to re-write the buddy
> >    in userland, can use migration interface, etc).
> >  - Per-workload dedicated memory nodes (disaggregated VM memory)
> > 
> 
> For accelerator part, take CXL Type-2 as example, it has its own protocol
> CXL.cache, CXL.mem which is the rule they need to obey during memory transaction,
> Adding more rules for them since they are NUMA node confused me, I wonder the reason ?
> 

CXL protocols simply state how to do the memory transaction at a
physical / transport level.  It does not make any statement on how the
operating systems are to make sense of these devices or what constructs
/ abstractions to build to actually manage the devices themselves.

This series is not necessarily attached to CXL in particular, you could
just as easily carve out memory from the general pool onto a private
node to ensure it only gets used for a particular use.

In fact, that is how I have been testing this with dax/kmem:
https://github.com/gourryinverse/linux/commit/f279e741d9c3643597525bfab916b29b95cb635b

> Because NUMA node, at least for me, representing topology/locality rather than
> something with ownership or capability or rules.
> 

The NUMA abstraction representing topology/locality is a construct we
(the OS developers) have decided on historically - but there's nothing
that dictates we can never create new useful abstractions with it.

Consider:
    N_NORMAL_MEMORY,        /* The node has regular memory */
    N_HIGH_MEMORY,          /* The node has regular or high memory */

These have nothing to do with topology or locality, they are node states
that only have meaning in the context of linux mm/.

To make something clear - there is no *requirement* for any particular
device to use this abstraction.  It simply enables a cleaner way for
devices carrying memory to re-utilize mm/ services while ensuring their
memory does not silently get used under system pressure (or some vagrant
in userspace doing `numactl --interleave all`).

Ignoring all the CAP bits entirely, if you just took the base series
your driver could re-use the buddy allocator without any special logic
AND have confidence that your driver is the only possible user of that
memory (barring some truly obscene bug).

> >  - isolation via a dedicated zonelist:
> >      - private nodes are omitted from FALLBACK/NOFALLBACK
> >      - added ZONELIST_PRIVATE(_NOFALLBACK)
> 
> I saw the reply in patch 5, so in fact there's not only one
> dedicate zonelist, but numerous ?
> I raise the question because zonelist was supposed to be a
> global, unbypasssable thing in MM design, but now what you
> are trying to do is to seperate the whole global list into several
> parts ?
> 
> Can you explain why in current design you don't consider to support
> something like ZONELIST_PRIVATE[n]={0, .. ,n-1} ? that's my imagination
> of what a global zonelist should look like, no matter private or non.
> 

First let me say that there's nothing that prevents us from doing this,
and we *could* make this the default case - but this decision was
intentional by me.

Having them all present in each-other's zonelists by default creates a
number of implicit opt-ins that are unclear:

  - any direct zonelist iterator now iterates all zones on all private
    nodes, even those nodes do not opt into the same services

    e.g. zonelist iteration in reclaim that targets Private Node A would
    attempt to reclaim Private Node B as well.  That would require and
    extra explicit filter.

    I ask: Why do this? Just isolate in the zonelist, and if there is
           a desire for intersections - make it explicit, not implicit.

  - fallback allocations can now occur across private nodes, even
    if those nodes are intended for different purposes.

    Obviously you can use nodemask to tighten the allocation target, but
    I use `numactl --interleave --all` as an example of a clear case
    where intersected nodelists may not give you the behavior you want.

So for consistency - everything including zonelists have full isolation.
This way all interactions with a private node must be explicit - always.

This is actually one of the problems with ZONE_DEVICE - and you can see
it in this patch set.  Some of the hooks in mm/ for zone_device only
apply to PTE cases, and are absent from PMD cases - only because PMD
mappings in ZONE_DEVICE aren't supported.

That kind of implicit behavior is quite bad.

That said, future improvements could include something like:

for_reclaimable_zone(ZONELIST_PRIVATE) {}

where we loosen this isolation, and formalize a filter, but I would
like to see the usecase for it first.  Loosening the isolation defeats
the entire purpose of the series, so there should be a strong reason to
do so.

> > Allocation Isolation
> > ====================
> > page_alloc presently controls whether a node's memory can be allocated
> > on a given call by 4 things (in order of authority)
> > 
> >   1) ZONELIST membership
> >      If a node is not in the walked zonelist, it's unreachable.
> >
> 
> So a device gets hotplugged in the system will get a dedicated zonelist here ?

Yes.

> And make sure it obeys the device's own protocol if it has one ?
> 

I'm not sure i follow this question, can you help me understand?

If by protocol you mean the CAP bits (opting into reclaim, demotion,
etc), then yes.  If you mean something else (CXL) then I think that's
orthogonal and unrelated.

> > Private nodes:
> >   1) Never appear in any ZONELIST_FALLBACK
> >   2) Have an empty ZONELIST_NOFALLBACK
> >   3) Only appear in their own ZONELIST_PRIVATE(_NOFALLBACK)
> > 
> > 1 & 2 mean all existing in-tree callers to page_alloc can NEVER
> > accidentally allocate from a private node.
> > 
> > An allocation must explicitly ask via a zonelist and a nodemask.
> > 
> >   alloc_flags |= ALLOC_ZONELIST_PRIVATE; /* use ZONELIST_PRIVATE */
> >   __alloc_pages(..., nodemask);     /* with the private node set */
> > 
> 
> As I stated above, NUMA concept was quite naive at first glance for my
> limited knowledge.
> 
> This is quite alot to add for NUMA node concept, I'll want to see
> more explanation in v6 and learn from it, thanks.
> 

Sure, I can expand on it.  I think it's not as much to add as you think
though - it's simply adding the concept of isolation to a NUMA node.

Some more explanation below, but if you think there is something i
should explicit spell out in v6 cover, please let me know.

---

I agree that NUMA as a concept was best-effort for, comically enough,
somewhat *Uniform* memory access - instead of *Non*-Uniform memory access.

The current abstraction quite nicely handles the case where all memory
on the system is roughly of the same calibre (DDR4, DDR5, etc) and
roughly for the same purpose (general system memory).

But it is quite incapable of handling truly heterogeneous memory systems
(precious HBM attached to the CPU, GPU's with HBM over a coherent link,
hardware-compressed memory expansion, network devices w/ memory, etc).

If you look at the history of ZONE_DEVICE, what it fundamentally does is
slaps an isolation mechanism on top of NUMA nodes because the NUMA
abstraction doesn't provide one. 

The problem with that approach is now you have to reason about a node
having both fungible and non-fungible memory.  It creates the need for
something like migrate_device.c when a properly isolated NUMA node could
just use migrate.c directly (with a coherent link).

One example of what isolation on the node enables:

A Private Node can hotplug memory in ZONE_NORMAL - which means it
can be GUP pinned.  That means driver support for GPU direct storage
is simply `alloc_pages_node() + pin()`. The driver doesn't have to
worry about something like SLAB accidentally using that same memory.

All without having to rewrite a bunch of mm/ in a driver, and with
having to do some kind of heroics with ZONE_DEVICE that causes even
more special mm/ interactions.

That only comes from adding an isolation primitive to a NUMA node.

> > Isolating private node folios from kernel services
> > ==================================================
> > We implement filter points in mm/ to prevent operations on
> > private node memory.  Where possible, we even re-use existing
> > filter points from ZONE_DEVICE.
> > 
> > Most filter points are one or two lines of code:
> > 
> > Combining ZONE_DEVICE and N_MEMORY_PRIVATE opt-out spots:
> >   -     if (folio_is_zone_device(folio))
> >   +     if (unlikely(folio_is_private_managed(folio)))
> > 
> > Disabling a service:
> >   +  if (!node_is_private(nid)) {
> >   +      kswapd_run(nid);
> >   +      kcompactd_run(nid);
> >   +  }
> > 
> > Disallowing a uapi interaction:
> >   +  if (node_state(nid, N_MEMORY_PRIVATE))
> >   +      return -EINVAL;
> > 
> 
> I'm not sure of why do we re-implement more filter and basically
> doing the same thing ? Any unavoidable scenario ?
> 
> re-using the exisintg filter would be nice if that's possible.
> 

We re-use (combine) existing filters where possible.

We add new ones where ZONE_DEVICE did not implement filters due to some
*implicit* filter already existing.

Two clear examples:
   - reclaim does not target ZONE_DEVICE
   - ZONE_DEVICE does not support PMD

Both cases result in private-node filters that otherwise would have to
exist for ZONE_DEVICE (and if ZONE_DEVICE ever grows PMD support, those
filters will have to be added).

> > Bonus Configuration: HBM device memory tiering
> > ==============================================
> > echo 1 > dax0.0/private       # make the node private
> > echo 0 > dax0.0/adistance     # highest tier
> > echo 1 > dax0.0/reclaim       # reclaim active
> > echo 1 > dax0.0/demotion      # may demote from the node
> > echo 1 > dax0.0/user_numa     # mbind()
> > echo online_movable > dax0.0/state
> > echo 1 > numa/demotion_enabled
> > 
> > This is an HBM device which is treated as the top-tier in the
> > system but for which memory can only enter via explicit mbind().
> > 
> > It can be overcommitted because it can be reclaimed (demotions
> > go to CPU DRAM, and reclaim can swap from it).
> > 
> > If the HBM is managed by an accelerator (GPU), the mmu_notifier
> > allows it to know when reclaim is moving memory out to do
> > device-mmu invalidation prior to migration.
> > 
> > Prereqs, base commit, references
> > ================================
> > akpm/mm-new - for Brendan Jackman's mm/page_alloc.h work[3]
> > 
> 
> Still thanks for the work, I learn alot from your work as well, thanks.
> 

Of course, and thank you for reading.

If nothing else I hope this series helps folks understand the page
allocator better (it certainly has helped me).  If there is anything
you think I can improve or better explain, I am happy to discuss.

I am planning a larger publication of all my research sometime in the
future, but I think code is more impactful and useful - so I am
prioritizing that for now :].

~Gregory

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Richard Cheng 18 hours ago

On Wed, Jul 22, 2026 at 11:40:06AM +0800, Gregory Price wrote:
> On Wed, Jul 22, 2026 at 10:20:00PM +0800, Richard Cheng wrote:
> > On Mon, Jul 20, 2026 at 03:33:54PM +0800, Gregory Price wrote:
> > Hi Gregory,
> > 
> > I applied your series on mm-next and give a quick review,
> > not thoroughly, still I have some questions below regarding the design.
> >  
> 
> Hi Richard,
> 
> Thank you for the read.
> 
> Just a heads up, this was based on mm-new because of some recent work
> on Brendan's page_alloc.h cleanup, but if you managed to get it applied
> then maybe that's made its way forward already.
> 
> > > The goal here is to flip that dynamic, isolate by default and then
> > > opt-in to specific services that the device says is safe.
> > > 
> > > Isolation at the NUMA/Zonelist layer provides a powerful mechanism for
> > > memory hosted on accelerators - re-use of the kernel mm/ code.
> > > 
> > >  - Accelerators (GPUs) can use demotion, numactl, and reclaim.
> > >  - Special memory devices (Compressed RAM) with special access controls
> > >    (promote-on-write) can have generic services written for them.
> > >  - Network devices with large memory regions intended for ring buffers
> > >    can use the buddy and standard networking stack.
> > >  - Slow, disaggregated memory pools which aren't suitable as general
> > >    purpose memory get cleaner interfaces (no need to re-write the buddy
> > >    in userland, can use migration interface, etc).
> > >  - Per-workload dedicated memory nodes (disaggregated VM memory)
> > > 
> > 
> > For accelerator part, take CXL Type-2 as example, it has its own protocol
> > CXL.cache, CXL.mem which is the rule they need to obey during memory transaction,
> > Adding more rules for them since they are NUMA node confused me, I wonder the reason ?
> > 
> 
> CXL protocols simply state how to do the memory transaction at a
> physical / transport level.  It does not make any statement on how the
> operating systems are to make sense of these devices or what constructs
> / abstractions to build to actually manage the devices themselves.
>

Ahhh thanks this clears my question.
Then this abstraction makes sense.

> This series is not necessarily attached to CXL in particular, you could
> just as easily carve out memory from the general pool onto a private
> node to ensure it only gets used for a particular use.
> 
> In fact, that is how I have been testing this with dax/kmem:
> https://github.com/gourryinverse/linux/commit/f279e741d9c3643597525bfab916b29b95cb635b
> 
> > Because NUMA node, at least for me, representing topology/locality rather than
> > something with ownership or capability or rules.
> > 
> 
> The NUMA abstraction representing topology/locality is a construct we
> (the OS developers) have decided on historically - but there's nothing
> that dictates we can never create new useful abstractions with it.
> 
> Consider:
>     N_NORMAL_MEMORY,        /* The node has regular memory */
>     N_HIGH_MEMORY,          /* The node has regular or high memory */
> 
> These have nothing to do with topology or locality, they are node states
> that only have meaning in the context of linux mm/.
> 

Ok, I see.

> 
> To make something clear - there is no *requirement* for any particular
> device to use this abstraction.  It simply enables a cleaner way for
> devices carrying memory to re-utilize mm/ services while ensuring their
> memory does not silently get used under system pressure (or some vagrant
> in userspace doing `numactl --interleave all`).
> 
> Ignoring all the CAP bits entirely, if you just took the base series
> your driver could re-use the buddy allocator without any special logic
> AND have confidence that your driver is the only possible user of that
> memory (barring some truly obscene bug).
> 
> > >  - isolation via a dedicated zonelist:
> > >      - private nodes are omitted from FALLBACK/NOFALLBACK
> > >      - added ZONELIST_PRIVATE(_NOFALLBACK)
> > 
> > I saw the reply in patch 5, so in fact there's not only one
> > dedicate zonelist, but numerous ?
> > I raise the question because zonelist was supposed to be a
> > global, unbypasssable thing in MM design, but now what you
> > are trying to do is to seperate the whole global list into several
> > parts ?
> > 
> > Can you explain why in current design you don't consider to support
> > something like ZONELIST_PRIVATE[n]={0, .. ,n-1} ? that's my imagination
> > of what a global zonelist should look like, no matter private or non.
> > 
> 
> First let me say that there's nothing that prevents us from doing this,
> and we *could* make this the default case - but this decision was
> intentional by me.
> 
> Having them all present in each-other's zonelists by default creates a
> number of implicit opt-ins that are unclear:
> 
>   - any direct zonelist iterator now iterates all zones on all private
>     nodes, even those nodes do not opt into the same services
> 
>     e.g. zonelist iteration in reclaim that targets Private Node A would
>     attempt to reclaim Private Node B as well.  That would require and
>     extra explicit filter.
> 
>     I ask: Why do this? Just isolate in the zonelist, and if there is
>            a desire for intersections - make it explicit, not implicit.
> 

Fair enough, thanks for the headup.

>   - fallback allocations can now occur across private nodes, even
>     if those nodes are intended for different purposes.
> 
>     Obviously you can use nodemask to tighten the allocation target, but
>     I use `numactl --interleave --all` as an example of a clear case
>     where intersected nodelists may not give you the behavior you want.
> 
> 
> So for consistency - everything including zonelists have full isolation.
> This way all interactions with a private node must be explicit - always.
> 

Good choice I think, avoiding those implicit opt-in assumption will make
future developers more aware of things and do not just stumble into them.

> 
> This is actually one of the problems with ZONE_DEVICE - and you can see
> it in this patch set.  Some of the hooks in mm/ for zone_device only
> apply to PTE cases, and are absent from PMD cases - only because PMD
> mappings in ZONE_DEVICE aren't supported.
> 
> That kind of implicit behavior is quite bad.
> 
> That said, future improvements could include something like:
> 
> for_reclaimable_zone(ZONELIST_PRIVATE) {}
> 
> where we loosen this isolation, and formalize a filter, but I would
> like to see the usecase for it first.  Loosening the isolation defeats
> the entire purpose of the series, so there should be a strong reason to
> do so.

Totally agree.

> 
> > > Allocation Isolation
> > > ====================
> > > page_alloc presently controls whether a node's memory can be allocated
> > > on a given call by 4 things (in order of authority)
> > > 
> > >   1) ZONELIST membership
> > >      If a node is not in the walked zonelist, it's unreachable.
> > >
> > 
> > So a device gets hotplugged in the system will get a dedicated zonelist here ?
> 
> Yes.
> 
> > And make sure it obeys the device's own protocol if it has one ?
> > 
> 
> I'm not sure i follow this question, can you help me understand?
> 

Nevermind, you just answer this part above, thanks alot.

> If by protocol you mean the CAP bits (opting into reclaim, demotion,
> etc), then yes.  If you mean something else (CXL) then I think that's
> orthogonal and unrelated.
> 
> > > Private nodes:
> > >   1) Never appear in any ZONELIST_FALLBACK
> > >   2) Have an empty ZONELIST_NOFALLBACK
> > >   3) Only appear in their own ZONELIST_PRIVATE(_NOFALLBACK)
> > > 
> > > 1 & 2 mean all existing in-tree callers to page_alloc can NEVER
> > > accidentally allocate from a private node.
> > > 
> > > An allocation must explicitly ask via a zonelist and a nodemask.
> > > 
> > >   alloc_flags |= ALLOC_ZONELIST_PRIVATE; /* use ZONELIST_PRIVATE */
> > >   __alloc_pages(..., nodemask);     /* with the private node set */
> > > 
> > 
> > As I stated above, NUMA concept was quite naive at first glance for my
> > limited knowledge.
> > 
> > This is quite alot to add for NUMA node concept, I'll want to see
> > more explanation in v6 and learn from it, thanks.
> > 
> 
> Sure, I can expand on it.  I think it's not as much to add as you think
> though - it's simply adding the concept of isolation to a NUMA node.
> 
> Some more explanation below, but if you think there is something i
> should explicit spell out in v6 cover, please let me know.
> 
> ---
> 
> I agree that NUMA as a concept was best-effort for, comically enough,
> somewhat *Uniform* memory access - instead of *Non*-Uniform memory access.
> 
> The current abstraction quite nicely handles the case where all memory
> on the system is roughly of the same calibre (DDR4, DDR5, etc) and
> roughly for the same purpose (general system memory).
> 
> But it is quite incapable of handling truly heterogeneous memory systems
> (precious HBM attached to the CPU, GPU's with HBM over a coherent link,
> hardware-compressed memory expansion, network devices w/ memory, etc).
> 
> If you look at the history of ZONE_DEVICE, what it fundamentally does is
> slaps an isolation mechanism on top of NUMA nodes because the NUMA
> abstraction doesn't provide one. 
> 
> The problem with that approach is now you have to reason about a node
> having both fungible and non-fungible memory.  It creates the need for
> something like migrate_device.c when a properly isolated NUMA node could
> just use migrate.c directly (with a coherent link).
> 
> 
> One example of what isolation on the node enables:
> 
> A Private Node can hotplug memory in ZONE_NORMAL - which means it
> can be GUP pinned.  That means driver support for GPU direct storage
> is simply `alloc_pages_node() + pin()`. The driver doesn't have to
> worry about something like SLAB accidentally using that same memory.
> 
> All without having to rewrite a bunch of mm/ in a driver, and with
> having to do some kind of heroics with ZONE_DEVICE that causes even
> more special mm/ interactions.
> 
> That only comes from adding an isolation primitive to a NUMA node.
> 
> 
> > > Isolating private node folios from kernel services
> > > ==================================================
> > > We implement filter points in mm/ to prevent operations on
> > > private node memory.  Where possible, we even re-use existing
> > > filter points from ZONE_DEVICE.
> > > 
> > > Most filter points are one or two lines of code:
> > > 
> > > Combining ZONE_DEVICE and N_MEMORY_PRIVATE opt-out spots:
> > >   -     if (folio_is_zone_device(folio))
> > >   +     if (unlikely(folio_is_private_managed(folio)))
> > > 
> > > Disabling a service:
> > >   +  if (!node_is_private(nid)) {
> > >   +      kswapd_run(nid);
> > >   +      kcompactd_run(nid);
> > >   +  }
> > > 
> > > Disallowing a uapi interaction:
> > >   +  if (node_state(nid, N_MEMORY_PRIVATE))
> > >   +      return -EINVAL;
> > > 
> > 
> > I'm not sure of why do we re-implement more filter and basically
> > doing the same thing ? Any unavoidable scenario ?
> > 
> > re-using the exisintg filter would be nice if that's possible.
> > 
> 
> We re-use (combine) existing filters where possible.
> 
> We add new ones where ZONE_DEVICE did not implement filters due to some
> *implicit* filter already existing.
> 
> Two clear examples:
>    - reclaim does not target ZONE_DEVICE
>    - ZONE_DEVICE does not support PMD
> 
> Both cases result in private-node filters that otherwise would have to
> exist for ZONE_DEVICE (and if ZONE_DEVICE ever grows PMD support, those
> filters will have to be added).
> 
> > > Bonus Configuration: HBM device memory tiering
> > > ==============================================
> > > echo 1 > dax0.0/private       # make the node private
> > > echo 0 > dax0.0/adistance     # highest tier
> > > echo 1 > dax0.0/reclaim       # reclaim active
> > > echo 1 > dax0.0/demotion      # may demote from the node
> > > echo 1 > dax0.0/user_numa     # mbind()
> > > echo online_movable > dax0.0/state
> > > echo 1 > numa/demotion_enabled
> > > 
> > > This is an HBM device which is treated as the top-tier in the
> > > system but for which memory can only enter via explicit mbind().
> > > 
> > > It can be overcommitted because it can be reclaimed (demotions
> > > go to CPU DRAM, and reclaim can swap from it).
> > > 
> > > If the HBM is managed by an accelerator (GPU), the mmu_notifier
> > > allows it to know when reclaim is moving memory out to do
> > > device-mmu invalidation prior to migration.
> > > 
> > > Prereqs, base commit, references
> > > ================================
> > > akpm/mm-new - for Brendan Jackman's mm/page_alloc.h work[3]
> > > 
> > 
> > Still thanks for the work, I learn alot from your work as well, thanks.
> > 
> 
> Of course, and thank you for reading.
> 
> If nothing else I hope this series helps folks understand the page
> allocator better (it certainly has helped me).  If there is anything
> you think I can improve or better explain, I am happy to discuss.
> 
> I am planning a larger publication of all my research sometime in the
> future, but I think code is more impactful and useful - so I am
> prioritizing that for now :].
> 
> ~Gregory

That would be helpful for me, thanks alot for all the explanation again !

--Richard

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Gregory Price 2 days, 4 hours ago

On Wed, Jul 22, 2026 at 11:40:06AM -0400, Gregory Price wrote:
> On Wed, Jul 22, 2026 at 10:20:00PM +0800, Richard Cheng wrote:
> > 
> > I saw the reply in patch 5, so in fact there's not only one
> > dedicate zonelist, but numerous ?
> > I raise the question because zonelist was supposed to be a
> > global, unbypasssable thing in MM design, but now what you
> > are trying to do is to seperate the whole global list into several
> > parts ?
> > 
> > Can you explain why in current design you don't consider to support
> > something like ZONELIST_PRIVATE[n]={0, .. ,n-1} ? that's my imagination
> > of what a global zonelist should look like, no matter private or non.
> > 
> 
> First let me say that there's nothing that prevents us from doing this,
> and we *could* make this the default case - but this decision was
> intentional by me.
> 

You caught me on something here - I am wrong about my own code.

I described to you how I originally implemented zonelist isolation,
which I walked back when I was testing mempolicy and demotion.

The current code and the comments in the code are correct, we get
the following zonelists.

N_MEMORY : 0,1
N_MEMORY_PRIVATE: 2,3

ZONELIST_PRIVATE[0]     = {0,1,2,3}
ZONELIST_PRIVATE[1]     = {1,0,2,3}
ZONELIST_PRIVATE[2]     = {2,0,1,3}
ZONELIST_PRIVATE[3]     = {3,0,1,2}

Meaning everything except for MPOL_F_RELATIVE_NODES works (because
of cpuset isolation, but more on this in a moment).

So what I shipped here is what you are describing.

Looking back at my notes - I realized a few things that lead me to
walk back that isolation:

1) mempolicy makes more sense this way.
2) reclaim/demotion is easier to implement without isolation.
3) The individual service checks and nodelists handle the rest.
4) It overally just makes more sense.  If something breaks isolation
   it's because there's a bug, not because of a structural issue.

For example in demotion we do this:

demote_folio_list():
    /* Get demotion targets, find the best one, and demote */
    node_get_allowed_targets(pgdat, &allowed_mask);
    target_nid = next_demotion_node(pgdat->node_id, &allowed_mask);
    migrate_pages(demote_folios, alloc_demote_folio, ...);

Where the first step already filters on CAP_DEMOTION.
The result is still clean - no special iterators needed.

What I realized from your questions here is that since i walked back the
zonelist isolation, I think i can actually include any CAP_USER_NUMA
node in cpuset.mems and allow parititioning fairly trivially.

I think this might be the last majority complexity falling out.

Thank you for the questions - this has helped.

~Gregory

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Zenghui Yu 3 days, 11 hours ago

On 2026/7/21 3:33, Gregory Price wrote:
>   KVM: guest_memfd: bind backing memory to a NUMA node at creation
>   KVM: selftests: add a guest_memfd FLAG_BIND_NODE test

I can't find the last patch in my inbox or on the lore. Not sure what went
wrong.

Thanks,
Zenghui

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Gregory Price 3 days, 8 hours ago

On Tue, Jul 21, 2026 at 09:26:09PM +0800, Zenghui Yu wrote:
> On 2026/7/21 3:33, Gregory Price wrote:
> >   KVM: guest_memfd: bind backing memory to a NUMA node at creation
> >   KVM: selftests: add a guest_memfd FLAG_BIND_NODE test
> 
> I can't find the last patch in my inbox or on the lore. Not sure what went
> wrong.
> 
> Thanks,
> Zenghui

Apologies, my email account got rate-limited to the number of recipients.

It should be in everyone's box shortly.

~Gregory

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Balbir Singh 3 days, 21 hours ago

On 7/21/26 5:33 AM, Gregory Price wrote:
> This series introduces the concept of "Private Memory Nodes", which
> are opted into two basic functionalities by default:
>   - page allocation (mm/page_alloc.c)
>   - OOM killing
> 
> All other features of mm/ are opted out of managing these NUMA
> nodes and its memory.  Then we add capability bits to allow node
> owners to opt back into those services (if supported).
> 
> NUMA-hosted memory is presently a most-privilege system (any memory on
> any node, except ZONE_DEVICE, is 100% fungible and accessible). We
> then slap restrictions on top of it (cpuset, mempolicy, ZONE type,
> page/folio flags, etc).
> 
> The goal here is to flip that dynamic, isolate by default and then
> opt-in to specific services that the device says is safe.
> 
> Isolation at the NUMA/Zonelist layer provides a powerful mechanism for
> memory hosted on accelerators - re-use of the kernel mm/ code.
> 
>  - Accelerators (GPUs) can use demotion, numactl, and reclaim.
>  - Special memory devices (Compressed RAM) with special access controls
>    (promote-on-write) can have generic services written for them.
>  - Network devices with large memory regions intended for ring buffers
>    can use the buddy and standard networking stack.
>  - Slow, disaggregated memory pools which aren't suitable as general
>    purpose memory get cleaner interfaces (no need to re-write the buddy
>    in userland, can use migration interface, etc).
>  - Per-workload dedicated memory nodes (disaggregated VM memory)
> 
> And more use cases I have collected over the past few years.
> 
> Not included here is a dax-extension [1] that exposes all the internal
> bits as userland controls for testing - along with a pile of selftests
> that prove correctness.
> 
> Changes Since V4
> ================
>  - Massive reduction in complexity.
>      - no ops struct
>      - no callback functions
>      - no __GFP_PRIVATE
>      - no __GFP_THISNODE requirement
>      - no task flags (no PF_MEMALLOC_* in the alloc path)

Very happy to see this

>  - isolation via a dedicated zonelist:
>      - private nodes are omitted from FALLBACK/NOFALLBACK
>      - added ZONELIST_PRIVATE(_NOFALLBACK)
>  - ALLOC_ZONELIST_PRIVATE alloc_flag
>      - on top of Brendan Jackman's recent mm/page_alloc.h work [3]
>      - Zonelist selection rides the allocator's alloc_flags
>  - rename OPS -> CAPS (capabilities)
>  - split base functionality (isolation) from opt-ins (CAPS)
>      - first half of series can be merged without CAPS
>  - dropped compressed ram example from series
>      - will submit separately if this moves forward
>  - Added KVM as first primary in-tree user (mempolicy / CAP_USER_NUMA)
>  - fully functional dax-kmem extension and huge suite of selftests
>    located at my github, to be discussed separately [1]
> 
> Patch Layout
> ============
> The series is broken into two sections:
> 
> 1) N_MEMORY_PRIVATE Introduction.
>    Introduce the node state.
>    Opt those nodes out of mm/ services.
> 
> 2) NODE_PRIVATE_CAP_* features
>    A set of mm/ service opt-in flags that augment private
>    nodes to make them more useful (i.e. reclaim = overcommit).
> 
>    NODE_PRIVATE_CAP_LTPIN for private node folio pinning
>    NODE_PRIVATE_CAP_NUMA_BALANCING for private-node NUMA balancing
>    NODE_PRIVATE_CAP_DEMOTION for private-node tiering demotion
>    NODE_PRIVATE_CAP_HOTUNPLUG for opted-in private nodes
>    NODE_PRIVATE_CAP_USER_NUMA for userland numa controls
>    NODE_PRIVATE_CAP_RECLAIM for opted-in private node reclaim
> 

Looks reasonable, I wonder why USER_NUMA/HOTUNPUG is an opt-in?

> My hope is to merge at least #1 pulled ahead while #2 is debated.
> 
> Allocation Isolation
> ====================
> page_alloc presently controls whether a node's memory can be allocated
> on a given call by 4 things (in order of authority)
> 
>   1) ZONELIST membership
>      If a node is not in the walked zonelist, it's unreachable.
> 
>   2) __GFP_THISNODE
>      If this flag is set and the only node in the ZONELIST is the
>      singular preferred node (or the local node, for -1)
> 
>   3) cpuset.mems membership
>      cpuset trims any node in its allowed list
> 
>   4) mempolicy nodemask
>      the allocator will skip any node not in the nodemask.
> 
> Except for #1 (Zonelist membership) there are all kinds of weird corner
> conditions in which 2-4 can be completely ignored (interrupt context,
> empty set because cpuset doesn't intersect mempolicy, shared vma, ...)
> 
> But ZONELIST membership is *absolute*.  If a zone is not in the
> zonelist being walked, IT CANNOT BE ALLOCATED FROM. PERIOD.
> 
> The existing zonelists are constructed like so:
>   ZONELIST_FALLBACK    : All N_MEMORY nodes
>   ZONELIST_NOFALLBACK  : A singleton N_MEMORY node
> 
> Private node isolation is implemented via ZONELIST isolation:
>   ZONELIST_PRIVATE            : The private node + N_MEMORY
>   ZONELIST_PRIVATE_NOFALLBACK : The private node alone.
> 
> (mirrors exactly the fallback/nofallback for __GFP_THISNODE)
> 
> Private nodes:
>   1) Never appear in any ZONELIST_FALLBACK
>   2) Have an empty ZONELIST_NOFALLBACK
>   3) Only appear in their own ZONELIST_PRIVATE(_NOFALLBACK)
> 
> 1 & 2 mean all existing in-tree callers to page_alloc can NEVER
> accidentally allocate from a private node.
> 
> An allocation must explicitly ask via a zonelist and a nodemask.
> 
>   alloc_flags |= ALLOC_ZONELIST_PRIVATE; /* use ZONELIST_PRIVATE */
>   __alloc_pages(..., nodemask);     /* with the private node set */
> 
> The page allocator keeps all its original interfaces which only
> ever touch the default zonelists - avoiding churn.
> 
> Making it accessible via Mempolicy: MPOL_F_PRIVATE and page_alloc
> =================================================================
> The vast majority of the kernel will never need to know about
> ZONELIST_PRIVATE, because we add MPOL_F_PRIVATE to mempolicy.
> 
> When a mempolicy has MPOL_F_PRIVATE, the alloc_mpol() interfaces
> do the zonelist selection for the source of the allocation.
> 
> That really is the whole explanation of the mechanism:
> 
>   alloc_flags = mpol_alloc_flags(pol);
>   page = __alloc_frozen_pages_noprof(..., alloc_flags);
> 
> On mm-new this rides the allocator's existing alloc_flags plumbing:
> ALLOC_ZONELIST_PRIVATE is just another alloc_flag, so no new
> parameter, enum, or alloc_context change is required.
> 
> For modules that want to implement their own special handling, they
> get the _private variants for the page allocator.  This lets modules
> re-use the buddy instead of rewriting it.
> 
>    - alloc_pages_node_private_noprof()
>    - folio_alloc_node_private_noprof()
> 
> This keeps ALLOC_ flags mm/ internal (these functions add the flags).
> 
> Isolating private node folios from kernel services
> ==================================================
> We implement filter points in mm/ to prevent operations on
> private node memory.  Where possible, we even re-use existing
> filter points from ZONE_DEVICE.
> 
> Most filter points are one or two lines of code:
> 
> Combining ZONE_DEVICE and N_MEMORY_PRIVATE opt-out spots:
>   -     if (folio_is_zone_device(folio))
>   +     if (unlikely(folio_is_private_managed(folio)))
> 
> Disabling a service:
>   +  if (!node_is_private(nid)) {
>   +      kswapd_run(nid);
>   +      kcompactd_run(nid);
>   +  }
> 
> Disallowing a uapi interaction:
>   +  if (node_state(nid, N_MEMORY_PRIVATE))
>   +      return -EINVAL;
> 
> In the second half of the series, we replace blanket N_MEMORY_PRIVATE
> filters with NODE_PRIVATE_CAP_* filters to opt those nodes into that
> interaction if CAP is set.
> 
> We abstract this with a nice clean interface to make it really clear
> what is happening (nodes have features!)
> 
>   -  if (node_state(pgdat->node_id, N_MEMORY_PRIVATE))
>   +  if (!node_allows_reclaim(pgdat->node_id))
> 
> NODE_PRIVATE_CAP_* features
> ===========================
> This series of commits opts private nodes into various mm/ services.
> 
> Capabilities:
>   NODE_PRIVATE_CAP_RECLAIM       - direct and kswapd reclaim
>   NODE_PRIVATE_CAP_USER_NUMA     - userland numa controls
>   NODE_PRIVATE_CAP_DEMOTION      - node is a demotion target
>   NODE_PRIVATE_CAP_HOTUNPLUG     - hotunplug may migrate
>   NODE_PRIVATE_CAP_NUMA_BALANCING - NUMAB may target node folios
>   NODE_PRIVATE_CAP_LTPIN         - Longterm pin operates normally
> 
> Some opt-in support is more intensive than others, so these features
> are broken out in a way that we can defer them as future work streams.
> 
> NODE_PRIVATE_CAP_RECLAIM:
>   Enabling reclaim for these nodes is actually surprisingly trivial.
> 
>   Without CAP_RECLAIM, when an allocation failure occurs, the system
>   will not attempt to swap the memory - and instead will OOM (typically
>   whatever task is using the most memory on *that* private node).
> 
>   This capability consists of:
>     1) enabling kswapd and kcompactd for that node at hotplug time.
>     2) formalizing opt-out hooks to node_allows_reclaim() opt-in hooks.
>     3) Sets normal watermarks for these nodes.
>     4) Allow madvise operations on that node (PAGEOUT).
>     5) A small tweak to how LRU decides which zones to visit.
> 
> NODE_PRIVATE_CAP_USER_NUMA
>   Enables the following userland interfaces to accept the node:
>     mbind()
>     set_mempolicy()
>     set_mempolicy_home_node()
>     move_pages()
>     migrate_pages()
> 
>   example:
>      buf = mmap(..., MAP_ANON);
>      mbind(buf, ..., {private_node});
>      buf[0] = 0xDEADBEEF; /* Page faults onto the private node */
> 
>   Later - the KVM example shows how in-kernel mempolicies can
>   also be bound by CAP_USER_NUMA.
> 
>   Otherwise, that's it - it's just a mempolicy with MPOL_F_PRIVATE.
> 
> NODE_PRIVATE_CAP_HOTUNPLUG
>   This is simple: allow hotunplug to migrate this nodes folios.
> 
>   Some devices may not be able to tolerate unexpected migrations,
>   so we prevent hotunplug from engaging in migration by default.
> 
>   Some devices may have an mmu_notifier in their driver that can
>   manage the migration and subsequent refault.
> 
>   CAP_HOTUNPLUG allows memory_hotplug.c to migrate normally.
> 
> NODE_PRIVATE_CAP_DEMOTION
>   This adds the private node as a valid demotion target, and allows
>   reclaim to demote memory from a private node to a demotion target.
> 
>   Requires:  NODE_PRIVATE_CAP_RECLAIM
> 
> NODE_PRIVATE_CAP_NUMA_BALANCING
>   This enables numa balancing to inject prot_none on private node
>   folio mappings and promote them when faults are taken.
> 
> NODE_PRIVATE_CAP_LTPIN
>   This allows GUP Longterm Pinning to operate normally.
> 
>   Normally, longterm pinning determines folio eligibility based
>   on its ZONE_* membership (among other things).
> 
>   ZONE_NORMAL is eligible, while ZONE_MOVABLE folios require
>   migration to ZONE_NORMAL before pinning.
> 
>   Neither operation is preferable by default on a private node,
>   so the base behavior of FOLL_LONGTERM is to FAIL.
> 
>   This capability allows longterm pinning to operate normally
>   based on the ZONE membership.  Private node memory may be
>   hotplugged as either ZONE_NORMAL or ZONE_MOVABLE.
> 
> In-tree User: KVM
> =================
> Dave Jiang proposed [2] dax-backed guest_memfd() memory as a way of
> enabling disaggregated memory pools to host dedicated KVM memory.
> 
> With private nodes, this is trivial (with a bit of basic plumbing):
> 
> static int kvm_gmem_bind_node(struct inode *inode, int node)
> {
> ...
>   /* Bind to a private node - gated on CAP_USER_NUMA */
>   pol = mpol_bind_node(node);
>   if (IS_ERR(pol))
>     return PTR_ERR(pol);
> 
>   /* Set the shared policy */
>   err = mpol_set_shared_policy_range(&GMEM_I(inode)->policy, ..., pol);
> ...
> }
> 
> KVM doesn't even need to know about private nodes at all, all it
> does is ask mempolicy whether the requested node is a valid bind.
> 
> mm/ component testing with dax driver
> =====================================
> The dax driver extensions[1] implements a simple interface to create
> a private node from a dax device created by any source.
> 
> I left the dax driver extensions out of this feature set because
> it locks in the CAP_ bits before anyone has input.  It's there
> primarily for testing at this point.
> 
> The simplest way to get a dax device is with the memmap= boot arg.
>    e.g.: "memmap=0x40000000!0x140000000"
> 
> The dax driver extension has the following sysfs entries:
>   dax0.0/private         - set the node to private
>   dax0.0/dax_file        - make /dev/dax0.0 mmap'able in kmem mode
>   dax0.0/adistance       - dictate memory_tierN membership
>   dax0.0/reclaim         - CAP_RECLAIM
>   dax0.0/demotion        - CAP_DEMOTION
>   dax0.0/user_numa       - CAP_USER_NUMA
>   dax0.0/hotunplug       - CAP_HOTUNPLUG
>   dax0.0/numa_balancing  - CAP_NUMA_BALANCING
>   dax0.0/ltpin           - CAP_LTPIN
> 
> Now consider the following...
> 
> Single node reclaim + mbind support:
>   echo 1 > dax0.0/private
>   echo 1 > dax0.0/reclaim
>   echo 1 > dax0.0/user_numa
>   echo online_movable > dax0.0/state
> 
> Test program:
>    /* node1: 1GB Private Memory Node, 4GB swap */
>    buf = mmap(NULL, TWO_GB, PROT_READ | PROT_WRITE,
>               MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>    sys_mbind(p, len, MPOL_BIND, mask, MAXNODE, MPOL_MF_STRICT);
>    memset(buf, 0xff, TWO_GB);
> 
> We can *guarantee* the ONLY reclaiming tasks are exactly:
>   - kswapdN (in theory we can even make this optional!)
>   - the memset task faulting pages in
> 
> This also means these are the only tasks capable of becoming
> locked up and OOMing (as long as it is not under pressure).
> 
> The rest of the system remains entirely functional for debugging.
> 
> It now becomes possible to micro-benchmark and A/B test reclaim
> changes with different scenarios (number of tasks, amount of
> memory, watermark targets, etc), because we have hard controls
> over exactly which tasks can access that node memory and how.
> 
> If we broke CAP_RECLAIM into subflags:
>   - CAP_RECLAIM_KSWAPD
>   - CAP_RECLAIM_DIRECT
>   - CAP_COMPACTION_KCOMPACTD
>   - CAP_COMPACTION_DIRECT
> 
> We can actually test the efficacy of each of these mechanisms in
> isolation to each other - something that is strictly impossible today.
> 
> Bonus Configuration: HBM device memory tiering
> ==============================================
> echo 1 > dax0.0/private       # make the node private
> echo 0 > dax0.0/adistance     # highest tier
> echo 1 > dax0.0/reclaim       # reclaim active
> echo 1 > dax0.0/demotion      # may demote from the node
> echo 1 > dax0.0/user_numa     # mbind()
> echo online_movable > dax0.0/state
> echo 1 > numa/demotion_enabled
> 
> This is an HBM device which is treated as the top-tier in the
> system but for which memory can only enter via explicit mbind().
> 
> It can be overcommitted because it can be reclaimed (demotions
> go to CPU DRAM, and reclaim can swap from it).
> 
> If the HBM is managed by an accelerator (GPU), the mmu_notifier
> allows it to know when reclaim is moving memory out to do
> device-mmu invalidation prior to migration.
> 
> Prereqs, base commit, references
> ================================
> akpm/mm-new - for Brendan Jackman's mm/page_alloc.h work[3]
> 
> Prereqs (all already in akpm/mm-new; listed for out-of-tree application):
> 
> page_alloc.h split + alloc_flags plumbing this series rides on:
> commit e81fae43cd69 ("mm: split out internal page_alloc.h")
> commit b4ff3b6d0a1d ("mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG")
> 
> dax atomic whole-device hotplug (used by the dax extension [1]):
> commit d7aa81b9a919 ("mm/memory: add memory_block_aligned_range() helper")
> commit 3b2f402a1754 ("dax/kmem: add sysfs interface for atomic whole-device hotplug")
> 
> [1] https://github.com/gourryinverse/linux/tree/scratch/gourry/managed_nodes/dax_private-mm-new
> [2] https://lore.kernel.org/all/20260423170219.281618-1-dave.jiang@intel.com/
> [3] https://lore.kernel.org/all/20260702-alloc-trylock-v4-0-0af8ff387e80@google.com/
> 
> base-commit: c872b70f5d6c742ad34b8e838c92af81c8920b3e
> 
> Gregory Price (36):
>   mm: refactor find_next_best_node to find_next_best_node_in
>   mm/page_alloc: refactor build_node_zonelist() out of build_zonelists()
>   mm/page_alloc: let the bulk and folio allocators carry alloc_flags
>   numa: introduce N_MEMORY_PRIVATE
>   mm: add ZONELIST_PRIVATE(_NOFALLBACK) for N_MEMORY_PRIVATE nodes.
>   cpuset: exclude private nodes from cpuset.mems (default-open)
>   mm/memory_hotplug: disallow migration-driven private node hotunplug
>   mm/mempolicy: skip private node folios when queueing for migration
>   mm/migrate: disallow userland driven migration for private nodes
>   mm/madvise: disallow madvise operations on private node folios
>   mm/compaction: disallow compaction on private nodes
>   mm/page_alloc: clear private node watermarks and system reserves
>   mm/mempolicy: disallow NUMA Balancing prot_none on private nodes
>   mm/damon: skip private node memory in DAMON migration and pageout
>   mm/ksm: skip KSM for managed-memory folios
>   mm/khugepaged: skip private node folios when trying to collapse.
>   mm/vmscan: disallow reclaim of private node memory
>   mm/gup: disallow longterm pin of private node folios
>   proc: include N_MEMORY_PRIVATE nodes in numa_maps output
>   mm/memcontrol: account private-node memory in per-node stats
>   proc/kcore: include private-node RAM in the kcore RAM map
>   mm/mempolicy: add MPOL_F_PRIVATE and zonelist selection
>   mm/mempolicy: apply policy at the kernel zone for private-node binds
>   mm/mempolicy: add in-kernel MPOL_BIND interfaces for drivers/services
>   mm/memory_hotplug: support N_MEMORY_PRIVATE node hotplug
>   mm: add NODE_PRIVATE_CAP_RECLAIM for opted-in private node reclaim
>   mm: add NODE_PRIVATE_CAP_USER_NUMA for userland numa controls
>   mm: add NODE_PRIVATE_CAP_HOTUNPLUG for opted-in private nodes
>   mm: add NODE_PRIVATE_CAP_DEMOTION for private-node tiering demotion
>   mm: add NODE_PRIVATE_CAP_NUMA_BALANCING for private-node NUMA
>     balancing
>   mm: add NODE_PRIVATE_CAP_LTPIN for private node folio pinning
>   mm/khugepaged: base private node collapse eligiblity on actor/cap bits
>   Documentation/mm: describe private (N_MEMORY_PRIVATE) memory nodes
>   mm/mempolicy: add mpol_set_shared_policy_range()
>   KVM: guest_memfd: bind backing memory to a NUMA node at creation
>   KVM: selftests: add a guest_memfd FLAG_BIND_NODE test
> 
>  Documentation/ABI/stable/sysfs-devices-node   |  10 +
>  Documentation/mm/index.rst                    |   1 +
>  Documentation/mm/numa_private_nodes.rst       | 160 ++++++++++
>  drivers/base/node.c                           | 118 +++++++
>  drivers/dax/kmem.c                            |   2 +-
>  fs/proc/kcore.c                               |   5 +-
>  fs/proc/task_mmu.c                            |  10 +-
>  include/linux/gfp.h                           |  22 +-
>  include/linux/kvm_host.h                      |   3 +
>  include/linux/memory_hotplug.h                |   5 +-
>  include/linux/mempolicy.h                     |  16 +
>  include/linux/mmzone.h                        |  26 +-
>  include/linux/node_private.h                  | 262 ++++++++++++++++
>  include/linux/nodemask.h                      |   7 +-
>  include/uapi/linux/kvm.h                      |   5 +-
>  include/uapi/linux/mempolicy.h                |   1 +
>  kernel/cgroup/cpuset.c                        |  26 +-
>  mm/compaction.c                               |  13 +
>  mm/damon/paddr.c                              |   9 +
>  mm/gup.c                                      |  28 +-
>  mm/huge_memory.c                              |   5 +
>  mm/internal.h                                 |  97 +++++-
>  mm/khugepaged.c                               |  18 +-
>  mm/ksm.c                                      |   8 +-
>  mm/madvise.c                                  |   8 +-
>  mm/memcontrol-v1.c                            |   8 +-
>  mm/memcontrol.c                               |  13 +-
>  mm/memory-tiers.c                             |  40 ++-
>  mm/memory_hotplug.c                           | 119 +++++++-
>  mm/mempolicy.c                                | 288 +++++++++++++++---
>  mm/migrate.c                                  |  19 +-
>  mm/mm_init.c                                  |   2 +-
>  mm/page_alloc.c                               | 189 +++++++++---
>  mm/page_alloc.h                               |  34 +++
>  mm/vmscan.c                                   |  57 +++-
>  tools/testing/selftests/kvm/Makefile.kvm      |   2 +
>  .../kvm/guest_memfd_bind_node_test.c          | 213 +++++++++++++
>  virt/kvm/guest_memfd.c                        |  39 ++-
>  38 files changed, 1728 insertions(+), 160 deletions(-)
>  create mode 100644 Documentation/mm/numa_private_nodes.rst
>  create mode 100644 include/linux/node_private.h
>  create mode 100644 tools/testing/selftests/kvm/guest_memfd_bind_node_test.c
> 

Thanks,
Balbir

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Gregory Price 3 days, 7 hours ago

On Tue, Jul 21, 2026 at 01:46:01PM +1000, Balbir Singh wrote:
> On 7/21/26 5:33 AM, Gregory Price wrote:
> > 
> > Patch Layout
> > ============
> > The series is broken into two sections:
> > 
> > 1) N_MEMORY_PRIVATE Introduction.
> >    Introduce the node state.
> >    Opt those nodes out of mm/ services.
> > 
> > 2) NODE_PRIVATE_CAP_* features
> >    A set of mm/ service opt-in flags that augment private
> >    nodes to make them more useful (i.e. reclaim = overcommit).
> > 
> >    NODE_PRIVATE_CAP_LTPIN for private node folio pinning
> >    NODE_PRIVATE_CAP_NUMA_BALANCING for private-node NUMA balancing
> >    NODE_PRIVATE_CAP_DEMOTION for private-node tiering demotion
> >    NODE_PRIVATE_CAP_HOTUNPLUG for opted-in private nodes
> >    NODE_PRIVATE_CAP_USER_NUMA for userland numa controls
> >    NODE_PRIVATE_CAP_RECLAIM for opted-in private node reclaim
> > 
> 
> Looks reasonable, I wonder why USER_NUMA/HOTUNPUG is an opt-in?
> 

Ideologically: Private nodes default to fully isolated, why should any
given feature be special?

Concretely: 

User-numa

   Some devices don't want the user to have control over placement.
   I have been working on compressed memory, for example, which only
   ever wants to be used as a reclaim-demotion target.
   
   (The reasoning for this is another thread, i plan on publishing
    my research on this this year)

Hot-unplug:

   Some devices can't necessarily handle unexpected migration, and
   hot-unplug is fundamentally a migration.  So the HOTUNPLUG cap
   actually means "hot-unplug can execute migrations".

   If the entire device has pre-drained the memory (all memory is free)
   then unplug works - it's just not very hot (no migrations) :]

   Maybe a naming issue?

~Gregory

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Balbir Singh 2 days, 16 hours ago

On 7/22/26 4:16 AM, Gregory Price wrote:
> On Tue, Jul 21, 2026 at 01:46:01PM +1000, Balbir Singh wrote:
>> On 7/21/26 5:33 AM, Gregory Price wrote:
>>>
>>> Patch Layout
>>> ============
>>> The series is broken into two sections:
>>>
>>> 1) N_MEMORY_PRIVATE Introduction.
>>>    Introduce the node state.
>>>    Opt those nodes out of mm/ services.
>>>
>>> 2) NODE_PRIVATE_CAP_* features
>>>    A set of mm/ service opt-in flags that augment private
>>>    nodes to make them more useful (i.e. reclaim = overcommit).
>>>
>>>    NODE_PRIVATE_CAP_LTPIN for private node folio pinning
>>>    NODE_PRIVATE_CAP_NUMA_BALANCING for private-node NUMA balancing
>>>    NODE_PRIVATE_CAP_DEMOTION for private-node tiering demotion
>>>    NODE_PRIVATE_CAP_HOTUNPLUG for opted-in private nodes
>>>    NODE_PRIVATE_CAP_USER_NUMA for userland numa controls
>>>    NODE_PRIVATE_CAP_RECLAIM for opted-in private node reclaim
>>>
>>
>> Looks reasonable, I wonder why USER_NUMA/HOTUNPUG is an opt-in?
>>
> 
> Ideologically: Private nodes default to fully isolated, why should any
> given feature be special?
> 
> Concretely: 
> 
> User-numa
> 
>    Some devices don't want the user to have control over placement.
>    I have been working on compressed memory, for example, which only
>    ever wants to be used as a reclaim-demotion target.
>    
>    (The reasoning for this is another thread, i plan on publishing
>     my research on this this year)
> 

I don't fully understand, how do we allocate memory on these devices then? Is
the driver expected to allocate and map via vm_insert_page()?

> Hot-unplug:
> 
>    Some devices can't necessarily handle unexpected migration, and
>    hot-unplug is fundamentally a migration.  So the HOTUNPLUG cap
>    actually means "hot-unplug can execute migrations".
> 
>    If the entire device has pre-drained the memory (all memory is free)
>    then unplug works - it's just not very hot (no migrations) :]
> 
>    Maybe a naming issue?
> 

Yes, I would prefer MIGRATION in the name, HOTUNPLUG made me wonder how these
devices come online and because it is a device, it can go offline while the
system is still online

> ~Gregory
> 

Balbir

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Gregory Price 2 days, 12 hours ago

On Wed, Jul 22, 2026 at 06:29:53PM +1000, Balbir Singh wrote:
> On 7/22/26 4:16 AM, Gregory Price wrote:
> > 
> > User-numa
> > 
> >    Some devices don't want the user to have control over placement.
> >    I have been working on compressed memory, for example, which only
> >    ever wants to be used as a reclaim-demotion target.
> >    
> >    (The reasoning for this is another thread, i plan on publishing
> >     my research on this this year)
> > 
> 
> I don't fully understand, how do we allocate memory on these devices then? Is
> the driver expected to allocate and map via vm_insert_page()?
>

It depends on your use case, but yes that's one possibility.

In another use case, you could enable (CAP_RECLAIM | CAP_DEMOTION) and
pages are allocated via alloc_demotion_target() in the reclaim path.

For general "device hosted memory", you can do a variety of things:

Direct alloc 
    ioctl(...) ->  explicit alloc call
    mmap(/dev/my_device) + mm_fault handler

Or a kernel-internal mempolicy
    ioctl(...) ->  set kernel-internal mempolicy on a vma
    mmap(/dev/my_device) + kernel-internal only mempolicy on the vma

The KVM guest_memfd patch included at the end of the series uses the
kernel-internal mempolicy mechanism:
https://lore.kernel.org/linux-mm/al-pkvmgIxGu3LzM@gourry-fedora-PF4VCD3F/T/#me04a7e3556677babd82bfe3947b1a83b7f0017e1

This pattern is needed because the memory is never mapped into userland,
so userland actually *can't* set a mempolicy on this memory itself.

and then of course: CAP_USER_NUMA + mbind/set_mempolicy directly

This is kind of the point - the source of the memory has some control
over how it can be used.

> > Hot-unplug:
> > 
> >    Some devices can't necessarily handle unexpected migration, and
> >    hot-unplug is fundamentally a migration.  So the HOTUNPLUG cap
> >    actually means "hot-unplug can execute migrations".
> > 
> >    If the entire device has pre-drained the memory (all memory is free)
> >    then unplug works - it's just not very hot (no migrations) :]
> > 
> >    Maybe a naming issue?
> > 
> 
> Yes, I would prefer MIGRATION in the name, HOTUNPLUG made me wonder how these
> devices come online and because it is a device, it can go offline while the
> system is still online
> 

There's a difference between device hotplug and memory hotplug.  The
kernel's memory hotunplug system can at best be described as "best
effort", and if hotplugged as ZONE_NORMAL (on a non-private node) you're
unlikely to ever be able to hotunplug.

It's possible that this CAP bit should just go away and if the device
doesn't want its memory to be run migratable then it needs to hold extra
references on the folios.  That's probably reasonable and looks a lot
like a long term pin.

(note: I have to respin for Sashiko fixes, i'm likely going to drop most
of the CAP bits except USER_NUMA in v6 and have them come in with
specific use cases).

~Gregory

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Arun George/Arun George 1 day, 16 hours ago

On 21-07-2026 01:03 am, Gregory Price wrote:
> Changes Since V4
> ================
>   - dropped compressed ram example from series
>       - will submit separately if this moves forward

We intend to test this series on a compression capable CXL expander 
hardware. Since compressed ram example (cram) is not part of this 
series, how do you suggest to do that? Do you have a version of cram 
module compatible with this series?

~Arun

Re: [PATCH v5 00/36] Private Memory NUMA Nodes

Posted by Gregory Price 1 day, 8 hours ago

On Thu, Jul 23, 2026 at 02:08:31PM +0530, Arun George/Arun George wrote:
> On 21-07-2026 01:03 am, Gregory Price wrote:
> > Changes Since V4
> > ================
> >   - dropped compressed ram example from series
> >       - will submit separately if this moves forward
> 
> We intend to test this series on a compression capable CXL expander 
> hardware. Since compressed ram example (cram) is not part of this 
> series, how do you suggest to do that? Do you have a version of cram 
> module compatible with this series?
> 
> ~Arun

Hi Arun,

I dropped it from the base series because i want to try to move forward
with this without RFC.  The compressed service itself will be a separate
submission after I get decent signal on whether this will be accepted.

I plan to RFC the new setup for compressed ram this a bit later, but I
will share my working branch with you for testing.

~Gregory