include/linux/kvm_host.h | 3 + include/linux/mempolicy.h | 5 + include/uapi/linux/kvm.h | 5 +- mm/mempolicy.c | 75 +++++++++- .../testing/selftests/kvm/guest_memfd_test.c | 134 ++++++++++++++++-- virt/kvm/guest_memfd.c | 44 +++++- 6 files changed, 248 insertions(+), 18 deletions(-)
guest_memfd allocates its folios through a per-inode shared mempolicy.
Today that policy can only be set after the fact, with mbind() on a host
mmap of the fd. That requires the fd to be mappable, and it cannot reach
folios that are only ever guest-faulted. Neither holds for a
non-mappable (confidential) guest_memfd.
Add GUEST_MEMFD_FLAG_BIND_NODE and a node field to struct
kvm_create_guest_memfd. When set, KVM builds an MPOL_BIND policy for the
requested node and installs it over the whole inode, so every folio is
allocated there with no userspace mbind().
1-2 mm/mempolicy prep. mempolicy_create() builds a validated,
cpuset-contextualised policy without installing it into the
calling task. mpol_set_shared_policy_range() installs one over a
pgoff range with no VMA.
3 The KVM flag.
4-5 Selftest harness support, and the test.
Why a single node and not a full mempolicy?
The fd is the unit of guest NUMA topology. A multi-node guest is one
guest_memfd per guest node - or one range per node, since
kvm_gmem_bind() is offset-based and mpol_set_shared_policy_range() is
already range-capable - each bound to a host node, with the guest
placing memory on top. This is the shape QEMU already builds with one
memory-backend per guest node.
Interleaving a single fd across host nodes would model a guest node
whose pages are scattered underneath it. That defeats every placement
decision the guest makes: guest NUMA balancing, tiering and weighted
interleave would all be reasoning about a topology that doesn't exist.
This narrow implementation enables the only clear use case.
User-visible behaviour:
mempolicy_create() constrains the request to the task's cpuset.
A node outside mems_allowed fails the ioctl with -EINVAL - the same
constraint mbind() carries. A task cannot grant a guest_memfd access
to a node it cannot reach itself.
Nothing rebinds an inode's shared policy on a later cpuset change:
mpol_rebind_task() walks tsk->mempolicy and mpol_rebind_mm() walks
vma->vm_policy, and neither reaches a struct shared_policy. The bind
is fixed for the life of the fd. This matches shmem's implementation.
Testing
guest_memfd_test under virtme-ng, nested KVM:
- 2-node guest, test pinned to the CPUs of the node it is not binding
to, with the task mempolicy aimed at that other node. Pages land on
the bound node, so the placement cannot be explained by the fault
being local. Stripping the flag from the harness puts them on the
other node, confirming the check has teeth.
- CONFIG_NUMA=n: the flag is not advertised and the tests skip.
- Single node: create/mmap/fault coverage, no placement claim.
Gregory Price (5):
mm/mempolicy: add mempolicy_create()
mm/mempolicy: add mpol_set_shared_policy_range()
KVM: guest_memfd: bind backing memory to a NUMA node at creation
selftests: KVM: guest_memfd: let the gmem_test() harness bind a node
selftests: KVM: guest_memfd: test GUEST_MEMFD_FLAG_BIND_NODE
include/linux/kvm_host.h | 3 +
include/linux/mempolicy.h | 5 +
include/uapi/linux/kvm.h | 5 +-
mm/mempolicy.c | 75 +++++++++-
.../testing/selftests/kvm/guest_memfd_test.c | 134 ++++++++++++++++--
virt/kvm/guest_memfd.c | 44 +++++-
6 files changed, 248 insertions(+), 18 deletions(-)
---
base-commit: da6c37ed8beb273e3308e42d4bca3ce11b4432fa
--
2.53.0-Meta
On 9/2/26 21:46, Gregory Price wrote: > guest_memfd allocates its folios through a per-inode shared mempolicy. > > Today that policy can only be set after the fact, with mbind() on a host > mmap of the fd. That requires the fd to be mappable, and it cannot reach > folios that are only ever guest-faulted. Neither holds for a > non-mappable (confidential) guest_memfd. That looks like an investment into a mechanism we will soon consider legacy, so I am not convinced this is the right approach? -- Cheers, David
On Thu, Sep 10, 2026 at 01:30:04PM +0200, David Hildenbrand (Arm) wrote: > On 9/2/26 21:46, Gregory Price wrote: > > guest_memfd allocates its folios through a per-inode shared mempolicy. > > > > Today that policy can only be set after the fact, with mbind() on a host > > mmap of the fd. That requires the fd to be mappable, and it cannot reach > > folios that are only ever guest-faulted. Neither holds for a > > non-mappable (confidential) guest_memfd. > > That looks like an investment into a mechanism we will soon consider legacy, so > I am not convinced this is the right approach? > Hm, I think i've missed recent work in this space / this context. Any suggested reading here? Don't want to eat too much of your time. ~Gregory
On 9/10/26 15:40, Gregory Price wrote: > On Thu, Sep 10, 2026 at 01:30:04PM +0200, David Hildenbrand (Arm) wrote: >> On 9/2/26 21:46, Gregory Price wrote: >>> guest_memfd allocates its folios through a per-inode shared mempolicy. >>> >>> Today that policy can only be set after the fact, with mbind() on a host >>> mmap of the fd. That requires the fd to be mappable, and it cannot reach >>> folios that are only ever guest-faulted. Neither holds for a >>> non-mappable (confidential) guest_memfd. >> >> That looks like an investment into a mechanism we will soon consider legacy, so >> I am not convinced this is the right approach? >> > > Hm, I think i've missed recent work in this space / this context. > Any suggested reading here? Don't want to eat too much of your time. So the big development happening right now is in-place conversion (having shared and private pages managed in guest_memfd): https://lore.kernel.org/r/20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com The final series where we added mempolicy support should be: https://lore.kernel.org/linux-mm/20250827175247.83322-2-shivankg@amd.com/ (I think you are aware of that, you commented on some) See my other reply regarding the use of GUEST_MEMFD_FLAG_MMAP on private-only guest_memfd. -- Cheers, David
On Thu, Sep 10, 2026 at 04:05:36PM +0200, David Hildenbrand (Arm) wrote: > On 9/10/26 15:40, Gregory Price wrote: > > On Thu, Sep 10, 2026 at 01:30:04PM +0200, David Hildenbrand (Arm) wrote: > >> On 9/2/26 21:46, Gregory Price wrote: > >>> guest_memfd allocates its folios through a per-inode shared mempolicy. > >>> > >>> Today that policy can only be set after the fact, with mbind() on a host > >>> mmap of the fd. That requires the fd to be mappable, and it cannot reach > >>> folios that are only ever guest-faulted. Neither holds for a > >>> non-mappable (confidential) guest_memfd. > >> > >> That looks like an investment into a mechanism we will soon consider legacy, so > >> I am not convinced this is the right approach? > >> > > > > Hm, I think i've missed recent work in this space / this context. > > Any suggested reading here? Don't want to eat too much of your time. > > So the big development happening right now is in-place conversion (having shared > and private pages managed in guest_memfd): > > https://lore.kernel.org/r/20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com > > > The final series where we added mempolicy support should be: > > https://lore.kernel.org/linux-mm/20250827175247.83322-2-shivankg@amd.com/ > > (I think you are aware of that, you commented on some) > > See my other reply regarding the use of GUEST_MEMFD_FLAG_MMAP on private-only > guest_memfd. > Ah yes, i do remember this series. In fact... https://lore.kernel.org/linux-mm/aPD-dbl5KWNSHu5R@gourry-fedora-PF4VCD3F/ ``` So this inode mempolicy in guest_memfd is really acting more as a the filesystem-default mempolicy, which you want to survive even if userland never maps the memory/unmaps the memory. So the relationship is more like guest_memfd -> creates fd/inode <- copies task mempolicy (if set) vm: allocates memory via filemap_get_folio_mpol() userland mmap(fd): creates new inode<->vma mapping vma->mpol = kvm_gmem_get_policy() calls to set/get_policy/mbind go through kvm_gmem ``` And quick skim of in-place conversion (wow that's dense) ``` #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) Set attributes for a range of offsets within a guest_memfd to KVM_MEMORY_ATTRIBUTE_PRIVATE to limit the specified guest_memfd backed memory range for guest use. Even if KVM_CAP_GUEST_MEMFD_MMAP is supported, after a successful call to set KVM_MEMORY_ATTRIBUTE_PRIVATE, the requested range will not be mappable into host userspace and will only be mappable by the guest. ``` I guess the thought here would be something like... 1. create guest memfd 2. mmap(fd) -> prior to full guest setup 3. mbind(buf) -> shared policy hits the inode 4. munmap(buf) -> policy is NOT reaped 5. set ATTRIBUTE_PRIVATE 6. no longer user mappable That doesn't seem unreasonable ~Gregory
Gregory Price <gourry@gourry.net> writes:
> guest_memfd allocates its folios through a per-inode shared mempolicy.
>
> Today that policy can only be set after the fact, with mbind() on a host
> mmap of the fd. That requires the fd to be mappable, and it cannot reach
> folios that are only ever guest-faulted. Neither holds for a
> non-mappable (confidential) guest_memfd.
>
> Add GUEST_MEMFD_FLAG_BIND_NODE and a node field to struct
> kvm_create_guest_memfd. When set, KVM builds an MPOL_BIND policy for the
> requested node and installs it over the whole inode, so every folio is
> allocated there with no userspace mbind().
>
Instead of a custom API to ensure all guest_memfd allocations come from
a single node, how about these options?
1. Using cgroups/cpuset to constrain allocations (could be troublesome
if the guest memory is not preallocated, unless the vCPU threads are
running with the cpuset config)
2. Process-level NUMA policy
3. Why not request the guest_memfd to be mmap-able just to be able to
set a memory policy?
4. How about something like fbind() that takes an fd and offset range
instead of mbind(), which has a prerequisite on mmap()?
This doesn't exist yet, but I'm hoping to discuss this at LPC 2026:
5. What if you could pass an fd representing a mount to guest_memfd at
creation time, so to make all the allocations come from a single node
Step 1: Create a tmpfs mount, specify mpol for mount to MPOL_BIND
Step 2: Get some fd representing the tmpfs mount, hand that to
guest_memfd at creation time
Step 3: guest_memfd allocations will always come from that tmpfs
mount, and abide by that tmpfs mount's memory policy.
May I know more about the use case behind this new feature?
>
> [...snip...]
>
On Wed, Sep 09, 2026 at 03:41:35PM -0700, Ackerley Tng wrote: > Gregory Price <gourry@gourry.net> writes: > > > guest_memfd allocates its folios through a per-inode shared mempolicy. > > > > Today that policy can only be set after the fact, with mbind() on a host > > mmap of the fd. That requires the fd to be mappable, and it cannot reach > > folios that are only ever guest-faulted. Neither holds for a > > non-mappable (confidential) guest_memfd. > > > > Add GUEST_MEMFD_FLAG_BIND_NODE and a node field to struct > > kvm_create_guest_memfd. When set, KVM builds an MPOL_BIND policy for the > > requested node and installs it over the whole inode, so every folio is > > allocated there with no userspace mbind(). > > > > Instead of a custom API to ensure all guest_memfd allocations come from > a single node, how about these options? > > 1. Using cgroups/cpuset to constrain allocations (could be troublesome > if the guest memory is not preallocated, unless the vCPU threads are > running with the cpuset config) > > 2. Process-level NUMA policy > for 1 and 2: the intent is to put the guest memory on the target node, not all system memory for a given process. so the scope here is not the same. in fact at that granularity, the desired node may not even have eligible memory to host the task's memory. > 3. Why not request the guest_memfd to be mmap-able just to be able to > set a memory policy? > the eventual intent is to enable this for fully confidential, host-unmapped guest, isolated to a particular memory device. Requiring a mapping to get node-placement is quite defeating the point. > 4. How about something like fbind() that takes an fd and offset range > instead of mbind(), which has a prerequisite on mmap()? > This was a consideration - although it has other limitations and larger complexities associated with it. Where does the policy live for random fd's? (inode? address_space?) How is a reclaimed inode's policy handled? (lost forever?) figured i'd start by reducing the scope to the narrowest and clearest use-case, but I did expect to have the fbind() conversation. I'm open to it, but it seems like over-engineering. If you look at tmpfs / shmem, you'll see there is the option for a default filesystem-wide mempolicy that can be plumbed, but i'm not sure there's a real usecase for per-file mempolicies that isn't literally guest_memfd. > This doesn't exist yet, but I'm hoping to discuss this at LPC 2026: > > 5. What if you could pass an fd representing a mount to guest_memfd at > creation time, so to make all the allocations come from a single node > > Step 1: Create a tmpfs mount, specify mpol for mount to MPOL_BIND > Step 2: Get some fd representing the tmpfs mount, hand that to > guest_memfd at creation time > Step 3: guest_memfd allocations will always come from that tmpfs > mount, and abide by that tmpfs mount's memory policy. > This i think this is more feasible than something like fbind, but devil is in the details. I definitely think it's interesting and would love to chat at LPC! > May I know more about the use case behind this new feature? > see above - confidential vm whose memory is quarantined to a particular device, without placing that same burden on the host memory. ~Gregory
On 9/10/26 01:10, Gregory Price wrote: > On Wed, Sep 09, 2026 at 03:41:35PM -0700, Ackerley Tng wrote: >> Gregory Price <gourry@gourry.net> writes: >> >>> guest_memfd allocates its folios through a per-inode shared mempolicy. >>> >>> Today that policy can only be set after the fact, with mbind() on a host >>> mmap of the fd. That requires the fd to be mappable, and it cannot reach >>> folios that are only ever guest-faulted. Neither holds for a >>> non-mappable (confidential) guest_memfd. >>> >>> Add GUEST_MEMFD_FLAG_BIND_NODE and a node field to struct >>> kvm_create_guest_memfd. When set, KVM builds an MPOL_BIND policy for the >>> requested node and installs it over the whole inode, so every folio is >>> allocated there with no userspace mbind(). >>> >> >> Instead of a custom API to ensure all guest_memfd allocations come from >> a single node, how about these options? >> >> 1. Using cgroups/cpuset to constrain allocations (could be troublesome >> if the guest memory is not preallocated, unless the vCPU threads are >> running with the cpuset config) >> >> 2. Process-level NUMA policy >> > > for 1 and 2: > > the intent is to put the guest memory on the target node, not all system > memory for a given process. so the scope here is not the same. > > in fact at that granularity, the desired node may not even have eligible > memory to host the task's memory. > >> 3. Why not request the guest_memfd to be mmap-able just to be able to >> set a memory policy? >> > > the eventual intent is to enable this for fully confidential, > host-unmapped guest, isolated to a particular memory device. > > Requiring a mapping to get node-placement is quite defeating the point. You only need a VMA, not actually mapped/faulted pages. So I don't immediately see the problem? -- Cheers, David
On Thu, Sep 10, 2026 at 01:32:08PM +0200, David Hildenbrand (Arm) wrote: > >> 3. Why not request the guest_memfd to be mmap-able just to be able to > >> set a memory policy? > > > > the eventual intent is to enable this for fully confidential, > > host-unmapped guest, isolated to a particular memory device. > > > > Requiring a mapping to get node-placement is quite defeating the point. > > You only need a VMA, not actually mapped/faulted pages. So I don't immediately > see the problem? > There is no VMA here - only an inode (GMEM_I), which is where the shared policy hangs off of. So yeah, if there was a vma, that's i suppose the missing component needed to hook up userland mempolicy to all of this - but I would have thought creating a VMA for guest_memfd is hacky and confusing (since its intent is to basically not have a VMA). Is there a series I missed that was proposing this? ~Gregory
On 9/10/26 15:39, Gregory Price wrote: > On Thu, Sep 10, 2026 at 01:32:08PM +0200, David Hildenbrand (Arm) wrote: >>> >>> the eventual intent is to enable this for fully confidential, >>> host-unmapped guest, isolated to a particular memory device. >>> >>> Requiring a mapping to get node-placement is quite defeating the point. >> >> You only need a VMA, not actually mapped/faulted pages. So I don't immediately >> see the problem? >> > > There is no VMA here - only an inode (GMEM_I), which is where the > shared policy hangs off of. > > So yeah, if there was a vma, that's i suppose the missing component > needed to hook up userland mempolicy to all of this - but I would have > thought creating a VMA for guest_memfd is hacky and confusing (since its > intent is to basically not have a VMA). The VMA is irrelevant, you just don't want to fault in the pages. In fact, you'd only need the VMA while setting the policy. (similar to shmem) > > Is there a series I missed that was proposing this? I recall that we definitely discussed this in on of our meetings, but I don't remember whether we decided to not fully support this case given that in-place conversation is on the horizon. I think you can open a private-only guest_memfd with GUEST_MEMFD_FLAG_MMAP, and it will reject to fault-in any pages, but GUEST_MEMFD_FLAG_MMAP also changes the way memory pages are obtained: "When the KVM MMU performs a PFN lookup to service a guest fault and the backing guest_memfd has the GUEST_MEMFD_FLAG_MMAP set, then the fault will always be consumed from guest_memfd, regardless of whether it is a shared or a private fault". For shared-only that makes perfect sense. For private-only, where shared memory pages would come from a user VMA, this wouldn't really work. But I don't know how your case would look like (where are shared pages? are there any ever?) -- Cheers, David
Gregory Price <gourry@gourry.net> writes: > On Wed, Sep 09, 2026 at 03:41:35PM -0700, Ackerley Tng wrote: >> Gregory Price <gourry@gourry.net> writes: >> >> > guest_memfd allocates its folios through a per-inode shared mempolicy. >> > >> > Today that policy can only be set after the fact, with mbind() on a host >> > mmap of the fd. That requires the fd to be mappable, and it cannot reach >> > folios that are only ever guest-faulted. Neither holds for a >> > non-mappable (confidential) guest_memfd. >> > >> > Add GUEST_MEMFD_FLAG_BIND_NODE and a node field to struct >> > kvm_create_guest_memfd. When set, KVM builds an MPOL_BIND policy for the >> > requested node and installs it over the whole inode, so every folio is >> > allocated there with no userspace mbind(). >> > >> >> Instead of a custom API to ensure all guest_memfd allocations come from >> a single node, how about these options? >> >> 1. Using cgroups/cpuset to constrain allocations (could be troublesome >> if the guest memory is not preallocated, unless the vCPU threads are >> running with the cpuset config) >> >> 2. Process-level NUMA policy >> > > for 1 and 2: > > the intent is to put the guest memory on the target node, not all system > memory for a given process. so the scope here is not the same. > > in fact at that granularity, the desired node may not even have eligible > memory to host the task's memory. > We have a similar problem where we wanted only guest memory to be charged a certain memcg. Currently guest_memfd memory is charge to mm->owner, which is the thread's parent, so doing the fallocate() in a thread wasn't good enough. The workaround was to have the fallocate() done in a separate process (fork) and have the child process's memcg be set to the desired memcg. This workaround is kind of awkward, but could it technically work for NUMA allocations? >> 3. Why not request the guest_memfd to be mmap-able just to be able to >> set a memory policy? >> > > the eventual intent is to enable this for fully confidential, > host-unmapped guest, isolated to a particular memory device. > > Requiring a mapping to get node-placement is quite defeating the point. > >> 4. How about something like fbind() that takes an fd and offset range >> instead of mbind(), which has a prerequisite on mmap()? >> > > This was a consideration - although it has other limitations and larger > complexities associated with it. > > Where does the policy live for random fd's? (inode? address_space?) > Off the top of my head the policy would be saved at gi->policy, where it's stored for mbind() now. fd+offset is just another way of referencing the offsets to set the policy. > How is a reclaimed inode's policy handled? (lost forever?) > > figured i'd start by reducing the scope to the narrowest and clearest > use-case, but I did expect to have the fbind() conversation. > > I'm open to it, but it seems like over-engineering. > > If you look at tmpfs / shmem, you'll see there is the option for a > default filesystem-wide mempolicy that can be plumbed, but i'm not sure > there's a real usecase for per-file mempolicies that isn't literally > guest_memfd. > I see, can't think of other use-cases for per-file mempolicies either. >> This doesn't exist yet, but I'm hoping to discuss this at LPC 2026: >> >> 5. What if you could pass an fd representing a mount to guest_memfd at >> creation time, so to make all the allocations come from a single node >> >> Step 1: Create a tmpfs mount, specify mpol for mount to MPOL_BIND >> Step 2: Get some fd representing the tmpfs mount, hand that to >> guest_memfd at creation time >> Step 3: guest_memfd allocations will always come from that tmpfs >> mount, and abide by that tmpfs mount's memory policy. >> > > This i think this is more feasible than something like fbind, but devil > is in the details. > > I definitely think it's interesting and would love to chat at LPC! > Great! :) Cya! Along the above lines, for HugeTLBfs we'd do something similar, and the mount's fd would also indicate the HugeTLB page size. Relating to memcg above, this doesn't really help though, since tmpfs doesn't offer a way to configure memcgs. >> May I know more about the use case behind this new feature? >> > > see above - confidential vm whose memory is quarantined to a particular > device, without placing that same burden on the host memory. > Is lazy allocation a requirement for you? (as opposed to fallocate()-ing before the guest touches pages) > ~Gregory
On Wed, Sep 09, 2026 at 04:23:07PM -0700, Ackerley Tng wrote: > Gregory Price <gourry@gourry.net> writes: > > > in fact at that granularity, the desired node may not even have eligible > > memory to host the task's memory. > > > > We have a similar problem where we wanted only guest memory to be > charged a certain memcg. Currently guest_memfd memory is charge to > mm->owner, which is the thread's parent, so doing the fallocate() in a > thread wasn't good enough. The workaround was to have the fallocate() > done in a separate process (fork) and have the child process's memcg be > set to the desired memcg. > > This workaround is kind of awkward, but could it technically work for > NUMA allocations? > That is a very clunky work-around, I'm not sure it's good to formalize that kind of design as an expected user behavior. Whatever the case, the issue of ineligible memory isn't resolved by this. A node may be 100% ZONE_MOVABLE, and later entirely unmapped - literally no eligible memory for kernel allocations for the task itself. > > > > This was a consideration - although it has other limitations and larger > > complexities associated with it. > > > > Where does the policy live for random fd's? (inode? address_space?) > > > > Off the top of my head the policy would be saved at gi->policy, where > it's stored for mbind() now. fd+offset is just another way of > referencing the offsets to set the policy. > It makes sense for anon use cases like this, but for some random file where fbind() means "put unmapped page cache on node X" the story is less clear. > > How is a reclaimed inode's policy handled? (lost forever?) > > > > figured i'd start by reducing the scope to the narrowest and clearest > > use-case, but I did expect to have the fbind() conversation. > > > > I'm open to it, but it seems like over-engineering. > > > > If you look at tmpfs / shmem, you'll see there is the option for a > > default filesystem-wide mempolicy that can be plumbed, but i'm not sure > > there's a real usecase for per-file mempolicies that isn't literally > > guest_memfd. > > > > I see, can't think of other use-cases for per-file mempolicies either. > The only thing i've considered is something like GPU wanting a file faulted directly onto its node - but that's a mapped region, and that's handled by existing semantics. > >> May I know more about the use case behind this new feature? > >> > > > > see above - confidential vm whose memory is quarantined to a particular > > device, without placing that same burden on the host memory. > > > > Is lazy allocation a requirement for you? (as opposed to fallocate()-ing > before the guest touches pages) > Yes, because eventually some kind of overcommit will be desired. May as well build it in from the start. Looking forward to LPC :] ~Gregory
© 2016 - 2026 Red Hat, Inc.