.../admin-guide/kernel-parameters.txt | 2 +- Documentation/filesystems/resctrl.rst | 81 ++++++++++++++ arch/x86/include/asm/cpufeatures.h | 3 +- arch/x86/include/asm/msr-index.h | 2 + arch/x86/kernel/cpu/resctrl/core.c | 101 +++++++++++++++++- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 90 +++++++++++++++- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/rdtgroup.c | 15 ++- arch/x86/kernel/cpu/scattered.c | 2 + fs/resctrl/ctrlmondata.c | 5 +- fs/resctrl/rdtgroup.c | 25 ++++- include/linux/resctrl.h | 3 + 12 files changed, 313 insertions(+), 17 deletions(-)
This series adds resctrl support for two new AMD memory-bandwidth
allocation features:
- GMBA - Global Memory Bandwidth Allocation (hardware name: GLBE).
Bounds DRAM bandwidth for groups of threads that span
multiple L3 QoS domains, rather than being per-L3 like MBA.
- GSMBA - Global Slow Memory Bandwidth Allocation (hardware name:
GLSBE). The CXL.memory / slow-memory counterpart of GMBA,
analogous to how SMBA relates to MBA.
Both features share a new "NPS-node" control domain: a set of QoS (L3)
domains grouped together and aligned to the system's NPS (Nodes Per
Socket) BIOS configuration. Although the control domain is NPS-scoped,
the underlying bandwidth-limit MSRs (MSR_IA32_GMBA_BW_BASE 0xc0000600,
MSR_IA32_GSMBA_BW_BASE 0xc0000680) are instantiated per L3. Programming
a single control domain therefore requires writing the MSR on one CPU
per L3 that the domain spans - a new pattern for resctrl. Patches 2/8
and 3/8 introduce that infrastructure so the new resources can reuse
it.
The features are documented in:
AMD64 Zen6 Platform Quality of Service (PQOS) Extensions,
Publication # 69193 Revision 1.00, Issue Date March 2026
available at https://bugzilla.kernel.org/show_bug.cgi?id=206537
Series overview
---------------
Patches 1-5 to enable GMBA:
1/8 x86,fs/resctrl: Add support for Global Bandwidth Enforcement (GLBE)
2/8 x86/resctrl: Add RESCTRL_NPS_NODE scope for AMD NPS-aligned domains
Add a new ctrl_scope value for resctrl resources whose control
domain spans multiple L3s within an NPS node.
3/8 x86/resctrl: Update control MSRs per L3 for NPS-scoped resources
Add resctrl_arch_update_nps(): builds a cpumask with one CPU per
distinct L3 in the domain, then issues rdt_ctrl_update() via
smp_call_function_many() on that mask. Falls back to the full
domain mask if the scratch masks cannot be built. Route
resctrl_arch_update_domains() and resctrl_arch_reset_all_ctrls()
through this helper when ctrl_scope == RESCTRL_NPS_NODE.
4/8 x86,fs/resctrl: Add the resource for Global Memory Bandwidth Allocation
Register RDT_RESOURCE_GMBA in rdt_resources_all[] with
ctrl_scope=RESCTRL_NPS_NODE and schema_fmt=RANGE, add commands to
discover feature details.
5/8 fs/resctrl: Add the documentation for Global Memory Bandwidth Allocation
Add examples in Documentation/filesystems/resctrl.rst.
Patches 6-8 to enable GSMBA in the same shape:
6/8 x86,fs/resctrl: Add support for Global Slow Memory Bandwidth Allocation
7/8 x86,fs/resctrl: Add the resource for Global Slow Memory Bandwidth Allocation
Register RDT_RESOURCE_GSMBA with ctrl_scope=RESCTRL_NPS_NODE.
8/8 fs/resctrl: Add the documentation for Global Slow Memory Bandwidth Allocation
Add examples in Documentation/filesystems/resctrl.rst.
Changes since v1
----------------
- Earlier sent RFC(v1) with Global Bandwidth Enforcement (GLBE) and
Privilege Level Zero Association (PLZA). This series only handles
Global Memory Bandwidth Allocation. Both the features are sent separately.
- Documentation
* Fixed grammar in the GMBA / GSMBA sections of resctrl.rst.
* Added examples to update GMBA and GSMBA in resctrl.rst documentation.
- Major changes are releated to RESCTRL_NPS_NODE scope handling.
- Commit messages
* Reworked the changelogs in all the patches.
Previous Revisions:
v1 : https://lore.kernel.org/lkml/cover.1769029977.git.babu.moger@amd.com/
---
base-commit: 216fe4b3e06754e73c79a88b1df7e9806e41f29d
Signed-off-by: Babu Moger <babu.moger@amd.com>
Babu Moger (8):
x86,fs/resctrl: Add support for Global Bandwidth Enforcement (GLBE)
x86/resctrl: Add RESCTRL_NPS_NODE scope for AMD NPS-aligned domains
x86/resctrl: Update control MSRs per L3 for NPS-scoped resources
x86,fs/resctrl: Add the resource for Global Bandwidth Allocation
fs/resctrl: Add the documentation for Global Memory Bandwidth
Allocation
x86,fs/resctrl: Add support for Global Slow Memory Bandwidth
Allocation
x86,fs/resctrl: Add the resource for Global Slow Memory Bandwidth
Allocation
fs/resctrl: Add the documentation for Global Slow Memory Bandwidth
Allocation
.../admin-guide/kernel-parameters.txt | 2 +-
Documentation/filesystems/resctrl.rst | 81 ++++++++++++++
arch/x86/include/asm/cpufeatures.h | 3 +-
arch/x86/include/asm/msr-index.h | 2 +
arch/x86/kernel/cpu/resctrl/core.c | 101 +++++++++++++++++-
arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 90 +++++++++++++++-
arch/x86/kernel/cpu/resctrl/internal.h | 1 +
arch/x86/kernel/cpu/resctrl/rdtgroup.c | 15 ++-
arch/x86/kernel/cpu/scattered.c | 2 +
fs/resctrl/ctrlmondata.c | 5 +-
fs/resctrl/rdtgroup.c | 25 ++++-
include/linux/resctrl.h | 3 +
12 files changed, 313 insertions(+), 17 deletions(-)
--
2.43.0
Hi Babu,
On 4/23/26 6:41 PM, Babu Moger wrote:
>
> This series adds resctrl support for two new AMD memory-bandwidth
> allocation features:
>
> - GMBA - Global Memory Bandwidth Allocation (hardware name: GLBE).
> Bounds DRAM bandwidth for groups of threads that span
> multiple L3 QoS domains, rather than being per-L3 like MBA.
>
> - GSMBA - Global Slow Memory Bandwidth Allocation (hardware name:
> GLSBE). The CXL.memory / slow-memory counterpart of GMBA,
> analogous to how SMBA relates to MBA.
>
> Both features share a new "NPS-node" control domain: a set of QoS (L3)
> domains grouped together and aligned to the system's NPS (Nodes Per
> Socket) BIOS configuration. Although the control domain is NPS-scoped,
> the underlying bandwidth-limit MSRs (MSR_IA32_GMBA_BW_BASE 0xc0000600,
> MSR_IA32_GSMBA_BW_BASE 0xc0000680) are instantiated per L3. Programming
> a single control domain therefore requires writing the MSR on one CPU
> per L3 that the domain spans - a new pattern for resctrl. Patches 2/8
> and 3/8 introduce that infrastructure so the new resources can reuse
> it.
>
> The features are documented in:
>
> AMD64 Zen6 Platform Quality of Service (PQOS) Extensions,
> Publication # 69193 Revision 1.00, Issue Date March 2026
>
> available at https://bugzilla.kernel.org/show_bug.cgi?id=206537
>
> Series overview
> ---------------
>
> Patches 1-5 to enable GMBA:
>
> 1/8 x86,fs/resctrl: Add support for Global Bandwidth Enforcement (GLBE)
>
> 2/8 x86/resctrl: Add RESCTRL_NPS_NODE scope for AMD NPS-aligned domains
> Add a new ctrl_scope value for resctrl resources whose control
> domain spans multiple L3s within an NPS node.
>
> 3/8 x86/resctrl: Update control MSRs per L3 for NPS-scoped resources
> Add resctrl_arch_update_nps(): builds a cpumask with one CPU per
> distinct L3 in the domain, then issues rdt_ctrl_update() via
> smp_call_function_many() on that mask. Falls back to the full
> domain mask if the scratch masks cannot be built. Route
> resctrl_arch_update_domains() and resctrl_arch_reset_all_ctrls()
> through this helper when ctrl_scope == RESCTRL_NPS_NODE.
>
> 4/8 x86,fs/resctrl: Add the resource for Global Memory Bandwidth Allocation
> Register RDT_RESOURCE_GMBA in rdt_resources_all[] with
> ctrl_scope=RESCTRL_NPS_NODE and schema_fmt=RANGE, add commands to
> discover feature details.
>
> 5/8 fs/resctrl: Add the documentation for Global Memory Bandwidth Allocation
> Add examples in Documentation/filesystems/resctrl.rst.
>
> Patches 6-8 to enable GSMBA in the same shape:
>
> 6/8 x86,fs/resctrl: Add support for Global Slow Memory Bandwidth Allocation
>
> 7/8 x86,fs/resctrl: Add the resource for Global Slow Memory Bandwidth Allocation
> Register RDT_RESOURCE_GSMBA with ctrl_scope=RESCTRL_NPS_NODE.
>
> 8/8 fs/resctrl: Add the documentation for Global Slow Memory Bandwidth Allocation
> Add examples in Documentation/filesystems/resctrl.rst.
>
> Changes since v1
> ----------------
> - Earlier sent RFC(v1) with Global Bandwidth Enforcement (GLBE) and
> Privilege Level Zero Association (PLZA). This series only handles
> Global Memory Bandwidth Allocation. Both the features are sent separately.
>
> - Documentation
> * Fixed grammar in the GMBA / GSMBA sections of resctrl.rst.
> * Added examples to update GMBA and GSMBA in resctrl.rst documentation.
>
> - Major changes are releated to RESCTRL_NPS_NODE scope handling.
>
> - Commit messages
> * Reworked the changelogs in all the patches.
>
> Previous Revisions:
> v1 : https://lore.kernel.org/lkml/cover.1769029977.git.babu.moger@amd.com/
What are your expectations from this submission? From what I can tell this ignores
v1 feedback in several ways:
- It introduces two new resources, GMBA and GSMBA, when the previous discussion agreed that
these are not actually new resources but instead new controls for the existing MBA/SMBA resources.
- It does not mention or attempt to address dependency on new resource schema descriptions [1]
to support user space in understanding how to interact with the new GMBA/GSMBA controls but
instead defers that to a snippet in the documentation that user space needs to
parse to know this control operates at multiples of 1GB/s.
Apart from ignoring v1 feedback this new version appears to complicate user interface even more
since now it is possible for there to be a single control that may operate at different scopes but from
what I can tell there is nothing that helps user understand whether, for example, domain "0" means
the whole system or a NUMA node?
We have discussed several times now how resctrl interface needs to be enhanced to support
this and other upcoming features from Intel, RISC-V, Arm MPAM, and NVidia. It is thus
unexpected that this submission ignores all the previous discussions.
Since there are so many dependencies on the new schema format support I am prioritizing this
and created a PoC that I am currently refining and hope to share soon. We can collaborate on this
to ensure that it provides a good foundation for the GMBA and GSMBA support.
Consider what I describe in [2] - even in that response I speculate that a "scope" may be needed and
this seems to be case. I believe would help this "NPS = 4" scenario. Adding "scope" to what I shared in
[2] may look like:
info/
└── MB/
└── resource_schemata/
├── GMB/
│ ├── max:4096
│ ├── min:1
│ ├── resolution:1
│ ├── scale:1
│ ├── tolerance:0
│ ├── type:scalar linear
│ ├── scope:NODE
│ └── unit:GBps
└── MB/
├── max:8192
├── min:1
├── resolution:8
├── scale:1
├── tolerance:0
├── type:scalar linear
├── scope:L3
└── unit:GBps
With a "scope" property of the control user space can know what the domain ID in the
schemata file refers to. In above example the "GMB" control has "NODE" scope so user space knows
that a domain ID refers to NUMA node. If the system is "NPS = 4" then the scope could be,
for example, "SYSTEM" (for the lack of a better term) so that user space knows that "0" means
entire system. What do you think?
Also note how the other control properties helps user understand what the schemata file control
values mean. This is what I expected the GMBA/GSMBA enabling to look like ... and you seemed to
agree [3] in v1 discussion. What changed?
Reinette
[1] https://lore.kernel.org/lkml/aPtfMFfLV1l%2FRB0L@e133380.arm.com/
[2] https://lore.kernel.org/lkml/06a237bd-c370-4d3f-99de-124e8c50e711@intel.com/
[3] https://lore.kernel.org/lkml/91d50431-41f3-49d7-a9e6-a3bee2de5162@amd.com/
Hi Reinette, On 4/29/2026 5:34 PM, Reinette Chatre wrote: > Hi Babu, > > On 4/23/26 6:41 PM, Babu Moger wrote: >> >> This series adds resctrl support for two new AMD memory-bandwidth >> allocation features: >> >> - GMBA - Global Memory Bandwidth Allocation (hardware name: GLBE). >> Bounds DRAM bandwidth for groups of threads that span >> multiple L3 QoS domains, rather than being per-L3 like MBA. >> >> - GSMBA - Global Slow Memory Bandwidth Allocation (hardware name: >> GLSBE). The CXL.memory / slow-memory counterpart of GMBA, >> analogous to how SMBA relates to MBA. >> >> Both features share a new "NPS-node" control domain: a set of QoS (L3) >> domains grouped together and aligned to the system's NPS (Nodes Per >> Socket) BIOS configuration. Although the control domain is NPS-scoped, >> the underlying bandwidth-limit MSRs (MSR_IA32_GMBA_BW_BASE 0xc0000600, >> MSR_IA32_GSMBA_BW_BASE 0xc0000680) are instantiated per L3. Programming >> a single control domain therefore requires writing the MSR on one CPU >> per L3 that the domain spans - a new pattern for resctrl. Patches 2/8 >> and 3/8 introduce that infrastructure so the new resources can reuse >> it. >> >> The features are documented in: >> >> AMD64 Zen6 Platform Quality of Service (PQOS) Extensions, >> Publication # 69193 Revision 1.00, Issue Date March 2026 >> >> available at https://bugzilla.kernel.org/show_bug.cgi?id=206537 >> >> Series overview >> --------------- >> >> Patches 1-5 to enable GMBA: >> >> 1/8 x86,fs/resctrl: Add support for Global Bandwidth Enforcement (GLBE) >> >> 2/8 x86/resctrl: Add RESCTRL_NPS_NODE scope for AMD NPS-aligned domains >> Add a new ctrl_scope value for resctrl resources whose control >> domain spans multiple L3s within an NPS node. >> >> 3/8 x86/resctrl: Update control MSRs per L3 for NPS-scoped resources >> Add resctrl_arch_update_nps(): builds a cpumask with one CPU per >> distinct L3 in the domain, then issues rdt_ctrl_update() via >> smp_call_function_many() on that mask. Falls back to the full >> domain mask if the scratch masks cannot be built. Route >> resctrl_arch_update_domains() and resctrl_arch_reset_all_ctrls() >> through this helper when ctrl_scope == RESCTRL_NPS_NODE. >> >> 4/8 x86,fs/resctrl: Add the resource for Global Memory Bandwidth Allocation >> Register RDT_RESOURCE_GMBA in rdt_resources_all[] with >> ctrl_scope=RESCTRL_NPS_NODE and schema_fmt=RANGE, add commands to >> discover feature details. >> >> 5/8 fs/resctrl: Add the documentation for Global Memory Bandwidth Allocation >> Add examples in Documentation/filesystems/resctrl.rst. >> >> Patches 6-8 to enable GSMBA in the same shape: >> >> 6/8 x86,fs/resctrl: Add support for Global Slow Memory Bandwidth Allocation >> >> 7/8 x86,fs/resctrl: Add the resource for Global Slow Memory Bandwidth Allocation >> Register RDT_RESOURCE_GSMBA with ctrl_scope=RESCTRL_NPS_NODE. >> >> 8/8 fs/resctrl: Add the documentation for Global Slow Memory Bandwidth Allocation >> Add examples in Documentation/filesystems/resctrl.rst. >> >> Changes since v1 >> ---------------- >> - Earlier sent RFC(v1) with Global Bandwidth Enforcement (GLBE) and >> Privilege Level Zero Association (PLZA). This series only handles >> Global Memory Bandwidth Allocation. Both the features are sent separately. >> >> - Documentation >> * Fixed grammar in the GMBA / GSMBA sections of resctrl.rst. >> * Added examples to update GMBA and GSMBA in resctrl.rst documentation. >> >> - Major changes are releated to RESCTRL_NPS_NODE scope handling. >> >> - Commit messages >> * Reworked the changelogs in all the patches. >> >> Previous Revisions: >> v1 : https://lore.kernel.org/lkml/cover.1769029977.git.babu.moger@amd.com/ > > What are your expectations from this submission? From what I can tell this ignores > v1 feedback in several ways: > - It introduces two new resources, GMBA and GSMBA, when the previous discussion agreed that > these are not actually new resources but instead new controls for the existing MBA/SMBA resources. > - It does not mention or attempt to address dependency on new resource schema descriptions [1] > to support user space in understanding how to interact with the new GMBA/GSMBA controls but > instead defers that to a snippet in the documentation that user space needs to > parse to know this control operates at multiples of 1GB/s. > > Apart from ignoring v1 feedback this new version appears to complicate user interface even more > since now it is possible for there to be a single control that may operate at different scopes but from > what I can tell there is nothing that helps user understand whether, for example, domain "0" means > the whole system or a NUMA node? > > We have discussed several times now how resctrl interface needs to be enhanced to support > this and other upcoming features from Intel, RISC-V, Arm MPAM, and NVidia. It is thus > unexpected that this submission ignores all the previous discussions. I think there may be some misunderstanding on this topic. Yes, we discussed it earlier. It depends on other requirements (region-aware aspects), so I assumed it would be handled by someone with full context and addressed as a separate feature. I didn’t have complete visibility into all the requirements. > > Since there are so many dependencies on the new schema format support I am prioritizing this > and created a PoC that I am currently refining and hope to share soon. We can collaborate on this > to ensure that it provides a good foundation for the GMBA and GSMBA support. That is good to know. Let me know when you are ready. Could you please share which parts of the feature (e.g., Part 1, Part 2, etc.) you are planning to cover in your PoC? > > Consider what I describe in [2] - even in that response I speculate that a "scope" may be needed and > this seems to be case. I believe would help this "NPS = 4" scenario. Adding "scope" to what I shared in > [2] may look like: > > info/ > └── MB/ > └── resource_schemata/ > ├── GMB/ > │ ├── max:4096 > │ ├── min:1 > │ ├── resolution:1 > │ ├── scale:1 > │ ├── tolerance:0 > │ ├── type:scalar linear > │ ├── scope:NODE > │ └── unit:GBps > └── MB/ > ├── max:8192 > ├── min:1 > ├── resolution:8 > ├── scale:1 > ├── tolerance:0 > ├── type:scalar linear > ├── scope:L3 > └── unit:GBps > > With a "scope" property of the control user space can know what the domain ID in the > schemata file refers to. In above example the "GMB" control has "NODE" scope so user space knows > that a domain ID refers to NUMA node. If the system is "NPS = 4" then the scope could be, > for example, "SYSTEM" (for the lack of a better term) so that user space knows that "0" means > entire system. What do you think? Yes. Sound good to me. > > Also note how the other control properties helps user understand what the schemata file control > values mean. This is what I expected the GMBA/GSMBA enabling to look like ... and you seemed to > agree [3] in v1 discussion. What changed? Again, I was under the impression that this would be handled as a separate feature and patch series. Thanks Babu
Hi Babu, On 4/30/26 4:04 PM, Moger, Babu wrote: > Hi Reinette, > > On 4/29/2026 5:34 PM, Reinette Chatre wrote: >> Hi Babu, >> >> On 4/23/26 6:41 PM, Babu Moger wrote: >>> >>> This series adds resctrl support for two new AMD memory-bandwidth >>> allocation features: >>> >>> - GMBA - Global Memory Bandwidth Allocation (hardware name: GLBE). >>> Bounds DRAM bandwidth for groups of threads that span >>> multiple L3 QoS domains, rather than being per-L3 like MBA. >>> >>> - GSMBA - Global Slow Memory Bandwidth Allocation (hardware name: >>> GLSBE). The CXL.memory / slow-memory counterpart of GMBA, >>> analogous to how SMBA relates to MBA. >>> >>> Both features share a new "NPS-node" control domain: a set of QoS (L3) >>> domains grouped together and aligned to the system's NPS (Nodes Per >>> Socket) BIOS configuration. Although the control domain is NPS-scoped, >>> the underlying bandwidth-limit MSRs (MSR_IA32_GMBA_BW_BASE 0xc0000600, >>> MSR_IA32_GSMBA_BW_BASE 0xc0000680) are instantiated per L3. Programming >>> a single control domain therefore requires writing the MSR on one CPU >>> per L3 that the domain spans - a new pattern for resctrl. Patches 2/8 >>> and 3/8 introduce that infrastructure so the new resources can reuse >>> it. >>> >>> The features are documented in: >>> >>> AMD64 Zen6 Platform Quality of Service (PQOS) Extensions, >>> Publication # 69193 Revision 1.00, Issue Date March 2026 >>> >>> available at https://bugzilla.kernel.org/show_bug.cgi?id=206537 >>> >>> Series overview >>> --------------- >>> >>> Patches 1-5 to enable GMBA: >>> >>> 1/8 x86,fs/resctrl: Add support for Global Bandwidth Enforcement (GLBE) >>> >>> 2/8 x86/resctrl: Add RESCTRL_NPS_NODE scope for AMD NPS-aligned domains >>> Add a new ctrl_scope value for resctrl resources whose control >>> domain spans multiple L3s within an NPS node. >>> >>> 3/8 x86/resctrl: Update control MSRs per L3 for NPS-scoped resources >>> Add resctrl_arch_update_nps(): builds a cpumask with one CPU per >>> distinct L3 in the domain, then issues rdt_ctrl_update() via >>> smp_call_function_many() on that mask. Falls back to the full >>> domain mask if the scratch masks cannot be built. Route >>> resctrl_arch_update_domains() and resctrl_arch_reset_all_ctrls() >>> through this helper when ctrl_scope == RESCTRL_NPS_NODE. >>> >>> 4/8 x86,fs/resctrl: Add the resource for Global Memory Bandwidth Allocation >>> Register RDT_RESOURCE_GMBA in rdt_resources_all[] with >>> ctrl_scope=RESCTRL_NPS_NODE and schema_fmt=RANGE, add commands to >>> discover feature details. >>> >>> 5/8 fs/resctrl: Add the documentation for Global Memory Bandwidth Allocation >>> Add examples in Documentation/filesystems/resctrl.rst. >>> >>> Patches 6-8 to enable GSMBA in the same shape: >>> >>> 6/8 x86,fs/resctrl: Add support for Global Slow Memory Bandwidth Allocation >>> >>> 7/8 x86,fs/resctrl: Add the resource for Global Slow Memory Bandwidth Allocation >>> Register RDT_RESOURCE_GSMBA with ctrl_scope=RESCTRL_NPS_NODE. >>> >>> 8/8 fs/resctrl: Add the documentation for Global Slow Memory Bandwidth Allocation >>> Add examples in Documentation/filesystems/resctrl.rst. >>> >>> Changes since v1 >>> ---------------- >>> - Earlier sent RFC(v1) with Global Bandwidth Enforcement (GLBE) and >>> Privilege Level Zero Association (PLZA). This series only handles >>> Global Memory Bandwidth Allocation. Both the features are sent separately. >>> >>> - Documentation >>> * Fixed grammar in the GMBA / GSMBA sections of resctrl.rst. >>> * Added examples to update GMBA and GSMBA in resctrl.rst documentation. >>> >>> - Major changes are releated to RESCTRL_NPS_NODE scope handling. >>> >>> - Commit messages >>> * Reworked the changelogs in all the patches. >>> >>> Previous Revisions: >>> v1 : https://lore.kernel.org/lkml/cover.1769029977.git.babu.moger@amd.com/ >> >> What are your expectations from this submission? From what I can tell this ignores >> v1 feedback in several ways: >> - It introduces two new resources, GMBA and GSMBA, when the previous discussion agreed that >> these are not actually new resources but instead new controls for the existing MBA/SMBA resources. >> - It does not mention or attempt to address dependency on new resource schema descriptions [1] >> to support user space in understanding how to interact with the new GMBA/GSMBA controls but >> instead defers that to a snippet in the documentation that user space needs to >> parse to know this control operates at multiples of 1GB/s. >> >> Apart from ignoring v1 feedback this new version appears to complicate user interface even more >> since now it is possible for there to be a single control that may operate at different scopes but from >> what I can tell there is nothing that helps user understand whether, for example, domain "0" means >> the whole system or a NUMA node? >> >> We have discussed several times now how resctrl interface needs to be enhanced to support >> this and other upcoming features from Intel, RISC-V, Arm MPAM, and NVidia. It is thus >> unexpected that this submission ignores all the previous discussions. > > I think there may be some misunderstanding on this topic. > > Yes, we discussed it earlier. It depends on other requirements (region-aware aspects), so I assumed it would be handled by someone with full context and addressed as a separate feature. I didn’t have complete visibility into all the requirements. Please read https://lore.kernel.org/lkml/06a237bd-c370-4d3f-99de-124e8c50e711@intel.com/ again. You should have complete visibility into the foundation of this work since one of the primary goals is to address the resctrl interface breakage that came with the initial AMD support for MBA that resctrl has been living with until now. With this series you completely disregard attempts to support users in understanding how to interact with the schemata file and instead introduce *another* obfuscated control. I will not support this. Also, no, this does not depend on region-aware work. Needing to support multiple controls for a single resource is independent from region-aware. >> Since there are so many dependencies on the new schema format support I am prioritizing this >> and created a PoC that I am currently refining and hope to share soon. We can collaborate on this >> to ensure that it provides a good foundation for the GMBA and GSMBA support. > > That is good to know. Let me know when you are ready. > > Could you please share which parts of the feature (e.g., Part 1, Part 2, etc.) you are planning to cover in your PoC? All three parts mentioned in https://lore.kernel.org/lkml/06a237bd-c370-4d3f-99de-124e8c50e711@intel.com/ This does not address all the features discussed, for example it does not support emulated controls, but I hope it is enough of a foundation to build on. >> Consider what I describe in [2] - even in that response I speculate that a "scope" may be needed and >> this seems to be case. I believe would help this "NPS = 4" scenario. Adding "scope" to what I shared in >> [2] may look like: >> >> info/ >> └── MB/ >> └── resource_schemata/ >> ├── GMB/ >> │ ├── max:4096 >> │ ├── min:1 >> │ ├── resolution:1 >> │ ├── scale:1 >> │ ├── tolerance:0 >> │ ├── type:scalar linear >> │ ├── scope:NODE >> │ └── unit:GBps >> └── MB/ >> ├── max:8192 >> ├── min:1 >> ├── resolution:8 >> ├── scale:1 >> ├── tolerance:0 >> ├── type:scalar linear >> ├── scope:L3 >> └── unit:GBps >> >> With a "scope" property of the control user space can know what the domain ID in the >> schemata file refers to. In above example the "GMB" control has "NODE" scope so user space knows >> that a domain ID refers to NUMA node. If the system is "NPS = 4" then the scope could be, >> for example, "SYSTEM" (for the lack of a better term) so that user space knows that "0" means >> entire system. What do you think? > > Yes. Sound good to me. > >> >> Also note how the other control properties helps user understand what the schemata file control >> values mean. This is what I expected the GMBA/GSMBA enabling to look like ... and you seemed to >> agree [3] in v1 discussion. What changed? > > Again, I was under the impression that this would be handled as a separate feature and patch series. Right. And since GMBA/GSMBA depends on it this submission without resolving this this dependency first is unexpected. Reinette
Hi Reinette, On 4/30/2026 6:40 PM, Reinette Chatre wrote: > Hi Babu, > > On 4/30/26 4:04 PM, Moger, Babu wrote: >> Hi Reinette, >> >> On 4/29/2026 5:34 PM, Reinette Chatre wrote: >>> Hi Babu, >>> >>> On 4/23/26 6:41 PM, Babu Moger wrote: >>>> >>>> This series adds resctrl support for two new AMD memory-bandwidth >>>> allocation features: >>>> >>>> - GMBA - Global Memory Bandwidth Allocation (hardware name: GLBE). >>>> Bounds DRAM bandwidth for groups of threads that span >>>> multiple L3 QoS domains, rather than being per-L3 like MBA. >>>> >>>> - GSMBA - Global Slow Memory Bandwidth Allocation (hardware name: >>>> GLSBE). The CXL.memory / slow-memory counterpart of GMBA, >>>> analogous to how SMBA relates to MBA. >>>> >>>> Both features share a new "NPS-node" control domain: a set of QoS (L3) >>>> domains grouped together and aligned to the system's NPS (Nodes Per >>>> Socket) BIOS configuration. Although the control domain is NPS-scoped, >>>> the underlying bandwidth-limit MSRs (MSR_IA32_GMBA_BW_BASE 0xc0000600, >>>> MSR_IA32_GSMBA_BW_BASE 0xc0000680) are instantiated per L3. Programming >>>> a single control domain therefore requires writing the MSR on one CPU >>>> per L3 that the domain spans - a new pattern for resctrl. Patches 2/8 >>>> and 3/8 introduce that infrastructure so the new resources can reuse >>>> it. >>>> >>>> The features are documented in: >>>> >>>> AMD64 Zen6 Platform Quality of Service (PQOS) Extensions, >>>> Publication # 69193 Revision 1.00, Issue Date March 2026 >>>> >>>> available at https://bugzilla.kernel.org/show_bug.cgi?id=206537 >>>> >>>> Series overview >>>> --------------- >>>> >>>> Patches 1-5 to enable GMBA: >>>> >>>> 1/8 x86,fs/resctrl: Add support for Global Bandwidth Enforcement (GLBE) >>>> >>>> 2/8 x86/resctrl: Add RESCTRL_NPS_NODE scope for AMD NPS-aligned domains >>>> Add a new ctrl_scope value for resctrl resources whose control >>>> domain spans multiple L3s within an NPS node. >>>> >>>> 3/8 x86/resctrl: Update control MSRs per L3 for NPS-scoped resources >>>> Add resctrl_arch_update_nps(): builds a cpumask with one CPU per >>>> distinct L3 in the domain, then issues rdt_ctrl_update() via >>>> smp_call_function_many() on that mask. Falls back to the full >>>> domain mask if the scratch masks cannot be built. Route >>>> resctrl_arch_update_domains() and resctrl_arch_reset_all_ctrls() >>>> through this helper when ctrl_scope == RESCTRL_NPS_NODE. >>>> >>>> 4/8 x86,fs/resctrl: Add the resource for Global Memory Bandwidth Allocation >>>> Register RDT_RESOURCE_GMBA in rdt_resources_all[] with >>>> ctrl_scope=RESCTRL_NPS_NODE and schema_fmt=RANGE, add commands to >>>> discover feature details. >>>> >>>> 5/8 fs/resctrl: Add the documentation for Global Memory Bandwidth Allocation >>>> Add examples in Documentation/filesystems/resctrl.rst. >>>> >>>> Patches 6-8 to enable GSMBA in the same shape: >>>> >>>> 6/8 x86,fs/resctrl: Add support for Global Slow Memory Bandwidth Allocation >>>> >>>> 7/8 x86,fs/resctrl: Add the resource for Global Slow Memory Bandwidth Allocation >>>> Register RDT_RESOURCE_GSMBA with ctrl_scope=RESCTRL_NPS_NODE. >>>> >>>> 8/8 fs/resctrl: Add the documentation for Global Slow Memory Bandwidth Allocation >>>> Add examples in Documentation/filesystems/resctrl.rst. >>>> >>>> Changes since v1 >>>> ---------------- >>>> - Earlier sent RFC(v1) with Global Bandwidth Enforcement (GLBE) and >>>> Privilege Level Zero Association (PLZA). This series only handles >>>> Global Memory Bandwidth Allocation. Both the features are sent separately. >>>> >>>> - Documentation >>>> * Fixed grammar in the GMBA / GSMBA sections of resctrl.rst. >>>> * Added examples to update GMBA and GSMBA in resctrl.rst documentation. >>>> >>>> - Major changes are releated to RESCTRL_NPS_NODE scope handling. >>>> >>>> - Commit messages >>>> * Reworked the changelogs in all the patches. >>>> >>>> Previous Revisions: >>>> v1 : https://lore.kernel.org/lkml/cover.1769029977.git.babu.moger@amd.com/ >>> >>> What are your expectations from this submission? From what I can tell this ignores >>> v1 feedback in several ways: >>> - It introduces two new resources, GMBA and GSMBA, when the previous discussion agreed that >>> these are not actually new resources but instead new controls for the existing MBA/SMBA resources. >>> - It does not mention or attempt to address dependency on new resource schema descriptions [1] >>> to support user space in understanding how to interact with the new GMBA/GSMBA controls but >>> instead defers that to a snippet in the documentation that user space needs to >>> parse to know this control operates at multiples of 1GB/s. >>> >>> Apart from ignoring v1 feedback this new version appears to complicate user interface even more >>> since now it is possible for there to be a single control that may operate at different scopes but from >>> what I can tell there is nothing that helps user understand whether, for example, domain "0" means >>> the whole system or a NUMA node? >>> >>> We have discussed several times now how resctrl interface needs to be enhanced to support >>> this and other upcoming features from Intel, RISC-V, Arm MPAM, and NVidia. It is thus >>> unexpected that this submission ignores all the previous discussions. >> >> I think there may be some misunderstanding on this topic. >> >> Yes, we discussed it earlier. It depends on other requirements (region-aware aspects), so I assumed it would be handled by someone with full context and addressed as a separate feature. I didn’t have complete visibility into all the requirements. > > Please read https://lore.kernel.org/lkml/06a237bd-c370-4d3f-99de-124e8c50e711@intel.com/ again. > > You should have complete visibility into the foundation of this work since one of the > primary goals is to address the resctrl interface breakage that came with the initial AMD > support for MBA that resctrl has been living with until now. > > With this series you completely disregard attempts to support users in understanding > how to interact with the schemata file and instead introduce *another* obfuscated control. I > will not support this. > > Also, no, this does not depend on region-aware work. Needing to support multiple controls for > a single resource is independent from region-aware. > >>> Since there are so many dependencies on the new schema format support I am prioritizing this >>> and created a PoC that I am currently refining and hope to share soon. We can collaborate on this >>> to ensure that it provides a good foundation for the GMBA and GSMBA support. >> >> That is good to know. Let me know when you are ready. >> >> Could you please share which parts of the feature (e.g., Part 1, Part 2, etc.) you are planning to cover in your PoC? > > All three parts mentioned in https://lore.kernel.org/lkml/06a237bd-c370-4d3f-99de-124e8c50e711@intel.com/ > > This does not address all the features discussed, for example it does not support emulated controls, > but I hope it is enough of a foundation to build on. Please share your code when you are ready. I can build GMB and GSMBA on top of your patches. Hopefully, I can reuse some of the code from this series. Thanks Babu
Hi Reinette, On 5/1/2026 9:38 AM, Moger, Babu wrote: > Hi Reinette, > > On 4/30/2026 6:40 PM, Reinette Chatre wrote: >> Hi Babu, >> >> On 4/30/26 4:04 PM, Moger, Babu wrote: >>> Hi Reinette, >>> >>> On 4/29/2026 5:34 PM, Reinette Chatre wrote: >>>> Hi Babu, >>>> >>>> On 4/23/26 6:41 PM, Babu Moger wrote: >>>>> >>>>> This series adds resctrl support for two new AMD memory-bandwidth >>>>> allocation features: >>>>> >>>>> - GMBA - Global Memory Bandwidth Allocation (hardware name: >>>>> GLBE). >>>>> Bounds DRAM bandwidth for groups of threads that span >>>>> multiple L3 QoS domains, rather than being per-L3 >>>>> like MBA. >>>>> >>>>> - GSMBA - Global Slow Memory Bandwidth Allocation (hardware name: >>>>> GLSBE). The CXL.memory / slow-memory counterpart of >>>>> GMBA, >>>>> analogous to how SMBA relates to MBA. >>>>> >>>>> Both features share a new "NPS-node" control domain: a set of QoS (L3) >>>>> domains grouped together and aligned to the system's NPS (Nodes Per >>>>> Socket) BIOS configuration. Although the control domain is NPS-scoped, >>>>> the underlying bandwidth-limit MSRs (MSR_IA32_GMBA_BW_BASE 0xc0000600, >>>>> MSR_IA32_GSMBA_BW_BASE 0xc0000680) are instantiated per L3. >>>>> Programming >>>>> a single control domain therefore requires writing the MSR on one CPU >>>>> per L3 that the domain spans - a new pattern for resctrl. Patches 2/8 >>>>> and 3/8 introduce that infrastructure so the new resources can reuse >>>>> it. >>>>> >>>>> The features are documented in: >>>>> >>>>> AMD64 Zen6 Platform Quality of Service (PQOS) Extensions, >>>>> Publication # 69193 Revision 1.00, Issue Date March 2026 >>>>> >>>>> available at https://bugzilla.kernel.org/show_bug.cgi?id=206537 >>>>> >>>>> Series overview >>>>> --------------- >>>>> >>>>> Patches 1-5 to enable GMBA: >>>>> >>>>> 1/8 x86,fs/resctrl: Add support for Global Bandwidth >>>>> Enforcement (GLBE) >>>>> >>>>> 2/8 x86/resctrl: Add RESCTRL_NPS_NODE scope for AMD NPS- >>>>> aligned domains >>>>> Add a new ctrl_scope value for resctrl resources whose >>>>> control >>>>> domain spans multiple L3s within an NPS node. >>>>> >>>>> 3/8 x86/resctrl: Update control MSRs per L3 for NPS-scoped >>>>> resources >>>>> Add resctrl_arch_update_nps(): builds a cpumask with one >>>>> CPU per >>>>> distinct L3 in the domain, then issues rdt_ctrl_update() via >>>>> smp_call_function_many() on that mask. Falls back to the full >>>>> domain mask if the scratch masks cannot be built. Route >>>>> resctrl_arch_update_domains() and >>>>> resctrl_arch_reset_all_ctrls() >>>>> through this helper when ctrl_scope == RESCTRL_NPS_NODE. >>>>> >>>>> 4/8 x86,fs/resctrl: Add the resource for Global Memory >>>>> Bandwidth Allocation >>>>> Register RDT_RESOURCE_GMBA in rdt_resources_all[] with >>>>> ctrl_scope=RESCTRL_NPS_NODE and schema_fmt=RANGE, add >>>>> commands to >>>>> discover feature details. >>>>> >>>>> 5/8 fs/resctrl: Add the documentation for Global Memory >>>>> Bandwidth Allocation >>>>> Add examples in Documentation/filesystems/resctrl.rst. >>>>> >>>>> Patches 6-8 to enable GSMBA in the same shape: >>>>> >>>>> 6/8 x86,fs/resctrl: Add support for Global Slow Memory >>>>> Bandwidth Allocation >>>>> >>>>> 7/8 x86,fs/resctrl: Add the resource for Global Slow Memory >>>>> Bandwidth Allocation >>>>> Register RDT_RESOURCE_GSMBA with ctrl_scope=RESCTRL_NPS_NODE. >>>>> >>>>> 8/8 fs/resctrl: Add the documentation for Global Slow Memory >>>>> Bandwidth Allocation >>>>> Add examples in Documentation/filesystems/resctrl.rst. >>>>> >>>>> Changes since v1 >>>>> ---------------- >>>>> - Earlier sent RFC(v1) with Global Bandwidth Enforcement (GLBE) >>>>> and >>>>> Privilege Level Zero Association (PLZA). This series only >>>>> handles >>>>> Global Memory Bandwidth Allocation. Both the features are >>>>> sent separately. >>>>> >>>>> - Documentation >>>>> * Fixed grammar in the GMBA / GSMBA sections of resctrl.rst. >>>>> * Added examples to update GMBA and GSMBA in resctrl.rst >>>>> documentation. >>>>> >>>>> - Major changes are releated to RESCTRL_NPS_NODE scope handling. >>>>> >>>>> - Commit messages >>>>> * Reworked the changelogs in all the patches. >>>>> >>>>> Previous Revisions: >>>>> v1 : https://lore.kernel.org/lkml/ >>>>> cover.1769029977.git.babu.moger@amd.com/ >>>> >>>> What are your expectations from this submission? From what I can >>>> tell this ignores >>>> v1 feedback in several ways: >>>> - It introduces two new resources, GMBA and GSMBA, when the previous >>>> discussion agreed that >>>> these are not actually new resources but instead new controls >>>> for the existing MBA/SMBA resources. >>>> - It does not mention or attempt to address dependency on new >>>> resource schema descriptions [1] >>>> to support user space in understanding how to interact with the >>>> new GMBA/GSMBA controls but >>>> instead defers that to a snippet in the documentation that user >>>> space needs to >>>> parse to know this control operates at multiples of 1GB/s. >>>> >>>> Apart from ignoring v1 feedback this new version appears to >>>> complicate user interface even more >>>> since now it is possible for there to be a single control that may >>>> operate at different scopes but from >>>> what I can tell there is nothing that helps user understand whether, >>>> for example, domain "0" means >>>> the whole system or a NUMA node? >>>> >>>> We have discussed several times now how resctrl interface needs to >>>> be enhanced to support >>>> this and other upcoming features from Intel, RISC-V, Arm MPAM, and >>>> NVidia. It is thus >>>> unexpected that this submission ignores all the previous discussions. >>> >>> I think there may be some misunderstanding on this topic. >>> >>> Yes, we discussed it earlier. It depends on other requirements >>> (region-aware aspects), so I assumed it would be handled by someone >>> with full context and addressed as a separate feature. I didn’t have >>> complete visibility into all the requirements. >> >> Please read https://lore.kernel.org/lkml/06a237bd- >> c370-4d3f-99de-124e8c50e711@intel.com/ again. >> >> You should have complete visibility into the foundation of this work >> since one of the >> primary goals is to address the resctrl interface breakage that came >> with the initial AMD >> support for MBA that resctrl has been living with until now. >> >> With this series you completely disregard attempts to support users in >> understanding >> how to interact with the schemata file and instead introduce *another* >> obfuscated control. I >> will not support this. >> >> Also, no, this does not depend on region-aware work. Needing to >> support multiple controls for >> a single resource is independent from region-aware. >> >>>> Since there are so many dependencies on the new schema format >>>> support I am prioritizing this >>>> and created a PoC that I am currently refining and hope to share >>>> soon. We can collaborate on this >>>> to ensure that it provides a good foundation for the GMBA and GSMBA >>>> support. >>> >>> That is good to know. Let me know when you are ready. >>> >>> Could you please share which parts of the feature (e.g., Part 1, Part >>> 2, etc.) you are planning to cover in your PoC? >> >> All three parts mentioned in https://lore.kernel.org/lkml/06a237bd- >> c370-4d3f-99de-124e8c50e711@intel.com/ >> >> This does not address all the features discussed, for example it does >> not support emulated controls, >> but I hope it is enough of a foundation to build on. > > Please share your code when you are ready. I can build GMB and GSMBA on > top of your patches. Hopefully, I can reuse some of the code from this > series. I didn’t see your acknowledgment on my previous note, so I wanted to follow up to ensure we’re aligned. Just to confirm—are you planning to share your PoC? My understanding is that I would build GMB/GSMBA on top of your patches. Please let me know if that’s correct. There’s no urgency on the patches at this point; I mainly wanted to get some clarity on the plan. Thanks, Babu
Hi Babu, On 5/15/26 8:31 AM, Moger, Babu wrote: > On 5/1/2026 9:38 AM, Moger, Babu wrote: >> On 4/30/2026 6:40 PM, Reinette Chatre wrote: >>>>> Since there are so many dependencies on the new schema format support I am prioritizing this >>>>> and created a PoC that I am currently refining and hope to share soon. We can collaborate on this >>>>> to ensure that it provides a good foundation for the GMBA and GSMBA support. Above is comment from me indicating plans to share the PoC and goal to have it provide a foundation for GMBA and GSMBA. >>>> >>>> That is good to know. Let me know when you are ready. >>>> >>>> Could you please share which parts of the feature (e.g., Part 1, Part 2, etc.) you are planning to cover in your PoC? >>> >>> All three parts mentioned in https://lore.kernel.org/lkml/06a237bd- c370-4d3f-99de-124e8c50e711@intel.com/ >>> >>> This does not address all the features discussed, for example it does not support emulated controls, >>> but I hope it is enough of a foundation to build on. >> >> Please share your code when you are ready. I can build GMB and GSMBA on top of your patches. Hopefully, I can reuse some of the code from this series. > > I didn’t see your acknowledgment on my previous note, so I wanted to follow up to ensure we’re aligned. I did not think a response was necessary since it essentially rephrased my earlier comment and did not contain a question. > Just to confirm—are you planning to share your PoC? Yes. The fixes needed in existing resctrl code are taking higher priority though. > > My understanding is that I would build GMB/GSMBA on top of your patches. Please let me know if that’s correct. That is my understanding also. > > There’s no urgency on the patches at this point; I mainly wanted to get some clarity on the plan. Reinette
Hi Reinette, On 5/15/2026 11:35 AM, Reinette Chatre wrote: > Hi Babu, > > On 5/15/26 8:31 AM, Moger, Babu wrote: >> On 5/1/2026 9:38 AM, Moger, Babu wrote: >>> On 4/30/2026 6:40 PM, Reinette Chatre wrote: > >>>>>> Since there are so many dependencies on the new schema format support I am prioritizing this >>>>>> and created a PoC that I am currently refining and hope to share soon. We can collaborate on this >>>>>> to ensure that it provides a good foundation for the GMBA and GSMBA support. > > Above is comment from me indicating plans to share the PoC and goal to have it provide a foundation > for GMBA and GSMBA. ok. > >>>>> >>>>> That is good to know. Let me know when you are ready. >>>>> >>>>> Could you please share which parts of the feature (e.g., Part 1, Part 2, etc.) you are planning to cover in your PoC? >>>> >>>> All three parts mentioned in https://lore.kernel.org/lkml/06a237bd- c370-4d3f-99de-124e8c50e711@intel.com/ >>>> >>>> This does not address all the features discussed, for example it does not support emulated controls, >>>> but I hope it is enough of a foundation to build on. >>> >>> Please share your code when you are ready. I can build GMB and GSMBA on top of your patches. Hopefully, I can reuse some of the code from this series. >> >> I didn’t see your acknowledgment on my previous note, so I wanted to follow up to ensure we’re aligned. > > I did not think a response was necessary since it essentially rephrased my earlier comment and did not contain > a question. > >> Just to confirm—are you planning to share your PoC? > > Yes. The fixes needed in existing resctrl code are taking higher priority though. > Ok. Sure. >> >> My understanding is that I would build GMB/GSMBA on top of your patches. Please let me know if that’s correct. > > That is my understanding also. Thanks for the confirmation. -Babu
© 2016 - 2026 Red Hat, Inc.