[RFC PATCH 00/36] arm_mpam: Add basic mpam driver

James Morse posted 36 patches 2 months, 3 weeks ago
There is a newer version of this series
.../devicetree/bindings/arm/arm,mpam-msc.yaml |  227 ++
MAINTAINERS                                   |    6 +-
arch/arm64/Kconfig                            |   19 +
arch/arm64/include/asm/cache.h                |   17 +
drivers/acpi/arm64/Kconfig                    |    3 +
drivers/acpi/arm64/Makefile                   |    1 +
drivers/acpi/arm64/mpam.c                     |  365 +++
drivers/acpi/pptt.c                           |  240 +-
drivers/acpi/tables.c                         |    2 +-
drivers/base/cacheinfo.c                      |   57 +
drivers/platform/arm64/Kconfig                |   73 +-
drivers/platform/arm64/Makefile               |   10 +-
drivers/platform/arm64/ec/Kconfig             |   73 +
drivers/platform/arm64/ec/Makefile            |   10 +
.../platform/arm64/{ => ec}/acer-aspire1-ec.c |    0
.../arm64/{ => ec}/huawei-gaokun-ec.c         |    0
.../arm64/{ => ec}/lenovo-yoga-c630.c         |    0
drivers/platform/arm64/mpam/Kconfig           |   23 +
drivers/platform/arm64/mpam/Makefile          |    4 +
drivers/platform/arm64/mpam/mpam_devices.c    | 2910 +++++++++++++++++
drivers/platform/arm64/mpam/mpam_internal.h   |  697 ++++
.../platform/arm64/mpam/test_mpam_devices.c   |  390 +++
include/linux/acpi.h                          |   17 +
include/linux/arm_mpam.h                      |   56 +
include/linux/cacheinfo.h                     |    1 +
25 files changed, 5117 insertions(+), 84 deletions(-)
create mode 100644 Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml
create mode 100644 drivers/acpi/arm64/mpam.c
create mode 100644 drivers/platform/arm64/ec/Kconfig
create mode 100644 drivers/platform/arm64/ec/Makefile
rename drivers/platform/arm64/{ => ec}/acer-aspire1-ec.c (100%)
rename drivers/platform/arm64/{ => ec}/huawei-gaokun-ec.c (100%)
rename drivers/platform/arm64/{ => ec}/lenovo-yoga-c630.c (100%)
create mode 100644 drivers/platform/arm64/mpam/Kconfig
create mode 100644 drivers/platform/arm64/mpam/Makefile
create mode 100644 drivers/platform/arm64/mpam/mpam_devices.c
create mode 100644 drivers/platform/arm64/mpam/mpam_internal.h
create mode 100644 drivers/platform/arm64/mpam/test_mpam_devices.c
create mode 100644 include/linux/arm_mpam.h
[RFC PATCH 00/36] arm_mpam: Add basic mpam driver
Posted by James Morse 2 months, 3 weeks ago
Hello,

This is just enough MPAM driver for the ACPI and DT pre-requisites.
It doesn't contain any of the resctrl code, meaning you can't actually drive it
from user-space yet.

This is the initial group of patches that allows the resctrl code to be built
on top. Including that will increase the number of trees that may need to
coordinate, so breaking it up make sense.

The locking looks very strange - but is influenced by the 'mpam-fb' firmware
interface specification that is still alpha. That thing needs to wait for an
interrupt after every system register write, which significantly impacts the
driver. Some features just won't work, e.g. reading the monitor registers via
perf.
The aim is to not have to make invasive changes to the locking to support the
firmware interface, hence it looks strange from day-1.

I've not found a platform that can test all the behaviours around the monitors,
so this is where I'd expect the most bugs.

It's unclear where in the tree this should be put. It affects memory bandwidth
and cache allocation, but doesn't (yet) interact with perf. The main interaction
is with resctrl in fs/resctrl - but there will be no filesystem code in here.
Its also likely there will be other in-kernel users. (in-kernel MSC emulation by
KVM being an obvious example).
(I'm not a fan of drivers/resctrl or drivers/mpam - its not the sort of thing
 that justifies being a 'subsystem'.)

For now, I've put this under drivers/platform/arm64. Other ideas welcome.

The first three patches are currently a series on the list, the PPTT stuff
has previously been posted - this is where the users of those helpers appear.


The MPAM spec that describes all the system and MMIO registers can be found
here:
https://developer.arm.com/documentation/ddi0598/db/?lang=en
(Ignored the 'RETIRED' warning - that is just arm moving the documentation
 around. This document has the best overview)

This series is based on v6.16-rc4, and can be retrieved from:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/driver/rfc

The rest of the driver can be found here:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/snapshot/v6.16-rc4

What is MPAM? Set your time-machine to 2020:
https://lore.kernel.org/lkml/20201030161120.227225-1-james.morse@arm.com/


Bugs welcome,
Thanks,

James Morse (31):
  cacheinfo: Add arch hook to compress CPU h/w id into 32 bits for
    cache-id
  arm64: cacheinfo: Provide helper to compress MPIDR value into u32
  cacheinfo: Expose the code to generate a cache-id from a device_node
  ACPI / PPTT: Add a helper to fill a cpumask from a processor container
  ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear
    levels
  ACPI / PPTT: Find cache level by cache-id
  ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
  arm64: kconfig: Add Kconfig entry for MPAM
  ACPI / MPAM: Parse the MPAM table
  platform: arm64: Move ec devices to an ec subdirectory
  arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
  arm_mpam: Add the class and component structures for ris firmware
    described
  arm_mpam: Add MPAM MSC register layout definitions
  arm_mpam: Add cpuhp callbacks to probe MSC hardware
  arm_mpam: Probe MSCs to find the supported partid/pmg values
  arm_mpam: Add helpers for managing the locking around the mon_sel
    registers
  arm_mpam: Probe the hardware features resctrl supports
  arm_mpam: Merge supported features during mpam_enable() into
    mpam_class
  arm_mpam: Reset MSC controls from cpu hp callbacks
  arm_mpam: Add a helper to touch an MSC from any CPU
  arm_mpam: Extend reset logic to allow devices to be reset any time
  arm_mpam: Register and enable IRQs
  arm_mpam: Use a static key to indicate when mpam is enabled
  arm_mpam: Allow configuration to be applied and restored during cpu
    online
  arm_mpam: Probe and reset the rest of the features
  arm_mpam: Add helpers to allocate monitors
  arm_mpam: Add mpam_msmon_read() to read monitor value
  arm_mpam: Track bandwidth counter state for overflow and power
    management
  arm_mpam: Add helper to reset saved mbwu state
  arm_mpam: Add kunit test for bitmap reset
  arm_mpam: Add kunit tests for props_mismatch()

Rob Herring (2):
  cacheinfo: Set cache 'id' based on DT data
  dt-bindings: arm: Add MPAM MSC binding

Rohit Mathew (2):
  arm_mpam: Probe for long/lwd mbwu counters
  arm_mpam: Use long MBWU counters if supported

Shanker Donthineni (1):
  arm_mpam: Add support for memory controller MSC on DT platforms

 .../devicetree/bindings/arm/arm,mpam-msc.yaml |  227 ++
 MAINTAINERS                                   |    6 +-
 arch/arm64/Kconfig                            |   19 +
 arch/arm64/include/asm/cache.h                |   17 +
 drivers/acpi/arm64/Kconfig                    |    3 +
 drivers/acpi/arm64/Makefile                   |    1 +
 drivers/acpi/arm64/mpam.c                     |  365 +++
 drivers/acpi/pptt.c                           |  240 +-
 drivers/acpi/tables.c                         |    2 +-
 drivers/base/cacheinfo.c                      |   57 +
 drivers/platform/arm64/Kconfig                |   73 +-
 drivers/platform/arm64/Makefile               |   10 +-
 drivers/platform/arm64/ec/Kconfig             |   73 +
 drivers/platform/arm64/ec/Makefile            |   10 +
 .../platform/arm64/{ => ec}/acer-aspire1-ec.c |    0
 .../arm64/{ => ec}/huawei-gaokun-ec.c         |    0
 .../arm64/{ => ec}/lenovo-yoga-c630.c         |    0
 drivers/platform/arm64/mpam/Kconfig           |   23 +
 drivers/platform/arm64/mpam/Makefile          |    4 +
 drivers/platform/arm64/mpam/mpam_devices.c    | 2910 +++++++++++++++++
 drivers/platform/arm64/mpam/mpam_internal.h   |  697 ++++
 .../platform/arm64/mpam/test_mpam_devices.c   |  390 +++
 include/linux/acpi.h                          |   17 +
 include/linux/arm_mpam.h                      |   56 +
 include/linux/cacheinfo.h                     |    1 +
 25 files changed, 5117 insertions(+), 84 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml
 create mode 100644 drivers/acpi/arm64/mpam.c
 create mode 100644 drivers/platform/arm64/ec/Kconfig
 create mode 100644 drivers/platform/arm64/ec/Makefile
 rename drivers/platform/arm64/{ => ec}/acer-aspire1-ec.c (100%)
 rename drivers/platform/arm64/{ => ec}/huawei-gaokun-ec.c (100%)
 rename drivers/platform/arm64/{ => ec}/lenovo-yoga-c630.c (100%)
 create mode 100644 drivers/platform/arm64/mpam/Kconfig
 create mode 100644 drivers/platform/arm64/mpam/Makefile
 create mode 100644 drivers/platform/arm64/mpam/mpam_devices.c
 create mode 100644 drivers/platform/arm64/mpam/mpam_internal.h
 create mode 100644 drivers/platform/arm64/mpam/test_mpam_devices.c
 create mode 100644 include/linux/arm_mpam.h

-- 
2.39.5
Re: [RFC PATCH 00/36] arm_mpam: Add basic mpam driver
Posted by Jonathan Cameron 2 months ago
On Fri, 11 Jul 2025 18:36:12 +0000
James Morse <james.morse@arm.com> wrote:

> Hello,
> 
> This is just enough MPAM driver for the ACPI and DT pre-requisites.
> It doesn't contain any of the resctrl code, meaning you can't actually drive it
> from user-space yet.
> 
> This is the initial group of patches that allows the resctrl code to be built
> on top. Including that will increase the number of trees that may need to
> coordinate, so breaking it up make sense.
> 
> The locking looks very strange - but is influenced by the 'mpam-fb' firmware
> interface specification that is still alpha. That thing needs to wait for an
> interrupt after every system register write, which significantly impacts the
> driver. Some features just won't work, e.g. reading the monitor registers via
> perf.
> The aim is to not have to make invasive changes to the locking to support the
> firmware interface, hence it looks strange from day-1.
> 
> I've not found a platform that can test all the behaviours around the monitors,
> so this is where I'd expect the most bugs.
> 
> It's unclear where in the tree this should be put. It affects memory bandwidth
> and cache allocation, but doesn't (yet) interact with perf. The main interaction
> is with resctrl in fs/resctrl - but there will be no filesystem code in here.
> Its also likely there will be other in-kernel users. (in-kernel MSC emulation by
> KVM being an obvious example).
> (I'm not a fan of drivers/resctrl or drivers/mpam - its not the sort of thing
>  that justifies being a 'subsystem'.)
> 
> For now, I've put this under drivers/platform/arm64. Other ideas welcome.
> 
> The first three patches are currently a series on the list, the PPTT stuff
> has previously been posted - this is where the users of those helpers appear.
> 
Hi James,

Whilst I get that this is minimal, I was a bit surprised that it doesn't
contain enough to have the driver actually bind to the platform devices
I think that needs the CPU hotplug handler to register a requester.
So about another 4 arch patches from your tree.  Maybe you can shuffle
things around to help with that.

That makes this a pain to test in isolation.

Given desire to poke the corners, I'm rebasing the old QEMU emulation and
will poke it some more.  Now we are getting close to upstream kernel support
maybe I'll even clean that up for potential upstream QEMU.

For bonus points I 'could' hook it up to the cache simulator and actually
generate real 'counts' but that's probably more for fun than because it's
useful. Fake numbers are a lot cheaper to get.

Jonathan
> 
> The MPAM spec that describes all the system and MMIO registers can be found
> here:
> https://developer.arm.com/documentation/ddi0598/db/?lang=en
> (Ignored the 'RETIRED' warning - that is just arm moving the documentation
>  around. This document has the best overview)
> 
> This series is based on v6.16-rc4, and can be retrieved from:
> https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/driver/rfc
> 
> The rest of the driver can be found here:
> https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/snapshot/v6.16-rc4
> 
> What is MPAM? Set your time-machine to 2020:
> https://lore.kernel.org/lkml/20201030161120.227225-1-james.morse@arm.com/
> 
> 
> Bugs welcome,
> Thanks,
> 
> James Morse (31):
>   cacheinfo: Add arch hook to compress CPU h/w id into 32 bits for
>     cache-id
>   arm64: cacheinfo: Provide helper to compress MPIDR value into u32
>   cacheinfo: Expose the code to generate a cache-id from a device_node
>   ACPI / PPTT: Add a helper to fill a cpumask from a processor container
>   ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear
>     levels
>   ACPI / PPTT: Find cache level by cache-id
>   ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
>   arm64: kconfig: Add Kconfig entry for MPAM
>   ACPI / MPAM: Parse the MPAM table
>   platform: arm64: Move ec devices to an ec subdirectory
>   arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
>   arm_mpam: Add the class and component structures for ris firmware
>     described
>   arm_mpam: Add MPAM MSC register layout definitions
>   arm_mpam: Add cpuhp callbacks to probe MSC hardware
>   arm_mpam: Probe MSCs to find the supported partid/pmg values
>   arm_mpam: Add helpers for managing the locking around the mon_sel
>     registers
>   arm_mpam: Probe the hardware features resctrl supports
>   arm_mpam: Merge supported features during mpam_enable() into
>     mpam_class
>   arm_mpam: Reset MSC controls from cpu hp callbacks
>   arm_mpam: Add a helper to touch an MSC from any CPU
>   arm_mpam: Extend reset logic to allow devices to be reset any time
>   arm_mpam: Register and enable IRQs
>   arm_mpam: Use a static key to indicate when mpam is enabled
>   arm_mpam: Allow configuration to be applied and restored during cpu
>     online
>   arm_mpam: Probe and reset the rest of the features
>   arm_mpam: Add helpers to allocate monitors
>   arm_mpam: Add mpam_msmon_read() to read monitor value
>   arm_mpam: Track bandwidth counter state for overflow and power
>     management
>   arm_mpam: Add helper to reset saved mbwu state
>   arm_mpam: Add kunit test for bitmap reset
>   arm_mpam: Add kunit tests for props_mismatch()
> 
> Rob Herring (2):
>   cacheinfo: Set cache 'id' based on DT data
>   dt-bindings: arm: Add MPAM MSC binding
> 
> Rohit Mathew (2):
>   arm_mpam: Probe for long/lwd mbwu counters
>   arm_mpam: Use long MBWU counters if supported
> 
> Shanker Donthineni (1):
>   arm_mpam: Add support for memory controller MSC on DT platforms
> 
>  .../devicetree/bindings/arm/arm,mpam-msc.yaml |  227 ++
>  MAINTAINERS                                   |    6 +-
>  arch/arm64/Kconfig                            |   19 +
>  arch/arm64/include/asm/cache.h                |   17 +
>  drivers/acpi/arm64/Kconfig                    |    3 +
>  drivers/acpi/arm64/Makefile                   |    1 +
>  drivers/acpi/arm64/mpam.c                     |  365 +++
>  drivers/acpi/pptt.c                           |  240 +-
>  drivers/acpi/tables.c                         |    2 +-
>  drivers/base/cacheinfo.c                      |   57 +
>  drivers/platform/arm64/Kconfig                |   73 +-
>  drivers/platform/arm64/Makefile               |   10 +-
>  drivers/platform/arm64/ec/Kconfig             |   73 +
>  drivers/platform/arm64/ec/Makefile            |   10 +
>  .../platform/arm64/{ => ec}/acer-aspire1-ec.c |    0
>  .../arm64/{ => ec}/huawei-gaokun-ec.c         |    0
>  .../arm64/{ => ec}/lenovo-yoga-c630.c         |    0
>  drivers/platform/arm64/mpam/Kconfig           |   23 +
>  drivers/platform/arm64/mpam/Makefile          |    4 +
>  drivers/platform/arm64/mpam/mpam_devices.c    | 2910 +++++++++++++++++
>  drivers/platform/arm64/mpam/mpam_internal.h   |  697 ++++
>  .../platform/arm64/mpam/test_mpam_devices.c   |  390 +++
>  include/linux/acpi.h                          |   17 +
>  include/linux/arm_mpam.h                      |   56 +
>  include/linux/cacheinfo.h                     |    1 +
>  25 files changed, 5117 insertions(+), 84 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml
>  create mode 100644 drivers/acpi/arm64/mpam.c
>  create mode 100644 drivers/platform/arm64/ec/Kconfig
>  create mode 100644 drivers/platform/arm64/ec/Makefile
>  rename drivers/platform/arm64/{ => ec}/acer-aspire1-ec.c (100%)
>  rename drivers/platform/arm64/{ => ec}/huawei-gaokun-ec.c (100%)
>  rename drivers/platform/arm64/{ => ec}/lenovo-yoga-c630.c (100%)
>  create mode 100644 drivers/platform/arm64/mpam/Kconfig
>  create mode 100644 drivers/platform/arm64/mpam/Makefile
>  create mode 100644 drivers/platform/arm64/mpam/mpam_devices.c
>  create mode 100644 drivers/platform/arm64/mpam/mpam_internal.h
>  create mode 100644 drivers/platform/arm64/mpam/test_mpam_devices.c
>  create mode 100644 include/linux/arm_mpam.h
>
Re: [RFC PATCH 00/36] arm_mpam: Add basic mpam driver
Posted by James Morse 1 month, 4 weeks ago
Hi Jonathan,

On 01/08/2025 17:09, Jonathan Cameron wrote:
> On Fri, 11 Jul 2025 18:36:12 +0000
> James Morse <james.morse@arm.com> wrote:
>> This is just enough MPAM driver for the ACPI and DT pre-requisites.
>> It doesn't contain any of the resctrl code, meaning you can't actually drive it
>> from user-space yet.

[..]

> Whilst I get that this is minimal, I was a bit surprised that it doesn't
> contain enough to have the driver actually bind to the platform devices
> I think that needs the CPU hotplug handler to register a requester.
> So about another 4 arch patches from your tree.  Maybe you can shuffle
> things around to help with that.

Ah, I hadn't spotted that. The register-requestor code should only serve to
reduce the available PARTID - just in case the CPUs support less than the
cache hierarchy.
It's likely its the 'system_supports_mpam()' that prevents the driver being
registered. I'll move that into the arm64 patches - its needed so any id
register overrides in the CPU knock out the driver too.


> That makes this a pain to test in isolation.
> 
> Given desire to poke the corners, I'm rebasing the old QEMU emulation and
> will poke it some more.  Now we are getting close to upstream kernel support
> maybe I'll even clean that up for potential upstream QEMU.
> 
> For bonus points I 'could' hook it up to the cache simulator and actually
> generate real 'counts' but that's probably more for fun than because it's
> useful. Fake numbers are a lot cheaper to get.

I've not found a good source of fake numbers to use. Ideally it would just
increment whenever the task was scheduled - but I've found that hard to hack
that up in linux.
I fall back to reading the counters instead of the MPAM registers ... but its
hard to test for overflow or double counting with that.


Thanks,

James