[PATCH v17 0/7] firmware: arm_rmm: Add RMM v2.0 base RMI support

Suzuki K Poulose posted 7 patches 2 weeks, 4 days ago
There is a newer version of this series
arch/arm64/Kconfig                |   1 +
arch/arm64/kernel/cpufeature.c    |   1 +
drivers/firmware/Kconfig          |   1 +
drivers/firmware/Makefile         |   1 +
drivers/firmware/arm_rmm/Kconfig  |  26 +
drivers/firmware/arm_rmm/Makefile |   2 +
drivers/firmware/arm_rmm/rmi.c    | 819 ++++++++++++++++++++++++++++++
include/linux/arm-rmi-cmds.h      | 678 +++++++++++++++++++++++++
include/linux/arm-smccc-rmi.h     | 494 ++++++++++++++++++
9 files changed, 2023 insertions(+)
create mode 100644 drivers/firmware/arm_rmm/Kconfig
create mode 100644 drivers/firmware/arm_rmm/Makefile
create mode 100644 drivers/firmware/arm_rmm/rmi.c
create mode 100644 include/linux/arm-rmi-cmds.h
create mode 100644 include/linux/arm-smccc-rmi.h
[PATCH v17 0/7] firmware: arm_rmm: Add RMM v2.0 base RMI support
Posted by Suzuki K Poulose 2 weeks, 4 days ago
This series adds the generic firmware layer for talking to the Realm
Management Monitor (RMM), as specified by the RMM v2.0-bet3
specification[1]. It is the first part of the Arm CCA host support that
was previously posted as part of the larger KVM series.

The split allows this RMM support to be used as a base for other work,
including Aneesh's PCI IDE support with Arm CCA RMM as the TSM,
 without depending on the KVM Realm support that will follow as separate
 series. (See more on that below)

The series adds:

 * The RMI SMC definitions and direct-call wrappers.

 * RMM discovery and version checks during firmware init.

 * RMM host configuration, including the host page size.

 * Stateful RMI Operation (SRO) infrastructure for commands which the RMM
   can complete across multiple SMC calls while requesting or returning
   memory to the host.

 * Verification that granule tracking is available at fine granularity.
   Fine-grained tracking allows each granule in the system to be tracked
   independently, which is required before individual granules can be
   delegated. A future series will add support for dynamically supplying
   memory to the RMM for this tracking.

 * Support for fully firmware-managed systems, where the Granule Protection
   Tables for memory regions are allocated and managed by firmware. RMM v2.0
   also allows dynamic GPT creation on demand; support for that will be added
   in a later series.

 * Wrappers for the RMI commands that are used for managing the "Realm VM"
   lifecycle. This is added in to make it easier for the on-going KVM support
   to evolve in parallel pieces.

If the platform firmware cannot manage the granule tracking or the GPTs, we
bail out and deactivate the RMM, reclaiming any memory that we have donated.

The RMM v2.0 spec introduces Stateful RMI Operations (SROs), which allow
the RMM to complete an operation over several SMC calls while requesting
or returning memory to the host. This allows interrupts to be handled in
the middle of an operation and lets the RMM dynamically allocate memory
for internal tracking purposes. For example, RMI_REC_CREATE no longer
needs auxiliary granules to be provided up front, and can instead
request memory during the operation.

This series applies on v7.3-rc1 and a branch is available at [2]. The KVM
CCA support that builds on this series is available at [3] as an integration
branch. The KVM support depends on guest-memfd-in-place conversion support v12
from Ackerley [4], we plan to split that into parts, which apply cleanly on
v7.3-rcx without any dependency and is in progress. This will be made available
as soon as it is ready. Until then [3] shows how this base series enables KVM
CCA support. You may find the tf-RMM [5] and kvmtool support [6] below.

[1] RMM spec : https://support.arm.com/documentation/den0137/2-0bet3/
[2] This series: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/fw_rmm/v17
[3] KVM CCA v17 integration branch: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/kvm-integration/v17
[4] Gmem inplace conversion https://github.com/googleprodkernel/linux-cc/tree/guest_memfd-inplace-conversion-v12
[5] TF-RMM https://git.trustedfirmware.org/TF-RMM/tf-rmm.git main (commit: 134266ae)
[6] kvmtool https://gitlab.arm.com/linux-arm/kvmtool-cca.git cca/kvm-v17

Known issues: RMMv2.0
 * RmiOpMemDonateReq:count (Uint14) is incompatible with RmiAddrRangeDesc4KB
  (Uint10) and RmiAddrRangeDesc16KB (Uint12). i.e., a larger contiguous request
  may not be satisfiable by the host. Arm is aware of this defect and a spec fix
  is in progress.


Steven Price (7):
  firmware: arm_rmm: Add SMC definitions for calling the RMM
  firmware: arm_rmm: Check for RMI support at init
  firmware: arm_rmm: Configure the RMM with the host's page size
  firmware: arm_rmm: Add support for SRO
  firmware: arm_rmm: Activate the RMM
  firmware: arm_rmm: Ensure the RMM has GPT entries for memory
  firmware: arm_rmm: Add wrappers for Realm related RMI commands

 arch/arm64/Kconfig                |   1 +
 arch/arm64/kernel/cpufeature.c    |   1 +
 drivers/firmware/Kconfig          |   1 +
 drivers/firmware/Makefile         |   1 +
 drivers/firmware/arm_rmm/Kconfig  |  26 +
 drivers/firmware/arm_rmm/Makefile |   2 +
 drivers/firmware/arm_rmm/rmi.c    | 819 ++++++++++++++++++++++++++++++
 include/linux/arm-rmi-cmds.h      | 678 +++++++++++++++++++++++++
 include/linux/arm-smccc-rmi.h     | 494 ++++++++++++++++++
 9 files changed, 2023 insertions(+)
 create mode 100644 drivers/firmware/arm_rmm/Kconfig
 create mode 100644 drivers/firmware/arm_rmm/Makefile
 create mode 100644 drivers/firmware/arm_rmm/rmi.c
 create mode 100644 include/linux/arm-rmi-cmds.h
 create mode 100644 include/linux/arm-smccc-rmi.h

-- 
2.43.0
Re: [PATCH v17 0/7] firmware: arm_rmm: Add RMM v2.0 base RMI support
Posted by Kohei Enju 2 weeks, 3 days ago
Hi Suzuki,

On 09/07 10:59, Suzuki K Poulose wrote:
> This series adds the generic firmware layer for talking to the Realm
> Management Monitor (RMM), as specified by the RMM v2.0-bet3
> specification[1]. It is the first part of the Arm CCA host support that
> was previously posted as part of the larger KVM series.
> 
> The split allows this RMM support to be used as a base for other work,
> including Aneesh's PCI IDE support with Arm CCA RMM as the TSM,
>  without depending on the KVM Realm support that will follow as separate
>  series. (See more on that below)
> 
> The series adds:
> 
>  * The RMI SMC definitions and direct-call wrappers.
> 
>  * RMM discovery and version checks during firmware init.
> 
>  * RMM host configuration, including the host page size.
> 
>  * Stateful RMI Operation (SRO) infrastructure for commands which the RMM
>    can complete across multiple SMC calls while requesting or returning
>    memory to the host.
> 
>  * Verification that granule tracking is available at fine granularity.
>    Fine-grained tracking allows each granule in the system to be tracked
>    independently, which is required before individual granules can be
>    delegated. A future series will add support for dynamically supplying
>    memory to the RMM for this tracking.
> 
>  * Support for fully firmware-managed systems, where the Granule Protection
>    Tables for memory regions are allocated and managed by firmware. RMM v2.0
>    also allows dynamic GPT creation on demand; support for that will be added
>    in a later series.
> 
>  * Wrappers for the RMI commands that are used for managing the "Realm VM"
>    lifecycle. This is added in to make it easier for the on-going KVM support
>    to evolve in parallel pieces.
> 
> If the platform firmware cannot manage the granule tracking or the GPTs, we
> bail out and deactivate the RMM, reclaiming any memory that we have donated.
> 
> The RMM v2.0 spec introduces Stateful RMI Operations (SROs), which allow
> the RMM to complete an operation over several SMC calls while requesting
> or returning memory to the host. This allows interrupts to be handled in
> the middle of an operation and lets the RMM dynamically allocate memory
> for internal tracking purposes. For example, RMI_REC_CREATE no longer
> needs auxiliary granules to be provided up front, and can instead
> request memory during the operation.
> 
> This series applies on v7.3-rc1 and a branch is available at [2]. The KVM
> CCA support that builds on this series is available at [3] as an integration
> branch. The KVM support depends on guest-memfd-in-place conversion support v12
> from Ackerley [4], we plan to split that into parts, which apply cleanly on
> v7.3-rcx without any dependency and is in progress. This will be made available
> as soon as it is ready. Until then [3] shows how this base series enables KVM
> CCA support. You may find the tf-RMM [5] and kvmtool support [6] below.
> 
> [1] RMM spec : https://support.arm.com/documentation/den0137/2-0bet3/
> [2] This series: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/fw_rmm/v17
> [3] KVM CCA v17 integration branch: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/kvm-integration/v17
> [4] Gmem inplace conversion https://github.com/googleprodkernel/linux-cc/tree/guest_memfd-inplace-conversion-v12
> [5] TF-RMM https://git.trustedfirmware.org/TF-RMM/tf-rmm.git main (commit: 134266ae)

I understand that QEMU-virt is not intended to model a realistic
production platform and that QEMU-SBSA is generally preferred. However,
I sometimes use QEMU-virt to test functionality quickly because it runs
slightly faster than QEMU-SBSA.

I tested this series with QEMU-virt and found that RMM initialization
failed during boot:

  [...]
  INFO:    BL31: Initializing RMM
  INFO:    RMM init start.
  ERROR:   RMM init failed: -8
  WARNING: BL31: RMM initialization failed

The following TF-A revision was used:
  https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git master (38269bb)

After investigating, I found that the failure was caused by the lack of
a plat_rmmd_reserve_memory() implementation for the QEMU-virt platform
in TF-A. Adding an implementation equivalent to the QEMU-SBSA one
resolved the issue:
    https://git.trustedfirmware.org/plugins/gitiles/TF-A/trusted-firmware-a.git/+/9d594743aa4d4e8160cdf025ea7538041648972c

Is QEMU virt still intended to be supported as an RME emulation
platform? In particular, is this functionality simply not implemented
yet, or is there a reason why RME support is not planned for QEMU virt?

If this is not the appropriate place to discuss this, please let me
know :)

Thanks,
Kohei

> [6] kvmtool https://gitlab.arm.com/linux-arm/kvmtool-cca.git cca/kvm-v17
> 
> Known issues: RMMv2.0
>  * RmiOpMemDonateReq:count (Uint14) is incompatible with RmiAddrRangeDesc4KB
>   (Uint10) and RmiAddrRangeDesc16KB (Uint12). i.e., a larger contiguous request
>   may not be satisfiable by the host. Arm is aware of this defect and a spec fix
>   is in progress.
> 
> 
> Steven Price (7):
>   firmware: arm_rmm: Add SMC definitions for calling the RMM
>   firmware: arm_rmm: Check for RMI support at init
>   firmware: arm_rmm: Configure the RMM with the host's page size
>   firmware: arm_rmm: Add support for SRO
>   firmware: arm_rmm: Activate the RMM
>   firmware: arm_rmm: Ensure the RMM has GPT entries for memory
>   firmware: arm_rmm: Add wrappers for Realm related RMI commands
> 
>  arch/arm64/Kconfig                |   1 +
>  arch/arm64/kernel/cpufeature.c    |   1 +
>  drivers/firmware/Kconfig          |   1 +
>  drivers/firmware/Makefile         |   1 +
>  drivers/firmware/arm_rmm/Kconfig  |  26 +
>  drivers/firmware/arm_rmm/Makefile |   2 +
>  drivers/firmware/arm_rmm/rmi.c    | 819 ++++++++++++++++++++++++++++++
>  include/linux/arm-rmi-cmds.h      | 678 +++++++++++++++++++++++++
>  include/linux/arm-smccc-rmi.h     | 494 ++++++++++++++++++
>  9 files changed, 2023 insertions(+)
>  create mode 100644 drivers/firmware/arm_rmm/Kconfig
>  create mode 100644 drivers/firmware/arm_rmm/Makefile
>  create mode 100644 drivers/firmware/arm_rmm/rmi.c
>  create mode 100644 include/linux/arm-rmi-cmds.h
>  create mode 100644 include/linux/arm-smccc-rmi.h
> 
> -- 
> 2.43.0
>
Re: [PATCH v17 0/7] firmware: arm_rmm: Add RMM v2.0 base RMI support
Posted by Gavin Shan 2 weeks, 2 days ago
Hi Kohei,

On 9/8/26 2:09 PM, Kohei Enju wrote:

[...]

>>
>> [1] RMM spec : https://support.arm.com/documentation/den0137/2-0bet3/
>> [2] This series: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/fw_rmm/v17
>> [3] KVM CCA v17 integration branch: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/kvm-integration/v17
>> [4] Gmem inplace conversion https://github.com/googleprodkernel/linux-cc/tree/guest_memfd-inplace-conversion-v12
>> [5] TF-RMM https://git.trustedfirmware.org/TF-RMM/tf-rmm.git main (commit: 134266ae)
> 
> I understand that QEMU-virt is not intended to model a realistic
> production platform and that QEMU-SBSA is generally preferred. However,
> I sometimes use QEMU-virt to test functionality quickly because it runs
> slightly faster than QEMU-SBSA.
> 
> I tested this series with QEMU-virt and found that RMM initialization
> failed during boot:
> 
>    [...]
>    INFO:    BL31: Initializing RMM
>    INFO:    RMM init start.
>    ERROR:   RMM init failed: -8
>    WARNING: BL31: RMM initialization failed
> 
I also ran into same issue and attached TF-A patch leads to a successful
RMM initialization, please have a try to see if it can resolve your issue.


Thanks,
Gavin
Re: [PATCH v17 0/7] firmware: arm_rmm: Add RMM v2.0 base RMI support
Posted by Kohei Enju 2 weeks, 1 day ago
Hi Gavin,

On 09/09 20:52, Gavin Shan wrote:
> Hi Kohei,
> 
> On 9/8/26 2:09 PM, Kohei Enju wrote:
> 
> [...]
> 
> > > 
> > > [1] RMM spec : https://support.arm.com/documentation/den0137/2-0bet3/
> > > [2] This series: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/fw_rmm/v17
> > > [3] KVM CCA v17 integration branch: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/kvm-integration/v17
> > > [4] Gmem inplace conversion https://github.com/googleprodkernel/linux-cc/tree/guest_memfd-inplace-conversion-v12
> > > [5] TF-RMM https://git.trustedfirmware.org/TF-RMM/tf-rmm.git main (commit: 134266ae)
> > 
> > I understand that QEMU-virt is not intended to model a realistic
> > production platform and that QEMU-SBSA is generally preferred. However,
> > I sometimes use QEMU-virt to test functionality quickly because it runs
> > slightly faster than QEMU-SBSA.
> > 
> > I tested this series with QEMU-virt and found that RMM initialization
> > failed during boot:
> > 
> >    [...]
> >    INFO:    BL31: Initializing RMM
> >    INFO:    RMM init start.
> >    ERROR:   RMM init failed: -8
> >    WARNING: BL31: RMM initialization failed
> > 
> I also ran into same issue and attached TF-A patch leads to a successful
> RMM initialization, please have a try to see if it can resolve your issue.

Thanks for the patch! It works perfectly for me on QEMU-virt.
Much appreciated.

Thanks,
Kohei

> 
> 
> Thanks,
> Gavin
Re: [PATCH v17 0/7] firmware: arm_rmm: Add RMM v2.0 base RMI support
Posted by Suzuki K Poulose 2 weeks, 3 days ago
Hi Kohei-San

Linaro is best suited to answer this. I have Cc'ed Mathieu (and
Lorenzo), who may be able to help you here.

On 08/09/2026 05:09, Kohei Enju wrote:
> Hi Suzuki,
> 
> On 09/07 10:59, Suzuki K Poulose wrote:
>> This series adds the generic firmware layer for talking to the Realm
>> Management Monitor (RMM), as specified by the RMM v2.0-bet3
>> specification[1]. It is the first part of the Arm CCA host support that
>> was previously posted as part of the larger KVM series.
>>
>> The split allows this RMM support to be used as a base for other work,
>> including Aneesh's PCI IDE support with Arm CCA RMM as the TSM,
>>   without depending on the KVM Realm support that will follow as separate
>>   series. (See more on that below)
>>
>> The series adds:
>>
>>   * The RMI SMC definitions and direct-call wrappers.
>>
>>   * RMM discovery and version checks during firmware init.
>>
>>   * RMM host configuration, including the host page size.
>>
>>   * Stateful RMI Operation (SRO) infrastructure for commands which the RMM
>>     can complete across multiple SMC calls while requesting or returning
>>     memory to the host.
>>
>>   * Verification that granule tracking is available at fine granularity.
>>     Fine-grained tracking allows each granule in the system to be tracked
>>     independently, which is required before individual granules can be
>>     delegated. A future series will add support for dynamically supplying
>>     memory to the RMM for this tracking.
>>
>>   * Support for fully firmware-managed systems, where the Granule Protection
>>     Tables for memory regions are allocated and managed by firmware. RMM v2.0
>>     also allows dynamic GPT creation on demand; support for that will be added
>>     in a later series.
>>
>>   * Wrappers for the RMI commands that are used for managing the "Realm VM"
>>     lifecycle. This is added in to make it easier for the on-going KVM support
>>     to evolve in parallel pieces.
>>
>> If the platform firmware cannot manage the granule tracking or the GPTs, we
>> bail out and deactivate the RMM, reclaiming any memory that we have donated.
>>
>> The RMM v2.0 spec introduces Stateful RMI Operations (SROs), which allow
>> the RMM to complete an operation over several SMC calls while requesting
>> or returning memory to the host. This allows interrupts to be handled in
>> the middle of an operation and lets the RMM dynamically allocate memory
>> for internal tracking purposes. For example, RMI_REC_CREATE no longer
>> needs auxiliary granules to be provided up front, and can instead
>> request memory during the operation.
>>
>> This series applies on v7.3-rc1 and a branch is available at [2]. The KVM
>> CCA support that builds on this series is available at [3] as an integration
>> branch. The KVM support depends on guest-memfd-in-place conversion support v12
>> from Ackerley [4], we plan to split that into parts, which apply cleanly on
>> v7.3-rcx without any dependency and is in progress. This will be made available
>> as soon as it is ready. Until then [3] shows how this base series enables KVM
>> CCA support. You may find the tf-RMM [5] and kvmtool support [6] below.
>>
>> [1] RMM spec : https://support.arm.com/documentation/den0137/2-0bet3/
>> [2] This series: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/fw_rmm/v17
>> [3] KVM CCA v17 integration branch: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/kvm-integration/v17
>> [4] Gmem inplace conversion https://github.com/googleprodkernel/linux-cc/tree/guest_memfd-inplace-conversion-v12
>> [5] TF-RMM https://git.trustedfirmware.org/TF-RMM/tf-rmm.git main (commit: 134266ae)
> 
> I understand that QEMU-virt is not intended to model a realistic
> production platform and that QEMU-SBSA is generally preferred. However,
> I sometimes use QEMU-virt to test functionality quickly because it runs
> slightly faster than QEMU-SBSA.
> 
> I tested this series with QEMU-virt and found that RMM initialization
> failed during boot:
> 
>    [...]
>    INFO:    BL31: Initializing RMM
>    INFO:    RMM init start.
>    ERROR:   RMM init failed: -8
>    WARNING: BL31: RMM initialization failed
> 
> The following TF-A revision was used:
>    https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git master (38269bb)
> 
> After investigating, I found that the failure was caused by the lack of
> a plat_rmmd_reserve_memory() implementation for the QEMU-virt platform
> in TF-A. Adding an implementation equivalent to the QEMU-SBSA one
> resolved the issue:
>      https://git.trustedfirmware.org/plugins/gitiles/TF-A/trusted-firmware-a.git/+/9d594743aa4d4e8160cdf025ea7538041648972c
> 
> Is QEMU virt still intended to be supported as an RME emulation
> platform? In particular, is this functionality simply not implemented
> yet, or is there a reason why RME support is not planned for QEMU virt?
> 
> If this is not the appropriate place to discuss this, please let me

Mathieu, Lorenzo,

Over to you folks :-)

Cheers
Suzuki


> know :)
> 
> Thanks,
> Kohei
> 
>> [6] kvmtool https://gitlab.arm.com/linux-arm/kvmtool-cca.git cca/kvm-v17
>>
>> Known issues: RMMv2.0
>>   * RmiOpMemDonateReq:count (Uint14) is incompatible with RmiAddrRangeDesc4KB
>>    (Uint10) and RmiAddrRangeDesc16KB (Uint12). i.e., a larger contiguous request
>>    may not be satisfiable by the host. Arm is aware of this defect and a spec fix
>>    is in progress.
>>
>>
>> Steven Price (7):
>>    firmware: arm_rmm: Add SMC definitions for calling the RMM
>>    firmware: arm_rmm: Check for RMI support at init
>>    firmware: arm_rmm: Configure the RMM with the host's page size
>>    firmware: arm_rmm: Add support for SRO
>>    firmware: arm_rmm: Activate the RMM
>>    firmware: arm_rmm: Ensure the RMM has GPT entries for memory
>>    firmware: arm_rmm: Add wrappers for Realm related RMI commands
>>
>>   arch/arm64/Kconfig                |   1 +
>>   arch/arm64/kernel/cpufeature.c    |   1 +
>>   drivers/firmware/Kconfig          |   1 +
>>   drivers/firmware/Makefile         |   1 +
>>   drivers/firmware/arm_rmm/Kconfig  |  26 +
>>   drivers/firmware/arm_rmm/Makefile |   2 +
>>   drivers/firmware/arm_rmm/rmi.c    | 819 ++++++++++++++++++++++++++++++
>>   include/linux/arm-rmi-cmds.h      | 678 +++++++++++++++++++++++++
>>   include/linux/arm-smccc-rmi.h     | 494 ++++++++++++++++++
>>   9 files changed, 2023 insertions(+)
>>   create mode 100644 drivers/firmware/arm_rmm/Kconfig
>>   create mode 100644 drivers/firmware/arm_rmm/Makefile
>>   create mode 100644 drivers/firmware/arm_rmm/rmi.c
>>   create mode 100644 include/linux/arm-rmi-cmds.h
>>   create mode 100644 include/linux/arm-smccc-rmi.h
>>
>> -- 
>> 2.43.0
>>
Re: [PATCH v17 0/7] firmware: arm_rmm: Add RMM v2.0 base RMI support
Posted by Kohei Enju 2 weeks, 3 days ago
On 09/08 06:46, Suzuki K Poulose wrote:
> Hi Kohei-San
> 
> Linaro is best suited to answer this. I have Cc'ed Mathieu (and
> Lorenzo), who may be able to help you here.

Thank you for pointing me in the right direction and Ccing Mathieu and
Lorenzo. 
I'll wait for their response.

> 
> On 08/09/2026 05:09, Kohei Enju wrote:
> > Hi Suzuki,
> > 
> > On 09/07 10:59, Suzuki K Poulose wrote:
> > > This series adds the generic firmware layer for talking to the Realm
> > > Management Monitor (RMM), as specified by the RMM v2.0-bet3
> > > specification[1]. It is the first part of the Arm CCA host support that
> > > was previously posted as part of the larger KVM series.
> > > 
> > > The split allows this RMM support to be used as a base for other work,
> > > including Aneesh's PCI IDE support with Arm CCA RMM as the TSM,
> > >   without depending on the KVM Realm support that will follow as separate
> > >   series. (See more on that below)
> > > 
> > > The series adds:
> > > 
> > >   * The RMI SMC definitions and direct-call wrappers.
> > > 
> > >   * RMM discovery and version checks during firmware init.
> > > 
> > >   * RMM host configuration, including the host page size.
> > > 
> > >   * Stateful RMI Operation (SRO) infrastructure for commands which the RMM
> > >     can complete across multiple SMC calls while requesting or returning
> > >     memory to the host.
> > > 
> > >   * Verification that granule tracking is available at fine granularity.
> > >     Fine-grained tracking allows each granule in the system to be tracked
> > >     independently, which is required before individual granules can be
> > >     delegated. A future series will add support for dynamically supplying
> > >     memory to the RMM for this tracking.
> > > 
> > >   * Support for fully firmware-managed systems, where the Granule Protection
> > >     Tables for memory regions are allocated and managed by firmware. RMM v2.0
> > >     also allows dynamic GPT creation on demand; support for that will be added
> > >     in a later series.
> > > 
> > >   * Wrappers for the RMI commands that are used for managing the "Realm VM"
> > >     lifecycle. This is added in to make it easier for the on-going KVM support
> > >     to evolve in parallel pieces.
> > > 
> > > If the platform firmware cannot manage the granule tracking or the GPTs, we
> > > bail out and deactivate the RMM, reclaiming any memory that we have donated.
> > > 
> > > The RMM v2.0 spec introduces Stateful RMI Operations (SROs), which allow
> > > the RMM to complete an operation over several SMC calls while requesting
> > > or returning memory to the host. This allows interrupts to be handled in
> > > the middle of an operation and lets the RMM dynamically allocate memory
> > > for internal tracking purposes. For example, RMI_REC_CREATE no longer
> > > needs auxiliary granules to be provided up front, and can instead
> > > request memory during the operation.
> > > 
> > > This series applies on v7.3-rc1 and a branch is available at [2]. The KVM
> > > CCA support that builds on this series is available at [3] as an integration
> > > branch. The KVM support depends on guest-memfd-in-place conversion support v12
> > > from Ackerley [4], we plan to split that into parts, which apply cleanly on
> > > v7.3-rcx without any dependency and is in progress. This will be made available
> > > as soon as it is ready. Until then [3] shows how this base series enables KVM
> > > CCA support. You may find the tf-RMM [5] and kvmtool support [6] below.
> > > 
> > > [1] RMM spec : https://support.arm.com/documentation/den0137/2-0bet3/
> > > [2] This series: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/fw_rmm/v17
> > > [3] KVM CCA v17 integration branch: https://gitlab.arm.com/linux-arm/linux-cca.git cca/cca-host/kvm-integration/v17
> > > [4] Gmem inplace conversion https://github.com/googleprodkernel/linux-cc/tree/guest_memfd-inplace-conversion-v12
> > > [5] TF-RMM https://git.trustedfirmware.org/TF-RMM/tf-rmm.git main (commit: 134266ae)
> > 
> > I understand that QEMU-virt is not intended to model a realistic
> > production platform and that QEMU-SBSA is generally preferred. However,
> > I sometimes use QEMU-virt to test functionality quickly because it runs
> > slightly faster than QEMU-SBSA.
> > 
> > I tested this series with QEMU-virt and found that RMM initialization
> > failed during boot:
> > 
> >    [...]
> >    INFO:    BL31: Initializing RMM
> >    INFO:    RMM init start.
> >    ERROR:   RMM init failed: -8
> >    WARNING: BL31: RMM initialization failed
> > 
> > The following TF-A revision was used:
> >    https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git master (38269bb)
> > 
> > After investigating, I found that the failure was caused by the lack of
> > a plat_rmmd_reserve_memory() implementation for the QEMU-virt platform
> > in TF-A. Adding an implementation equivalent to the QEMU-SBSA one
> > resolved the issue:
> >      https://git.trustedfirmware.org/plugins/gitiles/TF-A/trusted-firmware-a.git/+/9d594743aa4d4e8160cdf025ea7538041648972c
> > 
> > Is QEMU virt still intended to be supported as an RME emulation
> > platform? In particular, is this functionality simply not implemented
> > yet, or is there a reason why RME support is not planned for QEMU virt?
> > 
> > If this is not the appropriate place to discuss this, please let me
> 
> Mathieu, Lorenzo,
> 
> Over to you folks :-)
> 
> Cheers
> Suzuki
> 
> 
> > know :)
> > 
> > Thanks,
> > Kohei
> > 
> > > [6] kvmtool https://gitlab.arm.com/linux-arm/kvmtool-cca.git cca/kvm-v17
> > > 
> > > Known issues: RMMv2.0
> > >   * RmiOpMemDonateReq:count (Uint14) is incompatible with RmiAddrRangeDesc4KB
> > >    (Uint10) and RmiAddrRangeDesc16KB (Uint12). i.e., a larger contiguous request
> > >    may not be satisfiable by the host. Arm is aware of this defect and a spec fix
> > >    is in progress.
> > > 
> > > 
> > > Steven Price (7):
> > >    firmware: arm_rmm: Add SMC definitions for calling the RMM
> > >    firmware: arm_rmm: Check for RMI support at init
> > >    firmware: arm_rmm: Configure the RMM with the host's page size
> > >    firmware: arm_rmm: Add support for SRO
> > >    firmware: arm_rmm: Activate the RMM
> > >    firmware: arm_rmm: Ensure the RMM has GPT entries for memory
> > >    firmware: arm_rmm: Add wrappers for Realm related RMI commands
> > > 
> > >   arch/arm64/Kconfig                |   1 +
> > >   arch/arm64/kernel/cpufeature.c    |   1 +
> > >   drivers/firmware/Kconfig          |   1 +
> > >   drivers/firmware/Makefile         |   1 +
> > >   drivers/firmware/arm_rmm/Kconfig  |  26 +
> > >   drivers/firmware/arm_rmm/Makefile |   2 +
> > >   drivers/firmware/arm_rmm/rmi.c    | 819 ++++++++++++++++++++++++++++++
> > >   include/linux/arm-rmi-cmds.h      | 678 +++++++++++++++++++++++++
> > >   include/linux/arm-smccc-rmi.h     | 494 ++++++++++++++++++
> > >   9 files changed, 2023 insertions(+)
> > >   create mode 100644 drivers/firmware/arm_rmm/Kconfig
> > >   create mode 100644 drivers/firmware/arm_rmm/Makefile
> > >   create mode 100644 drivers/firmware/arm_rmm/rmi.c
> > >   create mode 100644 include/linux/arm-rmi-cmds.h
> > >   create mode 100644 include/linux/arm-smccc-rmi.h
> > > 
> > > -- 
> > > 2.43.0
> > > 
>