[v2] docs/system/arm/virt: Document user-creatable SMMUv3

[PATCH v2] docs/system/arm/virt: Document user-creatable SMMUv3

Posted by Shameer Kolothum 3 months, 2 weeks ago

The virt machine now supports creating multiple SMMUv3 instances, each
associated with a separate PCIe root complex.

Update the documentation with an example.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 Addressed feedback from v1.(Thanks!).
 https://lore.kernel.org/qemu-devel/20251006182900.100580-1-skolothumtho@nvidia.com/
---
 docs/system/arm/virt.rst | 34 ++++++++++++++++++++++++++++++++--
 1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
index 10cbffc8a7..339aa99ae1 100644
--- a/docs/system/arm/virt.rst
+++ b/docs/system/arm/virt.rst
@@ -37,7 +37,8 @@ The virt board supports:
 - An RTC
 - The fw_cfg device that allows a guest to obtain data from QEMU
 - A PL061 GPIO controller
-- An optional SMMUv3 IOMMU
+- An optional machine-wide SMMUv3 IOMMU
+- User-creatable SMMUv3 devices (see below for example)
 - hotpluggable DIMMs
 - hotpluggable NVDIMMs
 - An MSI controller (GICv2M or ITS). GICv2M is selected by default along
@@ -176,7 +177,7 @@ iommu
   ``none``
     Don't create an IOMMU (the default)
   ``smmuv3``
-    Create an SMMUv3
+    Create a machine-wide SMMUv3.
 
 default-bus-bypass-iommu
   Set ``on``/``off`` to enable/disable `bypass_iommu
@@ -219,6 +220,35 @@ x-oem-table-id
   Set string (up to 8 bytes) to override the default value of field OEM Table ID
   in ACPI table header.
 
+SMMU configuration
+""""""""""""""""""
+
+Machine-wide SMMUv3 IOMMU
+  See the machine-specific ``iommu`` option above. This allows specifying
+  a single, machine-wide SMMUv3 instance that applies to all devices in
+  the PCIe topology.
+
+  For information about selectively bypassing devices, refer to
+  ``docs/bypass-iommu.txt``.
+
+User-creatable SMMUv3 devices
+  Allows creating multiple user-defined SMMUv3 devices, each associated
+  with a separate PCIe root complex. This is only permitted if the
+  machine-wide SMMUv3 (``iommu=smmuv3``) option is not used.
+
+  This model is useful when you want to mirror a host configuration where
+  each NUMA node typically has its own SMMU, allowing the VM topology to
+  align more closely with the host’s hardware layout. Supporting multiple
+  SMMUv3 instances is also a prerequisite for future accelerated SMMUv3
+  support.
+
+  Example::
+
+      -device arm-smmuv3,primary-bus=pcie.0,id=smmuv3.0
+      ...
+      -device pxb-pcie,id=pcie.1,numa_node=1
+      -device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1
+
 Linux guest kernel configuration
 """"""""""""""""""""""""""""""""
 
-- 
2.43.0

Re: [PATCH v2] docs/system/arm/virt: Document user-creatable SMMUv3

Posted by Peter Maydell 3 months, 2 weeks ago

On Fri, 24 Oct 2025 at 09:46, Shameer Kolothum <skolothumtho@nvidia.com> wrote:
>
> The virt machine now supports creating multiple SMMUv3 instances, each
> associated with a separate PCIe root complex.
>
> Update the documentation with an example.
>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>

Thanks. I have some minor word-smithing tweaks, but I'll just
apply them to this patch in target-arm.next if you're OK with
that:


--- a/docs/system/arm/virt.rst
+++ b/docs/system/arm/virt.rst
@@ -224,23 +224,24 @@ SMMU configuration
 """"""""""""""""""

 Machine-wide SMMUv3 IOMMU
-  See the machine-specific ``iommu`` option above. This allows specifying
-  a single, machine-wide SMMUv3 instance that applies to all devices in
-  the PCIe topology.
+  Setting the machine-specific option ``iommu=smmuv3`` causes QEMU to
+  create a single, machine-wide SMMUv3 instance that applies to all
+  devices in the PCIe topology.

   For information about selectively bypassing devices, refer to
   ``docs/bypass-iommu.txt``.

 User-creatable SMMUv3 devices
-  Allows creating multiple user-defined SMMUv3 devices, each associated
-  with a separate PCIe root complex. This is only permitted if the
-  machine-wide SMMUv3 (``iommu=smmuv3``) option is not used.
+  You can use the ``-device arm-smmuv3`` option to create multiple
+  user-defined SMMUv3 devices, each associated with a separate PCIe
+  root complex. This is only permitted if the machine-wide SMMUv3
+  (``iommu=smmuv3``) option is not used. Each ``arm-smmuv3`` device
+  uses the ``primary-bus`` sub-option to specify which PCIe root
+  complex it is associated with.

   This model is useful when you want to mirror a host configuration where
   each NUMA node typically has its own SMMU, allowing the VM topology to
-  align more closely with the host’s hardware layout. Supporting multiple
-  SMMUv3 instances is also a prerequisite for future accelerated SMMUv3
-  support.
+  align more closely with the host’s hardware layout.

   Example::




(I deleted the sentence about SMMU acceleration because that
isn't upstream yet. When it lands then we can update the
docs if the version of it that lands can't support acceleration
of a single machine-wide accelerator.)

thanks
-- PMM

Re: [PATCH v2] docs/system/arm/virt: Document user-creatable SMMUv3

Posted by Eric Auger 3 months, 2 weeks ago


On 10/27/25 12:27 PM, Peter Maydell wrote:
> On Fri, 24 Oct 2025 at 09:46, Shameer Kolothum <skolothumtho@nvidia.com> wrote:
>> The virt machine now supports creating multiple SMMUv3 instances, each
>> associated with a separate PCIe root complex.
>>
>> Update the documentation with an example.
>>
>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> Thanks. I have some minor word-smithing tweaks, but I'll just
> apply them to this patch in target-arm.next if you're OK with
> that:
>
>
> --- a/docs/system/arm/virt.rst
> +++ b/docs/system/arm/virt.rst
> @@ -224,23 +224,24 @@ SMMU configuration
>  """"""""""""""""""
>
>  Machine-wide SMMUv3 IOMMU
> -  See the machine-specific ``iommu`` option above. This allows specifying
> -  a single, machine-wide SMMUv3 instance that applies to all devices in
> -  the PCIe topology.
> +  Setting the machine-specific option ``iommu=smmuv3`` causes QEMU to
> +  create a single, machine-wide SMMUv3 instance that applies to all
> +  devices in the PCIe topology.
>
>    For information about selectively bypassing devices, refer to
>    ``docs/bypass-iommu.txt``.
>
>  User-creatable SMMUv3 devices
> -  Allows creating multiple user-defined SMMUv3 devices, each associated
> -  with a separate PCIe root complex. This is only permitted if the
> -  machine-wide SMMUv3 (``iommu=smmuv3``) option is not used.
> +  You can use the ``-device arm-smmuv3`` option to create multiple
> +  user-defined SMMUv3 devices, each associated with a separate PCIe
> +  root complex. This is only permitted if the machine-wide SMMUv3
> +  (``iommu=smmuv3``) option is not used. Each ``arm-smmuv3`` device
> +  uses the ``primary-bus`` sub-option to specify which PCIe root
> +  complex it is associated with.
>
>    This model is useful when you want to mirror a host configuration where
>    each NUMA node typically has its own SMMU, allowing the VM topology to
> -  align more closely with the host’s hardware layout. Supporting multiple
> -  SMMUv3 instances is also a prerequisite for future accelerated SMMUv3
> -  support.
> +  align more closely with the host’s hardware layout.
>
>    Example::
>
>
>
>
> (I deleted the sentence about SMMU acceleration because that
> isn't upstream yet. When it lands then we can update the
> docs if the version of it that lands can't support acceleration
> of a single machine-wide accelerator.)
>
> thanks
> -- PMM
>
Looks good to me as well

Fell free to add my R-b

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks!

Eric

Re: [PATCH v2] docs/system/arm/virt: Document user-creatable SMMUv3

Posted by Donald Dutile 3 months, 1 week ago

On 10/27/25 10:13 AM, Eric Auger wrote:
> 
> 
> On 10/27/25 12:27 PM, Peter Maydell wrote:
>> On Fri, 24 Oct 2025 at 09:46, Shameer Kolothum <skolothumtho@nvidia.com> wrote:
>>> The virt machine now supports creating multiple SMMUv3 instances, each
>>> associated with a separate PCIe root complex.
>>>
>>> Update the documentation with an example.
>>>
>>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
>> Thanks. I have some minor word-smithing tweaks, but I'll just
>> apply them to this patch in target-arm.next if you're OK with
>> that:
>>
>>
>> --- a/docs/system/arm/virt.rst
>> +++ b/docs/system/arm/virt.rst
>> @@ -224,23 +224,24 @@ SMMU configuration
>>   """"""""""""""""""
>>
>>   Machine-wide SMMUv3 IOMMU
>> -  See the machine-specific ``iommu`` option above. This allows specifying
>> -  a single, machine-wide SMMUv3 instance that applies to all devices in
>> -  the PCIe topology.
>> +  Setting the machine-specific option ``iommu=smmuv3`` causes QEMU to
>> +  create a single, machine-wide SMMUv3 instance that applies to all
>> +  devices in the PCIe topology.
>>
>>     For information about selectively bypassing devices, refer to
>>     ``docs/bypass-iommu.txt``.
>>
>>   User-creatable SMMUv3 devices
>> -  Allows creating multiple user-defined SMMUv3 devices, each associated
>> -  with a separate PCIe root complex. This is only permitted if the
>> -  machine-wide SMMUv3 (``iommu=smmuv3``) option is not used.
>> +  You can use the ``-device arm-smmuv3`` option to create multiple
>> +  user-defined SMMUv3 devices, each associated with a separate PCIe
>> +  root complex. This is only permitted if the machine-wide SMMUv3
>> +  (``iommu=smmuv3``) option is not used. Each ``arm-smmuv3`` device
>> +  uses the ``primary-bus`` sub-option to specify which PCIe root
>> +  complex it is associated with.
>>
>>     This model is useful when you want to mirror a host configuration where
>>     each NUMA node typically has its own SMMU, allowing the VM topology to
>> -  align more closely with the host’s hardware layout. Supporting multiple
>> -  SMMUv3 instances is also a prerequisite for future accelerated SMMUv3
>> -  support.
>> +  align more closely with the host’s hardware layout.
>>
>>     Example::
>>
>>
>>
>>
>> (I deleted the sentence about SMMU acceleration because that
>> isn't upstream yet. When it lands then we can update the
>> docs if the version of it that lands can't support acceleration
>> of a single machine-wide accelerator.)
>>
>> thanks
>> -- PMM
>>
> Looks good to me as well
> 
> Fell free to add my R-b
> 
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> 
> Thanks!
> 
> Eric
> 
+1; looks good; can add my r-b if another is desired/needed.
- Don
> 
> 
>

Re: [PATCH v2] docs/system/arm/virt: Document user-creatable SMMUv3

Posted by Markus Armbruster 3 months, 1 week ago

Donald Dutile <ddutile@redhat.com> writes:

> On 10/27/25 10:13 AM, Eric Auger wrote:

[...]

>> Looks good to me as well
>> Fell free to add my R-b
>> Reviewed-by: Eric Auger <eric.auger@redhat.com>
>> Thanks!
>> Eric
>> 
> +1; looks good; can add my r-b if another is desired/needed.
> - Don

When you put in the review work, claim the review credit!  Suggest you
spell out your R-by.

Re: [PATCH v2] docs/system/arm/virt: Document user-creatable SMMUv3

Posted by Donald Dutile 3 months, 1 week ago

On 10/28/25 1:35 AM, Markus Armbruster wrote:
> Donald Dutile <ddutile@redhat.com> writes:
> 
>> On 10/27/25 10:13 AM, Eric Auger wrote:
> 
> [...]
> 
>>> Looks good to me as well
>>> Fell free to add my R-b
>>> Reviewed-by: Eric Auger <eric.auger@redhat.com>
>>> Thanks!
>>> Eric
>>>
>> +1; looks good; can add my r-b if another is desired/needed.
>> - Don
> 
> When you put in the review work, claim the review credit!  Suggest you
> spell out your R-by.
> 
Trivial doc; I reviewed it previously; Shameer modified it per Peter's suggestion/request.

I added my USD $0.01 ... :-/

RE: [PATCH v2] docs/system/arm/virt: Document user-creatable SMMUv3

Posted by Shameer Kolothum 3 months, 2 weeks ago


> -----Original Message-----
> From: Peter Maydell <peter.maydell@linaro.org>
> Sent: 27 October 2025 11:28
> To: Shameer Kolothum <skolothumtho@nvidia.com>
> Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> eric.auger@redhat.com; Nicolin Chen <nicolinc@nvidia.com>;
> ddutile@redhat.com; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
> <mochs@nvidia.com>; jonathan.cameron@huawei.com
> Subject: Re: [PATCH v2] docs/system/arm/virt: Document user-creatable
> SMMUv3
> 
> External email: Use caution opening links or attachments
> 
> 
> On Fri, 24 Oct 2025 at 09:46, Shameer Kolothum
> <skolothumtho@nvidia.com> wrote:
> >
> > The virt machine now supports creating multiple SMMUv3 instances, each
> > associated with a separate PCIe root complex.
> >
> > Update the documentation with an example.
> >
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> 
> Thanks. I have some minor word-smithing tweaks, but I'll just
> apply them to this patch in target-arm.next if you're OK with
> that:
> 
> 
> --- a/docs/system/arm/virt.rst
> +++ b/docs/system/arm/virt.rst
> @@ -224,23 +224,24 @@ SMMU configuration
>  """"""""""""""""""
> 
>  Machine-wide SMMUv3 IOMMU
> -  See the machine-specific ``iommu`` option above. This allows specifying
> -  a single, machine-wide SMMUv3 instance that applies to all devices in
> -  the PCIe topology.
> +  Setting the machine-specific option ``iommu=smmuv3`` causes QEMU to
> +  create a single, machine-wide SMMUv3 instance that applies to all
> +  devices in the PCIe topology.
> 
>    For information about selectively bypassing devices, refer to
>    ``docs/bypass-iommu.txt``.
> 
>  User-creatable SMMUv3 devices
> -  Allows creating multiple user-defined SMMUv3 devices, each associated
> -  with a separate PCIe root complex. This is only permitted if the
> -  machine-wide SMMUv3 (``iommu=smmuv3``) option is not used.
> +  You can use the ``-device arm-smmuv3`` option to create multiple
> +  user-defined SMMUv3 devices, each associated with a separate PCIe
> +  root complex. This is only permitted if the machine-wide SMMUv3
> +  (``iommu=smmuv3``) option is not used. Each ``arm-smmuv3`` device
> +  uses the ``primary-bus`` sub-option to specify which PCIe root
> +  complex it is associated with.
> 
>    This model is useful when you want to mirror a host configuration where
>    each NUMA node typically has its own SMMU, allowing the VM topology to
> -  align more closely with the host’s hardware layout. Supporting multiple
> -  SMMUv3 instances is also a prerequisite for future accelerated SMMUv3
> -  support.
> +  align more closely with the host’s hardware layout.
> 
>    Example::
> 
> 
> 
> 
> (I deleted the sentence about SMMU acceleration because that
> isn't upstream yet. When it lands then we can update the
> docs if the version of it that lands can't support acceleration
> of a single machine-wide accelerator.)

Ok. The changes look fine to me.

Thanks,
Shameer