hw/arm/smmuv3.c | 55 ++++++++++++++++++ hw/arm/virt-acpi-build.c | 119 +++++++++++++++++++++++++++++++++++---- hw/arm/virt.c | 109 ++++++++++++++++++++++++++--------- hw/core/sysbus-fdt.c | 3 + include/hw/arm/smmuv3.h | 16 ++++++ include/hw/arm/virt.h | 2 + 6 files changed, 266 insertions(+), 38 deletions(-)
Hi All, This patch series introduces support for a user-creatable SMMUv3 device (-device arm-smmuv3-dev) in QEMU. The implementation is based on feedback received from the RFCv2[0]: "hw/arm/virt: Add support for user-creatable accelerated SMMUv3" Currently, QEMU's SMMUv3 emulation (iommu=smmuv3) is tied to the machine and does not support instantiating multiple SMMUv3 devices—each associated with a separate PCIe root complex. In contrast, real-world ARM systems often include multiple SMMUv3 instances, each bound to a different PCIe root complex. This also lays the groundwork for supporting accelerated SMMUv3, as proposed in the aforementioned RFC. Please note, the accelerated SMMUv3 support is not part of this series and will be sent out as a separate series later on top of this one. Summary of changes: -Introduces support for multiple -device arm-smmuv3-dev,bus=pcie.x instances. Example usage: -device arm-smmuv3-dev,bus=pcie.0 -device virtio-net-pci,bus=pcie.0 -device pxb-pcie,id=pcie.1,bus_nr=2 -device arm-smmuv3-dev,bus=pcie.1 -device pcie-root-port,id=pcie.port1,bus=pcie.1 -device virtio-net-pci,bus=pcie.port1 -Supports either the legacy iommu=smmuv3 option or the new "-device arm-smmuv3-dev" model. -Adds device tree bindings for the new SMMUv3 device on the arm/virt machine only, and only for the default pcie.0 root complex. (Note: pxb-pcie root complexes are currently not supported with the device tree due to known limitations[1].) -Restricts usage of the new SMMUv3 device to virt machine versions > 9.2. This is to avoid creating new smmuv3 device for old virt machine versions and accidently breaking the migration support. Please take a look and let me know your feedback. Thanks, Shameer [0]:https://lore.kernel.org/qemu-devel/20250311141045.66620-1-shameerali.kolothum.thodi@huawei.com/ [1]:https://lore.kernel.org/qemu-devel/20230421165037.2506-1-Jonathan.Cameron@huawei.com/ Shameer Kolothum (5): hw/arm/smmuv3: Introduce SMMUv3 device hw/arm/virt-acpi-build: Update IORT for multiple smmuv3 devices hw/arm/virt: Factor out common SMMUV3 dt bindings code hw/arm/virt: Add support for smmuv3 device hw/arm/smmuv3: Enable smmuv3 device creation hw/arm/smmuv3.c | 55 ++++++++++++++++++ hw/arm/virt-acpi-build.c | 119 +++++++++++++++++++++++++++++++++++---- hw/arm/virt.c | 109 ++++++++++++++++++++++++++--------- hw/core/sysbus-fdt.c | 3 + include/hw/arm/smmuv3.h | 16 ++++++ include/hw/arm/virt.h | 2 + 6 files changed, 266 insertions(+), 38 deletions(-) -- 2.34.1
Shameer, Hi! First off, like the partitioning of these pieces. On 4/15/25 4:10 AM, Shameer Kolothum wrote: > Hi All, > > This patch series introduces support for a user-creatable SMMUv3 device > (-device arm-smmuv3-dev) in QEMU. > Can we drop the '-dev', as 'arm-smmu' is sufficient, as is '-device'? I know, internal to QEMU, there's already an ARM_SMMU, but as suggested later, if it uses the same struct, the qemu cmdline syntax is a bit less redundant. > The implementation is based on feedback received from the RFCv2[0]: > "hw/arm/virt: Add support for user-creatable accelerated SMMUv3" > > Currently, QEMU's SMMUv3 emulation (iommu=smmuv3) is tied to the machine > and does not support instantiating multiple SMMUv3 devices—each associated > with a separate PCIe root complex. In contrast, real-world ARM systems > often include multiple SMMUv3 instances, each bound to a different PCIe > root complex. > > This also lays the groundwork for supporting accelerated SMMUv3, as > proposed in the aforementioned RFC. Please note, the accelerated SMMUv3 > support is not part of this series and will be sent out as a separate > series later on top of this one. > > Summary of changes: > > -Introduces support for multiple -device arm-smmuv3-dev,bus=pcie.x > instances. > > Example usage: > > -device arm-smmuv3-dev,bus=pcie.0 > -device virtio-net-pci,bus=pcie.0 > -device pxb-pcie,id=pcie.1,bus_nr=2 > -device arm-smmuv3-dev,bus=pcie.1 > -device pcie-root-port,id=pcie.port1,bus=pcie.1 > -device virtio-net-pci,bus=pcie.port1 > > -Supports either the legacy iommu=smmuv3 option or the new > "-device arm-smmuv3-dev" model. > nice! :) Can it support both? i.e., some devices using a system-wide, legacy smmuv3, and some pcie devices using the -device arm-smmuv3 ? If not, then add a check to min-warn or max-fail, that config. I can see old machines being converted/upgraded to new machines, and they will want to switch from legacy->device smmuv3, and catching an edit/update to a machine change (or enabling automated conversion) would be helpful. > -Adds device tree bindings for the new SMMUv3 device on the arm/virt > machine only, and only for the default pcie.0 root complex. > (Note: pxb-pcie root complexes are currently not supported with the > device tree due to known limitations[1].) > -Restricts usage of the new SMMUv3 device to virt machine versions > 9.2. > This is to avoid creating new smmuv3 device for old virt machine versions > and accidently breaking the migration support. > > Please take a look and let me know your feedback. > > Thanks, > Shameer > [0]:https://lore.kernel.org/qemu-devel/20250311141045.66620-1-shameerali.kolothum.thodi@huawei.com/ > [1]:https://lore.kernel.org/qemu-devel/20230421165037.2506-1-Jonathan.Cameron@huawei.com/ > > Shameer Kolothum (5): > hw/arm/smmuv3: Introduce SMMUv3 device > hw/arm/virt-acpi-build: Update IORT for multiple smmuv3 devices > hw/arm/virt: Factor out common SMMUV3 dt bindings code > hw/arm/virt: Add support for smmuv3 device > hw/arm/smmuv3: Enable smmuv3 device creation > > hw/arm/smmuv3.c | 55 ++++++++++++++++++ > hw/arm/virt-acpi-build.c | 119 +++++++++++++++++++++++++++++++++++---- > hw/arm/virt.c | 109 ++++++++++++++++++++++++++--------- > hw/core/sysbus-fdt.c | 3 + > include/hw/arm/smmuv3.h | 16 ++++++ > include/hw/arm/virt.h | 2 + > 6 files changed, 266 insertions(+), 38 deletions(-) >
> -----Original Message----- > From: Donald Dutile <ddutile@redhat.com> > Sent: Friday, April 18, 2025 9:34 PM > To: Shameerali Kolothum Thodi > <shameerali.kolothum.thodi@huawei.com>; qemu-arm@nongnu.org; > qemu-devel@nongnu.org > Cc: eric.auger@redhat.com; peter.maydell@linaro.org; jgg@nvidia.com; > nicolinc@nvidia.com; berrange@redhat.com; nathanc@nvidia.com; > mochs@nvidia.com; smostafa@google.com; Linuxarm > <linuxarm@huawei.com>; Wangzhou (B) <wangzhou1@hisilicon.com>; > jiangkunkun <jiangkunkun@huawei.com>; Jonathan Cameron > <jonathan.cameron@huawei.com>; zhangfei.gao@linaro.org > Subject: Re: [PATCH 0/5] Add support for user creatable SMMUv3 device > > Shameer, > Hi! > > First off, like the partitioning of these pieces. > > On 4/15/25 4:10 AM, Shameer Kolothum wrote: > > Hi All, > > > > This patch series introduces support for a user-creatable SMMUv3 device > > (-device arm-smmuv3-dev) in QEMU. > > > Can we drop the '-dev', as 'arm-smmu' is sufficient, as is '-device'? > > I know, internal to QEMU, there's already an ARM_SMMU, but as suggested > later, > if it uses the same struct, the qemu cmdline syntax is a bit less redundant. Trust me I tried that😊. The problem is that, the legacy one doesn't have a bus associated with it and the moment we change that and have bus specified for the "-device arm-smmuv3, bus=pcie.0" the legacy smmuv3 creation in virt, create_smmu() --> qdev_new(TYPE_ARM_SMMUV3) will fails as it expects the bus type to be specified for it. I couldn't find a way to solve that. > > > The implementation is based on feedback received from the RFCv2[0]: > > "hw/arm/virt: Add support for user-creatable accelerated SMMUv3" > > > > Currently, QEMU's SMMUv3 emulation (iommu=smmuv3) is tied to the > machine > > and does not support instantiating multiple SMMUv3 devices—each > associated > > with a separate PCIe root complex. In contrast, real-world ARM systems > > often include multiple SMMUv3 instances, each bound to a different PCIe > > root complex. > > > > This also lays the groundwork for supporting accelerated SMMUv3, as > > proposed in the aforementioned RFC. Please note, the > accelerated SMMUv3 > > support is not part of this series and will be sent out as a separate > > series later on top of this one. > > > > Summary of changes: > > > > -Introduces support for multiple -device arm-smmuv3-dev,bus=pcie.x > > instances. > > > > Example usage: > > > > -device arm-smmuv3-dev,bus=pcie.0 > > -device virtio-net-pci,bus=pcie.0 > > -device pxb-pcie,id=pcie.1,bus_nr=2 > > -device arm-smmuv3-dev,bus=pcie.1 > > -device pcie-root-port,id=pcie.port1,bus=pcie.1 > > -device virtio-net-pci,bus=pcie.port1 > > > > -Supports either the legacy iommu=smmuv3 option or the new > > "-device arm-smmuv3-dev" model. > > > nice! :) > Can it support both? i.e., some devices using a system-wide, legacy > smmuv3, > and some pcie devices using the -device arm-smmuv3 ? Please see my reply to patch #2. In short No, it doesn't support both. Also I think we should slowly deprecate the use of legacy SMMUv3 going forward unless there is a strong use case/reason to support it. > If not, then add a check to min-warn or max-fail, that config. > I can see old machines being converted/upgraded to new machines, > and they will want to switch from legacy->device smmuv3, and catching > an edit/update to a machine change (or enabling automated conversion) > would be helpful. Please see Patch #4. It already checks and prevents if incompatible SMMUv3 types are specified. Or is the suggestion here is to add something extra? Please let me know. Thanks, Shameer
On 4/22/25 4:57 AM, Shameerali Kolothum Thodi wrote: > >> -----Original Message----- >> From: Donald Dutile <ddutile@redhat.com> >> Sent: Friday, April 18, 2025 9:34 PM >> To: Shameerali Kolothum Thodi >> <shameerali.kolothum.thodi@huawei.com>; qemu-arm@nongnu.org; >> qemu-devel@nongnu.org >> Cc: eric.auger@redhat.com; peter.maydell@linaro.org; jgg@nvidia.com; >> nicolinc@nvidia.com; berrange@redhat.com; nathanc@nvidia.com; >> mochs@nvidia.com; smostafa@google.com; Linuxarm >> <linuxarm@huawei.com>; Wangzhou (B) <wangzhou1@hisilicon.com>; >> jiangkunkun <jiangkunkun@huawei.com>; Jonathan Cameron >> <jonathan.cameron@huawei.com>; zhangfei.gao@linaro.org >> Subject: Re: [PATCH 0/5] Add support for user creatable SMMUv3 device >> >> Shameer, >> Hi! >> >> First off, like the partitioning of these pieces. >> >> On 4/15/25 4:10 AM, Shameer Kolothum wrote: >>> Hi All, >>> >>> This patch series introduces support for a user-creatable SMMUv3 device >>> (-device arm-smmuv3-dev) in QEMU. >>> >> Can we drop the '-dev', as 'arm-smmu' is sufficient, as is '-device'? >> >> I know, internal to QEMU, there's already an ARM_SMMU, but as suggested >> later, >> if it uses the same struct, the qemu cmdline syntax is a bit less redundant. > > Trust me I tried that😊. The problem is that, the legacy one doesn't have a bus > associated with it and the moment we change that and have bus specified for the > "-device arm-smmuv3, bus=pcie.0" the legacy smmuv3 creation in virt, > > create_smmu() --> qdev_new(TYPE_ARM_SMMUV3) > > will fails as it expects the bus type to be specified for it. I couldn't find a way > to solve that. > I guessed that's why there was new syntax for what is basically an extension of previous syntax. I'll dig more into it and see if there's a way to handle it. qemu does handle varying added params (id=blah, no id=blah), so I'm researching how that's done, to figure out how the legacy, smmu-for-all, and the smmu-for-specific pcie<RC> can both be supported. Given your stated trials and tribulations, this should be fun. ;-) - Don >> >>> The implementation is based on feedback received from the RFCv2[0]: >>> "hw/arm/virt: Add support for user-creatable accelerated SMMUv3" >>> >>> Currently, QEMU's SMMUv3 emulation (iommu=smmuv3) is tied to the >> machine >>> and does not support instantiating multiple SMMUv3 devices—each >> associated >>> with a separate PCIe root complex. In contrast, real-world ARM systems >>> often include multiple SMMUv3 instances, each bound to a different PCIe >>> root complex. >>> >>> This also lays the groundwork for supporting accelerated SMMUv3, as >>> proposed in the aforementioned RFC. Please note, the >> accelerated SMMUv3 >>> support is not part of this series and will be sent out as a separate >>> series later on top of this one. >>> >>> Summary of changes: >>> >>> -Introduces support for multiple -device arm-smmuv3-dev,bus=pcie.x >>> instances. >>> >>> Example usage: >>> >>> -device arm-smmuv3-dev,bus=pcie.0 >>> -device virtio-net-pci,bus=pcie.0 >>> -device pxb-pcie,id=pcie.1,bus_nr=2 >>> -device arm-smmuv3-dev,bus=pcie.1 >>> -device pcie-root-port,id=pcie.port1,bus=pcie.1 >>> -device virtio-net-pci,bus=pcie.port1 >>> >>> -Supports either the legacy iommu=smmuv3 option or the new >>> "-device arm-smmuv3-dev" model. >>> >> nice! :) >> Can it support both? i.e., some devices using a system-wide, legacy >> smmuv3, >> and some pcie devices using the -device arm-smmuv3 ? > > Please see my reply to patch #2. In short No, it doesn't support both. > > Also I think we should slowly deprecate the use of legacy SMMUv3 going > forward unless there is a strong use case/reason to support it. > >> If not, then add a check to min-warn or max-fail, that config. >> I can see old machines being converted/upgraded to new machines, >> and they will want to switch from legacy->device smmuv3, and catching >> an edit/update to a machine change (or enabling automated conversion) >> would be helpful. > > Please see Patch #4. It already checks and prevents if incompatible SMMUv3 > types are specified. Or is the suggestion here is to add something extra? > Please let me know. > > Thanks, > Shameer >
Hi Shameer, On 4/28/25 8:41 PM, Donald Dutile wrote: > > > On 4/22/25 4:57 AM, Shameerali Kolothum Thodi wrote: >> >>> -----Original Message----- >>> From: Donald Dutile <ddutile@redhat.com> >>> Sent: Friday, April 18, 2025 9:34 PM >>> To: Shameerali Kolothum Thodi >>> <shameerali.kolothum.thodi@huawei.com>; qemu-arm@nongnu.org; >>> qemu-devel@nongnu.org >>> Cc: eric.auger@redhat.com; peter.maydell@linaro.org; jgg@nvidia.com; >>> nicolinc@nvidia.com; berrange@redhat.com; nathanc@nvidia.com; >>> mochs@nvidia.com; smostafa@google.com; Linuxarm >>> <linuxarm@huawei.com>; Wangzhou (B) <wangzhou1@hisilicon.com>; >>> jiangkunkun <jiangkunkun@huawei.com>; Jonathan Cameron >>> <jonathan.cameron@huawei.com>; zhangfei.gao@linaro.org >>> Subject: Re: [PATCH 0/5] Add support for user creatable SMMUv3 device >>> >>> Shameer, >>> Hi! >>> >>> First off, like the partitioning of these pieces. >>> >>> On 4/15/25 4:10 AM, Shameer Kolothum wrote: >>>> Hi All, >>>> >>>> This patch series introduces support for a user-creatable SMMUv3 >>>> device >>>> (-device arm-smmuv3-dev) in QEMU. >>>> >>> Can we drop the '-dev', as 'arm-smmu' is sufficient, as is '-device'? >>> >>> I know, internal to QEMU, there's already an ARM_SMMU, but as suggested >>> later, >>> if it uses the same struct, the qemu cmdline syntax is a bit less >>> redundant. >> >> Trust me I tried that😊. The problem is that, the legacy one doesn't >> have a bus >> associated with it and the moment we change that and have bus >> specified for the >> "-device arm-smmuv3, bus=pcie.0" the legacy smmuv3 creation in virt, >> >> create_smmu() --> qdev_new(TYPE_ARM_SMMUV3) I have the impression you can achieve your goal by replacing sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); by qdev_realize_and_unref(dev, &bus->qbus, &error_fatal); See https://github.com/eauger/qemu/tree/arm-smmuv3-dev-rc0 and its top patch. indeed it would be better if we could reuse the same device. Eric >> >> will fails as it expects the bus type to be specified for it. I >> couldn't find a way >> to solve that. >> > I guessed that's why there was new syntax for what is basically an > extension of previous syntax. > I'll dig more into it and see if there's a way to handle it. > qemu does handle varying added params (id=blah, no id=blah), > so I'm researching how that's done, to figure out how the legacy, > smmu-for-all, > and the smmu-for-specific pcie<RC> can both be supported. > Given your stated trials and tribulations, this should be fun. ;-) > - Don > >>> >>>> The implementation is based on feedback received from the RFCv2[0]: >>>> "hw/arm/virt: Add support for user-creatable accelerated SMMUv3" >>>> >>>> Currently, QEMU's SMMUv3 emulation (iommu=smmuv3) is tied to the >>> machine >>>> and does not support instantiating multiple SMMUv3 devices—each >>> associated >>>> with a separate PCIe root complex. In contrast, real-world ARM systems >>>> often include multiple SMMUv3 instances, each bound to a different >>>> PCIe >>>> root complex. >>>> >>>> This also lays the groundwork for supporting accelerated SMMUv3, as >>>> proposed in the aforementioned RFC. Please note, the >>> accelerated SMMUv3 >>>> support is not part of this series and will be sent out as a separate >>>> series later on top of this one. >>>> >>>> Summary of changes: >>>> >>>> -Introduces support for multiple -device arm-smmuv3-dev,bus=pcie.x >>>> instances. >>>> >>>> Example usage: >>>> >>>> -device arm-smmuv3-dev,bus=pcie.0 >>>> -device virtio-net-pci,bus=pcie.0 >>>> -device pxb-pcie,id=pcie.1,bus_nr=2 >>>> -device arm-smmuv3-dev,bus=pcie.1 >>>> -device pcie-root-port,id=pcie.port1,bus=pcie.1 >>>> -device virtio-net-pci,bus=pcie.port1 >>>> >>>> -Supports either the legacy iommu=smmuv3 option or the new >>>> "-device arm-smmuv3-dev" model. >>>> >>> nice! :) >>> Can it support both? i.e., some devices using a system-wide, legacy >>> smmuv3, >>> and some pcie devices using the -device arm-smmuv3 ? >> >> Please see my reply to patch #2. In short No, it doesn't support both. >> >> Also I think we should slowly deprecate the use of legacy SMMUv3 going >> forward unless there is a strong use case/reason to support it. >> >>> If not, then add a check to min-warn or max-fail, that config. >>> I can see old machines being converted/upgraded to new machines, >>> and they will want to switch from legacy->device smmuv3, and catching >>> an edit/update to a machine change (or enabling automated conversion) >>> would be helpful. >> >> Please see Patch #4. It already checks and prevents if incompatible >> SMMUv3 >> types are specified. Or is the suggestion here is to add something >> extra? >> Please let me know. >> >> Thanks, >> Shameer >> >
Hi Eric, > -----Original Message----- > From: Eric Auger <eric.auger@redhat.com> > Sent: Wednesday, April 30, 2025 6:48 PM > To: Donald Dutile <ddutile@redhat.com>; Shameerali Kolothum Thodi > <shameerali.kolothum.thodi@huawei.com>; qemu-arm@nongnu.org; > qemu-devel@nongnu.org > Cc: peter.maydell@linaro.org; jgg@nvidia.com; nicolinc@nvidia.com; > berrange@redhat.com; nathanc@nvidia.com; mochs@nvidia.com; > smostafa@google.com; Linuxarm <linuxarm@huawei.com>; Wangzhou (B) > <wangzhou1@hisilicon.com>; jiangkunkun <jiangkunkun@huawei.com>; > Jonathan Cameron <jonathan.cameron@huawei.com>; > zhangfei.gao@linaro.org > Subject: Re: [PATCH 0/5] Add support for user creatable SMMUv3 device > > Hi Shameer, > > On 4/28/25 8:41 PM, Donald Dutile wrote: > > > > > > On 4/22/25 4:57 AM, Shameerali Kolothum Thodi wrote: > >> > >>> -----Original Message----- > >>> From: Donald Dutile <ddutile@redhat.com> > >>> Sent: Friday, April 18, 2025 9:34 PM > >>> To: Shameerali Kolothum Thodi > >>> <shameerali.kolothum.thodi@huawei.com>; qemu-arm@nongnu.org; > >>> qemu-devel@nongnu.org > >>> Cc: eric.auger@redhat.com; peter.maydell@linaro.org; jgg@nvidia.com; > >>> nicolinc@nvidia.com; berrange@redhat.com; nathanc@nvidia.com; > >>> mochs@nvidia.com; smostafa@google.com; Linuxarm > >>> <linuxarm@huawei.com>; Wangzhou (B) <wangzhou1@hisilicon.com>; > >>> jiangkunkun <jiangkunkun@huawei.com>; Jonathan Cameron > >>> <jonathan.cameron@huawei.com>; zhangfei.gao@linaro.org > >>> Subject: Re: [PATCH 0/5] Add support for user creatable SMMUv3 > device > >>> > >>> Shameer, > >>> Hi! > >>> > >>> First off, like the partitioning of these pieces. > >>> > >>> On 4/15/25 4:10 AM, Shameer Kolothum wrote: > >>>> Hi All, > >>>> > >>>> This patch series introduces support for a user-creatable SMMUv3 > >>>> device > >>>> (-device arm-smmuv3-dev) in QEMU. > >>>> > >>> Can we drop the '-dev', as 'arm-smmu' is sufficient, as is '-device'? > >>> > >>> I know, internal to QEMU, there's already an ARM_SMMU, but as > suggested > >>> later, > >>> if it uses the same struct, the qemu cmdline syntax is a bit less > >>> redundant. > >> > >> Trust me I tried that😊. The problem is that, the legacy one doesn't > >> have a bus > >> associated with it and the moment we change that and have bus > >> specified for the > >> "-device arm-smmuv3, bus=pcie.0" the legacy smmuv3 creation in virt, > >> > >> create_smmu() --> qdev_new(TYPE_ARM_SMMUV3) > I have the impression you can achieve your goal by replacing > sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); > by > qdev_realize_and_unref(dev, &bus->qbus, &error_fatal); > See https://github.com/eauger/qemu/tree/arm-smmuv3-dev-rc0 Cool!. That does the trick. Thanks. > and its top patch. > > indeed it would be better if we could reuse the same device. I will respin soon with the above and addressing other comments received. Thanks, Shameer
© 2016 - 2025 Red Hat, Inc.