[PATCH v7 17/36] hw/arm/virt: Set msi-gpa property

Shameer Kolothum posted 36 patches 4 weeks ago
There is a newer version of this series
[PATCH v7 17/36] hw/arm/virt: Set msi-gpa property
Posted by Shameer Kolothum 4 weeks ago
Set the MSI doorbell GPA property for accelerated SMMUv3 devices for use
by KVM MSI setup. Also, since any meaningful use of vfio-pci devices with
an accelerated SMMUv3 requires both KVM and a kernel irqchip, ensure
those are specified when accel=on is selected.

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/virt.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 9d0568a7d5..08feadf0a8 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -3052,6 +3052,26 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
             /* The new SMMUv3 device is specific to the PCI bus */
             object_property_set_bool(OBJECT(dev), "smmu_per_bus", true, NULL);
         }
+        if (object_property_find(OBJECT(dev), "accel") &&
+            object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
+            hwaddr db_start;
+
+            if (!kvm_enabled() || !kvm_irqchip_in_kernel()) {
+                error_setg(errp, "SMMUv3 accel=on requires KVM with "
+                           "kernel-irqchip=on support");
+                    return;
+            }
+
+            if (vms->msi_controller == VIRT_MSI_CTRL_ITS) {
+                /* GITS_TRANSLATER page + offset */
+                db_start = base_memmap[VIRT_GIC_ITS].base + 0x10000 + 0x40;
+            } else {
+                /* MSI_SETSPI_NS page + offset */
+                db_start = base_memmap[VIRT_GIC_V2M].base + 0x40;
+            }
+            object_property_set_uint(OBJECT(dev), "msi-gpa", db_start,
+                                     &error_abort);
+        }
     }
 }
 
-- 
2.43.0
Re: [PATCH v7 17/36] hw/arm/virt: Set msi-gpa property
Posted by Mohamed Mediouni 4 weeks ago

> On 11. Jan 2026, at 20:53, Shameer Kolothum <skolothumtho@nvidia.com> wrote:
> 
> Set the MSI doorbell GPA property for accelerated SMMUv3 devices for use
> by KVM MSI setup. Also, since any meaningful use of vfio-pci devices with
> an accelerated SMMUv3 requires both KVM and a kernel irqchip, ensure
> those are specified when accel=on is selected.
> 
> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
> hw/arm/virt.c | 20 ++++++++++++++++++++
> 1 file changed, 20 insertions(+)
> 
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 9d0568a7d5..08feadf0a8 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -3052,6 +3052,26 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
>             /* The new SMMUv3 device is specific to the PCI bus */
>             object_property_set_bool(OBJECT(dev), "smmu_per_bus", true, NULL);
>         }
> +        if (object_property_find(OBJECT(dev), "accel") &&
> +            object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
> +            hwaddr db_start;
> +
> +            if (!kvm_enabled() || !kvm_irqchip_in_kernel()) {
> +                error_setg(errp, "SMMUv3 accel=on requires KVM with "
> +                           "kernel-irqchip=on support");
> +                    return;
> +            }
> +
> +            if (vms->msi_controller == VIRT_MSI_CTRL_ITS) {
> +                /* GITS_TRANSLATER page + offset */
> +                db_start = base_memmap[VIRT_GIC_ITS].base + 0x10000 + 0x40;
> +            } else {
> +                /* MSI_SETSPI_NS page + offset */
> +                db_start = base_memmap[VIRT_GIC_V2M].base + 0x40;
> +            }
Hello,

Currently (but soon no longer the case for virt-11.0+), its=off means no MSI controller at all instead of
GICv3 + GICv2m. 

Would an else if with an error returned if no MSI controller is enabled be adequate?

> +            object_property_set_uint(OBJECT(dev), "msi-gpa", db_start,
> +                                     &error_abort);
> +        }
>     }
> }
> 
> -- 
> 2.43.0
> 
> 
RE: [PATCH v7 17/36] hw/arm/virt: Set msi-gpa property
Posted by Shameer Kolothum 3 weeks, 6 days ago

> -----Original Message-----
> From: Mohamed Mediouni <mohamed@unpredictable.fr>
> Sent: 11 January 2026 21:44
> To: Shameer Kolothum <skolothumtho@nvidia.com>
> Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> eric.auger@redhat.com; peter.maydell@linaro.org; Jason Gunthorpe
> <jgg@nvidia.com>; Nicolin Chen <nicolinc@nvidia.com>;
> ddutile@redhat.com; berrange@redhat.com; clg@redhat.com;
> alex@shazbot.org; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
> <mochs@nvidia.com>; smostafa@google.com; wangzhou1@hisilicon.com;
> jiangkunkun@huawei.com; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; yi.l.liu@intel.com;
> Krishnakant Jaju <kjaju@nvidia.com>
> Subject: Re: [PATCH v7 17/36] hw/arm/virt: Set msi-gpa property
> 
> External email: Use caution opening links or attachments
> 
> 
> > On 11. Jan 2026, at 20:53, Shameer Kolothum <skolothumtho@nvidia.com>
> wrote:
> >
> > Set the MSI doorbell GPA property for accelerated SMMUv3 devices for use
> > by KVM MSI setup. Also, since any meaningful use of vfio-pci devices with
> > an accelerated SMMUv3 requires both KVM and a kernel irqchip, ensure
> > those are specified when accel=on is selected.
> >
> > Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> > hw/arm/virt.c | 20 ++++++++++++++++++++
> > 1 file changed, 20 insertions(+)
> >
> > diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> > index 9d0568a7d5..08feadf0a8 100644
> > --- a/hw/arm/virt.c
> > +++ b/hw/arm/virt.c
> > @@ -3052,6 +3052,26 @@ static void
> virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
> >             /* The new SMMUv3 device is specific to the PCI bus */
> >             object_property_set_bool(OBJECT(dev), "smmu_per_bus", true,
> NULL);
> >         }
> > +        if (object_property_find(OBJECT(dev), "accel") &&
> > +            object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
> > +            hwaddr db_start;
> > +
> > +            if (!kvm_enabled() || !kvm_irqchip_in_kernel()) {
> > +                error_setg(errp, "SMMUv3 accel=on requires KVM with "
> > +                           "kernel-irqchip=on support");
> > +                    return;
> > +            }
> > +
> > +            if (vms->msi_controller == VIRT_MSI_CTRL_ITS) {
> > +                /* GITS_TRANSLATER page + offset */
> > +                db_start = base_memmap[VIRT_GIC_ITS].base + 0x10000 + 0x40;
> > +            } else {
> > +                /* MSI_SETSPI_NS page + offset */
> > +                db_start = base_memmap[VIRT_GIC_V2M].base + 0x40;
> > +            }
> Hello,
> 
> Currently (but soon no longer the case for virt-11.0+), its=off means no MSI
> controller at all instead of
> GICv3 + GICv2m.
> 
> Would an else if with an error returned if no MSI controller is enabled be
> adequate?

The MSI doorbell setup here is only required for MSI translation cases.
When ITS is off (and no MSI controller is present), passthrough devices
cannot use MSI/MSI-X, so no MSI translation is required. Is that right?

If so,  skipping the doorbell setup is expected and correct, and returning
an error would unnecessarily reject a valid configuration.

Thanks,
Shameer
Re: [PATCH v7 17/36] hw/arm/virt: Set msi-gpa property
Posted by Mohamed Mediouni 3 weeks, 6 days ago

> On 12. Jan 2026, at 10:45, Shameer Kolothum <skolothumtho@nvidia.com> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: Mohamed Mediouni <mohamed@unpredictable.fr>
>> Sent: 11 January 2026 21:44
>> To: Shameer Kolothum <skolothumtho@nvidia.com>
>> Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
>> eric.auger@redhat.com; peter.maydell@linaro.org; Jason Gunthorpe
>> <jgg@nvidia.com>; Nicolin Chen <nicolinc@nvidia.com>;
>> ddutile@redhat.com; berrange@redhat.com; clg@redhat.com;
>> alex@shazbot.org; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
>> <mochs@nvidia.com>; smostafa@google.com; wangzhou1@hisilicon.com;
>> jiangkunkun@huawei.com; jonathan.cameron@huawei.com;
>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; yi.l.liu@intel.com;
>> Krishnakant Jaju <kjaju@nvidia.com>
>> Subject: Re: [PATCH v7 17/36] hw/arm/virt: Set msi-gpa property
>> 
>> External email: Use caution opening links or attachments
>> 
>> 
>>> On 11. Jan 2026, at 20:53, Shameer Kolothum <skolothumtho@nvidia.com>
>> wrote:
>>> 
>>> Set the MSI doorbell GPA property for accelerated SMMUv3 devices for use
>>> by KVM MSI setup. Also, since any meaningful use of vfio-pci devices with
>>> an accelerated SMMUv3 requires both KVM and a kernel irqchip, ensure
>>> those are specified when accel=on is selected.
>>> 
>>> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
>>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
>>> ---
>>> hw/arm/virt.c | 20 ++++++++++++++++++++
>>> 1 file changed, 20 insertions(+)
>>> 
>>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>>> index 9d0568a7d5..08feadf0a8 100644
>>> --- a/hw/arm/virt.c
>>> +++ b/hw/arm/virt.c
>>> @@ -3052,6 +3052,26 @@ static void
>> virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
>>>            /* The new SMMUv3 device is specific to the PCI bus */
>>>            object_property_set_bool(OBJECT(dev), "smmu_per_bus", true,
>> NULL);
>>>        }
>>> +        if (object_property_find(OBJECT(dev), "accel") &&
>>> +            object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
>>> +            hwaddr db_start;
>>> +
>>> +            if (!kvm_enabled() || !kvm_irqchip_in_kernel()) {
>>> +                error_setg(errp, "SMMUv3 accel=on requires KVM with "
>>> +                           "kernel-irqchip=on support");
>>> +                    return;
>>> +            }
>>> +
>>> +            if (vms->msi_controller == VIRT_MSI_CTRL_ITS) {
>>> +                /* GITS_TRANSLATER page + offset */
>>> +                db_start = base_memmap[VIRT_GIC_ITS].base + 0x10000 + 0x40;
>>> +            } else {
>>> +                /* MSI_SETSPI_NS page + offset */
>>> +                db_start = base_memmap[VIRT_GIC_V2M].base + 0x40;
>>> +            }
>> Hello,
>> 
>> Currently (but soon no longer the case for virt-11.0+), its=off means no MSI
>> controller at all instead of
>> GICv3 + GICv2m.
>> 
>> Would an else if with an error returned if no MSI controller is enabled be
>> adequate?
> 
> The MSI doorbell setup here is only required for MSI translation cases.
> When ITS is off (and no MSI controller is present), passthrough devices
> cannot use MSI/MSI-X, so no MSI translation is required. Is that right?
> 
> If so,  skipping the doorbell setup is expected and correct, and returning
> an error would unnecessarily reject a valid configuration.
Hello,

Could it be better then to have an else if for the GICv2m case and a separate 
else with setting db_start to 0 or something instead of an invalid (because not 
present device) address to make things clearer?

Thank you,
-Mohamed 
> Thanks,
> Shameer
> 
RE: [PATCH v7 17/36] hw/arm/virt: Set msi-gpa property
Posted by Shameer Kolothum 3 weeks, 6 days ago

> -----Original Message-----
> From: Mohamed Mediouni <mohamed@unpredictable.fr>
> Sent: 12 January 2026 10:18
> To: Shameer Kolothum <skolothumtho@nvidia.com>
> Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> eric.auger@redhat.com; peter.maydell@linaro.org; Jason Gunthorpe
> <jgg@nvidia.com>; Nicolin Chen <nicolinc@nvidia.com>; ddutile@redhat.com;
> berrange@redhat.com; clg@redhat.com; alex@shazbot.org; Nathan Chen
> <nathanc@nvidia.com>; Matt Ochs <mochs@nvidia.com>;
> smostafa@google.com; wangzhou1@hisilicon.com;
> jiangkunkun@huawei.com; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; yi.l.liu@intel.com;
> Krishnakant Jaju <kjaju@nvidia.com>
> Subject: Re: [PATCH v7 17/36] hw/arm/virt: Set msi-gpa property
> 
> External email: Use caution opening links or attachments
> 
> 
> > On 12. Jan 2026, at 10:45, Shameer Kolothum
> <skolothumtho@nvidia.com> wrote:
> >
> >
> >
> >> -----Original Message-----
> >> From: Mohamed Mediouni <mohamed@unpredictable.fr>
> >> Sent: 11 January 2026 21:44
> >> To: Shameer Kolothum <skolothumtho@nvidia.com>
> >> Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> >> eric.auger@redhat.com; peter.maydell@linaro.org; Jason Gunthorpe
> >> <jgg@nvidia.com>; Nicolin Chen <nicolinc@nvidia.com>;
> >> ddutile@redhat.com; berrange@redhat.com; clg@redhat.com;
> >> alex@shazbot.org; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
> >> <mochs@nvidia.com>; smostafa@google.com; wangzhou1@hisilicon.com;
> >> jiangkunkun@huawei.com; jonathan.cameron@huawei.com;
> >> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; yi.l.liu@intel.com;
> >> Krishnakant Jaju <kjaju@nvidia.com>
> >> Subject: Re: [PATCH v7 17/36] hw/arm/virt: Set msi-gpa property
> >>
> >> External email: Use caution opening links or attachments
> >>
> >>
> >>> On 11. Jan 2026, at 20:53, Shameer Kolothum
> <skolothumtho@nvidia.com>
> >> wrote:
> >>>
> >>> Set the MSI doorbell GPA property for accelerated SMMUv3 devices for
> use
> >>> by KVM MSI setup. Also, since any meaningful use of vfio-pci devices with
> >>> an accelerated SMMUv3 requires both KVM and a kernel irqchip, ensure
> >>> those are specified when accel=on is selected.
> >>>
> >>> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
> >>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> >>> ---
> >>> hw/arm/virt.c | 20 ++++++++++++++++++++
> >>> 1 file changed, 20 insertions(+)
> >>>
> >>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> >>> index 9d0568a7d5..08feadf0a8 100644
> >>> --- a/hw/arm/virt.c
> >>> +++ b/hw/arm/virt.c
> >>> @@ -3052,6 +3052,26 @@ static void
> >> virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
> >>>            /* The new SMMUv3 device is specific to the PCI bus */
> >>>            object_property_set_bool(OBJECT(dev), "smmu_per_bus", true,
> >> NULL);
> >>>        }
> >>> +        if (object_property_find(OBJECT(dev), "accel") &&
> >>> +            object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
> >>> +            hwaddr db_start;
> >>> +
> >>> +            if (!kvm_enabled() || !kvm_irqchip_in_kernel()) {
> >>> +                error_setg(errp, "SMMUv3 accel=on requires KVM with "
> >>> +                           "kernel-irqchip=on support");
> >>> +                    return;
> >>> +            }
> >>> +
> >>> +            if (vms->msi_controller == VIRT_MSI_CTRL_ITS) {
> >>> +                /* GITS_TRANSLATER page + offset */
> >>> +                db_start = base_memmap[VIRT_GIC_ITS].base + 0x10000 +
> 0x40;
> >>> +            } else {
> >>> +                /* MSI_SETSPI_NS page + offset */
> >>> +                db_start = base_memmap[VIRT_GIC_V2M].base + 0x40;
> >>> +            }
> >> Hello,
> >>
> >> Currently (but soon no longer the case for virt-11.0+), its=off means no MSI
> >> controller at all instead of
> >> GICv3 + GICv2m.
> >>
> >> Would an else if with an error returned if no MSI controller is enabled be
> >> adequate?
> >
> > The MSI doorbell setup here is only required for MSI translation cases.
> > When ITS is off (and no MSI controller is present), passthrough devices
> > cannot use MSI/MSI-X, so no MSI translation is required. Is that right?
> >
> > If so,  skipping the doorbell setup is expected and correct, and returning
> > an error would unnecessarily reject a valid configuration.
> Hello,
> 
> Could it be better then to have an else if for the GICv2m case and a separate
> else with setting db_start to 0 or something instead of an invalid (because not
> present device) address to make things clearer?

Sure, will do.

Thanks,
Shameer