[PATCH v6 08/33] hw/pci/pci: Add optional supports_address_space() callback

Shameer Kolothum posted 33 patches 3 weeks, 2 days ago
[PATCH v6 08/33] hw/pci/pci: Add optional supports_address_space() callback
Posted by Shameer Kolothum 3 weeks, 2 days ago
Introduce an optional supports_address_space() callback in PCIIOMMUOps to
allow a vIOMMU implementation to reject devices that should not be attached
to it.

Currently, get_address_space() is the first and mandatory callback into the
vIOMMU layer, which always returns an address space. For certain setups, such
as hardware accelerated vIOMMUs (e.g. ARM SMMUv3 with accel=on), attaching
emulated endpoint devices is undesirable as it may impact the behavior or
performance of VFIO passthrough devices, for example, by triggering
unnecessary invalidations on the host IOMMU.

The new callback allows a vIOMMU to check and reject unsupported devices
early during PCI device registration.

Cc: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/pci/pci.c         | 20 ++++++++++++++++++++
 include/hw/pci/pci.h | 17 +++++++++++++++++
 2 files changed, 37 insertions(+)

diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index af32ab4adb..55647a6928 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -135,6 +135,21 @@ static void pci_set_master(PCIDevice *d, bool enable)
     d->is_master = enable; /* cache the status */
 }
 
+static bool
+pci_device_supports_iommu_address_space(PCIDevice *dev, Error **errp)
+{
+    PCIBus *bus;
+    PCIBus *iommu_bus;
+    int devfn;
+
+    pci_device_get_iommu_bus_devfn(dev, &iommu_bus, &bus, &devfn);
+    if (iommu_bus && iommu_bus->iommu_ops->supports_address_space) {
+        return iommu_bus->iommu_ops->supports_address_space(bus,
+                                iommu_bus->iommu_opaque, devfn, errp);
+    }
+    return true;
+}
+
 static void pci_init_bus_master(PCIDevice *pci_dev)
 {
     AddressSpace *dma_as = pci_device_iommu_address_space(pci_dev);
@@ -1424,6 +1439,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
     pci_dev->config_write = config_write;
     bus->devices[devfn] = pci_dev;
     pci_dev->version_id = 2; /* Current pci device vmstate version */
+    if (!pci_device_supports_iommu_address_space(pci_dev, errp)) {
+        do_pci_unregister_device(pci_dev);
+        bus->devices[devfn] = NULL;
+        return NULL;
+    }
     if (phase_check(PHASE_MACHINE_READY)) {
         pci_init_bus_master(pci_dev);
     }
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index a3ca54859c..dd1c4483a2 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -417,6 +417,23 @@ typedef struct IOMMUPRINotifier {
  * framework for a set of devices on a PCI bus.
  */
 typedef struct PCIIOMMUOps {
+    /**
+     * @supports_address_space: Optional pre-check to determine if a PCI
+     * device can have an IOMMU address space.
+     *
+     * @bus: the #PCIBus being accessed.
+     *
+     * @opaque: the data passed to pci_setup_iommu().
+     *
+     * @devfn: device and function number.
+     *
+     * @errp: pass an Error out only when return false
+     *
+     * Returns: true if the device can be associated with an IOMMU address
+     * space, false otherwise with errp set.
+     */
+    bool (*supports_address_space)(PCIBus *bus, void *opaque, int devfn,
+                                   Error **errp);
     /**
      * @get_address_space: get the address space for a set of devices
      * on a PCI bus.
-- 
2.43.0
Re: [PATCH v6 08/33] hw/pci/pci: Add optional supports_address_space() callback
Posted by Cédric Le Goater 2 days, 15 hours ago
Hello,

On 11/20/25 14:21, Shameer Kolothum wrote:
> Introduce an optional supports_address_space() callback in PCIIOMMUOps to
> allow a vIOMMU implementation to reject devices that should not be attached
> to it.

Why can't we use the existing .set_iommu_device() op instead ?

C.

> 
> Currently, get_address_space() is the first and mandatory callback into the
> vIOMMU layer, which always returns an address space. For certain setups, such
> as hardware accelerated vIOMMUs (e.g. ARM SMMUv3 with accel=on), attaching
> emulated endpoint devices is undesirable as it may impact the behavior or
> performance of VFIO passthrough devices, for example, by triggering
> unnecessary invalidations on the host IOMMU.
> 
> The new callback allows a vIOMMU to check and reject unsupported devices
> early during PCI device registration.
> 
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>   hw/pci/pci.c         | 20 ++++++++++++++++++++
>   include/hw/pci/pci.h | 17 +++++++++++++++++
>   2 files changed, 37 insertions(+)
> 
> diff --git a/hw/pci/pci.c b/hw/pci/pci.c
> index af32ab4adb..55647a6928 100644
> --- a/hw/pci/pci.c
> +++ b/hw/pci/pci.c
> @@ -135,6 +135,21 @@ static void pci_set_master(PCIDevice *d, bool enable)
>       d->is_master = enable; /* cache the status */
>   }
>   
> +static bool
> +pci_device_supports_iommu_address_space(PCIDevice *dev, Error **errp)
> +{
> +    PCIBus *bus;
> +    PCIBus *iommu_bus;
> +    int devfn;
> +
> +    pci_device_get_iommu_bus_devfn(dev, &iommu_bus, &bus, &devfn);
> +    if (iommu_bus && iommu_bus->iommu_ops->supports_address_space) {
> +        return iommu_bus->iommu_ops->supports_address_space(bus,
> +                                iommu_bus->iommu_opaque, devfn, errp);
> +    }
> +    return true;
> +}
> +
>   static void pci_init_bus_master(PCIDevice *pci_dev)
>   {
>       AddressSpace *dma_as = pci_device_iommu_address_space(pci_dev);
> @@ -1424,6 +1439,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
>       pci_dev->config_write = config_write;
>       bus->devices[devfn] = pci_dev;
>       pci_dev->version_id = 2; /* Current pci device vmstate version */
> +    if (!pci_device_supports_iommu_address_space(pci_dev, errp)) {
> +        do_pci_unregister_device(pci_dev);
> +        bus->devices[devfn] = NULL;
> +        return NULL;
> +    }
>       if (phase_check(PHASE_MACHINE_READY)) {
>           pci_init_bus_master(pci_dev);
>       }
> diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
> index a3ca54859c..dd1c4483a2 100644
> --- a/include/hw/pci/pci.h
> +++ b/include/hw/pci/pci.h
> @@ -417,6 +417,23 @@ typedef struct IOMMUPRINotifier {
>    * framework for a set of devices on a PCI bus.
>    */
>   typedef struct PCIIOMMUOps {
> +    /**
> +     * @supports_address_space: Optional pre-check to determine if a PCI
> +     * device can have an IOMMU address space.
> +     *
> +     * @bus: the #PCIBus being accessed.
> +     *
> +     * @opaque: the data passed to pci_setup_iommu().
> +     *
> +     * @devfn: device and function number.
> +     *
> +     * @errp: pass an Error out only when return false
> +     *
> +     * Returns: true if the device can be associated with an IOMMU address
> +     * space, false otherwise with errp set.
> +     */
> +    bool (*supports_address_space)(PCIBus *bus, void *opaque, int devfn,
> +                                   Error **errp);
>       /**
>        * @get_address_space: get the address space for a set of devices
>        * on a PCI bus.
RE: [PATCH v6 08/33] hw/pci/pci: Add optional supports_address_space() callback
Posted by Shameer Kolothum 1 day, 23 hours ago

> -----Original Message-----
> From: Cédric Le Goater <clg@redhat.com>
> Sent: 11 December 2025 14:40
> To: Shameer Kolothum <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: eric.auger@redhat.com; peter.maydell@linaro.org; Jason Gunthorpe
> <jgg@nvidia.com>; Nicolin Chen <nicolinc@nvidia.com>;
> ddutile@redhat.com; berrange@redhat.com; Nathan Chen
> <nathanc@nvidia.com>; Matt Ochs <mochs@nvidia.com>;
> smostafa@google.com; wangzhou1@hisilicon.com;
> jiangkunkun@huawei.com; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; yi.l.liu@intel.com;
> Krishnakant Jaju <kjaju@nvidia.com>; Michael S . Tsirkin <mst@redhat.com>
> Subject: Re: [PATCH v6 08/33] hw/pci/pci: Add optional
> supports_address_space() callback
> 
> External email: Use caution opening links or attachments
> 
> 
> Hello,
> 
> On 11/20/25 14:21, Shameer Kolothum wrote:
> > Introduce an optional supports_address_space() callback in PCIIOMMUOps
> > to allow a vIOMMU implementation to reject devices that should not be
> > attached to it.
> 
> Why can't we use the existing .set_iommu_device() op instead ?

Because it is called a bit later after the mandatory get_address_space() callback
during pci dev registration.  And would like to fail the dev registration before that.

Thanks,
Shameer

Re: [PATCH v6 08/33] hw/pci/pci: Add optional supports_address_space() callback
Posted by Nicolin Chen 3 weeks, 2 days ago
On Thu, Nov 20, 2025 at 01:21:48PM +0000, Shameer Kolothum wrote:
> Introduce an optional supports_address_space() callback in PCIIOMMUOps to

"supports_address_space" sounds a bit to wide to me than its
indication to supporting an IOMMU address space specifically,
since the "system address space" being used in this series is
a legit address space as well.

With that being said, I think we are fine for now, given the
API docs has clarified it. If someone shares the same concern,
we can rename it later.

> allow a vIOMMU implementation to reject devices that should not be attached
> to it.
> 
> Currently, get_address_space() is the first and mandatory callback into the
> vIOMMU layer, which always returns an address space. For certain setups, such
> as hardware accelerated vIOMMUs (e.g. ARM SMMUv3 with accel=on), attaching
> emulated endpoint devices is undesirable as it may impact the behavior or
> performance of VFIO passthrough devices, for example, by triggering
> unnecessary invalidations on the host IOMMU.
> 
> The new callback allows a vIOMMU to check and reject unsupported devices
> early during PCI device registration.
> 
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
RE: [PATCH v6 08/33] hw/pci/pci: Add optional supports_address_space() callback
Posted by Shameer Kolothum 3 weeks, 1 day ago

> -----Original Message-----
> From: Nicolin Chen <nicolinc@nvidia.com>
> Sent: 20 November 2025 20:51
> To: Shameer Kolothum <skolothumtho@nvidia.com>
> Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> eric.auger@redhat.com; peter.maydell@linaro.org; Jason Gunthorpe
> <jgg@nvidia.com>; ddutile@redhat.com; berrange@redhat.com; Nathan
> Chen <nathanc@nvidia.com>; Matt Ochs <mochs@nvidia.com>;
> smostafa@google.com; wangzhou1@hisilicon.com;
> jiangkunkun@huawei.com; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; yi.l.liu@intel.com;
> Krishnakant Jaju <kjaju@nvidia.com>; Michael S . Tsirkin <mst@redhat.com>
> Subject: Re: [PATCH v6 08/33] hw/pci/pci: Add optional
> supports_address_space() callback
> 
> On Thu, Nov 20, 2025 at 01:21:48PM +0000, Shameer Kolothum wrote:
> > Introduce an optional supports_address_space() callback in PCIIOMMUOps
> to
> 
> "supports_address_space" sounds a bit to wide to me than its
> indication to supporting an IOMMU address space specifically,
> since the "system address space" being used in this series is
> a legit address space as well.
> 
> With that being said, I think we are fine for now, given the
> API docs has clarified it. If someone shares the same concern,
> we can rename it later.

The intent here is just to let the vIOMMU decide whether a device should
be associated with its address_space before we call get_address_space().
If the check passes, the vIOMMU must provide the actual address_space
through get_address_space() callback.

Sure. Open to suggestion here.

Thanks,
Shameer
Re: [PATCH v6 08/33] hw/pci/pci: Add optional supports_address_space() callback
Posted by Nicolin Chen 3 weeks, 1 day ago
On Fri, Nov 21, 2025 at 02:38:06AM -0800, Shameer Kolothum wrote:
> > -----Original Message-----
> > From: Nicolin Chen <nicolinc@nvidia.com>
> > Sent: 20 November 2025 20:51
> > To: Shameer Kolothum <skolothumtho@nvidia.com>
> > Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> > eric.auger@redhat.com; peter.maydell@linaro.org; Jason Gunthorpe
> > <jgg@nvidia.com>; ddutile@redhat.com; berrange@redhat.com; Nathan
> > Chen <nathanc@nvidia.com>; Matt Ochs <mochs@nvidia.com>;
> > smostafa@google.com; wangzhou1@hisilicon.com;
> > jiangkunkun@huawei.com; jonathan.cameron@huawei.com;
> > zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; yi.l.liu@intel.com;
> > Krishnakant Jaju <kjaju@nvidia.com>; Michael S . Tsirkin <mst@redhat.com>
> > Subject: Re: [PATCH v6 08/33] hw/pci/pci: Add optional
> > supports_address_space() callback
> > 
> > On Thu, Nov 20, 2025 at 01:21:48PM +0000, Shameer Kolothum wrote:
> > > Introduce an optional supports_address_space() callback in PCIIOMMUOps
> > to
> > 
> > "supports_address_space" sounds a bit to wide to me than its
> > indication to supporting an IOMMU address space specifically,
> > since the "system address space" being used in this series is
> > a legit address space as well.
> > 
> > With that being said, I think we are fine for now, given the
> > API docs has clarified it. If someone shares the same concern,
> > we can rename it later.
> 
> The intent here is just to let the vIOMMU decide whether a device should
> be associated with its address_space before we call get_address_space().
> If the check passes, the vIOMMU must provide the actual address_space
> through get_address_space() callback.

The naming makes sense now. Yet, the API doc is a bit confusing..

Why it says "device can have an IOMMU address space"? If a device
only has a system address space (i.e. it doesn't support an IOMMU
address space), it still returns true, right?

Nicolin
RE: [PATCH v6 08/33] hw/pci/pci: Add optional supports_address_space() callback
Posted by Shameer Kolothum 3 weeks, 1 day ago

> -----Original Message-----
> From: Nicolin Chen <nicolinc@nvidia.com>
> Sent: 21 November 2025 17:28
> To: Shameer Kolothum <skolothumtho@nvidia.com>
> Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> eric.auger@redhat.com; peter.maydell@linaro.org; Jason Gunthorpe
> <jgg@nvidia.com>; ddutile@redhat.com; berrange@redhat.com; Nathan
> Chen <nathanc@nvidia.com>; Matt Ochs <mochs@nvidia.com>;
> smostafa@google.com; wangzhou1@hisilicon.com;
> jiangkunkun@huawei.com; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; yi.l.liu@intel.com;
> Krishnakant Jaju <kjaju@nvidia.com>; Michael S . Tsirkin <mst@redhat.com>
> Subject: Re: [PATCH v6 08/33] hw/pci/pci: Add optional
> supports_address_space() callback
> 
> On Fri, Nov 21, 2025 at 02:38:06AM -0800, Shameer Kolothum wrote:
> > > -----Original Message-----
> > > From: Nicolin Chen <nicolinc@nvidia.com>
> > > Sent: 20 November 2025 20:51
> > > To: Shameer Kolothum <skolothumtho@nvidia.com>
> > > Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> > > eric.auger@redhat.com; peter.maydell@linaro.org; Jason Gunthorpe
> > > <jgg@nvidia.com>; ddutile@redhat.com; berrange@redhat.com; Nathan
> > > Chen <nathanc@nvidia.com>; Matt Ochs <mochs@nvidia.com>;
> > > smostafa@google.com; wangzhou1@hisilicon.com;
> > > jiangkunkun@huawei.com; jonathan.cameron@huawei.com;
> > > zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; yi.l.liu@intel.com;
> > > Krishnakant Jaju <kjaju@nvidia.com>; Michael S . Tsirkin
> <mst@redhat.com>
> > > Subject: Re: [PATCH v6 08/33] hw/pci/pci: Add optional
> > > supports_address_space() callback
> > >
> > > On Thu, Nov 20, 2025 at 01:21:48PM +0000, Shameer Kolothum wrote:
> > > > Introduce an optional supports_address_space() callback in
> PCIIOMMUOps
> > > to
> > >
> > > "supports_address_space" sounds a bit to wide to me than its
> > > indication to supporting an IOMMU address space specifically,
> > > since the "system address space" being used in this series is
> > > a legit address space as well.
> > >
> > > With that being said, I think we are fine for now, given the
> > > API docs has clarified it. If someone shares the same concern,
> > > we can rename it later.
> >
> > The intent here is just to let the vIOMMU decide whether a device should
> > be associated with its address_space before we call get_address_space().
> > If the check passes, the vIOMMU must provide the actual address_space
> > through get_address_space() callback.
> 
> The naming makes sense now. Yet, the API doc is a bit confusing..
> 
> Why it says "device can have an IOMMU address space"? If a device
> only has a system address space (i.e. it doesn't support an IOMMU
> address space), it still returns true, right?

Yes. I think the API doc wording requires tightening. Will do.

Thanks,
Shameer