[v1] xio3130_downstream: Add ACS (Access Control Services) capability

[PATCH] xio3130_downstream: Add ACS (Access Control Services) capability

Posted by wlfightup 1 year, 2 months ago

When vfio-pci devices are attached to the downstream, pcie acs
capability may be needed, Consistent with physical machine.

It has been tested in our environment, and pcie acs capability
is required in some scenarios.

Claim ACS support in the downstream port to allow
passthrough of individual functions of a device to different
guests (in a nested virt.setting) with VFIO.
Without this patch, all functions of a device, such as all VFs of
an SR/IOV device, will end up in the same IOMMU group.
A similar situation occurs on Windows with Hyper-V.

Signed-off-by: wlfightup <wlfightup@gmail.com>
---
 hw/pci-bridge/xio3130_downstream.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/hw/pci-bridge/xio3130_downstream.c b/hw/pci-bridge/xio3130_downstream.c
index 38a2361fa2..2017cf42a3 100644
--- a/hw/pci-bridge/xio3130_downstream.c
+++ b/hw/pci-bridge/xio3130_downstream.c
@@ -40,6 +40,8 @@
 #define XIO3130_SSVID_SSID              0
 #define XIO3130_EXP_OFFSET              0x90
 #define XIO3130_AER_OFFSET              0x100
+#define XIO3130_ACS_OFFSET \
+        (XIO3130_AER_OFFSET + PCI_ERR_SIZEOF)
 
 static void xio3130_downstream_write_config(PCIDevice *d, uint32_t address,
                                          uint32_t val, int len)
@@ -111,6 +113,10 @@ static void xio3130_downstream_realize(PCIDevice *d, Error **errp)
         goto err;
     }
 
+    if (!s->disable_acs) {
+        pcie_acs_init(d, XIO3130_ACS_OFFSET);
+    }
+
     return;
 
 err:
@@ -137,6 +143,7 @@ static void xio3130_downstream_exitfn(PCIDevice *d)
 static Property xio3130_downstream_props[] = {
     DEFINE_PROP_BIT(COMPAT_PROP_PCP, PCIDevice, cap_present,
                     QEMU_PCIE_SLTCAP_PCP_BITNR, true),
+    DEFINE_PROP_BOOL("x-disable-acs", PCIESlot, disable_acs, true),
     DEFINE_PROP_END_OF_LIST()
 };
 
-- 
2.24.3 (Apple Git-128)

Re: [PATCH] xio3130_downstream: Add ACS (Access Control Services) capability

Posted by Michael S. Tsirkin 1 year, 1 month ago

On Tue, Jan 31, 2023 at 02:30:13PM +0800, wlfightup wrote:
> When vfio-pci devices are attached to the downstream, pcie acs
> capability may be needed, Consistent with physical machine.
> 
> It has been tested in our environment, and pcie acs capability
> is required in some scenarios.
> 
> Claim ACS support in the downstream port to allow
> passthrough of individual functions of a device to different
> guests (in a nested virt.setting) with VFIO.
> Without this patch, all functions of a device, such as all VFs of
> an SR/IOV device, will end up in the same IOMMU group.
> A similar situation occurs on Windows with Hyper-V.
> 
> Signed-off-by: wlfightup <wlfightup@gmail.com>

Hmm ok but this is an unstable property and nothing sets it.
If your product starts using it, it will break when qemu
changes the property in some way.

Do we want to maybe set this by default?
If not I am guessing at least a stable property is needed.

> ---
>  hw/pci-bridge/xio3130_downstream.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/hw/pci-bridge/xio3130_downstream.c b/hw/pci-bridge/xio3130_downstream.c
> index 38a2361fa2..2017cf42a3 100644
> --- a/hw/pci-bridge/xio3130_downstream.c
> +++ b/hw/pci-bridge/xio3130_downstream.c
> @@ -40,6 +40,8 @@
>  #define XIO3130_SSVID_SSID              0
>  #define XIO3130_EXP_OFFSET              0x90
>  #define XIO3130_AER_OFFSET              0x100
> +#define XIO3130_ACS_OFFSET \
> +        (XIO3130_AER_OFFSET + PCI_ERR_SIZEOF)
>  
>  static void xio3130_downstream_write_config(PCIDevice *d, uint32_t address,
>                                           uint32_t val, int len)
> @@ -111,6 +113,10 @@ static void xio3130_downstream_realize(PCIDevice *d, Error **errp)
>          goto err;
>      }
>  
> +    if (!s->disable_acs) {
> +        pcie_acs_init(d, XIO3130_ACS_OFFSET);
> +    }
> +
>      return;
>  
>  err:
> @@ -137,6 +143,7 @@ static void xio3130_downstream_exitfn(PCIDevice *d)
>  static Property xio3130_downstream_props[] = {
>      DEFINE_PROP_BIT(COMPAT_PROP_PCP, PCIDevice, cap_present,
>                      QEMU_PCIE_SLTCAP_PCP_BITNR, true),
> +    DEFINE_PROP_BOOL("x-disable-acs", PCIESlot, disable_acs, true),
>      DEFINE_PROP_END_OF_LIST()
>  };
>  
> -- 
> 2.24.3 (Apple Git-128)

Re: [PATCH] xio3130_downstream: Add ACS (Access Control Services) capability

Posted by Philippe Mathieu-Daudé 1 year, 2 months ago

Hi Paul,

On 31/1/23 07:30, wlfightup wrote:
> When vfio-pci devices are attached to the downstream, pcie acs
> capability may be needed, Consistent with physical machine.
> 
> It has been tested in our environment, and pcie acs capability
> is required in some scenarios.
> 
> Claim ACS support in the downstream port to allow
> passthrough of individual functions of a device to different
> guests (in a nested virt.setting) with VFIO.
> Without this patch, all functions of a device, such as all VFs of
> an SR/IOV device, will end up in the same IOMMU group.
> A similar situation occurs on Windows with Hyper-V.
> 
> Signed-off-by: wlfightup <wlfightup@gmail.com>

Please use your real name, "Paul Schlacter <wlfightup@gmail.com>"
See https://www.qemu.org/docs/master/devel/submitting-a-patch.html

Cc'ing VFIO maintainers.

Regards,

Phil.

> ---
>   hw/pci-bridge/xio3130_downstream.c | 7 +++++++
>   1 file changed, 7 insertions(+)
> 
> diff --git a/hw/pci-bridge/xio3130_downstream.c b/hw/pci-bridge/xio3130_downstream.c
> index 38a2361fa2..2017cf42a3 100644
> --- a/hw/pci-bridge/xio3130_downstream.c
> +++ b/hw/pci-bridge/xio3130_downstream.c
> @@ -40,6 +40,8 @@
>   #define XIO3130_SSVID_SSID              0
>   #define XIO3130_EXP_OFFSET              0x90
>   #define XIO3130_AER_OFFSET              0x100
> +#define XIO3130_ACS_OFFSET \
> +        (XIO3130_AER_OFFSET + PCI_ERR_SIZEOF)
>   
>   static void xio3130_downstream_write_config(PCIDevice *d, uint32_t address,
>                                            uint32_t val, int len)
> @@ -111,6 +113,10 @@ static void xio3130_downstream_realize(PCIDevice *d, Error **errp)
>           goto err;
>       }
>   
> +    if (!s->disable_acs) {
> +        pcie_acs_init(d, XIO3130_ACS_OFFSET);
> +    }
> +
>       return;
>   
>   err:
> @@ -137,6 +143,7 @@ static void xio3130_downstream_exitfn(PCIDevice *d)
>   static Property xio3130_downstream_props[] = {
>       DEFINE_PROP_BIT(COMPAT_PROP_PCP, PCIDevice, cap_present,
>                       QEMU_PCIE_SLTCAP_PCP_BITNR, true),
> +    DEFINE_PROP_BOOL("x-disable-acs", PCIESlot, disable_acs, true),
>       DEFINE_PROP_END_OF_LIST()
>   };
>

Re: [PATCH] xio3130_downstream: Add ACS (Access Control Services) capability

Posted by Paul Schlacter 1 year, 2 months ago

ping

On Tue, Jan 31, 2023 at 2:30 PM wlfightup <wlfightup@gmail.com> wrote:
>
> When vfio-pci devices are attached to the downstream, pcie acs
> capability may be needed, Consistent with physical machine.
>
> It has been tested in our environment, and pcie acs capability
> is required in some scenarios.
>
> Claim ACS support in the downstream port to allow
> passthrough of individual functions of a device to different
> guests (in a nested virt.setting) with VFIO.
> Without this patch, all functions of a device, such as all VFs of
> an SR/IOV device, will end up in the same IOMMU group.
> A similar situation occurs on Windows with Hyper-V.
>
> Signed-off-by: wlfightup <wlfightup@gmail.com>
> ---
>  hw/pci-bridge/xio3130_downstream.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/hw/pci-bridge/xio3130_downstream.c b/hw/pci-bridge/xio3130_downstream.c
> index 38a2361fa2..2017cf42a3 100644
> --- a/hw/pci-bridge/xio3130_downstream.c
> +++ b/hw/pci-bridge/xio3130_downstream.c
> @@ -40,6 +40,8 @@
>  #define XIO3130_SSVID_SSID              0
>  #define XIO3130_EXP_OFFSET              0x90
>  #define XIO3130_AER_OFFSET              0x100
> +#define XIO3130_ACS_OFFSET \
> +        (XIO3130_AER_OFFSET + PCI_ERR_SIZEOF)
>
>  static void xio3130_downstream_write_config(PCIDevice *d, uint32_t address,
>                                           uint32_t val, int len)
> @@ -111,6 +113,10 @@ static void xio3130_downstream_realize(PCIDevice *d, Error **errp)
>          goto err;
>      }
>
> +    if (!s->disable_acs) {
> +        pcie_acs_init(d, XIO3130_ACS_OFFSET);
> +    }
> +
>      return;
>
>  err:
> @@ -137,6 +143,7 @@ static void xio3130_downstream_exitfn(PCIDevice *d)
>  static Property xio3130_downstream_props[] = {
>      DEFINE_PROP_BIT(COMPAT_PROP_PCP, PCIDevice, cap_present,
>                      QEMU_PCIE_SLTCAP_PCP_BITNR, true),
> +    DEFINE_PROP_BOOL("x-disable-acs", PCIESlot, disable_acs, true),
>      DEFINE_PROP_END_OF_LIST()
>  };
>
> --
> 2.24.3 (Apple Git-128)
>