[v1] iommu: Report effective PCIe ATS support status

[PATCH 1/2] iommu: Add device ATS not supported capability

Posted by Shameer Kolothum 1 month, 1 week ago

PCIe ATS may be disabled by platform firmware, root complex limitations,
or kernel policy even when a device advertises the ATS capability in its
PCI configuration space.

Add a new IOMMU_CAP_PCI_ATS_NOT_SUPPORTED capability to allow IOMMU
drivers to report the effective ATS decision for a device.

When this capability is returned true for a device, ATS is not
supported and not used for that device, regardless of the presence
of the PCI ATS capability.

A subsequent patch will extend iommufd to expose the effective ATS
status to userspace.

Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 include/linux/iommu.h                       | 2 ++
 drivers/iommu/amd/iommu.c                   | 6 ++++++
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 +++
 drivers/iommu/intel/iommu.c                 | 2 ++
 4 files changed, 13 insertions(+)

diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 54b8b48c762e..f40ecdc5d761 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -271,6 +271,8 @@ enum iommu_cap {
 	 */
 	IOMMU_CAP_DEFERRED_FLUSH,
 	IOMMU_CAP_DIRTY_TRACKING,	/* IOMMU supports dirty tracking */
+	/* ATS is not supported and not used on this device */
+	IOMMU_CAP_PCI_ATS_NOT_SUPPORTED,
 };
 
 /* These are the possible reserved region types */
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 81c4d7733872..aa4399b6b6db 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -2985,6 +2985,12 @@ static bool amd_iommu_capable(struct device *dev, enum iommu_cap cap)
 
 		return amd_iommu_hd_support(iommu);
 	}
+	case IOMMU_CAP_PCI_ATS_NOT_SUPPORTED: {
+		struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
+
+		return !(amd_iommu_iotlb_sup &&
+			 (dev_data->flags & AMD_IOMMU_DEVICE_FLAG_ATS_SUP));
+	}
 	default:
 		break;
 	}
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 4d00d796f078..c20d2454ca14 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -107,6 +107,7 @@ static const char * const event_class_str[] = {
 };
 
 static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master);
+static bool arm_smmu_ats_supported(struct arm_smmu_master *master);
 
 static void parse_driver_options(struct arm_smmu_device *smmu)
 {
@@ -2494,6 +2495,8 @@ static bool arm_smmu_capable(struct device *dev, enum iommu_cap cap)
 		return true;
 	case IOMMU_CAP_DIRTY_TRACKING:
 		return arm_smmu_dbm_capable(master->smmu);
+	case IOMMU_CAP_PCI_ATS_NOT_SUPPORTED:
+		return !arm_smmu_ats_supported(master);
 	default:
 		return false;
 	}
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index ef7613b177b9..0be69695e88a 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -3220,6 +3220,8 @@ static bool intel_iommu_capable(struct device *dev, enum iommu_cap cap)
 		return ecap_sc_support(info->iommu->ecap);
 	case IOMMU_CAP_DIRTY_TRACKING:
 		return ssads_supported(info->iommu);
+	case IOMMU_CAP_PCI_ATS_NOT_SUPPORTED:
+		return !info->ats_supported;
 	default:
 		return false;
 	}
-- 
2.43.0

Re: [PATCH 1/2] iommu: Add device ATS not supported capability

Posted by Jason Gunthorpe 3 weeks, 6 days ago

On Tue, Mar 03, 2026 at 03:03:47PM +0000, Shameer Kolothum wrote:
> @@ -271,6 +271,8 @@ enum iommu_cap {
>  	 */
>  	IOMMU_CAP_DEFERRED_FLUSH,
>  	IOMMU_CAP_DIRTY_TRACKING,	/* IOMMU supports dirty tracking */
> +	/* ATS is not supported and not used on this device */
> +	IOMMU_CAP_PCI_ATS_NOT_SUPPORTED,

Actually, on second thought, the kdoc should have more details about
what supported means.

Does this mean ATS is available and could be turned on ?

Or does it mean ATS is on *right now* ?

The drivers should be changing ATS dynamically depending on what
domain is attached.

I think the implementation follows the first option?

Jason

Re: [PATCH 1/2] iommu: Add device ATS not supported capability

Posted by Jason Gunthorpe 3 weeks, 6 days ago

On Tue, Mar 03, 2026 at 03:03:47PM +0000, Shameer Kolothum wrote:
> PCIe ATS may be disabled by platform firmware, root complex limitations,
> or kernel policy even when a device advertises the ATS capability in its
> PCI configuration space.
> 
> Add a new IOMMU_CAP_PCI_ATS_NOT_SUPPORTED capability to allow IOMMU
> drivers to report the effective ATS decision for a device.
> 
> When this capability is returned true for a device, ATS is not
> supported and not used for that device, regardless of the presence
> of the PCI ATS capability.
> 
> A subsequent patch will extend iommufd to expose the effective ATS
> status to userspace.
> 
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  include/linux/iommu.h                       | 2 ++
>  drivers/iommu/amd/iommu.c                   | 6 ++++++
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 +++
>  drivers/iommu/intel/iommu.c                 | 2 ++
>  4 files changed, 13 insertions(+)
> 
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 54b8b48c762e..f40ecdc5d761 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -271,6 +271,8 @@ enum iommu_cap {
>  	 */
>  	IOMMU_CAP_DEFERRED_FLUSH,
>  	IOMMU_CAP_DIRTY_TRACKING,	/* IOMMU supports dirty tracking */
> +	/* ATS is not supported and not used on this device */
> +	IOMMU_CAP_PCI_ATS_NOT_SUPPORTED,

I think this one we should invert so it has saner semantics in-kernel
and for the drivers.

IOMMU_CAP_PCI_ATS_SUPPORTED

And then if not implemented it follows the usual cap protocol meaning
the iommu driver does nto support ATS.

It can be inverted by iommufd for the uapi.

Looks OK otherwise

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>

Jason

Re: [PATCH 1/2] iommu: Add device ATS not supported capability

Posted by Samiullah Khawaja 1 month ago

On Tue, Mar 03, 2026 at 03:03:47PM +0000, Shameer Kolothum wrote:
>PCIe ATS may be disabled by platform firmware, root complex limitations,
>or kernel policy even when a device advertises the ATS capability in its
>PCI configuration space.
>
>Add a new IOMMU_CAP_PCI_ATS_NOT_SUPPORTED capability to allow IOMMU
>drivers to report the effective ATS decision for a device.

nit: Maybe have IOMMU_CAP_PCI_ATS_SUPPORTED instead of the negative
variant of the capability. As this will affect only new users who might
check this capability so this does not really have to be negative.
>
>When this capability is returned true for a device, ATS is not
>supported and not used for that device, regardless of the presence
>of the PCI ATS capability.
>
>A subsequent patch will extend iommufd to expose the effective ATS
>status to userspace.
>
>Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
>Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
>---
> include/linux/iommu.h                       | 2 ++
> drivers/iommu/amd/iommu.c                   | 6 ++++++
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 +++
> drivers/iommu/intel/iommu.c                 | 2 ++
> 4 files changed, 13 insertions(+)
>
>diff --git a/include/linux/iommu.h b/include/linux/iommu.h
>index 54b8b48c762e..f40ecdc5d761 100644
>--- a/include/linux/iommu.h
>+++ b/include/linux/iommu.h
>@@ -271,6 +271,8 @@ enum iommu_cap {
> 	 */
> 	IOMMU_CAP_DEFERRED_FLUSH,
> 	IOMMU_CAP_DIRTY_TRACKING,	/* IOMMU supports dirty tracking */
>+	/* ATS is not supported and not used on this device */
>+	IOMMU_CAP_PCI_ATS_NOT_SUPPORTED,
> };
>
> /* These are the possible reserved region types */
>diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
>index 81c4d7733872..aa4399b6b6db 100644
>--- a/drivers/iommu/amd/iommu.c
>+++ b/drivers/iommu/amd/iommu.c
>@@ -2985,6 +2985,12 @@ static bool amd_iommu_capable(struct device *dev, enum iommu_cap cap)
>
> 		return amd_iommu_hd_support(iommu);
> 	}
>+	case IOMMU_CAP_PCI_ATS_NOT_SUPPORTED: {
>+		struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
>+
>+		return !(amd_iommu_iotlb_sup &&
>+			 (dev_data->flags & AMD_IOMMU_DEVICE_FLAG_ATS_SUP));
>+	}
> 	default:
> 		break;
> 	}
>diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>index 4d00d796f078..c20d2454ca14 100644
>--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>@@ -107,6 +107,7 @@ static const char * const event_class_str[] = {
> };
>
> static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master);
>+static bool arm_smmu_ats_supported(struct arm_smmu_master *master);
>
> static void parse_driver_options(struct arm_smmu_device *smmu)
> {
>@@ -2494,6 +2495,8 @@ static bool arm_smmu_capable(struct device *dev, enum iommu_cap cap)
> 		return true;
> 	case IOMMU_CAP_DIRTY_TRACKING:
> 		return arm_smmu_dbm_capable(master->smmu);
>+	case IOMMU_CAP_PCI_ATS_NOT_SUPPORTED:
>+		return !arm_smmu_ats_supported(master);
> 	default:
> 		return false;
> 	}
>diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>index ef7613b177b9..0be69695e88a 100644
>--- a/drivers/iommu/intel/iommu.c
>+++ b/drivers/iommu/intel/iommu.c
>@@ -3220,6 +3220,8 @@ static bool intel_iommu_capable(struct device *dev, enum iommu_cap cap)
> 		return ecap_sc_support(info->iommu->ecap);
> 	case IOMMU_CAP_DIRTY_TRACKING:
> 		return ssads_supported(info->iommu);
>+	case IOMMU_CAP_PCI_ATS_NOT_SUPPORTED:
>+		return !info->ats_supported;
> 	default:
> 		return false;
> 	}
>-- 
>2.43.0
>
>

Thanks,
Sami

Re: [PATCH 1/2] iommu: Add device ATS not supported capability

Posted by Jason Gunthorpe 1 month ago

On Tue, Mar 03, 2026 at 10:42:48PM +0000, Samiullah Khawaja wrote:
> On Tue, Mar 03, 2026 at 03:03:47PM +0000, Shameer Kolothum wrote:
> > PCIe ATS may be disabled by platform firmware, root complex limitations,
> > or kernel policy even when a device advertises the ATS capability in its
> > PCI configuration space.
> > 
> > Add a new IOMMU_CAP_PCI_ATS_NOT_SUPPORTED capability to allow IOMMU
> > drivers to report the effective ATS decision for a device.
> 
> nit: Maybe have IOMMU_CAP_PCI_ATS_SUPPORTED instead of the negative
> variant of the capability. As this will affect only new users who might
> check this capability so this does not really have to be negative.

It is deliberately negative for UAPI reasons.

Currently all userspace should assume ATS is supported.

The negative means new userspace that is sensitive will still work
correctly on old and new kernels that both return 0 as meaning ATS is
supported.

Perhaps this should be touched on in the commit message.

Jason

Re: [PATCH 1/2] iommu: Add device ATS not supported capability

Posted by Samiullah Khawaja 1 month ago

On Tue, Mar 03, 2026 at 07:28:35PM -0400, Jason Gunthorpe wrote:
>On Tue, Mar 03, 2026 at 10:42:48PM +0000, Samiullah Khawaja wrote:
>> On Tue, Mar 03, 2026 at 03:03:47PM +0000, Shameer Kolothum wrote:
>> > PCIe ATS may be disabled by platform firmware, root complex limitations,
>> > or kernel policy even when a device advertises the ATS capability in its
>> > PCI configuration space.
>> >
>> > Add a new IOMMU_CAP_PCI_ATS_NOT_SUPPORTED capability to allow IOMMU
>> > drivers to report the effective ATS decision for a device.
>>
>> nit: Maybe have IOMMU_CAP_PCI_ATS_SUPPORTED instead of the negative
>> variant of the capability. As this will affect only new users who might
>> check this capability so this does not really have to be negative.
>
>It is deliberately negative for UAPI reasons.
>
>Currently all userspace should assume ATS is supported.
>
>The negative means new userspace that is sensitive will still work
>correctly on old and new kernels that both return 0 as meaning ATS is
>supported.

Ah yes, for old userspace missing cap means ATS is supported.
>
>Perhaps this should be touched on in the commit message.

+1
>
>Jason