From: Kevin Tian <kevin.tian@intel.com>

SIOV devices allows driver to tag different PASIDs for the virtual devices
within it. Such driver should call iommufd_device_bind_pasid() to connect
the pasid of the device to iommufd, and then driver is able to attach the
virtual device to IOAS/HWPT with the iommufd_device_attach() API.

Unlike physical devices, for SIOV virtual devices, iommufd_device_attach()
eventually uses the idev->default_pasid when the virtual device is attached
to an IOAS/HWPT. Also, there is no need to do immediate_attach per iommu
domain allocation in the attach/replace path if any iommu domain allocation
happens since the attach/replace is eventually pasid attach/replace.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Yi Liu <yi.l.liu@intel.com>
---
 drivers/iommu/iommufd/device.c | 21 +++++++++++++++++----
 1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c
index 35c1419ee96b..4882e3106b2e 100644
--- a/drivers/iommu/iommufd/device.c
+++ b/drivers/iommu/iommufd/device.c
@@ -841,7 +841,11 @@ int iommufd_device_attach(struct iommufd_device *idev, u32 *pt_id)
 		.pasid = IOMMU_PASID_INVALID
 	};
 
-	rc = iommufd_device_change_pt(idev, pt_id, &data);
+	if (idev->igroup)
+		rc = iommufd_device_change_pt(idev, pt_id, &data);
+	else
+		/* SIOV device follows generic pasid attach flow */
+		rc = iommufd_device_pasid_attach(idev, idev->default_pasid, pt_id);
 	if (rc)
 		return rc;
 
@@ -876,7 +880,12 @@ int iommufd_device_replace(struct iommufd_device *idev, u32 *pt_id)
 		.pasid = IOMMU_PASID_INVALID
 	};
 
-	return iommufd_device_change_pt(idev, pt_id, &data);
+	if (idev->igroup) {
+		return iommufd_device_change_pt(idev, pt_id, &data);
+	} else {
+		/* SIOV device follows generic pasid replace flow */
+		return iommufd_device_pasid_replace(idev, idev->default_pasid, pt_id);
+	}
 }
 EXPORT_SYMBOL_NS_GPL(iommufd_device_replace, IOMMUFD);
 
@@ -891,8 +900,12 @@ void iommufd_device_detach(struct iommufd_device *idev)
 {
 	struct iommufd_hw_pagetable *hwpt;
 
-	hwpt = iommufd_hw_pagetable_detach(idev);
-	iommufd_hw_pagetable_put(idev->ictx, hwpt);
+	if (idev->igroup) {
+		hwpt = iommufd_hw_pagetable_detach(idev);
+		iommufd_hw_pagetable_put(idev->ictx, hwpt);
+	} else {
+		iommufd_device_pasid_detach(idev, idev->default_pasid);
+	}
 	refcount_dec(&idev->obj.users);
 }
 EXPORT_SYMBOL_NS_GPL(iommufd_device_detach, IOMMUFD);
-- 
2.34.1

> From: Liu, Yi L <yi.l.liu@intel.com>
> Sent: Monday, October 9, 2023 4:51 PM
> 
> From: Kevin Tian <kevin.tian@intel.com>
> 
> SIOV devices allows driver to tag different PASIDs for the virtual devices
> within it. Such driver should call iommufd_device_bind_pasid() to connect
> the pasid of the device to iommufd, and then driver is able to attach the
> virtual device to IOAS/HWPT with the iommufd_device_attach() API.
> 
> Unlike physical devices, for SIOV virtual devices, iommufd_device_attach()
> eventually uses the idev->default_pasid when the virtual device is attached

s/default_pasid/rid_pasid/? or just call it idev->pasid. 'default' adds slight
confusion instead...

> to an IOAS/HWPT. Also, there is no need to do immediate_attach per iommu
> domain allocation in the attach/replace path if any iommu domain allocation
> happens since the attach/replace is eventually pasid attach/replace.

immediate_attach rationale belongs to earlier pasid attach series when
iommufd_device_pasid_attach() was introduced. Not here.

On 2023/10/10 16:24, Tian, Kevin wrote:
>> From: Liu, Yi L <yi.l.liu@intel.com>
>> Sent: Monday, October 9, 2023 4:51 PM
>>
>> From: Kevin Tian <kevin.tian@intel.com>
>>
>> SIOV devices allows driver to tag different PASIDs for the virtual devices
>> within it. Such driver should call iommufd_device_bind_pasid() to connect
>> the pasid of the device to iommufd, and then driver is able to attach the
>> virtual device to IOAS/HWPT with the iommufd_device_attach() API.
>>
>> Unlike physical devices, for SIOV virtual devices, iommufd_device_attach()
>> eventually uses the idev->default_pasid when the virtual device is attached
> 
> s/default_pasid/rid_pasid/? or just call it idev->pasid. 'default' adds slight
> confusion instead...

then let's use rid_pasid as it is used as vRID for virtual device.

>> to an IOAS/HWPT. Also, there is no need to do immediate_attach per iommu
>> domain allocation in the attach/replace path if any iommu domain allocation
>> happens since the attach/replace is eventually pasid attach/replace.
> 
> immediate_attach rationale belongs to earlier pasid attach series when
> iommufd_device_pasid_attach() was introduced. Not here.
> 

sure, will drop it.

-- 
Regards,
Yi Liu