VMID owned by a vSMMU should be allocated in the viommu_init callback, as
HW like tegra241-cmdqv needs to setup VINTF with the VMID.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |  1 +
 .../arm/arm-smmu-v3/arm-smmu-v3-iommufd.c     | 21 +++++++++++++++++--
 .../iommu/arm/arm-smmu-v3/tegra241-cmdqv.c    |  1 +
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 696ebb89ffa3..8aecdbceb974 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -1190,6 +1190,7 @@ size_t arm_smmu_get_viommu_size(struct device *dev,
 int arm_vsmmu_init(struct iommufd_viommu *viommu,
 		   struct iommu_domain *parent_domain,
 		   const struct iommu_user_data *user_data);
+void arm_vsmmu_destroy(struct iommufd_viommu *viommu);
 int arm_smmu_attach_prepare_vmaster(struct arm_smmu_attach_state *state,
 				    struct arm_smmu_nested_domain *nested_domain);
 void arm_smmu_attach_commit_vmaster(struct arm_smmu_attach_state *state);
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
index 34c7bd4cfd84..9998871f69a0 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
@@ -404,7 +404,17 @@ int arm_vsmmu_cache_invalidate(struct iommufd_viommu *viommu,
 	return ret;
 }
 
+void arm_vsmmu_destroy(struct iommufd_viommu *viommu)
+{
+	struct arm_vsmmu *vsmmu = container_of(viommu, struct arm_vsmmu, core);
+
+	mutex_lock(&arm_smmu_asid_lock);
+	ida_free(&vsmmu->smmu->vmid_map, vsmmu->vmid);
+	mutex_unlock(&arm_smmu_asid_lock);
+}
+
 static const struct iommufd_viommu_ops arm_vsmmu_ops = {
+	.destroy = arm_vsmmu_destroy,
 	.alloc_domain_nested = arm_vsmmu_alloc_domain_nested,
 	.cache_invalidate = arm_vsmmu_cache_invalidate,
 };
@@ -454,14 +464,21 @@ int arm_vsmmu_init(struct iommufd_viommu *viommu,
 	struct arm_smmu_device *smmu =
 		container_of(viommu->iommu_dev, struct arm_smmu_device, iommu);
 	struct arm_smmu_domain *s2_parent = to_smmu_domain(parent_domain);
+	int id;
 
 	if (s2_parent->smmu != smmu)
 		return -EINVAL;
 
+	mutex_lock(&arm_smmu_asid_lock);
+	id = ida_alloc_range(&smmu->vmid_map, 1, (1 << smmu->vmid_bits) - 1,
+			     GFP_KERNEL);
+	mutex_unlock(&arm_smmu_asid_lock);
+	if (id < 0)
+		return id;
+
+	vsmmu->vmid = id;
 	vsmmu->smmu = smmu;
 	vsmmu->s2_parent = s2_parent;
-	/* FIXME Move VMID allocation from the S2 domain allocation to here */
-	vsmmu->vmid = s2_parent->s2_cfg.vmid;
 
 	if (viommu->type == IOMMU_VIOMMU_TYPE_ARM_SMMUV3) {
 		viommu->ops = &arm_vsmmu_ops;
diff --git a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
index 378104cd395e..67e72115d43a 100644
--- a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
+++ b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
@@ -1209,6 +1209,7 @@ static void tegra241_cmdqv_destroy_vintf_user(struct iommufd_viommu *viommu)
 		iommufd_viommu_destroy_mmap(&vintf->vsmmu.core,
 					    vintf->mmap_offset);
 	tegra241_cmdqv_remove_vintf(vintf->cmdqv, vintf->idx);
+	arm_vsmmu_destroy(viommu);
 }
 
 static void tegra241_vintf_destroy_vsid(struct iommufd_vdevice *vdev)
-- 
2.43.0

On Wed, Jan 21, 2026 at 05:24:24PM -0800, Nicolin Chen wrote:
> VMID owned by a vSMMU should be allocated in the viommu_init callback, as
> HW like tegra241-cmdqv needs to setup VINTF with the VMID.

Even the architected SMMU needs this, the VMS (not implemented in
Linux) should all share the same VMID for the same VM.

> +void arm_vsmmu_destroy(struct iommufd_viommu *viommu)
> +{
> +	struct arm_vsmmu *vsmmu = container_of(viommu, struct arm_vsmmu, core);
> +
> +	mutex_lock(&arm_smmu_asid_lock);
> +	ida_free(&vsmmu->smmu->vmid_map, vsmmu->vmid);
> +	mutex_unlock(&arm_smmu_asid_lock);

Need a comment explaining where the flush is.

It looks like arm_smmu_iotlb_tag_free() does the free in the case of
INV_TYPE_S2_VMID_VSMMU?

But this patch doesn't have that code yet. So maybe this should be
merged with the next patch..

Jason

On Mon, Jan 26, 2026 at 05:16:36PM -0400, Jason Gunthorpe wrote:
> On Wed, Jan 21, 2026 at 05:24:24PM -0800, Nicolin Chen wrote:
> > VMID owned by a vSMMU should be allocated in the viommu_init callback, as
> > HW like tegra241-cmdqv needs to setup VINTF with the VMID.
> 
> Even the architected SMMU needs this, the VMS (not implemented in
> Linux) should all share the same VMID for the same VM.

But for standard SMMU, the allocation/sharing could happen at the
device attachment to a nested (maybe bypass proxy) domain, right?

vmid is only needed for STE (attach) and invalidation.

Or do you see some other case where vmid must be allocated during
viommu_init?

> > +void arm_vsmmu_destroy(struct iommufd_viommu *viommu)
> > +{
> > +	struct arm_vsmmu *vsmmu = container_of(viommu, struct arm_vsmmu, core);
> > +
> > +	mutex_lock(&arm_smmu_asid_lock);
> > +	ida_free(&vsmmu->smmu->vmid_map, vsmmu->vmid);
> > +	mutex_unlock(&arm_smmu_asid_lock);
> 
> Need a comment explaining where the flush is.

Ack.

> It looks like arm_smmu_iotlb_tag_free() does the free in the case of
> INV_TYPE_S2_VMID_VSMMU?
> 
> But this patch doesn't have that code yet. So maybe this should be
> merged with the next patch..

Or maybe I should re-order the sequence of the patches. I'll see
what works the best.

Thanks
Nicolin

On Mon, Jan 26, 2026 at 07:06:24PM -0800, Nicolin Chen wrote:
> On Mon, Jan 26, 2026 at 05:16:36PM -0400, Jason Gunthorpe wrote:
> > On Wed, Jan 21, 2026 at 05:24:24PM -0800, Nicolin Chen wrote:
> > > VMID owned by a vSMMU should be allocated in the viommu_init callback, as
> > > HW like tegra241-cmdqv needs to setup VINTF with the VMID.
> > 
> > Even the architected SMMU needs this, the VMS (not implemented in
> > Linux) should all share the same VMID for the same VM.
> 
> But for standard SMMU, the allocation/sharing could happen at the
> device attachment to a nested (maybe bypass proxy) domain, right?
> 
> vmid is only needed for STE (attach) and invalidation.
> 
> Or do you see some other case where vmid must be allocated during
> viommu_init?

It has to be tied to the VSMMU, it doesn't matter if a single VMID is
allocated on-demand for the VSMMU or for the lifetime, it has to be a
single ID and shared across all STEs.

It is easy to understand that lifecycle by just allocating it for the
lifetime of the VSMMU..

Jason

On Tue, Jan 27, 2026 at 11:11:38AM -0400, Jason Gunthorpe wrote:
> On Mon, Jan 26, 2026 at 07:06:24PM -0800, Nicolin Chen wrote:
> > On Mon, Jan 26, 2026 at 05:16:36PM -0400, Jason Gunthorpe wrote:
> > > On Wed, Jan 21, 2026 at 05:24:24PM -0800, Nicolin Chen wrote:
> > > > VMID owned by a vSMMU should be allocated in the viommu_init callback, as
> > > > HW like tegra241-cmdqv needs to setup VINTF with the VMID.
> > > 
> > > Even the architected SMMU needs this, the VMS (not implemented in
> > > Linux) should all share the same VMID for the same VM.
> > 
> > But for standard SMMU, the allocation/sharing could happen at the
> > device attachment to a nested (maybe bypass proxy) domain, right?
> > 
> > vmid is only needed for STE (attach) and invalidation.
> > 
> > Or do you see some other case where vmid must be allocated during
> > viommu_init?
> 
> It has to be tied to the VSMMU, it doesn't matter if a single VMID is
> allocated on-demand for the VSMMU or for the lifetime, it has to be a
> single ID and shared across all STEs.

Attachment threads are mutexed. Any device that attaches firstly
will allocate a shareable tag and add it to the array. Any other
device will get the tag from the array and reuse it. And nothing
will change this.

> It is easy to understand that lifecycle by just allocating it for the
> lifetime of the VSMMU..

Yea, it makes sense for that matter.

So, perhaps:

VMID owned by a vSMMU should be allocated in the viommu_init callback for
 - a straightforward lifecycle for a VMID used by a vSMMU
 - HW like tegra241-cmdqv needs to setup VINTF with the VMID

Thanks
Nicolin

On Tue, Jan 27, 2026 at 09:11:14AM -0800, Nicolin Chen wrote:
> > It is easy to understand that lifecycle by just allocating it for the
> > lifetime of the VSMMU..
> 
> Yea, it makes sense for that matter.
> 
> So, perhaps:
> 
> VMID owned by a vSMMU should be allocated in the viommu_init callback for
>  - a straightforward lifecycle for a VMID used by a vSMMU
>  - HW like tegra241-cmdqv needs to setup VINTF with the VMID

Yeah, the cmdqv should take the VMID from the vSMMU and that is a good
reason to pre-allocate it since it won't go through the invalidation
list.

Jason