[PATCH v2 10/10] iommu/arm-smmu-v3: Allow sharing domain across SMMUs

Nicolin Chen posted 10 patches 2 weeks, 3 days ago
[PATCH v2 10/10] iommu/arm-smmu-v3: Allow sharing domain across SMMUs
Posted by Nicolin Chen 2 weeks, 3 days ago
VMM needs a domain holding the mappings between gPA to hPA. It can be an S1
domain or an S2 nesting parent domain, depending on whether the VM is built
with a vSMMU or not.

Given that the IOAS for this gPA mapping is the same across SMMU instances,
this domain can be shared across devices even if they sit behind different
SMMUs, so long as the underlying page table is compatible between the SMMU
instances.

There is no direct information about the page table from the master device,
but a comparison can be done between the physical SMMU that the domain was
allocated for and the physical SMMU that the device is behind.

Replace the smmu test in arm_smmu_attach_dev() and arm_vsmmu_init() with a
compatibility test for the S1 and S2 cases respectively. The compatibility
test goes through the physical SMMU parameters that were used to decide the
page table formats.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   | 20 +++++++++++++++++++
 .../arm/arm-smmu-v3/arm-smmu-v3-iommufd.c     |  2 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   |  2 +-
 3 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index c7b054eb062a..c4bea9f7f4f1 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -982,6 +982,26 @@ struct arm_smmu_nested_domain {
 	__le64 ste[2];
 };
 
+static inline bool
+arm_smmu_domain_can_share(struct arm_smmu_domain *smmu_domain,
+			  struct arm_smmu_device *new_smmu)
+{
+	struct arm_smmu_device *base_smmu = smmu_domain->smmu;
+
+	if (base_smmu == new_smmu)
+		return true;
+	/* Only support identical SMMUs for now */
+	if (base_smmu->features != new_smmu->features)
+		return false;
+	if (base_smmu->iommu.ops != new_smmu->iommu.ops)
+		return false;
+	if (base_smmu->pgsize_bitmap != new_smmu->pgsize_bitmap)
+		return false;
+	if (base_smmu->ias > new_smmu->ias || base_smmu->oas > new_smmu->oas)
+		return false;
+	return true;
+}
+
 /* The following are exposed for testing purposes. */
 struct arm_smmu_entry_writer_ops;
 struct arm_smmu_entry_writer {
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
index 33b336d494c3..6ecf98ca3bb8 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
@@ -467,7 +467,7 @@ int arm_vsmmu_init(struct iommufd_viommu *viommu,
 	struct arm_smmu_domain *s2_parent = to_smmu_domain(parent_domain);
 	int id;
 
-	if (s2_parent->smmu != smmu)
+	if (!arm_smmu_domain_can_share(s2_parent, smmu))
 		return -EINVAL;
 
 	mutex_lock(&arm_smmu_asid_lock);
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 19437ee6f4e1..4252418fc0a9 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3635,7 +3635,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev,
 	state.master = master = dev_iommu_priv_get(dev);
 	smmu = master->smmu;
 
-	if (smmu_domain->smmu != smmu)
+	if (!arm_smmu_domain_can_share(smmu_domain, smmu))
 		return -EINVAL;
 
 	if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) {
-- 
2.43.0
Re: [PATCH v2 10/10] iommu/arm-smmu-v3: Allow sharing domain across SMMUs
Posted by Jason Gunthorpe 1 week, 4 days ago
On Wed, Jan 21, 2026 at 05:24:28PM -0800, Nicolin Chen wrote:
> VMM needs a domain holding the mappings between gPA to hPA. It can be an S1
> domain or an S2 nesting parent domain, depending on whether the VM is built
> with a vSMMU or not.
> 
> Given that the IOAS for this gPA mapping is the same across SMMU instances,
> this domain can be shared across devices even if they sit behind different
> SMMUs, so long as the underlying page table is compatible between the SMMU
> instances.
> 
> There is no direct information about the page table from the master device,
> but a comparison can be done between the physical SMMU that the domain was
> allocated for and the physical SMMU that the device is behind.

I would very much prefer this works by inspecting the cfg from the
iopgtable..

You can get it by doing 

	struct io_pgtable_cfg *pgtbl_cfg =
		&io_pgtable_ops_to_pgtable(domain->pgtbl_ops)->cfg;

I think it is important that the domain->smmu pointer be removed as
well

Jason
Re: [PATCH v2 10/10] iommu/arm-smmu-v3: Allow sharing domain across SMMUs
Posted by Nicolin Chen 1 week, 4 days ago
On Tue, Jan 27, 2026 at 11:41:11AM -0400, Jason Gunthorpe wrote:
> On Wed, Jan 21, 2026 at 05:24:28PM -0800, Nicolin Chen wrote:
> > VMM needs a domain holding the mappings between gPA to hPA. It can be an S1
> > domain or an S2 nesting parent domain, depending on whether the VM is built
> > with a vSMMU or not.
> > 
> > Given that the IOAS for this gPA mapping is the same across SMMU instances,
> > this domain can be shared across devices even if they sit behind different
> > SMMUs, so long as the underlying page table is compatible between the SMMU
> > instances.
> > 
> > There is no direct information about the page table from the master device,
> > but a comparison can be done between the physical SMMU that the domain was
> > allocated for and the physical SMMU that the device is behind.
> 
> I would very much prefer this works by inspecting the cfg from the
> iopgtable..
> 
> You can get it by doing 
> 
> 	struct io_pgtable_cfg *pgtbl_cfg =
> 		&io_pgtable_ops_to_pgtable(domain->pgtbl_ops)->cfg;

OK. I will make it a detailed scan

> I think it is important that the domain->smmu pointer be removed as
> well

I will try that too.

There are smmu_domain->smmu validations in two SVA functions,
which presumably might be replaced with this can_share() too.

Thanks
Nicolin
Re: [PATCH v2 10/10] iommu/arm-smmu-v3: Allow sharing domain across SMMUs
Posted by Jason Gunthorpe 1 week, 4 days ago
On Tue, Jan 27, 2026 at 09:50:55AM -0800, Nicolin Chen wrote:
> On Tue, Jan 27, 2026 at 11:41:11AM -0400, Jason Gunthorpe wrote:
> > On Wed, Jan 21, 2026 at 05:24:28PM -0800, Nicolin Chen wrote:
> > > VMM needs a domain holding the mappings between gPA to hPA. It can be an S1
> > > domain or an S2 nesting parent domain, depending on whether the VM is built
> > > with a vSMMU or not.
> > > 
> > > Given that the IOAS for this gPA mapping is the same across SMMU instances,
> > > this domain can be shared across devices even if they sit behind different
> > > SMMUs, so long as the underlying page table is compatible between the SMMU
> > > instances.
> > > 
> > > There is no direct information about the page table from the master device,
> > > but a comparison can be done between the physical SMMU that the domain was
> > > allocated for and the physical SMMU that the device is behind.
> > 
> > I would very much prefer this works by inspecting the cfg from the
> > iopgtable..
> > 
> > You can get it by doing 
> > 
> > 	struct io_pgtable_cfg *pgtbl_cfg =
> > 		&io_pgtable_ops_to_pgtable(domain->pgtbl_ops)->cfg;
> 
> OK. I will make it a detailed scan
> 
> > I think it is important that the domain->smmu pointer be removed as
> > well
> 
> I will try that too.
> 
> There are smmu_domain->smmu validations in two SVA functions,
> which presumably might be replaced with this can_share() too.

Those are because we replicate the mmu notifier for every
instance. That can now be revised too, I think. Let's leave that for
another series after this and leave the smmu pointer for now, but
don't use it outside that SVA stuff.

Jason