This series introduces support for AMD IOMMU nested page table translation
with the host (v1) and guest (v2) page tables.

In this mode, the AMD IOMMU driver configures the Device Table Entry (DTE)
with host page table root pointer, which is configured by allocating domain
with page table type IOMMU_HWPT_ALLOC_NEST_PARENT.

The guest page tables and Guest CR3 (GCR3) tables are managed by Guest OS,
and stored in the guest DTE (gDTE) in guest memory. VMM is responsible for
passing gDTE information to the host IOMMU driver using struct
iommu_hwpt_amd_guest when allocating a domain type IOMMU_DOMAIN_NESTED.
Then, the gDTE is parsed and program onto host DTE by the AMD IOMMU driver.

In addition, this series introduces base code for IOMMUFD vIOMMU for AMD
IOMMU, and implements vIOMMU-based nested domain allocation interface.
The struct nested_domain to store nested domain information, and
set_dte_nested() helper function to handle DTE programing for the nested
domain.

The series is separated into two parts:
 * Patch 1-5 are preparatory patches.
 * Patch 6-17 implement nest-parent and nested domains support
   for IOMMUFD vIOMMU.

Note: This series is rebased on top of:
 * Git repo: git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git 
   Branch: next
   Commit: 91920a9d87f5 ("Merge branches 'arm/smmu/updates', 'arm/smmu/bindings',
                          'mediatek', 'nvidia/tegra', 'amd/amd-vi' and 'core' 
                          into next")
 * [PATCH v5] iommu/amd: Add support for hw_info for iommu capability query
   https://lore.kernel.org/linux-iommu/20250926141901.511313-1-suravee.suthikulpanit@amd.com/T/#u 

Changes from V4:
(https://lore.kernel.org/linux-iommu/20251009235755.4497-1-suravee.suthikulpanit@amd.com)
  * Remove unused patch to modify device_flush_dte().
  * Remove unused patch to modify iommu_completion_wait()
  * Remove unused patch to modify update_dte256()
  * Patch 5: (new)
  * Patch 11:
    - Move gdomid_array from protection_domain to struct amd_iommu_viommu.
    - Introduce struct guest_domain_mapping_info.
    - Rework logic to support multiple viommu.
    - Introduce amd_iommufd_viommu_destroy() to clean up data structurs
  * Patch 13:
    - Rework based on Jason's suggestion
    - Rebase on top of gen_pt changes, which affect the logic in set_dte_entry()
  * Patch 14
    - Modify to use the new amd_iommu_update_dte() per Jason's suggestion.

Thanks,
Suravee

Suravee Suthikulpanit (14):
  iommu/amd: Rename DEV_DOMID_MASK to DTE_DOMID_MASK
  iommu/amd: Make amd_iommu_pdom_id_alloc() non-static
  iommu/amd: Make amd_iommu_pdom_id_free() non-static
  iommu/amd: Make amd_iommu_make_clear_dte() non-static inline
  iommu/amd: Introduce helper function amd_iommu_update_dte()
  iommufd: Introduce data struct for AMD nested domain allocation
  iommu/amd: Always enable GCR3TRPMode when supported.
  iommu/amd: Add support for nest parent domain allocation
  iommu/amd: Introduce struct amd_iommu_viommu
  iommu/amd: Add support for nested domain allocation
  iommu/amd: Introduce gDomID-to-hDomID Mapping and handle parent domain
    invalidation
  iommu/amd: Refactor persistent DTE bits programming into
    amd_iommu_make_clear_dte()
  iommu/amd: Refactor logic to program the host page table in DTE
  iommu/amd: Add support for nested domain attach/detach

 drivers/iommu/amd/Makefile          |   2 +-
 drivers/iommu/amd/amd_iommu.h       |  36 ++++
 drivers/iommu/amd/amd_iommu_types.h |  48 +++++-
 drivers/iommu/amd/init.c            |   8 +
 drivers/iommu/amd/iommu.c           | 221 +++++++++++++++---------
 drivers/iommu/amd/iommufd.c         |  50 ++++++
 drivers/iommu/amd/iommufd.h         |   5 +
 drivers/iommu/amd/nested.c          | 259 ++++++++++++++++++++++++++++
 include/uapi/linux/iommufd.h        |  11 ++
 9 files changed, 558 insertions(+), 82 deletions(-)
 create mode 100644 drivers/iommu/amd/nested.c

-- 
2.34.1

On Wed, Nov 12, 2025 at 06:24:52PM +0000, Suravee Suthikulpanit wrote:
> Note: This series is rebased on top of:
>  * Git repo: git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git 
>    Branch: next
>    Commit: 91920a9d87f5 ("Merge branches 'arm/smmu/updates', 'arm/smmu/bindings',
>                           'mediatek', 'nvidia/tegra', 'amd/amd-vi' and 'core' 
>                           into next")
>  * [PATCH v5] iommu/amd: Add support for hw_info for iommu capability query
>    https://lore.kernel.org/linux-iommu/20250926141901.511313-1-suravee.suthikulpanit@amd.com/T/#u 

Nit: this patch doesn't apply cleanly on 91920a9d87f5 :-/

>  drivers/iommu/amd/Makefile          |   2 +-
>  drivers/iommu/amd/amd_iommu.h       |  36 ++++
>  drivers/iommu/amd/amd_iommu_types.h |  48 +++++-
>  drivers/iommu/amd/init.c            |   8 +
>  drivers/iommu/amd/iommu.c           | 221 +++++++++++++++---------
>  drivers/iommu/amd/iommufd.c         |  50 ++++++
>  drivers/iommu/amd/iommufd.h         |   5 +
>  drivers/iommu/amd/nested.c          | 259 ++++++++++++++++++++++++++++
>  include/uapi/linux/iommufd.h        |  11 ++

So, this seems to be a preparatory series for AMD vIOMMU, yet it
doesn't properly work since it's missing IOMMUFD_VIOMMU_TYPE_AMD
and the invalidation component (HW_QUEUE).

However, the series does declare IOMMU_HWPT_DATA_AMD_GUEST in the
uAPI header. I am afraid that might confuse user who might think
AMD now supports virtualization using the HWPT-based mode, like
Intel VT-d.

So, maybe we should either:
  - leave a note at IOMMU_HWPT_DATA_AMD_GUEST to declare it is
    incomplete yet, and remove later
  - keep IOMMU_HWPT_DATA_AMD_GUEST in an AMD driver header, and
    move to the uAPI header later

Jason?

Nicolin

On Thu, Nov 13, 2025 at 01:52:02PM -0800, Nicolin Chen wrote:
> On Wed, Nov 12, 2025 at 06:24:52PM +0000, Suravee Suthikulpanit wrote:
> > Note: This series is rebased on top of:
> >  * Git repo: git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git 
> >    Branch: next
> >    Commit: 91920a9d87f5 ("Merge branches 'arm/smmu/updates', 'arm/smmu/bindings',
> >                           'mediatek', 'nvidia/tegra', 'amd/amd-vi' and 'core' 
> >                           into next")
> >  * [PATCH v5] iommu/amd: Add support for hw_info for iommu capability query
> >    https://lore.kernel.org/linux-iommu/20250926141901.511313-1-suravee.suthikulpanit@amd.com/T/#u 
> 
> Nit: this patch doesn't apply cleanly on 91920a9d87f5 :-/
> 
> >  drivers/iommu/amd/Makefile          |   2 +-
> >  drivers/iommu/amd/amd_iommu.h       |  36 ++++
> >  drivers/iommu/amd/amd_iommu_types.h |  48 +++++-
> >  drivers/iommu/amd/init.c            |   8 +
> >  drivers/iommu/amd/iommu.c           | 221 +++++++++++++++---------
> >  drivers/iommu/amd/iommufd.c         |  50 ++++++
> >  drivers/iommu/amd/iommufd.h         |   5 +
> >  drivers/iommu/amd/nested.c          | 259 ++++++++++++++++++++++++++++
> >  include/uapi/linux/iommufd.h        |  11 ++
> 
> So, this seems to be a preparatory series for AMD vIOMMU, yet it
> doesn't properly work since it's missing IOMMUFD_VIOMMU_TYPE_AMD
> and the invalidation component (HW_QUEUE).
> 
> However, the series does declare IOMMU_HWPT_DATA_AMD_GUEST in the
> uAPI header. I am afraid that might confuse user who might think
> AMD now supports virtualization using the HWPT-based mode, like
> Intel VT-d.
> 
> So, maybe we should either:
>   - leave a note at IOMMU_HWPT_DATA_AMD_GUEST to declare it is
>     incomplete yet, and remove later
>   - keep IOMMU_HWPT_DATA_AMD_GUEST in an AMD driver header, and
>     move to the uAPI header later
> 
> Jason?

Yeah, I like to see this incremental work, but Alex recently raised
that we should be a bit more careful about how userspace perceives
these partially complete things.

I don't think tricks with head files work well, I think what you'd
want to do is leave some critical system call disabled until all the
work is finished so the VMM never has to see a half working
implementation?

The patch to get the info would have been a nice choice for this purpose..

Jason

On 11/18/2025 12:54 AM, Jason Gunthorpe wrote:
> On Thu, Nov 13, 2025 at 01:52:02PM -0800, Nicolin Chen wrote:
>> On Wed, Nov 12, 2025 at 06:24:52PM +0000, Suravee Suthikulpanit wrote:
>>> Note: This series is rebased on top of:
>>>   * Git repo: git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
>>>     Branch: next
>>>     Commit: 91920a9d87f5 ("Merge branches 'arm/smmu/updates', 'arm/smmu/bindings',
>>>                            'mediatek', 'nvidia/tegra', 'amd/amd-vi' and 'core'
>>>                            into next")
>>>   * [PATCH v5] iommu/amd: Add support for hw_info for iommu capability query
>>>     https://lore.kernel.org/linux-iommu/20250926141901.511313-1-suravee.suthikulpanit@amd.com/T/#u
>>
>> Nit: this patch doesn't apply cleanly on 91920a9d87f5 :-/
>>
>>>   drivers/iommu/amd/Makefile          |   2 +-
>>>   drivers/iommu/amd/amd_iommu.h       |  36 ++++
>>>   drivers/iommu/amd/amd_iommu_types.h |  48 +++++-
>>>   drivers/iommu/amd/init.c            |   8 +
>>>   drivers/iommu/amd/iommu.c           | 221 +++++++++++++++---------
>>>   drivers/iommu/amd/iommufd.c         |  50 ++++++
>>>   drivers/iommu/amd/iommufd.h         |   5 +
>>>   drivers/iommu/amd/nested.c          | 259 ++++++++++++++++++++++++++++
>>>   include/uapi/linux/iommufd.h        |  11 ++
>>
>> So, this seems to be a preparatory series for AMD vIOMMU, yet it
>> doesn't properly work since it's missing IOMMUFD_VIOMMU_TYPE_AMD
>> and the invalidation component (HW_QUEUE).
>>
>> However, the series does declare IOMMU_HWPT_DATA_AMD_GUEST in the
>> uAPI header. I am afraid that might confuse user who might think
>> AMD now supports virtualization using the HWPT-based mode, like
>> Intel VT-d.
>>
>> So, maybe we should either:
>>    - leave a note at IOMMU_HWPT_DATA_AMD_GUEST to declare it is
>>      incomplete yet, and remove later
>>    - keep IOMMU_HWPT_DATA_AMD_GUEST in an AMD driver header, and
>>      move to the uAPI header later
>>
>> Jason?
> 
> Yeah, I like to see this incremental work, but Alex recently raised
> that we should be a bit more careful about how userspace perceives
> these partially complete things.
> 
> I don't think tricks with head files work well, I think what you'd
> want to do is leave some critical system call disabled until all the
> work is finished so the VMM never has to see a half working
> implementation?

Yes, this is a preparatory series for nested translation w/ AMD vIOMMU 
support. Currently, the amd_iommu_alloc_domain_nested() has not been 
hooked with the struct iommufd_viommu_ops.alloc_domain_nested. This is 
should prevent the nested domain usage until it is fully enabled in 
subsequent series, which will introduce the support for IOMMUFD vIOMMU 
for AMD.

I am working on preparing and sending out the next series.

Thanks,
Suravee

> The patch to get the info would have been a nice choice for this purpose..
> 
> Jason