The vIOMMU cache_invalidate() and the nested-HWPT cache_invalidate_user()
ops are each handed the full user invalidation array and must report, via
array->entry_num, how many of its entries they handled. That makes every
driver open-code the same array walk, with real downsides:

 - each driver carries its own loop and sub-array bookkeeping;
 - the ARM SMMUv3 driver allocates a buffer sized to the whole array just
   to iterate over it;
 - hand-rolling the loop left the ARM SMMUv3 driver with two long-standing
   bugs:
    1) on a conversion failure it counts commands that it converted but
       never issued, so user space skips invalidations that never reached
       the cmdq;
    2) it rejects a zero-length array, which the uAPI documents as a valid
       request that only probes the data type.

The walk is identical for every driver, so move it into the iommufd core.

The core now drives the iteration:

 - it invokes the op on a sub-array starting at the first not-yet-handled
   entry;
 - the op handles one chunk from the front of that sub-array and reports
   the count via array->entry_num;
 - the core advances and re-invokes until the whole array is consumed or
   the op returns an error.

A driver then only has to handle one bounded chunk per call, e.g. the ARM
SMMUv3 op copies a single cmdq batch into a fixed on-stack buffer and drops
its whole-array allocation. An op still handling the entire array in one
call keeps working, so each driver converts independently.

All the bugs fixed here are long-standing ones rather than regressions in
this cycle, so the series targets for-next, not for-rc. The stable tags on
the first two patches take care of the backports.

This is on Github:
https://github.com/nicolinc/iommufd/commits/iommufd_invalidation_loop-v4

[Note to Jason and Will]
This has some conflicts with Ashish's ARM_SMMU_OPT_REPEAT_TLBI_CFGI series:
https://lore.kernel.org/all/20260609073204.1760077-1-amhetre@nvidia.com/

Changelog
v4
 * Add "Reviewed-by" from Baolu and Pranjal
 * Split the DS support out of the reject patch into a new patch-1,
   also listing DS in the iommu_hw_info_arm_smmuv3 kdoc
 * Patch-2: Add "!!" to the bool range assignment
 * Patch-2: Factor the allowlist into arm_vsmmu_validate_user_cmd()
v3
 https://lore.kernel.org/all/cover.1783539724.git.nicolinc@nvidia.com/
 * Patch-1: Add a minimal FEAT_DS detection and allow the two DS-only
   range encodings on a DS-capable SMMU
 * Patch-1: Mask the host's scale value to keep its 5-bit truncation
v2
 https://lore.kernel.org/all/cover.1783539724.git.nicolinc@nvidia.com/
 * Add "Reviewed-by" from Kevin to patches 2-5
 * Patch-1: Allow the ATC_INV Global bit gated on ssid_bits, correcting
   the wrong every-device claim: per the spec it only broadens a single
   device's invalidation across its PASIDs
 * Patch-1: Move the FEAT_ATS check into the allowlist switch
 * Patch-1: Gate the TTL range field on FEAT_RANGE_INV too
 * Patch-1: Accept only asid_bits of the ASID field
 * Patch-1: Reject Reserved range field value combinations
 * Patch-1: Reject an ATC_INV Size above 52
 * Patch-1: Add local smmu and data variables to simplify the long lines
 * Patch-1: Document the valid-command contract in the uAPI header
 * Patch-1: Note that unchecked out-of-range values are UNPREDICTABLE
 * Patch-1: Note that SSID/Global are IGNORED, not RES0, when SSV == 0
 * Patch-2: Consolidate the two invalidation loops into one
 * Patch-2: Multiply by the size_t entry_len to avoid a u32 overflow
 * Patch-3/5: Return 0 directly on a zero-length array
 * Patch-4: Use a processed counter and an out label like the mock driver
v1
 https://lore.kernel.org/all/cover.1782767398.git.nicolinc@nvidia.com/

Nicolin Chen (6):
  iommu/arm-smmu-v3: Support IDR5.DS and widen the TLBI SCALE field
  iommu/arm-smmu-v3-iommufd: Reject unsupported bits in invalidation
    commands
  iommufd: Iterate the cache invalidation array in the core
  iommufd/selftest: Convert cache invalidation mocks to the core array
    loop
  iommu/arm-smmu-v3-iommufd: Convert cache invalidation to the core
    array loop
  iommu/vt-d: Convert nested cache invalidation to the core array loop

 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |   4 +-
 include/linux/iommu.h                         |   6 +-
 include/linux/iommufd.h                       |   2 +
 include/uapi/linux/iommufd.h                  |   8 +-
 .../arm/arm-smmu-v3/arm-smmu-v3-iommufd.c     | 193 ++++++++++++++----
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   |   6 +-
 drivers/iommu/intel/nested.c                  |  54 ++---
 drivers/iommu/iommufd/hw_pagetable.c          |  25 ++-
 drivers/iommu/iommufd/selftest.c              | 147 +++++++------
 9 files changed, 288 insertions(+), 157 deletions(-)

-- 
2.43.0

On Tue, Jul 14, 2026 at 11:48:46AM -0700, Nicolin Chen wrote:
> Nicolin Chen (6):
>   iommu/arm-smmu-v3: Support IDR5.DS and widen the TLBI SCALE field
>   iommu/arm-smmu-v3-iommufd: Reject unsupported bits in invalidation
>     commands
>   iommufd: Iterate the cache invalidation array in the core
>   iommufd/selftest: Convert cache invalidation mocks to the core array
>     loop
>   iommu/arm-smmu-v3-iommufd: Convert cache invalidation to the core
>     array loop
>   iommu/vt-d: Convert nested cache invalidation to the core array loop

I was tempted to take this through the iommufd tree, but Will, I think
the first two ought to go to -stable through iommu ?

If you do that then maybe I'll pick up the rest

Thanks,
Jason

On Wed, Jul 15, 2026 at 05:15:21PM -0300, Jason Gunthorpe wrote:
> On Tue, Jul 14, 2026 at 11:48:46AM -0700, Nicolin Chen wrote:
> > Nicolin Chen (6):
> >   iommu/arm-smmu-v3: Support IDR5.DS and widen the TLBI SCALE field
> >   iommu/arm-smmu-v3-iommufd: Reject unsupported bits in invalidation
> >     commands
> >   iommufd: Iterate the cache invalidation array in the core
> >   iommufd/selftest: Convert cache invalidation mocks to the core array
> >     loop
> >   iommu/arm-smmu-v3-iommufd: Convert cache invalidation to the core
> >     array loop
> >   iommu/vt-d: Convert nested cache invalidation to the core array loop
> 
> I was tempted to take this through the iommufd tree, but Will, I think
> the first two ought to go to -stable through iommu ?
> 
> If you do that then maybe I'll pick up the rest

PATCH-5 touches arm-smmu-v3-iommufd. Merging via the iommufd tree
will conflict with the PATCH-2 in Ashish's series (via iommu):
https://lore.kernel.org/all/20260714104202.1664187-1-amhetre@nvidia.com/

Maybe it needs a shared branch; and then, Ashish can rebase?

Nicolin

On Wed, Jul 15, 2026 at 01:30:22PM -0700, Nicolin Chen wrote:
> On Wed, Jul 15, 2026 at 05:15:21PM -0300, Jason Gunthorpe wrote:
> > On Tue, Jul 14, 2026 at 11:48:46AM -0700, Nicolin Chen wrote:
> > > Nicolin Chen (6):
> > >   iommu/arm-smmu-v3: Support IDR5.DS and widen the TLBI SCALE field
> > >   iommu/arm-smmu-v3-iommufd: Reject unsupported bits in invalidation
> > >     commands
> > >   iommufd: Iterate the cache invalidation array in the core
> > >   iommufd/selftest: Convert cache invalidation mocks to the core array
> > >     loop
> > >   iommu/arm-smmu-v3-iommufd: Convert cache invalidation to the core
> > >     array loop
> > >   iommu/vt-d: Convert nested cache invalidation to the core array loop
> > 
> > I was tempted to take this through the iommufd tree, but Will, I think
> > the first two ought to go to -stable through iommu ?
> > 
> > If you do that then maybe I'll pick up the rest
> 
> PATCH-5 touches arm-smmu-v3-iommufd. Merging via the iommufd tree
> will conflict with the PATCH-2 in Ashish's series (via iommu):
> https://lore.kernel.org/all/20260714104202.1664187-1-amhetre@nvidia.com/
> 
> Maybe it needs a shared branch; and then, Ashish can rebase?

Urk this is a pain. I think Ashish's errata fix should go first so it
is easier to backport.

Jason

On Thu, Jul 16, 2026 at 12:18:45PM -0300, Jason Gunthorpe wrote:
> On Wed, Jul 15, 2026 at 01:30:22PM -0700, Nicolin Chen wrote:
> > On Wed, Jul 15, 2026 at 05:15:21PM -0300, Jason Gunthorpe wrote:
> > > On Tue, Jul 14, 2026 at 11:48:46AM -0700, Nicolin Chen wrote:
> > > > Nicolin Chen (6):
> > > >   iommu/arm-smmu-v3: Support IDR5.DS and widen the TLBI SCALE field
> > > >   iommu/arm-smmu-v3-iommufd: Reject unsupported bits in invalidation
> > > >     commands
> > > >   iommufd: Iterate the cache invalidation array in the core
> > > >   iommufd/selftest: Convert cache invalidation mocks to the core array
> > > >     loop
> > > >   iommu/arm-smmu-v3-iommufd: Convert cache invalidation to the core
> > > >     array loop
> > > >   iommu/vt-d: Convert nested cache invalidation to the core array loop
> > > 
> > > I was tempted to take this through the iommufd tree, but Will, I think
> > > the first two ought to go to -stable through iommu ?
> > > 
> > > If you do that then maybe I'll pick up the rest
> > 
> > PATCH-5 touches arm-smmu-v3-iommufd. Merging via the iommufd tree
> > will conflict with the PATCH-2 in Ashish's series (via iommu):
> > https://lore.kernel.org/all/20260714104202.1664187-1-amhetre@nvidia.com/
> > 
> > Maybe it needs a shared branch; and then, Ashish can rebase?
> 
> Urk this is a pain. I think Ashish's errata fix should go first so it
> is easier to backport.

OK. I can rebase and send a v5 after Will takes Ashish's series.

Nicolin