I've found that kdump does not work on some ARM host, such as AWS
Graviton metal, and the SMMU is what seems to kills it.

The capture kernel dies about a second after it resets the SMMU, part
way through PCI enumeration, and takes the machine with it.

There is no panic and no oops: the console stops mid-line and the box
reboots, leaving no vmcore, no log and nothing on disk. Crashes on this
hardware mostly produce no dump at all.

This is my theory, from an "outsider view":

1) A crash kexec does not run .device_shutdown(), so the capture
   kernel boots on a machine whose devices are still running.

2) They keep translating through the stream table the crashed kernel
   programmed, which sits in memory the capture kernel never touches.

3) arm_smmu_device_reset() installs a fresh table with no entry for
   those StreamIDs, so their in-flight DMA raises C_BAD_STREAMID. (seen
   when I cleared CR0_EVTQEN)

4) When the platform reports hardware errors to firmware first,
   firmware answers by resetting the machine.

If my  kdump initrd has no driver for the two ENA NICs, so nothing
resets them and they keep DMAing for as long as the capture kernel runs.

Things that will not crash/interrupt kdump:

   a) Get the ENA driver NIC into the kdump. This is costly.
   b) Removing ENA driver (or PCI device) before the crash

Debugging further, I  found that faults are invisible by default,
because the capture kernel clears CR0_EVTQEN. With the event queue left
enabled, one device accounts for all of them:

  [   36.191293] arm-smmu-v3 arm-smmu-v3.0.auto: SMMU currently enabled! Resetting...
  [   36.418009] event: C_BAD_STREAMID client: (unassigned sid) sid: 0x32d00 ssid: 0x0
  [   36.706345] event: C_BAD_STREAMID client: (unassigned sid) sid: 0x32d00 ssid: 0x0
  [   37.313703] pci 0003:02:02.2: Adding

with the platform resetting mid-line. sid 0x32d00 is 0003:2d:00.0, one of
ENA NICs.

My lovely robot and I came with two patches that solved the problem,
from a practical perspective and I want to share what has been tested.

Patch 1 carries the previous kernel's L1 descriptors into the new KDUMP table, so
those streams keep translating.

Patch 2 stops the core discarding that again: iommu_dma_init() already
enables deferred attach for a capture kernel, but the core only honours
it for drivers implementing .is_attach_deferred, which until now meant
amd and intel. Without it iommu_setup_default_domain() attaches
a default domain at probe and the fault returns as F_TRANSLATION.

The x86 IOMMU drivers solve the same/similar problem the same way; see
commit 38e5f33ee3596 ("iommu/amd: Reuse device table for kdump").

Signed-off-by: Breno Leitao <leitao@debian.org>
---
Breno Leitao (2):
      iommu/arm-smmu-v3: inherit the previous kernel's stream table in kdump
      iommu/arm-smmu-v3: defer attach in kdump so inherited entries survive probe

 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 104 ++++++++++++++++++++++++----
 1 file changed, 91 insertions(+), 13 deletions(-)
---
base-commit: f2bfbc3554ca6919484030729424b9dee2942d24
change-id: 20260911-smmu_fix_aws-95f486d8ee5d

Best regards,
--  
Breno Leitao <leitao@debian.org>

On Fri, Sep 11, 2026 at 07:54:43AM -0700, Breno Leitao wrote:
> My lovely robot and I came with two patches that solved the problem,
> from a practical perspective and I want to share what has been tested.
> 
> Patch 1 carries the previous kernel's L1 descriptors into the new KDUMP table, so
> those streams keep translating.
> 
> Patch 2 stops the core discarding that again: iommu_dma_init() already
> enables deferred attach for a capture kernel, but the core only honours
> it for drivers implementing .is_attach_deferred, which until now meant
> amd and intel. Without it iommu_setup_default_domain() attaches
> a default domain at probe and the fault returns as F_TRANSLATION.
> 
> The x86 IOMMU drivers solve the same/similar problem the same way; see
> commit 38e5f33ee3596 ("iommu/amd: Reuse device table for kdump").

This looks like my bigger series:
https://lore.kernel.org/linux-iommu/cover.1788130528.git.nicolinc@nvidia.com/

Would you please help check/test?

Thanks
Nicolin

On Fri, Sep 11, 2026 at 03:31:28PM -0700, Nicolin Chen wrote:
> On Fri, Sep 11, 2026 at 07:54:43AM -0700, Breno Leitao wrote:
> > My lovely robot and I came with two patches that solved the problem,
> > from a practical perspective and I want to share what has been tested.
> > 
> > Patch 1 carries the previous kernel's L1 descriptors into the new KDUMP table, so
> > those streams keep translating.
> > 
> > Patch 2 stops the core discarding that again: iommu_dma_init() already
> > enables deferred attach for a capture kernel, but the core only honours
> > it for drivers implementing .is_attach_deferred, which until now meant
> > amd and intel. Without it iommu_setup_default_domain() attaches
> > a default domain at probe and the fault returns as F_TRANSLATION.
> > 
> > The x86 IOMMU drivers solve the same/similar problem the same way; see
> > commit 38e5f33ee3596 ("iommu/amd: Reuse device table for kdump").
> 
> This looks like my bigger series:
> https://lore.kernel.org/linux-iommu/cover.1788130528.git.nicolinc@nvidia.com/
> 
> Would you please help check/test?

Tested, and it fixes the problem on Meta kdump kernel. Thanks for the message.
I am dropping my two patches in favour of your series.

Test it on AWS Graviton EC2 metal, 96 cores, two ENA NICs at
0003:2c:00.0 and 0003:2d:00.0. When the kernel crashes the NICs keep
bus-mastering, and the capture kernel builds a stream table with no L1
descriptor for the ENA's StreamID 0x32d00, so that DMA raises
C_BAD_STREAMID. 

Patch 11 is what makes the difference for us. "SMMU currently enabled!
Resetting..." opens every failed capture boot we have on record, and it
does not appear at all with your series applied.

Tested-by: Breno Leitao <leitao@kernel.org>

On Fri, Sep 11, 2026 at 07:54:43AM -0700, Breno Leitao wrote:
> I've found that kdump does not work on some ARM host, such as AWS
> Graviton metal, and the SMMU is what seems to kills it.
>
> The capture kernel dies about a second after it resets the SMMU, part
> way through PCI enumeration, and takes the machine with it.
>
> There is no panic and no oops: the console stops mid-line and the box
> reboots, leaving no vmcore, no log and nothing on disk. Crashes on this
> hardware mostly produce no dump at all.
>
> This is my theory, from an "outsider view":

This should really have been tagged RFC — apologies for missing that.