Kexec reboot consistently fails on ARM64 Hyper-V guests (Azure VMs).
During the kexec shutdown path, VMBus channels are never cleaned up,
so in the fresh kexec kernel startup hv_acpi_init() blocks because
the hypervisor still holds the old kernel's VMBus session open.

This is because ARM64 lacks the VMBus teardown that x86 performs during
kexec via hv_machine_shutdown(). On x86, machine_ops.shutdown is
overridden to send CHANNELMSG_UNLOAD and disable SynIC before CPUs
go offline. ARM64 has no equivalent mechanism.

Why existing notification mechanisms don't work:

- Reboot notifiers: Fire too early - they run inside
  kernel_restart_prepare(), BEFORE device_shutdown(). VMBus UNLOAD
  must happen AFTER device_shutdown() so that PCI drivers can clean
  up interrupt mappings (PCI_DELETE_INTERRUPT_MESSAGE) before UNLOAD
  force-closes channels. Sending UNLOAD first causes interrupt
  mapping leaks because the mappings aren't released on channel
  close.

- Device .shutdown callbacks: Too narrow - these handle per-device
  cleanup (e.g., individual VMBus channel teardown), but cannot
  perform bus-level operations like sending the global VMBus UNLOAD
  message or removing the SynIC CPU hotplug state via
  cpuhp_remove_state().

- VMBus parent device .shutdown callback: Also insufficient -
  cpuhp_remove_state() for SynIC teardown must run after all device
  shutdown completes, not during it. A parent .shutdown callback
  still executes within device_shutdown().

- The required window is: after device_shutdown() completes, after
  cpu_hotplug_enable(), but before smp_shutdown_nonboot_cpus(). That
  window exists inside machine_shutdown(), which currently has no hook.

The need for an ARM64 shutdown hook was previously discussed in [1]
but lacked a concrete failure case at the time. We now have one.

[1] https://lore.kernel.org/linux-arm-kernel/427a8277-49f0-4317-d6c3-4a15d7070e55@igalia.com/

This RFC proposes fixing kexec on ARM64 Hyper-V guests with:

Patch 1: A platform hook (arm64_pre_smp_shutdown_hook) in ARM64's
  machine_shutdown(), analogous to x86's machine_ops.shutdown. This
  runs after device_shutdown() and cpu_hotplug_enable(), allowing
  platform code to inject pre-shutdown logic at the right point in
  the kexec path.

  Design choices I'd like feedback on:
  - Single function pointer vs full machine_ops struct: ARM64 uses
    kernel-wide APIs (register_restart_handler, register_platform_power_off)
    for restart/poweroff rather than x86's monolithic machine_ops, so
    a targeted hook seemed more consistent with the ARM64 pattern.
  - On ARM64, machine_shutdown() is only called from kernel_kexec(),
    unlike x86/powerpc where it's also called from restart/halt/poweroff.

Patch 2: Uses this hook to call hv_kexec_handler() which performs:
  - vmbus_initiate_unload(): sends CHANNELMSG_UNLOAD to host
  - cpuhp_remove_state(): disables SynIC (SIMP, SIEFP, SINT) on
    all CPUs, ensuring the kexec'd kernel starts with clean state

Tested on ARM64 Azure VMs (Ubuntu 22.04, multiple vCPU configs):
- kexec reboot succeeds, VM comes back online with SSH
- Normal reboot/poweroff unaffected
- Multiple consecutive kexec cycles pass

Looking for feedback on:
1. Is the single function pointer hook acceptable for ARM64, or would
   the ARM64 maintainers prefer a registration API or __weak function?
2. Should this be arm64-specific or generic kexec infrastructure?

Shradha Gupta (2):
  arm64: Add pre-shutdown hook to machine_shutdown()
  arm64/hyperv: Add kexec handler using machine_shutdown hook

 arch/arm64/hyperv/mshyperv.c         | 37 ++++++++++++++++++++++++++++
 arch/arm64/include/asm/system_misc.h |  2 ++
 arch/arm64/kernel/process.c          | 13 ++++++++++
 3 files changed, 52 insertions(+)

-- 
2.43.0

On Fri, Aug 14, 2026 at 02:31:28AM -0700, Shradha Gupta wrote:
> Kexec reboot consistently fails on ARM64 Hyper-V guests (Azure VMs).
> During the kexec shutdown path, VMBus channels are never cleaned up,
> so in the fresh kexec kernel startup hv_acpi_init() blocks because
> the hypervisor still holds the old kernel's VMBus session open.
> 
> This is because ARM64 lacks the VMBus teardown that x86 performs during
> kexec via hv_machine_shutdown(). On x86, machine_ops.shutdown is
> overridden to send CHANNELMSG_UNLOAD and disable SynIC before CPUs
> go offline. ARM64 has no equivalent mechanism.

I'm really not keen on such hook, a bare pointer you set elsewhere. Not
sure it was discussed before but can you not use the syscore_shutdown()
mechanism? This hook got added in 2023, so the 2022 discussion you
referenced precedes it.

-- 
Catalin

On Fri, Aug 21, 2026 at 11:14:49AM +0100, Catalin Marinas wrote:
> On Fri, Aug 14, 2026 at 02:31:28AM -0700, Shradha Gupta wrote:
> > Kexec reboot consistently fails on ARM64 Hyper-V guests (Azure VMs).
> > During the kexec shutdown path, VMBus channels are never cleaned up,
> > so in the fresh kexec kernel startup hv_acpi_init() blocks because
> > the hypervisor still holds the old kernel's VMBus session open.
> > 
> > This is because ARM64 lacks the VMBus teardown that x86 performs during
> > kexec via hv_machine_shutdown(). On x86, machine_ops.shutdown is
> > overridden to send CHANNELMSG_UNLOAD and disable SynIC before CPUs
> > go offline. ARM64 has no equivalent mechanism.
> 
> I'm really not keen on such hook, a bare pointer you set elsewhere. Not
> sure it was discussed before but can you not use the syscore_shutdown()
> mechanism? This hook got added in 2023, so the 2022 discussion you
> referenced precedes it.
> 
> -- 
> Catalin

Thanks for the suggestion, Catalin.

I looked into syscore_shutdown() and it should work. The
syscore_shutdown() call in the kexec path runs after
device_shutdown() and before smp_shutdown_nonboot_cpus(),
which is exactly the window where VMBus teardown needs to happen.
It does run before cpu_hotplug_enable() (unlike the x86 path
where hv_kexec_handler runs inside machine_shutdown, after
cpu_hotplug_enable), but that should be safe because
cpuhp_remove_state() acquires cpus_read_lock, which should be
independent of the cpu_hotplug_disabled flag.

The implementation would override hv_setup_kexec_handler() in
arch/arm64/hyperv/mshyperv.c to register a syscore_ops.shutdown
callback, guarded by kexec_in_progress. No ARM64 arch changes needed,
and other shutdown paths (reboot, halt, poweroff, kdump) should also be
unaffected.

I am testing this now and will send a v2 with the updated approach.

Shradha

On Fri, Aug 14, 2026 at 02:31:28AM -0700, Shradha Gupta wrote:

Gentle Reminder

Regards,
Shradha.

> Kexec reboot consistently fails on ARM64 Hyper-V guests (Azure VMs).
> During the kexec shutdown path, VMBus channels are never cleaned up,
> so in the fresh kexec kernel startup hv_acpi_init() blocks because
> the hypervisor still holds the old kernel's VMBus session open.
> 
> This is because ARM64 lacks the VMBus teardown that x86 performs during
> kexec via hv_machine_shutdown(). On x86, machine_ops.shutdown is
> overridden to send CHANNELMSG_UNLOAD and disable SynIC before CPUs
> go offline. ARM64 has no equivalent mechanism.
> 
> Why existing notification mechanisms don't work:
> 
> - Reboot notifiers: Fire too early - they run inside
>   kernel_restart_prepare(), BEFORE device_shutdown(). VMBus UNLOAD
>   must happen AFTER device_shutdown() so that PCI drivers can clean
>   up interrupt mappings (PCI_DELETE_INTERRUPT_MESSAGE) before UNLOAD
>   force-closes channels. Sending UNLOAD first causes interrupt
>   mapping leaks because the mappings aren't released on channel
>   close.
> 
> - Device .shutdown callbacks: Too narrow - these handle per-device
>   cleanup (e.g., individual VMBus channel teardown), but cannot
>   perform bus-level operations like sending the global VMBus UNLOAD
>   message or removing the SynIC CPU hotplug state via
>   cpuhp_remove_state().
> 
> - VMBus parent device .shutdown callback: Also insufficient -
>   cpuhp_remove_state() for SynIC teardown must run after all device
>   shutdown completes, not during it. A parent .shutdown callback
>   still executes within device_shutdown().
> 
> - The required window is: after device_shutdown() completes, after
>   cpu_hotplug_enable(), but before smp_shutdown_nonboot_cpus(). That
>   window exists inside machine_shutdown(), which currently has no hook.
> 
> The need for an ARM64 shutdown hook was previously discussed in [1]
> but lacked a concrete failure case at the time. We now have one.
> 
> [1] https://lore.kernel.org/linux-arm-kernel/427a8277-49f0-4317-d6c3-4a15d7070e55@igalia.com/
> 
> This RFC proposes fixing kexec on ARM64 Hyper-V guests with:
> 
> Patch 1: A platform hook (arm64_pre_smp_shutdown_hook) in ARM64's
>   machine_shutdown(), analogous to x86's machine_ops.shutdown. This
>   runs after device_shutdown() and cpu_hotplug_enable(), allowing
>   platform code to inject pre-shutdown logic at the right point in
>   the kexec path.
> 
>   Design choices I'd like feedback on:
>   - Single function pointer vs full machine_ops struct: ARM64 uses
>     kernel-wide APIs (register_restart_handler, register_platform_power_off)
>     for restart/poweroff rather than x86's monolithic machine_ops, so
>     a targeted hook seemed more consistent with the ARM64 pattern.
>   - On ARM64, machine_shutdown() is only called from kernel_kexec(),
>     unlike x86/powerpc where it's also called from restart/halt/poweroff.
> 
> Patch 2: Uses this hook to call hv_kexec_handler() which performs:
>   - vmbus_initiate_unload(): sends CHANNELMSG_UNLOAD to host
>   - cpuhp_remove_state(): disables SynIC (SIMP, SIEFP, SINT) on
>     all CPUs, ensuring the kexec'd kernel starts with clean state
> 
> Tested on ARM64 Azure VMs (Ubuntu 22.04, multiple vCPU configs):
> - kexec reboot succeeds, VM comes back online with SSH
> - Normal reboot/poweroff unaffected
> - Multiple consecutive kexec cycles pass
> 
> Looking for feedback on:
> 1. Is the single function pointer hook acceptable for ARM64, or would
>    the ARM64 maintainers prefer a registration API or __weak function?
> 2. Should this be arm64-specific or generic kexec infrastructure?
> 
> Shradha Gupta (2):
>   arm64: Add pre-shutdown hook to machine_shutdown()
>   arm64/hyperv: Add kexec handler using machine_shutdown hook
> 
>  arch/arm64/hyperv/mshyperv.c         | 37 ++++++++++++++++++++++++++++
>  arch/arm64/include/asm/system_misc.h |  2 ++
>  arch/arm64/kernel/process.c          | 13 ++++++++++
>  3 files changed, 52 insertions(+)
> 
> -- 
> 2.43.0