From nobody Fri Jul 24 22:51:56 2026 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 539C5471267 for ; Wed, 22 Jul 2026 11:30:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784719858; cv=none; b=KKDSBxdSoWilr7sPvzwYzdERCou4ShcG/z1QkYcnj+vmJwPy1QTStnZuJyc5aIBOT4RPvtoUyP3YyV6Y8gLOFdXOR7nZ1lv2wJqweWAAQ1oyrKirQco1rwuRl7gSBnom8bCWscw/Mkztd2Oe0U/Fzw7Nz6tY5gsq4BCZU2UUaZc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784719858; c=relaxed/simple; bh=/u9OoXsgljB0s7KMVPfNj5/GLtrM63TLNc0Si6TjseM=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=WVzszG9tlYbjCeAB4TGT+FdhTdxlnlYD51UcTWA90eOiufzh0+6c0bzcQMjz6GnWYKClnnvt8H9qPGVmw30TgHgpj2gjG1yEg5BbGPhOaVuSgKXz9Jq1PpeI2LL7xJSDgDblPHnCWEYIk7naa6CcgfK9CTsXbUwZ403gvkRgGPo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=R0rX0gOs; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="R0rX0gOs" Received: from CPC-namja-026ON.redmond.corp.microsoft.com (unknown [4.213.232.21]) by linux.microsoft.com (Postfix) with ESMTPSA id 18F8520B7167; Wed, 22 Jul 2026 04:30:35 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 18F8520B7167 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1784719839; bh=mTxDia1v7efmMbdGqhuA+7Gs5LNYxD+Y+0UdQ3+MHck=; h=From:To:Cc:Subject:Date:From; b=R0rX0gOs4moWjU140aCobMbsvwwI6Ae+g4U+waXaldoH9UlxK9jwSK81kTxbPGGuP ENstCeBbMrxsa/6lbJohOJp320y2Jx8cyWDJUdqAH+PU5EzJ4qtxmvFnoENTmxORf9 5S9wWnyKtDEaSlA1A/gBva8pKmWRWoGAT5vnJ/KI= From: Naman Jain To: Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Marc Zyngier , Thomas Huth , Fuad Tabba , Jinjie Ruan , Thomas Gleixner , Pengjie Zhang , mrigendrachaubey , Saurabh Sengar Subject: [PATCH] arm64: smp: distinguish secondary CPUs that hang after reaching head.S Date: Wed, 22 Jul 2026 11:30:44 +0000 Message-ID: <20260722113044.1835365-1-namjain@linux.microsoft.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a secondary CPU fails to come online, __cpu_up() falls back to __early_cpu_boot_status, but boot status 0x0 is ambiguous: it cannot distinguish a CPU that never executed head.S (firmware/hypervisor never dispatched it, so it never ran a single instruction) from one that entered head.S, started executing, and then got stuck somewhere in kernel bring-up. Add a change to let us tell those two cases apart, which narrows down where to look when a CPU goes missing during boot. Write a sentinel (CPU_BOOT_ASM_ENTERED, 0x10) as the first thing in secondary_entry so the two cases can be told apart: 0x0 - CPU never ran head.S 0x10 - CPU entered head.S but did not come online __cpu_up() resets the word before releasing each secondary, so a stale value is not misattributed. Nothing clears the marker on the success path. Reason being, the CPU instead sets CPU_BOOT_SUCCESS in secondary_data.status, which __cpu_up() checks first, so __early_cpu_boot_status is only read on failure. A stall before CPU_BOOT_SUCCESS - including in secondary_start_kernel() - is therefore reported as 0x10. The sentinel sits in the low byte and sets no CPU_STUCK_REASON_* bits, so existing status decoding is unchanged. Limitations: best-effort only. The status variable is global with no CPU identity, and spin-table secondaries boot via secondary_holding_pen and never write it, so 0x0 is a strong hint rather than a guarantee. Signed-off-by: Naman Jain --- arch/arm64/include/asm/smp.h | 10 ++++++++++ arch/arm64/kernel/head.S | 24 +++++++++++++++++++++++- arch/arm64/kernel/smp.c | 21 +++++++++++++++++++++ 3 files changed, 54 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index 10ea4f5430690..ca330cd1b0615 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -20,6 +20,16 @@ /* Fatal system error detected by secondary CPU, crash the system */ #define CPU_PANIC_KERNEL (3) =20 +/* + * Best-effort marker written early in secondary_entry (PSCI/hotplug + * path) so __cpu_up() can tell whether a stuck CPU ever ran head.S. + * Not authoritative: spin-table secondaries use secondary_holding_pen + * and never write it, and the word has no CPU identity. Value must sit + * in the low byte, stay out of {0,1,2,3}, and set no CPU_STUCK_REASON_* + * bits. + */ +#define CPU_BOOT_ASM_ENTERED (0x10) + #define CPU_STUCK_REASON_52_BIT_VA (UL(1) << CPU_STUCK_REASON_SHIFT) #define CPU_STUCK_REASON_NO_GRAN (UL(2) << CPU_STUCK_REASON_SHIFT) =20 diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 87a822e5c4ca8..8ac3c49b4daf7 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -353,6 +353,22 @@ SYM_FUNC_END(secondary_holding_pen) * be used where CPUs are brought online dynamically by the kernel. */ SYM_FUNC_START(secondary_entry) + /* + * Record that this CPU reached head.S, before init_kernel_el() + * sets SCTLR_ELx.EE. A single-byte store to the reader's low + * byte (__cpu_up() zeroed the word) is endian-agnostic, so a + * big-endian kernel still reads 0x10. Flush it like + * update_early_cpu_boot_status. + */ + adr_l x2, __early_cpu_boot_status + mov w1, #CPU_BOOT_ASM_ENTERED +#ifdef CONFIG_CPU_BIG_ENDIAN + strb w1, [x2, #7] // low byte at top address +#else + strb w1, [x2] +#endif + dmb sy + dc ivac, x2 mov x0, xzr bl init_kernel_el // w0=3Dcpu_boot_mode b secondary_startup @@ -386,7 +402,6 @@ SYM_FUNC_START_LOCAL(__secondary_switched) mov x0, x20 bl finalise_el2 =20 - str_l xzr, __early_cpu_boot_status, x3 adr_l x5, vectors msr vbar_el1, x5 isb @@ -395,6 +410,13 @@ SYM_FUNC_START_LOCAL(__secondary_switched) ldr x2, [x0, #CPU_BOOT_TASK] cbz x2, __secondary_too_slow =20 + /* + * Don't clear the marker here; nothing on the secondary clears + * it. Once the CPU sets CPU_BOOT_SUCCESS in + * secondary_data.status, __cpu_up() stops reading + * __early_cpu_boot_status (and resets it before the next + * release), so a stall before that still reports 0x10. + */ init_cpu_task x2, x1, x3 =20 #ifdef CONFIG_ARM64_PTR_AUTH diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index cdcdd160e5b69..662205ae5a7a6 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -120,6 +120,17 @@ int __cpu_up(unsigned int cpu, struct task_struct *idl= e) secondary_data.task =3D idle; update_cpu_boot_status(CPU_MMU_OFF); =20 + /* + * Reset the marker before releasing this secondary: head.S + * never clears it, so without this a CPU that never reaches + * head.S would inherit a stale 0x10 from a previous CPU. Flush + * it so the MMU-off secondary sees the reset. + */ + WRITE_ONCE(__early_cpu_boot_status, 0); + dcache_clean_inval_poc((unsigned long)&__early_cpu_boot_status, + (unsigned long)&__early_cpu_boot_status + + sizeof(__early_cpu_boot_status)); + /* Now bring the CPU into our world */ ret =3D boot_secondary(cpu, idle); if (ret) { @@ -145,10 +156,20 @@ int __cpu_up(unsigned int cpu, struct task_struct *id= le) =20 switch (status & CPU_BOOT_STATUS_MASK) { default: + /* + * 0x0 usually means the CPU never ran head.S, but + * treat it as a hint: spin-table secondaries never + * write the marker and the word has no CPU identity. + */ pr_err("CPU%u: failed in unknown state : 0x%lx\n", cpu, status); cpus_stuck_in_kernel++; break; + case CPU_BOOT_ASM_ENTERED: + pr_err("CPU%u: entered head.S but not online : 0x%lx\n", + cpu, status); + cpus_stuck_in_kernel++; + break; case CPU_KILL_ME: if (!op_cpu_kill(cpu)) { pr_crit("CPU%u: died during early boot\n", cpu); base-commit: 290aaf24a551d5a0dce037e3fab30820f9113a10 --=20 2.43.0