From nobody Tue Dec 16 14:50:01 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3845A1F1518 for ; Sun, 14 Dec 2025 13:05:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765717502; cv=none; b=nF/Jc0X00Yg1uan0+V0nIwzGGeQ/NB3sFHhhXvFv3sA1oV1ztGr3x+B58FRujVBXF/BBpc62jZ4+X3805dsPWNXFIXIIY9UQX4CXnaYRZavZKmzMeo+5UxbnLDOdueM5XDbSekSQqwSt5s33pNBPTDm1iu3NFQyCb7/TQ86xMrg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765717502; c=relaxed/simple; bh=GrYtVAbpFu3RbWiQ5LxdN2wXKicn8vPVmlxf//QxmH0=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Tw4Be5zbMbLql5fHyr86UKAxOssi76W7/mZTRaA78dFoJZuHfrOvHz2TCWQ9NAvBV1cM6HVV/xxdh3q6UzQqFAeN2a73OAz767NdokhoxW60Mel4yymwpMuj+W7i2L22c7AZX+HBuBTJeBAkIIstTr6QabiEX3FywmZvm8e+omQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=aeLeNLdd; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="aeLeNLdd" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5BE9t20I029439; Sun, 14 Dec 2025 13:04:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h= content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=GJZAk8nDNKf44HMlB 0wt7NL+mPzbpnWEG07K4yl8V6U=; b=aeLeNLdd5rL0UWpsP345/BpIuq1O94zEE ZoqGbsxTSRf3gfwfsBOXhVEbHcy7k7j7q2i1WmXv05y6VVOzqUapEhsEGnY9upKO k2fAmVdazqHHlVk0ynX3Dcr9R3ttw8fyiUJ+Bn8Lt8E88F5CAiEW+LC9CWkOFqNd a5qp9W5t7/IlujqsaKrVA+OXibxUff2HHuh6fRFCLpdqoZQ+PfzORa2hYv2sGUPx TO8BLdSpFrXMB9JlRbCsLm9b4OEdLc+fGMa7afP38V784X8kiy3Eoe3hu+M64SiU 3m2oF4SQBIvVtZuhnhRXmXX4B8Kv8T+pIb5Jnes5WloegldbxL55Q== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4b0ytuwq4y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 14 Dec 2025 13:04:24 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 5BED4O6C001124; Sun, 14 Dec 2025 13:04:24 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4b0ytuwq4u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 14 Dec 2025 13:04:24 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 5BEAjrTi012816; Sun, 14 Dec 2025 13:04:22 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4b1juxt3s8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 14 Dec 2025 13:04:22 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 5BED4I3B40173874 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 14 Dec 2025 13:04:18 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B07DC20043; Sun, 14 Dec 2025 13:04:18 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 73C7420040; Sun, 14 Dec 2025 13:04:12 +0000 (GMT) Received: from li-1a3e774c-28e4-11b2-a85c-acc9f2883e29.ibm.com.com (unknown [9.124.210.21]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Sun, 14 Dec 2025 13:04:12 +0000 (GMT) From: Mukesh Kumar Chaurasiya To: maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, oleg@redhat.com, kees@kernel.org, luto@amacapital.net, wad@chromium.org, mchauras@linux.ibm.com, thuth@redhat.com, sshegde@linux.ibm.com, charlie@rivosinc.com, macro@orcam.me.uk, akpm@linux-foundation.org, ldv@strace.io, deller@gmx.de, ankur.a.arora@oracle.com, segher@kernel.crashing.org, tglx@linutronix.de, thomas.weissschuh@linutronix.de, peterz@infradead.org, menglong8.dong@gmail.com, bigeasy@linutronix.de, namcao@linutronix.de, kan.liang@linux.intel.com, mingo@kernel.org, atrajeev@linux.vnet.ibm.com, mark.barnett@arm.com, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 6/8] powerpc: Prepare for IRQ entry exit Date: Sun, 14 Dec 2025 18:32:42 +0530 Message-ID: <20251214130245.43664-7-mkchauras@linux.ibm.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20251214130245.43664-1-mkchauras@linux.ibm.com> References: <20251214130245.43664-1-mkchauras@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMjEzMDAyMyBTYWx0ZWRfX4hEa928hVlF9 zwkfXq7ShQSKiBfsSJsDalpmCqcnwOuk4b9fh4q9FOeFW3f94cSvAX/IHeS6BZ8mq+iWd9Q4YLp nNPf/9iMHIaS7nQtfPfZwGMRHPWBWF+di4E/sDiEZsGTPYQO3NHTwSnThlN2M3tUU4O5p8gyGZk h+Jy7BtUQ4hPQrC+fB12FszTelRggrWvCgIpqXDmO40cVO+h6a3CVaYztbmXjjmTXu/37j8CSq/ dYuLHGx+jgoqniM/nRNijD9IIP7GLODKVUKE7qhW1Pn+AQvuwflsU40J5VgQl6DcOwGi9c9IijX 7buiQoT6sW6EVeegjNfWt2bvcjFWlkBLM9XtXh04/OTAYSmpXeTY5kzByu4kNiYD3Rk+svvYg/h EWGCyY7y1HJcyocjWfcd6EZPLat1Cw== X-Proofpoint-ORIG-GUID: 5s6Vgn1uCoeR9lf_6KovSIdCAIpjM4eB X-Authority-Analysis: v=2.4 cv=QtRTHFyd c=1 sm=1 tr=0 ts=693eb5d8 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=wP3pNCr1ah4A:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=Ed76zSDasvk_NLogn_cA:9 X-Proofpoint-GUID: yUdzeqo1x4lFZjKHq5_V1Rt-7bieuoSa X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-12-14_03,2025-12-11_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 bulkscore=0 suspectscore=0 impostorscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2510240000 definitions=main-2512130023 Content-Type: text/plain; charset="utf-8" From: Mukesh Kumar Chaurasiya Move interrupt entry and exit helper routines from interrupt.h into the PowerPC-specific entry-common.h header as a preparatory step for enabling the generic entry/exit framework. This consolidation places all PowerPC interrupt entry/exit handling in a single common header, aligning with the generic entry infrastructure. The helpers provide architecture-specific handling for interrupt and NMI entry/exit sequences, including: - arch_interrupt_enter/exit_prepare() - arch_interrupt_async_enter/exit_prepare() - arch_interrupt_nmi_enter/exit_prepare() - Supporting helpers such as nap_adjust_return(), check_return_regs_valid(= ), debug register maintenance, and soft mask handling. The functions are copied verbatim from interrupt.h to avoid functional changes at this stage. Subsequent patches will integrate these routines into the generic entry/exit flow. No functional change intended. Signed-off-by: Mukesh Kumar Chaurasiya --- arch/powerpc/include/asm/entry-common.h | 422 ++++++++++++++++++++++++ 1 file changed, 422 insertions(+) diff --git a/arch/powerpc/include/asm/entry-common.h b/arch/powerpc/include= /asm/entry-common.h index e8ebd42a4e6d..e8bde4c67eaf 100644 --- a/arch/powerpc/include/asm/entry-common.h +++ b/arch/powerpc/include/asm/entry-common.h @@ -7,10 +7,432 @@ =20 #include #include +#include #include #include #include =20 +#ifdef CONFIG_PPC_IRQ_SOFT_MASK_DEBUG +/* + * WARN/BUG is handled with a program interrupt so minimise checks here to + * avoid recursion and maximise the chance of getting the first oops handl= ed. + */ +#define INT_SOFT_MASK_BUG_ON(regs, cond) \ +do { \ + if ((user_mode(regs) || (TRAP(regs) !=3D INTERRUPT_PROGRAM))) \ + BUG_ON(cond); \ +} while (0) +#else +#define INT_SOFT_MASK_BUG_ON(regs, cond) +#endif + +#ifdef CONFIG_PPC_BOOK3S_64 +extern char __end_soft_masked[]; +bool search_kernel_soft_mask_table(unsigned long addr); +unsigned long search_kernel_restart_table(unsigned long addr); + +DECLARE_STATIC_KEY_FALSE(interrupt_exit_not_reentrant); + +static inline bool is_implicit_soft_masked(struct pt_regs *regs) +{ + if (user_mode(regs)) + return false; + + if (regs->nip >=3D (unsigned long)__end_soft_masked) + return false; + + return search_kernel_soft_mask_table(regs->nip); +} + +static inline void srr_regs_clobbered(void) +{ + local_paca->srr_valid =3D 0; + local_paca->hsrr_valid =3D 0; +} +#else +static inline unsigned long search_kernel_restart_table(unsigned long addr) +{ + return 0; +} + +static inline bool is_implicit_soft_masked(struct pt_regs *regs) +{ + return false; +} + +static inline void srr_regs_clobbered(void) +{ +} +#endif + +static inline void nap_adjust_return(struct pt_regs *regs) +{ +#ifdef CONFIG_PPC_970_NAP + if (unlikely(test_thread_local_flags(_TLF_NAPPING))) { + /* Can avoid a test-and-clear because NMIs do not call this */ + clear_thread_local_flags(_TLF_NAPPING); + regs_set_return_ip(regs, (unsigned long)power4_idle_nap_return); + } +#endif +} + +static inline void booke_load_dbcr0(void) +{ +#ifdef CONFIG_PPC_ADV_DEBUG_REGS + unsigned long dbcr0 =3D current->thread.debug.dbcr0; + + if (likely(!(dbcr0 & DBCR0_IDM))) + return; + + /* + * Check to see if the dbcr0 register is set up to debug. + * Use the internal debug mode bit to do this. + */ + mtmsr(mfmsr() & ~MSR_DE); + if (IS_ENABLED(CONFIG_PPC32)) { + isync(); + global_dbcr0[smp_processor_id()] =3D mfspr(SPRN_DBCR0); + } + mtspr(SPRN_DBCR0, dbcr0); + mtspr(SPRN_DBSR, -1); +#endif +} + +static inline void booke_restore_dbcr0(void) +{ +#ifdef CONFIG_PPC_ADV_DEBUG_REGS + unsigned long dbcr0 =3D current->thread.debug.dbcr0; + + if (IS_ENABLED(CONFIG_PPC32) && unlikely(dbcr0 & DBCR0_IDM)) { + mtspr(SPRN_DBSR, -1); + mtspr(SPRN_DBCR0, global_dbcr0[smp_processor_id()]); + } +#endif +} + +static inline void check_return_regs_valid(struct pt_regs *regs) +{ +#ifdef CONFIG_PPC_BOOK3S_64 + unsigned long trap, srr0, srr1; + static bool warned; + u8 *validp; + char *h; + + if (trap_is_scv(regs)) + return; + + trap =3D TRAP(regs); + // EE in HV mode sets HSRRs like 0xea0 + if (cpu_has_feature(CPU_FTR_HVMODE) && trap =3D=3D INTERRUPT_EXTERNAL) + trap =3D 0xea0; + + switch (trap) { + case 0x980: + case INTERRUPT_H_DATA_STORAGE: + case 0xe20: + case 0xe40: + case INTERRUPT_HMI: + case 0xe80: + case 0xea0: + case INTERRUPT_H_FAC_UNAVAIL: + case 0x1200: + case 0x1500: + case 0x1600: + case 0x1800: + validp =3D &local_paca->hsrr_valid; + if (!READ_ONCE(*validp)) + return; + + srr0 =3D mfspr(SPRN_HSRR0); + srr1 =3D mfspr(SPRN_HSRR1); + h =3D "H"; + + break; + default: + validp =3D &local_paca->srr_valid; + if (!READ_ONCE(*validp)) + return; + + srr0 =3D mfspr(SPRN_SRR0); + srr1 =3D mfspr(SPRN_SRR1); + h =3D ""; + break; + } + + if (srr0 =3D=3D regs->nip && srr1 =3D=3D regs->msr) + return; + + /* + * A NMI / soft-NMI interrupt may have come in after we found + * srr_valid and before the SRRs are loaded. The interrupt then + * comes in and clobbers SRRs and clears srr_valid. Then we load + * the SRRs here and test them above and find they don't match. + * + * Test validity again after that, to catch such false positives. + * + * This test in general will have some window for false negatives + * and may not catch and fix all such cases if an NMI comes in + * later and clobbers SRRs without clearing srr_valid, but hopefully + * such things will get caught most of the time, statistically + * enough to be able to get a warning out. + */ + if (!READ_ONCE(*validp)) + return; + + if (!data_race(warned)) { + data_race(warned =3D true); + pr_warn("%sSRR0 was: %lx should be: %lx\n", h, srr0, regs->nip); + pr_warn("%sSRR1 was: %lx should be: %lx\n", h, srr1, regs->msr); + show_regs(regs); + } + + WRITE_ONCE(*validp, 0); /* fixup */ +#endif +} + +static inline void arch_interrupt_enter_prepare(struct pt_regs *regs) +{ +#ifdef CONFIG_PPC64 + irq_soft_mask_set(IRQS_ALL_DISABLED); + + /* + * If the interrupt was taken with HARD_DIS clear, then enable MSR[EE]. + * Asynchronous interrupts get here with HARD_DIS set (see below), so + * this enables MSR[EE] for synchronous interrupts. IRQs remain + * soft-masked. The interrupt handler may later call + * interrupt_cond_local_irq_enable() to achieve a regular process + * context. + */ + if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) { + INT_SOFT_MASK_BUG_ON(regs, !(regs->msr & MSR_EE)); + __hard_irq_enable(); + } else { + __hard_RI_enable(); + } + /* Enable MSR[RI] early, to support kernel SLB and hash faults */ +#endif + + if (!regs_irqs_disabled(regs)) + trace_hardirqs_off(); + + if (user_mode(regs)) { + kuap_lock(); + CT_WARN_ON(ct_state() !=3D CT_STATE_USER); + user_exit_irqoff(); + + account_cpu_user_entry(); + account_stolen_time(); + } else { + kuap_save_and_lock(regs); + /* + * CT_WARN_ON comes here via program_check_exception, + * so avoid recursion. + */ + if (TRAP(regs) !=3D INTERRUPT_PROGRAM) + CT_WARN_ON(ct_state() !=3D CT_STATE_KERNEL && + ct_state() !=3D CT_STATE_IDLE); + INT_SOFT_MASK_BUG_ON(regs, is_implicit_soft_masked(regs)); + INT_SOFT_MASK_BUG_ON(regs, regs_irqs_disabled(regs) && + search_kernel_restart_table(regs->nip)); + } + INT_SOFT_MASK_BUG_ON(regs, !regs_irqs_disabled(regs) && + !(regs->msr & MSR_EE)); + + booke_restore_dbcr0(); +} + +/* + * Care should be taken to note that arch_interrupt_exit_prepare and + * arch_interrupt_async_exit_prepare do not necessarily return immediately= to + * regs context (e.g., if regs is usermode, we don't necessarily return to + * user mode). Other interrupts might be taken between here and return, + * context switch / preemption may occur in the exit path after this, or a + * signal may be delivered, etc. + * + * The real interrupt exit code is platform specific, e.g., + * interrupt_exit_user_prepare / interrupt_exit_kernel_prepare for 64s. + * + * However arch_interrupt_nmi_exit_prepare does return directly to regs, b= ecause + * NMIs do not do "exit work" or replay soft-masked interrupts. + */ +static inline void arch_interrupt_exit_prepare(struct pt_regs *regs) +{ +} + +static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs) +{ +#ifdef CONFIG_PPC64 + /* Ensure arch_interrupt_enter_prepare does not enable MSR[EE] */ + local_paca->irq_happened |=3D PACA_IRQ_HARD_DIS; +#endif + arch_interrupt_enter_prepare(regs); +#ifdef CONFIG_PPC_BOOK3S_64 + /* + * RI=3D1 is set by arch_interrupt_enter_prepare, so this thread flags ac= cess + * has to come afterward (it can cause SLB faults). + */ + if (cpu_has_feature(CPU_FTR_CTRL) && + !test_thread_local_flags(_TLF_RUNLATCH)) + __ppc64_runlatch_on(); +#endif + irq_enter(); +} + +static inline void arch_interrupt_async_exit_prepare(struct pt_regs *regs) +{ + /* + * Adjust at exit so the main handler sees the true NIA. This must + * come before irq_exit() because irq_exit can enable interrupts, and + * if another interrupt is taken before nap_adjust_return has run + * here, then that interrupt would return directly to idle nap return. + */ + nap_adjust_return(regs); + + irq_exit(); + arch_interrupt_exit_prepare(regs); +} + +struct interrupt_nmi_state { +#ifdef CONFIG_PPC64 + u8 irq_soft_mask; + u8 irq_happened; + u8 ftrace_enabled; + u64 softe; +#endif +}; + +static inline bool nmi_disables_ftrace(struct pt_regs *regs) +{ + /* Allow DEC and PMI to be traced when they are soft-NMI */ + if (IS_ENABLED(CONFIG_PPC_BOOK3S_64)) { + if (TRAP(regs) =3D=3D INTERRUPT_DECREMENTER) + return false; + if (TRAP(regs) =3D=3D INTERRUPT_PERFMON) + return false; + } + if (IS_ENABLED(CONFIG_PPC_BOOK3E_64)) { + if (TRAP(regs) =3D=3D INTERRUPT_PERFMON) + return false; + } + + return true; +} + +static inline void arch_interrupt_nmi_enter_prepare(struct pt_regs *regs, + struct interrupt_nmi_state *state) +{ +#ifdef CONFIG_PPC64 + state->irq_soft_mask =3D local_paca->irq_soft_mask; + state->irq_happened =3D local_paca->irq_happened; + state->softe =3D regs->softe; + + /* + * Set IRQS_ALL_DISABLED unconditionally so irqs_disabled() does + * the right thing, and set IRQ_HARD_DIS. We do not want to reconcile + * because that goes through irq tracing which we don't want in NMI. + */ + local_paca->irq_soft_mask =3D IRQS_ALL_DISABLED; + local_paca->irq_happened |=3D PACA_IRQ_HARD_DIS; + + if (!(regs->msr & MSR_EE) || is_implicit_soft_masked(regs)) { + /* + * Adjust regs->softe to be soft-masked if it had not been + * reconcied (e.g., interrupt entry with MSR[EE]=3D0 but softe + * not yet set disabled), or if it was in an implicit soft + * masked state. This makes regs_irqs_disabled(regs) + * behave as expected. + */ + regs->softe =3D IRQS_ALL_DISABLED; + } + + __hard_RI_enable(); + + /* Don't do any per-CPU operations until interrupt state is fixed */ + + if (nmi_disables_ftrace(regs)) { + state->ftrace_enabled =3D this_cpu_get_ftrace_enabled(); + this_cpu_set_ftrace_enabled(0); + } +#endif + + /* If data relocations are enabled, it's safe to use nmi_enter() */ + if (mfmsr() & MSR_DR) { + nmi_enter(); + return; + } + + /* + * But do not use nmi_enter() for pseries hash guest taking a real-mode + * NMI because not everything it touches is within the RMA limit. + */ + if (IS_ENABLED(CONFIG_PPC_BOOK3S_64) && + firmware_has_feature(FW_FEATURE_LPAR) && + !radix_enabled()) + return; + + /* + * Likewise, don't use it if we have some form of instrumentation (like + * KASAN shadow) that is not safe to access in real mode (even on radix) + */ + if (IS_ENABLED(CONFIG_KASAN)) + return; + + /* + * Likewise, do not use it in real mode if percpu first chunk is not + * embedded. With CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK enabled there + * are chances where percpu allocation can come from vmalloc area. + */ + if (percpu_first_chunk_is_paged) + return; + + /* Otherwise, it should be safe to call it */ + nmi_enter(); +} + +static inline void arch_interrupt_nmi_exit_prepare(struct pt_regs *regs, + struct interrupt_nmi_state *state) +{ + if (mfmsr() & MSR_DR) { + // nmi_exit if relocations are on + nmi_exit(); + } else if (IS_ENABLED(CONFIG_PPC_BOOK3S_64) && + firmware_has_feature(FW_FEATURE_LPAR) && + !radix_enabled()) { + // no nmi_exit for a pseries hash guest taking a real mode exception + } else if (IS_ENABLED(CONFIG_KASAN)) { + // no nmi_exit for KASAN in real mode + } else if (percpu_first_chunk_is_paged) { + // no nmi_exit if percpu first chunk is not embedded + } else { + nmi_exit(); + } + + /* + * nmi does not call nap_adjust_return because nmi should not create + * new work to do (must use irq_work for that). + */ + +#ifdef CONFIG_PPC64 +#ifdef CONFIG_PPC_BOOK3S + if (regs_irqs_disabled(regs)) { + unsigned long rst =3D search_kernel_restart_table(regs->nip); + + if (rst) + regs_set_return_ip(regs, rst); + } +#endif + + if (nmi_disables_ftrace(regs)) + this_cpu_set_ftrace_enabled(state->ftrace_enabled); + + /* Check we didn't change the pending interrupt mask. */ + WARN_ON_ONCE((state->irq_happened | PACA_IRQ_HARD_DIS) !=3D local_paca->i= rq_happened); + regs->softe =3D state->softe; + local_paca->irq_happened =3D state->irq_happened; + local_paca->irq_soft_mask =3D state->irq_soft_mask; +#endif +} + static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs) { if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG)) --=20 2.52.0