From nobody Tue Sep 29 11:19:38 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3137338E5D7; Sat, 8 Aug 2026 20:48:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786222120; cv=none; b=lfOfbN5cRHJ6OnHNuvzRKREmH7MYK+g4xOnKdCc6sctBbEAmi5iTLlTm0liVUgu6TXvA0DGy3UiCCVmeAzxrHatA3+NNakQuHovDZVPvU5O4NPQ+lrHgthsZnsnyhnb6w4sZXydFdVI4M0+SYzBeGZhz6sP1TXC98/DrnwC0ph8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786222120; c=relaxed/simple; bh=p653BrvCTMoRBN9qmC5s5PmtYyyaQ8vRQMxKjDOxd8c=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=o/mZ4PaZuX9dS9Z1fovVPf2c8KbQzDeDSHOJglw9h2Rq1rwh0LDVajI9mZO+fq84u1I7DBrdmhXf/+YjgQU3ezbO+I6/hX6h+nQPEiPufyt53BAPhf7pLWp5fycjazO7JO1mB1SvjC5BlEqcFW+cJc9Yi+g1XMZH8JMpJlyuqh8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=deBYqxmO; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=QRxxiGMR; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="deBYqxmO"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="QRxxiGMR" Date: Sat, 08 Aug 2026 20:48:36 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1786222117; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cPAX0bEK48lOA0LM6Dgj4TRbDuPIH0EbGJbjGwLd2ms=; b=deBYqxmOKRCyTeiVBLPyQ/IobNruS5amJewBhzHR+rAuCMeznpd7UFQ0odFlyv6cDYxrOe cZ3W6Hgc2EyE2gxk9LwFIJoSNlesiDOXN5y9vxSlsBpWLdmfwNHp9wkub2v7cjC5nqQIbU UOpHoFY7zMlrPMM4EPaj+r+L/v/QkKMjb+RURyc34e1kXU5veOJUnaBuLpyUdLYBmij94e W22pBdybktHc8TMR7fxzC7/LNmJvvEA+m08XRHJBy/AQKmmR6jgVqtW9mGqdznkuwfspcK +iIUE/8bf93mB1Md9dHboQZwmDSQlm7Imdf9n5JJa+p+/2wk79bIHz4nRyLu4w== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1786222117; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cPAX0bEK48lOA0LM6Dgj4TRbDuPIH0EbGJbjGwLd2ms=; b=QRxxiGMR/AlweiMuqv4u2rJTD+Yj2oAblwtwvoh+1gUDwS+3qTJ7XNzd4wKlcIq2lFLLSZ 7LYXwo9PXtojE4Bg== From: "tip-bot2 for Joel Fernandes" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: locking/core] preempt: Track NMI nesting to separate per-CPU counter Cc: Boqun Feng , Joel Fernandes , Lyude Paul , "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260804161447.84806-2-boqun@kernel.org> References: <20260804161447.84806-2-boqun@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178622211605.442315.18132644859641387330.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the locking/core branch of tip: Commit-ID: b54aa0edf0594a6e1138e626bfd8a352ed582ae9 Gitweb: https://git.kernel.org/tip/b54aa0edf0594a6e1138e626bfd8a352e= d582ae9 Author: Joel Fernandes AuthorDate: Tue, 04 Aug 2026 09:14:23 -07:00 Committer: Peter Zijlstra CommitterDate: Sat, 08 Aug 2026 22:44:05 +02:00 preempt: Track NMI nesting to separate per-CPU counter Move NMI nesting tracking from the preempt_count bits to a separate per-CPU counter (nmi_nesting). This is to free up the NMI bits in the preempt_count, allowing those bits to be repurposed for other uses. Reduce NMI_BITS from 4 to 1, using it only to detect if we're in an NMI. The per-CPU counter currently caps nesting at 15. [boqun: Address Steven Rostedt's comment on the BUG_ON() condition] [boqun: Use preempt_count_set() in __nmi_exit() to avoid underflow] Suggested-by: Boqun Feng Signed-off-by: Joel Fernandes Signed-off-by: Lyude Paul Signed-off-by: Boqun Feng Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260121223933.1568682-3-lyude@redhat.com Link: https://patch.msgid.link/20260804161447.84806-2-boqun@kernel.org --- include/linux/hardirq.h | 17 +++++++++++++---- include/linux/preempt.h | 9 +++++++-- kernel/softirq.c | 2 ++- tools/testing/selftests/bpf/bpf_experimental.h | 2 +- 4 files changed, 23 insertions(+), 7 deletions(-) diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h index d57cab4..8d48955 100644 --- a/include/linux/hardirq.h +++ b/include/linux/hardirq.h @@ -10,6 +10,8 @@ #include #include =20 +DECLARE_PER_CPU(unsigned int, nmi_nesting); + extern void synchronize_irq(unsigned int irq); extern bool synchronize_hardirq(unsigned int irq); =20 @@ -102,14 +104,17 @@ void irq_exit_rcu(void); */ =20 /* - * nmi_enter() can nest up to 15 times; see NMI_BITS. + * nmi_enter() can nest - nesting is tracked in a per-CPU counter. */ #define __nmi_enter() \ do { \ lockdep_off(); \ arch_nmi_enter(); \ - BUG_ON(in_nmi() =3D=3D NMI_MASK); \ - __preempt_count_add(NMI_OFFSET + HARDIRQ_OFFSET); \ + /* Maximum NMI nesting is 15. */ \ + BUG_ON(__this_cpu_read(nmi_nesting) >=3D 15); \ + __this_cpu_inc(nmi_nesting); \ + __preempt_count_add(HARDIRQ_OFFSET); \ + preempt_count_set(preempt_count() | NMI_MASK); \ } while (0) =20 #define nmi_enter() \ @@ -124,8 +129,12 @@ void irq_exit_rcu(void); =20 #define __nmi_exit() \ do { \ + unsigned int nesting; \ BUG_ON(!in_nmi()); \ - __preempt_count_sub(NMI_OFFSET + HARDIRQ_OFFSET); \ + __preempt_count_sub(HARDIRQ_OFFSET); \ + nesting =3D __this_cpu_dec_return(nmi_nesting); \ + if (!nesting) \ + preempt_count_set(preempt_count() & ~NMI_MASK); \ arch_nmi_exit(); \ lockdep_on(); \ } while (0) diff --git a/include/linux/preempt.h b/include/linux/preempt.h index d964f96..586f966 100644 --- a/include/linux/preempt.h +++ b/include/linux/preempt.h @@ -17,6 +17,8 @@ * * - bits 0-7 are the preemption count (max preemption depth: 256) * - bits 8-15 are the softirq count (max # of softirqs: 256) + * - bits 16-19 are the hardirq count (max # of hardirqs: 16) + * - bit 20 is the NMI flag (no nesting count, tracked separately) * * The hardirq count could in theory be the same as the number of * interrupts in the system, but we run all interrupt handlers with @@ -24,16 +26,19 @@ * there are a few palaeontologic drivers which reenable interrupts in * the handler, so we need more than one bit here. * + * NMI nesting depth is tracked in a separate per-CPU variable + * (nmi_nesting) to save bits in preempt_count. + * * PREEMPT_MASK: 0x000000ff * SOFTIRQ_MASK: 0x0000ff00 * HARDIRQ_MASK: 0x000f0000 - * NMI_MASK: 0x00f00000 + * NMI_MASK: 0x00100000 * PREEMPT_NEED_RESCHED: 0x80000000 */ #define PREEMPT_BITS 8 #define SOFTIRQ_BITS 8 #define HARDIRQ_BITS 4 -#define NMI_BITS 4 +#define NMI_BITS 1 =20 #define PREEMPT_SHIFT 0 #define SOFTIRQ_SHIFT (PREEMPT_SHIFT + PREEMPT_BITS) diff --git a/kernel/softirq.c b/kernel/softirq.c index 4425d8d..10af5ed 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -88,6 +88,8 @@ EXPORT_PER_CPU_SYMBOL_GPL(hardirqs_enabled); EXPORT_PER_CPU_SYMBOL_GPL(hardirq_context); #endif =20 +DEFINE_PER_CPU(unsigned int, nmi_nesting); + /* * SOFTIRQ_OFFSET usage: * diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing= /selftests/bpf/bpf_experimental.h index 67ff788..e4e1200 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -367,7 +367,7 @@ extern int bpf_cgroup_read_xattr(struct cgroup *cgroup,= const char *name__str, #define PREEMPT_BITS 8 #define SOFTIRQ_BITS 8 #define HARDIRQ_BITS 4 -#define NMI_BITS 4 +#define NMI_BITS 1 =20 #define PREEMPT_SHIFT 0 #define SOFTIRQ_SHIFT (PREEMPT_SHIFT + PREEMPT_BITS)