From nobody Mon Jun 15 05:19:15 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3186035B631; Wed, 8 Apr 2026 10:10:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775643046; cv=none; b=hyhuxdRAlh8TgX4DaCmRAT4JupbgQLpgMp9H6dJYlUFqVjYzd6L0b0Xt+/Npe55whxP/gf0JBTa7j+V8TvpqIfGhvCbEQ/qxNHQg1LRthHbI1xbR7D3GbtdHoYMv30fcArJBH/Ro9S8UtxVnVGOwPCu+lu0WsnR+XQ0kETQMqfw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775643046; c=relaxed/simple; bh=9F1JEQxaijZhTVZjhY+DmZ+KnS5arcCIzCWFR//03Jg=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=EmqnX/B58jpsDq2TK0M696NsH+nZjcF3gzwrHGRh9jqO1bT12+KAbSmgG9UgvZzdVHqNJzXCSzpboWgyWjWEoXo8hVNR6khOpL5qhj2+uiEERUjKEyDnhNo0AaiyY1tM135iFGDZqbJJl7DO59PtMKVnEJu43YxgN7Q9R1hKd6s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=dr8PjnCm; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=odDG918V; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="dr8PjnCm"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="odDG918V" Date: Wed, 08 Apr 2026 10:10:35 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1775643037; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QVTFf/ltJrt9OcWxgvu2gyrEetDToaEqfvFU7ohmACM=; b=dr8PjnCm+tmvYaVi0gaqIF8XlZonFoEOIx6hDdmilYlpQ46FY1V0QzlO/0c/D7rSYBpyl5 DgdcEIl1juUeDEQ8XCJ7SUIb5+BrrcmsL/q3e1lVortR/fmewOsfF0Mn9vbGKrVIGPX/0I /NdAf0ruvnX6UWka7Xixf6lTR3m4U4ELPhK3B9y3AB3a1+HaiySuBVMU0bSqktvt9b+htI eA7Cqj2QUnbVwoswMyBeDhNTTu+7Vwj95Z2hJuLt72LMZH4YK6bZuZmdvxlIt+xYjWHcKj U/re0TBFx9FrWGh64qxkIiH4BSEEKDDNt/b/+369PRQVm/Ofuv6kBrMY/H9VRQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1775643037; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QVTFf/ltJrt9OcWxgvu2gyrEetDToaEqfvFU7ohmACM=; b=odDG918VfDMPXV3Qlcy+uysHkajfkkyrvL/HY5/cAaFzaKia4R0CpFfRmntiXd+cL34g60 +ZARcQDmYfgy+6AA== From: "tip-bot2 for Mark Rutland" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/hrtick] entry: Split preemption from irqentry_exit_to_kernel_mode() Cc: Mark Rutland , Thomas Gleixner , Jinjie Ruan , "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260407131650.3813777-6-mark.rutland@arm.com> References: <20260407131650.3813777-6-mark.rutland@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <177564303548.801717.13573050814101927362.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the sched/hrtick branch of tip: Commit-ID: 041aa7a85390c99b1de86dc28eddcff0890d8186 Gitweb: https://git.kernel.org/tip/041aa7a85390c99b1de86dc28eddcff08= 90d8186 Author: Mark Rutland AuthorDate: Tue, 07 Apr 2026 14:16:45 +01:00 Committer: Thomas Gleixner CommitterDate: Wed, 08 Apr 2026 11:43:32 +02:00 entry: Split preemption from irqentry_exit_to_kernel_mode() Some architecture-specific work needs to be performed between the state management for exception entry/exit and the "real" work to handle the exception. For example, arm64 needs to manipulate a number of exception masking bits, with different exceptions requiring different masking. Generally this can all be hidden in the architecture code, but for arm64 the current structure of irqentry_exit_to_kernel_mode() makes this particularly difficult to handle in a way that is correct, maintainable, and efficient. The gory details are described in the thread surrounding: https://lore.kernel.org/lkml/acPAzdtjK5w-rNqC@J2N7QTR9R3/ The summary is: * Currently, irqentry_exit_to_kernel_mode() handles both involuntary preemption AND state management necessary for exception return. * When scheduling (including involuntary preemption), arm64 needs to have all arm64-specific exceptions unmasked, though regular interrupts must be masked. * Prior to the state management for exception return, arm64 needs to mask a number of arm64-specific exceptions, and perform some work with these exceptions masked (with RCU watching, etc). While in theory it is possible to handle this with a new arch_*() hook called somewhere under irqentry_exit_to_kernel_mode(), this is fragile and complicated, and doesn't match the flow used for exception return to user mode, which has a separate 'prepare' step (where preemption can occur) prior to the state management. To solve this, refactor irqentry_exit_to_kernel_mode() to match the style of {irqentry,syscall}_exit_to_user_mode(), moving preemption logic into a new irqentry_exit_to_kernel_mode_preempt() function, and moving state management in a new irqentry_exit_to_kernel_mode_after_preempt() function. The existing irqentry_exit_to_kernel_mode() is left as a caller of both of these, avoiding the need to modify existing callers. There should be no functional change as a result of this change. [ tglx: Updated kernel doc ] Signed-off-by: Mark Rutland Signed-off-by: Thomas Gleixner Reviewed-by: Jinjie Ruan Acked-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260407131650.3813777-6-mark.rutland@arm.com --- include/linux/irq-entry-common.h | 73 +++++++++++++++++++++++++------ 1 file changed, 59 insertions(+), 14 deletions(-) diff --git a/include/linux/irq-entry-common.h b/include/linux/irq-entry-com= mon.h index 66bc168..3845202 100644 --- a/include/linux/irq-entry-common.h +++ b/include/linux/irq-entry-common.h @@ -438,24 +438,46 @@ static __always_inline irqentry_state_t irqentry_ente= r_from_kernel_mode(struct p } =20 /** - * irqentry_exit_to_kernel_mode - Run preempt checks and establish state a= fter - * invoking the interrupt handler + * irqentry_exit_to_kernel_mode_preempt - Run preempt checks on return to = kernel mode * @regs: Pointer to current's pt_regs * @state: Return value from matching call to irqentry_enter_from_kernel_m= ode() * - * This is the counterpart of irqentry_enter_from_kernel_mode() and runs t= he - * necessary preemption check if possible and required. It returns to the = caller - * with interrupts disabled and the correct state vs. tracing, lockdep and= RCU - * required to return to the interrupted context. + * This is to be invoked before irqentry_exit_to_kernel_mode_after_preempt= () to + * allow kernel preemption on return from interrupt. + * + * Must be invoked with interrupts disabled and CPU state which allows ker= nel + * preemption. * - * It is the last action before returning to the low level ASM code which = just - * needs to return. + * After returning from this function, the caller can modify CPU state bef= ore + * invoking irqentry_exit_to_kernel_mode_after_preempt(), which is require= d to + * re-establish the tracing, lockdep and RCU state for returning to the + * interrupted context. */ -static __always_inline void irqentry_exit_to_kernel_mode(struct pt_regs *r= egs, - irqentry_state_t state) +static inline void irqentry_exit_to_kernel_mode_preempt(struct pt_regs *re= gs, + irqentry_state_t state) { - lockdep_assert_irqs_disabled(); + if (regs_irqs_disabled(regs) || state.exit_rcu) + return; + + if (IS_ENABLED(CONFIG_PREEMPTION)) + irqentry_exit_cond_resched(); +} =20 +/** + * irqentry_exit_to_kernel_mode_after_preempt - Establish trace, lockdep a= nd RCU state + * @regs: Pointer to current's pt_regs + * @state: Return value from matching call to irqentry_enter_from_kernel_m= ode() + * + * This is to be invoked after irqentry_exit_to_kernel_mode_preempt() and = before + * actually returning to the interrupted context. + * + * There are no requirements for the CPU state other than being able to co= mplete + * the tracing, lockdep and RCU state transitions. After this function ret= urns + * the caller must return directly to the interrupted context. + */ +static __always_inline void +irqentry_exit_to_kernel_mode_after_preempt(struct pt_regs *regs, irqentry_= state_t state) +{ if (!regs_irqs_disabled(regs)) { /* * If RCU was not watching on entry this needs to be done @@ -474,9 +496,6 @@ static __always_inline void irqentry_exit_to_kernel_mod= e(struct pt_regs *regs, } =20 instrumentation_begin(); - if (IS_ENABLED(CONFIG_PREEMPTION)) - irqentry_exit_cond_resched(); - /* Covers both tracing and lockdep */ trace_hardirqs_on(); instrumentation_end(); @@ -491,6 +510,32 @@ static __always_inline void irqentry_exit_to_kernel_mo= de(struct pt_regs *regs, } =20 /** + * irqentry_exit_to_kernel_mode - Run preempt checks and establish state a= fter + * invoking the interrupt handler + * @regs: Pointer to current's pt_regs + * @state: Return value from matching call to irqentry_enter_from_kernel_m= ode() + * + * This is the counterpart of irqentry_enter_from_kernel_mode() and combin= es + * the calls to irqentry_exit_to_kernel_mode_preempt() and + * irqentry_exit_to_kernel_mode_after_preempt(). + * + * The requirement for the CPU state is that it can schedule. After the fu= nction + * returns the tracing, lockdep and RCU state transitions are completed an= d the + * caller must return directly to the interrupted context. + */ +static __always_inline void irqentry_exit_to_kernel_mode(struct pt_regs *r= egs, + irqentry_state_t state) +{ + lockdep_assert_irqs_disabled(); + + instrumentation_begin(); + irqentry_exit_to_kernel_mode_preempt(regs, state); + instrumentation_end(); + + irqentry_exit_to_kernel_mode_after_preempt(regs, state); +} + +/** * irqentry_enter - Handle state tracking on ordinary interrupt entries * @regs: Pointer to pt_regs of interrupted context *