From nobody Tue Dec 30 14:41:01 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 759E9C4167B for ; Tue, 14 Nov 2023 19:38:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231725AbjKNTiv (ORCPT ); Tue, 14 Nov 2023 14:38:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53274 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229619AbjKNTiu (ORCPT ); Tue, 14 Nov 2023 14:38:50 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EA6B10D for ; Tue, 14 Nov 2023 11:38:47 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3054C433C9; Tue, 14 Nov 2023 19:38:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1699990726; bh=AI6BXRLyDCRzfgTcAPcBGNu2Xr1HniWMknoBjvYmHug=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f9T3QtOfwyMU3SM/xpQ/Glx/rMUj5Plw6/3/0jabnR7kXRucPmXtaqD7J5jFCUn41 oxM9PHRB/h3uCCnw33AuBgmcE8xk7xj+4tIzJTIykuS6tssltjN2WiQg2LQiCCoNZ7 wZTIq6PY48L30SmyEo9A0JX/mHxpibSL3/Qi7WV6uEP3H3kBeTOfU9df70r8xKDNTs pLQ1rULivGrP1b1aYrQLQRUIpVXsjDrDFauE/ajgFac79dbCxR0om9zv5cZ1tYf/DV 94l1q7oQDep6BYh1w1Osnyx6zgxrByT9+SOLzTzFbVWLwrCLdIcgVzcFXochoSzalG eGDZu7ibPoFUQ== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , "Rafael J . Wysocki" , Daniel Lezcano , Thomas Gleixner , Anna-Maria Behnsen , Ingo Molnar Subject: [PATCH 1/2] sched/cpuidle: Comment about timers requirements VS idle handler Date: Tue, 14 Nov 2023 14:38:39 -0500 Message-ID: <20231114193840.4041-2-frederic@kernel.org> X-Mailer: git-send-email 2.42.1 In-Reply-To: <20231114193840.4041-1-frederic@kernel.org> References: <20231114193840.4041-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add missing explanation concerning IRQs re-enablement constraints in the cpuidle path against timers. Acked-by: Rafael J. Wysocki Signed-off-by: Frederic Weisbecker --- kernel/sched/idle.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 565f8374ddbb..31231925f1ec 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -258,6 +258,36 @@ static void do_idle(void) while (!need_resched()) { rmb(); =20 + /* + * Interrupts shouldn't be re-enabled from that point on until + * the CPU sleeping instruction is reached. Otherwise an interrupt + * may fire and queue a timer that would be ignored until the CPU + * wakes from the sleeping instruction. And testing need_resched() + * doesn't tell about pending needed timer reprogram. + * + * Several cases to consider: + * + * - SLEEP-UNTIL-PENDING-INTERRUPT based instructions such as + * "wfi" or "mwait" are fine because they can be entered with + * interrupt disabled. + * + * - sti;mwait() couple is fine because the interrupts are + * re-enabled only upon the execution of mwait, leaving no gap + * in-between. + * + * - ROLLBACK based idle handlers with the sleeping instruction + * called with interrupts enabled are NOT fine. In this scheme + * when the interrupt detects it has interrupted an idle handler, + * it rolls back to its beginning which performs the + * need_resched() check before re-executing the sleeping + * instruction. This can leak a pending needed timer reprogram. + * If such a scheme is really mandatory due to the lack of an + * appropriate CPU sleeping instruction, then a FAST-FORWARD + * must instead be applied: when the interrupt detects it has + * interrupted an idle handler, it must resume to the end of + * this idle handler so that the generic idle loop is iterated + * again to reprogram the tick. + */ local_irq_disable(); =20 if (cpu_is_offline(cpu)) { --=20 2.42.1 From nobody Tue Dec 30 14:41:01 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFB41C4332F for ; Tue, 14 Nov 2023 19:39:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231874AbjKNTix (ORCPT ); Tue, 14 Nov 2023 14:38:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231694AbjKNTiv (ORCPT ); Tue, 14 Nov 2023 14:38:51 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53BF210D for ; Tue, 14 Nov 2023 11:38:48 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 219ACC433C7; Tue, 14 Nov 2023 19:38:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1699990727; bh=TiiyIUwqEqNWx33Q4JpkMzZZTKBz7XGXy64YGa5f2BI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BJCok1LbgCXsmVwz+wvwuwA+GtIUohmHHu9CjSL0aGZe6xFbQwsvWhCAEvG5pOHZm +g6RREpNN+IcN3bU0RHg1S9tWVOtmP9QQ5SVqJp0PDEmjhgrh+1haH4b1LrXE9j05r PWwDMvHiOdJ1BSJhpqg/ixBHCGfQ606CufxQ7BD95I/Kk35lkIwP03b4qb0/0KAWgy 5luM9/GwumxshVaRAwlLHDglSTj3JtEKby3SnUHdYAiy2LSZS/0gtMellSmYlCUWvk cLYSbvN6TDMB8b+uy0TFtvInwWUCkxH9Th3iV179IeDBc27MztCMriYsHTN0uC0SG/ 0hIl//2IHIw0Q== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , "Rafael J . Wysocki" , Daniel Lezcano , Thomas Gleixner , Anna-Maria Behnsen , Ingo Molnar Subject: [PATCH 2/2] sched/timers: Explain why idle task schedules out on remote timer enqueue Date: Tue, 14 Nov 2023 14:38:40 -0500 Message-ID: <20231114193840.4041-3-frederic@kernel.org> X-Mailer: git-send-email 2.42.1 In-Reply-To: <20231114193840.4041-1-frederic@kernel.org> References: <20231114193840.4041-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Trying to avoid that didn't bring much value after testing, add comment about this. Acked-by: Rafael J. Wysocki Signed-off-by: Frederic Weisbecker --- kernel/sched/core.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index a708d225c28e..50abc7eddb82 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1131,6 +1131,28 @@ static void wake_up_idle_cpu(int cpu) if (cpu =3D=3D smp_processor_id()) return; =20 + /* + * Set TIF_NEED_RESCHED and send an IPI if in the non-polling + * part of the idle loop. This forces an exit from the idle loop + * and a round trip to schedule(). Now this could be optimized + * because a simple new idle loop iteration is enough to + * re-evaluate the next tick. Provided some re-ordering of tick + * nohz functions that would need to follow TIF_NR_POLLING + * clearing: + * + * - On most archs, a simple fetch_or on ti::flags with a + * "0" value would be enough to know if an IPI needs to be sent. + * + * - x86 needs to perform a last need_resched() check between + * monitor and mwait which doesn't take timers into account. + * There a dedicated TIF_TIMER flag would be required to + * fetch_or here and be checked along with TIF_NEED_RESCHED + * before mwait(). + * + * However, remote timer enqueue is not such a frequent event + * and testing of the above solutions didn't appear to report + * much benefits. + */ if (set_nr_and_not_polling(rq->idle)) smp_send_reschedule(cpu); else --=20 2.42.1