From nobody Fri Oct 24 20:28:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36BF0C00140 for ; Mon, 15 Aug 2022 23:02:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243556AbiHOXCl (ORCPT ); Mon, 15 Aug 2022 19:02:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352794AbiHOXBI (ORCPT ); Mon, 15 Aug 2022 19:01:08 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 463415D120; Mon, 15 Aug 2022 12:57:47 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 06ACB6113B; Mon, 15 Aug 2022 19:57:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 17BEEC433C1; Mon, 15 Aug 2022 19:57:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1660593465; bh=5T4neKr3Jd7AuPZGVA14c6D8Q45msYSSscOIE8m2h88=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y9Fd99E4ToYyTfqcuSOan+9UggqRPLwoYUctFPKj9AYfqIertvlhwqB3irmnUu+V6 9c84OhqhGkikivImG5FYn4eBJGBdQ22IxGms7BVB2h+oqGhSS9xSG3guKylnLd4LSN hKWlEcjK9TeMcSYbT8xan9vN0umSJw4iRWGLYSS4= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Valentin Schneider , Tianchen Ding , "Peter Zijlstra (Intel)" , Sasha Levin Subject: [PATCH 5.18 0950/1095] sched: Fix the check of nr_running at queue wakelist Date: Mon, 15 Aug 2022 20:05:49 +0200 Message-Id: <20220815180508.412896351@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220815180429.240518113@linuxfoundation.org> References: <20220815180429.240518113@linuxfoundation.org> User-Agent: quilt/0.67 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianchen Ding [ Upstream commit 28156108fecb1f808b21d216e8ea8f0d205a530c ] The commit 2ebb17717550 ("sched/core: Offload wakee task activation if it the wakee is descheduling") checked rq->nr_running <=3D 1 to avoid task stacking when WF_ON_CPU. Per the ordering of writes to p->on_rq and p->on_cpu, observing p->on_cpu (WF_ON_CPU) in ttwu_queue_cond() implies !p->on_rq, IOW p has gone through the deactivate_task() in __schedule(), thus p has been accounted out of rq->nr_running. As such, the task being the only runnable task on the rq implies reading rq->nr_running =3D=3D 0 at that point. The benchmark result is in [1]. [1] https://lore.kernel.org/all/e34de686-4e85-bde1-9f3c-9bbc86b38627@linux.= alibaba.com/ Suggested-by: Valentin Schneider Signed-off-by: Tianchen Ding Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Valentin Schneider Link: https://lore.kernel.org/r/20220608233412.327341-2-dtcccc@linux.alibab= a.com Signed-off-by: Sasha Levin --- kernel/sched/core.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index b0f3ded147cb..4e99f0cf727a 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -3832,8 +3832,12 @@ static inline bool ttwu_queue_cond(int cpu, int wake= _flags) * CPU then use the wakelist to offload the task activation to * the soon-to-be-idle CPU as the current CPU is likely busy. * nr_running is checked to avoid unnecessary task stacking. + * + * Note that we can only get here with (wakee) p->on_rq=3D0, + * p->on_cpu can be whatever, we've done the dequeue, so + * the wakee has been accounted out of ->nr_running. */ - if ((wake_flags & WF_ON_CPU) && cpu_rq(cpu)->nr_running <=3D 1) + if ((wake_flags & WF_ON_CPU) && !cpu_rq(cpu)->nr_running) return true; =20 return false; --=20 2.35.1