From nobody Thu Sep 11 22:44:34 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80249C636D4 for ; Wed, 15 Feb 2023 08:34:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233945AbjBOIeV (ORCPT ); Wed, 15 Feb 2023 03:34:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232166AbjBOIeT (ORCPT ); Wed, 15 Feb 2023 03:34:19 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 179D01714 for ; Wed, 15 Feb 2023 00:34:17 -0800 (PST) Received: from canpemm100009.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4PGrs931PWzRs49; Wed, 15 Feb 2023 16:31:41 +0800 (CST) Received: from canpemm500010.china.huawei.com (7.192.105.118) by canpemm100009.china.huawei.com (7.192.105.213) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.17; Wed, 15 Feb 2023 16:34:14 +0800 Received: from canpemm500010.china.huawei.com ([7.192.105.118]) by canpemm500010.china.huawei.com ([7.192.105.118]) with mapi id 15.01.2507.017; Wed, 15 Feb 2023 16:34:14 +0800 From: "liujian (CE)" To: John Stultz CC: "tglx@linutronix.de" , "sboyd@kernel.org" , "linux-kernel@vger.kernel.org" , "peterz@infradead.org" , "Paul E. McKenney" Subject: RE: [Question] softlockup in run_timer_softirq Thread-Topic: [Question] softlockup in run_timer_softirq Thread-Index: AQHZP+XzD5YJ48ci7USTV771E31fma7PkEww Date: Wed, 15 Feb 2023 08:34:14 +0000 Message-ID: References: In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.174.176.93] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > -----Original Message----- > From: John Stultz [mailto:jstultz@google.com] > Sent: Tuesday, February 14, 2023 4:01 AM > To: liujian (CE) > Cc: tglx@linutronix.de; sboyd@kernel.org; linux-kernel@vger.kernel.org; > peterz@infradead.org; Paul E. McKenney > Subject: Re: [Question] softlockup in run_timer_softirq >=20 > On Fri, Feb 10, 2023 at 1:51 AM liujian (CE) wrote: > > > > During the syz test, we encountered many problems with various timer > > handler functions softlockup. > > > > We analyze __run_timers() and find the following problem. > > > > In the while loop of __run_timers(), because there are too many timers > > or improper timer handler functions, if the processing time of the > > expired timers is always greater than the time wheel's next_expiry, > > the function will loop infinitely. > > > > The following extreme test case can be used to reproduce the problem. > > An extreme test case[1] is constructed to reproduce the problem. >=20 > Thanks for reporting and sending out this data: >=20 > First, any chance you might submit this as a in-kernel-stress test? > Maybe utilizing the kernel/torture.c framework? >=20 Okay, I'll learn this framework and do this thing. > (Though the test may need to occasionally take a break so the system can > eventually catch up) >=20 > > Is this a problem or an unreasonable use? > > > > Can we limit the running time of __run_timers() [2]? > > > > Does anyone have a good idea to solve this problem? >=20 > So your patch reminds me of Peter's softirq_needs_break() logic: >=20 > https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h= =3Dco > re/softirq >=20 > Maybe it could extend that series for the timer softirq as well? >=20 Thank you. Yes. Base on the patchset and the extended patch for timer [1], the soft lockup = problem does not occur. By the way, I see this is a very old patchset? Will this patchset push the= main line? @John @Peter [1] Author: Liu Jian Date: Tue Feb 14 09:53:46 2023 +0800 softirq, timer: Use softirq_needs_break() =20 In the while loop of __run_timers(), because there are too many timers = or improper timer handler functions, if the processing time of the expired timers is always greater than the time wheel's next_expiry, the function will loop infinitely. =20 To prevent this, use the timeout/break logic provided by SoftIRQs.If the running time exceeds the limit, break the loop and an additional TIMER_SOFTIRQ is triggered. =20 Signed-off-by: Liu Jian diff --git a/kernel/time/timer.c b/kernel/time/timer.c index 63a8ce7177dd..70744a469a39 100644 Tested-by: tags if it works for them as well. --- a/kernel/time/timer.c +++ b/kernel/time/timer.c @@ -1992,7 +1992,7 @@ void timer_clear_idle(void) * __run_timers - run all expired timers (if any) on this CPU. * @base: the timer vector to be processed. */ -static inline void __run_timers(struct timer_base *base) +static inline void __run_timers(struct timer_base *base, struct softirq_ac= tion *h) { struct hlist_head heads[LVL_DEPTH]; int levels; @@ -2020,6 +2020,12 @@ static inline void __run_timers(struct timer_base *b= ase) =20 while (levels--) expire_timers(base, heads + levels); + + if (softirq_needs_break(h)) { + if (time_after_eq(jiffies, base->next_expiry)) + __raise_softirq_irqoff(TIMER_SOFTIRQ); + break; + } } raw_spin_unlock_irq(&base->lock); timer_base_unlock_expiry(base); @@ -2032,9 +2038,9 @@ static __latent_entropy void run_timer_softirq(struct= softirq_action *h) { struct timer_base *base =3D this_cpu_ptr(&timer_bases[BASE_STD]); =20 - __run_timers(base); + __run_timers(base, h); if (IS_ENABLED(CONFIG_NO_HZ_COMMON)) - __run_timers(this_cpu_ptr(&timer_bases[BASE_DEF])); + __run_timers(this_cpu_ptr(&timer_bases[BASE_DEF]), h); } =20 /* > thanks > -john