From nobody Thu Apr 16 00:53:08 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AD44C4332F for ; Wed, 23 Nov 2022 01:21:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235066AbiKWBVc (ORCPT ); Tue, 22 Nov 2022 20:21:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233625AbiKWBVT (ORCPT ); Tue, 22 Nov 2022 20:21:19 -0500 Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [IPv6:2607:f8b0:4864:20::82e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECBF1B5A for ; Tue, 22 Nov 2022 17:21:17 -0800 (PST) Received: by mail-qt1-x82e.google.com with SMTP id cg5so10430958qtb.12 for ; Tue, 22 Nov 2022 17:21:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9GtnXpguQj4O0Y4VxU8SlCD6RocW4OIg2BaI4YRFC28=; b=AA9IN5HCivPYCzFmqlbWwF8baNeqOR9Ccmdvk3ZCGGkEayW3L++azUK+9Bgah47C5E /GHOjjDLal9oMLaNvJhcWfo8jCo0FSsdC2sfXTZ3f4KI2WJhp6EuE1nIObGJTo7N5phH V/nSlNvCnKkrWRiDgKlRqujAL6x+cz3UM3T6I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9GtnXpguQj4O0Y4VxU8SlCD6RocW4OIg2BaI4YRFC28=; b=2pGoYYO4rSpJ0OfJxrExhWlBAL9X0MuT0MkJO8VDwZ/GufdWCpsieqMzdmHOUXCJKI gunjGLggNilw7i852bJ5iiAWFMh18bjR72X7VYq49QaGFP739GvMAdugpE8Y9lGqbV3f Aqw7/ddkI9LUxTFsyrtlDJdrflwDhayCrAYU4xEci66q3VMDCAwNfY7P03f1kTFA/XiR 1H3TVurU16UHQm0K6FGNfCYa/8qco1hSTLBVPn/gf9IY9qDQa0DG8sdNfbGhbzz+yK0q 9pqYgYldb1GAS5bJUdVZ7snoAsvPr8r5NFrKaY+Ab7XisgzbnXe/vaZ1w7njVH/cFWjH 3Arw== X-Gm-Message-State: ANoB5pm1irUAKxBIGvBwxEUcxFQ27LFIELlChh4Yu0l/hMtfQfVOxVGE YYKs9aj76uf5OGi+7nqdN4FPtmkCUZvWSw== X-Google-Smtp-Source: AA0mqf4C1zrZVf02my366Mut7UBgrffo7WKuTROXtE64lO8/bFc9S6NaTqSqltNwXqzsRCMakHJreA== X-Received: by 2002:ac8:5551:0:b0:3a6:5573:5851 with SMTP id o17-20020ac85551000000b003a655735851mr4334524qtr.313.1669166476848; Tue, 22 Nov 2022 17:21:16 -0800 (PST) Received: from joelboxx.c.googlers.com.com (228.221.150.34.bc.googleusercontent.com. [34.150.221.228]) by smtp.gmail.com with ESMTPSA id gc7-20020a05622a59c700b0035ce8965045sm9096026qtb.42.2022.11.22.17.21.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Nov 2022 17:21:16 -0800 (PST) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org Cc: "Joel Fernandes (Google)" , Dietmar Eggemann , Ben Segall , Daniel Bristot de Oliveira , Davidlohr Bueso , Ingo Molnar , Josh Triplett , Juri Lelli , Mel Gorman , "Paul E. McKenney" , Peter Zijlstra , Steven Rostedt , Valentin Schneider , Vincent Guittot , kernel-team@android.com, John Stultz , Joel Fernandes , Qais Yousef , Will Deacon , Waiman Long , Boqun Feng , "Connor O'Brien" Subject: [PATCH RFC 1/3] sched/pe: Exclude balance callback queuing during proxy()'s migrate Date: Wed, 23 Nov 2022 01:21:02 +0000 Message-Id: <20221123012104.3317665-2-joel@joelfernandes.org> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog In-Reply-To: <20221123012104.3317665-1-joel@joelfernandes.org> References: <20221123012104.3317665-1-joel@joelfernandes.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In commit 565790d28b1e ("sched: Fix balance_callback()"), it is clear that rq lock needs to be held from when balance callbacks are queued to when __balance_callbacks() in schedule() is called. This has to be done without dropping the runqueue lock in between. If dropped between queuing and executing callbacks, it is possible that another CPU, say in __sched_setscheduler() can queue balancing callbacks and cause issues as show in that commit. This is precisely what happens in proxy(). During a proxy(), the current CPU's rq lock is temporary dropped when moving the tasks in the migrate list to the owner CPU. This commit therefore make proxy() exclude balance callback queuing on other CPUs, in the section where proxy() temporarily drops the rq lock of current CPU. CPUs that acquire a remote CPU's rq lock but later queue a balance callback, are made to call the new helpers in this commit to check whether balance_lock is held. If it is held, then the rq lock is released and a re-attempt is made to acquire it in the hopes that the ban on balancing callback queuing has elapsed. Reported-by: Dietmar Eggemann Signed-off-by: Joel Fernandes (Google) --- kernel/sched/core.c | 72 ++++++++++++++++++++++++++++++++++++++++++-- kernel/sched/sched.h | 7 ++++- 2 files changed, 76 insertions(+), 3 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 88a5fa34dc06..aba90b3dc3ef 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -633,6 +633,29 @@ struct rq *__task_rq_lock(struct task_struct *p, struc= t rq_flags *rf) } } =20 +/* + * Helper to call __task_rq_lock safely, in scenarios where we might be ab= out to + * queue a balance callback on a remote CPU. That CPU might be in proxy(),= and + * could have released its rq lock while holding balance_lock. So release = rq + * lock in such a situation to avoid deadlock, and retry. + */ +struct rq *__task_rq_lock_balance(struct task_struct *p, struct rq_flags *= rf) +{ + struct rq *rq; + bool locked =3D false; + + do { + if (locked) { + __task_rq_unlock(rq, rf); + cpu_relax(); + } + rq =3D __task_rq_lock(p, rf); + locked =3D true; + } while (raw_spin_is_locked(&rq->balance_lock)); + + return rq; +} + /* * task_rq_lock - lock p->pi_lock and lock the rq @p resides on. */ @@ -675,6 +698,29 @@ struct rq *task_rq_lock(struct task_struct *p, struct = rq_flags *rf) } } =20 +/* + * Helper to call task_rq_lock safely, in scenarios where we might be abou= t to + * queue a balance callback on a remote CPU. That CPU might be in proxy(),= and + * could have released its rq lock while holding balance_lock. So release = rq + * lock in such a situation to avoid deadlock, and retry. + */ +struct rq *task_rq_lock_balance(struct task_struct *p, struct rq_flags *rf) +{ + struct rq *rq; + bool locked =3D false; + + do { + if (locked) { + task_rq_unlock(rq, p, rf); + cpu_relax(); + } + rq =3D task_rq_lock(p, rf); + locked =3D true; + } while (raw_spin_is_locked(&rq->balance_lock)); + + return rq; +} + /* * RQ-clock updating methods: */ @@ -6739,6 +6785,12 @@ proxy(struct rq *rq, struct task_struct *next, struc= t rq_flags *rf) p->wake_cpu =3D wake_cpu; } =20 + /* + * Prevent other CPUs from queuing balance callbacks while we migrate + * tasks in the migrate_list with the rq lock released. + */ + raw_spin_lock(&rq->balance_lock); + rq_unpin_lock(rq, rf); raw_spin_rq_unlock(rq); raw_spin_rq_lock(that_rq); @@ -6758,7 +6810,21 @@ proxy(struct rq *rq, struct task_struct *next, struc= t rq_flags *rf) } =20 raw_spin_rq_unlock(that_rq); + + /* + * This may make lockdep unhappy as we acquire rq->lock with + * balance_lock held. But that should be a false positive, as the + * following pattern happens only on the current CPU with interrupts + * disabled: + * rq_lock() + * balance_lock(); + * rq_unlock(); + * rq_lock(); + */ raw_spin_rq_lock(rq); + + raw_spin_unlock(&rq->balance_lock); + rq_repin_lock(rq, rf); =20 return NULL; /* Retry task selection on _this_ CPU. */ @@ -7489,7 +7555,7 @@ void rt_mutex_setprio(struct task_struct *p, struct t= ask_struct *pi_task) if (p->pi_top_task =3D=3D pi_task && prio =3D=3D p->prio && !dl_prio(prio= )) return; =20 - rq =3D __task_rq_lock(p, &rf); + rq =3D __task_rq_lock_balance(p, &rf); update_rq_clock(rq); /* * Set under pi_lock && rq->lock, such that the value can be used under @@ -8093,7 +8159,8 @@ static int __sched_setscheduler(struct task_struct *p, * To be able to change p->policy safely, the appropriate * runqueue lock must be held. */ - rq =3D task_rq_lock(p, &rf); + rq =3D task_rq_lock_balance(p, &rf); + update_rq_clock(rq); =20 /* @@ -10312,6 +10379,7 @@ void __init sched_init(void) =20 rq =3D cpu_rq(i); raw_spin_lock_init(&rq->__lock); + raw_spin_lock_init(&rq->balance_lock); rq->nr_running =3D 0; rq->calc_load_active =3D 0; rq->calc_load_update =3D jiffies + LOAD_FREQ; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 354e75587fed..17947a4009de 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1057,6 +1057,7 @@ struct rq { unsigned long cpu_capacity_orig; =20 struct callback_head *balance_callback; + raw_spinlock_t balance_lock; =20 unsigned char nohz_idle_balance; unsigned char idle_balance; @@ -1748,18 +1749,22 @@ queue_balance_callback(struct rq *rq, void (*func)(struct rq *rq)) { lockdep_assert_rq_held(rq); + raw_spin_lock(&rq->balance_lock); =20 /* * Don't (re)queue an already queued item; nor queue anything when * balance_push() is active, see the comment with * balance_push_callback. */ - if (unlikely(head->next || rq->balance_callback =3D=3D &balance_push_call= back)) + if (unlikely(head->next || rq->balance_callback =3D=3D &balance_push_call= back)) { + raw_spin_unlock(&rq->balance_lock); return; + } =20 head->func =3D (void (*)(struct callback_head *))func; head->next =3D rq->balance_callback; rq->balance_callback =3D head; + raw_spin_unlock(&rq->balance_lock); } =20 #define rcu_dereference_check_sched_domain(p) \ --=20 2.38.1.584.g0f3c55d4c2-goog From nobody Thu Apr 16 00:53:08 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0255C4332F for ; Wed, 23 Nov 2022 01:21:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235264AbiKWBVm (ORCPT ); Tue, 22 Nov 2022 20:21:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234403AbiKWBVU (ORCPT ); Tue, 22 Nov 2022 20:21:20 -0500 Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13173D8 for ; Tue, 22 Nov 2022 17:21:19 -0800 (PST) Received: by mail-qt1-x829.google.com with SMTP id z6so10450546qtv.5 for ; Tue, 22 Nov 2022 17:21:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XtElyjrt1s1nmV78/tq8sM5iYOsbz+FSac/YAvZjzfY=; b=FeWKJLVgavtLMAmkacg3nndLLRX3v5gjRpFC0Pvsl6Tjzl2psoWHhpo+MXDXYXNHYL CfF3y36WuiB0WuNkYGPtOyBLiDpWU0Oo+9AnYz0ibA2m9CGi3S8fN7tzPoPnFN3BO6ad 3wAYQbtsrcAEoSEtuag95BJWifgzzKIJQxrvY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XtElyjrt1s1nmV78/tq8sM5iYOsbz+FSac/YAvZjzfY=; b=K1djOJtzfzqm7Xb0ZZXWElIlN96zxZb2cR+g4j0QDzulaVrKm0bvG4qTshojRFl3TE rV8aKKxpwBEm9GkIEZQg+DA6MBsyyoRNp8Ky8mz6px06WQMBwsYQGqX1PdXmFf6JGBIP 7Xtmg4/pWLSU1rF/QO2pUqD1+muH8pfaZL4j9tcO2hBQ/XIJaq67DoYKwOScVVvltmYU j/NpfAwt3hi5p022BrckeAuwda1iaGFyWgOGkp4QCR6vSUa7MgAHqCSEALl4nMRwRObI rYJQBrp60ZrVhPndZf2Ac4RkgTIpQpc69ZkLHKPfSY1D5xF/ni+gXFZTuh0m70qLcjd1 vC4g== X-Gm-Message-State: ANoB5pn8L72SmcfBJ/NdYFPzcBS8TEU6XmQk7EfHzePXezDHOX2jErDq D5j/zNZfuVQr72SRma1TWnXZyumnw+SFPA== X-Google-Smtp-Source: AA0mqf78RcsF5kXE9EyATFRoC96kRCwekCycihHv9lCsxitfRfZ9GBr1gXiCvR+h38OKBZACSr09kA== X-Received: by 2002:a05:622a:5a0c:b0:3a5:1dde:dcee with SMTP id fy12-20020a05622a5a0c00b003a51ddedceemr24264139qtb.639.1669166477975; Tue, 22 Nov 2022 17:21:17 -0800 (PST) Received: from joelboxx.c.googlers.com.com (228.221.150.34.bc.googleusercontent.com. [34.150.221.228]) by smtp.gmail.com with ESMTPSA id gc7-20020a05622a59c700b0035ce8965045sm9096026qtb.42.2022.11.22.17.21.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Nov 2022 17:21:17 -0800 (PST) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org Cc: "Joel Fernandes (Google)" , Ben Segall , Daniel Bristot de Oliveira , Davidlohr Bueso , Dietmar Eggemann , Ingo Molnar , Josh Triplett , Juri Lelli , Mel Gorman , "Paul E. McKenney" , Peter Zijlstra , Steven Rostedt , Valentin Schneider , Vincent Guittot , kernel-team@android.com, John Stultz , Joel Fernandes , Qais Yousef , Will Deacon , Waiman Long , Boqun Feng , "Connor O'Brien" Subject: [PATCH RFC 2/3] locktorture: Allow non-rtmutex lock types to be boosted Date: Wed, 23 Nov 2022 01:21:03 +0000 Message-Id: <20221123012104.3317665-3-joel@joelfernandes.org> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog In-Reply-To: <20221123012104.3317665-1-joel@joelfernandes.org> References: <20221123012104.3317665-1-joel@joelfernandes.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Currently RT boosting is only done for rtmutex_lock, however with proxy execution, we also have the mutex_lock participating in priorities. To exercise the testing better, add RT boosting to other lock testing types as well, using a new knob (rt_boost). Tested with boot parameters: locktorture.torture_type=3Dmutex_lock locktorture.onoff_interval=3D1 locktorture.nwriters_stress=3D8 locktorture.stutter=3D0 locktorture.rt_boost=3D1 locktorture.rt_boost_factor=3D1 locktorture.nlocks=3D3 For the rtmutex test, rt_boost is always enabled even if disabling is requested. Signed-off-by: Joel Fernandes (Google) --- kernel/locking/locktorture.c | 91 +++++++++++++++++++----------------- 1 file changed, 48 insertions(+), 43 deletions(-) diff --git a/kernel/locking/locktorture.c b/kernel/locking/locktorture.c index bc3557677eed..5a388ac96a9b 100644 --- a/kernel/locking/locktorture.c +++ b/kernel/locking/locktorture.c @@ -46,6 +46,7 @@ torture_param(int, shutdown_secs, 0, "Shutdown time (j), = <=3D zero to disable."); torture_param(int, stat_interval, 60, "Number of seconds between stats printk()s"); torture_param(int, stutter, 5, "Number of jiffies to run/halt test, 0=3Ddi= sable"); +torture_param(int, rt_boost, 0, "Perform an rt-boost from the writer, alwa= ys 1 for rtmutex_lock"); torture_param(int, verbose, 1, "Enable verbose debugging printk()s"); torture_param(int, nlocks, 1, @@ -129,15 +130,44 @@ static void torture_lock_busted_write_unlock(int tid = __maybe_unused) /* BUGGY, do not use in real life!!! */ } =20 -static void torture_boost_dummy(struct torture_random_state *trsp) +static void torture_rt_boost(struct torture_random_state *trsp) { - /* Only rtmutexes care about priority */ + const unsigned int factor =3D 50000; /* yes, quite arbitrary */ + + if (!rt_boost) + return; + + if (!rt_task(current)) { + /* + * Boost priority once every ~50k operations. When the + * task tries to take the lock, the rtmutex it will account + * for the new priority, and do any corresponding pi-dance. + */ + if (trsp && !(torture_random(trsp) % + (cxt.nrealwriters_stress * factor))) { + sched_set_fifo(current); + } else /* common case, do nothing */ + return; + } else { + /* + * The task will remain boosted for another ~500k operations, + * then restored back to its original prio, and so forth. + * + * When @trsp is nil, we want to force-reset the task for + * stopping the kthread. + */ + if (!trsp || !(torture_random(trsp) % + (cxt.nrealwriters_stress * factor * 2))) { + sched_set_normal(current, 0); + } else /* common case, do nothing */ + return; + } } =20 static struct lock_torture_ops lock_busted_ops =3D { .writelock =3D torture_lock_busted_write_lock, .write_delay =3D torture_lock_busted_write_delay, - .task_boost =3D torture_boost_dummy, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_lock_busted_write_unlock, .readlock =3D NULL, .read_delay =3D NULL, @@ -181,7 +211,7 @@ __releases(torture_spinlock) static struct lock_torture_ops spin_lock_ops =3D { .writelock =3D torture_spin_lock_write_lock, .write_delay =3D torture_spin_lock_write_delay, - .task_boost =3D torture_boost_dummy, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_spin_lock_write_unlock, .readlock =3D NULL, .read_delay =3D NULL, @@ -208,7 +238,7 @@ __releases(torture_spinlock) static struct lock_torture_ops spin_lock_irq_ops =3D { .writelock =3D torture_spin_lock_write_lock_irq, .write_delay =3D torture_spin_lock_write_delay, - .task_boost =3D torture_boost_dummy, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_lock_spin_write_unlock_irq, .readlock =3D NULL, .read_delay =3D NULL, @@ -277,7 +307,7 @@ __releases(torture_rwlock) static struct lock_torture_ops rw_lock_ops =3D { .writelock =3D torture_rwlock_write_lock, .write_delay =3D torture_rwlock_write_delay, - .task_boost =3D torture_boost_dummy, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_rwlock_write_unlock, .readlock =3D torture_rwlock_read_lock, .read_delay =3D torture_rwlock_read_delay, @@ -320,7 +350,7 @@ __releases(torture_rwlock) static struct lock_torture_ops rw_lock_irq_ops =3D { .writelock =3D torture_rwlock_write_lock_irq, .write_delay =3D torture_rwlock_write_delay, - .task_boost =3D torture_boost_dummy, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_rwlock_write_unlock_irq, .readlock =3D torture_rwlock_read_lock_irq, .read_delay =3D torture_rwlock_read_delay, @@ -362,7 +392,7 @@ __releases(torture_mutex) static struct lock_torture_ops mutex_lock_ops =3D { .writelock =3D torture_mutex_lock, .write_delay =3D torture_mutex_delay, - .task_boost =3D torture_boost_dummy, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_mutex_unlock, .readlock =3D NULL, .read_delay =3D NULL, @@ -460,7 +490,7 @@ static struct lock_torture_ops ww_mutex_lock_ops =3D { .exit =3D torture_ww_mutex_exit, .writelock =3D torture_ww_mutex_lock, .write_delay =3D torture_mutex_delay, - .task_boost =3D torture_boost_dummy, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_ww_mutex_unlock, .readlock =3D NULL, .read_delay =3D NULL, @@ -471,6 +501,11 @@ static struct lock_torture_ops ww_mutex_lock_ops =3D { #ifdef CONFIG_RT_MUTEXES static DEFINE_RT_MUTEX(torture_rtmutex); =20 +static void torture_rtmutex_init(void) +{ + rt_boost =3D 1; +} + static int torture_rtmutex_lock(int tid __maybe_unused) __acquires(torture_rtmutex) { @@ -478,37 +513,6 @@ __acquires(torture_rtmutex) return 0; } =20 -static void torture_rtmutex_boost(struct torture_random_state *trsp) -{ - const unsigned int factor =3D 50000; /* yes, quite arbitrary */ - - if (!rt_task(current)) { - /* - * Boost priority once every ~50k operations. When the - * task tries to take the lock, the rtmutex it will account - * for the new priority, and do any corresponding pi-dance. - */ - if (trsp && !(torture_random(trsp) % - (cxt.nrealwriters_stress * factor))) { - sched_set_fifo(current); - } else /* common case, do nothing */ - return; - } else { - /* - * The task will remain boosted for another ~500k operations, - * then restored back to its original prio, and so forth. - * - * When @trsp is nil, we want to force-reset the task for - * stopping the kthread. - */ - if (!trsp || !(torture_random(trsp) % - (cxt.nrealwriters_stress * factor * 2))) { - sched_set_normal(current, 0); - } else /* common case, do nothing */ - return; - } -} - static void torture_rtmutex_delay(struct torture_random_state *trsp) { const unsigned long shortdelay_us =3D 2; @@ -535,9 +539,10 @@ __releases(torture_rtmutex) } =20 static struct lock_torture_ops rtmutex_lock_ops =3D { + .init =3D torture_rtmutex_init, .writelock =3D torture_rtmutex_lock, .write_delay =3D torture_rtmutex_delay, - .task_boost =3D torture_rtmutex_boost, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_rtmutex_unlock, .readlock =3D NULL, .read_delay =3D NULL, @@ -604,7 +609,7 @@ __releases(torture_rwsem) static struct lock_torture_ops rwsem_lock_ops =3D { .writelock =3D torture_rwsem_down_write, .write_delay =3D torture_rwsem_write_delay, - .task_boost =3D torture_boost_dummy, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_rwsem_up_write, .readlock =3D torture_rwsem_down_read, .read_delay =3D torture_rwsem_read_delay, @@ -656,7 +661,7 @@ static struct lock_torture_ops percpu_rwsem_lock_ops = =3D { .exit =3D torture_percpu_rwsem_exit, .writelock =3D torture_percpu_rwsem_down_write, .write_delay =3D torture_rwsem_write_delay, - .task_boost =3D torture_boost_dummy, + .task_boost =3D torture_rt_boost, .writeunlock =3D torture_percpu_rwsem_up_write, .readlock =3D torture_percpu_rwsem_down_read, .read_delay =3D torture_rwsem_read_delay, --=20 2.38.1.584.g0f3c55d4c2-goog From nobody Thu Apr 16 00:53:08 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 745FCC4332F for ; Wed, 23 Nov 2022 01:21:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235207AbiKWBVi (ORCPT ); Tue, 22 Nov 2022 20:21:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234575AbiKWBVU (ORCPT ); Tue, 22 Nov 2022 20:21:20 -0500 Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E821623B for ; Tue, 22 Nov 2022 17:21:19 -0800 (PST) Received: by mail-qk1-x72e.google.com with SMTP id x18so11567196qki.4 for ; Tue, 22 Nov 2022 17:21:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3Xn44YesMZuZ8Y4WLRq4kq1k9pek4ng6uS2E49EtvTo=; b=X1sloOlgGllcTjloaNOVgImsa3WWDB0rKo/V4rhEpZ6lwZVuZCGd5Ry70nxVZf3tCD Kwd1kV+FCfUtXRqFb5UhftZHXOx2NDkiV9j1kNON7jw4FxVtyzI+o3GjiYcP57lxHRyX l5PuyYEojNSISBrTYWZMPdiOMOxz/AtOwfM3s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3Xn44YesMZuZ8Y4WLRq4kq1k9pek4ng6uS2E49EtvTo=; b=Qn1ox2QKolfF/711kQpySiRanXZEX4Ukqq3tHAlpaQy/NoaKIikZszxo3zo33DR8sR vj22dr9gzgCkLnKywOjBeG8MBU9BWSFYZ+dw2cVK0hhWKi4+aqQS+lLPUEBNM7PibWOl wDz7JaaV3dJdFOwVhD34fCbvUyN7qOlZASVNXDqC3QSUSIqdh4ntc/QwC68udPRPtv7D 9SZajbQu4xRVdCaskciQv/xxq+loditdyZZKhFhl+GekiEm4PDDGPqe+2cGIZieuo2ho +JhKILfkgKKTxbGdDNhoA4NseHaHJEORK764vYjqtjj5olJJAj8Jn/bZfavFNd+OfkvB kd+w== X-Gm-Message-State: ANoB5pnwavNiEUlmT5Vsse9UlxGD80Vd5MCMY8guUxdv+g1OzC456ssU AKCf4PYkmetrX3J6tQrOhJ6hilNUanmH1g== X-Google-Smtp-Source: AA0mqf7MLAkSje6Pt2VcpKfquMHKbWhD/gitl9fSJQ2nxO3ekqrUFA8tF3Uid0LuBU0OPi8/kaD3HQ== X-Received: by 2002:a37:cd7:0:b0:6fa:1e59:4b72 with SMTP id 206-20020a370cd7000000b006fa1e594b72mr22925074qkm.247.1669166478871; Tue, 22 Nov 2022 17:21:18 -0800 (PST) Received: from joelboxx.c.googlers.com.com (228.221.150.34.bc.googleusercontent.com. [34.150.221.228]) by smtp.gmail.com with ESMTPSA id gc7-20020a05622a59c700b0035ce8965045sm9096026qtb.42.2022.11.22.17.21.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Nov 2022 17:21:18 -0800 (PST) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org Cc: "Joel Fernandes (Google)" , Ben Segall , Daniel Bristot de Oliveira , Davidlohr Bueso , Dietmar Eggemann , Ingo Molnar , Josh Triplett , Juri Lelli , Mel Gorman , "Paul E. McKenney" , Peter Zijlstra , Steven Rostedt , Valentin Schneider , Vincent Guittot , kernel-team@android.com, John Stultz , Joel Fernandes , Qais Yousef , Will Deacon , Waiman Long , Boqun Feng , "Connor O'Brien" Subject: [PATCH RFC 3/3] locktorture: Make the rt_boost factor a tunable Date: Wed, 23 Nov 2022 01:21:04 +0000 Message-Id: <20221123012104.3317665-4-joel@joelfernandes.org> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog In-Reply-To: <20221123012104.3317665-1-joel@joelfernandes.org> References: <20221123012104.3317665-1-joel@joelfernandes.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The rt boosting in locktorture has a factor variable large enough that boosting only happens once every minute or so. Add a tunable to educe the factor so that boosting happens more often, to test paths and arrive at failure modes earlier. With this change, I can set the factor to like 50 and have the boosting happens every 10 seconds or so. Tested with boot parameters: locktorture.torture_type=3Dmutex_lock locktorture.onoff_interval=3D1 locktorture.nwriters_stress=3D8 locktorture.stutter=3D0 locktorture.rt_boost=3D1 locktorture.rt_boost_factor=3D50 locktorture.nlocks=3D3 Signed-off-by: Joel Fernandes (Google) --- kernel/locking/locktorture.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/kernel/locking/locktorture.c b/kernel/locking/locktorture.c index 5a388ac96a9b..e4529c2166e9 100644 --- a/kernel/locking/locktorture.c +++ b/kernel/locking/locktorture.c @@ -47,6 +47,7 @@ torture_param(int, stat_interval, 60, "Number of seconds between stats printk()s"); torture_param(int, stutter, 5, "Number of jiffies to run/halt test, 0=3Ddi= sable"); torture_param(int, rt_boost, 0, "Perform an rt-boost from the writer, alwa= ys 1 for rtmutex_lock"); +torture_param(int, rt_boost_factor, 50000, "A factor determining how often= rt-boost happens"); torture_param(int, verbose, 1, "Enable verbose debugging printk()s"); torture_param(int, nlocks, 1, @@ -132,15 +133,15 @@ static void torture_lock_busted_write_unlock(int tid = __maybe_unused) =20 static void torture_rt_boost(struct torture_random_state *trsp) { - const unsigned int factor =3D 50000; /* yes, quite arbitrary */ + const unsigned int factor =3D rt_boost_factor; /* yes, quite arbitrary */ =20 if (!rt_boost) return; =20 if (!rt_task(current)) { /* - * Boost priority once every ~50k operations. When the - * task tries to take the lock, the rtmutex it will account + * Boost priority once every rt_boost_factor operations. When + * the task tries to take the lock, the rtmutex it will account * for the new priority, and do any corresponding pi-dance. */ if (trsp && !(torture_random(trsp) % @@ -150,8 +151,9 @@ static void torture_rt_boost(struct torture_random_stat= e *trsp) return; } else { /* - * The task will remain boosted for another ~500k operations, - * then restored back to its original prio, and so forth. + * The task will remain boosted for another 10*rt_boost_factor + * operations, then restored back to its original prio, and so + * forth. * * When @trsp is nil, we want to force-reset the task for * stopping the kthread. --=20 2.38.1.584.g0f3c55d4c2-goog