From nobody Sun Feb 8 00:26:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 966B3C0015E for ; Tue, 25 Jul 2023 23:29:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231811AbjGYX3d (ORCPT ); Tue, 25 Jul 2023 19:29:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231705AbjGYX3a (ORCPT ); Tue, 25 Jul 2023 19:29:30 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E6D819BA for ; Tue, 25 Jul 2023 16:29:29 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-7878e573827so224020839f.1 for ; Tue, 25 Jul 2023 16:29:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327767; x=1690932567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SjqXIuVtq0u8YshojVwovW+pCVt0mgB/LilQjFvvm4k=; b=YpU7e+Pj0/atr4EbsOJuU3upQRDKhMb6J0aC/Kwb5sK5X1H4MDa7KtxhGNk+JprQa8 xnDQkG8c0nNaGaiUAxLBwoHST4BNpZ2nFNZPTPeF7D0/5SyEMgWEAfwg/HQLLaNJ5GPS CVrYBTezNf2/CmiE7wAzXkmnNs+/HVvnLeI5w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327767; x=1690932567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SjqXIuVtq0u8YshojVwovW+pCVt0mgB/LilQjFvvm4k=; b=edg07nKGog7ElI3oqPFwG5ySav9qCI2L6R4kWtOf6hmOa1MVaH5exUq5D3++PgpNKL OD3UEF/kGCIC3i0EtE88+ug8zTCaF0vlMqITHcbwFN6cpG0w0KoqzxVzXSDCRD/kT1j6 VG1o4aqIDTbr71Z+WQXSjekwzaTFFDsKMP2U903IyAKC/cZ12N9HuwVj9IXdJTQTQFbD qpZMpaF9ZomnwMxUJ5vwo3pD52GdAy1Re8ad6FvQxHdzx97jdALkl4mQVpYeUF96Q9Hu mZ8uY6RGoteMOqJojW0NghSy3YqCxTFYAeFU4DB9clo1+jBCLkMGnnw4cBbZASepbzvD gzVg== X-Gm-Message-State: ABy/qLbs5NwSFpStFVETaLYVUNDX4FgdrwEV1vEjYAyRgywESrv7VN1V 4Fd1dIFRFM7yd/E4WhCPWdzEa21BIajgUWDk3a0= X-Google-Smtp-Source: APBJJlHSQsPaCUv83pDozY5eFiLygkk647/+RNJH82m94E6xEn7hjRbJJFpy2CXENNqGa4wtwOyDwg== X-Received: by 2002:a6b:e818:0:b0:783:6ff9:a6c7 with SMTP id f24-20020a6be818000000b007836ff9a6c7mr334374ioh.6.1690327767704; Tue, 25 Jul 2023 16:29:27 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:26 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, Davidlohr Bueso , "Paul E. McKenney" , Josh Triplett Cc: rcu@vger.kernel.org, "Joel Fernandes (Google)" Subject: [PATCH 1/5] rcutorture: Fix stuttering races and other issues Date: Tue, 25 Jul 2023 23:29:06 +0000 Message-ID: <20230725232913.2981357-2-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The stuttering code isn't functioning as expected. Ideally, it should pause the torture threads for a designated period before resuming. Yet, it fails to halt the test for the correct duration. Additionally, a race condition exists, potentially causing the stuttering code to pause for an extended period if the 'spt' variable is non-zero due to the stutter orchestration thread's inadequate CPU time. Moreover, over-stuttering can hinder RCU's progress on TREE07 kernels. This happens as the stuttering code may run within a softirq due to RCU callbacks. Consequently, ksoftirqd keeps a CPU busy for several seconds, thus obstructing RCU's progress. This situation triggers a warning message in the logs: [ 2169.481783] rcu_torture_writer: rtort_pipe_count: 9 This warning suggests that an RCU torture object, although invisible to RCU readers, couldn't make it past the pipe array and be freed -- a strong indication that there weren't enough grace periods during the stutter interval. To address these issues, this patch sets the "stutter end" time to an absolute point in the future set by the main stutter thread. This is then used for waiting in stutter_wait(). While the stutter thread still defines this absolute time, the waiters' waiting logic doesn't rely on the stutter thread receiving sufficient CPU time to halt the stuttering as the halting is now self-controlled. Signed-off-by: Joel Fernandes (Google) --- kernel/torture.c | 46 +++++++++++++--------------------------------- 1 file changed, 13 insertions(+), 33 deletions(-) diff --git a/kernel/torture.c b/kernel/torture.c index 68dba4ecab5c..63f8f2a7d960 100644 --- a/kernel/torture.c +++ b/kernel/torture.c @@ -719,7 +719,7 @@ static void torture_shutdown_cleanup(void) * suddenly applied to or removed from the system. */ static struct task_struct *stutter_task; -static int stutter_pause_test; +static ktime_t stutter_till_abs_time; static int stutter; static int stutter_gap; =20 @@ -729,30 +729,17 @@ static int stutter_gap; */ bool stutter_wait(const char *title) { - unsigned int i =3D 0; bool ret =3D false; - int spt; + ktime_t now_ns, till_ns; =20 cond_resched_tasks_rcu_qs(); - spt =3D READ_ONCE(stutter_pause_test); - for (; spt; spt =3D READ_ONCE(stutter_pause_test)) { - if (!ret && !rt_task(current)) { - sched_set_normal(current, MAX_NICE); - ret =3D true; - } - if (spt =3D=3D 1) { - torture_hrtimeout_jiffies(1, NULL); - } else if (spt =3D=3D 2) { - while (READ_ONCE(stutter_pause_test)) { - if (!(i++ & 0xffff)) - torture_hrtimeout_us(10, 0, NULL); - cond_resched(); - } - } else { - torture_hrtimeout_jiffies(round_jiffies_relative(HZ), NULL); - } - torture_shutdown_absorb(title); + now_ns =3D ktime_get(); + till_ns =3D READ_ONCE(stutter_till_abs_time); + if (till_ns && ktime_before(now_ns, till_ns)) { + torture_hrtimeout_ns(ktime_sub(till_ns, now_ns), 0, NULL); + ret =3D true; } + torture_shutdown_absorb(title); return ret; } EXPORT_SYMBOL_GPL(stutter_wait); @@ -763,23 +750,16 @@ EXPORT_SYMBOL_GPL(stutter_wait); */ static int torture_stutter(void *arg) { - DEFINE_TORTURE_RANDOM(rand); - int wtime; + ktime_t till_ns; =20 VERBOSE_TOROUT_STRING("torture_stutter task started"); do { if (!torture_must_stop() && stutter > 1) { - wtime =3D stutter; - if (stutter > 2) { - WRITE_ONCE(stutter_pause_test, 1); - wtime =3D stutter - 3; - torture_hrtimeout_jiffies(wtime, &rand); - wtime =3D 2; - } - WRITE_ONCE(stutter_pause_test, 2); - torture_hrtimeout_jiffies(wtime, NULL); + till_ns =3D ktime_add_ns(ktime_get(), + jiffies_to_nsecs(stutter)); + WRITE_ONCE(stutter_till_abs_time, till_ns); + torture_hrtimeout_jiffies(stutter - 1, NULL); } - WRITE_ONCE(stutter_pause_test, 0); if (!torture_must_stop()) torture_hrtimeout_jiffies(stutter_gap, NULL); torture_shutdown_absorb("torture_stutter"); --=20 2.41.0.487.g6d72f3e995-goog From nobody Sun Feb 8 00:26:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27F70EB64DD for ; Tue, 25 Jul 2023 23:29:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231909AbjGYX3i (ORCPT ); Tue, 25 Jul 2023 19:29:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230211AbjGYX3b (ORCPT ); Tue, 25 Jul 2023 19:29:31 -0400 Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F645120 for ; Tue, 25 Jul 2023 16:29:30 -0700 (PDT) Received: by mail-io1-xd2e.google.com with SMTP id ca18e2360f4ac-783549ef058so316767839f.2 for ; Tue, 25 Jul 2023 16:29:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327769; x=1690932569; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t1/tyaZq/EJBhi/7l+hIL3z3pGtmH2Zjtk0EiAh2ooA=; b=W4LU5bSa/QHlkTdxIb4sY4pM7LWuU9noqwcduFR+0CmKenXKzGAI5fNoSvgjHOSlUm IwiPuf4Gm81MY4HrBIdQsBc4MnQ8nMUuNbJ7L7Lgmjqm9m9qXFc6vriyFqRhMkbcPGUE QC142xSTHhGMr/9RiF3+UQBA1dMQ1hLm+EzgM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327769; x=1690932569; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t1/tyaZq/EJBhi/7l+hIL3z3pGtmH2Zjtk0EiAh2ooA=; b=DjQCKqWFruR70cs4N1PbOOVJbtd4uisKEaDfq21zdtj29Z+X2L+xNuqkUYLAttpPcX dqu99BWwTPbbg8CRyJ4wFy0TDDY9zwnO9xYJt9GqFEv1o5Krn+WLgQWBCQTp1JAeNLSk emvhw+SaAqqcPpNpnLOR6p3LAlL+9SPONngD4OXNntsO8Ez4VAVFJXaxoDRxIs29sC/Y uqlleXBDn4nJNK6LZ6a/6LMxpHumk9ji4rlV/CmEodrWmJqhq68LZg36q+L/I/Tt1ZFC bIuZJzCjcfRrYvjul4dKeiKqYqhcZ8NDVMPXOprwwrEAkqznY2dlJFr8/T8CoqAF2kjw jrfA== X-Gm-Message-State: ABy/qLapS+TYA81cPtDyU6MGD0HouGy3/UxIp8OE6FxtdeyAO0dHQtxJ rB+/i2/N+yRcNbhQZAgN9LMnp5vNtxJDix2J98g= X-Google-Smtp-Source: APBJJlFTSX89ddGUxW0LY7A4uygDts9vWsSV2ZCphFzDYxJVS7ZYCAeaT9OC8EaxmdhM7TusKTii5Q== X-Received: by 2002:a6b:5c10:0:b0:783:72d4:8c38 with SMTP id z16-20020a6b5c10000000b0078372d48c38mr341104ioh.13.1690327769166; Tue, 25 Jul 2023 16:29:29 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:28 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, Lai Jiangshan , "Paul E. McKenney" , Josh Triplett , Steven Rostedt , Mathieu Desnoyers , rcu@vger.kernel.org (open list:SLEEPABLE READ-COPY UPDATE (SRCU)) Cc: rcu@vger.kernel.org, "Joel Fernandes (Google)" Subject: [PATCH 2/5] srcu: Fix error handling in init_srcu_struct_fields() Date: Tue, 25 Jul 2023 23:29:07 +0000 Message-ID: <20230725232913.2981357-3-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The current error handling in init_srcu_struct_fields() is a bit inconsistent. If init_srcu_struct_nodes() fails, the function either returns -ENOMEM or 0 depending on whether ssp->sda_is_static is true or false. This can make init_srcu_struct_fields() return 0 even if memory allocation failed! Simplify the error handling by always returning -ENOMEM if either init_srcu_struct_nodes() or the per-CPU allocation fails. This makes the control flow easier to follow and avoids the inconsistent return values. Add goto labels to avoid duplicating the error cleanup code. Link: https://lore.kernel.org/r/20230404003508.GA254019@google.com Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/srcutree.c | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 20d7a238d675..cbc37cbc1805 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -255,29 +255,32 @@ static int init_srcu_struct_fields(struct srcu_struct= *ssp, bool is_static) ssp->srcu_sup->sda_is_static =3D is_static; if (!is_static) ssp->sda =3D alloc_percpu(struct srcu_data); - if (!ssp->sda) { - if (!is_static) - kfree(ssp->srcu_sup); - return -ENOMEM; - } + if (!ssp->sda) + goto err_free_sup; init_srcu_struct_data(ssp); ssp->srcu_sup->srcu_gp_seq_needed_exp =3D 0; ssp->srcu_sup->srcu_last_gp_end =3D ktime_get_mono_fast_ns(); if (READ_ONCE(ssp->srcu_sup->srcu_size_state) =3D=3D SRCU_SIZE_SMALL && S= RCU_SIZING_IS_INIT()) { - if (!init_srcu_struct_nodes(ssp, GFP_ATOMIC)) { - if (!ssp->srcu_sup->sda_is_static) { - free_percpu(ssp->sda); - ssp->sda =3D NULL; - kfree(ssp->srcu_sup); - return -ENOMEM; - } - } else { + if (!init_srcu_struct_nodes(ssp, GFP_ATOMIC)) + goto err_free_sda; + else WRITE_ONCE(ssp->srcu_sup->srcu_size_state, SRCU_SIZE_BIG); - } } ssp->srcu_sup->srcu_ssp =3D ssp; smp_store_release(&ssp->srcu_sup->srcu_gp_seq_needed, 0); /* Init done. */ return 0; + +err_free_sda: + if (!is_static) { + free_percpu(ssp->sda); + ssp->sda =3D NULL; + } +err_free_sup: + if (!is_static) { + kfree(ssp->srcu_sup); + ssp->srcu_sup =3D NULL; + } + return -ENOMEM; } =20 #ifdef CONFIG_DEBUG_LOCK_ALLOC --=20 2.41.0.487.g6d72f3e995-goog From nobody Sun Feb 8 00:26:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9C78EB64DD for ; Tue, 25 Jul 2023 23:29:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232039AbjGYX3m (ORCPT ); Tue, 25 Jul 2023 19:29:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231821AbjGYX3e (ORCPT ); Tue, 25 Jul 2023 19:29:34 -0400 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EC971BF8 for ; Tue, 25 Jul 2023 16:29:32 -0700 (PDT) Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-78363cc070aso316961539f.1 for ; Tue, 25 Jul 2023 16:29:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327770; x=1690932570; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B6faQrb3vVDF+1ZA3HcLro9nEgsaXGn2RtvbxbNNEvc=; b=dOKqCkxitVYH7c0pdupRnZpK5/l2nGavKWDlS+ALoRAPtAH4m3BstqX7xtaGRRb1Wf d3seuANAUNcBh2wORYzQfdJRgZyYXkIXgQWow/2GUL1DlZMFSx0ijE0bHqYlnFjrE4P5 mHXFGWUEOy2hIGqzMfbeyqOFGR7zJNac5++X4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327770; x=1690932570; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B6faQrb3vVDF+1ZA3HcLro9nEgsaXGn2RtvbxbNNEvc=; b=YN6xrwjxb5s2Sviu/cB5IHl0EKDzY64EshVVsmgQr01PN5p1r2QHkENBJE8f+k2JLw gEZNfa7mlvt4o3dnVOmwKKtgCUrtRKdtNDcFfJP2Py+ipm4/r/X6xML1yvExqTm1oxkv 7S4WjlmWAVUssznpA1Hg2WowlxSQWZ11WfZNeg0zFPYse5G4Kg0QsLNdBoh69tEbUSSg puePU9mfRyiK1BkWBbOSGBRWVFpGIkfZC2eETNmF5mnmjVQPvuBydqpIjExN9FKgXwT/ +qnLjTGIy0UI3N0t6fKMCC2tmiggk0dLnNlaAlLbbEQV8AY8BQPMEm8jjs+tDzDUbK9W Ne3Q== X-Gm-Message-State: ABy/qLaEo+J2fMkvAB0qaPwEHCL/ke1A4O78im/UAXcEpNg3ZQ2IY0M8 EZEEjwCJLdXgHwyJVHxACi9MuuZtyj1Qbq5WsBo= X-Google-Smtp-Source: APBJJlEZFnEQQK9J1v8M71kPDaqQ8ZTeYcDEjV+OUIXcmUtYKVYoXTApZUTw4E9efh8aj3vI2iBVkA== X-Received: by 2002:a6b:e80e:0:b0:786:7100:72de with SMTP id f14-20020a6be80e000000b00786710072demr346606ioh.16.1690327770577; Tue, 25 Jul 2023 16:29:30 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:29 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , rcu@vger.kernel.org (open list:READ-COPY UPDATE (RCU)) Cc: rcu@vger.kernel.org Subject: [PATCH 3/5] tree/nocb: Adjust RCU_NOCB_WAKE_* macros from weaker to stronger Date: Tue, 25 Jul 2023 23:29:08 +0000 Message-ID: <20230725232913.2981357-4-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This is needed to make the next patch work correctly as we rely on the strength of the wakeup when comparing deferred-wakeup types across different CPUs. Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/tree.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h index 192536916f9a..0f40a9c2b78d 100644 --- a/kernel/rcu/tree.h +++ b/kernel/rcu/tree.h @@ -288,8 +288,8 @@ struct rcu_data { =20 /* Values for nocb_defer_wakeup field in struct rcu_data. */ #define RCU_NOCB_WAKE_NOT 0 -#define RCU_NOCB_WAKE_BYPASS 1 -#define RCU_NOCB_WAKE_LAZY 2 +#define RCU_NOCB_WAKE_LAZY 1 +#define RCU_NOCB_WAKE_BYPASS 2 #define RCU_NOCB_WAKE 3 #define RCU_NOCB_WAKE_FORCE 4 =20 --=20 2.41.0.487.g6d72f3e995-goog From nobody Sun Feb 8 00:26:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41F6AEB64DD for ; Tue, 25 Jul 2023 23:29:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232355AbjGYX3w (ORCPT ); Tue, 25 Jul 2023 19:29:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231896AbjGYX3o (ORCPT ); Tue, 25 Jul 2023 19:29:44 -0400 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4E981FC9 for ; Tue, 25 Jul 2023 16:29:33 -0700 (PDT) Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-78374596182so307959339f.0 for ; Tue, 25 Jul 2023 16:29:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327772; x=1690932572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0DIiXFGh73Irk8k8jfSTlgn3pzuU9BPg/xYZpIge3SA=; b=devazoCc1ePSllifONKGQCl8V8kDvW5uvDEsIy/u3SDJfI/vz1wlFS0SUxKZdfqrcB NEV4tcQXSLdIR+lnufehU/yii4Le4bCblWZXsvn8b0MbCrfA5CbEFQh/KvLUcVyexJgD bAOucRRZCGzD+cJEPlkY3nJk/aUwJlJqC6WxA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327772; x=1690932572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0DIiXFGh73Irk8k8jfSTlgn3pzuU9BPg/xYZpIge3SA=; b=TqQKXbC151OYB6lHYeEu1+4y3EsVejkemOHuwCC5mXEzcVZsndUBOpeOnzlhmHgDr2 pLpIavvBrnCJY2tIskgtpB3Qeaqr5HbpYPEP2mzrEPPxfrgLq/FjpZ96yNMhYk4LKKN+ xLbCKv5lZBin2wradNaWt/9HneYW09VHTMInglmHITPRtOmXK54R9ajZ1hw+4wrDHjKk IcgW3Me+ORGjgdohGnkjECEkSnD1ETa7T7P4Bri0p90higfek48t9643+W+Y9tgolGH1 fw/164n4f2Z3rHj0Y9PoKNKR6nHnusaldKubwxdISrJCLgyalkTgZmFwKeFrWO4Mkz9P mJTA== X-Gm-Message-State: ABy/qLZizpOKLZvKZTd7YKhvigAb3GpfNo6qQ8On/kSuaVeGpFF+epO5 ggA3xb0eBB/JyM+ctZbywGpJC4SdNmWVVtIoglU= X-Google-Smtp-Source: APBJJlH9/vaRwB5vMSyP0PFSl3tgH3jguGqOapInYGQWm35VNrAOnNszvyRrx5BqXu6IQCKvacXg1g== X-Received: by 2002:a05:6602:257a:b0:783:5fd9:3789 with SMTP id dj26-20020a056602257a00b007835fd93789mr367880iob.1.1690327772055; Tue, 25 Jul 2023 16:29:32 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:31 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , rcu@vger.kernel.org (open list:READ-COPY UPDATE (RCU)) Cc: rcu@vger.kernel.org Subject: [PATCH 4/5] tree/nocb: Improve readability of nocb_gp_wait() Date: Tue, 25 Jul 2023 23:29:09 +0000 Message-ID: <20230725232913.2981357-5-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The nocb_gp_wait() function contains logic to check each rdp's bypass list, flush if needed, and decide on wakeups. This makes the function hard to follow. Split out the bypass checking and flushing into a separate helper nocb_gp_flush_wake(). The new function encapsulates the logic to: - Check if the bypass needs to be flushed - Flush if needed - Return info on wakeups (lazy, bypass or none) nocb_gp_wait() now becomes simpler by calling the helper to handle the bypass flushing and waking logic. This splitting improves the readability and maintainability of the code by encapsulating related logic into a function with a clear purpose. Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/tree_nocb.h | 113 ++++++++++++++++++++++++----------------- 1 file changed, 66 insertions(+), 47 deletions(-) diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index 5598212d1f27..c805825c3f00 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -670,22 +670,68 @@ static void nocb_gp_sleep(struct rcu_data *my_rdp, in= t cpu) trace_rcu_nocb_wake(rcu_state.name, cpu, TPS("EndSleep")); } =20 +/* + * Given an rdp, flush its bypass list if needed and return information ab= out + * if a deferred-wakeup needs to be organized depending on whether things = are + * still in the bypass list. Also tell caller if the list was flushed and = if it + * is still empty after any flushing. + */ +static int nocb_gp_flush_wake(struct rcu_data *rdp, bool *empty, bool *flu= sh) +{ + long bypass_ncbs; + long lazy_ncbs; + unsigned long j =3D jiffies; + + trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("Check")); + lockdep_assert_held(&rdp->nocb_lock); + bypass_ncbs =3D rcu_cblist_n_cbs(&rdp->nocb_bypass); + lazy_ncbs =3D READ_ONCE(rdp->lazy_len); + + *flush =3D false; + *empty =3D false; + if (bypass_ncbs && (lazy_ncbs =3D=3D bypass_ncbs) && + (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + jiffies_till_flush= ) || + bypass_ncbs > 2 * qhimark)) { + *flush =3D true; + } else if (bypass_ncbs && (lazy_ncbs !=3D bypass_ncbs) && + (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + 1) || + bypass_ncbs > 2 * qhimark)) { + *flush =3D true; + } else if (!bypass_ncbs && rcu_segcblist_empty(&rdp->cblist)) { + *empty =3D true; + return RCU_NOCB_WAKE_NOT; + } + + if (*flush) { + // Bypass full or old, so flush it. + (void)rcu_nocb_try_flush_bypass(rdp, j); + bypass_ncbs =3D rcu_cblist_n_cbs(&rdp->nocb_bypass); + lazy_ncbs =3D READ_ONCE(rdp->lazy_len); + } + + if (bypass_ncbs) { + trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, + bypass_ncbs =3D=3D lazy_ncbs ? TPS("Lazy") : TPS("Bypass")); + return (bypass_ncbs =3D=3D lazy_ncbs ? RCU_NOCB_WAKE_LAZY : + RCU_NOCB_WAKE_BYPASS); + } + return RCU_NOCB_WAKE_NOT; +} + /* * No-CBs GP kthreads come here to wait for additional callbacks to show up * or for grace periods to end. */ static void nocb_gp_wait(struct rcu_data *my_rdp) { - bool bypass =3D false; int __maybe_unused cpu =3D my_rdp->cpu; unsigned long cur_gp_seq; unsigned long flags; bool gotcbs =3D false; - unsigned long j =3D jiffies; - bool lazy =3D false; bool needwait_gp =3D false; // This prevents actual uninitialized use. bool needwake; bool needwake_gp; + int defer_wake_type =3D RCU_NOCB_WAKE_NOT; struct rcu_data *rdp, *rdp_toggling =3D NULL; struct rcu_node *rnp; unsigned long wait_gp_seq =3D 0; // Suppress "use uninitialized" warning. @@ -712,44 +758,24 @@ static void nocb_gp_wait(struct rcu_data *my_rdp) * won't be ignored for long. */ list_for_each_entry(rdp, &my_rdp->nocb_head_rdp, nocb_entry_rdp) { - long bypass_ncbs; - bool flush_bypass =3D false; - long lazy_ncbs; + int defer_wake_type_one =3D RCU_NOCB_WAKE_NOT; + bool flushed; + bool empty; =20 - trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("Check")); rcu_nocb_lock_irqsave(rdp, flags); - lockdep_assert_held(&rdp->nocb_lock); - bypass_ncbs =3D rcu_cblist_n_cbs(&rdp->nocb_bypass); - lazy_ncbs =3D READ_ONCE(rdp->lazy_len); + defer_wake_type_one =3D nocb_gp_flush_wake(rdp, &empty, &flushed); =20 - if (bypass_ncbs && (lazy_ncbs =3D=3D bypass_ncbs) && - (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + jiffies_till_flus= h) || - bypass_ncbs > 2 * qhimark)) { - flush_bypass =3D true; - } else if (bypass_ncbs && (lazy_ncbs !=3D bypass_ncbs) && - (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + 1) || - bypass_ncbs > 2 * qhimark)) { - flush_bypass =3D true; - } else if (!bypass_ncbs && rcu_segcblist_empty(&rdp->cblist)) { - rcu_nocb_unlock_irqrestore(rdp, flags); - continue; /* No callbacks here, try next. */ - } + // We may need to do a deferred wakeup later for bypass/lazy + // So note down what we learnt from the rdp. + defer_wake_type =3D max(defer_wake_type_one, defer_wake_type); =20 - if (flush_bypass) { - // Bypass full or old, so flush it. - (void)rcu_nocb_try_flush_bypass(rdp, j); - bypass_ncbs =3D rcu_cblist_n_cbs(&rdp->nocb_bypass); - lazy_ncbs =3D READ_ONCE(rdp->lazy_len); + // Did we make any updates to main cblist? If not, no + // non-deferred wake up to do for this rdp. + if (!flushed && empty) { + rcu_nocb_unlock_irqrestore(rdp, flags); + continue; } =20 - if (bypass_ncbs) { - trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, - bypass_ncbs =3D=3D lazy_ncbs ? TPS("Lazy") : TPS("Bypass")); - if (bypass_ncbs =3D=3D lazy_ncbs) - lazy =3D true; - else - bypass =3D true; - } rnp =3D rdp->mynode; =20 // Advance callbacks if helpful and low contention. @@ -792,23 +818,16 @@ static void nocb_gp_wait(struct rcu_data *my_rdp) rcu_gp_kthread_wake(); } =20 - my_rdp->nocb_gp_bypass =3D bypass; + my_rdp->nocb_gp_bypass =3D (defer_wake_type =3D=3D RCU_NOCB_WAKE_BYPASS); my_rdp->nocb_gp_gp =3D needwait_gp; my_rdp->nocb_gp_seq =3D needwait_gp ? wait_gp_seq : 0; =20 // At least one child with non-empty ->nocb_bypass, so set // timer in order to avoid stranding its callbacks. - if (!rcu_nocb_poll) { - // If bypass list only has lazy CBs. Add a deferred lazy wake up. - if (lazy && !bypass) { - wake_nocb_gp_defer(my_rdp, RCU_NOCB_WAKE_LAZY, - TPS("WakeLazyIsDeferred")); - // Otherwise add a deferred bypass wake up. - } else if (bypass) { - wake_nocb_gp_defer(my_rdp, RCU_NOCB_WAKE_BYPASS, - TPS("WakeBypassIsDeferred")); - } - } + if (!rcu_nocb_poll && defer_wake_type !=3D RCU_NOCB_WAKE_NOT) + wake_nocb_gp_defer(my_rdp, defer_wake_type, + defer_wake_type =3D=3D RCU_NOCB_WAKE_LAZY ? + TPS("WakeLazyIsDeferred") : TPS("WakeBypassIsDeferred")); =20 if (rcu_nocb_poll) { /* Polling, so trace if first poll in the series. */ --=20 2.41.0.487.g6d72f3e995-goog From nobody Sun Feb 8 00:26:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35FA5EB64DD for ; Tue, 25 Jul 2023 23:29:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232199AbjGYX3r (ORCPT ); Tue, 25 Jul 2023 19:29:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231886AbjGYX3n (ORCPT ); Tue, 25 Jul 2023 19:29:43 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 019491FE2 for ; Tue, 25 Jul 2023 16:29:34 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-77a62a84855so261093439f.1 for ; Tue, 25 Jul 2023 16:29:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327773; x=1690932573; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ynoALsv1aca8y8xpjhuiunc/CTMUKjN2qeAE0MoxzWk=; b=T1/qgW80ZW0ruvAexTl3SwPC/VD8efNMinccPq8SOmll1PeYaL7nJcC+rT2y/ICCI/ Zm7HzPisHr2CWscQ8DNq4ATfmZQ4copg8Byy3hFGTtKvq0+ing3UNCdbIzc4z9yskHeM i6XYgDFBYacc8aSIAdToK+pj+H/fS21Ias0jU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327773; x=1690932573; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ynoALsv1aca8y8xpjhuiunc/CTMUKjN2qeAE0MoxzWk=; b=BmhENbGzzU2H1WTFNohYWsDAnjGcgw7mxMbuoCk1FE9kS7HBzjveNVMdbsyMznwrPM mHvGuButh43zvqOT/UY9UUD79WqtRHcEVyxyafzeqv/cLCiM2BILkmuxj1KjqhpIO2hj UrymULVR/8OUkMavZqM2JT4Zoc1SYsK9Dy+rAlrR4OW4AiYn7/IUZanyLYhr/jQxQstq y9ntZJ5LIJUYmLWmy21QQR6DnYd1UhNRYIi23FE5yRsMxLHb/XvCzJDAvRF8kgGQKdGy k4wpSH/zLd+hLlcVYLZFfwovpo836amUy/i3gPzjdmikQyvBoqiDbLseW+xoC9iAhUFw Gxjw== X-Gm-Message-State: ABy/qLarL7LrvOv8dy1yAp7oZiZR9SEn9KBPSpsK9f76m6kwYQ+gRNfJ buR0T71MffvyxPhQnN5zfDABtEKRMPjOon+GHYs= X-Google-Smtp-Source: APBJJlFOYFaGXIDd2DLLQ0HZ+PPI+wBB+NlGxhegixuv5PbL5oqsOrir68iUsUdxnVKRboGwsYrdZA== X-Received: by 2002:a5e:8a4c:0:b0:787:34d:ee9e with SMTP id o12-20020a5e8a4c000000b00787034dee9emr356394iom.7.1690327773744; Tue, 25 Jul 2023 16:29:33 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:32 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, Andy Whitcroft , Joe Perches , Dwaipayan Ray , Lukas Bulwahn Cc: rcu@vger.kernel.org, paulmck@kernel.org, "Joel Fernandes (Google)" Subject: [PATCH 5/5] Revert "checkpatch: Error out if deprecated RCU API used" Date: Tue, 25 Jul 2023 23:29:10 +0000 Message-ID: <20230725232913.2981357-6-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The check for single-argument kfree_rcu() is no longer needed as all users have been converted. This reverts commit 1eacac3255495be7502d406e2ba5444fb5c3607c. Signed-off-by: Joel Fernandes (Google) --- scripts/checkpatch.pl | 9 --------- 1 file changed, 9 deletions(-) diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index a9841148cde2..528f619520eb 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -6422,15 +6422,6 @@ sub process { } } =20 -# check for soon-to-be-deprecated single-argument k[v]free_rcu() API - if ($line =3D~ /\bk[v]?free_rcu\s*\([^(]+\)/) { - if ($line =3D~ /\bk[v]?free_rcu\s*\([^,]+\)/) { - ERROR("DEPRECATED_API", - "Single-argument k[v]free_rcu() API is deprecated, please pass r= cu_head object or call k[v]free_rcu_mightsleep()." . $herecurr); - } - } - - # check for unnecessary "Out of Memory" messages if ($line =3D~ /^\+.*\b$logFunctions\s*\(/ && $prevline =3D~ /^[ \+]\s*if\s*\(\s*(\!\s*|NULL\s*=3D=3D\s*)?($Lval)(= \s*=3D=3D\s*NULL\s*)?\s*\)/ && --=20 2.41.0.487.g6d72f3e995-goog