From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 221CB1F12F8; Sat, 3 Jan 2026 00:23:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399838; cv=fail; b=QeH6qJNDQigkniRPTB0do6l2tBIVt9spMKV4dLlw1MDTosHkPLraXi8LLno7cZsOnyOkJnhxgyK70Ln6y2KOu/W7aAu3tGJxQXXKJ8SBCa/iALA6tMWdWGmDNJWTravnZXR7j88XOQqygVUC3GKYoQjof78t0RVgyKxIYFTkC60= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399838; c=relaxed/simple; bh=ExcAwuJ2csLNXiS13QHqPi/N+MxLxiKs+x9M5EunpfA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=Dt8Teg67CxJymC6oVXy2LcOXsgTOwHfbldrRBJD0q9v1p9lpby7Gnj1BQ7xtehGyvX/bAH+WUgfkSm3IvOLeolzKyiHvNr16fklq9/eihYZsCVrbBDAFBjXt85u/W1RtWJPW/+x0U6bNpktG1Qb92fIaF0kEBqMR42f3wl7xlnk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=aYOqufjb; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="aYOqufjb" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Ebb5JgUACX4+tXuPtZZtEsZekF3bFSszWtCZAqTesXD12vCQ0hCIw29g8edjcyrauv5NKdgnrCzGbp0/6mblQevahS0E26x/23gPSq602L/V97F0hPaZlXtjDgWFkhkQElaDbHOxrt06LVpsdrQRRCnwVqZzlNncgZUF1WtEqavD55XCKvQkrCLTwn7hnHsaUKssFsBzs9mqmURdvX9jp3+Nnat9tDrs1zu2xesyzhKXKVWAgq0iQDLHv0axXcFoGiKKdLTJ1rA2VBTg3DZGYQyogZd0X1fQjxmGk+o73ZnPnGCBcuIR4ZvU4c/Onjinr/1OwDfZxGAq2gZ8yu7TCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=90vscRQZ/II7DHa0WnbCqijbVsbgy9+W9TxBcfWWipc=; b=tCd/Z0ykjLHhuXM1qwZJ87uz4Og9vhR3ulzQluqry1OxgOZubpdzmSItc5YUcYkPy4+8+HIIbCumfHwEjUl4d9YIBsWe50UWM/9y6Wt/EkYvauIZcA6nF6rn3fSTqb2/8HIg8oKUGp5tZUQCJx9f2MjanRf1Wq5UKgOUIWd74NbeZCWs3BXhyOx8g3enzNvAFzMFFa9vcDL+ezUDg0tVAcxCz4GmKJ6h5HSmhJEIn7R77vmcwMi45OiDo/wyTAljCeiazeqkEDaTaaAmcugDWtW+bGCSDN+JJx1SVHYfUGnWAtazRvz8s2//QrytYlthBxTdNOxDB6cTn+6Prki8fw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=90vscRQZ/II7DHa0WnbCqijbVsbgy9+W9TxBcfWWipc=; b=aYOqufjbcwwWel7RjvArXlTwM7mZwVU7D5l9gM5ypgKdV6dVGp/kxGZ9HahECJr7Ye0p6r+jqeV4j4WlPYzkadYOBSRbysKz9QTRBXoT5mjpo7Vtg/0HQcO23Q2WU+1Nsu0MFOFycJ44xTKChN6Vg6zKBmQ7FGdt4JAIzreshyzIu6eObzg0LL72/ciAcENScDFbaESASUbA9HKs6gp+BFXgPHQrIsy7+gGGLivy0o0H832hPJqzi5Bqd20uvYrQ1uPpziqGL8fOm98wBoWhvZxGoePQpvW0m8Y7Q2jBLjLOUO54r5NFwa74TDNww/X/rmgYfsVIgwowTlDx+5bXyQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:23:53 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:23:52 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 01/14] rcu: Add WARN_ON_ONCE for blocked flag invariant in exit_rcu() Date: Fri, 2 Jan 2026 19:23:30 -0500 Message-Id: <20260103002343.6599-2-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BL1PR13CA0178.namprd13.prod.outlook.com (2603:10b6:208:2bd::33) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 2869a382-d6cf-4b0b-eb0f-08de4a5e5fd4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ElHraaGc1cCgAbcaEuTjkqzghLX/c5Hu0I1EbXNnHNmhy66Ml2bqjiMYhihU?= =?us-ascii?Q?2UczMflupsDzcEXvRnIeOOMllFStPizhVKg5BrMlUsmHaOQnc/D1ATIChNBQ?= =?us-ascii?Q?8EdxSeyGG3DG0bnd8wSxt8KVp5jYdcjKAKqX2N6m2K0LMWC4+3+AOkdhOsud?= =?us-ascii?Q?qgwnBaQFzXLnHNxZYfATPMXqxkcU2muzqMl9ipwIW6B4p/ZYU2bSn+azB4Ru?= =?us-ascii?Q?nT1JF8W8IA4D71K6tjihk4z2AIrwwNtgJFRHBQVDvzQcS+vCZsdVUsahCOwN?= =?us-ascii?Q?lzCtAI6xWn1dWMxbUIbomjOKai7F+D5c8WkswssgwZmBLTlUb5olKD7jZBhy?= =?us-ascii?Q?EL2iH8aj96x2oaByRLuM8hpR0qJDfDnb9Y/YZZoHrgLVyGAwphbOa1VCaH6H?= =?us-ascii?Q?P/pi9xTDtz+wdeIb0U5btNnOdUQuoUra/YRjQ9irAFdBY5rkazpFOVlUQy/n?= =?us-ascii?Q?h6CShsigJYoPMiEYPVBHnC+/e1tEThFBrJkxNTfZqj4pVW7QXyOJIs4qExe5?= =?us-ascii?Q?zbCHTI38Hxyr9I01udyw9bpidDlomDObJvougmYU4cMBtKtTx9Qu51ouA9f2?= =?us-ascii?Q?Jjo8J5HM9mdZ8iq3O+vCD0okm6OKGqbrUDZlueXL7UI4MG3biIPM6KFW19Jy?= =?us-ascii?Q?urCj8U6aqOuAAolFC1HrAPIh0kLi39EahGFR/oeA4j35aOpcgIZ05lWIWvAJ?= =?us-ascii?Q?XAdK2wA9dQ7jD0Z89oSBTFKnxfgvaJMMi9SzHu1PhWvYJTfk0xRrola2Dtax?= =?us-ascii?Q?6HTvCWtTmWpNpb1fcBAaP5PAfxjSVWZd4Zc88zLEE14sdX+F0iz5RTsxPe8V?= =?us-ascii?Q?riDrsMB0m1/4M+vn6h/8oxoVFmLHUjZep4dGXNl54sGUDpk5RBm0UCVnq+lS?= =?us-ascii?Q?NwlN8ybr6U9G+pxFoE3bLeLOuiSefeN+o1zm3lQDuMge7lkl/YSo999OYcun?= =?us-ascii?Q?MjBis/n5ac1uRHypOg1FIx4odgZAZPz42g2018RHcHaVj4AJOFK2ma5bZ5Hi?= =?us-ascii?Q?zuJpqmic6ptm7BsuCfLWgymM9wtynTtYqaCzU7Vbrr7RspioQk8ahK82MiPO?= =?us-ascii?Q?vpScLPgE/fNM0UJXBEbD/wsbZN3N7bH8K8D/RvQbPHGFsPPje06I5TqX/CGX?= =?us-ascii?Q?YKHcwqZuyUECiS/TT4hphIBCh4hpHuVK5erzH1rE2qbAvDGWiomBB/gjdYrR?= =?us-ascii?Q?AO/mb9dS816Mtf5YQw09YYTiim4/RYGhQJf64GHrFYE27LqDc8IXTeClEKDT?= =?us-ascii?Q?AXWIpysi9ajmYKcgxrdoy5F0cNnWcLMhhj7clz6jvvJY/HU3lcypbn1RobHO?= =?us-ascii?Q?7+j9mH8FLQPFAQF9CWjSl819stAfBCtrzA0zVcR9E8vEObtt0CMy+GZvOsGn?= =?us-ascii?Q?wc81NjA/rYwR/VfqRq5neX07H46cQczlAWIaJNb46sSfeDGRwcGJLz4BOM9E?= =?us-ascii?Q?6TFUyXsGQ1WQAXQyIb9+ZYFBuBSoSABH?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?QsjgvLbl4bLv87AjjKOq8LRAlC2gVgYbk5GU1a+e2iWk44sguIerCF2zzycj?= =?us-ascii?Q?pseWfyfo0e6NZGbz+tWcZ4OhbhxlWNgkQoihNlY4cHYtbNfJ1H9D+m03S/wp?= =?us-ascii?Q?lTQbwS3IuB8SEI9mdhCc1kQXqL3BueoQT0cCbZMs5upIq3ZenbIs2rXi/qFc?= =?us-ascii?Q?tERIlSrPvqY9n3bTFIazWljf7WtKojeT5sk3hJeKcr6eIEOE7htDhuikovm/?= =?us-ascii?Q?raK2FJ53umzC3TPRESg1G2jQy/YVwFIWQramCQYCaLBzTfu97POTUH8kL/Z+?= =?us-ascii?Q?bvBTl5PUGlhgul3qqe5lLKx1FxPhqZcoAspFldm5Ycmjbe1FLqHULWgYk5n7?= =?us-ascii?Q?arwImGiPW6R5LOysLh8/6rlUuPUq5O83YhOulJBVNlPVqC8H0gip5ZVVUgoE?= =?us-ascii?Q?WZE3tg12DfAenuj9r8MOAxCzrpZkWH3eQb84tSugWdYdiyR1F8MGIdej/f1O?= =?us-ascii?Q?rJ05+IAtaHAPS7qqIEMuJSz+L6Omz6Ps3UgBaBCpceaioZu/ZHAXqGu6yEIa?= =?us-ascii?Q?YpIL6ZsA0Te1nrtKkIBBj9oBrjzKemVlnUMYHiT4UlICdr2ko8VdoC2EQCL7?= =?us-ascii?Q?9LxKEI4UyDm4Iq2zB6YMWujRld4zOwFm3TPrTHW0apoXRmv9xSqCyan/3e+K?= =?us-ascii?Q?6d27y0zTtAhSeKe/myTlPQr7n6uhToqnq4xR00SX0jvjGudM3+c+EULToEsF?= =?us-ascii?Q?AVCRBZzaXhXiGTwG/CTI7Bt2d+E8CXGxXbWGUrDDYeaOCR4V555wtz5DlZKv?= =?us-ascii?Q?qDm2/iOLJfx9dr48HNbn0d3GTEj8uppxZQRiOpwiyqf1S2bytHsW9MdIulMn?= =?us-ascii?Q?xj/0JetIfMuQrvJggPFdcXAU+rpnd299i4keZedHP2+K/dvqmq3cmiT2ECPy?= =?us-ascii?Q?xRvGc98o5MzynTsKBJV1NmKTZR5T9qAHBx3sfoSZb+k6rUoPPQP2iBL2RXzg?= =?us-ascii?Q?DuS6mFrfVZC2Gv42HUcUcyIesIoY2OMjyN3ANN3qwNCC5A4BFfDeEb2xSWjZ?= =?us-ascii?Q?Cpc1DiLtgG5fheK6P8N/SkOLsVombWmO3sb6Ah/nslCD2Z7uaN9b0FmCCWLm?= =?us-ascii?Q?rdKIvqkURg0SKgSpHZUWEzuEtHv/PIIN2bgKLuw9AK+MR3wIzXePya37PXOW?= =?us-ascii?Q?xdDkOhPz2lL9esqBntd/8fLvHmBvtq2hDAKS9g9VCC8a0600lmQBAYmAjMNr?= =?us-ascii?Q?H12QOzw6L/fzBljbGEnt9jm0IiNxZx6SbixqVzhum885Pa9vNnhQdbQAwG5A?= =?us-ascii?Q?Rfm6zTNUNDE5C22i7yiaYBIz9NKm3DfD8jA3JWxq3EeKu8gx0SXFzVlhwJhU?= =?us-ascii?Q?5G2ejVfuuMl53Mpsck4z+7gnZIgGQTMpMys1Y3Xuk9/xEneoAUJYm2XU+VvP?= =?us-ascii?Q?JGKae0EAPGV7SeO/rCkdwgKtp+oji2rLoHLuUr+eLDArP6xhV8vb0qlRatz8?= =?us-ascii?Q?13n8Dv62k8hyyCecJ+dqxcC+dGeVvKvndr6fC/qNVGMlztZChvvpZQe6+WXp?= =?us-ascii?Q?XAcFLpYT5NZsj4aKBHZoNe9+8hQknr+2HJ+6FROo+aBplXKvBLEPdo7hGatw?= =?us-ascii?Q?f7Ow2awXTE29/lwJ85+Cr4OLSNsZo29RaKWRvYqtdqr2h5qaBRde6QGbiKUg?= =?us-ascii?Q?acW+nHSR5FWJWXgSUqnplgItsa7eh5BrXYFZjxGn61oTG7oVZI9xQ97Ti1Kp?= =?us-ascii?Q?1SVCI24pdHxV7NpYRdaXh677qqUvLK+KRp5j87OM+zAjfDytokW6vlivZ5jj?= =?us-ascii?Q?o52vd41NSw=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2869a382-d6cf-4b0b-eb0f-08de4a5e5fd4 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:23:52.8935 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: H7pgCF5kee7ldy1ptrVtVO0xOe8W12FNwuZMe9KLZgATQQBbiWWHiGmNCh53RDmJQ7rvjG+qmLThL8XoV/5U/Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" If a task is on the rcu_node_entry list, its blocked flag should already be set (it's set before adding to any list in rcu_note_context_switch()). The current code silently re-sets it, which could mask bugs. Add a WARN_ON_ONCE to detect this invariant violation. If this warning ever fires, it indicates a bug where a task was added to a blocked list without properly setting the blocked flag first. Signed-off-by: Joel Fernandes --- kernel/rcu/tree_plugin.h | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index dbe2d02be824..73ba5f4a968d 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -846,6 +846,7 @@ void exit_rcu(void) if (unlikely(!list_empty(¤t->rcu_node_entry))) { rcu_preempt_depth_set(1); barrier(); + WARN_ON_ONCE(!t->rcu_read_unlock_special.b.blocked); WRITE_ONCE(t->rcu_read_unlock_special.b.blocked, true); } else if (unlikely(rcu_preempt_depth())) { rcu_preempt_depth_set(1); --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C646721773F; Sat, 3 Jan 2026 00:23:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399840; cv=fail; b=EugOdLHeMqsfKZaBld0idfGicAXSi+UwpqoqQcEYgX9/RliPGN0lISoKUz32uWWrJaGqG5ynmxNUf93iIeHbvaMOkWbOUdbaSuYuU850VG/dh3x1ocHFEBuXels5sS86eSJZZkj+nyqu0TJALquKSgam4GBEZEy/QdxKkz3WUTQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399840; c=relaxed/simple; bh=cTrfbJ0QLAGFF/eW1Xu8zpiZhFNvS/Vv+hgloiQO7B4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=IbJ8FSAEKOLPZLUokelHg7n+BBLUtFLdbPPsSNxpRlNhbmYgrLFSJubRzD8LT8cbjALHJrNgUvAXWG9aJ9IJgxmfZ/BISJ3TeFXMT94qyltK0Jh1NLWp5kTwQUiXamoykJ1G6MKmDG6uC6/+5TlNO0M6y9lmwkYBPEw0fC/rrb0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=VCOSvJ39; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="VCOSvJ39" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NXft0Zp9Jvjshqfl5TfUS0l5f+OVvpnhTlXWzIclC6xUwV53ZBQ37BI3zCTRxQBZfng/1NPzvs4i+e2zo8VTlxybLQrx1iRGUuhQNwSUE9N8sxvBBBKhPeNFGWDfSfLj0a6sUsgmwXXXdfFkFLI681ZMau+OgQWUpeeAEtPjAVrjlPnYK0KvZagXIUoevwOx5Cb1AdXh+ZRBJydR/L9j2wsOc4Nj4kavXmb1n7q7IQML+/Oc93OLRRJ6Nb7jRN5K+oXKKeZ5e+JOT2zrZekUz1zKtFG/yNuYBqjr+nEBbtSReHrIzKwIQrvZUacW6wIe8ixGj7yNGrVN3rvLgkptFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lkwgaKDVo/gi1rc8+Angb8g13zjXdUqYwSl80TJ6JCE=; b=gj/y8abRHxrkJU/bwy8T61gxpNX4vBoAOd+lU6GlhDXde7P2n41wGAYhGoW9SvRwkhJghzAhSr0gxye5dp51dGjvoDo36c1G0i4VN14epq2aq9XAASfLsduR+RwFJO0DzKVwp9no6/BdMsUn4JDduhdkRtrBCakeKRc9ONN+CvRRssFwxKX1wkvbnUCooQ9SCp0+MqS/1QVKDrPaCblzCBlq/aMmedn+5a9E8QqheGXZJ4f4tmdgyWUBdK0F3OR/qrIVE0F1uf4PxNnGT+zy+aIEH0v8ulTF9hQ4UdPx1DS+TmfER8dN/lwRw1/aaeXbW+Z2zh4KXLSErx142/ubAg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lkwgaKDVo/gi1rc8+Angb8g13zjXdUqYwSl80TJ6JCE=; b=VCOSvJ39yvyZyfVtErYJirOd3L3spRi17drK0FkRqv+nyzKwk5g1hePQYMV0C6vLDQRrSjn//uET2aJzUfK9TNeALinYMydgVcvNKCbwsCkY3tWdHmyMy+nX+xGWxFiPLs3hlzyaGJZG0OemItoFUJhZwroaxIpnq4c6l47a4ZafzhqNWoRaeBkU8MRsbVWyUmoV1hmPVd8Drwhiwcf4PDqUOKo63Qobo7+ApKs+DQrqc2h42p+fYJCY7J1Es9tsGDDi2Mv4PfJT7nquWk1ShFx5lWtcjQRHIeYEqwbBuXyTLzSQORXSXzBRw9eGPA3VyaPA13YfP4Gc+DMPsS6JlA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:23:54 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:23:54 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 02/14] rcu: Add per-CPU blocked task lists for PREEMPT_RCU Date: Fri, 2 Jan 2026 19:23:31 -0500 Message-Id: <20260103002343.6599-3-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR20CA0013.namprd20.prod.outlook.com (2603:10b6:208:e8::26) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 3e23db2c-8ae9-4378-f017-08de4a5e6108 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?rR0cAu/ooNzA8Kq/NumV8fLxsE+ReHbh1BPcVYh6HAJN+qID4SChuIsL0qWL?= =?us-ascii?Q?Hd33c3s2RXazGtYZVA+hU963bYue+kf1tfmYwhIe2ayf2soMQya2oSjovCMJ?= =?us-ascii?Q?wQt6Dt70tr6XZ+TrUtnZj30t4J/dkb67HijmfW05DLFnlTyNULu3vx07roi+?= =?us-ascii?Q?kyB1lj58a5J5TDanq7gkhiIWB7qac63Xeke0Pq1829zGgP7hSJtTaJbBPt1n?= =?us-ascii?Q?u4cbnFImv9J2TtHVcqRsuLkrLKYPQPQmSDkBP3e3Pss9Zz0hW4YmRhYHq6XP?= =?us-ascii?Q?XgvRnIIae3/t/SYDTaM+vlgUBgk/JYj3/JwWgWNa8BZjl0rer6akIbnzeaAV?= =?us-ascii?Q?tI7rmfQ7meGp/Hq6BwAFIX9SFm5yCO4Gsqj5+87NkYqsl+oLLbU9XLPHVPNV?= =?us-ascii?Q?HPRb0XXIiNIeCAUiVsMMzko6SEYawyL1E1opSDyy2ZDdGcENjmzCulX0e+2d?= =?us-ascii?Q?RTgL92ow9J+lJSW/VoZYpbiBWbJE5VWZyzyb8kO6W9/Xn+HKh+6DjACY4IEy?= =?us-ascii?Q?epGE5bLjEboghqWU+hZeOlMB7OMmS/M/CuiQ//CAYqEBR5sWME3T/rVBUlwM?= =?us-ascii?Q?RyZ3+kwyv2zTAF3RbbOJlSxg0d71wV06H/EyCk621kXZndS6KYQlgzGVUcZE?= =?us-ascii?Q?6xIFeA55tcb2wnpq5Vil17cC51JcRUvo5wW1BMSXui+y/3eGzkusWXwkZVTt?= =?us-ascii?Q?/gHC77xrkrGYKeh8BGJHXG+E9X8bV0KnwMGXMAiKsrB88OBWVMJgIsQkA9Pc?= =?us-ascii?Q?SqyAuzJYe0kp9vKH2Enayz9As4zxHHgf85CBxC1MXQkAZHLNyaKR+Ye86laK?= =?us-ascii?Q?0fYBY13n4FCsyLdqcqSo/U187v0Q068YC4RQWgEXbBQw4Z7DAKUkBJItC9lB?= =?us-ascii?Q?UlRrwK/dx9Fjkl5Vbluhlk7t2QrAtpDaWY7OGgYQKP5Kqf+3fQffzAB5oqwr?= =?us-ascii?Q?O1dw0ANiUpK5HHkLVRvhpzdF1m4FgNlFmxU/DwhW9B3oxP+I2ySDQu0IVr1k?= =?us-ascii?Q?d6icH9wiGBopfKLM4BC/fC3zfwHDNjXrTti6OsHDg10QoRN5UGh/70ePgs0Q?= =?us-ascii?Q?8dKGHrXZ0z2M0OQe3rMvHPGeiTKSYvBSzGhMtQ/5YHoJ3JPiNa/fmJZ7M+1X?= =?us-ascii?Q?kyWv6OboSenR04tyX5OvRwTTkmg72XaCNY16yU6AqEGOR54XXS1G6tI2ZoPS?= =?us-ascii?Q?9Jn2hjMrggvG3IekjIp2j0mAEkHT46VnbjXC6hF+rpTIXms5ipydH10CFqFt?= =?us-ascii?Q?O4ea15IhREfiv+/u7LFgYi8sTVJAknRC1FA/wmwMUaI/Se8vQkXam+OJL3r6?= =?us-ascii?Q?4CTL/b3ozTnDcMhKimd08PVdPOcuhyspiB/NKL4i4Jb7gTKj81AtO87FPbLO?= =?us-ascii?Q?rA5ZJd6AFKJwHorGdDE/qRJuwO9gBlW2S6mYA/mzfw8qFQDhkUPrUkZRvoAn?= =?us-ascii?Q?fpJ+41/jvksIgN549r61D8A1mUbU4YLw?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?pmkoR9NIFzUskAtB0WQajEGDEBdv4H320a2pXlkww2xXwjaDgrTNtQmEWBpz?= =?us-ascii?Q?ISCiro0ZYOpV+ZP6xToJnjcr9/6675DolW7lf4c1cvT/ESe6WgNdH8u6JiLW?= =?us-ascii?Q?hY6VtmF7NSGRgeSA7vUFItRzV8IXFPsb0qgMxVBJFTb/uVUdyfau5zg+T2kC?= =?us-ascii?Q?FkaZK+aCQ1myAjlVVtaKzXu1hV28SZLZNEVg7lujiEYebU4GnEzapmHnWK89?= =?us-ascii?Q?UGt93DhTP1Jm8kzwML9kJau4LB5XZaAF8wlNxCynrvh5Et5wxRbSNnVYqR0E?= =?us-ascii?Q?u0/OP/T6jOk38oGi9cO8rFe3blLFJ94YpPxoL8vpqNuI40lEK9oVx35xYmuD?= =?us-ascii?Q?YvbCKHjx5KYLf8SMyUQemib1oa3c9latZ2QfKV7tInTjiK3eEG6vtfiXmBei?= =?us-ascii?Q?OnLX8SX78+o0curbLB5cJLKQmq235nE2eQTwVbmjdO9TCWqTT2kXOElxzlpC?= =?us-ascii?Q?nyWQf7lXKtRln9otRr0cKHbZHQam3Fobb0DjwhMdaJwLKlP3HNHyrdQd7Jl9?= =?us-ascii?Q?xr79tUJ8C8+0rFjcxt2HDSRTBWfKr71X50VkkDltifnKauZTcZlSlusrJOXY?= =?us-ascii?Q?zdH/p68HXI1VUjotQACjbrLToAjWZZugouwJD8vNoaEC/iW66KccqKbKQbzk?= =?us-ascii?Q?VSOuOmOu4J1ozM1aHGWrcw2eU9s/H71jr7J/lKlTbq9lyFiSgTNcjCfrTfcZ?= =?us-ascii?Q?Vpsnn82dCQrvwO4qDGyp9SlXaCWZrNSnN8xWJb1EP4LoFqM/gCw1fp0UUoUb?= =?us-ascii?Q?WTxZdOP/nJ5jcs09TxENWIz2TuhbHwkIeN/qkyvY5LpsKU4aFr9U+x6TOucK?= =?us-ascii?Q?sS7/meahK5oH8pNo4KHqIxhElO5/v0aGYWjHeZOjBfsiGaM9H4jt9Pp9J+EJ?= =?us-ascii?Q?fGNZ4bwHDxJKOAHZxjNoO2SNbYCGRLosI+/sr569BObazVTg2/wHPk9DlYYx?= =?us-ascii?Q?sCwaxyJqRoMA4SgJq4T1vCVdFjCdevh4G8Nd+7uYdXAjq6cYRAVzzc8docB9?= =?us-ascii?Q?evLzdG/Gcf3hoBn8WVACzsTDzvTHK5v6KVp0xEOUcbalIx/+UINP66kba4ZE?= =?us-ascii?Q?axcXnDwK2XiXxsI4s3qK7sUefnrJT3D+wc7Hss+ihQAzxraCWVkT1SuOu5sE?= =?us-ascii?Q?wwzAg18FLixgZ8+fOBCJXvaE0o5bPdE8CZBE+Xf7loYzEzDWzgvA/sxPGy7Y?= =?us-ascii?Q?JrPvDO0h31B/gTBEButG2WWLMfxg1KQ0obD3DzFD7ubDQhbTNRQAd8VrPdvf?= =?us-ascii?Q?l2gF5/eRhe9uvzb4b9LCD1WIXYIjYss2iM9qvqFfPyCo5qhpEc/cgh65WzHT?= =?us-ascii?Q?hyJoiLmCfu4xTZdh5HBVVTRp9rhgiW5i8EdWMNe7mSzls1/iiWtl6THjQJf+?= =?us-ascii?Q?pgd4WCN0EA3mNpXWgTo6Ce3wfy3C6NRO5w5WkpfmiMqOejykw0mQisF1/FG3?= =?us-ascii?Q?A16QfXuDrKrf8cM9EMSotGjfuO11uI9Zry7jqWxzzkK3kRbm0xABiydIMIXy?= =?us-ascii?Q?UEdqKZyy3/gQyTtoZfZo7mZhd1DikwVk1ETY6c/t/Egs1Ju8HjU1PVDMgmdz?= =?us-ascii?Q?IyosGlfmk+/+/HMnNqJydNQMj5XQb25zE+FouxpsQ9jlE1vFlt+iGftQnCv5?= =?us-ascii?Q?MJ1+Oa0pPCB3Gp8PAzBZ5MC6AS4rpZdZPTkAM1Oc6saO9gBMHseMsGcaJTjG?= =?us-ascii?Q?SyLDVPPv1K8GOUZd9ayu2vkXR6VTBfcW3nhrmer+11frwGRKAlMB7Ch7w+f2?= =?us-ascii?Q?vj1lBB/Kmg=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3e23db2c-8ae9-4378-f017-08de4a5e6108 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:23:54.9177 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Xrd6d0bSVOfwDPxx9Oqb8vVMwGKguoitfco+fEawo7B+9JeSblKCNJHys132NrZ21NHOqbHXgMP5iG4MR/ipXw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" Add per-CPU tracking of tasks blocked in RCU read-side critical sections. Each rcu_data gets a blkd_list protected by blkd_lock, mirroring the rcu_node blkd_tasks list at per-CPU granularity. Tasks are added on preemption and removed on rcu_read_unlock. A WARN_ON_ONCE in rcu_gp_init verifies list consistency. Signed-off-by: Joel Fernandes --- include/linux/sched.h | 4 ++++ kernel/fork.c | 4 ++++ kernel/rcu/Kconfig | 12 ++++++++++++ kernel/rcu/tree.c | 32 ++++++++++++++++++++++++++++++++ kernel/rcu/tree.h | 6 ++++++ kernel/rcu/tree_plugin.h | 21 +++++++++++++++++++++ 6 files changed, 79 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index d395f2810fac..90ce501a568e 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -931,6 +931,10 @@ struct task_struct { union rcu_special rcu_read_unlock_special; struct list_head rcu_node_entry; struct rcu_node *rcu_blocked_node; +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + struct list_head rcu_rdp_entry; + int rcu_blocked_cpu; +#endif #endif /* #ifdef CONFIG_PREEMPT_RCU */ =20 #ifdef CONFIG_TASKS_RCU diff --git a/kernel/fork.c b/kernel/fork.c index b1f3915d5f8e..7a5ba2d2c1b5 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1819,6 +1819,10 @@ static inline void rcu_copy_process(struct task_stru= ct *p) p->rcu_read_unlock_special.s =3D 0; p->rcu_blocked_node =3D NULL; INIT_LIST_HEAD(&p->rcu_node_entry); +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + INIT_LIST_HEAD(&p->rcu_rdp_entry); + p->rcu_blocked_cpu =3D -1; +#endif #endif /* #ifdef CONFIG_PREEMPT_RCU */ #ifdef CONFIG_TASKS_RCU p->rcu_tasks_holdout =3D false; diff --git a/kernel/rcu/Kconfig b/kernel/rcu/Kconfig index 4d9b21f69eaa..4bb12f1fed09 100644 --- a/kernel/rcu/Kconfig +++ b/kernel/rcu/Kconfig @@ -248,6 +248,18 @@ config RCU_EXP_KTHREAD =20 Accept the default if unsure. =20 +config RCU_PER_CPU_BLOCKED_LISTS + bool "Use per-CPU blocked task lists in PREEMPT_RCU" + depends on PREEMPT_RCU + default n + help + Enable per-CPU tracking of tasks blocked in RCU read-side + critical sections. This allows to quickly toggle the feature. + Eventually the config will be removed, in favor of always keeping + the optimization enabled. + + Accept the default if unsure. + config RCU_NOCB_CPU bool "Offload RCU callback processing from boot-selected CPUs" depends on TREE_RCU diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 293bbd9ac3f4..e2b6a4579086 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -1809,6 +1809,14 @@ static noinline_for_stack bool rcu_gp_init(void) struct rcu_node *rnp =3D rcu_get_root(); bool start_new_poll; unsigned long old_gp_seq; +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + struct task_struct *t_verify; + int cpu_verify; + int rnp_count; + int rdp_total; + struct rcu_data *rdp_cpu; + struct task_struct *t_rdp; +#endif =20 WRITE_ONCE(rcu_state.gp_activity, jiffies); raw_spin_lock_irq_rcu_node(rnp); @@ -1891,6 +1899,26 @@ static noinline_for_stack bool rcu_gp_init(void) */ arch_spin_lock(&rcu_state.ofl_lock); raw_spin_lock_rcu_node(rnp); +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + /* + * Verify rdp lists consistent with rnp list. Since the unlock + * path removes from rdp before rnp, we can have tasks that are + * on rnp but not on rdp (in the middle of being removed). + * Therefore rnp_count >=3D rdp_total is the expected invariant. + */ + rnp_count =3D 0; + rdp_total =3D 0; + list_for_each_entry(t_verify, &rnp->blkd_tasks, rcu_node_entry) + rnp_count++; + for (cpu_verify =3D rnp->grplo; cpu_verify <=3D rnp->grphi; cpu_verify++= ) { + rdp_cpu =3D per_cpu_ptr(&rcu_data, cpu_verify); + raw_spin_lock(&rdp_cpu->blkd_lock); + list_for_each_entry(t_rdp, &rdp_cpu->blkd_list, rcu_rdp_entry) + rdp_total++; + raw_spin_unlock(&rdp_cpu->blkd_lock); + } + WARN_ON_ONCE(rnp_count < rdp_total); +#endif if (rnp->qsmaskinit =3D=3D rnp->qsmaskinitnext && !rnp->wait_blkd_tasks) { /* Nothing to do on this leaf rcu_node structure. */ @@ -4143,6 +4171,10 @@ rcu_boot_init_percpu_data(int cpu) rdp->rcu_onl_gp_state =3D RCU_GP_CLEANED; rdp->last_sched_clock =3D jiffies; rdp->cpu =3D cpu; +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + raw_spin_lock_init(&rdp->blkd_lock); + INIT_LIST_HEAD(&rdp->blkd_list); +#endif rcu_boot_init_nocb_percpu_data(rdp); } =20 diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h index b8bbe7960cda..13d5649a80fb 100644 --- a/kernel/rcu/tree.h +++ b/kernel/rcu/tree.h @@ -294,6 +294,12 @@ struct rcu_data { =20 long lazy_len; /* Length of buffered lazy callbacks. */ int cpu; + +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + /* 8) Per-CPU blocked task tracking. */ + raw_spinlock_t blkd_lock; /* Protects blkd_list. */ + struct list_head blkd_list; /* Tasks blocked on this CPU. */ +#endif }; =20 /* Values for nocb_defer_wakeup field in struct rcu_data. */ diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 73ba5f4a968d..5d2bde19131a 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -338,6 +338,12 @@ void rcu_note_context_switch(bool preempt) raw_spin_lock_rcu_node(rnp); t->rcu_read_unlock_special.b.blocked =3D true; t->rcu_blocked_node =3D rnp; +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + t->rcu_blocked_cpu =3D rdp->cpu; + raw_spin_lock(&rdp->blkd_lock); + list_add(&t->rcu_rdp_entry, &rdp->blkd_list); + raw_spin_unlock(&rdp->blkd_lock); +#endif =20 /* * Verify the CPU's sanity, trace the preemption, and @@ -485,6 +491,10 @@ rcu_preempt_deferred_qs_irqrestore(struct task_struct = *t, unsigned long flags) struct rcu_data *rdp; struct rcu_node *rnp; union rcu_special special; +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + int blocked_cpu; + struct rcu_data *blocked_rdp; +#endif =20 rdp =3D this_cpu_ptr(&rcu_data); if (rdp->defer_qs_iw_pending =3D=3D DEFER_QS_PENDING) @@ -530,6 +540,17 @@ rcu_preempt_deferred_qs_irqrestore(struct task_struct = *t, unsigned long flags) * to loop. Retain a WARN_ON_ONCE() out of sheer paranoia. */ rnp =3D t->rcu_blocked_node; +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + /* Remove from per-CPU list if task was added to it. */ + blocked_cpu =3D t->rcu_blocked_cpu; + if (blocked_cpu !=3D -1) { + blocked_rdp =3D per_cpu_ptr(&rcu_data, blocked_cpu); + raw_spin_lock(&blocked_rdp->blkd_lock); + list_del_init(&t->rcu_rdp_entry); + t->rcu_blocked_cpu =3D -1; + raw_spin_unlock(&blocked_rdp->blkd_lock); + } +#endif raw_spin_lock_rcu_node(rnp); /* irqs already disabled. */ WARN_ON_ONCE(rnp !=3D t->rcu_blocked_node); WARN_ON_ONCE(!rcu_is_leaf_node(rnp)); --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF4772248A5; Sat, 3 Jan 2026 00:24:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399842; cv=fail; b=rml+8mH2S0k9fkafPyl9rMqCGkP4WzDABnZiYbjK9wglKhyrLhobxHfwCXX9uWrXDM+ASaLexPfIQqvOrePvzh+MX1B2iKqROFsvSTAMvXVQBmWJN1SDKfGt2ZwgeJUThhBuNaJu0jIengmHa0c48IjhNHSuhqH75oLutu4AHhY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399842; c=relaxed/simple; bh=Y4xaspahvaqWA2tMBkQmDy8HemRYJkkNVWZQGRgXeFg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=SIAT2ZzjtMtFsBrGrAEvSdtriHk3lxb3lWxJw4CRdF3CEjGOsyiz9kH/2z3kMEmfj2GAK0P4mpS5eo9Z0GSiee9UksSZc51Ke4uTt7669PsP7msUVuFoYd69Hhei7Zn3lLWH0PnvB0fsqDXgZJzIzVs9EMI3ulfB+RU5nGzql4k= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=qhmEGZwn; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="qhmEGZwn" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bBVYhIxukGZzq1Hf3/YEPT4lCPi3UWgLKWbV2Vd0nrKwS2l1X9wY1LJ+Dlj7JsB/ZTGhVvktnwezuqHCGys1XwphijwyE/MIzmwanfvTnDWGovdAuWfgxwTZ5EF8cP5Bh+jDzlPtiFCMjD3jp8M0yYYGDbe9XkNZ6oIwRtgOd04e6U4P0Bug+OUUoso4Wywa+I/kQG/weMA7hHDN+PFjdRmaalVkrQlT+QHZrJVmZO2aXAdBq13mfXSLaPC0HRtrFg0DMNC4p8vvjRK6pPRrRpyahra3rNqRIqwJNHwJLhLn6WbqPCsmd9zycfZYV8hM1FumRBgrlNl/YPvRZXV9Pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SSorV68PZBpyBabwz73RRZm/aldQEKcxT8Q+1oh4my8=; b=Tv49zEW9eCRBj9MdENhH/b7MbtUI/1oAp20XL2ONNJsLTOOX4QYKBqqAHKO16c1tBPcoqC4x5ZJQEqjp8cnzwpYDrQkX8Ldr6iQEUZiii21Y8mIkNTtzTa1KFzRwvrjI60XyhvjwlMINqsIfxYI4OpE2MUkKAlzPwTDVTSsjSM07YzNGUnrnweWxLdF7+oQQMKgmWM/Gy3eGGc1yrZnvqPsC9aumm0m3v4OdxDCpCtNEEvifIETggVsUemufBQa8FB53G84z0wUMK3sah5WkBO5+IWg9UXAPdU1CZZZlzUqtHQAI0rF91sHWWozHLUj8fQDoYEinbsndx2UlZYNxmw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SSorV68PZBpyBabwz73RRZm/aldQEKcxT8Q+1oh4my8=; b=qhmEGZwnlvnRM5mWA+XZY7V1m/9s3uPtWhT16tcc0BoRyJf8i8q4mh6qJTHtlSmJzpQA340cPynKBNGaZSztRYwsV+kFV3JjsVZLdwh+qQTeZsSFxrb3mCQX4V2p63q6KTHmga3vyAI8mqngXrxI7wwi4vV4qoYDPWCQYsoRUwZFAyWYvXRcnV4deroBD0U3B5yuxrsVcV3JxW+uakJuicwkp43vV4+piAB9nptVScpYhF5pZHtJQNBmHmcVRM0w6+x9uJ3YCQSZd6XIfcpXbuk/RVzMEio/bGOa66F6P2uyB08u4GCWH19cXszvVsc6h+gY8FNwWYl0MMwzY84b4g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:23:57 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:23:57 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 03/14] rcu: Early return during unlock for tasks only on per-CPU blocked list Date: Fri, 2 Jan 2026 19:23:32 -0500 Message-Id: <20260103002343.6599-4-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0P220CA0026.NAMP220.PROD.OUTLOOK.COM (2603:10b6:208:52e::19) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 09cb19bf-562a-47b4-7929-08de4a5e6241 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?i5oAhuliuf2+9iNK55Wwp7tmYcA8BWFKwyDYJq2f+kK1cxRvwdWw5/R9sQod?= =?us-ascii?Q?FQYN3IHtSD6XhwVU2V/eBE81qJXEBH6T3jYCMLaZhvw0VX4Ad8t7srFW0Y6z?= =?us-ascii?Q?eRswFC7ncClSYOr49FrvWcCSYSKppyWdYn7ng9xOPx1IP7sumQymZlRZJS84?= =?us-ascii?Q?fpGilobAXsX7SktIuZk2Eal7NaExcnIKqaFWstXgqua6sd+L+s77hLHuq4To?= =?us-ascii?Q?qWuNUkbrVp6gm04tPIXDzOxA49r4+rgvjYnQc2QeIWKSirgIzBm8p2v+D4ak?= =?us-ascii?Q?stym+vjPNl654/pgir0Q1b3wSjiUe2W8BfjAomL+LB2oNDOBWVDEHs4d/hNo?= =?us-ascii?Q?IGdyPoxa1ivhAXxhwffP4IA8Ps/h+tYoRlVArHFwUHQ7xTWg/6thlVoP6B0f?= =?us-ascii?Q?Ps2dk738xPW2VZWinuVqSk84sQIsyZvZqZQZH2orIqj2Lben977AzdPDasFA?= =?us-ascii?Q?tt9FGsprP7vFm5veslHzFQHn/HudIpAPx7XUuPSSbIRShTOYBZIq5MBfjQq4?= =?us-ascii?Q?82R2JaOiNIO8gimolpEJzUFUbg1JRqqbHSzO81yZ1SKix7in8MHPX/EI5c5E?= =?us-ascii?Q?+xUw7xpYc2Mf4k2YYJ2xDx7KWcw+O4ezumgJkGTIv6hwNK1GrwyZ0qHoL0+4?= =?us-ascii?Q?Ka+fSkbZz2dnb9WngNKkvTletQAt6gGSM0aTIPrzLWSNvQI/iA4wLzvxiM12?= =?us-ascii?Q?puMV38gQ+d8uzw3/Ks1b5jcDBoWXYOoRnZSBZpxF9baXA9MyiSeCSUd4Py7q?= =?us-ascii?Q?/n4xrl6WWDC9QHO/qeKWtAEI1w3oKotwZ34Z2PPFG9NwZSkgeO/foNcTxs5N?= =?us-ascii?Q?229duiItdPoBOm6EDSxqqPl8INFMK9zSn+YRWHRMC1AAxuMm8JPXaoPaYrrJ?= =?us-ascii?Q?8IqG8cWGxccGZmXogwPeVKabzsNF/TCakHKFcBkrhKzKimrSTMFWoZogqCMc?= =?us-ascii?Q?XIgvIlQCwTmOhhJrTLu0dNmWuP0rBp1E7u3VHw09Hi0yq7PMQxadmtPYPfTV?= =?us-ascii?Q?y3opL/zjD3he/ZRjnwQOgI+gqKL+hab6CV8uyOJ4cr3pI+2MijErDfiBeAi/?= =?us-ascii?Q?KTNx3owD70WZFCGZjX5z2er36HJzmSFya0fXsjUHLPrnPaAmkirIg8EyDmCQ?= =?us-ascii?Q?9DsnypCE/szGiBqaeE4Eniv9Cacu0sEL/khSCvET1lQlCrnSRsjhRv58SAEP?= =?us-ascii?Q?dn9kQxvZx0ATae3SUPxIjqNfSdLqjfF2vUCnHPo1I3Ne9aFWOeAp9MgWVo/5?= =?us-ascii?Q?6hTN+4xmRDaimzImXDCFs1jcZf2VxPI2Hm53T0/eeJVBCR0knzXQW7vki78d?= =?us-ascii?Q?0IIDgtCwz9LZbFqv9F289eRNZNmzp7UkfwVfzV6hh8+lLQfauG9uttBMWZI0?= =?us-ascii?Q?0gf970iFhZ2LKC0OARgiPA/zDfbGZiSyWVAv6wdh78FPVGvBiEZZ2DPwdAAA?= =?us-ascii?Q?6/j2SSr9zrqkdr6VpN5Ca9ARe0L5iHmN?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?LzF1GHO4RLRQD20oTsstLzBi4aQvAxGxYDXUpER6N0Mv54kS5nKYzI9HjCEb?= =?us-ascii?Q?6BPcc/No/pj1XiPyg9gghqe1TZHm5HlaqHkwTeY7X0sFAzKJM35W/bnyzUTZ?= =?us-ascii?Q?y/xRSeZVx9jxhCSwpI/3n8cuJrt6BLym4SchOuwu46tXcC9j8U8V7mC2+GON?= =?us-ascii?Q?4FtXIFHvwwNvhkcvvYlRAJNlsAxlnoP5MKWp42dUGOSXFlJplmE79W80X8lY?= =?us-ascii?Q?wxkLteBGgIt9Qwf07woU25Efz6oA5JNMQdUGMxvqCzQ40dcfuCiM03RIELLn?= =?us-ascii?Q?zPlBY8rkn3gq8eMhJHjx+aE0pc14GlSM8vHtNy1jTha94ggo8kpukk/n43qz?= =?us-ascii?Q?Jd0bVpODlNjBnk+7n+qQQIUVgXLBHm+AIkt9JCctw8cUVYx2yZyR2SrTn95U?= =?us-ascii?Q?jSrFhZBrYIIGxX5hcXozWxUy4Be7aFo9xhLhPApMpOd39M/Xcd+Edyjznv08?= =?us-ascii?Q?Vb26bnq+VZ+t15LYR+w56gSP+aaelH9eoytW/u7OQ9GhB+msmva4L/Dvkyk8?= =?us-ascii?Q?/QAaPTpyDGVPBKL7gdmQP8Qvxcqfchzr6h84RBbpgqtsTubz8J4tEF79q30q?= =?us-ascii?Q?6U8Votw8e2tk/PKpvEOLxpFzJ/xEfRBlsTVCLwC50RDwBbpy7rO/DVP5a+th?= =?us-ascii?Q?jSWhNFFxWZpjrnXQGUTQhqCaoEKVlKoc7SKKjDo2WFroSUES+2HjWWnJO2Bh?= =?us-ascii?Q?Ad4n78xZ46DkXEba52k1h5zxoU720fO1qC4IQFWvyin5OLq7uLgwQ9LhV4it?= =?us-ascii?Q?Bk3To7tjta82VUZQ3LwQmQgUdcJiNUNDHyGABaIqJqwbDQWyk/WT+rLErnco?= =?us-ascii?Q?4rQ5rgvX0lb9AKyMVKF7ToQTYS2r5VGfpOr15vnpUE4jbiXGjf0Afmf6fRj9?= =?us-ascii?Q?78JHipBIY1hV1Ona5MSCGjJown20B7HeWojn0032hM6f017k3Rr8wPCzGkvA?= =?us-ascii?Q?cR2YXUgVKmnLvjvXu2XLJRh/6Su0C70CPk7jZ360b/ey7bcEbAFlfDIeULM7?= =?us-ascii?Q?UniMEih8i8pbfGA9rFz6tyGnrQQ0fVNBzqO/ixbe7Z0yjksDeDG5Q3wz/w2Q?= =?us-ascii?Q?lN0U2SPsyWWJ/FxnRQOHzaxNGKURudXySWkur57I3OG6JCBZ8zmjINIsAzdh?= =?us-ascii?Q?UmEvgtGmdLdYSfFwWvlwoWQd2apc7c6IkmGHw0bOJmGxMvpJlRUIdSUn6xNJ?= =?us-ascii?Q?kDEiXe1V1vpwVwburEO90xhNMG4mu/wMhQnoPlmPthyUXV4WZoe6xLzax1ag?= =?us-ascii?Q?/oyrWyTMdC4d3K/2HDuya3nzF5Rkd0KIT2Dm3gQ4zzNiRbmKikp0UZ5tFH3X?= =?us-ascii?Q?UELdnTM8UACWB5k3REXfUpgygliDXut1ta47MS8QwaYhT0+Uh1DY4tDkmhr6?= =?us-ascii?Q?MYi6/g/eyKySe7ZuzdJ6ADyBhgybTXrT/nB0gU5KvUUvzK/f2vJPGnnhd/dC?= =?us-ascii?Q?3nAZZzk3gIOA5wMIvUEaZusJu5yDMr2TTguRIRRuwL88nG5u9VWjUKTCDVKH?= =?us-ascii?Q?m/GMWThm61ddfZYCPqr23Q6s2HTKPWiIqQ/9u6NR0Z2wZLeag0oOI/i2qbBY?= =?us-ascii?Q?kjvz7JpixE+Ij+JGwXln4Z6Xz9jQpxwc5PPqsijfMkRf82zqG5rTm+QaD3ZF?= =?us-ascii?Q?H0RdJMK4RAT/JmPVRkPArMme1H0Az18S7YY/DeEPUr553xgZIOaBBjcyKDSm?= =?us-ascii?Q?/dI/HmPovL6V/VXFUdf5fE3W08KucP7srGTewzlhtY+cadq32B3vBqEPOS2q?= =?us-ascii?Q?toIRub+uCw=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 09cb19bf-562a-47b4-7929-08de4a5e6241 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:23:57.0359 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9UR3YMT01craYrUUcjQ25/RvdtCbR2ljzzmY8cMhuHp+XKDCN1bYg1tlT8iGv4WRCqYjjyYlqogjD6YOkDpcXQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" Add a check for t->rcu_blocked_node being NULL after removing the task from the per-CPU blocked list. If NULL, the task was only on the per-CPU list and not on the rcu_node's blkd_tasks list, so we can skip all the rnp lock acquisition and quiescent state reporting. Currently this path is not taken since tasks are always added to both lists. This prepares for a future optimization where tasks will initially be added only to the per-CPU list and promoted to the rnp list only when a grace period needs to wait for them. Signed-off-by: Joel Fernandes --- kernel/rcu/tree_plugin.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 5d2bde19131a..ee26e87c72f8 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -549,6 +549,22 @@ rcu_preempt_deferred_qs_irqrestore(struct task_struct = *t, unsigned long flags) list_del_init(&t->rcu_rdp_entry); t->rcu_blocked_cpu =3D -1; raw_spin_unlock(&blocked_rdp->blkd_lock); + /* + * TODO: This should just be "WARN_ON_ONCE(rnp); return;" since after + * the last patches, the task can only be in either the rdp or the rnp + * list, not both. Since blocked_cpu !=3D -1, it is clearly not in the = rnp + * so we activate the benefits of this patchset by removing the task + * from the rdp blocked list and early returning. + */ + if (!rnp) { + /* + * Task was only on per-CPU list, not on rnp list. + * This can happen in future when tasks are added + * only to rdp initially and promoted to rnp later. + */ + local_irq_restore(flags); + return; + } } #endif raw_spin_lock_rcu_node(rnp); /* irqs already disabled. */ --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C289922370A; Sat, 3 Jan 2026 00:24:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399844; cv=fail; b=C/2sm1fslQ63W3G/DKru5A7xgUYo213LRdVIX80lvaIEYY4nQ5CJKQnEtXZ8wHU2rCu6+mBqhrPvZ3nkqoDGmDxXC10k1HfkEtWrv39f9SD/GYkVQlDleBBlH3RhoQ4uPoKeLbDc8ohAMSrHiDmIBdS/mxQcv/uaEIfdgPyTGt0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399844; c=relaxed/simple; bh=TFqodCAeZo1Ogkm0r0Im9Ln16atkSOpInZPpjXXepxI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=TmMLYg58yYOnDyInp0OrYc63JVoJ8oqlGdMYR0tTArnWFBNwYQpPMCxoMWq4sSBgJAstNu0OPonTa09BmSXoSIwDISQIe4EctWlzVxBB3CB/Rsbwmzn4RV6+JqOAHKZ7n5vMs4jxmSPKnk+bx3Xb0byQaqkVnfyKGlzteTW0ACg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=KpoI5gv2; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="KpoI5gv2" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=snUTo+oqEKYRQGD22NifRVVLuu1JVjhUtJYqU0deYZmA8nDLD+xXBnJtUHV6Es8cXth2xcw5GXUl+yNC5YymnkkjBtu5gkTfqAjhJZKp58/kXqEAC0u6wIi2a8fBJoKcWtbarWpIYQJvbXDEkN2aLRpacMWanMY+ybjuXzIFZbSuyc6pycRyVX5ZoBR6gsy+qHxyZbNYjMUbB12Wjp3yvp5KGSQxP4+IjJGGJm0XzC6eqwOXFlGAO+Mui3kwDDRulU1NmMR+Prr0POG7pp/qIoAxkoxImQ883kZipW2iXSqLDZei8YEok+Wn/6CupGch6R4AiRdDEONjVB549qsfTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=e2lo6cA9yYpv3Dg7pG7VtvfShhErAmK5ziF796zGRy4=; b=IkT+zeOy2NrxdnvKxrJwGnqMh5HycbU1vGnMwXdr/iJrYnXWCyfb5m6sqJ2dcYmNVH7FZba9Q3HRl89Ceu8Y8EaY4+UeGDlTPVpfH55ktjAWI3SsrOeEU1Y2CAgTyymLNMLKhS9NeWgVi4liSgvpxIW56pXQ6ZS3/g2CDCFbDrlJkTKdYwV2hznBJOpYdQpVKTLowMpscbWgbRypO/5q2pvUMOA+JJAWNRtMUnOx1ScCV6tam/znAd5NeIUZuvN5a2+5vyjUqbyOxjtKlWOF5DmsIO9tDzIjUsQAEmL8TOBIvfB00NLinndIzll4DfqM0ngshZdIxywJz0WkbX0EjQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e2lo6cA9yYpv3Dg7pG7VtvfShhErAmK5ziF796zGRy4=; b=KpoI5gv26sEm4OeBWoBnXqMrDPjeC7RNzGIv6nTyYQlhN5SxHKnUysikbPXozeUemPBcZ3/L0fqRMlIQzyJNpZV9WecHVMheoprg87O+1ffZHT6W5yGORbkHDsVZNQ5pj9ru8vmnHn572sOxWtWnaU/4d5u+gWYSOhvoY33n0l4UL/66d5ry+41BnlyCr2afDJkcINAp4Y9/dFEtgy5u02j2eurDsENqYBE0JwCwhZuip1utK43XQmLs9ApLXTAUl5jjcsSNKSTr76aE+e6Qmpq9EoaMS5Tr4E5xrrivJHqckg0zPwkGEJxwFfF6EW9tZPqLvaAZxDv4T1oCVZiQEg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:23:58 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:23:58 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 04/14] rcu: Promote blocked tasks from per-CPU to rnp lists Date: Fri, 2 Jan 2026 19:23:33 -0500 Message-Id: <20260103002343.6599-5-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BL1PR13CA0163.namprd13.prod.outlook.com (2603:10b6:208:2bd::18) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: f3d1a058-4b4b-4501-7576-08de4a5e6365 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?gV7PzrWS6Z8CVqJi8S/+74zU5Pt+dWs4U8fzC7+1mtDWUaojhO+No7qHP/Gl?= =?us-ascii?Q?AjSsQkNpORsD7VbF+4gYJLiTMD96DcOT81AZPeTDq/s8BitRg5Nm0an6DYvV?= =?us-ascii?Q?H+olDaxWa6uh6YDF3/LXdvZS7bW4on+pkbWSoNSvZoZCqNH7jK8m7NWcjVC6?= =?us-ascii?Q?nv9KM6qIJb7lirTBrA/UT0ttJiVDA+br7AVLYgH/COPrmOodF7NE5CTrZ+lo?= =?us-ascii?Q?LrIvdq1Ynu2tpVnby+vlEAp9FUyZNIoktjizBRMGfV94lPKsG8H4Nka19fXR?= =?us-ascii?Q?g5rJKWLt2CuRrBm4mT6NwbgmbnTAOiIXyW655GVsApB5IhkaevDL061Itufd?= =?us-ascii?Q?GFa2n3yu+NIrnRqdzGX3vHZq+guAQkaje8o042gN352KAJxTkJQ8OKRkyUM8?= =?us-ascii?Q?VBPwUcnBAHTXfnSFd5GKdUjaFnrHjNyb+Qz+TSAJ+fRuyAmZhq1N3qrT1+wr?= =?us-ascii?Q?ON7X3jvVPHfWo+iho8K5DtNvmrl+BTwf5fqYVOtWu37x6OhVhoDsVYH78vjW?= =?us-ascii?Q?mZJ4pm9p0wjw1iDJCqajmBs0eFJGIMI4wq51OH1nXVkS3wmHQ/hdUKuk1GQr?= =?us-ascii?Q?2+lBtXPRWcaPPCwVSGweKaAHN295TFViaHAM7yJsTSsW7mdEg0wOzdUjhggC?= =?us-ascii?Q?MgACwrOgQOIztRVTrh6wV0TJF0DZVU/0rYrgr6P7pa91V8+epMHtIbImlppM?= =?us-ascii?Q?kdsndAdTl/JiKP4XjOl58c/g34gRjR2swXF7Szbf11L+EbvPs913D8GzBvDZ?= =?us-ascii?Q?LceL8JUq0gQGalYkr1z4x7uCMtALisCL2Tyq+mSJv/j/3LSq34ySmXQtAz8j?= =?us-ascii?Q?F4qweo6a1nJSjp6otSBDz8onPiEnjJ3oikbhDV4L2hFfXQBhNrbcOj/LEqkL?= =?us-ascii?Q?V853cEPclVyTxMQb7st4ZY6N3Hh+Wr57jtb4JyDjetGtWPHuUITgHuNED54w?= =?us-ascii?Q?YG4b0XKtfayT8I5tg4vnQjAO4bvnCDrzaIhIjGgg4IWgoBBPVTzX3TZBx0WP?= =?us-ascii?Q?vbRtfaCkRis+6Jg9CiOfaFBSnIJQHZKid/6Rzetv1tHUTUvDovgEcQpkMefn?= =?us-ascii?Q?I7ywZ2RnFYCwv+QRs3ZiOiH6HLj2WRaX7zUOYz7+Oja8NIb9NuTk2F/G6EZT?= =?us-ascii?Q?8u+u0BsF1MkypvEO/Ccs4eiuPN9IWqPZ3hIVRZTJZioZqlFhVF8beLUUbHFO?= =?us-ascii?Q?fYutRsLCI86B9chFL52HiWVtwFoeB/os3BP8dM7+i/OwJ638uke2YXIIpIbQ?= =?us-ascii?Q?SPmqH7HIWEXuZ0hKY46p5B3ceG3Z6bOsy2Z+gjIgD5Df5JTz+Ng6iufBxTIL?= =?us-ascii?Q?VX7uIWqwOvvHcUYh2nJYwXW98RvnA1mTcKiffGf2v/1G96XRFyxtSD033E/w?= =?us-ascii?Q?UfXwyDCbu8OQKCKKjTe9IMf1Ul1Wp/ZsVeYKe5XmwoZbPbUPQIJXmPBnOyBH?= =?us-ascii?Q?9z0hv3rP8BB8Cx2g3MSFCPhEYK0WKLtj?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?4ZfpUPfivA9rsKZm6Mkx5FN9SLyIfNXYqtnDEpIBDVYJ/BoJB3F4eteLbKrh?= =?us-ascii?Q?jAy5LjKd9ok8m9pUxN28TTverNjtJLH5w/0404YHp24FqFgF6AL0hPy6EY1E?= =?us-ascii?Q?EWirkBgR4AbH2dkv4Da5ss57Y6xmqR8GcrGMC2Ygt83Lb0dN5oJrdrfGH58x?= =?us-ascii?Q?9GxOZrvu+Qpi9vaaYZ0foKKhSHtYhRuUlNQ5g/8FvcCn0M2xarwVFwYv3riv?= =?us-ascii?Q?voxiT47CouFeJFB9rr0hmKssQcWzZcxWKNgompb7wmfRdLzJG/Pi/N2YSDpc?= =?us-ascii?Q?eRRCRhFfFLfZK3bARG4aJAiqXxkBGQfp3/C8kxNHX4mBKYhvqHNDHA8wxtsB?= =?us-ascii?Q?O31FpgolFITboPJE1bYMsSsz96zszE6fhKCV/iWqGyrxvJ+3kcCuNWE744h+?= =?us-ascii?Q?VEC2OnEEsaCDYTbYd2zKnx5W0/zzvZ9lr4IEvBAr2Z/3g9YVlrq4XlH9gc/R?= =?us-ascii?Q?KI3LAWZ4578vWrOtD0sYM8c1XS71/XB71zUGX9PXXMVtQQq+w6Q+iAVlaWyD?= =?us-ascii?Q?iA7kOucUdHRnvch29yrwpPktmJZXEK5zmxdRJ3yGrYdxzel5pN+y0AhP8LT9?= =?us-ascii?Q?O+loc6anrtYR/f/D+j/ZALnNDOCxFUeyeRxn7Ic/dTNjp/F8n67TDYJppKDk?= =?us-ascii?Q?CgRTSKJMi6CnEz4nhXbWKewK3wPfivsKXQUKzHARm4JuG+0BRqVyk84A06Xd?= =?us-ascii?Q?a3nAUxkJ3vb79xk8pt15F8dKv7lyeK8TNVAuJBpjHja2wF5gG56UjDU6o1db?= =?us-ascii?Q?uoXQS97SNgX5ORw7b+Y4N4eb4Pamj8F16PasceRTc3r3sFJfEtyJZRIsygFf?= =?us-ascii?Q?PvpEXs+5sjnBazEqMLb8y3ohsp7qQN1+f0qjtJndy0b56OF91hqzbvYqLqfg?= =?us-ascii?Q?kyf9qAQtN6zwDUHF/UZlQcncf6244lGoSgs49tfKKIwNmLamUlgBmILu/wIL?= =?us-ascii?Q?GsA1TqJMXWqFQExCWVotxiJMfnPyINkE4ANaXbU8HSZPRCW287Wh621mc2Yo?= =?us-ascii?Q?eVxce19hQ3LAaTL4SpJ9/IoLUP4ljN7N6CAAk7G5M5C/f/uVhTZmyY5srNtI?= =?us-ascii?Q?i8prdeGFMrAWRL8RxMjEqjrZmL9DMlg+XSrlqWj4Whh9RjhfMp094SyqwBwQ?= =?us-ascii?Q?RtG1NoeNKgnOBk3t6WYdCXV3vK/K9XumbNTGUV6cNQaK4/7ShUcC/N5V+sBV?= =?us-ascii?Q?BYAPMFN5dnPvF5uPle+BaPK+20o72qv1abLZGs9BcZ9DMQN+y7XJpfUioFh6?= =?us-ascii?Q?0pFfioDydK66aPQf2eUHVUM0Cj4ASAUtoP3E2I+KZUMYD/O/wSd/APX1mUmA?= =?us-ascii?Q?eLO3B40CQuiX14IjX2Wnx+HCU4kCp61LlyDSTOkBBiltr6o/hS5HBqlUymGn?= =?us-ascii?Q?8uaS1PKjtX6bn7O33GIjd7cPWgi72OTdE+d+pqpPiIWlzpYZzcBKQN3BcL6X?= =?us-ascii?Q?8gyis8ulMdRZcqfxmCoaa7lyEg3nYXRhguN77aAdRKUMnFZrd/wyZv9K5dAS?= =?us-ascii?Q?MR3hy3lkhZVL9g8/lN0tw1ocwMYFAk6MZLObSKQBOAPeFg/CZ+rLOAItdoPc?= =?us-ascii?Q?K2nukLdnnM5cJrnMIf64hMslBDmGYiBGnxJ5lDvDuhXgaKvsRVU5r7sNe+tq?= =?us-ascii?Q?yQ7JlEYg+2mUv176xQxZkiyuCYzyDZ3tmFZQOBMnSRVj319QiW+QaK8AYJrB?= =?us-ascii?Q?qzeHp02vzvfjGMzZ2+7xg90yoiPTFCC8+3M6ZFVjx8r7pXus7Dcv/Le6xmM1?= =?us-ascii?Q?WHOqm02wdQ=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f3d1a058-4b4b-4501-7576-08de4a5e6365 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:23:58.8722 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0qDJ65VLJkzUBV34C+oI7ihfUB/wqp3lR5CBp/HUamoBkvrycoDYLP9e1NRkaV9tMdelOv4w6yqwKAjxoIv21g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" Add rcu_promote_blocked_tasks() helper that moves blocked tasks from per-CPU rdp->blkd_list to the rcu_node's blkd_tasks list during grace period initialization. This is a prerequisite for deferring rnp list addition until gp_init. Signed-off-by: Joel Fernandes --- kernel/rcu/tree.c | 2 + kernel/rcu/tree_plugin.h | 80 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index e2b6a4579086..5837e9923642 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -1899,6 +1899,7 @@ static noinline_for_stack bool rcu_gp_init(void) */ arch_spin_lock(&rcu_state.ofl_lock); raw_spin_lock_rcu_node(rnp); + rcu_promote_blocked_tasks(rnp); #ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS /* * Verify rdp lists consistent with rnp list. Since the unlock @@ -1982,6 +1983,7 @@ static noinline_for_stack bool rcu_gp_init(void) rcu_gp_slow(gp_init_delay); raw_spin_lock_irqsave_rcu_node(rnp, flags); rdp =3D this_cpu_ptr(&rcu_data); + rcu_promote_blocked_tasks(rnp); rcu_preempt_check_blocked_tasks(rnp); rnp->qsmask =3D rnp->qsmaskinit; WRITE_ONCE(rnp->gp_seq, rcu_state.gp_seq); diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index ee26e87c72f8..6810f1b72d2a 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -806,6 +806,84 @@ static void rcu_read_unlock_special(struct task_struct= *t) rcu_preempt_deferred_qs_irqrestore(t, flags); } =20 +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS +/* + * Promote blocked tasks from a single CPU's per-CPU list to the rnp list. + * + * If there are no tracked blockers (gp_tasks NULL) and this CPU + * is still blocking the corresponding GP (bit set in qsmask), set + * the pointer to ensure the GP machinery knows about the blocking task. + * This handles late promotion during QS reporting, where tasks may have + * blocked after rcu_gp_init() or sync_exp_reset_tree() ran their scans. + */ +static void rcu_promote_blocked_tasks_rdp(struct rcu_data *rdp, + struct rcu_node *rnp) +{ + struct task_struct *t, *tmp; + + raw_lockdep_assert_held_rcu_node(rnp); + + raw_spin_lock(&rdp->blkd_lock); + list_for_each_entry_safe(t, tmp, &rdp->blkd_list, rcu_rdp_entry) { + /* + * Skip tasks already on rnp list. A non-NULL + * rcu_blocked_node indicates the task was already + * promoted or added directly during blocking. + * TODO: Should be WARN_ON_ONCE() after the last patch? + */ + if (t->rcu_blocked_node !=3D NULL) + continue; + + /* + * Add to rnp list and remove from per-CPU list. We must add to + * TAIL so that the task blocks any ongoing GPs. + */ + list_add_tail(&t->rcu_node_entry, &rnp->blkd_tasks); + t->rcu_blocked_node =3D rnp; + list_del_init(&t->rcu_rdp_entry); + t->rcu_blocked_cpu =3D -1; + + /* + * Set gp_tasks if this is the first blocker and + * this CPU is still blocking the corresponding GP. + */ + if (!rnp->gp_tasks && (rnp->qsmask & rdp->grpmask)) + WRITE_ONCE(rnp->gp_tasks, &t->rcu_node_entry); + } + raw_spin_unlock(&rdp->blkd_lock); +} + +/* + * Promote blocked tasks from per-CPU lists to the rcu_node's blkd_tasks l= ist. + * This is called during grace period initialization to move tasks that we= re + * blocked on per-CPU lists to the rnp list where they will block the new = GP. + * rnp->lock must be held by the caller. + */ +static void rcu_promote_blocked_tasks(struct rcu_node *rnp) +{ + int cpu; + struct rcu_data *rdp_cpu; + + raw_lockdep_assert_held_rcu_node(rnp); + + /* + * Only leaf nodes have per-CPU blocked task lists. + * TODO: Should be WARN_ON_ONCE()? + */ + if (!rcu_is_leaf_node(rnp)) + return; + + for (cpu =3D rnp->grplo; cpu <=3D rnp->grphi; cpu++) { + rdp_cpu =3D per_cpu_ptr(&rcu_data, cpu); + rcu_promote_blocked_tasks_rdp(rdp_cpu, rnp); + } +} +#else /* #ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS */ +static inline void rcu_promote_blocked_tasks_rdp(struct rcu_data *rdp, + struct rcu_node *rnp) { } +static void rcu_promote_blocked_tasks(struct rcu_node *rnp) { } +#endif /* #else #ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS */ + /* * Check that the list of blocked tasks for the newly completed grace * period is in fact empty. It is a serious bug to complete a grace @@ -1139,6 +1217,8 @@ dump_blkd_tasks(struct rcu_node *rnp, int ncheck) =20 static void rcu_preempt_deferred_qs_init(struct rcu_data *rdp) { } =20 +static void rcu_promote_blocked_tasks(struct rcu_node *rnp) { } + #endif /* #else #ifdef CONFIG_PREEMPT_RCU */ =20 /* --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9DFE0209F43; Sat, 3 Jan 2026 00:24:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399849; cv=fail; b=UZoABGr1d6rqNsZ7N/3F7LQtUuTQban/t6dML3QfWlQbtKt2e9YndbYqmR70xoYyylb672jbrjXTPtdMsUJUj7qeJIdYJrcz20pcDpnw/RupRymLZVVuowNyFHHLEzXF8xDMN6NkZlNjMYoDDOiJ1jyH4RatLegQzAHmg0tnYrQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399849; c=relaxed/simple; bh=gDMUMcMoeaJzV9MPVFUWcQsM2xWLYbhjMazIrIlTqVM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=iLuvNIw4HLiz3qdkaHalXnPPTKdijR9dptPOhOz2zryObnEh7Z5Rt0Vn8UmxoMh0UrvXf+M0rgSP7YUl4sLw5+Z91MfnZCIUQsexVfiTFqFYP43vsxra/cH8/Ml5oDv8bUtaRaViC4SQn1widTkQlm8bZ4Te3mLgavRaprGrANE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=K5ccbA6k; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="K5ccbA6k" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Jo31ccyfAY84BcBnDoEo63d/wlHrDYM+Ru3Wa66U9ndTqAblhO6GnKiXCD7dNEVSrEG5I2CGi+8+Y2GhuZw3liF28OmEdbb9oYN0Xk9xvx0CNlBtG6OhWur0hxqbSsXaP7n7oGNvPd1m+KjLhjLs+LlDOhrkrGClz5Vty+Tk63u88eMdLbzdXETW432f0nYAIGuf693HBqRO1ib5LEFkXUKd0E4q9GFcASSY4Qmj4FwDDM8yMzFEx5NqbDsDcZfCvk1V7QAMzoQxUxoX0kDcWE/i5uHfb8mzXrkwwKJnAfnY4dI1cKiCzZtoOCUGVeY6M6jN2CdY3aDZLE5qdYEP8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K0fs3VuuDbaSEzz79VIjVpmfRvpDUT2HcWXnuI7kxr0=; b=jFB0EiM5I1cqifb3l68f0gWb2QBkus1F6I9NynDOcWF0aKJmgfHHigVixzbnOY7HNwuerHKvYhRBQmpnpdLBdSmY1MIg+9kTff82KlJtftwFcJOiMNPnQWK3jZaYM0gaLfnvNBImVMgDyOajxN6BN7VZwRBc0Dfl384+0K3Lmuq+mApTBLplskssOwUD+6qTJTcBbTTGTpKykgdogup/aFAJWci9DBWji0eWWLkn3tWq77PABQlAFoKCepBBAM80BGwTh9Ti8ienMAkztbMBfu/jBUNqPGTAxXpsBrFHgpMgro8gANO1bkUgwTyM/2kDts4Xp1EuCnpUx2Psu9DFuQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K0fs3VuuDbaSEzz79VIjVpmfRvpDUT2HcWXnuI7kxr0=; b=K5ccbA6kiItqWDdB3kZa4BgsoDCr+6PXU4yhVlnVQmGTVtpSRVqVUnw1WWZ492LEMaBOBj5wSYCHhluvewqWdPyuUVg2aCq9EYIhvW7Z5w8p+o2QbHnBFdfFMQkC7DhSVQL1cp34pySw+4Ihz2hAtCVv3FVP4QksoBiM0h5BEyabFuoi27MTeGnUmC1T7mar9tOvDE+9H1yUrJphlyspeNVhCqy9FY1bxt1HQAxzzsiSIJmdkTCd0KDnWgjPVb1KeODwDt11UK171qO5pPLp0h40z8kYlFVmt20bQPijr4YGlTDUtXd1vBeDxCGm6tiRJPLg9PjtS4r3qMKhzKDexQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:00 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:00 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 05/14] rcu: Promote blocked tasks for expedited GPs Date: Fri, 2 Jan 2026 19:23:34 -0500 Message-Id: <20260103002343.6599-6-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BLAPR03CA0004.namprd03.prod.outlook.com (2603:10b6:208:32b::9) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 9b649807-8dba-47b7-c2a8-08de4a5e6482 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Pf9HrEI9BPG0SsUG07rYhzc9DfWk0mUfQaznw9UrS/zqBgnibIrl1Efv9ZmK?= =?us-ascii?Q?S6FDWQwhFGhtBViQ7B+Bh6HrB3GxlhUTk7pS26VpPYkhREJ9Zr7lRGY0XF5Q?= =?us-ascii?Q?apj6xacBPAoJXK2Xg8wCl4iDYUws7bkfUJ64y9phABRLIFZH0QAugMKja0An?= =?us-ascii?Q?OKizaZZDvQ8Vajjy8fHUnrarcQywCgr9gQP/MkA46nQEnIz3n1MyS22vc7Pl?= =?us-ascii?Q?flSf2/+05Q5hwvZLrg4cD+gygt1if9ZDNV6zysa3OmKNHyQR5GjnXFDqt/Qq?= =?us-ascii?Q?P/786VkcH4K5DjEig35yiaTD2t5G5TAo78efqU8Yia5VL6Kzz6cDH3Dhh6WB?= =?us-ascii?Q?QXrMiWkNVTwR0RsVigxm815JEPsgmFMawXo0mVVrAoaLQYEqSOPMjfOOWoXg?= =?us-ascii?Q?EyATOqcwvzVKDsFQ2DyI7b3Lan68iroLTUzbagvJQ732fJ9T/ncunxNal//c?= =?us-ascii?Q?XFIbV2a5roRzk+BfYLAzzqDsN7t1hoA6TYo4qLBOqQvEo5zpjGtJK5UwS4Bl?= =?us-ascii?Q?FpmWGHCrGGXr4ChL7T7eRiF1AEaETGF++6dCY4q2RNGyMdcMqm43eAfEnFvI?= =?us-ascii?Q?W89MdMCZmyWjMQ48R2myLnhH1jOynStdIC5XhFlvdsj2+1NKFgmsb5pRvwoG?= =?us-ascii?Q?oL6e7cdNxSNGRApTsV/FU2EHluA9BK0dGAzqQWFm3q9Jgf8j4V4OiZ/kEvCH?= =?us-ascii?Q?uTL5xqsaZiISRvMfn47lLJEluGcl9XzGsYNT04mrt2sTE+RGqTXBIHQrxkV7?= =?us-ascii?Q?s00etAMHperwQsBhdR9JwABjuKprf9laa7r0gI9Cai51eS31lOjfsMkTLNjB?= =?us-ascii?Q?gQkuJh/4yvh49AkghGqxWp5eTfLd3E8I9+a2h48ovg4f3VAPGLQKK2PFeVic?= =?us-ascii?Q?OcN+Ii0O7OEKFb2QZphjR6t9MU7/qdz89zOz3lcVsAoWJUpwHJgwzXPKlVgN?= =?us-ascii?Q?8hQ+Q+1h/LJWf9T74bmBcSJMwLtzaZx1Y1J773V4rGCM6OxRBBbaWHKaU+vD?= =?us-ascii?Q?+xauVXqsUzo0FT05e3Ipmg1Vq7YDbdRAK/1yGmr07aTq9myjSKfDO27anKDr?= =?us-ascii?Q?QfaDbgoDo4Dr6JjwbaoG8uY+coi2vviRpNYzgmf2/yWx5z/1yKLqQKJEMW8b?= =?us-ascii?Q?62doLCUSHEsVdjVnkrvoIlH0FGMS4u6OSOjDo20HnHARGr3Gcf71qjbcR3tZ?= =?us-ascii?Q?qdJdPlGGlP2oPcuxb5nIM/5XRjD9ZRMYHpEg5WOlos2aC+IX/YmV70uPXtsi?= =?us-ascii?Q?ClDdFIxMDvlxm6x3oz1tZa5tNJR16zAFDNrQtyRBz0H4R9Y6vbgvndVgTucY?= =?us-ascii?Q?+Spfm6/DPMLrL8wmUmpRfdDt0jGp42kSMca7kEookHOkCyprnMJ6GJA2pG8f?= =?us-ascii?Q?10Bn6bOvJJvQSAZvw6+XeDWCeCbuw0+SPFeDsV1Ip9R0rgVHkro0o8UFfsgb?= =?us-ascii?Q?e7yLITooVqrRT3V33pjn0FyO5G9Q/3uz?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?LMgwoo0u0K3OWVO5yCAMwlXttIxSpYwgXqD9BfquZrCwGCxK4++EljUpD+VW?= =?us-ascii?Q?3bG634w266IKsaERufxHOTDyEHmFUpJ6CzK2B9DtkbMbGPkrPMGod1/UYcKJ?= =?us-ascii?Q?4G7Hg6zsELnzZ87FGuoH2VfxckltsI0wRVBHqbgEJloI7maCKWdFHFxQSUuA?= =?us-ascii?Q?54llvrR7lhaAcX+1JfhMhgLDXGbMxWC00gdlYu/Hb2ERvluVQBnJMv5L0MZU?= =?us-ascii?Q?yoUPCfWMYEps/pb7sX+StPv0VyilE8Y/CyNudXBQ8Pn0x1IDyHxIkWFSLzAB?= =?us-ascii?Q?fvuYDSUqhL0WkJDz0nzuASD/PJ8QWbelZceLLX7hUUKePFL4KcR1Z3xezvzk?= =?us-ascii?Q?chFDVVTtMHnSMCMDn9I0sllniYfHUUOQ/8AaxCPk09JoaLmUm5kY5fgpwC2p?= =?us-ascii?Q?ZcIpqX+gHQmsL82X/8S/URy8mjkbvbGbYZ5+w32fSY8D8WKkLGeDmqi+R07e?= =?us-ascii?Q?fDOr5xh+EGD78r3pYW9m4+SYmOxkeTxIEA1jF6kbw/7S4aLMkYxc3AGo/RQH?= =?us-ascii?Q?RYCkLElPidIU9c5qbVtST7z2HprCeuXaD4ddhqKEfQ8/FFPrDbZBqVDUc6Lv?= =?us-ascii?Q?fxKVjDLpOTYvDZTIZMtSBGSXrBdPv+HIanIX0Zg+YB+/0mfqmobGB4B3W6hi?= =?us-ascii?Q?kjcJV6ItqkPWhcIQK62O8tiHtsIQ1g1XaheFdBXDNy2ucJUkxnApeYlkxVrq?= =?us-ascii?Q?4Sc4sqKaR/qGGK2/hnG3PEjCVQGqiS24plmyEf6Oaq4YoKVdwbvaX9dbA7si?= =?us-ascii?Q?/fkcxk2VtRVTaao68zgXeEPVNHmFXFL4XeT8clG333l9BRLJUsrCpPRcTDFs?= =?us-ascii?Q?BnAhJHj2pcp9pvKKLl6pCZpi2L4+81Q8pyB2pkyqHhNmNbbnb8ssojrGk4on?= =?us-ascii?Q?pT7BbO0hzNnXEqsueIkF+YVCKdpQAVtaKUJEQlr3Pt5itOOLeYMTi11GImId?= =?us-ascii?Q?7ApnT2Bg3nAGilBEHQFl/y5xlW4WoWcs2tnPrBoQc97gKxOQV92cGDy5uOsl?= =?us-ascii?Q?s3AG3N19AKvuil5iDCNZ4LQdkUJZX7Hj52hkoj3GuDoL+pzkzFE/JESHfVU1?= =?us-ascii?Q?SDDAiw1CoiVj6pPkZv2r7ELdkw3J+xCqYKe0n6RUGiCDZxPaB2aGZQ7+l2p+?= =?us-ascii?Q?b7CBchTitQL8cUxpf+gy6LPQhgTvXLTqX7xuPx8d/L7EOmNq1XF6aLWZRioP?= =?us-ascii?Q?lW1g6DJOsSCHFhxOlKBIv7DP8pqLhZvDy/R654kGZ/gWS9RWuSo5ESIP4sVI?= =?us-ascii?Q?x6wevmtMYvak6aidsOkNlhK2wVUXfkRzLN5xBwBp5db8cwwSpnW0gxU/WrzZ?= =?us-ascii?Q?SD0Ibx3f3mJNauBfO7MmCIVr02OgOmcYbMdp6oKLxUferfUy2cg/xFvxnkyo?= =?us-ascii?Q?26RdLLDXfo97n0IHbNvJfGPcTjcBLPOlkTNZNC1D8k2ryMOS2j+fI6Vj/1Oj?= =?us-ascii?Q?yxpP4sNmGABkTAtzoMp0WPnvMKTEkmTw3CaM8T4BPDvrNGMKZF42N9T2Nz0T?= =?us-ascii?Q?pk8n+eD1c6K0YRa7agvUkgAKtpAOby0feXWCrOOwLQONCpqTkyL3mxmVIufm?= =?us-ascii?Q?+xSRJZ7M9CLUeUCTPwlPxnNUmte7833+OFqA47/l2U9cT6sFJdcM7qXekOvW?= =?us-ascii?Q?+349RSTE6P6TZj3jCovJ+2FG99yLu3vVlI2atgtZLkQiiuQIxYFFywkzD5Lo?= =?us-ascii?Q?NLURzD4IZKGAgiA0/aaJWek6CT1nF4Kz8VW/nVuXXg4oP1FO7kucr5L8REsw?= =?us-ascii?Q?kzN00b0h5w=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b649807-8dba-47b7-c2a8-08de4a5e6482 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:00.7489 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: E6ALlCReZxjqF/cDCn/yp1u7MzftEZdk4AZvsAUDeb5iYtboOXIYg0sn0odUcxFCFZfXNnZg7h/31BpXonGA/g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" Add a call to rcu_promote_blocked_tasks() in sync_exp_reset_tree() before checking for blocked tasks. This ensures that expedited grace periods properly wait for tasks that were blocked on per-CPU lists before the expedited GP was initiated. Signed-off-by: Joel Fernandes --- kernel/rcu/tree.h | 1 + kernel/rcu/tree_exp.h | 5 +++++ kernel/rcu/tree_plugin.h | 8 +++++--- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h index 13d5649a80fb..b71c6c1de8d3 100644 --- a/kernel/rcu/tree.h +++ b/kernel/rcu/tree.h @@ -501,6 +501,7 @@ static bool rcu_is_callbacks_kthread(struct rcu_data *r= dp); static void rcu_cpu_kthread_setup(unsigned int cpu); static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp); static bool rcu_preempt_has_tasks(struct rcu_node *rnp); +static void rcu_promote_blocked_tasks(struct rcu_node *rnp); static bool rcu_preempt_need_deferred_qs(struct task_struct *t); static void zero_cpu_stall_ticks(struct rcu_data *rdp); static struct swait_queue_head *rcu_nocb_gp_get(struct rcu_node *rnp); diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h index 96c49c56fc14..f6cb0e3147c4 100644 --- a/kernel/rcu/tree_exp.h +++ b/kernel/rcu/tree_exp.h @@ -141,6 +141,11 @@ static void __maybe_unused sync_exp_reset_tree(void) raw_spin_lock_irqsave_rcu_node(rnp, flags); WARN_ON_ONCE(rnp->expmask); WRITE_ONCE(rnp->expmask, rnp->expmaskinit); + /* + * Promote tasks from per-CPU lists before checking blkd_tasks. + * This ensures expedited GPs see tasks blocked. + */ + rcu_promote_blocked_tasks(rnp); /* * Need to wait for any blocked tasks as well. Note that * additional blocking tasks will also block the expedited GP diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 6810f1b72d2a..ad33fdd0efe8 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -810,8 +810,8 @@ static void rcu_read_unlock_special(struct task_struct = *t) /* * Promote blocked tasks from a single CPU's per-CPU list to the rnp list. * - * If there are no tracked blockers (gp_tasks NULL) and this CPU - * is still blocking the corresponding GP (bit set in qsmask), set + * If there are no tracked blockers (gp_tasks/exp_tasks NULL) and this CPU + * is still blocking the corresponding GP (bit set in qsmask/expmask), set * the pointer to ensure the GP machinery knows about the blocking task. * This handles late promotion during QS reporting, where tasks may have * blocked after rcu_gp_init() or sync_exp_reset_tree() ran their scans. @@ -844,11 +844,13 @@ static void rcu_promote_blocked_tasks_rdp(struct rcu_= data *rdp, t->rcu_blocked_cpu =3D -1; =20 /* - * Set gp_tasks if this is the first blocker and + * Set gp_tasks/exp_tasks if this is the first blocker and * this CPU is still blocking the corresponding GP. */ if (!rnp->gp_tasks && (rnp->qsmask & rdp->grpmask)) WRITE_ONCE(rnp->gp_tasks, &t->rcu_node_entry); + if (!rnp->exp_tasks && (rnp->expmask & rdp->grpmask)) + WRITE_ONCE(rnp->exp_tasks, &t->rcu_node_entry); } raw_spin_unlock(&rdp->blkd_lock); } --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96F541F12F8; Sat, 3 Jan 2026 00:24:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399851; cv=fail; b=K7mIo9MaGEVWPTbr4kTisGsa89ZPyg22Laz7L1uu/2X1av8j/7Uy2EHeT3TuJKuTNC4GDI5fQ3DMf954tXmBFyPTbAajAQteiQtMxT8NB3E3wQY8sHtPHJ3Dy4wugSLSIGKrNOpMDprmx2xsBdL3cGcF6iZ7xuQLXLsD1ajPo0E= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399851; c=relaxed/simple; bh=XiUGzi8CeJ+7CbRjkqb3vbv0g6LJH1fXTyVWgiZbLyg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=lq9e6j3MVfQ2V3wc8XRehxcct6uKhHxB9Gdlp1jaE9tw0cyvTgU9RdU1ueF4XfQwHSOFBD2lAHwbeIpoGVaf6/5TRm1J8V/2YwVeieCQdUoOSnf1YmGPsiLHfRzdvfhIYyBC/3l5G9jV8EQQxXq4NBMfjL0pse+0FgnLjZV1edk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=R3Z9lTvz; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="R3Z9lTvz" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mytgmENNDfQaino8bYNEEiqWVogSZFvLWXpvIon1HI6s2oQbM9SgGXT9dK2k+8WHaIrgU3HnFmrmJ6V2EgJryyakLmJ14FAoD1ZUMKDGMzZ8GX7O77nWqO9/EjR4dPWGTdGqqALlunF8LlCoOh67GAf3Iss6xLB/3lOGo8AcRKMnrf4X9q0UDeEdPt4u4805XJp7Z7beflVJYeO5cXUDqUURZlvDiuU4DLfQm03IcAEqAwZTpAE7J1K/qTD+M95o69n0937hVtEjRz8YMy9cxwsrfKAK7wkmuOw78ynzsY+xxre99s8XPrB6HVSD+z8fM/gUP+5vLvBYaq760aIYtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ED3V/0n1tDKypUfjw0CgWkQqYcApQ0qmipVTYedxtiQ=; b=LDSQVxVPPckfbDHS7GivaBOHW4w7y1cWIYVwpPylSe4S/mhTFfRknO//K7Pb+ZAZvrfuDMnKroLqNLIBJ2Cy1Un4HguwyvTSHy+a47cHWjwEixPkOmMaRYMYDGw+OrXk9pNWddOpzNJtrf674oK2aAo0MHqoDMTVa9Tljwq4o5X2fKuOTeRiC9cVuAzZftxXcsA/lDi7Ulo7T1+sxpsHqgWJ1OGZHsUtoUwfQpvV2axFTbU+0ASh34iRrR0g6MgD7cqoHrvsQVqgHjEHHbfUunJeLm8sJHR8B/bb+iEMooVMCjpkXTv1be8OW5lBuRLja+/92KJ7sFfkMyXW5nOozw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ED3V/0n1tDKypUfjw0CgWkQqYcApQ0qmipVTYedxtiQ=; b=R3Z9lTvz0hX1TxrviRev9z3UKrtguZHAp9uSF3l5L4K2vy43qlVtIriQ6dg913vY9fVXD0EDYFKE9IdQo1bG20Af49fFWbubEdidOacHs7kR8mFRkwOrZXt7JVvWxJKLMaOTrgyDEhU23bX4tyqvWceHsHKEe+nRSlS3VSAWi6x0NJxb5tIl6Sbge+MHqn69Rlp5jiAEDOIgqq9/UQKJDNLuoW4pRAUhg0snhOQpU1TETBHXOAagLBCKHxWP0GFqKokS5yJ/KFfINZJqXeOxNmsBYzWwbW7+TUki1hbUo7Vkdf8lnVhI+8XMe9XKL6YkzwEnPHzDR6DEbTAV52tEXg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:02 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:02 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 06/14] rcu: Promote per-CPU blocked tasks before checking for blocked readers Date: Fri, 2 Jan 2026 19:23:35 -0500 Message-Id: <20260103002343.6599-7-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BLAPR03CA0140.namprd03.prod.outlook.com (2603:10b6:208:32e::25) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: cfed9769-b8ed-49a0-3503-08de4a5e65aa X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?5YP9LNZ+wnd9Z0Egqyoro8BTiuvqSaFIVqvmLcC7yUWccADXomPEMIFEif9/?= =?us-ascii?Q?UMf5aptxJV4ct2GWiu6NodeoExkjzQ3rNp1lbsn5mh9OwtvtwTvvGYUJfj9N?= =?us-ascii?Q?HV/cGjLAGWQmhnfHX8C/NtUcWg6a9eEBSEVXpSYlb0Tjx6VVtJZKSwSLW6me?= =?us-ascii?Q?wpeH9t5MkIMm2gtVmy3nGRYghyraXsecj1lr+6kPLIv+JkDLURDHx5ki/OlC?= =?us-ascii?Q?5sz5yuAf3dCLPK3atlB4ds3KqFrwOCaAcDtIJnqA8Yz1z5RdVsqezK/LsEZ1?= =?us-ascii?Q?DaUA69IYrEL/ywmNv4Ru0phxax/JmSliZLFwbzpLHSO2Zsj6LdkBV6gwvunZ?= =?us-ascii?Q?nMHQjNtCbkcaVkOE5AQQvdBufT56tlHsdnl7Kmfs5R7uWgOK9j1PNpReMN2O?= =?us-ascii?Q?GllVNetOc+a5t9Tpt0q+P+RAT4RLfbDEo6aso044SLOI0Wbd5F296vPn04O5?= =?us-ascii?Q?b9EKfQ3yXdUkl9PR8kJ9M9PpAKlpr+aOYfWHUGuofFPh3mUFJNc5X5N0a4GR?= =?us-ascii?Q?P2srm1+zRyYKxWrfl8uIf/OXk15i8ALrWo3YjviPeN1BaYv8WJs5COtAbKOY?= =?us-ascii?Q?7vZ6H4dGWFT5nx0FGsvoDA9e8OadnWeUoC7PLWG0dNJmDtox3POGKn28XIpB?= =?us-ascii?Q?4mClsEsMQT5AuAgN/K+i6CsX+2vOccEFsn4Df8B3F+cR6tJVYOD2n/FeCvOR?= =?us-ascii?Q?7Ga5/8mPKaOpoW6GUKzt9ai8pCv8rbFuAlTIDdHJPFHHjwEZymH2AxKOIKIj?= =?us-ascii?Q?dfieI9Zsz3NAXjOqpxcNLpYKMuElis498BNEtzWQ5z9fjf7uprXOT0Pv3QO2?= =?us-ascii?Q?liHhcLnEkb+Bl7WkfNQM0g2l/ExdrR6vjsaTFQukx6p9B3tRmpPYgQk8uJdR?= =?us-ascii?Q?26IjKwD1f9Bs8J0hVA7HniICmOLAme1aEDQIhvTU+tukp87ehE5qAoBntMaO?= =?us-ascii?Q?9R7nLzNqa7mCbX4XDSElqpnQujnP2NEZm6lOoD9AuEOsOOT20bJAAzNb2FQ6?= =?us-ascii?Q?CA62Kw7dWi8cNP36Az3ZH4SX0m4AxBbXPxFYqh3VejDp3Aa817uiFN9YoFDx?= =?us-ascii?Q?v5dAAKAFVKzCIV9AM07YtqUstMjsCGgW9ahA8lTt5wvKMpEUR164F6f9G4pO?= =?us-ascii?Q?aMrmliXdkoeDpVHM1bbRmS98/jQvjmyH1jWYo3nWoytnPuxpIVhe1SqBum4k?= =?us-ascii?Q?O883GaUJJbrO2JK+7byTvHzIcjzKDpNtc+BE4Jgk3KVmjxTD1NZs2JQLe/BX?= =?us-ascii?Q?ZkJOe+mzmxlP9GK3GQ4iPi8dbiWO7WBIhRyUFzw+Y/mYRVM/BHJFw+5VgnMP?= =?us-ascii?Q?OnUVVIo/EQTCW1EukpRZln7KAZwoSGi/NIoFQIWo4l42OX5zBl0EC1QpJ5XA?= =?us-ascii?Q?Ess9HvZTI5qTI9Zb892ADS7jDe7dCX1X9rywT2M26NFCKeSITpGfhMyEzRzN?= =?us-ascii?Q?9CvtGXl+XqGBZtGVyD+196Lltba3ibOH?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?NkivzFhnltLxXHVczAMwwnAhDCWB4sh2A+pn2i76naNq09EXg+XDgMEKXUdl?= =?us-ascii?Q?VmhH4fB2I4bDkV63dZBDodkwtfN07+6vm0OUV2OJ6SOZbK/vUt7f3y8ge0NT?= =?us-ascii?Q?boWW2qrKzKJMqPhkjUUGX9ex2Sy7/cZgLuAITBZeahbJaZCw7fcR8MPVDaiq?= =?us-ascii?Q?k6LrTN7XD59/gO57ayDa8Z/Q8iTCI1Irv4znKN0FN48t0koqvinZIH6P3z5j?= =?us-ascii?Q?ZEFLF2AY5wPcJ0HE9fCgcePgdIpU4RcM6BUn16pVZmmajcLwJBZHFqOgnnKc?= =?us-ascii?Q?yxc207fstBQ7rR4MFIyp868e5LtFdiwoEKDQLqofzepThpb+oKa1jALXi1pV?= =?us-ascii?Q?hAaqvBJUiCdOYaPMEQntaTMojeut8dDGuo1X/MbkSFT51IQWdVws7Xc2JZm5?= =?us-ascii?Q?ZOaKyk+INzLG63Fvr7BR9I1kpBbcf/gIg9i5Y4vhCRzF3wvlM1LjkxT5CZx8?= =?us-ascii?Q?EWqrVaXGTzpz3Ay03BDVr4Ay8gyCGb8gMJtTiAii/Pj6TpIefPbpeCPlvxQN?= =?us-ascii?Q?yD/bTbGr6NF5X7izTPzwABsobuNtx1d6YLr3/EgxuLRJv65z1vXwtvRnPU/Q?= =?us-ascii?Q?7LlVtO4MjYoJjciOa+8oT7mzAdxPnhHJGubyKHd0Vs1CHiEZVm0E2LM6fO3n?= =?us-ascii?Q?6ggXWA3T9RQeastnQLFI4nNkv/WZofp09zcOBxyA0684icmSQ2KTE3Viro0P?= =?us-ascii?Q?18c8/HkUrcBat5wFt5ZtNK4EouB7vCC34xK4b9T2gKycdDdD/X/LLrIxRF98?= =?us-ascii?Q?I4U/JZb8YJGcVl1TrY2KNnZiOSzi7AOrGEMK7LYVayz+z4hkIljIg10X0+ma?= =?us-ascii?Q?AUCO8woewDCt6Ba9LYSPgbvSbaRTi5BnX6nuaCIsu9uk2EaMoGK9glxT583I?= =?us-ascii?Q?N0lKZ5RIn5UL4b4DjV8BY9SsRqymgRKzlmnkS4MSkt29T+ftUHM+GkqkuHse?= =?us-ascii?Q?W+jM5zFd4lqsSnkOz/mxlQne2V5N6G5Az6b0UMRPGA1H7KX2Q3eJHKFJzXdX?= =?us-ascii?Q?TPz6ByJZZIazYDkbSCNvamw4lgKbV1fntdhWrL739lqfhTxZIwptze5NPS9p?= =?us-ascii?Q?1xnHF+lHm+KGDLjFyzFY5qVFgwqrFyyvN22I0m0obnUqWLqxGz2qie40s/d5?= =?us-ascii?Q?PRt6UEesUopV+VlvyJcVU6NesUgT30UGByvuxF6ugRkdnHF8fSYlg+eSkN5L?= =?us-ascii?Q?w3gjIjo34Ssz05nxJCFvPbuE3Ko5QF3nxRNSvcjuy4xvD64ysnAoZTDx0g9X?= =?us-ascii?Q?EIdLAOKb+MwIzfn1J4ICdLRfyVHac9E6WAYZHVIUjsUWT1Zj19pa0B+fShUX?= =?us-ascii?Q?2vXfgFCHpuXaawxNQc04OKzzJQsTb/VIXF3jHVeJBAxA8jm2e7qOrF+h36U1?= =?us-ascii?Q?Nv5rFvwAhqaEB70pl+3k/i/oHethTAsR45AdVKGLRD+Zb7e0iivkgbPhPBT0?= =?us-ascii?Q?2xMjDk09Ygnt3dK0uHa7MHrjaorw07IqsJhMbsx1ZtZjlrL1gdordSZX/Gvu?= =?us-ascii?Q?eC2Fesh0TJcbxx19HLdX6kZ1ao6RgGQ5+Nl8lV72GQsdFKBAkDBZUaxJwCuQ?= =?us-ascii?Q?7sHVXLRleKlKIszlmBVbPfiUW7jLmBmXOYHR7hVhYztWgRUrwIb4iSjnfOXr?= =?us-ascii?Q?gwVm5vvlkD2Hedo4RPzBGoQHU6BpogMPDkrpWVbl9CHPTn9nxtiWevQpNO25?= =?us-ascii?Q?XsmRXYBVBeamN9FSKAc5EugiPaAiv+TTFAs53nZ7A0X4curOvyXkdIA0soGX?= =?us-ascii?Q?sv5HR/QBew=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: cfed9769-b8ed-49a0-3503-08de4a5e65aa X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:02.7538 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WNgx9Htrewp5zXiyMIO0dlePNHF0mSoJlxiVjIA1596Ao1es3i2X1c7e5+5AP+otxlIMk10B88HtMRuB2x+e1w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" When CONFIG_RCU_PER_CPU_BLOCKED_LISTS is enabled, tasks that block in RCU read-side critical sections may be placed on per-CPU lists rather than directly on the rcu_node's blkd_tasks list. It is possible that a task can block just after rcu_gp_init()'s promotion scan completes, leaving it only on the per-CPU list while a GP is active. The RCU priority boosting mechanism only looks at rnp->gp_tasks and rnp->exp_tasks, which point into rnp->blkd_tasks. Tasks on per-CPU lists are invisible to the boost kthread and cannot be boosted. Address this by adding a "promote" parameter to rcu_preempt_blocked_readers_cgp(). When promote is true and the caller the function first promotes any tasks from per-CPU blocked lists to the rcu_node's blkd_tasks list before checking if there are blocked readers. This ensures that late-arriving tasks are visible for priority boosting and other operations. Callers that hold the rnp lock pass promote=3Dtrue to get an accurate answer including late arrivals. Lockless callers (GP loop, FQS check) pass promote=3Dfalse for an approximate snapshot (TODO: need to check if we can always just set "promote" to true and remove the parameter). Signed-off-by: Joel Fernandes --- kernel/rcu/tree.c | 14 +++++++------- kernel/rcu/tree.h | 2 +- kernel/rcu/tree_plugin.h | 34 ++++++++++++++++++++++++++++------ kernel/rcu/tree_stall.h | 4 ++-- 4 files changed, 38 insertions(+), 16 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 5837e9923642..f8f43f94adbb 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2034,7 +2034,7 @@ static bool rcu_gp_fqs_check_wake(int *gfp) return true; =20 // The current grace period has completed. - if (!READ_ONCE(rnp->qsmask) && !rcu_preempt_blocked_readers_cgp(rnp)) + if (!READ_ONCE(rnp->qsmask) && !rcu_preempt_blocked_readers_cgp(rnp, fals= e)) return true; =20 return false; @@ -2125,7 +2125,7 @@ static noinline_for_stack void rcu_gp_fqs_loop(void) * the corresponding leaf nodes have passed through their quiescent stat= e. */ if (!READ_ONCE(rnp->qsmask) && - !rcu_preempt_blocked_readers_cgp(rnp)) + !rcu_preempt_blocked_readers_cgp(rnp, false)) break; /* If time for quiescent-state forcing, do it. */ if (!time_after(rcu_state.jiffies_force_qs, jiffies) || @@ -2207,7 +2207,7 @@ static noinline void rcu_gp_cleanup(void) rcu_seq_end(&new_gp_seq); rcu_for_each_node_breadth_first(rnp) { raw_spin_lock_irq_rcu_node(rnp); - if (WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp))) + if (WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp, true))) dump_blkd_tasks(rnp, 10); WARN_ON_ONCE(rnp->qsmask); WRITE_ONCE(rnp->gp_seq, new_gp_seq); @@ -2376,13 +2376,13 @@ static void rcu_report_qs_rnp(unsigned long mask, s= truct rcu_node *rnp, } WARN_ON_ONCE(oldmask); /* Any child must be all zeroed! */ WARN_ON_ONCE(!rcu_is_leaf_node(rnp) && - rcu_preempt_blocked_readers_cgp(rnp)); + rcu_preempt_blocked_readers_cgp(rnp, true)); WRITE_ONCE(rnp->qsmask, rnp->qsmask & ~mask); trace_rcu_quiescent_state_report(rcu_state.name, rnp->gp_seq, mask, rnp->qsmask, rnp->level, rnp->grplo, rnp->grphi, !!rnp->gp_tasks); - if (rnp->qsmask !=3D 0 || rcu_preempt_blocked_readers_cgp(rnp)) { + if (rnp->qsmask !=3D 0 || rcu_preempt_blocked_readers_cgp(rnp, true)) { =20 /* Other bits still set at this level, so done. */ raw_spin_unlock_irqrestore_rcu_node(rnp, flags); @@ -2428,7 +2428,7 @@ rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsig= ned long flags) =20 raw_lockdep_assert_held_rcu_node(rnp); if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_PREEMPT_RCU)) || - WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp)) || + WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp, true)) || rnp->qsmask !=3D 0) { raw_spin_unlock_irqrestore_rcu_node(rnp, flags); return; /* Still need more quiescent states! */ @@ -2763,7 +2763,7 @@ static void force_qs_rnp(int (*f)(struct rcu_data *rd= p)) raw_spin_lock_irqsave_rcu_node(rnp, flags); rcu_state.cbovldnext |=3D !!rnp->cbovldmask; if (rnp->qsmask =3D=3D 0) { - if (rcu_preempt_blocked_readers_cgp(rnp)) { + if (rcu_preempt_blocked_readers_cgp(rnp, true)) { /* * No point in scanning bits because they * are all zero. But we might need to diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h index b71c6c1de8d3..25eb9200e6ef 100644 --- a/kernel/rcu/tree.h +++ b/kernel/rcu/tree.h @@ -486,7 +486,7 @@ static const char *tp_rcu_varname __used __tracepoint_s= tring =3D rcu_name; /* Forward declarations for tree_plugin.h */ static void rcu_bootup_announce(void); static void rcu_qs(void); -static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp); +static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp, bool prom= ote); #ifdef CONFIG_HOTPLUG_CPU static bool rcu_preempt_has_tasks(struct rcu_node *rnp); #endif /* #ifdef CONFIG_HOTPLUG_CPU */ diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index ad33fdd0efe8..6ed3815bb912 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -383,9 +383,28 @@ EXPORT_SYMBOL_GPL(rcu_note_context_switch); * Check for preempted RCU readers blocking the current grace period * for the specified rcu_node structure. If the caller needs a reliable * answer, it must hold the rcu_node's ->lock. + * + * If @promote is true and CONFIG_RCU_PER_CPU_BLOCKED_LISTS is enabled, + * this function first promotes any tasks from per-CPU blocked lists to + * the rcu_node's blkd_tasks list before checking. This ensures that + * late-arriving tasks (blocked after GP init's promotion scan) are + * visible for priority boosting and other operations. When promoting, + * the caller must hold rnp->lock. */ -static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp) +static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp, bool prom= ote) { +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + if (promote && rcu_is_leaf_node(rnp)) { + int cpu; + struct rcu_data *rdp; + + raw_lockdep_assert_held_rcu_node(rnp); + for (cpu =3D rnp->grplo; cpu <=3D rnp->grphi; cpu++) { + rdp =3D per_cpu_ptr(&rcu_data, cpu); + rcu_promote_blocked_tasks_rdp(rdp, rnp); + } + } +#endif return READ_ONCE(rnp->gp_tasks) !=3D NULL; } =20 @@ -570,7 +589,7 @@ rcu_preempt_deferred_qs_irqrestore(struct task_struct *= t, unsigned long flags) raw_spin_lock_rcu_node(rnp); /* irqs already disabled. */ WARN_ON_ONCE(rnp !=3D t->rcu_blocked_node); WARN_ON_ONCE(!rcu_is_leaf_node(rnp)); - empty_norm =3D !rcu_preempt_blocked_readers_cgp(rnp); + empty_norm =3D !rcu_preempt_blocked_readers_cgp(rnp, true); WARN_ON_ONCE(rnp->completedqs =3D=3D rnp->gp_seq && (!empty_norm || rnp->qsmask)); empty_exp =3D sync_rcu_exp_done(rnp); @@ -597,7 +616,7 @@ rcu_preempt_deferred_qs_irqrestore(struct task_struct *= t, unsigned long flags) * so we must take a snapshot of the expedited state. */ empty_exp_now =3D sync_rcu_exp_done(rnp); - if (!empty_norm && !rcu_preempt_blocked_readers_cgp(rnp)) { + if (!empty_norm && !rcu_preempt_blocked_readers_cgp(rnp, true)) { trace_rcu_quiescent_state_report(TPS("preempt_rcu"), rnp->gp_seq, 0, rnp->qsmask, @@ -901,7 +920,7 @@ static void rcu_preempt_check_blocked_tasks(struct rcu_= node *rnp) =20 RCU_LOCKDEP_WARN(preemptible(), "rcu_preempt_check_blocked_tasks() invoke= d with preemption enabled!!!\n"); raw_lockdep_assert_held_rcu_node(rnp); - if (WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp))) + if (WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp, true))) dump_blkd_tasks(rnp, 10); if (rcu_preempt_has_tasks(rnp) && (rnp->qsmaskinit || rnp->wait_blkd_tasks)) { @@ -1127,7 +1146,7 @@ EXPORT_SYMBOL_GPL(rcu_note_context_switch); * Because preemptible RCU does not exist, there are never any preempted * RCU readers. */ -static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp) +static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp, bool prom= ote) { return 0; } @@ -1221,6 +1240,9 @@ static void rcu_preempt_deferred_qs_init(struct rcu_d= ata *rdp) { } =20 static void rcu_promote_blocked_tasks(struct rcu_node *rnp) { } =20 +static void rcu_promote_blocked_tasks_rdp(struct rcu_data *rdp, + struct rcu_node *rnp) { } + #endif /* #else #ifdef CONFIG_PREEMPT_RCU */ =20 /* @@ -1378,7 +1400,7 @@ static void rcu_initiate_boost(struct rcu_node *rnp, = unsigned long flags) { raw_lockdep_assert_held_rcu_node(rnp); if (!rnp->boost_kthread_task || - (!rcu_preempt_blocked_readers_cgp(rnp) && !rnp->exp_tasks)) { + (!rcu_preempt_blocked_readers_cgp(rnp, true) && !rnp->exp_tasks)) { raw_spin_unlock_irqrestore_rcu_node(rnp, flags); return; } diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h index b67532cb8770..5aa65130ab5c 100644 --- a/kernel/rcu/tree_stall.h +++ b/kernel/rcu/tree_stall.h @@ -277,7 +277,7 @@ static void rcu_print_detail_task_stall_rnp(struct rcu_= node *rnp) struct task_struct *t; =20 raw_spin_lock_irqsave_rcu_node(rnp, flags); - if (!rcu_preempt_blocked_readers_cgp(rnp)) { + if (!rcu_preempt_blocked_readers_cgp(rnp, true)) { raw_spin_unlock_irqrestore_rcu_node(rnp, flags); return; } @@ -331,7 +331,7 @@ static int rcu_print_task_stall(struct rcu_node *rnp, u= nsigned long flags) struct task_struct *ts[8]; =20 lockdep_assert_irqs_disabled(); - if (!rcu_preempt_blocked_readers_cgp(rnp)) { + if (!rcu_preempt_blocked_readers_cgp(rnp, true)) { raw_spin_unlock_irqrestore_rcu_node(rnp, flags); return 0; } --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF99D2222C5; Sat, 3 Jan 2026 00:24:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399853; cv=fail; b=HwX5ZmWg7mk42g/U9+eIoGDR1vTieAUWFpw5M/Wox2mxmAI+kgdI8I5++TzeQicWwdIhjKyYuNVhksGHPtL7JiLrMqWDt1uCjUm0AD2c035+97LFYuAtwSKCjXyvH54EibGPEdS1V9Xit4ud+dq0ZXnyel8va/nT2Qk83hrpVLs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399853; c=relaxed/simple; bh=DuF4SS0S7fuJ+G/0CP1smmshLYwknJrtBOCz3NO7BEs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=I7oCATHfg8uTqNVIj9YiOtHagenbW9lzPcTAszM3ywTgmCga/dYTWE0C8QwRr+fP9NJ8mSTv4mUsfmonvHaqEYufJ4uU4n324MRYl/D96YNiu5kGDOT7OFXc9EsTHIOWi/ocRQ2b+0V5cGeBi3xgngC6SQ96LJk131yo9FLrlr4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=tEtjxeVJ; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="tEtjxeVJ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uCcJj0LKU48+4BuabRBddGsRCHkYlNrK+zny5GKhfSdxbn5co4tVAOgVEonHNqnZRi1ePVPxJCzdH7pZ8VSfduQGyheoFkRC6dbwzb0dGEqf/UxJAp5abPCyafojb6Z/0WvKv6aKssaQiOeipfIBX72PtpC3t+04lpnuDJdRk3S4Vk2C9tc3h69axDIHuWRsUii36oY66Nr5EGUATeH/A+hDc4oSR3M+LEgvNFAsXOIZ4EWa1If6vGJDCYp/WOyGxL8iRuS216Ugp27McFFzMpb1mGQxBvl76P6K+9vbrep5S9eUEuNnE6LJtQ5eoJvICbmxYiSBX/UQKCUj53urLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WIXXk4W5M2aE04XrkKAC21LgHUjioYhSRCnjUEwGpW4=; b=x8wpd40ZDQfJHYGNJi+TS9B77EdIyNTsCbBlbZi0Gfy81HyM5rr10rIN0sGZoiDdI/epPhr7pS2wryKoKJFMmbnp36Wzg2bLX8R71qRuhMIUxcI0Man5aucKPyBXE0keZYi30b1D1rifB0BtS4rZcwKfspmVVNY1TorSWSunv4vl/nLapIvfLBQMjQmq/XNl1GD5vdb633vAzmVeGy5m4Xp7/AmoDkgtwnH9gGbgOhSVA40tWzo9DwXMJ4Y4CEz0qJB2SDrs4euoz1SXozQyPzElGi6hVtqkLkUcCXLOdkEqfT1ck5HtL9E+YwDtYas+98LQZxDOiaReuprgQ9srCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WIXXk4W5M2aE04XrkKAC21LgHUjioYhSRCnjUEwGpW4=; b=tEtjxeVJbu662kzT4lzhqh0TeBT5AAYmJsIwe/hFZ1srahQJ8vDgd7PbzE9YFseuVkHXNqeMvokfxUmlJ7c33Bu+u5Gp3VH3YZo68h5zEqTpnjx8LTUhl/RExNhVASRXU8lHqEP7IO4beccahs102Pm4sVsdwLVbzIHH5P6Rzb09oMDR1FmNYl+sWf+wMHwW3R3eGH0Ghp3+prjorHH2c1C6fiAOXxXYFBf/iB7eJMzKDBxWYG2DzKNypl6ILorZmvofuP0PBs9ZOwLWXG9V5IVCIq5XG6jfjtUS3aMmtvPNGlXxbOgQRRWt3DNL8F2we2qXL5M1GWywtqc5/VRgBQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:07 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:04 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 07/14] rcu: Promote late-arriving blocked tasks before reporting QS Date: Fri, 2 Jan 2026 19:23:36 -0500 Message-Id: <20260103002343.6599-8-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0PR03CA0027.namprd03.prod.outlook.com (2603:10b6:208:52f::33) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 5d5c3356-8848-4ab9-6558-08de4a5e66fa X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?xFZqCDtxACXk02hKiVpED5O87oWOYGb/5PIzg5gdwyxO+W6V024sXZ+cAJlV?= =?us-ascii?Q?p6JHFT9pVKjnA+YC70ypBYv7JCuPbX+fM2HHEVbbxQPdRhhbu5aHGfuhMPUp?= =?us-ascii?Q?hnpNgWMk6jcayvqZv3227D7An1hVRqRy/y5QNLCqTbgHaYQefr541z5UavJn?= =?us-ascii?Q?XGN5SL8dCdctkrA+oNgEZ/Nyn4efsF8/kRylnV9ZmnVyXiDY8mBXv9/eUvDY?= =?us-ascii?Q?1drYZ4cAuGGhtpclMvd4l426Cg503CsHSqPTp0qaqt+xdqYMtMX86k9rRTxf?= =?us-ascii?Q?a7uVbex+L9h64j4qfZR+CQlowIamsoZeMKGRwo6I0DH9t1vlaRWXiag2uj/P?= =?us-ascii?Q?Ro2VmgGWrMHz1H4Poo0SpqIJ0uZiGunkUoBngBiRnINWQrl0Juu9vPNvryH8?= =?us-ascii?Q?KZ9nSzpaczrIiCDG41/4/1l2gHw9r6Agk6Efie8ixxdh0/evlvbyYjdfIIpt?= =?us-ascii?Q?HtDLo4gXyktDdXOHPv6XpRSWlpqheelzfXG5FiqH9v4JQ1+GKIB5JpCAKzAc?= =?us-ascii?Q?tHFQNjQKaEowUsbH1FkeMSf/WNUmKdegU9afrl4bMD58WgVM2RSALRyfq1Zg?= =?us-ascii?Q?BiL4++rTYQl6ECiOqikrfknzUu/6i2DE2b1PDJBoo8y+KNzfHJunAQZ/fwsS?= =?us-ascii?Q?sUuG9DXi4XUYFxWRC0DQ+b9ZuRoQ8afVP8w9M3rzg7whef4y6BfTDA8K+dVi?= =?us-ascii?Q?pDZ5+NPAgFxBp/bBCfqScL1ImhRpBxwslrlzaHj9lbgMbjtJEM14A3P+TTGc?= =?us-ascii?Q?t8UawBkMoUQWst2Ed5z+IhFdZCNTqyLuT+d7G3CPxEbQBY9V9ollmHvbNA31?= =?us-ascii?Q?kotQH2dRDuJZDs+JhaNAbC2Vud0n3m9yg1EIsLI8j4BXExay+B6IjMEFfON5?= =?us-ascii?Q?9xkcX0Gnk8RQGiDcYU4WOdPMmiuBgFILGVFT2IkGUfymD+LSe4DnlySQ+U7p?= =?us-ascii?Q?DSw36gns7R3GOClPAdZKnGLtxO+T2ordaqD4oBrfYJIfG2J+L0mFzTDcNN6/?= =?us-ascii?Q?vWGFInwD56JVB4kalPp6x8zBUAQeU7BMTQKiH5vhMn4BdbJMx6On2mOD2lvW?= =?us-ascii?Q?qVMvfGUeYOaNouiuY6KNfiXafsuy8YmDwbrzMC8Pq8IEhxrmURpRycuvGFvs?= =?us-ascii?Q?WrfsCYncOtRaz0EIKYc1IbIfuxBvmKO1nrwBSMznGUy3mONy6lF/ZiXNCXrB?= =?us-ascii?Q?Utc9GMJKqNGJqCXb88l+9NtVLVwOIGaPd0KrrN0EJsTJtAtqBRK8VjJC1kmX?= =?us-ascii?Q?i1l931uBrNJ5ME/jZ9K1xm17tKJbW4JYCzXDXHdO1zhMP43IfDl+I6eo5opJ?= =?us-ascii?Q?nFy56zYY4M8PkDAh2WlwcoMijA2i9WtFK1ZpFYva4imRD2sV2bpGGFs6X0Cx?= =?us-ascii?Q?3PDtf4n5DvCnTZ6rlVKxmSaMNVsfzz8RsL+2D/JIFWNVJjqiB+AeJtxzeLAg?= =?us-ascii?Q?xgPctjdHVGj6Kaxm3E1mwxgh8RINf7UZ?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?tJzV7XdqVyRcbWVxWYvXBazpyp5GI5JNgIkWyIsucuYHL3EGjSDlFn7eN39q?= =?us-ascii?Q?tin0I2TIyFjZyI1OC3aVLiav90EaNnuDj5epyy4MRqtJXVwlmJMeybOzG2Ma?= =?us-ascii?Q?kN9nAZZuBj6+BpNtVon+Y8E07NWq/EoqxA++TX/QmGxoeR9aIgH5G9KNARAb?= =?us-ascii?Q?F9GK+K/w5399v/OvTBE1v1B0YOz/69VC//IY7A7w3sRsiY3isFFmEsBYypo+?= =?us-ascii?Q?mmX6i1ZewDcrKGdOvC2Drn/UdkmA4kGsjLcOPqZkXgS/aoP83AMH8c+ykxLh?= =?us-ascii?Q?3/LYtUw0kPgIyqyJu9FkJOQCvhrJHSy/QeGQWEZ4JvRtnRTEhUu0Fiq6ah6W?= =?us-ascii?Q?0VGPVifLr7B/q3GInmZnZop9fXyY+9rIR/U+IDYelsFz+JLtb0PoSA4uFfGD?= =?us-ascii?Q?mzN52KNWAH0olrdYyZ0Wb09pV2/8x9CSiDqW/zNkJlgNB9zrBcGiL+8FAcKO?= =?us-ascii?Q?PTDlJBiFvsmpIZ72MbfeVWQHebwCVW8qLujtUO0hfvpjJQj6OXjyYS8JH3MY?= =?us-ascii?Q?uY/qXn8DU6sALj5FzUeOpvYiaHsYUokPcKtdffpb+TY5wQdEfTjahbrwJQuG?= =?us-ascii?Q?X3EZ0q1hzkuAN9ObgB1YMrnTDqf6dfrVxOh1kpgvSrguxyKcKF2Om+g4YCDq?= =?us-ascii?Q?x8/IXZuPo6ZbsYoUmqF9dyo3cp5DhzPbxO9bRBb4SayLfrdAhyfkXIjP9g4s?= =?us-ascii?Q?ZCX8m5trbpuzKBgqi3jq4gePBUg+JP3PMt4LYaWR7D8DrQlG2aszC31LjhO8?= =?us-ascii?Q?wr8Uk310XYVVgB8pcMQqlS5t8qMBTlBPB7o5Q7cdaNW/9xQI2mkQ7lPrCLkn?= =?us-ascii?Q?rZRfWeHRkr7LBr0NTQUaul2uD3q8w0pjlzeJc8KKH1mPcPakT94BygLMNIBk?= =?us-ascii?Q?eXhnQekf3vGObRP2QUHEejTR4aP3Q5ZQm8WYVnEF0t0BrHkkUjg2hVBRleAh?= =?us-ascii?Q?I4STFs95KEvTpaHUKXXfZvXVFdEJO9YzGu7YRUa+mgQfMkXFEYRpxySAXAe2?= =?us-ascii?Q?JnTugDjIiwgeSILeTbxrs2JqPkAn7uCjSDzYS0rtNRdNLQH0kMMXPuoXZehE?= =?us-ascii?Q?mhXcAcagHX93D0HCWAZmParJ/Gnb6RY9bQn+GC8Kb/m/ujPayAVr2mSM4p+r?= =?us-ascii?Q?eg65LTKgmuiId1/phixkvPfhgypSG15oyDpVGErxtCF7sjmfAUdxAJksw2Mq?= =?us-ascii?Q?Kno4JHvkCIUK7IQnd3yYyrIdz/PpQt9cGpiU9vXad7yS/rssf/8AKC0o1DdA?= =?us-ascii?Q?nxBLiRSEWXtBzI3FjaIizYPT71bapAKWlixsH2yhIlRaoThiUaCpF3GG95a7?= =?us-ascii?Q?YZ9lJ0YrNYxxPewOveDv3oQF0XAj1F/67llT8qU087C1Uun54bid2DBLU6lu?= =?us-ascii?Q?i4FMFDVwDlptTySC6NCafuzeeV8Uuz1o/saqbBgpjZO4l+Cw8SsUPbQhFRap?= =?us-ascii?Q?HCfeR8HILBWP2pNo9UhGB6WceGKAONEh4TS/8VwNPr4AfLxASNjLUU4onSv3?= =?us-ascii?Q?wpzJtik06rGuDJary7Boe2xfQD0M3IqIJkFwEMK8rZLRsTEZTlBKLXkIBqjw?= =?us-ascii?Q?YRA4EB0Uac23TiTCgupYl39+A0rr4Hy7TKzs8vBXMn35f+Ze6/wxFEgIFkow?= =?us-ascii?Q?5T6FSOthEJUgtUik97x/UavmxLE6tDh93PET5LM+K+Q66Xa/B9R0xJHI9Wf9?= =?us-ascii?Q?qh8PduiXGZFrWNkOmbT7RPBLywA2IMquujXxWGlccclZumDMJmFSnh5xe3q9?= =?us-ascii?Q?JWP0z/tXWQ=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5d5c3356-8848-4ab9-6558-08de4a5e66fa X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:04.8811 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SoqvFrBb+f84a/uq+DiAOaQq8fVXSXXUiWFVD3iQtkw6y5fa09mbBfhXOI2h9jrXdTMtwa8B/0GLq7Cd2gx/xQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" Blocked tasks need to be promoted before rdp QS reporting, so that the QS reporting infrastructure considers this and does not prematurely end the GP. Therefore, this patch adds support for the same. Signed-off-by: Joel Fernandes --- kernel/rcu/tree.c | 7 +++++++ kernel/rcu/tree.h | 2 ++ 2 files changed, 9 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index f8f43f94adbb..2a20b1a8c5d3 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2500,6 +2500,13 @@ rcu_report_qs_rdp(struct rcu_data *rdp) WARN_ON_ONCE(rcu_accelerate_cbs(rnp, rdp)); } =20 + /* + * Promote any late-arriving blocked tasks before reporting QS. + * This handles the case where a task blocks just as a GP is + * starting, missing the initial promotion in rcu_gp_init(). + */ + rcu_promote_blocked_tasks_rdp(rdp, rnp); + rcu_disable_urgency_upon_qs(rdp); rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags); /* ^^^ Released rnp->lock */ diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h index 25eb9200e6ef..809aa77f57f8 100644 --- a/kernel/rcu/tree.h +++ b/kernel/rcu/tree.h @@ -502,6 +502,8 @@ static void rcu_cpu_kthread_setup(unsigned int cpu); static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp); static bool rcu_preempt_has_tasks(struct rcu_node *rnp); static void rcu_promote_blocked_tasks(struct rcu_node *rnp); +static void rcu_promote_blocked_tasks_rdp(struct rcu_data *rdp, + struct rcu_node *rnp); static bool rcu_preempt_need_deferred_qs(struct task_struct *t); static void zero_cpu_stall_ticks(struct rcu_data *rdp); static struct swait_queue_head *rcu_nocb_gp_get(struct rcu_node *rnp); --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B235E221DB9; Sat, 3 Jan 2026 00:24:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399855; cv=fail; b=rAHsblZIQG9YT9BnuGhR0R40JNdYghTTi9tWHVeCpdj3Av8FeZSfZcHkwCAKRLEEOpwkduhxAqvdqBuvDOSpEo+5qugYUOeHhPGDv1a3+WIQWXWeFNtCsKsGN94tosv0viWoAzyPr+G1N7bSzRFtEsX6c8rftvM24VpuDrPLTSw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399855; c=relaxed/simple; bh=WvbXjEMy8QsIs5sGAeuW9GMNfQmjUUYd+FRZjaOGRgw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=Cz6YuPyBU5A7VHk4idYuIB/3sVXQAl8Fk+qKb8cYd3m+g4x5sxgWe7nPz7CEffsdiR2BPspGvGhHXGS7bUH8X6vb+lhn8uCECFslPcDPlt9eIDnVtHZh7By2Akvu5Q602x9ouxcGMhpsxNsWUpnk14Q+YdJq0B3/+0jaWm/wtyw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Ux6Cdj+l; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Ux6Cdj+l" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jzppuMBtB9LjwlKQILWUacM/fUv8Z0z7jMpS1UVJPgjB7Q1X+SCYVjZHqiblP2+621TzNUR1nCPqf+7V6otxoRMHNQzpCu/Ksv4KJRKJwWPWt9nVBg2WtIk0IB2qHo6EU3MZD9cImvUBBcvI//oBsUL5rRPVF18DN1PArOwk9jKNdTgZ/KSISmXsH8pk2B0tYr70eijoowGdDGImewoNTZfU9ikehocAwajAuk7dZN3qHkSOtLjjp46syuKz3tVW21Uw/d4B62MzIBTwKM/iWrrvbYBDl4vG+TGHGfAdoJhX1Ol0gMLsEWjaKXWO+kDKVGDk3XE3e6rurxcfs0GlHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8HHUbotGhN7nmfwLlZe853APgpUr+MDloD9ZjD/nP9M=; b=PSD+xTEq9E4KKEEDDTHPpIRbLE5w9O6VvMQ9ulCiAXw0yJVi78jmXa3lIDN8+MqtxBm46AuDn54MkoELmV5d1iagtxJ20bZrAzORBnK7yTrmdWfehBU2LnAOkYXqx07h+ybGrBd/1aEKf1ynaRP+Q/vpyWp/oOEJczM/r9HSWNks6MDqv0sETL9FJLPPp476k+v2flnaufR+nXCXw1Z076siXw9lzCs5ve8x9fNMg2lR7bYGQ56jA73f4hYT4xClvqTTt4CDHX4+yc+dcs/cnur81HbJUNO0ElWjKSULPt2JOStzG6AGtdKL6FjKkTXfxnlSyuwC9mtm4i2MssSPvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8HHUbotGhN7nmfwLlZe853APgpUr+MDloD9ZjD/nP9M=; b=Ux6Cdj+lshzIkh0gLoVIGy7AMVGSiH6LYJsrwtAnJpacwq8iRQliq7wXX9PaNnUit6f/tMyVkT3Gg5AORiRqXWQO5DKbXA4i7cwN/GZsO2emdpZAZLlDGsmNDj5n137xMNdO5DNB3QYQg9kX+BfkwBmGolDIKarC7KiCrrgoVnb5nYKZZmqLgjGswmY0oEuif+Rk5+HX5JzIBmK42zz2CC0TBVSawJSEfmp2EWHDHEVPWMv9sGTXLwbyrPX8KyMSeFO51VIXI+AHEZ/VjZFvLuAwIfG/4mBzJOVjX3BHAtR4H2jy0Lh7dTV0KdPC+pCmkKpMdgln95o1Fe6npednjg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:07 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:07 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 08/14] rcu: Promote blocked tasks before QS report in force_qs_rnp() Date: Fri, 2 Jan 2026 19:23:37 -0500 Message-Id: <20260103002343.6599-9-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0PR03CA0012.namprd03.prod.outlook.com (2603:10b6:208:52f::25) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 026b9a54-23b5-47dd-b49a-08de4a5e682e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?h8W0nt09pXrIFWAZyZgAY72hsT4kYUsXgdxBDmwYM5dsUEpVWptLMQnavPiN?= =?us-ascii?Q?DZ08D1O+nMkXfnT05vhKVax2u7wMVuaT+cpBnhREfyDJG9glnJpZ1IBw7j5R?= =?us-ascii?Q?y9xTTg8iF9LF52l2LewcwbodiMHbKfy9j/bWCla7Lw7opC+fpB/vH4wS5FaW?= =?us-ascii?Q?YyBysyb4WWPkz0xHYnQVcxRKPVk3jXTYjm+zKObTFivEXyV2waShmZ+wsdaX?= =?us-ascii?Q?EIiQxwurDN8lqx5DGYqvAh2OxeENmTN1YNxsmnN0TvLi5tp1DXTytIIQFAiE?= =?us-ascii?Q?L0WbWiey1vjwrz+fDUGtzro3pE3bcus4cqrAaPfR6T7erbQYER3S8ojl9bVW?= =?us-ascii?Q?hd9CTQq1o9zcCGqsTOZoeE5HEPHjLCaACH0QPwzwaUh4OaqPgIxhdy3/OYQP?= =?us-ascii?Q?k5R6+DKeDoesVFnbJ+6Yhqynmk97VcrNBn8opci6tpCtbdsGHQyzdVTvXNml?= =?us-ascii?Q?D6vAcu7waO8Q9ES1AI3RR0jdrhSOA2FMgDGsQ515qbSVLrF5ZEcFK/lQELzi?= =?us-ascii?Q?DBG+TpKumJz54gmJDpHIZULIZSiebrWafa6xnIgXtjV4OgGbCneViHuNnqHS?= =?us-ascii?Q?9gmVGWKKgsRnFZsEHnajBSqG0qMDdLuJUiedq/cJkYzceU4PE+4nb7t8Fo21?= =?us-ascii?Q?0mb8h78LnYAvW4Qwurxf+Uzr1F+0wKgm4SSm3OPYJthsSWPuQdsfMpjjFNfQ?= =?us-ascii?Q?MOQTHPvUlTVUSiYpMI1qdsZVNG/dHI55Hut7AVbg+lqevaAa+oEXe8TnTCDS?= =?us-ascii?Q?nHl2i9uo5uj0mr46VAJiev8YhqdFL7BeupIZLrPOMdLYNTSaXD91jS3xIHZ9?= =?us-ascii?Q?VO9ueqbI+UIVjInxC0BDtS4JgwdsmF+/6tRafVSVRuPBZ451H3i88Db98ER5?= =?us-ascii?Q?iPJO3OgFqTXNUhggd1nIfD6Uil1q/PzGXjMioks9+bKIswtbDfVQG5pc1OBc?= =?us-ascii?Q?EhZn4Cv7M4M4tOLqrb94LnwTZhvWHkWeVVBzNByzlKdaKGhBnkA9+OmybdOV?= =?us-ascii?Q?9ZA9PmNbdE2O1nUkMsnXbtGh50oV3j6RUTj4d8GLBuVsoA1b7eN22yZCtyiZ?= =?us-ascii?Q?LXTiufcmSw5KIyORjLS7o51b2pop05SyyUejXGKUDu7z1rA1jlGALRxHzk8w?= =?us-ascii?Q?3VKZEIVJ0X3M5G3evlp3UI0FjMYoSSMgFvrsxemnSHrvozhd1TjIRHWHoMft?= =?us-ascii?Q?4KeDVAGxm/e0hrnGAHyA3hzimVgtqimD/ExvHLYjEmHBzn3FAgDUTSinAWWE?= =?us-ascii?Q?5ZFa8lhqRI7E+4Cyy40QULt7/pF/ZuwrCu4PvQnYrLQC9nhVI/Ndanjx4ePh?= =?us-ascii?Q?ix5LWl1CW7zyIk5Pomgo860AR81IRlnS/PnC/90BAcz5mkIs+xWUXr8hs6XG?= =?us-ascii?Q?gjCRu1uDehnXQXIG5AQCDut5d43xDddMAud43ziUgmGDSWx2pAWMUC5Eb/1S?= =?us-ascii?Q?TZUNGCOg6iOzTMuvLjyZ1KAE+dgcqcqt?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?5HSAynQkSp/tOXXIXXUMxgmCnzIKF7W4sJLLjor1hM1paKCadKc7GlJ8DnTH?= =?us-ascii?Q?699X1fpDUP3oT0sJs8wF4QF21oL3itHMvZo7kViTuxsjBBUddB5M6L1QCzjI?= =?us-ascii?Q?3EBJ9fjeGTgK5MKUyBh7Lfq4kLbOYZjY1KoOcLYLKzQ6g56JFWMxKF0O3Tt4?= =?us-ascii?Q?48dHTKSBaI5doAOgGJMPmaBauaBoMhRffaQ5fTZfhrbIcOd3jjZWfI3k/2Uj?= =?us-ascii?Q?9u23CkuBJhyKF97QOB7yC60ld4pB5srjxOeQM4ksy9X9Zu3scAl1aU2b8bqq?= =?us-ascii?Q?gsPZNlm/FA3MzwDm31xVZOUxtpLYCm2lApK7WSxuotatKv8sYvdDYHhI9+BM?= =?us-ascii?Q?vtgwy3+ywPP+N+dzr/C3c5D7lZT7tXxsabHt/vPQUGbSDgQHh42iTxdGUl/g?= =?us-ascii?Q?6MqhvVgyXXiNwZ+YJqy/qVDdYMRrSWYVR2FUjiWRKQtkVM4LLOEX6uJw2CbN?= =?us-ascii?Q?Q8Ho/jT5q2oGDTatvheOWQYM43xSRpwfUv5pGA4cPxJE3yqeGpuoAQcHIveV?= =?us-ascii?Q?x7dPpn/hN6HuGOJCiQP3uWZF6c7bhFpoR3Y3xLqBkuSqon5zOKd8O3omoGBO?= =?us-ascii?Q?J7H0mNUotRN/DxrTY6mkD3wm597HR/dS1CM0LC7tDo9G101//Tobv867Ndt8?= =?us-ascii?Q?bdxIxpOQ0uZgNNkzNWC/wdaRB5RrNYvvvCATUtaALsmx/vaW2dECHkDXpgG1?= =?us-ascii?Q?X/0hBrSTJcPgrS1AkYWY2Y/1b3sKk4PPTjD6/MPog+7rTEs/v0jW8CRWUBqn?= =?us-ascii?Q?VXsV1v1rd0Qc9omLgTHUKhxsVLyvnHHfennh7e1XutFUJ1dJdnzdA11hjU14?= =?us-ascii?Q?RAMD3VPHVgi9sRgqoRrek7rFaKPZWiOUkgQPDIeXy1xGnYX02JY17gseYqIf?= =?us-ascii?Q?44VgYkkYXl63UoL4tODp29kYlIlpS87FivTnom0F3bYxb8DjZmmbo+Vp38xI?= =?us-ascii?Q?mTnRvH6vAHrVcPaQXzQNcawJx+WMCyNuRtTTguxg4wW6Ic+OIEIuJ8hn/Igx?= =?us-ascii?Q?njao4aVpyblH7u1fuyZNqDw/QCf67zUZalwtp5b8Wv+jt2wNf5hmMj8d2G7v?= =?us-ascii?Q?brJbRrb8BNLiAKcXgwaNBnUqrGWI0+5T7ZsVUSOnj3CSJe6KInWdO1d/WTGp?= =?us-ascii?Q?4jFL1xPzEjQwrXifdviBVoUq9OngsVBn1B8Yp8m2jK+6zoppwbnA87yYRexq?= =?us-ascii?Q?sNcriaUQSyCNNRbY0z7jDdOYKK+PX+zfDADDPX86Uqn5FhqbIKu1BUSlhg07?= =?us-ascii?Q?h1q2/AM5Nb0hUaAnJPWOTs+wLp0FfG4qXjI0Ow+UboBbQJe2Km16ApwZCU8d?= =?us-ascii?Q?87YV0cAjvjZm4E94Zd61njNyvt4nPhPi7uGDCJn6DHMd5yYKtf47zdnaZEtl?= =?us-ascii?Q?+SzEAY+k/tK8wIwQ0qgVFwMGc18dqt5yaq/PVo1EBGwujwjmvPN655n9HgBf?= =?us-ascii?Q?KfQ07mBVxxXsJ/bTFD5aPG4Riz3+pvSQ2O3kNn5Xmoo4/d0Ue6kMtp3BomjK?= =?us-ascii?Q?61q0Zr1xncz/PrAUeZt3SZ/QtVfBqbsYWIzygn9uXNi5IGWI3vxY+MLAQyJO?= =?us-ascii?Q?6mSuhe+EGGVHYTKseaccueLONnyxG+Q9CLwNdIULdbFNM6Arhaw3/p4I5sXF?= =?us-ascii?Q?DhGdaoxmijpSfiEwTNg5l713fRdZoi77MeYvrVe2ff63Oe2IOgTq0xCOQVMr?= =?us-ascii?Q?XcMCOdWh0r7fix8h5YMZYkvJDHpPjF92jPGBoeS+9jS2Rms86/lY1VhodvKw?= =?us-ascii?Q?XizdAGqJOA=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 026b9a54-23b5-47dd-b49a-08de4a5e682e X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:06.9093 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mFSyDmvHgfir+kOUo6Q4Fiqj46jnie4WIoNa4eAMThfnjR8QYY0PKSo8IusetLyHpHqhXoyJlmSJPCWZJsUi3Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" When force_qs_rnp() forces quiescent states for idle or offline CPUs, any tasks blocked on those CPUs' per-CPU blocked lists must first be promoted to the rcu_node's blkd_tasks list. Without this promotion, blocked tasks on per-CPU lists won't have gp_tasks point to them, so the GP machinery won't wait for them. This can cause "Wrong-GP reads" errors where a GP completes while readers are still in their critical sections. Therefore, call rcu_promote_blocked_tasks_rdp() before reporting QS. Signed-off-by: Joel Fernandes --- kernel/rcu/tree.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 2a20b1a8c5d3..19fd13c1e6be 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2790,6 +2790,11 @@ static void force_qs_rnp(int (*f)(struct rcu_data *r= dp)) rdp =3D per_cpu_ptr(&rcu_data, cpu); ret =3D f(rdp); if (ret > 0) { + /* + * Promote blocked tasks before reporting QS. + * Otherwise tasks on per-CPU list aren't tracked. + */ + rcu_promote_blocked_tasks_rdp(rdp, rnp); mask |=3D rdp->grpmask; rcu_disable_urgency_upon_qs(rdp); } --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7134F22256B; Sat, 3 Jan 2026 00:24:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399857; cv=fail; b=In+oKMaKqSFbWA99rAnIIE+j8N3UStEsDfqmEpxsFVRC8NYny7rmxW8KXWBolZjHbqfuDG8wpxDJ6n9t+IwjI0/lLewf1bzW9zEKDtyPm9RZ0Z6E5pcLau1FV7l+6qrTCqp5r+BAdvW3Lv+vJ039vPPS+v/VmMw+6mTuozQtTwI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399857; c=relaxed/simple; bh=8Nfx/YgLwz2WOv6Yf5okEol4Ptd6zOufx6U2qYBV/aA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=jEuPaMjT1WsZShRmPuDin8Vl6A9GvUmem8/bGn5d2a5ksyKCIGp2T8MbYf7zOul7Ea2QFY0d7cX9Jvn5ksumTINuwTVi5ahGGbaK/1Sf6sTV+lFi/SnOK6EnL0WccM/3BQx+Z6+uQetEFREFbi4GCkylk77ul2XvFDi6e89Q3Dg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=YbBJ/rIe; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="YbBJ/rIe" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=oy9HqLPiPsqwmbIInEIilKFc013lAUlyRdbJS8zcQUM+IPBvDqa15Qk+SnYEsJvouZxwCOxdKyxkCyT3iJ9hbCg3QAh+Jt5CFoi5rQkRT9iFe9PCmkKIp9pz0JnL4J+BVuH1XFfVIbx8dhdeHEt46ncHACuLFPFi7II5kC5Atl3jw71tPaV5UHhzsaauUMLpMy0mYnfutMK7S6/oMFbchZ3YO8NjL62iFLehoV2DZYBsi3iY0wg3jN/QIr8BeizxcoVaUQqW8kxsz7gG3Ferv0CFQEFdxqNTJeTM3zgD53Yj6mru4VHIDDElaSlNpCu8rZDTz4IjNZn2tegdeCVDjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EmCyp1Uwwr9vGbXn0bGkp4zU2iN84nXbhCFMqJBrsjc=; b=CJslop7ErWciy1Wm0ZU5JFFBIW8I7Fm9qxfAjohHHZLkfctwkMix+tgmAA2dbw7GxLRtO09KGWaJNsI4ihbVNjydEtgvBysUMX9cpdJtn8AQFpcaLMo5B0x162P3ia0fNqSAeVCN8stQDHEs1U/8cFW4t2pibw7IrG0Br2R3CWXEmy1YW20wN4efVYrhuGaD7hdwPN2fTVg28ne9642rvIxVYuQiUwN81RSp8ngthDtL9s6nE0U+aJnXiZM+hEuhzRX3XC0phjVgzlah5I9x7GRdOhXG4dou0kLoytCNthdEkITQ15zTt1BdFDAgFOZF/TzsyD5n2cLWZYhAHUgthg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EmCyp1Uwwr9vGbXn0bGkp4zU2iN84nXbhCFMqJBrsjc=; b=YbBJ/rIeYQ8ZPFlJ9rek+XQx2ScKVSDhEVqeoa7q/hnm2ccm2wJAl0w+rYFw5H3JnzieBKmMPrepizPXkEDPA6pTjrLztfyvuNapg7INiaKBsPemvsKxJOlOAGaOVdmvB6rtIypl7Vgl8EfEREXdZfE9SYPNDP+XXIMsZj5i8TBQiAqZzsMAFzGSl5nIxcRJSuxD+wx/6mHCxX6yvRtrOON0YuKASgkXtpPucSVo7Fi+WP7TGH280d48RXzcyUdbFdgaKQMbPu4qhpclA65PT95XVbm15bdgvMLDGSLXmiLXaOuizaDlDh4jHA9yoDb4pJl6bCudvg1KFqQxM9JEfA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:08 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:08 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 09/14] rcu: Promote blocked tasks before QS report in rcutree_report_cpu_dead() Date: Fri, 2 Jan 2026 19:23:38 -0500 Message-Id: <20260103002343.6599-10-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: IA4P221CA0011.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:559::14) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: db563b26-70b7-4089-a5c9-08de4a5e6954 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?cQtAjpfq89fvfFVCqiv11NGIIiC4LxfkirqznZPUP3JFKKl3gNoViRr4sb/c?= =?us-ascii?Q?A6UJNMGPXOr+u1LSZ8WgX1JnUQmWU9mKJru6KG49X7X3cVK4XnIxpMhQHNVL?= =?us-ascii?Q?eVSsrKYczGdvrMfTfdjg0vFDOO1QO8VKgmlsdkhR3Agb0vwPBCn/2/GhHtIh?= =?us-ascii?Q?8oXL3lydp2WbpL44I4Uw9VmjvGPiBgJ02gCgSpaoDT/X4f1cxFZveFF3wTxg?= =?us-ascii?Q?WdgdhkCbVKGPtZFKFB/11gOV1zHnf/DGxBq9KPZA7/L6tpmfHJTxoNJtkrhi?= =?us-ascii?Q?cGbtDtzkmUQCApnQEEHjuBeKCVPKETWcFe/kORW04csRbVeBi/Wn2dFevM6R?= =?us-ascii?Q?GKRzFKVjiSmw5a7YL5AqwMwWLdaGrzzLU8Kf3yb7gsEbvWUMSoir5WC81o9w?= =?us-ascii?Q?7RBM7s49WLFWuVPA7a4Xr2u4m0YNUCIMsr+tCvG8W4qRHYAZ6vy8TY2G8zD5?= =?us-ascii?Q?/I6ePqwTPzg28tvrJeitVlfs1xkB+XSdpPv9/zFyf3auLGKQMG2vaWE+1r4E?= =?us-ascii?Q?Ac5yWd8INFmYhsJR4aS/ThxJ/5mg3srdP1JMWHTc46mUsk4PNVWzJKl7oY42?= =?us-ascii?Q?qnByJlzzdC2/J/EBaOGP08uJFFRBZ6Blkbot/onEDSg/WSgvjkzrMOnQuyvm?= =?us-ascii?Q?90ZOIOjY6KzKILBapzJafI9PZU0nXbyMNVNV9qIF6UdJx8q8vw/ifc3pLRNS?= =?us-ascii?Q?jy0ueU3rWug6vh4v69IJoh63fcf3Km90bvilunftvt6hicKP7ZrlBsos7yG8?= =?us-ascii?Q?xYBrfdf9wkGiAbfkwigBeqD5naxhG2e5Qql22e59G1Q+0JeEgGvNjga02vUM?= =?us-ascii?Q?lhhkvSS6QjbaeedmDi8xAk7GDPqbFMxt9iez1N7FXEoh/IhzqmmJayvMmoZa?= =?us-ascii?Q?YRkE5bi3hLllCsNVuPLkYmTQizzuxO5tgswqiGRBoJ5YnEagK54dLB1MMGF9?= =?us-ascii?Q?kAVl8jgbtxMUlhgxkAOiTszhZ87voVdhDSxGqV9SuIGDPCnnaw9oZ5yAkC/n?= =?us-ascii?Q?5MTKPOX5UyRXq5GpxAI7QKKJ/I3card/4nBWaWKIw8GOaokd+GspckiYkr0U?= =?us-ascii?Q?5osdDbeVQrwH4KAbW7DHaRoYO3jhA7cEr/wUoZjLwk1ZkWypVo2GYI15ZPiE?= =?us-ascii?Q?b1nA/Db7oBJbIFwHnDxteKPglQYQrwyddsAQH+uFk3vcFTkTALgzIAaCFSut?= =?us-ascii?Q?Lc6AKua7kJL0Fhrm2GKaZKvvIvpTZ6N2bBX7G9c3MijItoJh/BI4MRsJvQ3Q?= =?us-ascii?Q?VPNVV9KKQ9EcnhLwkM9v9pjm4A4vRhdNKDysfRYNV1zl6fTdDre4DESK9Yh3?= =?us-ascii?Q?SAGj29/mB+pTnNE+I6aufVuzcbTrGjeRcNxYmRrriERiWEvaSAjam8rjkt6G?= =?us-ascii?Q?1n0c3t2Bf8AAL1ckf7zbrII0i4ucJ1XjQwNjl06jYzQ3V45Hp0yQOvlw+07J?= =?us-ascii?Q?EBy6VNXgfhkdXuTY050srF8oEVnI1WKb?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?b2Zm9W+ebfPCqVjKyRh3TahDtRXba7v+Vmel3Do/90lhQ1m+MJkBi4aGUa/8?= =?us-ascii?Q?6CpTyLNohyjOCMl8hT3osCNbDwcg3CQuMiTGQjgpKXKk0XIVP20wiLu2zgBo?= =?us-ascii?Q?hz2+pTvL0qXSJMq/kJ25KrsROibXPaR6KfUQlpYl7HEGXIcceSG45EgYTkGp?= =?us-ascii?Q?dxZXi94+mBqF+yasLEaJlJeg6hiufaBSkbmeQETqDpQjkqRwLUjaKNzWR5xC?= =?us-ascii?Q?Ot1iK1mWI0ntyL9h7CYAV2IIO3MEyRmfUBQrRZqCLJVx5FNoZZMjf3gNDTpP?= =?us-ascii?Q?6W1U7e9cnVvgy/6HpDKE/eCTNHd7JkTxxV3LepGduO8cvDRTUXhLCLTIJ0JB?= =?us-ascii?Q?RcrsqcrANprXMLefyPul7FVX7Mv1H5df7ZPrAfpwp9ma2lAkaZVP4aFzCj84?= =?us-ascii?Q?D2XsRJDmpLlgYyEu5l5ly5ilXhvGQtKfdChZ/AV9Xdov7N7EwSsfN9oox+cC?= =?us-ascii?Q?EeF9aeYzQ1+JiYYOWxBLMYRV10v7f+mMbolfuEWYy7pJ/ywRWPJ67u7gbykw?= =?us-ascii?Q?uNYREtRYLg9tp6M5mFQkfsS62sgtOs5oXaEdTbpVghSaTUmw1iqcTaVrjEoZ?= =?us-ascii?Q?jw+XjcrE/FyMbnILoOJ9hLbjNEjhyT0JB78WwM3UyhC226xrV4OwTiehG4EA?= =?us-ascii?Q?2fy0y9jWiNc6K/BGlcS9shjiPy8WB/Y9OpORCDxWL63pAMsFsa6LrF5sRbPY?= =?us-ascii?Q?6dW0nZN6tQKHtaRx585jZazo/Ph8zBT/oW5rSs5UTDYJuoeKhwG4Q6Thb+0k?= =?us-ascii?Q?8WHAoIhU/I6sZBtZ2zUPHno10+MVMzGgX3VTIg/zD/IXLNwkDH0P1VOqVSiY?= =?us-ascii?Q?svlOUaQw6PHTwaVO5J8+eFc14h3VGx8SfbuUwLcqzs64jHjru7lYWPf66RAe?= =?us-ascii?Q?4bQ5DYgteWSZPajFJ+zjrb8OA/o1abVAkigNcfQRQf/l8TUSvxnSfL88+8z2?= =?us-ascii?Q?iVNJOP6tk+t8xXL0mjPF7HtJx3f+lB6A8fxnplTCa/pK0NVKlOJ6w2xQxgrw?= =?us-ascii?Q?s9kwkBcUYQikBfsIqC+RUqZTXAQYOUdpXqTyGj8cNVzLc+i8D5IzYgNz3Dfp?= =?us-ascii?Q?KWbZ04peQrsrmjvS1OUypZcL8A4aoyOYqldTS7SoXUiAaoIqXyykGPZSzCb5?= =?us-ascii?Q?tq2RPvEXTJ14KQZ3yWsDSn+R2Le9aqVqUAJtG1woEVtCyfdpQ2kf1DNQChZ8?= =?us-ascii?Q?jvafQslMFIqcFdkY6UlCRsmBlfMYEAzR79KzCJjMi7JQszmylPyEJyd92P6Q?= =?us-ascii?Q?z0octP3ReCZYUt5d20tzFhBZjEEWbgyG1q5BPzZN9o2yzYIEk4V9hZRiIBe7?= =?us-ascii?Q?xIt4mDhT7qfzFmm3DDACw8C2SVv84qXDVasjMf/M4YJw647YhpbL2O+4LscB?= =?us-ascii?Q?EpsrZhK/PvWZ7ZkaxXK5wJTfRF9xCSMZ7V7rcV4kVYKQDDLOcDEh+Jsjk7/F?= =?us-ascii?Q?idDxfzQu4F3Xc+mKduXMvR78wnfc3wVcQN4SgqGhvGgY2jmxQKbub++O36V1?= =?us-ascii?Q?J3cWDFbKRjKp/2lPDMF/tezvP33jypmcxec56a7UAwWXBVKnpMzz/mS0mg3c?= =?us-ascii?Q?ko/6YWqbDHJEfir7h28ZORpBSaq7mHEztjjP45eCyJfRbOR8taLrBw4W9x35?= =?us-ascii?Q?yrV847CrdvU201RhqMhwgJ+MxCgrRgMxyKN7BNTxq4Wm87etvwWzN4/s4OU9?= =?us-ascii?Q?J/4D47kNuyNKgONS73l+kmgCC/XEMqgvoCnto6PmItSwxrPxsH57YZ5Dx0rz?= =?us-ascii?Q?q1dQGplnJA=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: db563b26-70b7-4089-a5c9-08de4a5e6954 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:08.8345 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mT5686qcAYNHoTmF3iJbwoUySgdZxIZJQmchmaYdvaeJ9Hl/8jxgkUHP+ytYO9c6Kn8GaCxYuLy8ZSGIRClLJw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" When a CPU dies and reports QS via rcutree_report_cpu_dead(), any tasks blocked on that CPU's per-CPU blocked list must first be promoted to the rcu_node's blkd_tasks list. Without this promotion, blocked tasks on the dying CPU's per-CPU list won't have gp_tasks point to them, so the GP machinery won't wait for them. This can cause "Wrong-GP reads" errors where a GP completes while readers are still in their critical sections. Therefore, call rcu_promote_blocked_tasks_rdp() before reporting QS. Signed-off-by: Joel Fernandes --- kernel/rcu/tree.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 19fd13c1e6be..5e73ebb260e3 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4460,6 +4460,11 @@ void rcutree_report_cpu_dead(void) rdp->rcu_ofl_gp_seq =3D READ_ONCE(rcu_state.gp_seq); rdp->rcu_ofl_gp_state =3D READ_ONCE(rcu_state.gp_state); if (rnp->qsmask & mask) { /* RCU waiting on outgoing CPU? */ + /* + * Promote blocked tasks from dying CPU's per-CPU list before + * reporting QS. Otherwise those tasks won't block the GP. + */ + rcu_promote_blocked_tasks_rdp(rdp, rnp); /* Report quiescent state -before- changing ->qsmaskinitnext! */ rcu_disable_urgency_upon_qs(rdp); rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags); --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D9ACA14B977; Sat, 3 Jan 2026 00:24:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399859; cv=fail; b=LjiiPHS189YBsDBzQA8zOkLccYPP/yKI0VQavy0lc7GfYBRkh81s/Q3i9lLdneLjK4ULYifNbN3zve+3VD5Lk7Seso4nV0WjBcG/yuH2R13gD6VSlla+Oc/yMo/o/e9Da11ZedVX9dLxISD+U5XXv0ksdDFQn5mUa6saEuOZbKE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399859; c=relaxed/simple; bh=g+H3zouk+COKMVrSYcs4UtpcnZ/qteKCRBfpU9VsFVE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=RxhCvaOE5DJfaZPj8LUgiNulssv1uu2+RERC3uHfgzWdPlKmXF+ZZI9dFmcaedh+DEM0/t3cIaWo9slVlJjtY3LKRThXufJgIXLx+pu6SMBJCdBrj+wS48ZPOZ9tpzf4k/lUrDZsR3OwAP9gsWaXHpcB7EeJIBg7U6XJMabcBWU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=pVd+ejRq; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="pVd+ejRq" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=f7CTZdsFdhpGY0aSpLX9kaSo7bV1BnE3uZHLkC2wrfI9F1lhB8lGvkUkBYEOA42kkgdVkrF9OqPeZicZgBENQWwkpqoFl59TW6Gcu+4wDF0w3qtl6yQUu9nplPgJaab5dCbRqmWDgXOMaadoaRGhMPYuXtX2tBe3o7b9StQVsFqm8paTKLpLKD+jWeIis72bn2NtxlzeKWIRnjnOrEhlGarv39B2Gmp6poUdswIhVEi6IpUzvEH3bqKmHMIIdfmn+oAka/3w31uu5BFzounRNfr0kHdbXWz5UZpClbTQ9foU5sdM++/SP/bTK2GnFjHxa1FKYWuOfOagOj2fhse+3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=M8A8psrX9Xcy1zRMGpJ6ObU1owk6vnH6LQzQ/6mDHP8=; b=u6X5keYoM8nUHklqNwBTNQh/agfQ9lOAlBt+YDCE3I4aB7wyiFizH0pt9uHnRQpqfX+w924KIKuy5Bpm4ca2Zd47KFLUqdOnzSXWcq+7PiNlvdjBP5J8GkYFyCC4eWfpbd30ebt8LgK9hVMPVxAZsiK6PG9DMCawLqAD0nYR6SU4PRFMTQEPXSkQl8fqkjaZrpKZ+xFnXP27RGVp/cYopKpPekgO1ktwGzRXRPrgD7wTzlgN+pZdyv11B1ZWTVTLReHs3kb2pvGkAXNkr8aKy3l082+/ALxaLb2CLpEYUwRKouvtBPsABxhC/Q7/JCAz6fn4QcxD0ghNAKNa89O1/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=M8A8psrX9Xcy1zRMGpJ6ObU1owk6vnH6LQzQ/6mDHP8=; b=pVd+ejRqedBRl16oSExKkctmVQen2+kr6BdcmNBfzTxmeWDElQKZPQPsKy7ggEMqkAoPJkF4M5NDRcbuNUKG8wC+H91gGj+eY5ySXxOdzLVq5hEVvENlPgarAnLqDxZ+VMrSn2c3LqOxdXQ4pBxk9ebhJqN+ijSlMpJXUfb/BKtcvqUdTBbqf5L0fE+GbliWCXVBX56QiLYFpwuLJxr50oZF3bmlOteBf/RQ63F8fzIf0rxwiNWF+f3EYc/i0Daluf8UhwiT+AbG/QEiVi0D09eYEao1lFVhM2sOW7ubIQVR00xRDbcFLSM2y77Z4cB98cOqvadUlS+P7AbrMIwF/g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:10 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:10 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 10/14] rcu: Promote blocked tasks before QS report in rcu_gp_init() Date: Fri, 2 Jan 2026 19:23:39 -0500 Message-Id: <20260103002343.6599-11-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0P221CA0029.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:52a::20) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: e81bede6-de6e-4b47-a0c5-08de4a5e6a58 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ve4ifGD9u3cpCwJBpIcHTiiVI9H00e/BCXcLZyeU6tLxqsc5duTE3hIEfGFl?= =?us-ascii?Q?YXGxQbZovhh9uD5cXYu+krDv+M+LfVNAFNH6uIH9N0lsRlQRr4iuqgLFyoOM?= =?us-ascii?Q?Rx0oJDQLALi/3vMuWJ+sRs7OMR8Urx7sBjQLSDdY5ADvMzILI1a9Yis7cyf7?= =?us-ascii?Q?jKOX8HL7jC/aa06quKnwTJhS7UAqUhnSGJiN5TN/HpVtfn/+XjMSljBmC/Uf?= =?us-ascii?Q?0G/vKPeTsrb7ABVeFOsqgdaJVMWhBcJOaXByo4CZ/E8wE6/x8PTmQvwV7rYq?= =?us-ascii?Q?Jex2hR7NMns6y7/QYk5ixy4kDLJjm45DKYPSD04tzrJFzFyU2yq0k4cnqYLo?= =?us-ascii?Q?03dT/M0M/No4wJILAP9arIRJQSshOZ4E+yKoPS/Bpbto0uHI9zJPALXRD6Kr?= =?us-ascii?Q?MtGmts62UxGagSne5xXYFfcrY+2nVD/f8Oy+R/mNp7iuUZIO2toXJPPjYHlY?= =?us-ascii?Q?/iFA0c+5JjTPExQFKcgh+6N8nbaltHUpN/oGi6FWYG8ucoAV1bYa9VESOJcO?= =?us-ascii?Q?Tk7jEC8K/A0bBnYWTSlFZ8goaMmQkRYgEi+jYZ4OKyli5zirtIMiZ+PS3OhH?= =?us-ascii?Q?kCXPmaFbJqOzg2H/mjMI59L2gtAOvUUK2mGSs4WMpb9SltIFdhM8lQjoav/f?= =?us-ascii?Q?dUoDDmC1iJ7yo3PX6CGQCFGijp/gpDpEG/miA0LzVaTr8aLex6SlBSgMR0Tc?= =?us-ascii?Q?qiqiJN1FskVLoq1STve1wSVLBxjb8PyXel575sHM7FvPiXozJ7VtHtUiHgIP?= =?us-ascii?Q?W5jw+ankulqDC5Yhgx2qxFqM6g61/gjbZ8pkfy9/TiCWoCJuHrWq2Pds595M?= =?us-ascii?Q?2IbmoAhuxK0LcOjpy9hpY37WjWXe+FG4ZdbMTYipvtMT0fb34P3Xe2FbJEPd?= =?us-ascii?Q?XgdjGb2Cj3JM5H2sUViiMe6PvGH25hDIpvpeZ017myIEUrxib+v1z7Nmpx8S?= =?us-ascii?Q?0CzEDoYqR8DUi/l42L0Oj83eQNKE01gJgcDzSBjkxAhEZJX9PwiYf2iCveC7?= =?us-ascii?Q?nuZSLqL7UGvP6wGCKtzsjh/xmG6zPrQQOuibi81LjIp2AsKKU8liBtwJW2RM?= =?us-ascii?Q?fPpmmgyblwdfwfw+OONCUSOUkvVDSuqaEaqhwp395fvU5VxGzVuRLhSKc7id?= =?us-ascii?Q?sE46kZfY4cBUXq+jr5DRENLOxCQSSpkNXTHWJ0tyAwk4N6UU8UCQ9gNf5ySI?= =?us-ascii?Q?NfjP9wi9jMMwC4rfechIY4EXjMWpYU4MKfUoODc9KnC9jsppEFiYmGDEYKDp?= =?us-ascii?Q?ISyOPpbQh8Gq88gk95Fl5nML/V7Y/PYGMpT4r4wNr5Rhqu+UdkTRSzkP4D16?= =?us-ascii?Q?tRISxHwPvPgasbPJe34iUkWCdxYrepw6SJeu05xTnoOfhChHBxmLWg2LrHdF?= =?us-ascii?Q?5KIJx24TZzQVGBYId9NOBiXNRgaQFyobvX2Ven/v2FF5X4SNLEbLKxDJ/TFr?= =?us-ascii?Q?fF1Krp8ZGjJ100UgJVQDMaIzOLhZ+SUU?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?jtP9EqMAalxMZmDFm8XVSerx5nx57PzG5Rs5NIokJNG07SbWL4xKZhS8CLWM?= =?us-ascii?Q?kauxSAPqVg3iSabgd6vE1u4dYFLoVKSAT77fWGpalObCxPvj2swzdip+xESq?= =?us-ascii?Q?JfHVpzfvoqH0k8sJ5vs8BoGEbPBz7CCZowSQ87V+zHVakMKSgpZXHVNW+Hbb?= =?us-ascii?Q?ON93IaxD+/diazxqsw8bDHb4ZETf7qOPGMZFnlVonaPdIS9qANCL3dXoJ6SD?= =?us-ascii?Q?Dy/U6X6BZn1vUTmUkaQqMT54FSYVMqYUJqKDQnUnURZUTKDo4ILgHUK0iiWd?= =?us-ascii?Q?RGDBpyEEAavIwSImpmfdLF9aUTel6dWay1p4N0hvAPycGJ2j5Xd1t3z6relQ?= =?us-ascii?Q?W2+P4qY9CoIgl1D043sRZUzNjS/5HWTHcjy6oFvky74PZB0JCEyKlUta2oU5?= =?us-ascii?Q?IB1T811LgjaxsN0n/9CY1HtLeK29ldjSS28vSLe88iqJO8sgrzGpGaTutulw?= =?us-ascii?Q?D9ACbJWlrL4go4APxhCVXtZUSbi5pFku1pog7cvsvZd5kBBT8Xwt8Jrqe50F?= =?us-ascii?Q?tMNF5JUQRyzQ3VXC/2bV9bod36C6Qsdogw+zSAL8+t0KbjrAcn0fhp1Kj1jv?= =?us-ascii?Q?/eb57c6dtSfHDq41GWhW+tqbr4oskGlHlYB9YRHBsfD84aN8JdO+Lk52ZfnH?= =?us-ascii?Q?mQQltXBPvJQDoXcVUrohz1asQCGbS/XIzj8eJ4LVdFuZaHw4gTHXTpOBLBl+?= =?us-ascii?Q?OUa8KawgPs47YeLNSmnAkHCymBvzw8fGcNPQnHOBakRZ6WrFVN9QrHja4w3j?= =?us-ascii?Q?sCWiOUaVmjU+gzzoTfuYqsklmqwCTuIs+RgP1uQcvJzmPGA7AF0RufzlxhER?= =?us-ascii?Q?qTbgKiaJksUVw3GJMaYeNczUnMq2iK1/blSE0hlewJPpIZ5obv+M328VczMw?= =?us-ascii?Q?RXti4lEyq5NSDc186+TfETZFEzp70w3YIMNCwlEmx3hlgSJ2SlRKyBCr451X?= =?us-ascii?Q?61MYNcYl+ZuajxIriepalWSLnvdEfGWMOddGVv6rnzPy5G4yeUJJmrcKQ70b?= =?us-ascii?Q?Q0JlZIOcCJ/koz4ABP0LM6R0th+OG0u+n3QOgEV0fCj0RWlK5t4Mqz+zspzA?= =?us-ascii?Q?hXJlhJzHSERiXXJ4JPlb3X2+AmSaHChjWHxbUS9OA6//UJBlNHj3TRLyqjkn?= =?us-ascii?Q?7H0ws7DdYqSCwSM2QE7j88GU8SELZ/8LuR0K2kOEEEW8KUbNYK5xVHdfUAo9?= =?us-ascii?Q?GAPqa0sRE8O1PWqULTcGEfEFXspNmW/Q48zcqnSF1Jn9CM49bV6YUioGNm06?= =?us-ascii?Q?djUGb+v5+E9xGqhD623CGPGh4lz5ZtsTz92gEXybEwqI2MqVeh3vTxoZ4Bha?= =?us-ascii?Q?2vG9JCzAMkPWr855lcJg72V22mxo9EppciU7PQUAV0bgz5fjJ6o29T8wBi4i?= =?us-ascii?Q?8NQZMs0O2vxI3tKupkj6Jwn4MNwspFmpNldGS0mFXAdXxYZdiDj+4DdTP8dZ?= =?us-ascii?Q?hhh7zH32IfhPALdkipuO57hvTJgd+bDR7CxTkjuY6ifVI+ThJ0pkArBxkELX?= =?us-ascii?Q?awEL/wtZFO4+2R2MvS16QkVV5ShZplLTnPXb+tWWScfsjDEvcdmIKckI43vq?= =?us-ascii?Q?31rmdyHlofInkKAkl1xSJNbsJYZhl33Go5CjI8xub0f3i8e25e0Os9C6955k?= =?us-ascii?Q?HF5nD6mEa6chJkO90kRIIT6onusCpgzoOv3z3meh2KrjCHEecnN9NOo1kswJ?= =?us-ascii?Q?gAkI+2cAV/UHleKcfzivw4DvXXR0wEvhxiK06hf2GCNbEJHnp1chLwV5Z+lA?= =?us-ascii?Q?MFIEJm9WLg=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e81bede6-de6e-4b47-a0c5-08de4a5e6a58 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:10.5880 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Rm+tNjwtPA41yHeWclEokBZV6k6bnVehVoqVOGkLpDSp3qVDF0Q8+AhOy7hOYs4R3qfTvpV+lsfkruWNDLZibA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" During grace period initialization, when rcu_gp_init() reports QS for offline CPUs, any tasks blocked on those CPUs' per-CPU blocked lists must first be promoted to the rcu_node's blkd_tasks list. Without this promotion, blocked tasks on offline CPUs' per-CPU lists won't have gp_tasks point to them, so the GP machinery won't wait for them. This can cause "Wrong-GP reads" errors where a GP completes while readers are still in their critical sections. Therefore, call call rcu_promote_blocked_tasks_rdp() for each offline CPU before reporting QS for them. Signed-off-by: Joel Fernandes --- kernel/rcu/tree.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 5e73ebb260e3..468388970c98 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2001,8 +2001,21 @@ static noinline_for_stack bool rcu_gp_init(void) */ mask =3D rnp->qsmask & ~rnp->qsmaskinitnext; rnp->rcu_gp_init_mask =3D mask; - if ((mask || rnp->wait_blkd_tasks) && rcu_is_leaf_node(rnp)) + if ((mask || rnp->wait_blkd_tasks) && rcu_is_leaf_node(rnp)) { + int cpu; + + /* + * Promote blocked tasks from offline CPUs before + * reporting QS, so they properly block the GP. + */ + for_each_leaf_node_cpu_mask(rnp, cpu, mask) { + struct rcu_data *rdp_cpu; + + rdp_cpu =3D per_cpu_ptr(&rcu_data, cpu); + rcu_promote_blocked_tasks_rdp(rdp_cpu, rnp); + } rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags); + } else raw_spin_unlock_irq_rcu_node(rnp); cond_resched_tasks_rcu_qs(); --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09DF822F767; Sat, 3 Jan 2026 00:24:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399861; cv=fail; b=h7sudyg1aImsmz5MSlS+/GhO61xm7KpEaub0ld4BmrBY4YTcU2yG2l2gbfd9QCBLwAdGFucNtwG1auNtp0zn5y8ggtYxYZJCoUIr8KCn5tqeML44UdJUlGsskFP4adXwplls5+pyEFZiXI0LOWgqiSLZkBFKhwPHQqIUxHpyTUs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399861; c=relaxed/simple; bh=jEWbHUjCwphvHrvcoVoT77xXsMIhyhbuKUFFK9T1VOQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=T0HnuPbhMLIPPW+T+yY+XwcLhzNgKflnaoaHbxZy2Nq/A27yFg1QWZeGcyrCJeII5sX8O8t6i9W1olDNGFyuD6Egjv32hsw7X92WgcrjATehfjwWTP98OD+S0m8rDU/KOKKLq72sd0BTIOYolY/B4CDY2DaCNFTnkLrBuRz9UFg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=EFgAPbbx; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="EFgAPbbx" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=U/MZcPxji7SvLXgVL7Ew+6fxKI/pL7eRNKQOgnIlt8Ke7dtZ9Bkm+5zto7h7yEoXDoRZhk9RSbSAMw3T0HN+AXvNIWwMk2NkjDSDanhRJviGnAYtqvZCfhBJAskMxoCRRCXK3f3BmgNYN3V/rW5pmmMXLfPRQ+b1fGre++iniBuF2Wfwt4aIv77mQ8dIQ0lVUf+brSCKsL2cDwGku110874ji2qLGIVO8ujefLPKGTmI4FVRdTb/DOokcBL4pGwHW9cmePltjcsP+ScQM5f70o+n8E7d4Ndfnn9lMU/T6aTIKTtU1hl2wxeyLgQaOglv1eLoG06107aqqayMdUyA7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EiAmPKH8lCQkwA9G5ucoxCYPHfi4lO5/dkSbinQzAAg=; b=R3DZOhHduoWugE4U9DYIp7xvvny5FyjBBEaW23v56AsDEy0hwYe3lWQdRNLOG7TPWK3247Hu2FFo0kXr4lqn/7QE5JhXq5j7FAPwevxLnbgaC+yR/gGbiun90i8baKGWjl6KJjUPxWklBcZHQQe0UCIkdYJxC5/OvpSFL2tXn59yHhjYA4kbN5HiYgPqO2dseGu+sIkjub3NqHaYTsyKQngVCOT6LMfE91kgQPWUBub6ZcpmozdoqOU9Ihua2tDUavUQnJFN3JWT3i5sr1EfPMI+25WutEPkh7HIVG/hDNCzDe4vXeYKZQKfcyaE6wOs7+V37XVMOHBHmfiK9l8b2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EiAmPKH8lCQkwA9G5ucoxCYPHfi4lO5/dkSbinQzAAg=; b=EFgAPbbxVwLfV6T9yKLI54tN4vVMfU5KMLDliHGdCKS0wGrY64AHh7NubtvfZFx95KKxw8b2YyejSVU/4gD28SoMzyYIqFsxiOBHs6ZztodQg9JFeSem8iIBfwIb2Bp+0zjm3fQFyemZAAb2Ket0uXg5YDc27sDegIMEYw+0Xa0rgPkX8Zh4pU1hKpA65/caAVTZ4ohEJa3I5kO60Ouq3OPntL4YSr4nnf8hbNcklvj9vDup139VXtr5fFiDWxwH3MAPoW/n3uuruYZNNefE14glVbegzUFXxv1NfD0Yp7N9XnTQiqme/y29Tm+FzRffrxarhXs8IPB2bkQFmvNH/Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:12 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:12 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 11/14] rcu: Add per-CPU blocked list check in exit_rcu() Date: Fri, 2 Jan 2026 19:23:40 -0500 Message-Id: <20260103002343.6599-12-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BL1P221CA0024.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::31) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 2a740cc7-2aec-410c-6dcc-08de4a5e6b87 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?rb5zvQ8ZzHLHBByhiYYmcjCpo/GF011/BLnSzaligzYVdzL9pM0k/xHGub+G?= =?us-ascii?Q?xEUerVPz+kvKfvywHGhzVAdhl0zCfrdIPnoIiV59tH2OxQS07uoY5mTqvvkM?= =?us-ascii?Q?a/tbi+ot/c4232mUGbvZTaO1UYFMtfmlv5k5FHwMCaiS4IGrS44LH0wGsx0Q?= =?us-ascii?Q?M0h+uuaVAAkX/WPYr1kytFJ3QaVGMy1jf6YCKyPUi4mZ8vBec0pYa1OQb8sZ?= =?us-ascii?Q?3iplmtuo8OYonqUktjFk0k8Lkcyb3ri1Ao1LZav1YTuRvzB2lfdPz0k/kqC4?= =?us-ascii?Q?IOOAEirC2+W/rYE17uHVKHIlNpcF+3tBB23RAoDFqQhkRSfCpP/HLx2omCM9?= =?us-ascii?Q?ztTPTcI4tVJ6EAr4OCjFzdj7TL4xNvV7zUu8T5S+kdNgynYIBOmdOwAoEzFC?= =?us-ascii?Q?jl6tOy0ijB64nfusNfMrFljHngRo/L3d4SzGVQ07xUyWJ+8vwDctZkiwNYdQ?= =?us-ascii?Q?r5Pb0JuqW4dio5SdZ74wW88VE/rqakreb328jU8GUKPgunObdQzvK8OgbyM3?= =?us-ascii?Q?59HTR8B2dr6o+VT4bH6/zbXp6bIsVHOtItQ0T3jcRqBiGlMW6vOljVOzs28K?= =?us-ascii?Q?9tQTM9aHGzqYlMbkqux/FUBmFJ5jlzaDZwY6YgkNS6qFRiOsJV7erKmYDU6G?= =?us-ascii?Q?JmuCBZfJ2WF/T4ynK8G5P+m3Vo6hBCwdl/1kGpy84NaklRtBPByJ87GI5lrw?= =?us-ascii?Q?ZclvfZjalssUZOBUzGXiTeosBe84ZGIUegJDVbTnaSEEA6hjWszzyFu6NpBK?= =?us-ascii?Q?Wiplecu05hpbl0onCJCfvVQ4r8Itwg+/K9HdkuNGLuATR81sI99yvxmSZyvX?= =?us-ascii?Q?9PR0wt7xzay8KIOPTPEE3TJ3gzmJYxy0KVMQYovYi8vhgaPm2PbxGdXWyWzU?= =?us-ascii?Q?qPd3XAgYkwET/VQvT+TZIompn6zAKwH0EETeLr6XWOwzH91+nVKWbqIiSDye?= =?us-ascii?Q?LpYMQDsBxF7GETIa8zUUOUGFI2xVZphwObNKrhurTkNyS38NLOPeA855RBKv?= =?us-ascii?Q?HKYOP97he7ltc7bE1/ggn8U5TUFMVnt2XG3ciUNV6EHQSr8CnSGsinvhuWiR?= =?us-ascii?Q?H3+LoxyYgmbPT+eJf5r3hf/OrX7WjsHw0a6YG5uH/++8vWmVoOFE+e5OZJp3?= =?us-ascii?Q?Bj+b5XqiW5SFcM+6eiKpRTckk8eUoJkrbemum29eRu/PW77i53hU78HTD9nT?= =?us-ascii?Q?bTTQ00ZHunW50JEO+2tjS5AOjjpmP32jwjkWC8L3FrV/WnJU1W9/fOwUy1v5?= =?us-ascii?Q?6vN6iaRUFwMn36tGa8mqJemaO6cZTw74wxBRll8XzB1TH+PBwiSXSV7c0Rnv?= =?us-ascii?Q?DhXvmPNre9QmpP3C5TelVU4UadT23go7k5dLCL2gxRG0Qv56JjDAXYkIjWEn?= =?us-ascii?Q?Tb7Cy/efCAT+1dxd0gKSzyoEZSCgzc2Pe2ssk7fMDMKSS6qU/PdRSsw/Xilh?= =?us-ascii?Q?2CaxFAmopJO8ol0FfY6tKxAXT670rR9U?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?HyFkrN3GSMe/LQyRuNwQFS/6JlmhvkmxvY+YdKvQwhpOXQaqNijV/fkPGTv0?= =?us-ascii?Q?Ulxeon9aVUJLT3bvhLqLpGaNTy5JP05vXNl7lZhgR/PNNVb+YdOKamzk8nH1?= =?us-ascii?Q?8J1vpbLPtbyLZ+TtRR5uLjJXM3p/F6rnUudYQ42aTU+aNCthU7qb+c4Upe+G?= =?us-ascii?Q?xTyDyOcYMwEQiY4Es9Y2ZA2Xv58aF1ljh1av4BvjT2CgKSQnFyDTmIl4QCO6?= =?us-ascii?Q?nfnnkFjRoz2bmz/DOHevropWaI02ABTIyBwQQgIZ54i3zYNaXF8B+M82SZEt?= =?us-ascii?Q?aUs7Ksj4RK8pigOUOnZj1oVf2VnzunLlnVe2wtaR6m5HID4pUD6iuyClybyf?= =?us-ascii?Q?OyW666E3kwpNOJYusSdnZDC5gPMj2bUoQHRNuhXhEgubNh8kNPDyUp0OX0MQ?= =?us-ascii?Q?vsavoaBiiOCxabxeSaFNLEIkgLATJbvmnStv9sTzcqgGELAnjlL0g+mkGqyW?= =?us-ascii?Q?yAHEOEDR4skmE5BtV9abDTLkUps0s0PaUibS1bT8hPOw9zm1E8U76t/Oy5Rl?= =?us-ascii?Q?1KUVVTYsgWgYiEX+xoPHPvRUldfPCYOAGJjqxMsmJe0AwF+k+oamrxqu7OEo?= =?us-ascii?Q?DrHRCPG3jIjQUgVQc7IQlXmGUKKTLF+KK4zwcAvVdA9PRm/7tJUJNoF5vm5d?= =?us-ascii?Q?mNdYJbAWu0WcP2ou12X44zndDFEjX9qJnmqRA3F/sCJ0W7DjuFYR7usKOCXz?= =?us-ascii?Q?qqzQT1pQOtDBWGkaka1Jtm/u1fouD84lk4PFfUrYoyZmp/Ihj2XTmQ0igFfV?= =?us-ascii?Q?Qs3XUFfVn6AiNoNpC5reQDntxQS8HPPGwRqL9xMGPLSM6GJqvBIvXoqAO/Rg?= =?us-ascii?Q?KzQmyIk7bdvFrbjh6cZ2uZFfMvJBy/trjsWbOZNgrVJIXUFcBrhmuYr+X/qO?= =?us-ascii?Q?bxBJ2SsJA+iuZu9oLs7n+L0Q68UI723qfZsWsAZGs9lBmJpVqTY7vVcPqBzT?= =?us-ascii?Q?XgR5ZwXdzvUw20dNjotH7nUTKPKjjvkICWj2yLz3T/gsdvRdMnxgw7Z1IhD7?= =?us-ascii?Q?OV+7vRHxjs8uviRAPVmEatIzVbRoYUcHvaQ9xJ+e0isjfFWLidh/cHfHMrbG?= =?us-ascii?Q?ZiXn+BlbmUAZcDGqAU7cAEXfiXkmExKTfJqkEKlf2hrVhdU6nYqg6X7v0fqy?= =?us-ascii?Q?YLNh37D5vsdkPILhxEpwM5ZW0V1Wce2vWY+x+Tm/xYjI4HljNNYANpVYS7sg?= =?us-ascii?Q?c7AlBU7bC1crIVETTi9QHSzfPt7nHaSwVClNn75LavQL57I0nus5HrF+JEyD?= =?us-ascii?Q?iTRK8coaKujy+mCuG1W3Zuu94OH5J2VsqESN63IDKvIZp5gMdFjoS5qPUBUM?= =?us-ascii?Q?o94RfEchHttYXy0wC6lnnmReblybgxexaBLurqFmzC46Z63ahB+q0fBL/L1O?= =?us-ascii?Q?DpVaoKM8PFtKQmjfL2Su2A9O3tNHVbFcUpgF6BdvJB99TPi/zHsuO684Jv+u?= =?us-ascii?Q?J6tOd8bOsbfhqcJOesXcvxQt9MNi+KqOKm598Z1xK2GaMtxsT/H6Bcg/JBb4?= =?us-ascii?Q?B5AiO1Jfokcwmh7WTUocTqDHG3bpBjvZyI3Z7Y3x5cCklh3yIOmi9XNuQea6?= =?us-ascii?Q?4mdXHoy0uy1p9xDPqpkX27foGJ8RyW8bvyG8CLCnXDRao7cgz7Cy9fe76Sy2?= =?us-ascii?Q?3rZbyM0ELtyxPE6s0Zn3q+2VJcpzIHMv5brTQ1zsKfQzMz6FBNiniHB8u8qp?= =?us-ascii?Q?tdW9n6XYXRaCIV6vRwFp2czA9kMk9egyM77D3GJnil5mpK/DimgbuEbHttcp?= =?us-ascii?Q?x0ojXzRdcA=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2a740cc7-2aec-410c-6dcc-08de4a5e6b87 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:12.5372 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: o6vRALycva02/F946Bw7va7tmrnVXCBIx7TkCTtQ/NEDZa3K0kApmFVjev0IlLRPH8gtu+WjoBB6elJ0VKhUvA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" With per-CPU blocked lists, a task can be on either the rcu_node's blkd_tasks list or on the per-CPU blocked list. Therefore, extend exit_rcu() to check both lists. This ensures that if a task exits while on any blocked list, the cleanup path will properly handle it. Signed-off-by: Joel Fernandes --- kernel/rcu/tree_plugin.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 6ed3815bb912..8622e79660ed 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -978,8 +978,15 @@ static void rcu_flavor_sched_clock_irq(int user) void exit_rcu(void) { struct task_struct *t =3D current; + bool on_list; =20 - if (unlikely(!list_empty(¤t->rcu_node_entry))) { + /* Check if task is on any blocked list (rnp or per-CPU). */ + on_list =3D !list_empty(¤t->rcu_node_entry); +#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS + on_list =3D on_list || !list_empty(¤t->rcu_rdp_entry); +#endif + + if (unlikely(on_list)) { rcu_preempt_depth_set(1); barrier(); WARN_ON_ONCE(!t->rcu_read_unlock_special.b.blocked); --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA50822A4FC; Sat, 3 Jan 2026 00:24:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399864; cv=fail; b=k+p6OZI/FQursT7bw5ZpGGdSsZssTYU7UGkjtIsw1Lb5SrSkiLQXFn7v5LTj+KU04sKZwo2wIaOrPU+EAMo3UXH37kcEbri/d5e1iNh672yjNPwt3ACSsZ59rpCU8NHuJHMG5HFayzBgqfZRT7ypaJNe06YrnHApXKQAxnuJxB4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399864; c=relaxed/simple; bh=7G4Df2BE8oQqTwqd5jiZu9xWFZJqsy7Eq6V7SOvDBa8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=VCVWrOLr39ah4rWwCqYUKw+D9yvBMEfTuDtVkFawlYXXjGeMM5pB6tSRPZiVLxctMypcU+x123I6hMkRIsZJEN3GuvXxwQ0k0CnK9gGXEaA1BjM4+0b/M3Chf9SNr8B+62XhotoqBnHYl1cfPPUJd1WWQFFlUkk7D349CukH28Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=pmRpH5bR; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="pmRpH5bR" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GRwwkIYFUxE6d6VjxD2lTMvi/LGaKDEWb3fgyTHZCfStbknlrGenMNjbtFWZ0tg5KUmDwzdKG+H1MUi4HsZ/zxsVeJTrXo1f75G7AT7goG1SKQEJTrjJa1g6ACqocr0E2xIZeCM7qyFhPI0Q+d8OdwNKXMUsOVSanlDMoNPKdgw1IUCWj0ebmInQh91biW3+np1DDJlnZ6XwiciVpluvLj+ti/E4bK++CP13TFGJTkGGN8/aZX8weNoWRutTR7XEel97PrPxmnD2EBrgIYqoh7m6sS3jNHP2ekmXaPpJkTMFWKIvHYaTfkdGWYyX98C8xDI9aiCQ4OtaGjzC5FDd/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tpyYOXXYxhw6wVA4IV8amj/dHk7al1UPtBbe5IGyY0Q=; b=MF7oQ79GOHCeTlnXGPjpA5n8McRz17OEF7SGu8AgjrfHEk9uE96WQ1LwJB2bGedxS+4bR2q08f2xsVgFa+XDKLJCS3e/TQoCuxsdLjAaq8OOPotoOqnnRHEdKj6KQvFiQQyDGFTMyjyM8dn7lKa+XjV+qoHpkxQv7mb+3M/pyaL5cJSvCD31lLey7OAzdYUzv5QfXtWROeaNq8M7xPrK1i/kNv3Shp0hhnhfXM7oDhKmU3HBwbtYZ7/ccc4UlB1pxpbdQ3VaDeh1DsK3RV1X2RMrSAcqYZHDQqwPD8D5ne7L3f31iZmQLntL+IDEih2VRFjAXTlYc4odeLmhTg4OZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tpyYOXXYxhw6wVA4IV8amj/dHk7al1UPtBbe5IGyY0Q=; b=pmRpH5bRzPlsHH/9FRVGyYHiNwDzbpg/26F+7ctk/wrO3O6qxqvDAH1FM+H0f9ygCkxd3H2pXZtGuL/X0sh8puN/B5AWjx1XCXFuoRXiXVOYKKsAgeFsiB/3P3DTrLDOSMJMSJCnOLkPktTZ/hx2Wg5zGz2GEXzywsKjKcuToUcYIECoJd+gtaDWOSPztUXMTJJR+ZAXryO4gBpAC2fLy6GgC5vMfEttkY97kij5SWKOIHy29ffy1St/4oBcLkz9Ac8B4Dv/ik8PXerwYvThsjIb4LWqossFCKfftwPAMHdRwAYWk28xtMmLpFyi60HxLpuBHJQhfDGIztQvrknaYw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:14 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:14 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 12/14] rcu: Skip per-CPU list addition when GP already started Date: Fri, 2 Jan 2026 19:23:41 -0500 Message-Id: <20260103002343.6599-13-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BL1P221CA0009.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::24) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 45399d8b-4b95-4210-ae16-08de4a5e6ca3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?y3DyquioVm0LQq8AxTdT3OUSeSqursFoVGwsEwZ8AYyYC/Ua2sqYmkuaHdhx?= =?us-ascii?Q?eGZ10oRRNzUjqzmGeGbV/TzRG6XU09YR6gfOSThcpZ5sLKjcViF13FpF0fdM?= =?us-ascii?Q?4BYGzafJ2/fL4uoHe9yfm/2/og/L1BiNF70nDiYWxt4uaB/tqxckYmzEkEdK?= =?us-ascii?Q?qmpC41n8hgp85t6BoMrMveTYtPbgUiamj83fCfKkzvqKnhaYJLnAG4A/M01G?= =?us-ascii?Q?oS555wnDHdx5d35CX2S7Baj3j2UpwomwRQgtCm5AJLBZ/JU+z4LATyAH1M1O?= =?us-ascii?Q?expLwdfZUfRpOLP7IOdFwaNOmyiSizaOJogdxyZEpzrR0PbAJE+/EeDRikbg?= =?us-ascii?Q?EUIjh+xrbuBhYBxKVM+Wg/zSKXds8w0cJnEFIWpn/1sqmkQrKpY/BUjxnjxF?= =?us-ascii?Q?1oJOza0OSu5znnd3uTd9QlnnvGDAkDxQP7nlg0BPW9wvQM9r46gDzlwbRNF3?= =?us-ascii?Q?ED03OOAsYiPLOrd9QaERROKDWY9RmDJcKpE6OvIZ7eC2hF4SnFa0APap8Zjk?= =?us-ascii?Q?uzx+2DTd1GdqNYl+gbWvjVOuvPYSRc/zb8ghzodRra1yM1QtHi/+Jo/3GQjI?= =?us-ascii?Q?kiySYfnbf8pbHlDgKGYrcKdkQ8u5esc3n6+KlVhSyihpMTVZIP4rmkqEHpln?= =?us-ascii?Q?fgvZ41g0h+3MKZHXg/DMvLEFI/UVKuEl0O++al513olESMKnj0hzDtGtpu9Z?= =?us-ascii?Q?b4+SJLzY236mjLIuoIDLmzEEau2g68yjyObIafrbFe19rCHrWPhot59/krVr?= =?us-ascii?Q?lJNGFmQW7ivISF6i1akN/M6S2gHQgd6H8NX1sor+v8i4FucMynWZr9hO4z5U?= =?us-ascii?Q?0JkBcXrNH9dzSuJAE3zEnevSz2NrCul0IgKvy6XouSdf7lVBZwY1anpL1OEo?= =?us-ascii?Q?q/BtPudCeLmzwOefbl3JPI+rgTLkoKEihQm728XSisfF5ZgT2/F5Fsu6lrHt?= =?us-ascii?Q?VW33GVssQfhq+zkGtFpM2cXcJtejFZnBvw+hOviKamlHpCg6wd0fIz5SfPTV?= =?us-ascii?Q?uNwnfG/7lh7hIp2H9LZEAc0iORIFgf0h5zaFLP4l8FHxR4X73Nz5pS58s+YZ?= =?us-ascii?Q?vMg6zKz3QJEKsJuYyV+mBOteLEGNAL1JUt2D4BOFcQ0gzzTYqWryaR3LAoYF?= =?us-ascii?Q?lbwn1F0KGXa8HKsAALRiE/Ow5na0vAz171BxruW3jfP8gjQdHC1/YtomAybp?= =?us-ascii?Q?ICWqFCwu1w3Plwjk8sqpxKlQBcOu4b2OQ3PwFECc5A/OAvMw7ep+OIu1fWd4?= =?us-ascii?Q?5ANBId+ZX1N0YHP9O9DA50PO4g2WYKCK3xxozAh438hLEtNWg7vTRB/osMFs?= =?us-ascii?Q?kc/0Q9O9M2ROTYqT483d07sqReGe6YBpCDt8VEryvg18D2AQ7od65H8cEBeM?= =?us-ascii?Q?ZQGo1x0w1Gu997to8skOGj+5jMdH7L12H3ROVSnaY8ZqD4Gbsn47XY2XJ87a?= =?us-ascii?Q?dIg1TS/BoDIUv1jnsHoMIxiTQZQWEegf?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?mpsmgnIWNzfZl786VlXvzBm7NpHsViPSZxQF5JeLkUNB7gsqFJoTzJgpT0tC?= =?us-ascii?Q?qOr9Xe24TsOi7OVN5pYHAJOPYtZoPYVfxMpw93nSt5h3w5t4ia8QDpVuLWyD?= =?us-ascii?Q?nK0rHqKOCCct5DTmpnYpRIx1342PjA/2+mC5WRpp1oLYR354hf3SH1elc4tO?= =?us-ascii?Q?9hsC5S7986ya/G2rW1KWAgQOknz3jG+YghzHOFeeQYnwtRjhjbPEaFL6bbWE?= =?us-ascii?Q?pTr4qXY+CVZv/G20emcm1sk4VSZhA2x98R5kdj7r5wj8Q2mBjdfrNok1EOlM?= =?us-ascii?Q?VuqDdnrN5JW3hE4errBNYONY+HNevzeWVz2gztR6q8m+ELEKJ2CWMa0a1zhM?= =?us-ascii?Q?BdJIhwq3dG1eHnzKsMCr0mEm4om5Hj5TjUR6O6KHYBtx+EHR5f9Zzj+jhZJp?= =?us-ascii?Q?/7psSZl4WGObJ0WkfI8z7I19aqumqW4d7ajmmUq28NFE7Lagt/i1J12U0FY0?= =?us-ascii?Q?H/XmYsgBl15n96dSVcvV/z/gCEPNkPgPkC+yCw1P4SO+WRbInRKWXZhMCXen?= =?us-ascii?Q?Gs7UWn39ZbamSNc9bdzYlRMDCGbBIAXepaPH2i2DfeyeuT3puQ03N4OAqRUZ?= =?us-ascii?Q?IgLxoUbwHZTLCFEUkidwN5VSsp9oqAkxfhxi7MxXg8s+M9Su1dracUeBBxHL?= =?us-ascii?Q?rHaYz7bCOFdJJtOi9lC0hXUK4nA8WygaHyot+hR2HVovt1rtZhOjtKTYwntx?= =?us-ascii?Q?RQgvNBZbli8n2rfsuCknmUM1g0hr8nad2XZ8+HTF3vIIUFwr8wU16X8kMJx4?= =?us-ascii?Q?WeJ2IujsdzZT68H1tv5a60HGx3XOJFzIOB1NkzPWXlRUPpMXn96J3ALKhBXS?= =?us-ascii?Q?blPXjF3heXF1n7EQN1Uos44qr/P1A4Ng5iCmZck/TbIqvdPhcZ2MS89VCH+o?= =?us-ascii?Q?S2zVYVRpU6rKulyUX+O45u5ng1mSedg5pFzq2X7ZlZP1V9e0ZNtQw/nWVdhh?= =?us-ascii?Q?tif6srSmya1cSdGbV8u8/0/PIN4OVXRU7I9rs4IZ9Mhu0TxcOTPKGuqDHl3Q?= =?us-ascii?Q?PlSdA15r0GgfBocLnZ2iPWNYmL17amvvv9POwhLJqZiIXTEZkXmxWP+CmxE5?= =?us-ascii?Q?YhD+ADw9Uu6KSnsITnh7BgZyGl/XjnYLb4ZvQGYbk/iQdwhHH28506fy7Ak9?= =?us-ascii?Q?wT7/+d6VY6cUHl7WUKsjby6lZxnv6Pvi60b5wF2Rd3eDoqiAhSwxYDjGPbm6?= =?us-ascii?Q?B6gM1OwZYdAHPyRwxmvQ4JHhgODthvxUYBQ3xTjCaL/+6NgsqrGrTf8wmExs?= =?us-ascii?Q?40SL773g6rvQryF2zSuh/RYCNW0WRoY3PiM003z6uzGXsXg98nwtuxmFfcSl?= =?us-ascii?Q?8a4Xpi9kw0BDw07dGT9B+rrJB0PEEWEBo94GEF4Nn4S9GD103fZOKUP7SRpi?= =?us-ascii?Q?1i1TxbTx8JpNhCYDiwduRos9uff/DEQzMithesCJnoUiRQ7Yuw52mJrCv9Um?= =?us-ascii?Q?Nw6CstxOiXWDgp6nyZtMCLibhL6jSKV6OiNjlbU475jB9LqOJnb+n0hCiETZ?= =?us-ascii?Q?fjmcA2Auz4x5ZTmX47DkaWQCDa4pTPolwoESgACc3dJkO2/35X1ozsGkWPmI?= =?us-ascii?Q?I42lCaeS52nyscxashPYqIvzPMLENcjhdURkAsVQnA7EU68zOzpk6lgyjJPz?= =?us-ascii?Q?KewTaGBHdmR0DgtiLVJCkXRsBcf7xgo5jLPfqB3+3N3cQ07Hk9wVVGeSD+W5?= =?us-ascii?Q?Nh/1W6d/SjdFuCTAa3deuSBHat2RA0VUf3f5ui6wQl59TomZVeYynU1ZSuW/?= =?us-ascii?Q?5nvo37eJPw=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 45399d8b-4b95-4210-ae16-08de4a5e6ca3 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:14.3827 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SSI/U5eqa3UtAwSqUgWUKE1MjTsUvvjuV/GqdjDGwdWft1AUJWiSEz4oJ/6gyj4SGH+V/lN7kbTaX18wy2WmzQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" When a grace period is already started or waiting on this CPU, skip adding the blocked task to the per-CPU rdp->blkd_list. The task goes directly to rnp->blkd_tasks via rcu_preempt_ctxt_queue(), which is the same behavior as before per-CPU lists were added. When no GP is waiting, add the task to BOTH lists as before this patch. This maintains the existing behavior while preparing for the next patch which wi= ll skip rnp blocked list addition when no GP is waiting. Because the rnp->blkd_tasks handling remains unchanged (tasks still go through rcu_preempt_ctxt_queue() in all cases), this work same as before this patch. Signed-off-by: Joel Fernandes --- kernel/rcu/tree_plugin.h | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 8622e79660ed..d43dd153c152 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -339,10 +339,18 @@ void rcu_note_context_switch(bool preempt) t->rcu_read_unlock_special.b.blocked =3D true; t->rcu_blocked_node =3D rnp; #ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS - t->rcu_blocked_cpu =3D rdp->cpu; - raw_spin_lock(&rdp->blkd_lock); - list_add(&t->rcu_rdp_entry, &rdp->blkd_list); - raw_spin_unlock(&rdp->blkd_lock); + /* + * If no GP is waiting on this CPU, add to per-CPU list as well + * so promotion can find it if a GP starts later. If GP waiting, + * skip per-CPU list - task goes only to rnp->blkd_tasks (same + * behavior as before per-CPU lists were added). + */ + if (!rcu_gp_in_progress() && !rdp->cpu_no_qs.b.norm && !rdp->cpu_no_qs.b= .exp) { + t->rcu_blocked_cpu =3D rdp->cpu; + raw_spin_lock(&rdp->blkd_lock); + list_add(&t->rcu_rdp_entry, &rdp->blkd_list); + raw_spin_unlock(&rdp->blkd_lock); + } #endif =20 /* --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51D23225762; Sat, 3 Jan 2026 00:24:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399866; cv=fail; b=BiFaJOcbYj/Yd4SFgEW4Q6g2/S1zozAnttXuvtqFrorDMBtXeJHtC0fFIQhM+qRJRfqrsSSIz7wlpmi5D2eICb+1BIZdCUH8QQda7HnUBgIIz8IvmR876ai44B5nzUwFMT6XaepX4X+tUGgJ365VL3Tz2OdBUiMWbmZuU6fyeHk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399866; c=relaxed/simple; bh=cMeTEwMPFmXWhjRtlPV+wwoqz1UHvOSyRC8L7TSBT64=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=THBmzgNB4wwcM5wo85Y4Epye5gkulTE7WGwIFh9/OIilS1ep3haR+Kj0npUdXmzN1MqAeGemD4McuVPmHGC3kUrarkmJVw3RMHn0fCDIjlxxYr1rH0BUpAxDpk0u7vadlbnrq11KCITas25yFzpHBQH9ekCO9g4nsQTzb96MbrU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=RNAWJgTj; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="RNAWJgTj" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SjiEApQ66/Qk4EZrqayoOZU0CGtSCzNEUyh6JHABxcbx4+khWvqdwb2aSU47hxsKPSapECeJvv7H7lugqyQ/10LAcosC8onYUFEgdScXeP4T/CghBFQRemuJIgT9nF9nm4LdYVB1G2CnYYzxHJLMnD+JXMBDt7bdzlzTQBK6hf7OV6gY8IwclUh3qh6p0lRgpzDX711ux9fxRy4OYWoYiNFMBGSd9/ksFTOV+o4WEkL+OKziOJEDxxJD3ESZLnEtID6jGIA7PQZwMRGlojrYbKUSmfLsz+65e5dHo4VkuFHV+9eF85aj7uFjgUjDR13cD0iLmo9Xp5W2O+uFNA/+WA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=i+mTgRnY5Fxs712rqui9HWx9XxSvsOL/1QRUm7jH9K0=; b=Ypfvrda0ak1g8EWCdMTPZaKi8cNZuikuiLaVHA4FGRDTK56Jdd5EJzblfzUIeDwQiRc4kO544KTMh8wc8RZ+hIswyrdd2s+i6gdtROp1YFSeBDKRHNf5IAlJIAdekpcbR27iYyWZzXax6kIbPaw+bqiggJxCrV4kxMbVYa/KZp/Lyu3kOe6PLnJ64BDxqRVHy6tu26ucLlPZhcnU0iOL1cDEd9rY4pj9R/EQCa1l9lio/Hacovxg9BU9fHBTxFBojlHDJUty+D2shiuStSQVCAErgbqNmVNYjbNpjmkWB7+hMp9BM4rF7QpZA6cwC6glbdKLHvlIfSBSXdoSzN3rFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i+mTgRnY5Fxs712rqui9HWx9XxSvsOL/1QRUm7jH9K0=; b=RNAWJgTjwnCsQPhAqHeMQX6sIIF8zRknA99WaBFoJktaCmCgbAYG7PLWVx5cAIHtCM1cuJYaYQut1Re9UXOEtia4zmT40aZhF07QeceX0YLSzXtpdlrPC6wDvy5G2Cio5Dxc+lmQKxK7cvcNMDcIJdFDCxqpy1m3+arVG3ApTBsMESuGS8+yxRh4RHdnFMH5D0WWie7IgvP74rCLpT5piT/YWGkIsUidxtxg1IeDLDMgq6w5pYhOJhDVjdexFuJYIhtlQ5SRUD/3vbKov+eK0d9ZNFP81pC+yTeV4MjkOJaLzFmJnjsMPgv/GYyiJeSw5gnVNDuT/Tgu8aB6X2mUrw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:16 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:16 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 13/14] rcu: Skip rnp addition when no grace period waiting Date: Fri, 2 Jan 2026 19:23:42 -0500 Message-Id: <20260103002343.6599-14-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0PR04CA0015.namprd04.prod.outlook.com (2603:10b6:208:52d::26) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 186ff6e6-a80e-463e-9409-08de4a5e6dc1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?W+ABO329QmvOakNif1vQMg8rgueCA0vmK82ER/Y+AXoLLK5GoHAYXGpAjin4?= =?us-ascii?Q?nhDl40fNYREDAVRdZfSkH6dJv6BPPK7yVsHyn2p8y9z8QAdfXKYb5ZRkWTya?= =?us-ascii?Q?UoiiDhRW+6hjxvuLOBhc0fNyKHeqvnPusvgQyrCjeuuKowey0+57LzGdB5vL?= =?us-ascii?Q?L4kK5ZNdMKMwzi3EBv8aTc4sZ6hnJkSC9AGVxCnQ8J3hQnNOjpVtnJMOd7Rb?= =?us-ascii?Q?gzxZnNGiU9TWzGJw0U8AI6prg+J0RTnrkMweOcPoTVK2OBiwxUYqmklV6aWv?= =?us-ascii?Q?CnBhMfKYjzLjcW7RPqg5OzDjlZAfu4ZiOzwI1G28/skQJTRAg5Cuc208qw8w?= =?us-ascii?Q?huKye4ZjEZt4Dtrmrza5mJePlqapzONjTSdGZi8L0UzWa4WIhFrek5j9J6ne?= =?us-ascii?Q?FhfOh5lGEImu50UXGQvnnh5YqVVV7/s24nFvSU0dflrDe3dFkiALV3NZ1Qxq?= =?us-ascii?Q?wwcreF2XE7DQjLKkP2+VQobAAa6fDts+xJEpcEGp1rbxwlWmzh3ypVGUIzdY?= =?us-ascii?Q?PgrjtQJ2FWlmB9U8drMpHiWgYRYjOQk5z8Eg0lLhLP76iu+9KuJOgTxMpgVd?= =?us-ascii?Q?YLAGpPHno+oVkwsTApaDEPAz1Chk2/PBFcISdJpbpLAQCWuXkKkBvm+yMEN+?= =?us-ascii?Q?Zp0CjeVj3jghKCAE3gZuqki07r5xGf6SHeK41LQOzDgw9IO+a3n1WVwRu4CZ?= =?us-ascii?Q?yPIbtKvOYD3VousuVgL9uzeP5HHiF0AaO+PCuI1uokBHbXLwtPz6EEB5LSDJ?= =?us-ascii?Q?K99bl2GeTDJ0PEDCdBcCyzbFR+R7sfW+Yj1hron+0xerlEA1c9jf2N/9XUc9?= =?us-ascii?Q?5Ga7j7JpDnCIUGq/9VrOwXZ6i2ulyO8kPkuCTizY5Ul7VrtkxrMkSkzEuOJl?= =?us-ascii?Q?SKBYma/sHqWpbNykaRlmB02zmu22X37v5kwCfGqyV8NxrJVQFotdm1hcviUU?= =?us-ascii?Q?nNbZrCsHpxmNHUjDJf9Eman6Sz6y658v4Y1cNNAxCHEQ4iXnLQ8+Bgvz5+cV?= =?us-ascii?Q?fSlycm8Epq/QyfyEYUB5zQpOtga7WvfOfhUOiw9CXrNVgiQDoc0W01Gwf7NK?= =?us-ascii?Q?y2xuQXWRPZQREuH+i8Z2NRYxzrkjMyp0fKoIITRdt9HaHqPXJG2GbXt3kQ3X?= =?us-ascii?Q?k3jgUCmDwUt8Nd6yKBEZ5m++SrujcE5N7FORJlGTvkxObn4CK+qHr+yd/Ghr?= =?us-ascii?Q?f28RAJ3Iv98iSyYmtni2+a27hUyQyr40RsmDIHOtxCN3roXrsNvTkaVMqDAE?= =?us-ascii?Q?ZoZoGnrlna5OfeNapoaK4Q5HOhGkJJ1HU5NCq0bqB0RP+t3IpgoFopGJG/Ly?= =?us-ascii?Q?BRhDjoXZG0tkbOm5Y2fjKdLnhJ++HIKgqfxXIJgJNb/v+XbVViNTLFkv8Ixv?= =?us-ascii?Q?jUqLe6b3Wf5MkY+omCfXSyV5n0JCM+zW6chNRO3IShj2Rm2cQnx0eck5ylpt?= =?us-ascii?Q?pK5sehpc2dzh8QmEoWPEvx/pvT8FPdeO?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?KfHbU9ERQMJL+lMwAB9fNbQgD91Jq9evI6hehc0U5xD+pr2rtcn1p6AXSa0G?= =?us-ascii?Q?3ekVNLPfz+cTVfd/jHKFgBqX4b0AqKCRgg+Gbe6YJ4xWS93//bhJrqkTON35?= =?us-ascii?Q?gHCbiy8PZ/KaoUFCuneUfX/s7/r3KTajq+eQ5mC/QVpTWz9UaQy7iI/P8mZw?= =?us-ascii?Q?ugUyIQRvaqbOSzNi6qx8WwV2xvvwTYHDKFr6vQXlbxmqFP1t+jvpzZSreblj?= =?us-ascii?Q?ZgKF3fvYEDnbS9+LVHDVrfE/EQpEEu6p+eDUvQO91hxwlh7MN1TNajZ06b+m?= =?us-ascii?Q?Z7/Wg6WVmxh1WhLxo0yt4BS48ktbDF9/zLEqvp131ROahS/fsH6uxPxA5YHj?= =?us-ascii?Q?fjAvaGsgEjIT5UGyKo9r1oX+KAwULs+cxRfWUWEI1esRk30khPy4ES1+vjx/?= =?us-ascii?Q?emhzQ7ELuisMSOhM/LiU/Svp3nIkJBzvFFF1/H+MsthWCFZUnxqNcdLTRg87?= =?us-ascii?Q?//1f4XrfYWplX670H1l8kL3x3XCBH4idoXMwOernWOkSJcJhbnhjJDguRS0s?= =?us-ascii?Q?kAUjiBxFfu90nOXG0BCqUKms5qAA4Sj3ORg8gc3EJSBnPxO4A9WJszjeA6aD?= =?us-ascii?Q?xv0dMNp6lZHRDvlooh9jLLJOQDbCmzTIHqqyIxTT2Rzvu0VdwdyuZlvrEdij?= =?us-ascii?Q?VHj6XivbuqriWL5HPSwhFR/ftor3Hf1l9ukYPZsKPJ30fqsQdNsKdev8Nq5S?= =?us-ascii?Q?fmiVFC3mfSeYDSweoMflR0VjcRckTztAOmufLFgZ5p+0ivIjpKYhu9kHfwmk?= =?us-ascii?Q?D36zFkdqZi7UsV4vYvkVQKn9gsDv3YZDyBzcI1gGDmhLFwmeV5tNNre/7qkP?= =?us-ascii?Q?OuLEvNVDLD799FkVzv8EVj+QeJ4LVjGSK8cHV4aiCG84Cy4ekuxAuP4QRJgs?= =?us-ascii?Q?kRJ3UGOcuoSd2Ui3zbXKclURAx+7yHitLO6gxF/Ls0kKTjtfGX587KTJZ527?= =?us-ascii?Q?RC1shzeDkoTBHB63jbNh4EBWhTs5tpASaJr/zll5wR98xS9z+04kU1Kv+oS3?= =?us-ascii?Q?iabEXkbaYKrxrc+6MAACbTe/DFV5ObgkSSHVRVMvVSwUfzHvdlEoKdvoFOO6?= =?us-ascii?Q?peedKOzUW7SU7fy2mVKBWWDagghdyYMQiyQLjMlEBVxl4sLFt3/EiUMDCf1G?= =?us-ascii?Q?/6S9Kzl5SC0VLQRjvklJaAYGP8HiJf+w3HDCRpGdSJoKIxXvyc07+KqAzT02?= =?us-ascii?Q?tb0uxaakR+jdoBUPWZ6nzKPw79RErYXugZjtWG5JwketLpx88wVQCw2V0qnS?= =?us-ascii?Q?J4yA7wGPBL03CeMQVRsBrAbFLat4BPGBC2SHnkBM1MLBSYXxHHRUY/8zivHt?= =?us-ascii?Q?Lnq+meeH6lmDnktP0ETmhlw4uEYpcpUu/omtNXwvCsUmCihtDr70gL65Z92W?= =?us-ascii?Q?jhLtmUt70Ar8nQAkNhWa9+DfK49K7vYrFFys8J103JK0lbGs9qQuFsRs4MMj?= =?us-ascii?Q?tzffCLXLxZqZlppn1geCxJAFWpNOwTzmJeJ/u+lm/EDwKUPmGx+hIo9cQMhG?= =?us-ascii?Q?o6iknXgfsRhvUtoCHxMc3nURFD/s63gD1sqlSTUjv21Q0ufGAWLm2hHh8uC0?= =?us-ascii?Q?Cji8nbx3rJtclE0mU1CVyIQ1/gUPtyiV1Qy3wKvfqTrt6NG8NcscMb69HQny?= =?us-ascii?Q?dl80AmLjaCG2XyFMmzJEWPUQ1MXPlRqSRFfgQHyI2LJ1jE9f7RFCrO42RBbM?= =?us-ascii?Q?c4x933k6uGZiUrtCsqdyX+hSQE2FHUAjC6wFYtdkPRsOmAeBrhl1DtcGhFbR?= =?us-ascii?Q?Yojst7RcyQ=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 186ff6e6-a80e-463e-9409-08de4a5e6dc1 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:16.3031 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: It781brFJ7KuDC24nH+1qCGnZ+DQwneO8Mm7JflvuSJn6V+926yq63VHmGgos9jw5TbAssSu1oUo+tw+eSYV4w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" This is the key optimization commit that triggers the per-CPU blocked task list promotion mechanism. When a GP is waiting, add directly to rnp->blkd_tasks via rcu_preempt_ctxt_queue(), but NOT to the per-CPU list. However, when no GP is waiting on this CPU, skip adding to rnp->blkd_tasks entirely. This completely avoids rnp->lock acquisition in this path triggering the optimization. Signed-off-by: Joel Fernandes --- kernel/rcu/tree_plugin.h | 64 ++++++++++++++++++++++++---------------- 1 file changed, 38 insertions(+), 26 deletions(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index d43dd153c152..a0cd50f1e6c5 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -335,37 +335,43 @@ void rcu_note_context_switch(bool preempt) =20 /* Possibly blocking in an RCU read-side critical section. */ rnp =3D rdp->mynode; - raw_spin_lock_rcu_node(rnp); t->rcu_read_unlock_special.b.blocked =3D true; - t->rcu_blocked_node =3D rnp; #ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS /* - * If no GP is waiting on this CPU, add to per-CPU list as well - * so promotion can find it if a GP starts later. If GP waiting, - * skip per-CPU list - task goes only to rnp->blkd_tasks (same - * behavior as before per-CPU lists were added). + * Check if a GP is in progress. */ if (!rcu_gp_in_progress() && !rdp->cpu_no_qs.b.norm && !rdp->cpu_no_qs.b= .exp) { + /* + * No GP waiting on this CPU. Add to per-CPU list only, + * skipping rnp->lock for better scalability. + */ + t->rcu_blocked_node =3D NULL; t->rcu_blocked_cpu =3D rdp->cpu; raw_spin_lock(&rdp->blkd_lock); list_add(&t->rcu_rdp_entry, &rdp->blkd_list); raw_spin_unlock(&rdp->blkd_lock); - } + trace_rcu_preempt_task(rcu_state.name, t->pid, + rcu_seq_snap(&rnp->gp_seq)); + } else #endif + /* GP waiting (or per-CPU lists disabled) - add to rnp. */ + { + raw_spin_lock_rcu_node(rnp); + t->rcu_blocked_node =3D rnp; =20 - /* - * Verify the CPU's sanity, trace the preemption, and - * then queue the task as required based on the states - * of any ongoing and expedited grace periods. - */ - WARN_ON_ONCE(!rcu_rdp_cpu_online(rdp)); - WARN_ON_ONCE(!list_empty(&t->rcu_node_entry)); - trace_rcu_preempt_task(rcu_state.name, - t->pid, - (rnp->qsmask & rdp->grpmask) - ? rnp->gp_seq - : rcu_seq_snap(&rnp->gp_seq)); - rcu_preempt_ctxt_queue(rnp, rdp); + /* + * Verify the CPU's sanity, trace the preemption, and + * then queue the task as required based on the states + * of any ongoing and expedited grace periods. + */ + WARN_ON_ONCE(!rcu_rdp_cpu_online(rdp)); + WARN_ON_ONCE(!list_empty(&t->rcu_node_entry)); + trace_rcu_preempt_task(rcu_state.name, t->pid, + (rnp->qsmask & rdp->grpmask) + ? rnp->gp_seq + : rcu_seq_snap(&rnp->gp_seq)); + rcu_preempt_ctxt_queue(rnp, rdp); + } } else { rcu_preempt_deferred_qs(t); } @@ -568,13 +574,22 @@ rcu_preempt_deferred_qs_irqrestore(struct task_struct= *t, unsigned long flags) */ rnp =3D t->rcu_blocked_node; #ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS - /* Remove from per-CPU list if task was added to it. */ blocked_cpu =3D t->rcu_blocked_cpu; if (blocked_cpu !=3D -1) { + /* + * Task is on per-CPU list. Remove it and check if + * it was promoted to rnp->blkd_tasks. + */ blocked_rdp =3D per_cpu_ptr(&rcu_data, blocked_cpu); raw_spin_lock(&blocked_rdp->blkd_lock); list_del_init(&t->rcu_rdp_entry); t->rcu_blocked_cpu =3D -1; + + /* + * Read rcu_blocked_node while holding blkd_lock to + * serialize with rcu_promote_blocked_tasks(). + */ + rnp =3D t->rcu_blocked_node; raw_spin_unlock(&blocked_rdp->blkd_lock); /* * TODO: This should just be "WARN_ON_ONCE(rnp); return;" since after @@ -584,15 +599,12 @@ rcu_preempt_deferred_qs_irqrestore(struct task_struct= *t, unsigned long flags) * from the rdp blocked list and early returning. */ if (!rnp) { - /* - * Task was only on per-CPU list, not on rnp list. - * This can happen in future when tasks are added - * only to rdp initially and promoted to rnp later. - */ + /* Not promoted - no GP waiting for this task. */ local_irq_restore(flags); return; } } + /* else: Task went directly to rnp->blkd_tasks. */ #endif raw_spin_lock_rcu_node(rnp); /* irqs already disabled. */ WARN_ON_ONCE(rnp !=3D t->rcu_blocked_node); --=20 2.34.1 From nobody Sat Feb 7 18:20:16 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99AF1229B36; Sat, 3 Jan 2026 00:24:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399868; cv=fail; b=QwdcH6v413UcmpRLKTck+MBdlvqIX4+ZowUmjHWcgvBVI7nlADffZudFqvov/VyQ4b649dio1jNH+s6aZHv2Mcm8g7qkk7HiP1nC0GxOyU/ebLmtqS6du0UAmWHFxcf1G6L0kUgU9Rk3sOXLQKDt9UWYtAFZTefb6TMa4MnKp/o= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767399868; c=relaxed/simple; bh=auXS3nBE1ms1lXRA/1dM7+gfICzNyXbR7cGGll9tBGY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=MMuiQ47YdxwwkZfHbsKUdtQ5OnNxDKm0ZHOTLvf1nYLpH5gaXMAZj+JEDFWDvTCfi+B67gSgSyj+hFheJRmuIfkcM1BXowlfiCeQcxXyF7ytIKUfr1TlgyVp+yJIIMwkGHwkT7x/Y2Qu/Rs/yL6ljEnKnnxkw8Lu6XxrYZadSFs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Nqd8yuBE; arc=fail smtp.client-ip=52.101.43.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Nqd8yuBE" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qGCJkxiLzgtyrfZxVs92I5QZm0dYNb/hQiAMxmEKkf2OEu1/uw4fc2+oTjqdeZe/wstfSqP7n04l22uLkcqvNPpVJQ2csiee2zH7jZeb5rJsQ+M839lOqmJFbn7EpPkHI+jK6WVaVXXRGvNxMB4zedoaZ+X0cApVu0Qiz+kd/o7fm3B4zcrBcC7+8mfrOmIcLh0TLRdB7QjrJ2l1EBkjif36nl2q2V+exN5fXqnsQDQa4QoebmjxJReyZvLL+OjzmIMm4DI1O3nZMd0o4iEBfMoh4/Rdxkp7UVdngfyu4z7zyC/6GEeJRg+OJeKKSIdMn+6kqwlpL7GLp+XZTRQDOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XLXFvBuacqAvwTlEjmNruDXadCCLcHE8n7LXChvo1zU=; b=ruUnbI/u1/UxzeYpUTzWBlspKDX6Deqe/iDAVeY3bmtNsU2XJTufEdrLlS3TLkr87zg81FvCajpMZ9pzepPWCbvSkPxCl4CLcWpPx3BoU/O0hX96NAGPiOvW+7AnGeYsopNXPQMT2KVGiMPXdt5iJWycUIfGtxAMzJ8PLY6pyxU2HBVq92Bt1pJfCBESThShZO958y8qRiXVOBomU9beDX0bCB5qcj+bZyGVHN8324nVwNCLunDFkEtGyCb9RzdrVt6n2j81rYxuCzbhOzWi2x3ztRLf8KuBQfBuxr3Qqj6V7bYICYsCYMPFHPw0kSRMx6UKkrvK6ls7DlFu8XNgqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XLXFvBuacqAvwTlEjmNruDXadCCLcHE8n7LXChvo1zU=; b=Nqd8yuBEFICGe+rg87x6tooxP0wyJYOnn9yv5kurmrSCB26alCGHZnZMpxoiJo+jQWwWkkU1XuiCc4+S7cztpeXE5NJ3e+AosGNQuyeF23TUyxdWyA+eGiEBdzDw721w9B24NuUoqOflUZVwh3gM5oYxpJ852FXkXEjJSP2FX9rR5vUJCBCtj36s6kV96sYkqHLuBFxo1JM9DfkHg3UVrtzwI1M4PprbLi1sJQn8yWd6V8ddUbDwrYErGqZ4m3Xn20O/nS7XQkecOnyHsm6jN3PcICT3hC3mN6dASQn2ON1IOjcCLFQfO02H5zW/gEVwwqI6xMh8OX0ovdNbgsgJ/Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) by CYYPR12MB8872.namprd12.prod.outlook.com (2603:10b6:930:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Sat, 3 Jan 2026 00:24:18 +0000 Received: from SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91]) by SN7PR12MB8059.namprd12.prod.outlook.com ([fe80::4ee2:654e:1fe8:4b91%2]) with mapi id 15.20.9478.004; Sat, 3 Jan 2026 00:24:18 +0000 From: Joel Fernandes To: linux-kernel@vger.kernel.org Cc: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Uladzislau Rezki , joel@joelfernandes.org, rcu@vger.kernel.org Subject: [PATCH RFC 14/14] rcu: Remove checking of per-cpu blocked list against the node list Date: Fri, 2 Jan 2026 19:23:43 -0500 Message-Id: <20260103002343.6599-15-joelagnelf@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260103002343.6599-1-joelagnelf@nvidia.com> References: <20260103002343.6599-1-joelagnelf@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0PR04CA0004.namprd04.prod.outlook.com (2603:10b6:208:52d::6) To SN7PR12MB8059.namprd12.prod.outlook.com (2603:10b6:806:32b::7) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB8059:EE_|CYYPR12MB8872:EE_ X-MS-Office365-Filtering-Correlation-Id: 543ff174-a8c6-48c0-6262-08de4a5e6ee3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Z8xAkYOK77nRaJARbEqbDKH2OehNkM3ZGpQLRaZ/W1HtqnKYP0y9w+NOVmTk?= =?us-ascii?Q?IuOBoEY0ruNk2gz6rRHwUvj1tnsslFN4fz8H04YG4VKyhUbG7vrtacc5Qn5p?= =?us-ascii?Q?blozonGdfPUaWrKBYVl448d4w4uhnw00Brlyuqmt02uI65iONMCGpo+EL7Xh?= =?us-ascii?Q?wT+UllTzF7IyhNmWG/EiBGC3EJqPGPkfgOn0PVjtSnb1lspdBjhnZlD6yHbi?= =?us-ascii?Q?wOU61Osz/JFJb6jz+59MBOkdlR+6o3MzhGEkNuvbyNV/VGKqwM1WAZQaoV2q?= =?us-ascii?Q?JEl00PIKkzpvwbxTbMJLHji0oFvKNpx3brFxPpLp7tokvzZTbh3tz0OjalYf?= =?us-ascii?Q?2RW1Jr4IeJVCjlaOxFD0KyL/9gEnQunMUF4bGwcpA9n2ilKm3SWTDfIjO6RU?= =?us-ascii?Q?KgorTt7yjOBcpV34RdwWCRb91ZNs8C4/XCgOFYBsL7YaZvoETUjuduaEeT9Z?= =?us-ascii?Q?5khul8iWa1i6JfmrViyIaCqgW7hI1HxV77hOfllnYmkS1UJQ1CTTtSePOhU9?= =?us-ascii?Q?ceaPzp9r++++Caf5YzDaCWL4KEouVIjQyvMT13XICKKLzReK6n+NZggm8Jcm?= =?us-ascii?Q?jjxknHbtFLEjM/FYEa+YUL8wo35cLF7afbRlC7qhSrntJzDuWrI4G9G87XyN?= =?us-ascii?Q?rid241k+paC6nNqwu+ThAFG/RqPCeRCh/6Y+WuVew13mubNzMfDVKkLE+NhO?= =?us-ascii?Q?9MS7TCC1jgvLwlr5C8Ej23kPSwxPIRLSg6crmcvGTo7+MqPTB9MMiI0qf5l9?= =?us-ascii?Q?jESRCHGVJmp6lrn79EXAHgyPqQSQgQD1e3wdlg5uhjrAD9vRXWIEh6NSRlkU?= =?us-ascii?Q?2IHS5byOOXLugy+GI1Kgemni7eIPv+6ZEJjAESDTjM8gb+SPYgswq2YHPJZZ?= =?us-ascii?Q?16xEQlB9uSPovPfBCzuJljXKPqGU9gdLyQThJWFeuLS9God7Wi5QD0535jX8?= =?us-ascii?Q?smqY55gPVQEBzDSvREDMod4Scwrp4L1BI33H0jPjyNQwcAEA/stcz9Te5iKn?= =?us-ascii?Q?HVDVYmbd6ZYfbZVswhiUZRGkTYwy8EFyuK+P/lAzooQPcHHlneEqCtoZnrXf?= =?us-ascii?Q?/ac3r0NdFSZKqLfLKvlkT3Kos9xAzd4jc5zUI7/aSV6BfqvErSGzzF9jrZUa?= =?us-ascii?Q?sBBhEhazSea2vNdUtkG3nMeKWl8qMfsnjH9dCODzKg0yAE6YdqJjFNZhPAUm?= =?us-ascii?Q?K5byCbJMBGf+kFNgLntTHJqlBp0Dif/mmUSO+HiWtRGqzHELfQNVF5EEudew?= =?us-ascii?Q?uMW+B2t+n6ZP4zZUgpQbpsCPoQHD9y95g03Gq7THCzU60Yu20YVWDEsBEENe?= =?us-ascii?Q?f67QcnfFIxNuFSF+WBmR3HWcOckHOUImbbPzZsnm8Qf0aFncNZz45yI5AElp?= =?us-ascii?Q?lvavIZYTYXbcjKZEpJ+ebxb89u7TJlx5MugLV9ojbzL3NaZYCCjQ+qNWuAiZ?= =?us-ascii?Q?J0hena3qju0iVUMdIe/HtA+u3bVR/tKn?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB8059.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?S81S0XnCpFtq4qJXk2N4mhR2XbYkCsxnVaR41uZNp2IfWNlhdCxHSTnL9A3z?= =?us-ascii?Q?8WHSyM9eUa/GrTjhTt6rF3ErYeJxy9i0ZWq/G54qEdwrmrWdZvv64KuRX19A?= =?us-ascii?Q?RUO+mXm1d1Ow30IQyfPXVbzjktx7sLSyaOGSWdeSwYAbVJmWml55wzkp2m9N?= =?us-ascii?Q?oxpNNdkleT5oPprgtYkuIa8aENHN4xBGOr9dFDYeYTaNJcxoaF8UIau9Xn8x?= =?us-ascii?Q?kLAcxFxI31xPbnB68Hj8dnZVeB1lC/mhnu2uat8JDVGZILXqsMzsMmRTJlHn?= =?us-ascii?Q?hkJ2g7kFWxbA8SKv4yHZABlQ+AG8hKwb0mFJxaAYoNDi1AMXysBdubddG/uY?= =?us-ascii?Q?t2Tj2YdEFfaCx3WrLDz/K2CI4pYo4KAPXxkxNf52Z9VkS/Da0jUpTmerZ4yu?= =?us-ascii?Q?MjgmtaXshu2hXqaHbXYu+Nd48i0J9ydS/7ii3vVeAqbpsuylfMu2Na563idc?= =?us-ascii?Q?JnTox0RW0dvOPjCdxF6194e3X90kUvaR6ondgh+rKPIkH/VWk6KI4fGbR4J8?= =?us-ascii?Q?7nD8JvBMoW9TODTs3DHiQIEHBxyGiO2HUeb8B5026bZxwwGjn9FVbpbeJud+?= =?us-ascii?Q?vSrlY9/yPvkw2g6YCx1uhX1URVXiOtgtvQKVuUXxRNiPSR5+Eb/fjkxzdBdI?= =?us-ascii?Q?ofxlANRJF157tiXmrBlc0TqFIm2Dv7Sga+TsnyZ/9Rvri+t9gMk5OoMG2M+J?= =?us-ascii?Q?Vc6EmYu8X0BpBw+bdF9hBzJM1VVOKhCYmJD1xtn2mInsMthpT0Cj3yHYFs0W?= =?us-ascii?Q?Ogbx1+GSS8PCbac5FxkSVXEaRHmqbu6C3LOX2pBcEdMunKwCmy+f0VukKuMe?= =?us-ascii?Q?e1qdsXUbKZovXgjIwBrRhVHdPJ2n+U6fPJU1Lq/dlJhpQYxpnlWklFPfI+BF?= =?us-ascii?Q?E/zHQO7O+R5TTV1tNEQDyXY7akg8AzlwoBUOVthYKn5sBmx3reenaL7wkafa?= =?us-ascii?Q?7QFyua/VYVBhXUZq3jNv+sPi6Ah+FIrae3pbPk6orxm4YPxJ2NJYIOqxUNEx?= =?us-ascii?Q?lg8c64XFGb1RjUy1xA3mFpF63j+kX3BvVZFIEoMyTdmb19fw/rMPBi38klId?= =?us-ascii?Q?eZunweSLkYx1exO4Q4M+sZff0Ir4cE4mQLu0pxpOgOzIxHlebOp9LjNB0Hoi?= =?us-ascii?Q?cE+lF7REgJtT3fUYrpf7GqnNEHl0RN0CU5I5SMqc1NtIEjFNKV4I7sZnQJrT?= =?us-ascii?Q?kxas0BPBnhPH9lzfWHkD5JBBQywniZxF0JfElUPXdmPVg6qrUfHsLVacCL+/?= =?us-ascii?Q?2vMJmd719gFXmNxWW6C6Xul/dbM1KutIpc3QjmzOCfITE0tw+NxXwiCZGeU4?= =?us-ascii?Q?pvJcPO8BsZXOK+eX3mhj78Qib8GJPG7/2RIWM0boXfcXT6ZjAtxp4Y5QvtZB?= =?us-ascii?Q?5hK3VSqURMZclERDTKijWoJqJQMK53z9VY1vqXRK/0mexBqBR4C3AKbP5FIh?= =?us-ascii?Q?SfCG3E6XEcbBzyLncbHOhkWVvFnRj3FZPVacbMUoK+w5EhABL/KDkXAMwtTy?= =?us-ascii?Q?STLMMXRC7HFL5MVSKJAUIREQpMxuRwpaj0HiegIzB7HgPEZwYWBw6YpWWRQf?= =?us-ascii?Q?OFtLlO+aQpb7u/7Hkp6umCw9y8bU7ROJVxOy7fWMm8itVm+7RqjlPqtPYwWv?= =?us-ascii?Q?kwpzEZtt2OP/5D4c+1nA/XMH/K3H2yQNuidKh5V+aqdHehKVvP6FkGPrgPqu?= =?us-ascii?Q?rwiw1LU86wmNfAUWMvu3v13e0degCSN9yKvia/5td54sO85OeK4VTvhIGHMS?= =?us-ascii?Q?rzkMhE1jZg=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 543ff174-a8c6-48c0-6262-08de4a5e6ee3 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB8059.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2026 00:24:18.1505 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: S++CW4ysbRllb9RbKiUZ69a5c8zvpcTr/xC1g6tHx8WyzEbeFuHtd69k0+wpaJM++IKd81GSThfqWm7H4PnpSQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8872 Content-Type: text/plain; charset="utf-8" Now that the verification check consistently is verified, remove it. It is still kept in the patch series for illustration/testing purposes. Signed-off-by: Joel Fernandes --- kernel/rcu/tree.c | 20 -------------------- 1 file changed, 20 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 468388970c98..9d9d7c5ff3fc 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -1900,26 +1900,6 @@ static noinline_for_stack bool rcu_gp_init(void) arch_spin_lock(&rcu_state.ofl_lock); raw_spin_lock_rcu_node(rnp); rcu_promote_blocked_tasks(rnp); -#ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS - /* - * Verify rdp lists consistent with rnp list. Since the unlock - * path removes from rdp before rnp, we can have tasks that are - * on rnp but not on rdp (in the middle of being removed). - * Therefore rnp_count >=3D rdp_total is the expected invariant. - */ - rnp_count =3D 0; - rdp_total =3D 0; - list_for_each_entry(t_verify, &rnp->blkd_tasks, rcu_node_entry) - rnp_count++; - for (cpu_verify =3D rnp->grplo; cpu_verify <=3D rnp->grphi; cpu_verify++= ) { - rdp_cpu =3D per_cpu_ptr(&rcu_data, cpu_verify); - raw_spin_lock(&rdp_cpu->blkd_lock); - list_for_each_entry(t_rdp, &rdp_cpu->blkd_list, rcu_rdp_entry) - rdp_total++; - raw_spin_unlock(&rdp_cpu->blkd_lock); - } - WARN_ON_ONCE(rnp_count < rdp_total); -#endif if (rnp->qsmaskinit =3D=3D rnp->qsmaskinitnext && !rnp->wait_blkd_tasks) { /* Nothing to do on this leaf rcu_node structure. */ --=20 2.34.1