From nobody Mon Jun 8 20:53:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B13A2571B8; Tue, 26 May 2026 16:39:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779813546; cv=none; b=JYwkGrjxVzh5QCpKcZGwSa6G5sKPpxhihPClSzQWhAxwjE1p/ewi/7ct1S7GsUxKYdqV50MoQOLjEu3bcISCwRic15lDuqBucg8oFYI3VKoqPNNlwaYNOHlsOMguEVfbjhQScRiOzhSvPu5wS8Qd9xUC5zM53R1GGEV+/HgPc8A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779813546; c=relaxed/simple; bh=EuhaeTvZUer9TPm4hog0MOssp4P5pZU6P3ibMtaXW2o=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=lW0nmf+4s8j5YkTcSk8/TSxwC/c8wvbn29/ttFMW5AWnQ+cUC10PcUlyoq1uzAiIDRjDEmfatU0Avuij13ouQtkVaU6jXldlGHHPGcWke14xrj6NOi1z0jq/LdXLWn7L+J2RiVkqYHDz8OI8mRBzNiz1PvQaV1DhoAM9sAfVNZE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZR+wPEWt; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZR+wPEWt" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A8E1F1F00A3C; Tue, 26 May 2026 16:39:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1779813545; bh=BzHDn0s1mv/4IfZh/7mfkYvYs848g+6l7oL8JYM3GB4=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=ZR+wPEWtsVxUgmW2/0MFgzMXDtmVMD0l3lLiAUMIST5uvbMXoSm9L+blTW/5SiiE6 a6qHJK1ThRibMWamqbKn4sTr6r7ptoeJfCl1xr/ZTCyEHiMuAQPoUtD82o3EJEHJV5 36PlYiFI5+llSTo93nP2HaPsBJhHqKZvj8bQGt/qjiYj/wphd/HjliKe4iNy0Q7f8M KsDfuePZVG5CCe+bWL+e8GSRhmp8WHcW9vyVdpjqSNH3bDaoWIoDAVwL/ZZytVGt6x Y4RhxwXGWjHA5kl/T5F1A+cASs86utl4N0eAq6hh9wpqHNUMgMZm/d7YFF0OX0PsJC 1YeQ34J/Zjq5A== From: Jeff Layton Date: Tue, 26 May 2026 12:38:45 -0400 Subject: [PATCH 1/2] nfsd: defer setting NFSD4_CALLBACK_RUNNING in deleg_reaper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260526-cb_recall_any_callback_running_stuck-v1-1-310011a028f3@kernel.org> References: <20260526-cb_recall_any_callback_running_stuck-v1-0-310011a028f3@kernel.org> In-Reply-To: <20260526-cb_recall_any_callback_running_stuck-v1-0-310011a028f3@kernel.org> To: Chuck Lever , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1386; i=jlayton@kernel.org; h=from:subject:message-id; bh=EuhaeTvZUer9TPm4hog0MOssp4P5pZU6P3ibMtaXW2o=; b=owEBbQKS/ZANAwAKAQAOaEEZVoIVAcsmYgBqFcyneTPqlf6nE46AihFmbKARJVO9iyGdiVCk5 pSQ9klotlqJAjMEAAEKAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCahXMpwAKCRAADmhBGVaC FSh+D/9Gmcr1/QJAqHBZBZO6W6H1U4lYIk548bP1D91TbT4K5VbDzBCcPpbzzidC/Lsz/8MLqJI u73jiv3wT4DIPPXgcoccw8DqWyqVRs9NYffna6dC9BMv3AcNjHnBOasLoor27e4cdvwiyuM/xhm mWdScIlfle//DTr3ynhD43Bl2YstKmx6V6QShMF+5zCioG8QD5emD5RzafUu7teeWOU4AlVt33m w349+cE8fZkd//MDr4Y15E94BM/I17YEY0cRo+rQ2sWA8EGXzntkmHAW3+igaCmtgS7PZPXDeBF FuDQ0/PaI8IqjtOcLQmB6Vvp3B/MrOPyZsUnA6NfiOr8kxoGzqcygcX5HzlAru0/0GKqKhuS8U1 X1kZDXQ2QgmRyMlaQ6hSzB5E6Y484xwSVFNCMp0KhMW5EXx3B3CIDjCq4sVnN8vzi0JiDLNk/Ku 9eliLEOBPiaWlb3DSh57ImCP6ygJo942QbSiWIv5+kYtgUxfAJySEXObJ2fNLVAGeK8LKJrMCqN 0Z+U853fxTB1oABXoBs+/H4NzvKx/C1QcwuzCY2B/fVZbGFcOYEzV/tmBLJy35kMqppDYgLBmsx Z0Zsmz8LK6FrZ3NFIpFFFxRzo27pgEroXG8KBtV8mh78IRCT238RWDvVLqKk6NE0/Om1ds+aY3W Hs0zHa8KrYHJh9w== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 deleg_reaper() sets NFSD4_CALLBACK_RUNNING before checking the 5-second rate limit and cl_cb_state gates. When either gate fires the loop continues without queuing callback work, so the bit's only clear site in nfsd41_destroy_cb() is never reached and RECALL_ANY dispatch is permanently disabled for the affected client. Move the test_and_set_bit() below both non-queueing gates so the bit is taken only when nfsd4_run_cb() will be called. Fixes: 424dd3df1f99 ("nfsd: eliminate cl_ra_cblist and NFSD4_CLIENT_CB_RECA= LL_ANY") Signed-off-by: Jeff Layton --- fs/nfsd/nfs4state.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index d5cbf626ab9b..57b99d1e74a6 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -7478,12 +7478,12 @@ deleg_reaper(struct nfsd_net *nn) continue; if (atomic_read(&clp->cl_delegs_in_recall)) continue; - if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags= )) - continue; if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5) continue; if (clp->cl_cb_state !=3D NFSD4_CB_UP) continue; + if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags= )) + continue; =20 /* release in nfsd4_cb_recall_any_release */ kref_get(&clp->cl_nfsdfs.cl_ref); --=20 2.54.0 From nobody Mon Jun 8 20:53:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9AB3D3C9EED; Tue, 26 May 2026 16:39:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779813547; cv=none; b=e7c3ERCoTxsFkJExay+YuaELbqy55/bQ/uwzVpv57M3JyR9ys8QfT/k3KKKvA5BdWwamI73uCV4Ls1S85eDTOBs5qKNGGsPA2BsqhTKatsNzyAZgSHWbeb9nhyHuD5d1DSesSYu0B4uP/Fc3RWg9Dgw/D7Apl6YQzIcSo2lYRAQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779813547; c=relaxed/simple; bh=oBiyUV3sb+iaaUesrbu3m1O986pr2EXCTNctR4cskBk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=axrmBOHi6X+v1cMflsxg+d7O2uh9tQYWKf9KdkwxQSml8QVsPGTeyBo3pTTBzyvQzKgMExHH9HKIMkCr+Owp+YYlN0EuHS03eq/NP3l9KTUvKIMRPtVgl8qevhfygg+GzltTN6D5OtaiIBo1uH1BK/5BibS732Y8kfw/EsQOz5I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WRJ9GdpF; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WRJ9GdpF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E4651F000E9; Tue, 26 May 2026 16:39:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1779813546; bh=DWieJrj4IvMbaWC9sJ0oUMZcM7jWqQaqKc83yl2am7s=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=WRJ9GdpFRpe+N2Cq7Rt6aFV9nbEdU9C4gVIaaGmSHQG1RXQqOcypLrkUB9K0fUagC v54UrR8akKrvqr6ETlWqxOBW1YOJmoagJnTMgDo2vXzWt1MPMWoR6i63wq0yWvw24u yM/nRKgyvaDkS8sLNggO2oXdxGpnXBKwbsZFu6XADc+SroiuGedMvVfsmf5tuNEQxZ TJzg5d0eUHqSDbnlRYtXWg9i/ZnTA0b/zTqO8auHQVyoO0wiN5FnJfLmS68Up+8f1Z Udxbi63Ex+EmBxIBxGPca4w/J3n4O+5oxlE5VvltGF3mROEMPWjp3GRy7xl7+sD3d6 7VllyN8Tlh4Cw== From: Jeff Layton Date: Tue, 26 May 2026 12:38:46 -0400 Subject: [PATCH 2/2] nfsd: clear CALLBACK_RUNNING on failed delegation recall queue Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260526-cb_recall_any_callback_running_stuck-v1-2-310011a028f3@kernel.org> References: <20260526-cb_recall_any_callback_running_stuck-v1-0-310011a028f3@kernel.org> In-Reply-To: <20260526-cb_recall_any_callback_running_stuck-v1-0-310011a028f3@kernel.org> To: Chuck Lever , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1531; i=jlayton@kernel.org; h=from:subject:message-id; bh=oBiyUV3sb+iaaUesrbu3m1O986pr2EXCTNctR4cskBk=; b=owEBbQKS/ZANAwAKAQAOaEEZVoIVAcsmYgBqFcynpfyLsYChSKYRHZqyANynVbhFNJ0Pme+Nz 39VYtDOyeeJAjMEAAEKAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCahXMpwAKCRAADmhBGVaC FQY7D/9EvrdvD8OOnHtjf6Z1vBO86TKXEMv/XB0qcaNsCd6TBZtDo8L6doE3S1aMX7cpjyjnGHn zEev+pAu83oLWeOgAUh94iuKHPZGyhYk3nV2Qf0OPg/8OVYFpTyvt/otbjhiMQnXqhLrQPjglEN SPbIkdm8aE+GuutFV5NTsJqAWa1qpK5Khvdq79BWQ1jCDHjTdAu3tD+KUD/5fhVduC+ZFf2RC/G DzH6/ziLCwyo5s6B4BR9wzxFLwQt2dUhX+2o3epf5pi6AWGHRFuaDakolWCH+L0SbQpzM4eSVPT nkXF3FQNIeelU6ZHMDeNF+i7UidWGY/hdvfmPYdM2VVwMvbkHk5OkXKwkI/See4LdmzTU5LPaQY ArKmqkQbm660Q2e6D4bb3b6aZSSk5+UazJ2LRggMg2vvm83KIELx5ejZUJgD3X4w8OrjBGWQ5Yy Yew4u+h0FdX7HlZKvgbBYZUytJ+2EfpYi/9SE+5+M3ftpD0riloZ+lN2DpuC1w51F7HMGkuLAO7 aZSXQ4TlpYZKSvRGtpQ4MaqhgyYqTmfK57h6ScVxRFY2cUZ0fsWnaB5rQ4+g4nIQArxCU5rM3qC tWod7vGzm0Mk2qDCVEYmNURZRiV0+PXQjaHSETUCEE8r/a0TTNz3nSVBkDVcushPBS0AhLbEZSG y6xMig2zAOJJ28g== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 nfsd_break_one_deleg() sets NFSD4_CALLBACK_RUNNING via test_and_set_bit at entry to serialize recall work, then calls nfsd4_run_cb() to queue the recall. When the queue attempt fails the refcount bump is undone, but the RUNNING bit is left set. The only site that clears the bit is nfsd41_destroy_cb() (fs/nfsd/nfs4callback.c), which runs from the workqueue and is therefore unreachable when nothing was queued. The bit becomes a permanent latch on dp->dl_recall.cb_flags: every subsequent break_lease() on the same delegation hits the early-return guard in nfsd_break_one_deleg() and silently skips the recall, so the delegation is never broken and the conflicting open or lock stalls. Fix by clearing NFSD4_CALLBACK_RUNNING on the !queued branch alongside the refcount_dec. Fixes: 1054e8ffc5c4 ("nfsd: prevent callback tasks running concurrently") Signed-off-by: Jeff Layton --- fs/nfsd/nfs4state.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 57b99d1e74a6..0b4d7afc42c6 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -3479,8 +3479,10 @@ static void nfsd_break_one_deleg(struct nfs4_delegat= ion *dp) refcount_inc(&dp->dl_stid.sc_count); queued =3D nfsd4_run_cb(&dp->dl_recall); WARN_ON_ONCE(!queued); - if (!queued) + if (!queued) { refcount_dec(&dp->dl_stid.sc_count); + clear_bit(NFSD4_CALLBACK_RUNNING, &dp->dl_recall.cb_flags); + } } =20 static bool --=20 2.54.0