From nobody Sat Sep 26 23:53:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E4C4381B0D; Fri, 28 Aug 2026 07:19:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787901592; cv=none; b=jHxDU2nqa3ze4eIDyhZ/2FshgGsyn7mXVPPvFBzA6+/4dyedWW8qTs0GvaPLxln3PVE7oWDsy23ay1xnSxFgbepbbFcIuuC38jSvJW1M0ynf3896gXkXRQnzp+LxzSeZWA2gzoHLtLJr0PIa7Qjl03iIWHw1swYj3eo/YxzmCIg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787901592; c=relaxed/simple; bh=Qwh1evNey1RqwtlKCdO/sm2TJvcyqoG+uj8ZqTaUQMw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=qB23uOImzCIqR/VkDJpU7y/a23YbiBuCXW9or4FH/mz2Zy6RDPQWGOSoEbTCIB/6qhl/q9O4rDft5pBZiSLKYmf6K2j8UjoRMTuyrPF1qQWAuFgUQapq6xt/gppF322Tw7g4w35DN6QOAr+zns7rUoydhwTPxwNp7gYhYVODx1k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PGT54BDi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PGT54BDi" Received: by smtp.kernel.org (Postfix) with ESMTPS id BFA66C19425; Fri, 28 Aug 2026 07:19:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1787901591; bh=Qwh1evNey1RqwtlKCdO/sm2TJvcyqoG+uj8ZqTaUQMw=; h=From:Date:Subject:To:Cc:Reply-To:From; b=PGT54BDipAbYTAmpUB3efGFR/nZWOenF0Ibr8fNmuGR1ofdm2NGUOoA/bBMflEbg7 bxUOByComQAz1+UazEqXAeZ3FnCTNH6Ptr9+0XNXgjPgZnvcdsE8KqK7mF+fYQIt1L p/3FDwgOrmK1fxcrMDaaQyx6ABrdDjDi5mDtdfrzselBz2QWr2kRE/Z8butfG5VtMM LM536rmfTM1XwjiFBAmGWhdIEzZPjkGceDbuZ7QkRJKxFyQkYieoqS0teQoK/phs2/ Bowtu+eg3qLhL8acXTWyYdcEaZM0ZxL45Td+dp2GfvjNEwBpePCjGXhEa1FbOCfXp/ 85Rt75EBs7y3Q== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C8CDC61DC7; Fri, 28 Aug 2026 07:19:51 +0000 (UTC) From: Xiubo Li via B4 Relay Date: Fri, 28 Aug 2026 00:19:44 -0700 Subject: [PATCH] ceph: unregister aborted requests in ceph_mdsc_wait_request() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260828-b4-ceph-fix-abort-request-uaf-new-v1-1-96e480d5a028@clyso.com> X-B4-Tracking: v=1; b=H4sIAAAAAAAC/yXNywrCMBSE4VcpZ+2BGKRGX0VcJOnEHhdpzUWF0 ndvqssPhn8WykiCTNduoYS3ZJliw/HQkR9tfIBlaCatdK+MPrM7scc8cpAvWzelwgmvily42sA RH7YXE4zvB6sUqHXmhDb+fdzuf+fqnvBlD9O6botkd5+FAAAA X-Change-ID: 20260827-b4-ceph-fix-abort-request-uaf-new-a98f8c6da00e To: Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , Sage Weil Cc: ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, Xiubo Li X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1787901589; l=5446; i=xiubo.li@clyso.com; s=20260625; h=from:subject:message-id; bh=xOjNIYx5UpPI1bdwdUUnY3dY/FlR7eDo3///O/wyWVM=; b=LvkX9rMT+9tEYkPvxddgvQpdntIdw59wdI46lYJbusiYeTXuEA2GcuQQfrMvX024FCCZ46uqn 2E+CNs242arBulo8VUgVk5fVqkAQc5Kr0xDx2D9+N9q/KJqTlIUc+aO X-Developer-Key: i=xiubo.li@clyso.com; a=ed25519; pk=V3NGr0AgAopiUhaLY51ipBkLN5LlcLhjOEfLEq1RoZ8= X-Endpoint-Received: by B4 Relay for xiubo.li@clyso.com/20260625 with auth_id=840 X-Original-From: Xiubo Li Reply-To: xiubo.li@clyso.com From: Xiubo Li When a request is aborted because the waiter was killed or the wait timed out, the abort path marks it with CEPH_MDS_R_ABORTED but leaves it parked on a wait list (r_wait) and registered in the request tree. The waiter's put can then be the last reference while the request is still linked, freeing it with r_wait non-empty: WARNING: fs/ceph/mds_client.c:1233 ceph_mdsc_release_request+0x247/0x250 WARN_ON_ONCE(!list_empty(&req->r_wait)) The freed request stays behind as a dangling entry on the session's waiting list, and the next __wake_requests() or kick_requests() walk uses it after free (a stress test that SIGKILLs waiters during MDS failovers reproduces this reliably). Fix the abort path so that the request is removed from its wait list and from the request tree when the waiter gives up, making the waiter's put the final reference on a request that is no longer linked anywhere. Both operations happen under mdsc->mutex; a racing reply or forward may already have unregistered the request, so only unregister it when it is still in the tree. Fixes: e1518c7c0a67a ("ceph: clean up mds reply, error handling") Cc: stable@vger.kernel.org Signed-off-by: Xiubo Li --- The full call trace: ------------[ cut here ]------------ refcount_t: addition on 0; use-after-free. WARNING: lib/refcount.c:25 at refcount_warn_saturate+0x6a/0x90, CPU#1: kwo= rker/1:1/99 Modules linked in: ceph libceph krb5 netfs xsk_diag vsock_diag uinput snd_= seq_dummy snd_hrtimer rfkill nf_conntrack_netbios_ns nf_conntrack_broadcast= nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_i= pv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_d= efrag_ipv6 nf_defrag_ipv4 nf_tables qrtr sunrpc intel_rapl_msr intel_rapl_c= ommon intel_uncore_frequency_common intel_pmc_core pmt_telemetry pmt_discov= ery pmt_class intel_pmc_ssram_telemetry intel_pmc_pwrm_telemetry intel_vsec= kvm_intel kvm snd_hda_codec_generic irqbypass snd_hda_intel rapl snd_hda_c= odec snd_hda_core snd_intel_dspcfg snd_intel_sdw_acpi snd_hwdep snd_seq iTC= O_wdt snd_seq_device intel_pmc_bxt snd_pcm i2c_i801 snd_timer pcspkr i2c_sm= bus snd lpc_ich virtio_balloon soundcore joydev zram lz4hc_compress vmw_vso= ck_virtio_transport vmw_vsock_virtio_transport_common vsock virtio_net net_= failover virtio_gpu failover virtio_dma_buf serio_raw i2c_dev qemu_fw_cfg v= irtiofs fuse CPU: 1 UID: 0 PID: 99 Comm: kworker/1:1 Tainted: G D 7.2.= 0-rc7-lockdep+ #13 PREEMPT(lazy) Tainted: [D]=3DDIE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.17.0-10.fc44 06= /10/2025 Workqueue: ceph-msgr ceph_con_workfn [libceph] RIP: 0010:refcount_warn_saturate+0x6a/0x90 Code: cc 48 8d 3d b8 f6 d4 01 67 48 0f b9 3a c3 cc cc cc cc 48 8d 3d b7 f6= d4 01 67 48 0f b9 3a c3 cc cc cc cc 48 8d 3d b6 f6 d4 01 <67> 48 0f b9 3a = c3 cc cc cc cc 48 8d 3d b5 f6 d4 01 67 48 0f b9 3a RSP: 0018:ffffcdb6403a7cc0 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffff8d5e6441c000 RCX: ffff8d5e43de6a60 RDX: ffff8d5e5afbd468 RSI: 0000000000000002 RDI: ffffffffa7fbe040 RBP: ffffcdb6403a7ce8 R08: 0000000000000001 R09: ffff8d5e5afbd550 R10: ffffcdb6403a7cc8 R11: 0000000000000000 R12: ffff8d5e6441c000 R13: ffff8d5e5afbd028 R14: ffff8d5e515c0000 R15: ffff8d5e5a621008 FS: 0000000000000000(0000) GS:ffff8d61f7904000(0000) knlGS:00000000000000= 00 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f0541dbf000 CR3: 000000011ad8b005 CR4: 0000000000772ef0 PKRU: 55555554 Call Trace: __wake_requests+0x3af/0x3d0 [ceph] mds_dispatch+0xad/0x240 [ceph] ceph_con_process_message+0x88/0x1c0 [libceph] ceph_con_v1_try_read+0x316/0x6f0 [libceph] ceph_con_workfn+0x1f4/0x500 [libceph] ? process_one_work+0x20d/0x600 process_one_work+0x234/0x600 worker_thread+0x1e5/0x3c0 ? __pfx_worker_thread+0x10/0x10 kthread+0xf5/0x130 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x24a/0x360 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 irq event stamp: 7923424 hardirqs last enabled at (7923423): [] _raw_spin_unlock= _irq+0x28/0x50 hardirqs last disabled at (7923424): [] __schedule+0x685= /0x820 --- fs/ceph/mds_client.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c index 85f8ceb10377..6b27146d0c9f 100644 --- a/fs/ceph/mds_client.c +++ b/fs/ceph/mds_client.c @@ -4024,6 +4024,19 @@ int ceph_mdsc_wait_request(struct ceph_mds_client *m= dsc, if (req->r_parent && (req->r_op & CEPH_MDS_OP_WRITE)) ceph_invalidate_dir_request(req); + + /* + * The waiter is about to drop its reference, which is + * the last one held by the request's caller. Make sure + * the request is no longer parked on a wait list or + * registered in the request tree, or it could be walked + * (and in the tree case, replied to) after it is freed. + * A racing reply may already have unregistered it, in + * which case it is no longer in the tree. + */ + list_del_init(&req->r_wait); + if (lookup_request(&mdsc->request_tree, req->r_tid) =3D=3D req) + __unregister_request(mdsc, req); } else { err =3D req->r_err; } --- base-commit: dee30ce1286a0d18b14545ecac345e4cf4a80511 change-id: 20260827-b4-ceph-fix-abort-request-uaf-new-a98f8c6da00e Best regards, -- =20 Xiubo Li