From nobody Sat Sep 26 03:17:41 2026 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78655499F1B for ; Fri, 4 Sep 2026 15:03:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788534195; cv=none; b=iieu/uy9W1b0G7kxxoH9sl+xqY+usK+ndaY4BfG2Vsw6i3nxilHrMTOB/Rsv8UVImfuI5D9HAjNQXnxprL3UOt4jkI1Co0VZ2TZ9ySBla2OKDh1b3psrklo76R0QYYRcmiYw2G1/d0ddo5E6cJuewVWo5iiESWYdXNz/SwcA7TI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788534195; c=relaxed/simple; bh=rJMGxltpZAJR+jvcsSWb1sWqev/1OdfAjfWAfOaf5p0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=blwJgl+hlpCdfrkSBxDLM/S/5QQUrclpIJhumtKtR9+emVCddXQRgNGKGp9tkbh6cmIPN+frTrp8TvEiBHMnXnkvyM+bBq3uqkX/XBRCIWyuE0OfqyLGGj29vxTSOmAMEERm2J7wAaOGbeQyzo9MiTpukF6NYsvfwaX6p6pFFeQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=NOPau+o7; arc=none smtp.client-ip=209.85.218.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="NOPau+o7" Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-c2531f453eeso147624966b.3 for ; Fri, 04 Sep 2026 08:03:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1788534191; x=1789138991; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=nzoyfcOHeKv+tpIRgc2EL6jwYs5rJEcNEzi5ToP2Q7A=; b=NOPau+o7lFVkyOQu82/0x2kbPX/Qjl1gUl2J4269yEES58mAR6CYHq3AinAEVjG1FN CNF/6oHiGApxRGEI8ifXSp/7t29VbxLHTwqSJOqxtOWkba5yUqVveRko4SvBqcJas6gY kW7npbpnfQUiX0mgOZyKcJLgEljcPv414UvlLNj3IqRg116kj1hCDIjSwex2KOT7F0/7 eyndeizrwnR6PEhl9lEF4d7fRTEcJfgHKxDB/mlUQ7Pz78rx9Jy2xef9MhMl3hob2x9E esj8mFsH/YCyISw8CPjeGQdUsQ8IgoTHRY264QlGcXZms9dy/+2VPG1tPYoGT63ieHHP mdVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788534191; x=1789138991; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=nzoyfcOHeKv+tpIRgc2EL6jwYs5rJEcNEzi5ToP2Q7A=; b=SdpstBt+odfD+18nY103WsFqT8oIvN9GMGyc+tfV2qjGZepTxMRLyOZeNOFEkeFjvh XJK/8pxPIuJ1rGvodjrLosnZpTVJ7Jkxp+c7DKEm0yYPs87/65XrKc6O2V2iFMujC4zn jxMmJsylYKYNXbtBIUB7RYS7spcGJgDXSrSWu1J2tX9NiyM3dG9dbmb5397AhwU845EF QHPiz0RrVrzJ4tJe8kHZWPqgNkL0ztEYbKJNxmvqyQWggIspneWXaGei/MbUIHUfMH5b vjVuh93tUbwfqbaSnbWvEsemGF/FvHgWamFT57g0yXY+oOR9WJpMpML0g+HT6XW7A+rE wTJw== X-Forwarded-Encrypted: i=1; AKwUvBwLWk7vuoeEWZBUmHq/uE8ChyuEVWfPOP5NGWdq4D91q2KQ9Qd4J3q+b7aixarOrtKiq4RWGkauM6EdhH4=@vger.kernel.org X-Gm-Message-State: AFuF++mwKwgajN8ekJFONoxABQiQQTAxcLbBqR6oMCbYMp2T3hoUbVA2 O34ia4Xklahc3MqoWqrF5yCvEqfTH4S8PuI9aFqkaRcqJfUlhidPWJWSqcHcmZZwM3k= X-Gm-Gg: AYBFou25xZqDTzMy+9ZQn6Hfi0msx7mqTpuPIyH24NP5G3vbfMJLiON7MEm2GLlCEYu TaSkdWzUhR0AsrbLYeVjc/2u2W9mSXTbemlAUYFWHwvG0gAy92e42KWoqfVT/nS0Gh181C9Louh HHVI9+OPqLWnhOXR/YVeNtU9BV25lkNHq5fCVzMaxEsTH1zlqupE43MGNzH7lMr9Ym2CVRQ2Vd0 IgOPFiepmItRbl2L4EifQd9gG83wliu5iNWdnLGFJjFIvwFBk0VEbykkLSn9d66+FDn3n+wG28r yhc2RugOM0BwkBeA+aYWtcF4Ny6eKC2emQG4bqLQj4OE/0rZoQ0UgQA9kY0wDkZ5njkpZss9A+5 ErXEiNirI+u8lpQp8o35QT0tvWar0e5Qkj2gy4uVSnPwHBDBngLD9y851xgIRjwU3ki+d7lQPjl /+Rmw0j9cTM3ZZg9klypB46gqsRRUQbeLSMMa1YSUsGKmYmzSibWCbrOQWa+fgZfGD94V2aH0B3 sZc6RbKCX49t6IpA4850WxMWq2VgYAIPeNfftXklcAjbVqrdgCdZhVlDzffrU+PKBE/1+oNTHsw fQ== X-Received: by 2002:a17:906:f586:b0:c20:1bc5:7002 with SMTP id a640c23a62f3a-c260c6a0e2emr465899466b.3.1788534189151; Fri, 04 Sep 2026 08:03:09 -0700 (PDT) Received: from raven.intern.cm-ag (p200300dc6f04b700023064fffe740809.dip0.t-ipconnect.de. [2003:dc:6f04:b700:230:64ff:fe74:809]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c260d6e1c93sm122438866b.63.2026.09.04.08.03.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Sep 2026 08:03:08 -0700 (PDT) From: Max Kellermann To: idryomov@gmail.com, amarkuze@redhat.com, xiubo.li@clyso.com, ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Max Kellermann , stable@vger.kernel.org Subject: [PATCH v4 1/3] ceph: fix use-after-free in check_new_map() after early session put Date: Fri, 4 Sep 2026 17:03:02 +0200 Message-ID: <20260904150304.49104-2-max.kellermann@ionos.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260904150304.49104-1-max.kellermann@ionos.com> References: <20260904150304.49104-1-max.kellermann@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" check_new_map() drops mdsc->mutex while it locks a session, prepares a reconnect, or kicks flushing caps. A concurrent teardown can call __unregister_session() in that window and drop the sessions[] reference. When check_new_map() reacquires mdsc->mutex, its temporary reference may therefore be the only reference keeping the local variable `s` alive. Commit ee611a750955 ("ceph: fix UAF in check_new_map() on session freed during unlock") addressed this by taking a temporary reference around each unlock window, but it drops that reference as soon as mdsc->mutex is reacquired, while `s` is still in use: ceph_get_mds_session(s); mutex_unlock(&mdsc->mutex); mutex_lock(&s->s_mutex); mutex_lock(&mdsc->mutex); ceph_put_mds_session(s); /* may drop the last reference */ ceph_con_close(&s->s_con); /* use-after-free */ mutex_unlock(&s->s_mutex); /* use-after-free */ s->s_state =3D CEPH_MDS_SESSION_RESTARTING; If the session was unregistered during the window, the sessions[] reference is already gone, so this ceph_put_mds_session() frees it. Additionally, the session could be freed while its s_mutex is locked, which trips the WARN_ON(mutex_is_locked(&s->s_mutex)) in ceph_put_mds_session() and then unlocks freed memory. Fix this by holding a single reference for the whole loop iteration: look the session up with __ceph_lookup_mds_session(), which returns it with a reference held, and release that reference on every exit from the loop body. This subsumes the per-window get/put pairs, so remove them. Fixes: ee611a750955 ("ceph: fix UAF in check_new_map() on session freed dur= ing unlock") Cc: stable@vger.kernel.org Signed-off-by: Max Kellermann Reviewed-by: Xiubo Li --- Note the stable maintainers: this is a fixup for ee611a750955, but the bug has existed before; see https://lore.kernel.org/ceph-devel/20260828174504.1247038-2-max.kellermann@= ionos.com/ for a patch that applies to pre-7.2 kernel versions. --- fs/ceph/mds_client.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c index a091f77cedaf..d36a114747ae 100644 --- a/fs/ceph/mds_client.c +++ b/fs/ceph/mds_client.c @@ -5860,9 +5860,9 @@ static void check_new_map(struct ceph_mds_client *mds= c, } =20 for (i =3D 0; i < oldmap->possible_max_rank && i < mdsc->max_sessions; i+= +) { - if (!mdsc->sessions[i]) + s =3D __ceph_lookup_mds_session(mdsc, i); + if (!s) continue; - s =3D mdsc->sessions[i]; oldstate =3D ceph_mdsmap_get_state(oldmap, i); newstate =3D ceph_mdsmap_get_state(newmap, i); =20 @@ -5875,7 +5875,6 @@ static void check_new_map(struct ceph_mds_client *mds= c, =20 if (i >=3D newmap->possible_max_rank) { /* force close session for stopped mds */ - ceph_get_mds_session(s); __unregister_session(mdsc, s); __wake_requests(mdsc, &s->s_waiting); mutex_unlock(&mdsc->mutex); @@ -5896,15 +5895,14 @@ static void check_new_map(struct ceph_mds_client *m= dsc, ceph_mdsmap_get_addr(newmap, i), sizeof(struct ceph_entity_addr))) { /* just close it */ - ceph_get_mds_session(s); mutex_unlock(&mdsc->mutex); mutex_lock(&s->s_mutex); mutex_lock(&mdsc->mutex); - ceph_put_mds_session(s); ceph_con_close(&s->s_con); mutex_unlock(&s->s_mutex); s->s_state =3D CEPH_MDS_SESSION_RESTARTING; } else if (oldstate =3D=3D newstate) { + ceph_put_mds_session(s); continue; /* nothing new with this mds */ } =20 @@ -5915,7 +5913,6 @@ static void check_new_map(struct ceph_mds_client *mds= c, newstate >=3D CEPH_MDS_STATE_RECONNECT) { int rc; =20 - ceph_get_mds_session(s); mutex_unlock(&mdsc->mutex); clear_bit(i, targets); rc =3D send_mds_reconnect(mdsc, s); @@ -5924,7 +5921,6 @@ static void check_new_map(struct ceph_mds_client *mds= c, "mds%d reconnect failed: %d\n", i, rc); mutex_lock(&mdsc->mutex); - ceph_put_mds_session(s); } =20 /* @@ -5937,15 +5933,14 @@ static void check_new_map(struct ceph_mds_client *m= dsc, pr_info_client(cl, "mds%d recovery completed\n", s->s_mds); kick_requests(mdsc, i); - ceph_get_mds_session(s); mutex_unlock(&mdsc->mutex); mutex_lock(&s->s_mutex); mutex_lock(&mdsc->mutex); - ceph_put_mds_session(s); ceph_kick_flushing_caps(mdsc, s); mutex_unlock(&s->s_mutex); wake_up_session_caps(s, RECONNECT); } + ceph_put_mds_session(s); } =20 /* --=20 2.47.3 From nobody Sat Sep 26 03:17:41 2026 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A74DB4E8E1E for ; Fri, 4 Sep 2026 15:03:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788534197; cv=none; b=CCqcTJuFsCF/CVywRsKE55gkTQcvKKLcAgyXAoAikcdgcyeaPB+FfKopdgYdUL74HCG4ySvsogB2PqRKKgXnrFM/rBc1szr7Y5WrKA8QGJIMiEzWY2ljMWtdq84wfpablcVJ2mPvNhmUqY9/upZ/F/FpnLqoQGNJxUvHjiENLvQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788534197; c=relaxed/simple; bh=RaXX8VJyPOJNRhZBa3bCoTSLDAkhaKdHEayJvkI1wiw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F7NinsnOp5bgOPXaUAyOFlyV03hr4gNjZYdL+I5YE2E3NToAUw+Ubv/7S1qn2h+Fp0UMSZ6FgawKiZHLt0MxOPprcO60HoH19FJ9qEgVj6iN9TjFSm2Xl57+g3ecvJOd8/G9juP/U0M1v2mePMEFYY0k2L8+SKq5aGWFSWRwL7k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=bU+1iRh3; arc=none smtp.client-ip=209.85.208.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="bU+1iRh3" Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-6a20319d030so1629948a12.2 for ; Fri, 04 Sep 2026 08:03:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1788534192; x=1789138992; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=wHU7q3pRFtkKql8fEyqkLOSTSzzZohiVwSCb3KFef/E=; b=bU+1iRh3L54e2zbgbo5LDZ6pzXO4k+4SdX7gTmzc26AWX/bocayi+xvPpIf+lKNuke IpJbRYD8Tsp6GvIVtmnksDXgxyHQJEU7GgzE4uTyAcrDNSzXPhZe0egZ2TJ6PB9a0FHl iL+EqkLDRvOxhGzD3XtxFl20kCAKkG6TTGYT0DaQYFs6hDf6m6usnWsmz6q5+fUb0mpX weY1GAvwhNl023jzuaN+gs7E+RE55b/HQGX8QPP4DF5R2S+PpTHHhMfFZLU66GobfU4o 00mdiTSJVqm/c2suXs6T2UQq5svuwFkE/GOPKBe0MEIDiwwqqfkRwaJSs4NWnbsrW1S5 hj9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788534192; x=1789138992; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=wHU7q3pRFtkKql8fEyqkLOSTSzzZohiVwSCb3KFef/E=; b=QzfqFXyY8+c+NSOYtczJZ2Un7W3G5BXizkz/ripTTRkVQnvidwvKeh3M0j3l5x3Uug F0JUyYW5BVQmUQXCaW9Qje99vi5VpEgJoqxzG5JxbtGtedhLrkDXcEV3am7hjZzB3Dv7 L5asaocRiVCghPhUXpPb2sqC3GpkqrMvnZGNYyXUP5yJDvW9SwyYbfhDsN/avIN1bY0K Tt0n7tlTeo2DvbOfUX+bMFJ2RaKyZgiaekXciLEZYL2xSgd3cRZjcNLMRH0xas599CuI 71BFZKkHCw40wNxENcPHajfJ6zZH7QPo3VlULFSbusokEMQp5q8MgaJY6ZOg1nYi+xs+ xZjQ== X-Forwarded-Encrypted: i=1; AKwUvBzl9Wcxx8dCR4eBfOvuFsoI9lI2ZOYqEhL+3tfQRyS8aPSo6ql1Nc9YHtt1VHOAmYENXstYZRQlwfHP3Gk=@vger.kernel.org X-Gm-Message-State: AFuF++lsoxRjPMvxlTbZ8AuO42KpRyvJgFqOmi6hCHt6laG3F1E5Ngf8 /l4pgD4Z19z9vZr9pciLhHT0fMrfh3NnJ9WlA1V20/yXSOk/QHW2jP0uzsJjbskTmmI= X-Gm-Gg: AYBFou31lK/hPTGy7Wb6khaLYrdBuqWjYl7rwpulWiQLoJzAr0UipRwkjjIn9mXVfYx WDwsAemQaPQvqou+AwhgkwpvNZ4zIkPL4JhO8i/xrFJzVoCqBcSt12QdknH1gXkH+pZgN5fUETW MgiWa15SQcGvlX/qAmzyLjHwz/7iPoTNZ7z0A2o8wBI4vmn09H/Nv2ztwSmqHP6Y+tDPReD+Uhh 84Cs8OZ9TapYBrHylUokZNTjgjD1GPveNb2JltTow+XTykSkh9Wgfl4IN4S0Xhio2vsLrazwrUY lRj37+19AiP2fum11k/QMArYoZsOMkJIZ/CZm3djlr/puDHOYnH5JoVUK0qULcC/Tb5PqZumMWL HX4ZlTO74YGmEU2TCSV34Gq44gaMgTveoTO6oRUgvLNWZnDafGYXFQQgnCzg7o9VZ0E+DV/F6Ib z0QNW0eGLpO7Ipg0wBblggT2oCbD3FRHNAJJK9cmCDCRbdaC0brPuCqgLUO84xnykSrH/B3cF49 IYWfcBQKCjaWKIcmJz13gAUbmrQpmEtzNCRflJESX0m/vvGikT95WrFQ+zIetlx X-Received: by 2002:a17:907:3f95:b0:c25:ee8d:99ff with SMTP id a640c23a62f3a-c260cb45800mr321415566b.21.1788534191627; Fri, 04 Sep 2026 08:03:11 -0700 (PDT) Received: from raven.intern.cm-ag (p200300dc6f04b700023064fffe740809.dip0.t-ipconnect.de. [2003:dc:6f04:b700:230:64ff:fe74:809]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c260d6e1c93sm122438866b.63.2026.09.04.08.03.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Sep 2026 08:03:11 -0700 (PDT) From: Max Kellermann To: idryomov@gmail.com, amarkuze@redhat.com, xiubo.li@clyso.com, ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Max Kellermann , stable@vger.kernel.org Subject: [PATCH v4 2/3] ceph: stop checking a stale MDS map after dropping mutex Date: Fri, 4 Sep 2026 17:03:03 +0200 Message-ID: <20260904150304.49104-3-max.kellermann@ionos.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260904150304.49104-1-max.kellermann@ionos.com> References: <20260904150304.49104-1-max.kellermann@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" check_new_map() drops `mdsc->mutex` in several slow paths. Another map handler can install a newer map and destroy the map being checked before the original invocation re-locks the mutex. Continuing to dereference newmap after that results in a use-after-free. Use the `mdsmap->m_epoch` field to identify stale maps. Fixes: 2f2dc053404f ("ceph: MDS client") Cc: stable@vger.kernel.org Signed-off-by: Max Kellermann Reviewed-by: Xiubo Li --- fs/ceph/mds_client.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c index d36a114747ae..86d592f06196 100644 --- a/fs/ceph/mds_client.c +++ b/fs/ceph/mds_client.c @@ -5844,6 +5844,7 @@ static void check_new_map(struct ceph_mds_client *mds= c, struct ceph_mdsmap *newmap, struct ceph_mdsmap *oldmap) { + u32 map_epoch =3D newmap->m_epoch; int i, j, err; int oldstate, newstate; struct ceph_mds_session *s; @@ -5887,6 +5888,8 @@ static void check_new_map(struct ceph_mds_client *mds= c, ceph_put_mds_session(s); =20 mutex_lock(&mdsc->mutex); + if (mdsc->mdsmap->m_epoch !=3D map_epoch) + return; kick_requests(mdsc, i); continue; } @@ -5898,6 +5901,11 @@ static void check_new_map(struct ceph_mds_client *md= sc, mutex_unlock(&mdsc->mutex); mutex_lock(&s->s_mutex); mutex_lock(&mdsc->mutex); + if (mdsc->mdsmap->m_epoch !=3D map_epoch) { + mutex_unlock(&s->s_mutex); + ceph_put_mds_session(s); + return; + } ceph_con_close(&s->s_con); mutex_unlock(&s->s_mutex); s->s_state =3D CEPH_MDS_SESSION_RESTARTING; @@ -5921,6 +5929,10 @@ static void check_new_map(struct ceph_mds_client *md= sc, "mds%d reconnect failed: %d\n", i, rc); mutex_lock(&mdsc->mutex); + if (mdsc->mdsmap->m_epoch !=3D map_epoch) { + ceph_put_mds_session(s); + return; + } } =20 /* @@ -5936,6 +5948,11 @@ static void check_new_map(struct ceph_mds_client *md= sc, mutex_unlock(&mdsc->mutex); mutex_lock(&s->s_mutex); mutex_lock(&mdsc->mutex); + if (mdsc->mdsmap->m_epoch !=3D map_epoch) { + mutex_unlock(&s->s_mutex); + ceph_put_mds_session(s); + return; + } ceph_kick_flushing_caps(mdsc, s); mutex_unlock(&s->s_mutex); wake_up_session_caps(s, RECONNECT); @@ -5991,6 +6008,8 @@ static void check_new_map(struct ceph_mds_client *mds= c, i, err); ceph_put_mds_session(s); mutex_lock(&mdsc->mutex); + if (mdsc->mdsmap->m_epoch !=3D map_epoch) + return; } =20 for (i =3D 0; i < newmap->possible_max_rank && i < mdsc->max_sessions; i+= +) { --=20 2.47.3 From nobody Sat Sep 26 03:17:41 2026 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBC0B4EBAEC for ; Fri, 4 Sep 2026 15:03:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788534197; cv=none; b=aqC36SAZi1wSppnzlOYQDWiABmFmy6Xeu8iAGffgUvvnELqYKsQo70HN4M8mWsT/ln55YraUz1PZUMuHiqW6d/b2aK5BjvR3dSXsivm2wS0GUxMI44iOdr4Y969vFRqR8bClooZeBNPLnCE7OgeNTNokAuFP3w4SEFkMEqf+JQQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788534197; c=relaxed/simple; bh=6RcqfWTpR9cSAA7gQiwQ0i4hUp/8e4bbFr3Xg/BPp9k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p70aUH7D+0/ShoRc3LnJBB462AOGk7v9IantFGe5qhQeQqu2CZHv5EKzW/OuO9tRAvFbla86u3FcmRHH5VjNpG+McW9DxbwyI0WbkJJDVx8cxYlFC9E5crD0H2R95did67QaxuUK5ptUJpS2bxz4FfxyIt1ty4ONatyRsGfPRWY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=W0KNwZ9z; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="W0KNwZ9z" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-c15e592da74so132479366b.1 for ; Fri, 04 Sep 2026 08:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1788534193; x=1789138993; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=NPdoZec4iRzjfMWUlv4D9QHImQ8qSXuSr3RPA6jeV48=; b=W0KNwZ9z8LmrZGcIAtOzKp+85vNnLoxXfmkkKfcKvn3ZSHhX4eZYcEmyhly1T1/yJF B+bxLSxaitpKR2aWAtXNy0YSG6s2ZL/BUN+pZBXrId212byn+FluHRKV7B1PtxKBMLcq +/kBMqzjy6ZETHZtMIrLvzG+0/FKRX8uxs9PZiyWLmNVBx5gwTE+kHVjcfdDJbDFppg5 +BQCJb39l+P0DsaD8Eh7ON6INsFDl+CrclB8fN/FvdxJL6+p+BtRU8sl2OrGCPb1gJvd Fj1U9owcnSRahjo2d//07lW1sUmB2d0BuJPdeS3D1JX7XP+nFIjmZ1bK3K8WfwIiduuy VNDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788534193; x=1789138993; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=NPdoZec4iRzjfMWUlv4D9QHImQ8qSXuSr3RPA6jeV48=; b=RSHctd2AYb9/+LI6cnEUb0jXsLXmlrgDGDQekSjGxAjH02SpH5XaymDB7AbZKKCPZt nbh32Gs9AUL+mju0kWJoucSjiZbuM+yN7SMAvsrJlAZt9/Rfn19f+DR2e6ciqatbi+k0 CTH/MQkJiZd/YKi5Sz8S+nnW/MpyEK9lCbdtFlcQSbMYuBNLorLyxjus1lYjRfq7AMp6 OQr/0kMaj2xTMp9EwmXycViFF6jKVyepLYlKcGcnwJB6VRapTQlJ9IZqB1S3CIjEeSSd BQz2ffoVu6CPP9PE182qUiftU5+nDddcePzaUKUaMQHdATA0sqF5FuvdzgFxnJ6b7Fq5 CBLQ== X-Forwarded-Encrypted: i=1; AKwUvBxBIPyygeg/jUkj/WiltMDD7Zm583NrTMtbMXWDgoL3WmzpZJG38RGRnyL/DpJwf6Yv9AbTj6IfH7G+i5g=@vger.kernel.org X-Gm-Message-State: AFuF++my5oDMWjQPxo231PCsQM6c9B9cA7JyyneIq7nrWhScaN9Tsnhg fq+r/DKgqY4ORWj0F6in1kzqhmvGu4KhVE2NRErCdIymDz5zK4ANXbIhlIUg5PtkiLs= X-Gm-Gg: AYBFou3fSpbbDfjbXDA51vSMR8gTqrJDTTWsm/fEbp6gS3N/obb8SOWUIstfeU3hQGM 9Nuh6ibYKcp7XX/FSpWlb9SnyfxmF5Qo8pyciOCjt5524H98/MS1J5io4t6cBqNca/sf89Wq986 OHCtp07UJ07zxCb6bCHEcJh5sNuE94LXgkpupbu6/SoO3iEe6dviByh9jPzzwytOXV2vUNUaHgY 6aaS9afoj2yYFdfgmRy+unKawDG05ZKu190KcrPi4EKH7WFl4Hl7O+PL2UHy4u+eeF93nmuX2+Q LQOUQT7kzCxMZz2e6XloC20Pa2FGWBDKcVcgBX56FkE7vqmu7GNX63hGfN4n0VGe9E4cbwrP3VU JWAzHUbn98C8MhasUq/BbghpIdPiYTwMGbBh4lB3XX3YT4IyUqjxrClFpZWWHvwrqiovGEkLJ3/ C/8segHX/Nvb7jN/TI4ouem1HNM7jEDGpuYaiVb5V1P//SmnpY/KujIIDvu43IFR2HCS5v5UGX1 QgPFwfgmKM/tfNJ3A4nTtlEVl8TMha1f1+vErp2jRxe+J/YQMi2cBmqYT0rOtQ/ X-Received: by 2002:a17:907:a892:b0:c20:3402:bd35 with SMTP id a640c23a62f3a-c260c988d8cmr286054566b.12.1788534192795; Fri, 04 Sep 2026 08:03:12 -0700 (PDT) Received: from raven.intern.cm-ag (p200300dc6f04b700023064fffe740809.dip0.t-ipconnect.de. [2003:dc:6f04:b700:230:64ff:fe74:809]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c260d6e1c93sm122438866b.63.2026.09.04.08.03.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Sep 2026 08:03:12 -0700 (PDT) From: Max Kellermann To: idryomov@gmail.com, amarkuze@redhat.com, xiubo.li@clyso.com, ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Max Kellermann , stable@vger.kernel.org Subject: [PATCH v4 3/3] ceph: don't unregister an MDS session before removing its caps Date: Fri, 4 Sep 2026 17:03:04 +0200 Message-ID: <20260904150304.49104-4-max.kellermann@ionos.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260904150304.49104-1-max.kellermann@ionos.com> References: <20260904150304.49104-1-max.kellermann@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" handle_session() removed the session from mdsc->sessions[] at the very top of `CEPH_SESSION_CLOSE` handling, before taking `s_mutex`. Between session unregistration and remove_session_caps(), the MDS rank has no registered session while the old session still owns all caps it was granted. Any concurrent filesystem operation may walk into that and the next __do_request() call registers a new session for this rank. Once it is open and the MDS issues caps, ceph_fill_inode() calls ceph_add_cap(), which looks caps up by rank, not by session identity, finding old caps linked to the old session. The list_move_tail() call then moves the cap object to the new session, which is already a bad thing to do. Since it doesn't decrement `old_session->s_nr_caps`, this quickly triggers: kernel BUG at fs/ceph/mds_client.c:1959! Internal error: Oops - BUG: 00000000f2000800 [#1] SMP [...] Workqueue: ceph-msgr ceph_con_workfn pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=3D--) pc : remove_session_caps+0x2bc/0x2d8 lr : remove_session_caps+0x74/0x2d8 [...] Call trace: remove_session_caps+0x2bc/0x2d8 (P) mds_dispatch+0xf48/0x1b60 ceph_con_process_message+0x74/0xa0 ceph_con_v1_try_read+0x3a0/0x1510 ceph_con_workfn+0x260/0x460 process_one_work+0x168/0x3b8 worker_thread+0x1bc/0x3a0 kthread+0x118/0x1e0 ret_from_fork+0x10/0x20 That's BUG_ON(session->s_nr_caps > 0). I was able to reproduce this reliably by delaying the close and starting I/O during the delay. This patch keeps the session registered with `CEPH_MDS_SESSION_CLOSED`. New requests will be put on the `s_waiting` list where they will be resumed on the new session. Since `CLOSE` now sets `CLOSED` before taking `s_mutex`, make send_mds_reconnect() validate and update the session state under `mdsc->mutex`. Preserve `CLOSED` if reconnect preparation later fails. Fixes: 2600d2dd5085 ("ceph: drop messages on unregistered mds sessions; cle= anup") Cc: stable@vger.kernel.org Signed-off-by: Max Kellermann Reviewed-by: Xiubo Li --- fs/ceph/caps.c | 4 + fs/ceph/mds_client.c | 182 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 156 insertions(+), 30 deletions(-) diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index bcb04c6cb92c..024714d7f34e 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -4261,6 +4261,10 @@ static void handle_cap_export(struct inode *inode, s= truct ceph_mds_caps *ex, } new_cap =3D ceph_get_cap(mdsc, NULL); } else { + if (tsession =3D=3D ERR_PTR(-EAGAIN)) + /* locks already dropped */ + return; + WARN_ON(1); tsession =3D NULL; target =3D -1; diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c index 86d592f06196..f091f0eaafc2 100644 --- a/fs/ceph/mds_client.c +++ b/fs/ceph/mds_client.c @@ -1783,6 +1783,30 @@ static int __open_session(struct ceph_mds_client *md= sc, return 0; } =20 +/* Is this rank still occupied by a session being torn down? */ +static bool __mds_rank_closing(struct ceph_mds_client *mdsc, int mds) +{ + return mds < mdsc->max_sessions && mdsc->sessions[mds] && + mdsc->sessions[mds]->s_state =3D=3D CEPH_MDS_SESSION_CLOSED; +} + +/* + * Wait until the rank is no longer occupied by a CLOSED session. + * + * The caller must hold mdsc->mutex. The mutex is dropped while sleeping = and + * held again on return. Another session may occupy the rank by then. The + * wait also ends when shutdown starts, so the caller must check + * mdsc->stopping before proceeding. + */ +static void wait_for_mds_rank_not_closing(struct ceph_mds_client *mdsc, + int mds) +{ + wait_event_cmd(mdsc->session_close_wq, + !__mds_rank_closing(mdsc, mds) || mdsc->stopping, + mutex_unlock(&mdsc->mutex), + mutex_lock(&mdsc->mutex)); +} + /* * open sessions for any export targets for the given mds * @@ -1800,6 +1824,16 @@ __open_export_target_session(struct ceph_mds_client = *mdsc, int target) if (IS_ERR(session)) return session; } + if (session->s_state =3D=3D CEPH_MDS_SESSION_CLOSED) { + /* + * handle_session() is currently closing this session; + * it stays registered until its caps are gone. Do + * not return it to our caller because we don't want + * it to attach new caps to it. + */ + ceph_put_mds_session(session); + return ERR_PTR(-EAGAIN); + } if (session->s_state =3D=3D CEPH_MDS_SESSION_NEW || session->s_state =3D=3D CEPH_MDS_SESSION_CLOSING) { ret =3D __open_session(mdsc, session); @@ -1819,7 +1853,19 @@ ceph_mdsc_open_export_target_session(struct ceph_mds= _client *mdsc, int target) doutc(cl, "to mds%d\n", target); =20 mutex_lock(&mdsc->mutex); - session =3D __open_export_target_session(mdsc, target); + for (;;) { + session =3D __open_export_target_session(mdsc, target); + if (session !=3D ERR_PTR(-EAGAIN) || mdsc->stopping) + break; + + /* + * Keep the exported cap on its old session until the + * target rank is vacant. Dropping it here can discard a + * dirty auth cap; handle_session() wakes us after removing + * the old target session's caps and unregistering it. + */ + wait_for_mds_rank_not_closing(mdsc, target); + } mutex_unlock(&mdsc->mutex); =20 return session; @@ -4553,8 +4599,20 @@ static void handle_session(struct ceph_mds_session *= session, ceph_metric_bind_session(mdsc, session); } if (op =3D=3D CEPH_SESSION_CLOSE) { + /* + * Pin the session for the rest of this function. The + * __unregister_session() call is deferred until after + * remove_session_caps() below, or else other + * processes may find caps still assigned to this + * session while working with a new session object. + */ ceph_get_mds_session(session); - __unregister_session(mdsc, session); + + if (session->s_state =3D=3D CEPH_MDS_SESSION_RECONNECTING) + pr_info_client(cl, "mds%d reconnect denied\n", + session->s_mds); + + session->s_state =3D CEPH_MDS_SESSION_CLOSED; } /* FIXME: this ttl calculation is generous */ session->s_ttl =3D jiffies + HZ*mdsc->mdsmap->m_session_autoclose; @@ -4612,12 +4670,24 @@ static void handle_session(struct ceph_mds_session = *session, break; =20 case CEPH_SESSION_CLOSE: - if (session->s_state =3D=3D CEPH_MDS_SESSION_RECONNECTING) - pr_info_client(cl, "mds%d reconnect denied\n", - session->s_mds); - session->s_state =3D CEPH_MDS_SESSION_CLOSED; cleanup_session_requests(mdsc, session); remove_session_caps(session); + + /* + * Now that all caps are removed, it is safe to release + * the MDS rank and allow other processes to create a + * new session object. + * + * A concurrent ceph_mdsc_close_sessions() or + * check_new_map() may have unregistered the session + * already, so check __verify_registered_session() + * first. + */ + mutex_lock(&mdsc->mutex); + if (!__verify_registered_session(mdsc, session)) + __unregister_session(mdsc, session); + mutex_unlock(&mdsc->mutex); + wake =3D 2; /* for good measure */ wake_up_all(&mdsc->session_close_wq); break; @@ -5168,16 +5238,6 @@ static int send_mds_reconnect(struct ceph_mds_client= *mdsc, /* Serialized by s_mutex against concurrent ceph_get_deleg_ino(). */ xa_destroy(&session->s_delegated_inos); atomic_set(&session->s_num_deleg_inos, 0); - if (session->s_state =3D=3D CEPH_MDS_SESSION_CLOSED || - session->s_state =3D=3D CEPH_MDS_SESSION_REJECTED) { - pr_info_client(cl, "mds%d skipping reconnect, session %s\n", - mds, - ceph_session_state_name(session->s_state)); - mutex_unlock(&session->s_mutex); - ceph_msg_put(reply); - err =3D -ESTALE; - goto fail_return; - } =20 /* s_mutex -> mdsc->mutex matches cleanup_session_requests() order. */ mutex_lock(&mdsc->mutex); @@ -5191,12 +5251,23 @@ static int send_mds_reconnect(struct ceph_mds_clien= t *mdsc, err =3D -ENOENT; goto fail_return; } - mutex_unlock(&mdsc->mutex); + if (session->s_state =3D=3D CEPH_MDS_SESSION_CLOSED || + session->s_state =3D=3D CEPH_MDS_SESSION_REJECTED) { + mutex_unlock(&mdsc->mutex); + pr_info_client(cl, "mds%d skipping reconnect, session %s\n", + mds, + ceph_session_state_name(session->s_state)); + mutex_unlock(&session->s_mutex); + ceph_msg_put(reply); + err =3D -ESTALE; + goto fail_return; + } =20 pr_info_client(cl, "mds%d reconnect start\n", mds); old_state =3D session->s_state; session->s_state =3D CEPH_MDS_SESSION_RECONNECTING; session->s_seq =3D 0; + mutex_unlock(&mdsc->mutex); =20 doutc(cl, "session %p state %s\n", session, ceph_session_state_name(session->s_state)); @@ -5338,7 +5409,10 @@ static int send_mds_reconnect(struct ceph_mds_client= *mdsc, * (check_new_map) can retry. Without this, a transient build * failure strands the session in RECONNECTING indefinitely. */ - session->s_state =3D old_state; + mutex_lock(&mdsc->mutex); + if (session->s_state =3D=3D CEPH_MDS_SESSION_RECONNECTING) + session->s_state =3D old_state; + mutex_unlock(&mdsc->mutex); mutex_unlock(&session->s_mutex); fail_nomsg: ceph_pagelist_release(recon_state.pagelist); @@ -5746,23 +5820,29 @@ static void ceph_mdsc_reset_workfn(struct work_stru= ct *work) continue; } sessions[i]->s_state =3D CEPH_MDS_SESSION_CLOSED; - __unregister_session(mdsc, sessions[i]); - __wake_requests(mdsc, &sessions[i]->s_waiting); mutex_unlock(&mdsc->mutex); =20 mutex_lock(&sessions[i]->s_mutex); cleanup_session_requests(mdsc, sessions[i]); remove_session_caps(sessions[i]); + + /* Keep the rank occupied until all old-session caps are gone. */ + mutex_lock(&mdsc->mutex); + if (!__verify_registered_session(mdsc, sessions[i])) + __unregister_session(mdsc, sessions[i]); + mutex_unlock(&mdsc->mutex); + mutex_unlock(&sessions[i]->s_mutex); =20 wake_up_all(&mdsc->session_close_wq); =20 - ceph_put_mds_session(sessions[i]); - mutex_lock(&mdsc->mutex); + __wake_requests(mdsc, &sessions[i]->s_waiting); kick_requests(mdsc, mds); mutex_unlock(&mdsc->mutex); =20 + ceph_put_mds_session(sessions[i]); + torn_down++; pr_info_client(cl, "mds%d session reset complete\n", mds); } @@ -5864,6 +5944,12 @@ static void check_new_map(struct ceph_mds_client *md= sc, s =3D __ceph_lookup_mds_session(mdsc, i); if (!s) continue; + + if (s->s_state =3D=3D CEPH_MDS_SESSION_CLOSED) { + ceph_put_mds_session(s); + continue; + } + oldstate =3D ceph_mdsmap_get_state(oldmap, i); newstate =3D ceph_mdsmap_get_state(newmap, i); =20 @@ -5876,18 +5962,24 @@ static void check_new_map(struct ceph_mds_client *m= dsc, =20 if (i >=3D newmap->possible_max_rank) { /* force close session for stopped mds */ - __unregister_session(mdsc, s); - __wake_requests(mdsc, &s->s_waiting); + s->s_state =3D CEPH_MDS_SESSION_CLOSED; mutex_unlock(&mdsc->mutex); =20 mutex_lock(&s->s_mutex); cleanup_session_requests(mdsc, s); remove_session_caps(s); + + mutex_lock(&mdsc->mutex); + if (!__verify_registered_session(mdsc, s)) + __unregister_session(mdsc, s); + mutex_unlock(&mdsc->mutex); mutex_unlock(&s->s_mutex); =20 - ceph_put_mds_session(s); + wake_up_all(&mdsc->session_close_wq); =20 mutex_lock(&mdsc->mutex); + __wake_requests(mdsc, &s->s_waiting); + ceph_put_mds_session(s); if (mdsc->mdsmap->m_epoch !=3D map_epoch) return; kick_requests(mdsc, i); @@ -5906,6 +5998,16 @@ static void check_new_map(struct ceph_mds_client *md= sc, ceph_put_mds_session(s); return; } + if (s->s_state =3D=3D CEPH_MDS_SESSION_CLOSED) { + /* + * handle_session() set state=3DCLOSED + * in the mutex gap above and is tearing it + * down + */ + mutex_unlock(&s->s_mutex); + ceph_put_mds_session(s); + continue; + } ceph_con_close(&s->s_con); mutex_unlock(&s->s_mutex); s->s_state =3D CEPH_MDS_SESSION_RESTARTING; @@ -5964,6 +6066,9 @@ static void check_new_map(struct ceph_mds_client *mds= c, * Only open and reconnect sessions that don't exist yet. */ for (i =3D 0; i < newmap->possible_max_rank; i++) { + if (mdsc->stopping) + return; + /* * In case the import MDS is crashed just after * the EImportStart journal is flushed, so when @@ -5989,6 +6094,19 @@ static void check_new_map(struct ceph_mds_client *md= sc, * reconnection request in up:reconnect state. */ s =3D __ceph_lookup_mds_session(mdsc, i); + if (s && s->s_state =3D=3D CEPH_MDS_SESSION_CLOSED) { + /* + * Wait for handle_session() to remove the caps and + * unregister this session, so the reconnect below + * uses a fresh session on the now vacant rank + */ + ceph_put_mds_session(s); + wait_for_mds_rank_not_closing(mdsc, i); + if (mdsc->stopping || + mdsc->mdsmap->m_epoch !=3D map_epoch) + return; + s =3D NULL; + } if (likely(!s)) { s =3D __open_export_target_session(mdsc, i); if (IS_ERR(s)) { @@ -7149,9 +7267,7 @@ static void mds_peer_reset(struct ceph_connection *co= n) * Snapshot session state with READ_ONCE, then revalidate under * mdsc->mutex before acting. The subsequent mdsc->mutex * section rechecks s_state to catch concurrent transitions, so - * the lockless snapshot here is safe. s->s_mutex is taken - * separately for cleanup after unregistration, which avoids - * introducing a new s->s_mutex + mdsc->mutex nesting. + * the lockless snapshot here is safe. */ session_state =3D READ_ONCE(s->s_state); =20 @@ -7176,18 +7292,24 @@ static void mds_peer_reset(struct ceph_connection *= con) =20 ceph_get_mds_session(s); s->s_state =3D CEPH_MDS_SESSION_CLOSED; - __unregister_session(mdsc, s); - __wake_requests(mdsc, &s->s_waiting); mutex_unlock(&mdsc->mutex); =20 mutex_lock(&s->s_mutex); cleanup_session_requests(mdsc, s); remove_session_caps(s); + + /* Keep the rank occupied until all old-session caps are gone. */ + mutex_lock(&mdsc->mutex); + if (!__verify_registered_session(mdsc, s)) + __unregister_session(mdsc, s); + mutex_unlock(&mdsc->mutex); + mutex_unlock(&s->s_mutex); =20 wake_up_all(&mdsc->session_close_wq); =20 mutex_lock(&mdsc->mutex); + __wake_requests(mdsc, &s->s_waiting); kick_requests(mdsc, s->s_mds); mutex_unlock(&mdsc->mutex); =20 --=20 2.47.3