From nobody Sat Sep 26 08:48:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FCF83F929C; Thu, 3 Sep 2026 06:59:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788418745; cv=none; b=tssff7g3fAQOyNlwy1kjwvk+pwq4U7/H5BipMyu7ZK71ZC2KXHTSpbL4TyiFtPoyaniyvudHUwvywamF8U2FT45/yTQnpPcjrthwJGPW1cRqq8LHB1b9/W4VHIZTm96IVi8EuGtXqKLAAFDeHKgWhGM4LOKCJElMsgYY9GjjvLM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788418745; c=relaxed/simple; bh=fFHoj6gAKkFWjGmwc9Pnj0rlvfKC1Wk7kIjSuw78k3k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Nl+vqKJop9SaLGVpyS6NjBykVlMrYFAJP0wfPigYYYIGyBhOdhQu/dt2s/sa3jAgW+BJfzBOn2XusRV24KFg+Q8bml9RUFQn6lmzmv8fK4d05ZBFkbfDVhA7cty7cvWHeTYttzPwDabWVeuIPbMyTHCXlGTjKNRJpI4BdxJ7+yE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=o+OD3rc4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="o+OD3rc4" Received: by smtp.kernel.org (Postfix) with ESMTPS id C480CC2BCB8; Thu, 3 Sep 2026 06:59:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788418744; bh=fFHoj6gAKkFWjGmwc9Pnj0rlvfKC1Wk7kIjSuw78k3k=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=o+OD3rc4YPWYSgEGcHBvSUlQ1q9KpCF6VnzV7ZbMCe633Y2rVqntkw42HQ/Rx/HBZ BqJS9yUMmbhtYKajnPIMsAgHLD0DjGu1pTWsp326SKKoK1qs9JUk/HRC0jX1U8pJSF +GFlG/DXSnr52fYPmUBuOHIzFVcVZg+h1eVWjzlhzfhiNCN+vuel9wd69xYXT6sWYY 0KCps7pkzt7FjXh5dwfzsjzbG3ov58pM+3XOwWI7rpHpB5Ld2Qv5KDcow4bascHz0N PpJZDmJ+8yLrp/Q6/g2qd4KWBkMtLoWMHyujNnIZoIBkOz1XCqERdv2bcadtBErXpx qs3B59mP6cedA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0A26C624D4; Thu, 3 Sep 2026 06:59:04 +0000 (UTC) From: Xiubo Li via B4 Relay Date: Wed, 02 Sep 2026 23:59:01 -0700 Subject: [PATCH v7 1/2] ceph: wake up i_cap_wq waiters on inode shutdown Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260902-lazyio-v7-1-18c2afa06352@clyso.com> References: <20260902-lazyio-v7-0-18c2afa06352@clyso.com> In-Reply-To: <20260902-lazyio-v7-0-18c2afa06352@clyso.com> To: Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko Cc: ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org, Xiubo Li X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788418742; l=1160; i=xiubo.li@clyso.com; s=20260625; h=from:subject:message-id; bh=/w+13aDT7byuE64KSZZv8Qvteagzw9g49d8Bw9ILBEg=; b=Wd2gkxu4c96IAMJZwP9U5PTF/BKNol+7CKZej5rUGleR6M7qAg/xDl9TxxEtTCQXWsrOTdbjc YaEEDI20Q3rDLfcN8aYhMXtcg5NzEEd9m/fnHZWRPvNA/nuWUdqsQaO X-Developer-Key: i=xiubo.li@clyso.com; a=ed25519; pk=V3NGr0AgAopiUhaLY51ipBkLN5LlcLhjOEfLEq1RoZ8= X-Endpoint-Received: by B4 Relay for xiubo.li@clyso.com/20260625 with auth_id=840 X-Original-From: Xiubo Li Reply-To: xiubo.li@clyso.com From: Xiubo Li ceph_inode_shutdown() purges every cap on the inode but never wakes i_cap_wq, unlike remove_session_caps_cb() which does so after the same kind of purge. Anyone blocked in ceph_get_caps() is waiting on that queue for caps that are now gone for good, so the wait only ends when the task is killed. Wake the queue after purging so the waiters get a chance to notice the shutdown and bail out. Link: https://tracker.ceph.com/issues/77594 Signed-off-by: Xiubo Li --- fs/ceph/inode.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c index 209e8972287d..619a429f0d45 100644 --- a/fs/ceph/inode.c +++ b/fs/ceph/inode.c @@ -3323,6 +3323,13 @@ void ceph_inode_shutdown(struct inode *inode) } spin_unlock(&ci->i_ceph_lock); =20 + /* + * The caps are gone now, so wake up anyone waiting for them -- they + * are never coming back and the waiters need to notice the shutdown + * instead. remove_session_caps_cb() does the same after purging. + */ + wake_up_all(&ci->i_cap_wq); + if (invalidate) ceph_queue_invalidate(inode); while (iputs--) --=20 2.53.0 From nobody Sat Sep 26 08:48:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F67E3F58E9; Thu, 3 Sep 2026 06:59:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788418745; cv=none; b=qeNEaIrXtA6AozheoLkpWUHkquqwRvRvm5XAs3QBo0wZCMkJ5wQB0OdhnECBnLS693OheJ0q/gD20UYDR+jK9OyCytAGi0rG6Ld8aR6e+4Yc+vb4hBc7ZcGUXDhOlBIj2lUBKj/QpcpnBJQ4AV+G8zyae1myZeoTJ+h5ZthXiQI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788418745; c=relaxed/simple; bh=EPzc67U7aIwAheGp+40UzjkVlFy1PO2Zypy1bm+2ae8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=IicNdgKQ+697FhBSbKX/C1H1FfosDfDV+/VDgalDfyYP4KxCdxxA+s1b2LKrLrNk6qYWpCmo15a81hfa3vP7sli9Gk09rPHG54W423RYxJWsaLzGEFIHPb6nNSaigRQ8q3P2bm4GDib0HaUpxrQ3ll/2efWFhLqjof2DGg8Zv6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hZIHikb+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hZIHikb+" Received: by smtp.kernel.org (Postfix) with ESMTPS id CE3F6C2BCFC; Thu, 3 Sep 2026 06:59:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788418744; bh=EPzc67U7aIwAheGp+40UzjkVlFy1PO2Zypy1bm+2ae8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=hZIHikb+EgnlY1c+kvpwymMVaDKIh4HT7druJrba+TC+9CSqU9Sc1Oa6Qjam5GBNQ 8NpBxDlZEHY3jPFlfVi9gF4rOsAYnfmEF8vXWrLktOOvlN9cBifrTRW0S8PQ9RYwWo UChrwfGDlxvNldHi9DXo3GKNU97xxKrGztIhzvGv2wqxcmtzFykpW96nBlZ2Qzubdd O2bWRZ9rTbZWqFpEt+rZXAjmuzywmgvTWZdiVwZmEYUqCkRutd1TAPxqvUoYQeDwLx K4FdtnSLFDJshf4wqPBl8YFtVfW0qZQn2JzF7bA++xYg6Qb7b68vCxEuvFUbU+sEJG ObxxbnDtaIdKw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1340C624D7; Thu, 3 Sep 2026 06:59:04 +0000 (UTC) From: Xiubo Li via B4 Relay Date: Wed, 02 Sep 2026 23:59:02 -0700 Subject: [PATCH v7 2/2] ceph: add 'lazyio' mount option to kclient Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260902-lazyio-v7-2-18c2afa06352@clyso.com> References: <20260902-lazyio-v7-0-18c2afa06352@clyso.com> In-Reply-To: <20260902-lazyio-v7-0-18c2afa06352@clyso.com> To: Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko Cc: ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org, Xiubo Li X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788418742; l=28002; i=xiubo.li@clyso.com; s=20260625; h=from:subject:message-id; bh=L/4vOWF5T8yL6iOaiyMOxkagdrBdS3DMo2PDy2dUTmg=; b=1mPQvXdHI9134UAhlx/vomcYBnRgXRdditQi1iAhZmBNPNB9tucTkbtW9MLpWIX/cIWK1LWZd 7zSB9UqGSICD/I0oGV4abFFL+CFd0KrbtUDVKcT5t30XRydPCWyjXYK X-Developer-Key: i=xiubo.li@clyso.com; a=ed25519; pk=V3NGr0AgAopiUhaLY51ipBkLN5LlcLhjOEfLEq1RoZ8= X-Endpoint-Received: by B4 Relay for xiubo.li@clyso.com/20260625 with auth_id=840 X-Original-From: Xiubo Li Reply-To: xiubo.li@clyso.com From: Xiubo Li Add a 'lazyio' mount option to the kernel Ceph client that enables LazyIO globally for all regular file opens on a mount. This is the kclient equivalent of the 'client_force_lazyio=3Dtrue' config option in the ceph-fuse userspace client. When 'lazyio' is specified, CEPH_FILE_MODE_LAZY is automatically added to every regular file's fmode at open time in ceph_open() and ceph_atomic_open(), causing the I/O paths to request CEPH_CAP_FILE_LAZYIO from the MDS. This permits buffered I/O via the page cache even when multiple clients have the file open for write =E2=80=94 beneficial for HPC workloads that can tolerate relaxed cache coherency. The mount option is exposed as 'lazyio' / 'nolazyio' via the VFS fsparam_flag_no mechanism and supports remount. Link: https://tracker.ceph.com/issues/77594 Signed-off-by: Xiubo Li --- fs/ceph/addr.c | 138 +++++++++++++++++++++++++++ fs/ceph/caps.c | 218 +++++++++++++++++++++++++++++++++++++++= +--- fs/ceph/file.c | 33 ++++++- fs/ceph/super.c | 15 +++ fs/ceph/super.h | 17 ++++ fs/ceph/util.c | 4 - include/linux/ceph/ceph_fs.h | 1 + 7 files changed, 406 insertions(+), 20 deletions(-) diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index f8844390f88d..b4bfef11ac6f 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -500,6 +500,10 @@ static int ceph_init_request(struct netfs_io_request *= rreq, struct file *file) rreq->netfs_priv =3D priv; return 0; } + + /* If this is a lazy fd, also try to get LAZYIO caps */ + if (fi->fmode & CEPH_FILE_MODE_LAZY) + want |=3D CEPH_CAP_FILE_LAZYIO; } =20 /* @@ -1990,6 +1994,42 @@ static void ceph_restore_sigs(sigset_t *oldset) sigprocmask(SIG_SETMASK, oldset, NULL); } =20 +/* + * True if the page cache is only being kept alive by LAZYIO: CACHE is no + * longer present in the implemented cap set, while LAZYIO still is, so + * handle_cap_grant() left the pages in place for the lazy openers. + * + * Such pages are only valid under the lazy semantics, and a non-lazy read= er + * must not be served from them. try_get_cap_refs() already refuses to + * satisfy a non-lazy CACHE want with LAZYIO for read_iter()/write_iter(); + * faults need the same gate. + * + * Unlike __prep_cap(), which has to reason about a single cap, the questi= on + * here is whether the cached data can have gone stale, so the aggregate of + * all caps is the right granularity: as long as any one of them still has + * CACHE implemented, its revocation has not been ACKed and the MDS cannot + * have handed CACHE to another client. Note that __ceph_caps_issued() + * reports i_snap_caps in "have" only and not in "implemented", hence the + * OR; snap caps never include LAZYIO, so folding them in can only make th= is + * more conservative. + */ +static bool ceph_pages_retained_for_lazyio(struct inode *inode) +{ + struct ceph_inode_info *ci =3D ceph_inode(inode); + int have, implemented; + + if (!inode->i_data.nrpages) + return false; + + spin_lock(&ci->i_ceph_lock); + have =3D __ceph_caps_issued(ci, &implemented); + implemented |=3D have; + spin_unlock(&ci->i_ceph_lock); + + return !(implemented & CEPH_CAP_FILE_CACHE) && + (implemented & CEPH_CAP_FILE_LAZYIO); +} + /* * vm ops */ @@ -2025,6 +2065,43 @@ static vm_fault_t ceph_filemap_fault(struct vm_fault= *vmf) doutc(cl, "%llx.%llx %llu got cap refs on %s\n", ceph_vinop(inode), off, ceph_cap_string(got)); =20 + /* + * Don't fault a non-lazy mapping in from page cache that only LAZYIO + * covers. The "every opener is lazy" gate in handle_cap_grant() can't + * catch this on its own: an open that happens after the cache was + * retained sends no cap message, so nothing re-evaluates the decision. + * Drop the range here instead and let the fault below re-read it. + */ + if (!(fi->fmode & CEPH_FILE_MODE_LAZY) && + !(got & CEPH_CAP_FILE_CACHE) && + ceph_pages_retained_for_lazyio(inode)) { + struct address_space *mapping =3D inode->i_mapping; + + doutc(cl, "%llx.%llx %llu dropping LAZYIO page cache\n", + ceph_vinop(inode), off); + err =3D invalidate_inode_pages2_range(mapping, vmf->pgoff, + vmf->pgoff); + if (err =3D=3D -EBUSY) { + /* + * The folio is dirty and ceph has no launder_folio(), + * so invalidate_inode_pages2_range() cannot drop it. + * Write it back and retry -- handing a folio that only + * LAZYIO covers to a non-lazy vma is not an option. + */ + err =3D filemap_write_and_wait_range(mapping, off, + off + PAGE_SIZE - 1); + if (!err) + err =3D invalidate_inode_pages2_range(mapping, + vmf->pgoff, vmf->pgoff); + } + if (err) { + doutc(cl, "%llx.%llx %llu failed to drop LAZYIO page cache: %d\n", + ceph_vinop(inode), off, err); + ceph_put_cap_refs(ci, got); + goto out_restore; + } + } + if ((got & (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_LAZYIO)) || !ceph_has_inline_data(ci)) { CEPH_DEFINE_RW_CONTEXT(rw_ctx, got); @@ -2084,6 +2161,30 @@ static vm_fault_t ceph_filemap_fault(struct vm_fault= *vmf) return ret; } =20 +/* + * Return true if the MDS has issued us a cap that covers buffered + * dirtying: either BUFFER, or LAZYIO (as long as LAZYIO itself is not + * being revoked). This mirrors the conditions under which + * try_get_cap_refs() will hand out a BUFFER or LAZYIO ref. + */ +static bool ceph_have_dirtyable_caps(struct inode *inode) +{ + struct ceph_inode_info *ci =3D ceph_inode(inode); + int have, implemented; + bool ret =3D false; + + spin_lock(&ci->i_ceph_lock); + have =3D __ceph_caps_issued(ci, &implemented); + if (have & CEPH_CAP_FILE_BUFFER) { + ret =3D true; + } else if ((have & CEPH_CAP_FILE_LAZYIO) && + !((implemented & ~have) & CEPH_CAP_FILE_LAZYIO)) { + ret =3D true; + } + spin_unlock(&ci->i_ceph_lock); + return ret; +} + static vm_fault_t ceph_page_mkwrite(struct vm_fault *vmf) { struct vm_area_struct *vma =3D vmf->vma; @@ -2122,6 +2223,7 @@ static vm_fault_t ceph_page_mkwrite(struct vm_fault *= vmf) else want =3D CEPH_CAP_FILE_BUFFER; =20 +retry_caps: got =3D 0; err =3D ceph_get_caps(vma->vm_file, CEPH_CAP_FILE_WR, want, off + len, &g= ot); if (err < 0) @@ -2130,6 +2232,42 @@ static vm_fault_t ceph_page_mkwrite(struct vm_fault = *vmf) doutc(cl, "%llx.%llx %llu~%zd got cap refs on %s\n", ceph_vinop(inode), off, len, ceph_cap_string(got)); =20 + /* + * ceph_write_iter() makes the same check and falls back to + * synchronous writes, but a page fault has no such fallback: + * dirtying the folio without BUFFER or LAZYIO would leave dirty + * data uncovered by any issued cap (e.g. while LAZYIO is being + * revoked, or after it has been released). Wait for the MDS to + * (re)grant a covering cap, matching how the exclude gate in + * try_get_cap_refs() blocks buffered writes while BUFFER is + * revoking. + */ + if ((fi->fmode & CEPH_FILE_MODE_LAZY) && + (got & (CEPH_CAP_FILE_BUFFER | CEPH_CAP_FILE_LAZYIO)) =3D=3D 0) { + ceph_put_cap_refs(ci, got); + got =3D 0; + doutc(cl, "%llx.%llx %llu~%zd waiting for BUFFER or LAZYIO\n", + ceph_vinop(inode), off, len); + /* + * Bail out if the inode is being shut down: the caps are gone + * for good, so ceph_have_dirtyable_caps() would never become + * true again. Note that we cannot rely on ceph_get_caps() + * failing with -ESTALE on the retry either -- if WR is still + * held it would succeed without BUFFER or LAZYIO and send us + * straight back here. + */ + err =3D wait_event_killable(ci->i_cap_wq, + ceph_have_dirtyable_caps(inode) || + ceph_inode_is_shutdown(inode)); + if (err) + goto out_free; + if (ceph_inode_is_shutdown(inode)) { + err =3D -ESTALE; + goto out_free; + } + goto retry_caps; + } + /* Update time before taking folio lock */ file_update_time(vma->vm_file); inode_inc_iversion_raw(inode); diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index 1847badddd87..131a23edbf95 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -1028,6 +1028,53 @@ int __ceph_caps_used(struct ceph_inode_info *ci) return used; } =20 +/* + * Substitute LAZYIO for CACHE/BUFFER when they are not issued. + * If we have LAZYIO but not CACHE/BUFFER, report LAZYIO as used instead + * so the MDS knows we're fine with the weaker consistency guarantee. + * + * Base the substitution on "implemented" rather than "issued": while + * LAZYIO is being revoked, "issued" no longer contains it but + * "implemented" still does. If used reverted to CACHE/BUFFER at that + * point, ceph_check_caps() would see (revoking & cap_used) =3D=3D 0 and + * ACK the revoke while dirty or stale pages were still present. Only + * once the revoke is ACKed does "implemented" drop LAZYIO. + * + * Caller must hold i_ceph_lock. + */ +static inline int ceph_adjust_caps_used_for_lazyio(struct ceph_inode_info = *ci, + int used, int issued, + int implemented) +{ + if (!(used & (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_BUFFER))) + return used; + if (!(implemented & CEPH_CAP_FILE_LAZYIO)) + return used; + /* + * While LAZYIO is still issued, it only covers the cached data if + * every opener has accepted the lazy semantics. With a non-lazy + * opener around, keep reporting CACHE/BUFFER so that a pending + * revocation is not ACKed before the invalidation queued by + * handle_cap_grant() has emptied the page cache. + * + * Once LAZYIO is itself being revoked ("implemented" has it but + * "issued" no longer does) the substitution is unconditional: the + * revoke must not be ACKed while pages are still around, no matter + * who has the file open. + */ + if ((issued & CEPH_CAP_FILE_LAZYIO) && !__ceph_all_opens_lazy(ci)) + return used; + if (!(issued & CEPH_CAP_FILE_CACHE)) { + used &=3D ~CEPH_CAP_FILE_CACHE; + used |=3D CEPH_CAP_FILE_LAZYIO; + } + if (!(issued & CEPH_CAP_FILE_BUFFER)) { + used &=3D ~CEPH_CAP_FILE_BUFFER; + used |=3D CEPH_CAP_FILE_LAZYIO; + } + return used; +} + #define FMODE_WAIT_BIAS 1000 =20 /* @@ -1449,6 +1496,20 @@ static void __prep_cap(struct cap_msg_args *arg, str= uct ceph_inode_info *ci, * dirty pages _before_ allowing sync writes to avoid reordering. */ arg->wake =3D cap->implemented & ~cap->issued; + /* + * "used" decides which of the revoked caps stay in cap->implemented, + * i.e. which revocations this message does _not_ ack yet. Callers on + * the flush paths (try_flush_caps(), __kick_flushing_caps()) hand us + * the raw __ceph_caps_used(), which reports CACHE/BUFFER as long as + * the page cache is populated. While LAZYIO is being revoked it is no + * longer in cap->issued, so a raw used set would drop it from + * cap->implemented and implicitly ack the revoke with pages still + * cached -- the MDS could then hand CACHE to another client. Report + * LAZYIO as used instead, and only stop doing so once writeback and + * invalidation have emptied the page cache. + */ + used =3D ceph_adjust_caps_used_for_lazyio(ci, used, cap->issued, + cap->implemented); cap->implemented &=3D cap->issued | used; cap->mds_wanted =3D want; =20 @@ -2094,6 +2155,7 @@ void ceph_check_caps(struct ceph_inode_info *ci, int = flags) * usually because they have outstanding references). */ issued =3D __ceph_caps_issued(ci, &implemented); + revoking =3D implemented & ~issued; =20 want =3D file_wanted; @@ -2185,6 +2247,19 @@ void ceph_check_caps(struct ceph_inode_info *ci, int= flags) if (ci->i_auth_cap && cap !=3D ci->i_auth_cap) cap_used &=3D ~ci->i_auth_cap->issued; =20 + /* + * Substitute LAZYIO for CACHE/BUFFER when they are not issued. + * Do this per cap and not once for the whole inode: only the + * cap that actually holds LAZYIO may report it as used. A cap + * that is revoking CACHE without holding LAZYIO must keep + * reporting CACHE as used, or the (revoking & cap_used) test + * below would treat the revocation as completed while the page + * cache is still populated. + */ + cap_used =3D ceph_adjust_caps_used_for_lazyio(ci, cap_used, + cap->issued, + cap->implemented); + revoking =3D cap->implemented & ~cap->issued; doutc(cl, " mds%d cap %p used %s issued %s implemented %s revoking %s\n", cap->mds, cap, ceph_cap_string(cap_used), @@ -2209,10 +2284,13 @@ void ceph_check_caps(struct ceph_inode_info *ci, in= t flags) * at most 5 seconds. That means the MDS needs to wait at * most 5 seconds to finished the Fb capability's revocation. * - * Let's queue a writeback for it. + * Let's queue a writeback for it. The same applies when + * LAZYIO is revoked while it was covering for BUFFER + * (dirty pages exist, but BUFFER isn't issued). */ if (S_ISREG(inode->i_mode) && ci->i_wrbuffer_ref && - (revoking & CEPH_CAP_FILE_BUFFER)) + (revoking & (CEPH_CAP_FILE_BUFFER | + CEPH_CAP_FILE_LAZYIO))) queue_writeback =3D true; } =20 @@ -2957,9 +3035,45 @@ static int try_get_cap_refs(struct inode *inode, int= need, int want, } snap_rwsem_locked =3D true; } - if ((have & want) =3D=3D want) + /* + * Allow LAZYIO to act as a substitute for + * CACHE or BUFFER when those caps are not + * issued, but only for callers that + * explicitly requested LAZYIO. This + * prevents a non-lazy fd from having its + * CACHE/BUFFER wants satisfied by LAZYIO + * on an inode where a different fd is lazy. + * + * A missing LAZYIO cap, however, must never + * cost us the CACHE/BUFFER refs that are + * actually issued: the MDS only grants + * LAZYIO for files opened with CEPH_O_LAZY + * and it can be revoked at any time. If we + * made the whole want unsatisfiable without + * it, the I/O paths would silently drop + * CACHE/BUFFER (e.g. take no wrbuffer refs) + * and degrade to synchronous writes. + */ + if ((have & (want & ~CEPH_CAP_FILE_LAZYIO)) =3D=3D + (want & ~CEPH_CAP_FILE_LAZYIO)) { + *got =3D need | (want & ~exclude & + ~CEPH_CAP_FILE_LAZYIO); + if ((want & CEPH_CAP_FILE_LAZYIO) && + (have & CEPH_CAP_FILE_LAZYIO) && + !(exclude & CEPH_CAP_FILE_LAZYIO)) + *got |=3D CEPH_CAP_FILE_LAZYIO; + } else if ((want & CEPH_CAP_FILE_LAZYIO) && + (have & CEPH_CAP_FILE_LAZYIO) && + !(exclude & CEPH_CAP_FILE_LAZYIO) && + ((have & want) =3D=3D + (want & ~(CEPH_CAP_FILE_CACHE | + CEPH_CAP_FILE_BUFFER)))) { + /* + * LAZYIO substitutes for missing CACHE/BUFFER; + * it is already included via (want & ~exclude). + */ *got =3D need | (want & ~exclude); - else + } else *got =3D need; ceph_take_cap_refs(ci, *got, true); ret =3D 1; @@ -3048,6 +3162,14 @@ static void check_max_size(struct inode *inode, loff= _t endoff) ceph_check_caps(ci, CHECK_CAPS_AUTHONLY); } =20 +/* + * The inverse of ceph_caps_for_mode(). LAZYIO has to be mapped back as + * well: __ceph_get_caps() feeds the result to ceph_get_fmode() to bias + * i_nr_by_mode[] by FMODE_WAIT_BIAS while it waits, and dropping LAZY + * here would bias the open count without biasing the lazy count, making + * __ceph_all_opens_lazy() report a phantom non-lazy opener for as long + * as a lazy fd is waiting for caps. + */ static inline int get_used_fmode(int caps) { int fmode =3D 0; @@ -3055,6 +3177,8 @@ static inline int get_used_fmode(int caps) fmode |=3D CEPH_FILE_MODE_RD; if (caps & CEPH_CAP_FILE_WR) fmode |=3D CEPH_FILE_MODE_WR; + if (caps & CEPH_CAP_FILE_LAZYIO) + fmode |=3D CEPH_FILE_MODE_LAZY; return fmode; } =20 @@ -3616,13 +3740,29 @@ static void handle_cap_grant(struct inode *inode, =20 =20 /* - * If CACHE is being revoked, and we have no dirty buffers, - * try to invalidate (once). (If there are dirty buffers, we - * will invalidate _after_ writeback.) + * Check the revocation of *both* CACHE and LAZYIO, because + * CACHE may have been revoked earlier and cap->issued no + * longer contains it -- at that point only LAZYIO was + * covering us. If LAZYIO is now also being revoked and no + * cache cap remains, we must invalidate the page cache. + * Without this, a CACHE-revoked-then-LAZYIO-revoked sequence + * leaves stale pages in memory until the next periodic + * check_caps (up to 60s). Also invalidate when we have no + * dirty buffers (if dirty, invalidate after writeback). + * + * Keeping the page cache alive on a remaining LAZYIO is only + * safe if every opener has accepted the lazy semantics: those + * pages are retained for the lazy fds alone, and a non-lazy + * reader (mmap, or an fd that never asked for LAZYIO) must not + * be served from them. So drop the cache as well whenever a + * non-lazy opener is around. */ if (S_ISREG(inode->i_mode) && /* don't invalidate readdir cache */ - ((cap->issued & ~newcaps) & CEPH_CAP_FILE_CACHE) && - (newcaps & CEPH_CAP_FILE_LAZYIO) =3D=3D 0 && + ((cap->issued & ~newcaps) & + (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_LAZYIO)) && + !(newcaps & CEPH_CAP_FILE_CACHE) && + (!(newcaps & CEPH_CAP_FILE_LAZYIO) || + !__ceph_all_opens_lazy(ci)) && !(ci->i_wrbuffer_ref || ci->i_wb_ref)) { if (try_nonblocking_invalidate(inode)) { /* there were locked pages.. invalidate later @@ -3634,6 +3774,47 @@ static void handle_cap_grant(struct inode *inode, } } =20 + /* + * The counterpart on the grant arm: pages that survived a CACHE + * revocation are only kept alive by LAZYIO, and they may have + * gone stale while CACHE was away. Once a later grant brings + * CACHE back, ceph_pages_retained_for_lazyio() (addr.c) turns + * false the moment CACHE shows up in the implemented set, and a + * non-lazy opener would be served those folios as coherent + * CACHE data. Drop the cache here, before a non-lazy + * read_iter/fault can use it. + * + * The aggregate check below still sees the pre-grant state: + * cap->issued and cap->implemented are only updated further + * down, so CACHE is absent from "implemented" here, exactly + * as it was when the pages were retained. + * + * Dirty folios (this client's own lazy writes) are not + * droppable -- ceph has no launder_folio() -- and are left in + * place by the invalidate, which is fine: they hold current + * local data. + */ + if (S_ISREG(inode->i_mode) && + ((newcaps & ~cap->issued) & CEPH_CAP_FILE_CACHE) && + inode->i_data.nrpages) { + int have, implemented; + + have =3D __ceph_caps_issued(ci, &implemented); + implemented |=3D have; + if (!(implemented & CEPH_CAP_FILE_CACHE) && + (implemented & CEPH_CAP_FILE_LAZYIO) && + try_nonblocking_invalidate(inode)) { + /* + * There were locked pages.. invalidate later in + * a separate thread. + */ + if (ci->i_rdcache_revoking !=3D ci->i_rdcache_gen) { + queue_invalidate =3D true; + ci->i_rdcache_revoking =3D ci->i_rdcache_gen; + } + } + } + if (was_stale) cap->issued =3D cap->implemented =3D CEPH_CAP_PIN; =20 @@ -3766,6 +3947,8 @@ static void handle_cap_grant(struct inode *inode, /* check cap bits */ wanted =3D __ceph_caps_wanted(ci); used =3D __ceph_caps_used(ci); + used =3D ceph_adjust_caps_used_for_lazyio(ci, used, cap->issued, + cap->implemented); dirty =3D __ceph_caps_dirty(ci); doutc(cl, " my wanted =3D %s, used =3D %s, dirty %s\n", ceph_cap_string(wanted), ceph_cap_string(used), @@ -3793,13 +3976,26 @@ static void handle_cap_grant(struct inode *inode, doutc(cl, "revocation: %s -> %s (revoking %s)\n", ceph_cap_string(cap->issued), ceph_cap_string(newcaps), ceph_cap_string(revoking)); + /* + * If BUFFER is being revoked and we have dirty data, + * trigger writeback before acking. When LAZYIO was + * covering for BUFFER (BUFFER not issued, dirty refs + * held), also trigger writeback. Clean cached pages + * under LAZYIO are handled by queue_invalidate below. + */ if (S_ISREG(inode->i_mode) && (revoking & used & CEPH_CAP_FILE_BUFFER)) { writeback =3D true; /* initiate writeback; will delay ack */ revoke_wait =3D true; + } else if (S_ISREG(inode->i_mode) && + (revoking & used & CEPH_CAP_FILE_LAZYIO) && + (ci->i_wrbuffer_ref || ci->i_wb_ref)) { + /* LAZYIO was covering for dirty data =E2=80=94 flush first */ + writeback =3D true; + revoke_wait =3D true; } else if (queue_invalidate && - revoking =3D=3D CEPH_CAP_FILE_CACHE && - (newcaps & CEPH_CAP_FILE_LAZYIO) =3D=3D 0) { + (revoking & (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_LAZYIO)) && + !(newcaps & (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_LAZYIO))) { revoke_wait =3D true; /* do nothing yet, invalidation will be queued */ } else if (cap =3D=3D ci->i_auth_cap) { check_caps =3D 1; /* check auth cap only */ diff --git a/fs/ceph/file.c b/fs/ceph/file.c index d5606456dbd5..132b5878f924 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -347,10 +347,6 @@ int ceph_renew_caps(struct inode *inode, int fmode) flags =3D O_RDONLY; else if (wanted & CEPH_CAP_FILE_WR) flags =3D O_WRONLY; -#ifdef O_LAZY - if (wanted & CEPH_CAP_FILE_LAZYIO) - flags |=3D O_LAZY; -#endif =20 req =3D prepare_open_request(inode->i_sb, flags, 0); if (IS_ERR(req)) { @@ -358,6 +354,10 @@ int ceph_renew_caps(struct inode *inode, int fmode) goto out; } =20 + if (wanted & CEPH_CAP_FILE_LAZYIO) { + req->r_fmode |=3D CEPH_FILE_MODE_LAZY; + req->r_args.open.flags |=3D cpu_to_le32(CEPH_O_LAZY); + } req->r_inode =3D inode; ihold(inode); req->r_num_caps =3D 1; @@ -409,6 +409,19 @@ int ceph_open(struct inode *inode, struct file *file) doutc(cl, "%p %llx.%llx file %p flags %d (%d)\n", inode, ceph_vinop(inode), file, flags, file->f_flags); fmode =3D ceph_flags_to_mode(flags); + + /* + * If lazyio mount option is set, enable lazyio for all regular + * files. Skip snapped files: snap caps never include LAZYIO, + * so including it in wanted would force an unnecessary MDS + * round-trip for every open of a snapped file. + */ + if (S_ISREG(inode->i_mode) && + ceph_snap(inode) =3D=3D CEPH_NOSNAP && + (fsc->mount_options->flags & CEPH_MOUNT_OPT_LAZYIO)) { + fmode |=3D CEPH_FILE_MODE_LAZY; + } + wanted =3D ceph_caps_for_mode(fmode); =20 if (fmode & CEPH_FILE_MODE_WR) @@ -485,13 +498,16 @@ int ceph_open(struct inode *inode, struct file *file) err =3D PTR_ERR(req); goto out; } + req->r_fmode |=3D fmode & CEPH_FILE_MODE_LAZY; + if (fmode & CEPH_FILE_MODE_LAZY) + req->r_args.open.flags |=3D cpu_to_le32(CEPH_O_LAZY); req->r_inode =3D inode; ihold(inode); =20 req->r_num_caps =3D 1; err =3D ceph_mdsc_do_request(mdsc, NULL, req); if (!err) - err =3D ceph_init_file(inode, file, req->r_fmode); + err =3D ceph_init_file(inode, file, fmode); ceph_mdsc_put_request(req); doutc(cl, "open result=3D%d on %llx.%llx\n", err, ceph_vinop(inode)); out: @@ -836,6 +852,9 @@ int ceph_atomic_open(struct inode *dir, struct dentry *= dentry, } else { int fmode =3D ceph_flags_to_mode(flags); =20 + if (fsc->mount_options->flags & CEPH_MOUNT_OPT_LAZYIO) + fmode |=3D CEPH_FILE_MODE_LAZY; + mask =3D MAY_READ; if (fmode & CEPH_FILE_MODE_WR) mask |=3D MAY_WRITE; @@ -878,6 +897,10 @@ int ceph_atomic_open(struct inode *dir, struct dentry = *dentry, err =3D PTR_ERR(req); goto out_ctx; } + if (fsc->mount_options->flags & CEPH_MOUNT_OPT_LAZYIO) { + req->r_fmode |=3D CEPH_FILE_MODE_LAZY; + req->r_args.open.flags |=3D cpu_to_le32(CEPH_O_LAZY); + } req->r_dentry =3D dget(dentry); req->r_num_caps =3D 2; mask =3D CEPH_STAT_CAP_INODE | CEPH_CAP_AUTH_SHARED; diff --git a/fs/ceph/super.c b/fs/ceph/super.c index c631844828d3..108095ef68b4 100644 --- a/fs/ceph/super.c +++ b/fs/ceph/super.c @@ -181,6 +181,7 @@ enum { Opt_pagecache, Opt_sparseread, Opt_nearfull_sync, + Opt_lazyio, }; =20 enum ceph_recover_session_mode { @@ -207,6 +208,7 @@ static const struct fs_parameter_spec ceph_mount_parame= ters[] =3D { fsparam_flag_no ("fsc", Opt_fscache), // fsc|nofsc fsparam_string ("fsc", Opt_fscache), // fsc=3D... fsparam_flag_no ("ino32", Opt_ino32), + fsparam_flag_no ("lazyio", Opt_lazyio), fsparam_string ("mds_namespace", Opt_mds_namespace), fsparam_string ("mon_addr", Opt_mon_addr), fsparam_flag_no ("nearfull_sync", Opt_nearfull_sync), @@ -604,6 +606,12 @@ static int ceph_parse_mount_param(struct fs_context *f= c, else fsopt->flags |=3D CEPH_MOUNT_OPT_NEARFULL_SYNC; break; + case Opt_lazyio: + if (result.negated) + fsopt->flags &=3D ~CEPH_MOUNT_OPT_LAZYIO; + else + fsopt->flags |=3D CEPH_MOUNT_OPT_LAZYIO; + break; case Opt_test_dummy_encryption: #ifdef CONFIG_FS_ENCRYPTION fscrypt_free_dummy_policy(&fsopt->dummy_enc_policy); @@ -762,6 +770,8 @@ static int ceph_show_options(struct seq_file *m, struct= dentry *root) seq_puts(m, ",sparseread"); if (fsopt->flags & CEPH_MOUNT_OPT_NEARFULL_SYNC) seq_puts(m, ",nearfull_sync"); + if (fsopt->flags & CEPH_MOUNT_OPT_LAZYIO) + seq_puts(m, ",lazyio"); =20 fscrypt_show_test_dummy_encryption(m, ',', root->d_sb); =20 @@ -1437,6 +1447,11 @@ static int ceph_reconfigure_fc(struct fs_context *fc) else ceph_clear_mount_opt(fsc, NEARFULL_SYNC); =20 + if (fsopt->flags & CEPH_MOUNT_OPT_LAZYIO) + ceph_set_mount_opt(fsc, LAZYIO); + else + ceph_clear_mount_opt(fsc, LAZYIO); + if (strcmp_null(fsc->mount_options->mon_addr, fsopt->mon_addr)) { kfree(fsc->mount_options->mon_addr); fsc->mount_options->mon_addr =3D fsopt->mon_addr; diff --git a/fs/ceph/super.h b/fs/ceph/super.h index f3d99027e193..2cf81934c891 100644 --- a/fs/ceph/super.h +++ b/fs/ceph/super.h @@ -46,6 +46,7 @@ #define CEPH_MOUNT_OPT_NOPAGECACHE (1<<16) /* bypass pagecache altoget= her */ #define CEPH_MOUNT_OPT_SPARSEREAD (1<<17) /* always do sparse reads */ #define CEPH_MOUNT_OPT_NEARFULL_SYNC (1<<18) /* sync writes when nearful= l */ +#define CEPH_MOUNT_OPT_LAZYIO (1<<19) /* force lazyio for all fil= e opens */ =20 #define CEPH_MOUNT_OPT_DEFAULT \ (CEPH_MOUNT_OPT_DCACHE | \ @@ -927,6 +928,22 @@ static inline bool __ceph_is_file_opened(struct ceph_i= node_info *ci) { return ci->i_nr_by_mode[0]; } + +/* + * True if the inode is open and every opener is a lazy one. Bit 0 of + * i_nr_by_mode[] is set for every open regardless of mode (see + * ceph_get_fmode()), so it doubles as the total open count. + * + * Callers must hold i_ceph_lock. + */ +static inline bool __ceph_all_opens_lazy(struct ceph_inode_info *ci) +{ + int nr_open =3D ci->i_nr_by_mode[ffs(CEPH_FILE_MODE_PIN)]; + int nr_lazy =3D ci->i_nr_by_mode[ffs(CEPH_FILE_MODE_LAZY)]; + + return nr_open > 0 && nr_lazy >=3D nr_open; +} + extern int __ceph_caps_file_wanted(struct ceph_inode_info *ci); extern int __ceph_caps_wanted(struct ceph_inode_info *ci); =20 diff --git a/fs/ceph/util.c b/fs/ceph/util.c index 2c34875675bf..be3db3f20344 100644 --- a/fs/ceph/util.c +++ b/fs/ceph/util.c @@ -73,10 +73,6 @@ int ceph_flags_to_mode(int flags) mode =3D CEPH_FILE_MODE_RDWR; break; } -#ifdef O_LAZY - if (flags & O_LAZY) - mode |=3D CEPH_FILE_MODE_LAZY; -#endif =20 return mode; } diff --git a/include/linux/ceph/ceph_fs.h b/include/linux/ceph/ceph_fs.h index 0adc526ec321..d70bbce1a1b2 100644 --- a/include/linux/ceph/ceph_fs.h +++ b/include/linux/ceph/ceph_fs.h @@ -414,6 +414,7 @@ extern const char *ceph_mds_op_name(int op); #define CEPH_O_CREAT 00000100 #define CEPH_O_EXCL 00000200 #define CEPH_O_TRUNC 00001000 +#define CEPH_O_LAZY 00020000 #define CEPH_O_DIRECTORY 00200000 #define CEPH_O_NOFOLLOW 00400000 =20 --=20 2.53.0