From nobody Sun Sep 27 02:19:24 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 352D541F5F3; Thu, 27 Aug 2026 10:16:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787825800; cv=none; b=PuXfEeYHwtOa7D5+nGn/FTEfO3dGRnm0WoPOZlBymTcSQSV7zlBQ/D2iGsIXePgwCzNkU9KY0tfMKv4pVZmgZDb8oYbupNfUHBIV8VY0pV3Whlb/G38auMVFXQCMkj8sebFQK4jXvotOx4bGTtF8znLows4ui9vF/nNA8+jk1fs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787825800; c=relaxed/simple; bh=YNjJLTxfNMZ4O4V4pvNh7HO8NADO9hmH1voDuRHPkvw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fB+PsstLDvVTL5ZqYq1uwyqzGOMxusu577bIGztvE/ONlaiFyBThHJv2n6fJc7RLmjtP3FGG2rh6YvCdaPTrLaIqsvaWb6sOM5K9UedfsPghRYPnemKb2s8eCUklThzfihjQ51rlFI3rwSmEBwc7iFyLycqCSsf9yua8qOa+oWc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Azivbd+2; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Azivbd+2" Received: by smtp.kernel.org (Postfix) with ESMTPS id 21E58C2BCF6; Thu, 27 Aug 2026 10:16:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1787825799; bh=YNjJLTxfNMZ4O4V4pvNh7HO8NADO9hmH1voDuRHPkvw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Azivbd+2TqBMBm42A4H4RjqmTNBjF9e7f596hCOGd8d3+wpFrKCuL7qkDasGt+bQQ EhOV4wjiOP+qbWpiS1lwJBWXEhMOY0gYJuCONf/WfxaXuV0rlDvZrENZGXOP5XyVrV +N6/dGtXDxoptsknkGye6acBssayrgKoeigMSxzMo92pQ2X48WisGaOvU6uJEj3wtI HCfmGLwp56rGvmnpgaL7UTAY36mJIKd1exVLagS2c/3aEOmg6RRluDO3eJJjvccp+s 5446+S+TM41eRBkiwxZmRf6ekCZ++RQnWwyCngDWz1t3QkufZtr4gxQXtXjtqBqCpA SR9Q0/3A7Rg8Q== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 061B5C61DB9; Thu, 27 Aug 2026 10:16:39 +0000 (UTC) From: Xiubo Li via B4 Relay Date: Thu, 27 Aug 2026 03:16:03 -0700 Subject: [PATCH v6 1/2] ceph: wake up i_cap_wq waiters on inode shutdown Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260827-lazyio-v6-1-2755b379b8de@clyso.com> References: <20260827-lazyio-v6-0-2755b379b8de@clyso.com> In-Reply-To: <20260827-lazyio-v6-0-2755b379b8de@clyso.com> To: Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko Cc: ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org, Xiubo Li X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1787825763; l=1160; i=xiubo.li@clyso.com; s=20260625; h=from:subject:message-id; bh=wOzreLw8o1CmL+OtUmksL+iiCd6T7ivHrgbY3g5NRtM=; b=RrrSt9Q/ciUxfmOZJ9+OMrK7qDcWQ2dMILOZtiH3YIz3VA+PbgbRlnmTvf+f3Ad2oLsAtQyLf ESKO/7quyY8DHbVz8eGmX0l5FZ/xJEJipBHPjqW6gsFaR6JSyeDUX3g X-Developer-Key: i=xiubo.li@clyso.com; a=ed25519; pk=V3NGr0AgAopiUhaLY51ipBkLN5LlcLhjOEfLEq1RoZ8= X-Endpoint-Received: by B4 Relay for xiubo.li@clyso.com/20260625 with auth_id=840 X-Original-From: Xiubo Li Reply-To: xiubo.li@clyso.com From: Xiubo Li ceph_inode_shutdown() purges every cap on the inode but never wakes i_cap_wq, unlike remove_session_caps_cb() which does so after the same kind of purge. Anyone blocked in ceph_get_caps() is waiting on that queue for caps that are now gone for good, so the wait only ends when the task is killed. Wake the queue after purging so the waiters get a chance to notice the shutdown and bail out. Link: https://tracker.ceph.com/issues/77594 Signed-off-by: Xiubo Li --- fs/ceph/inode.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c index d52e2b389e0b..153a5242080f 100644 --- a/fs/ceph/inode.c +++ b/fs/ceph/inode.c @@ -3252,6 +3252,13 @@ void ceph_inode_shutdown(struct inode *inode) } spin_unlock(&ci->i_ceph_lock); =20 + /* + * The caps are gone now, so wake up anyone waiting for them -- they + * are never coming back and the waiters need to notice the shutdown + * instead. remove_session_caps_cb() does the same after purging. + */ + wake_up_all(&ci->i_cap_wq); + if (invalidate) ceph_queue_invalidate(inode); while (iputs--) --=20 2.53.0 From nobody Sun Sep 27 02:19:24 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 351C841CB39; Thu, 27 Aug 2026 10:16:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787825800; cv=none; b=OArkGiAzsv/7sY76/lYQ3jearbo5NRgHzG3WmZdI9YIzhbI+Z/+PjeqSpFdyApFyYKlJRb5CwAyQlwfA46rMyEgYMJ5COhFLW7FsqMUrz3+oC7vLLgy2SqsygK2CnP++D33Ve0nJG/uueFfDeDCVYg1L9Ku2kYMxsCwe2QX13pk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787825800; c=relaxed/simple; bh=t/npOUta5D5kvnhxV+B9zZPNVl3Rws5yfaKtbmemjD8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RjphEpctYilHR3H2Bjum5cfpbmXYuxWCmZRV9otNT7fb6ZlZK2iuK1f4j5ZFHZ+FneVAv5DD6jLEztJdOB9ojYqRCx0VlRB/tKFMqoJnHOV4FCyh9dw0E5OfifPcRnZt+Zm57QXNqlABV2dGtcT/ZNnNaldRj52Mio1bgJ2n2yw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KKMJ8o+3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KKMJ8o+3" Received: by smtp.kernel.org (Postfix) with ESMTPS id 384FAC2BCF7; Thu, 27 Aug 2026 10:16:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1787825799; bh=t/npOUta5D5kvnhxV+B9zZPNVl3Rws5yfaKtbmemjD8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=KKMJ8o+3u8R4K3rizYMy5hGGTLueR2BjG2pyE4CH8OfbRsOGfR4MdOIxnedjD0Qul yqW/JpnzpaMRwwcnfoWzob2Huzfz+iU76w4zTPVdePO+CLyBO+dfjWMsCRC3Iw+8+d bvfHPrcJTDpldCc9qJJIKh/0panxE2r93dVOH4FagdFzyLFnM5Eq3xM8NFdokS7OTr XZRSMZpneLEfBRHx7nuqE/qozlsjda2lwTt2W1/NHeyOk7BzjDiL3cefgDpyfqUssc 2Lx0TkmvhdISeAo8DG84f0hPiwPsCTKdz0/1PZFiw4XJP6QRd861U3pOCoQhwrXYaS avqOmuBSkeHLQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C177C61DCB; Thu, 27 Aug 2026 10:16:39 +0000 (UTC) From: Xiubo Li via B4 Relay Date: Thu, 27 Aug 2026 03:16:04 -0700 Subject: [PATCH v6 2/2] ceph: add 'lazyio' mount option to kclient Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260827-lazyio-v6-2-2755b379b8de@clyso.com> References: <20260827-lazyio-v6-0-2755b379b8de@clyso.com> In-Reply-To: <20260827-lazyio-v6-0-2755b379b8de@clyso.com> To: Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko Cc: ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org, Xiubo Li X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1787825763; l=26264; i=xiubo.li@clyso.com; s=20260625; h=from:subject:message-id; bh=dzkxCsGeXf7Y/WXJ3aen2qYOzyHPlNjEyoxRy6CMCLw=; b=VPg660incPT39F3EJBRgHayfyOKoKDnRgXE3nVYT/RkBmc5FiR8n7N2loBthkdpDxOb/pIlZE FvgB9UxUrW9BLEydRIlhpLdDMDc8oKUG2/+sQdcj1oNDGnl7j4IF6hw X-Developer-Key: i=xiubo.li@clyso.com; a=ed25519; pk=V3NGr0AgAopiUhaLY51ipBkLN5LlcLhjOEfLEq1RoZ8= X-Endpoint-Received: by B4 Relay for xiubo.li@clyso.com/20260625 with auth_id=840 X-Original-From: Xiubo Li Reply-To: xiubo.li@clyso.com From: Xiubo Li Add a 'lazyio' mount option to the kernel Ceph client that enables LazyIO globally for all regular file opens on a mount. This is the kclient equivalent of the 'client_force_lazyio=3Dtrue' config option in the ceph-fuse userspace client. When 'lazyio' is specified, CEPH_FILE_MODE_LAZY is automatically added to every regular file's fmode at open time in ceph_open() and ceph_atomic_open(), causing the I/O paths to request CEPH_CAP_FILE_LAZYIO from the MDS. This permits buffered I/O via the page cache even when multiple clients have the file open for write =E2=80=94 beneficial for HPC workloads that can tolerate relaxed cache coherency. The mount option is exposed as 'lazyio' / 'nolazyio' via the VFS fsparam_flag_no mechanism and supports remount. Link: https://tracker.ceph.com/issues/77594 Signed-off-by: Xiubo Li --- fs/ceph/addr.c | 138 +++++++++++++++++++++++++++++++++ fs/ceph/caps.c | 177 +++++++++++++++++++++++++++++++++++++++= +--- fs/ceph/file.c | 33 ++++++-- fs/ceph/super.c | 15 ++++ fs/ceph/super.h | 17 +++++ fs/ceph/util.c | 4 - include/linux/ceph/ceph_fs.h | 1 + 7 files changed, 365 insertions(+), 20 deletions(-) diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index 18cbb18eee75..1e4f03625912 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -493,6 +493,10 @@ static int ceph_init_request(struct netfs_io_request *= rreq, struct file *file) rreq->netfs_priv =3D priv; return 0; } + + /* If this is a lazy fd, also try to get LAZYIO caps */ + if (fi->fmode & CEPH_FILE_MODE_LAZY) + want |=3D CEPH_CAP_FILE_LAZYIO; } =20 /* @@ -1994,6 +1998,42 @@ static void ceph_restore_sigs(sigset_t *oldset) sigprocmask(SIG_SETMASK, oldset, NULL); } =20 +/* + * True if the page cache is only being kept alive by LAZYIO: CACHE is no + * longer present in the implemented cap set, while LAZYIO still is, so + * handle_cap_grant() left the pages in place for the lazy openers. + * + * Such pages are only valid under the lazy semantics, and a non-lazy read= er + * must not be served from them. try_get_cap_refs() already refuses to + * satisfy a non-lazy CACHE want with LAZYIO for read_iter()/write_iter(); + * faults need the same gate. + * + * Unlike __prep_cap(), which has to reason about a single cap, the questi= on + * here is whether the cached data can have gone stale, so the aggregate of + * all caps is the right granularity: as long as any one of them still has + * CACHE implemented, its revocation has not been ACKed and the MDS cannot + * have handed CACHE to another client. Note that __ceph_caps_issued() + * reports i_snap_caps in "have" only and not in "implemented", hence the + * OR; snap caps never include LAZYIO, so folding them in can only make th= is + * more conservative. + */ +static bool ceph_pages_retained_for_lazyio(struct inode *inode) +{ + struct ceph_inode_info *ci =3D ceph_inode(inode); + int have, implemented; + + if (!inode->i_data.nrpages) + return false; + + spin_lock(&ci->i_ceph_lock); + have =3D __ceph_caps_issued(ci, &implemented); + implemented |=3D have; + spin_unlock(&ci->i_ceph_lock); + + return !(implemented & CEPH_CAP_FILE_CACHE) && + (implemented & CEPH_CAP_FILE_LAZYIO); +} + /* * vm ops */ @@ -2029,6 +2069,43 @@ static vm_fault_t ceph_filemap_fault(struct vm_fault= *vmf) doutc(cl, "%llx.%llx %llu got cap refs on %s\n", ceph_vinop(inode), off, ceph_cap_string(got)); =20 + /* + * Don't fault a non-lazy mapping in from page cache that only LAZYIO + * covers. The "every opener is lazy" gate in handle_cap_grant() can't + * catch this on its own: an open that happens after the cache was + * retained sends no cap message, so nothing re-evaluates the decision. + * Drop the range here instead and let the fault below re-read it. + */ + if (!(fi->fmode & CEPH_FILE_MODE_LAZY) && + !(got & CEPH_CAP_FILE_CACHE) && + ceph_pages_retained_for_lazyio(inode)) { + struct address_space *mapping =3D inode->i_mapping; + + doutc(cl, "%llx.%llx %llu dropping LAZYIO page cache\n", + ceph_vinop(inode), off); + err =3D invalidate_inode_pages2_range(mapping, vmf->pgoff, + vmf->pgoff); + if (err =3D=3D -EBUSY) { + /* + * The folio is dirty and ceph has no launder_folio(), + * so invalidate_inode_pages2_range() cannot drop it. + * Write it back and retry -- handing a folio that only + * LAZYIO covers to a non-lazy vma is not an option. + */ + err =3D filemap_write_and_wait_range(mapping, off, + off + PAGE_SIZE - 1); + if (!err) + err =3D invalidate_inode_pages2_range(mapping, + vmf->pgoff, vmf->pgoff); + } + if (err) { + doutc(cl, "%llx.%llx %llu failed to drop LAZYIO page cache: %d\n", + ceph_vinop(inode), off, err); + ceph_put_cap_refs(ci, got); + goto out_restore; + } + } + if ((got & (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_LAZYIO)) || !ceph_has_inline_data(ci)) { CEPH_DEFINE_RW_CONTEXT(rw_ctx, got); @@ -2088,6 +2165,30 @@ static vm_fault_t ceph_filemap_fault(struct vm_fault= *vmf) return ret; } =20 +/* + * Return true if the MDS has issued us a cap that covers buffered + * dirtying: either BUFFER, or LAZYIO (as long as LAZYIO itself is not + * being revoked). This mirrors the conditions under which + * try_get_cap_refs() will hand out a BUFFER or LAZYIO ref. + */ +static bool ceph_have_dirtyable_caps(struct inode *inode) +{ + struct ceph_inode_info *ci =3D ceph_inode(inode); + int have, implemented; + bool ret =3D false; + + spin_lock(&ci->i_ceph_lock); + have =3D __ceph_caps_issued(ci, &implemented); + if (have & CEPH_CAP_FILE_BUFFER) { + ret =3D true; + } else if ((have & CEPH_CAP_FILE_LAZYIO) && + !((implemented & ~have) & CEPH_CAP_FILE_LAZYIO)) { + ret =3D true; + } + spin_unlock(&ci->i_ceph_lock); + return ret; +} + static vm_fault_t ceph_page_mkwrite(struct vm_fault *vmf) { struct vm_area_struct *vma =3D vmf->vma; @@ -2126,6 +2227,7 @@ static vm_fault_t ceph_page_mkwrite(struct vm_fault *= vmf) else want =3D CEPH_CAP_FILE_BUFFER; =20 +retry_caps: got =3D 0; err =3D ceph_get_caps(vma->vm_file, CEPH_CAP_FILE_WR, want, off + len, &g= ot); if (err < 0) @@ -2134,6 +2236,42 @@ static vm_fault_t ceph_page_mkwrite(struct vm_fault = *vmf) doutc(cl, "%llx.%llx %llu~%zd got cap refs on %s\n", ceph_vinop(inode), off, len, ceph_cap_string(got)); =20 + /* + * ceph_write_iter() makes the same check and falls back to + * synchronous writes, but a page fault has no such fallback: + * dirtying the folio without BUFFER or LAZYIO would leave dirty + * data uncovered by any issued cap (e.g. while LAZYIO is being + * revoked, or after it has been released). Wait for the MDS to + * (re)grant a covering cap, matching how the exclude gate in + * try_get_cap_refs() blocks buffered writes while BUFFER is + * revoking. + */ + if ((fi->fmode & CEPH_FILE_MODE_LAZY) && + (got & (CEPH_CAP_FILE_BUFFER | CEPH_CAP_FILE_LAZYIO)) =3D=3D 0) { + ceph_put_cap_refs(ci, got); + got =3D 0; + doutc(cl, "%llx.%llx %llu~%zd waiting for BUFFER or LAZYIO\n", + ceph_vinop(inode), off, len); + /* + * Bail out if the inode is being shut down: the caps are gone + * for good, so ceph_have_dirtyable_caps() would never become + * true again. Note that we cannot rely on ceph_get_caps() + * failing with -ESTALE on the retry either -- if WR is still + * held it would succeed without BUFFER or LAZYIO and send us + * straight back here. + */ + err =3D wait_event_killable(ci->i_cap_wq, + ceph_have_dirtyable_caps(inode) || + ceph_inode_is_shutdown(inode)); + if (err) + goto out_free; + if (ceph_inode_is_shutdown(inode)) { + err =3D -ESTALE; + goto out_free; + } + goto retry_caps; + } + /* Update time before taking folio lock */ file_update_time(vma->vm_file); inode_inc_iversion_raw(inode); diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index bcb04c6cb92c..ba079483b145 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -1028,6 +1028,53 @@ int __ceph_caps_used(struct ceph_inode_info *ci) return used; } =20 +/* + * Substitute LAZYIO for CACHE/BUFFER when they are not issued. + * If we have LAZYIO but not CACHE/BUFFER, report LAZYIO as used instead + * so the MDS knows we're fine with the weaker consistency guarantee. + * + * Base the substitution on "implemented" rather than "issued": while + * LAZYIO is being revoked, "issued" no longer contains it but + * "implemented" still does. If used reverted to CACHE/BUFFER at that + * point, ceph_check_caps() would see (revoking & cap_used) =3D=3D 0 and + * ACK the revoke while dirty or stale pages were still present. Only + * once the revoke is ACKed does "implemented" drop LAZYIO. + * + * Caller must hold i_ceph_lock. + */ +static inline int ceph_adjust_caps_used_for_lazyio(struct ceph_inode_info = *ci, + int used, int issued, + int implemented) +{ + if (!(used & (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_BUFFER))) + return used; + if (!(implemented & CEPH_CAP_FILE_LAZYIO)) + return used; + /* + * While LAZYIO is still issued, it only covers the cached data if + * every opener has accepted the lazy semantics. With a non-lazy + * opener around, keep reporting CACHE/BUFFER so that a pending + * revocation is not ACKed before the invalidation queued by + * handle_cap_grant() has emptied the page cache. + * + * Once LAZYIO is itself being revoked ("implemented" has it but + * "issued" no longer does) the substitution is unconditional: the + * revoke must not be ACKed while pages are still around, no matter + * who has the file open. + */ + if ((issued & CEPH_CAP_FILE_LAZYIO) && !__ceph_all_opens_lazy(ci)) + return used; + if (!(issued & CEPH_CAP_FILE_CACHE)) { + used &=3D ~CEPH_CAP_FILE_CACHE; + used |=3D CEPH_CAP_FILE_LAZYIO; + } + if (!(issued & CEPH_CAP_FILE_BUFFER)) { + used &=3D ~CEPH_CAP_FILE_BUFFER; + used |=3D CEPH_CAP_FILE_LAZYIO; + } + return used; +} + #define FMODE_WAIT_BIAS 1000 =20 /* @@ -1449,6 +1496,20 @@ static void __prep_cap(struct cap_msg_args *arg, str= uct ceph_inode_info *ci, * dirty pages _before_ allowing sync writes to avoid reordering. */ arg->wake =3D cap->implemented & ~cap->issued; + /* + * "used" decides which of the revoked caps stay in cap->implemented, + * i.e. which revocations this message does _not_ ack yet. Callers on + * the flush paths (try_flush_caps(), __kick_flushing_caps()) hand us + * the raw __ceph_caps_used(), which reports CACHE/BUFFER as long as + * the page cache is populated. While LAZYIO is being revoked it is no + * longer in cap->issued, so a raw used set would drop it from + * cap->implemented and implicitly ack the revoke with pages still + * cached -- the MDS could then hand CACHE to another client. Report + * LAZYIO as used instead, and only stop doing so once writeback and + * invalidation have emptied the page cache. + */ + used =3D ceph_adjust_caps_used_for_lazyio(ci, used, cap->issued, + cap->implemented); cap->implemented &=3D cap->issued | used; cap->mds_wanted =3D want; =20 @@ -2094,6 +2155,7 @@ void ceph_check_caps(struct ceph_inode_info *ci, int = flags) * usually because they have outstanding references). */ issued =3D __ceph_caps_issued(ci, &implemented); + revoking =3D implemented & ~issued; =20 want =3D file_wanted; @@ -2185,6 +2247,19 @@ void ceph_check_caps(struct ceph_inode_info *ci, int= flags) if (ci->i_auth_cap && cap !=3D ci->i_auth_cap) cap_used &=3D ~ci->i_auth_cap->issued; =20 + /* + * Substitute LAZYIO for CACHE/BUFFER when they are not issued. + * Do this per cap and not once for the whole inode: only the + * cap that actually holds LAZYIO may report it as used. A cap + * that is revoking CACHE without holding LAZYIO must keep + * reporting CACHE as used, or the (revoking & cap_used) test + * below would treat the revocation as completed while the page + * cache is still populated. + */ + cap_used =3D ceph_adjust_caps_used_for_lazyio(ci, cap_used, + cap->issued, + cap->implemented); + revoking =3D cap->implemented & ~cap->issued; doutc(cl, " mds%d cap %p used %s issued %s implemented %s revoking %s\n", cap->mds, cap, ceph_cap_string(cap_used), @@ -2209,10 +2284,13 @@ void ceph_check_caps(struct ceph_inode_info *ci, in= t flags) * at most 5 seconds. That means the MDS needs to wait at * most 5 seconds to finished the Fb capability's revocation. * - * Let's queue a writeback for it. + * Let's queue a writeback for it. The same applies when + * LAZYIO is revoked while it was covering for BUFFER + * (dirty pages exist, but BUFFER isn't issued). */ if (S_ISREG(inode->i_mode) && ci->i_wrbuffer_ref && - (revoking & CEPH_CAP_FILE_BUFFER)) + (revoking & (CEPH_CAP_FILE_BUFFER | + CEPH_CAP_FILE_LAZYIO))) queue_writeback =3D true; } =20 @@ -2957,9 +3035,45 @@ static int try_get_cap_refs(struct inode *inode, int= need, int want, } snap_rwsem_locked =3D true; } - if ((have & want) =3D=3D want) + /* + * Allow LAZYIO to act as a substitute for + * CACHE or BUFFER when those caps are not + * issued, but only for callers that + * explicitly requested LAZYIO. This + * prevents a non-lazy fd from having its + * CACHE/BUFFER wants satisfied by LAZYIO + * on an inode where a different fd is lazy. + * + * A missing LAZYIO cap, however, must never + * cost us the CACHE/BUFFER refs that are + * actually issued: the MDS only grants + * LAZYIO for files opened with CEPH_O_LAZY + * and it can be revoked at any time. If we + * made the whole want unsatisfiable without + * it, the I/O paths would silently drop + * CACHE/BUFFER (e.g. take no wrbuffer refs) + * and degrade to synchronous writes. + */ + if ((have & (want & ~CEPH_CAP_FILE_LAZYIO)) =3D=3D + (want & ~CEPH_CAP_FILE_LAZYIO)) { + *got =3D need | (want & ~exclude & + ~CEPH_CAP_FILE_LAZYIO); + if ((want & CEPH_CAP_FILE_LAZYIO) && + (have & CEPH_CAP_FILE_LAZYIO) && + !(exclude & CEPH_CAP_FILE_LAZYIO)) + *got |=3D CEPH_CAP_FILE_LAZYIO; + } else if ((want & CEPH_CAP_FILE_LAZYIO) && + (have & CEPH_CAP_FILE_LAZYIO) && + !(exclude & CEPH_CAP_FILE_LAZYIO) && + ((have & want) =3D=3D + (want & ~(CEPH_CAP_FILE_CACHE | + CEPH_CAP_FILE_BUFFER)))) { + /* + * LAZYIO substitutes for missing CACHE/BUFFER; + * it is already included via (want & ~exclude). + */ *got =3D need | (want & ~exclude); - else + } else *got =3D need; ceph_take_cap_refs(ci, *got, true); ret =3D 1; @@ -3048,6 +3162,14 @@ static void check_max_size(struct inode *inode, loff= _t endoff) ceph_check_caps(ci, CHECK_CAPS_AUTHONLY); } =20 +/* + * The inverse of ceph_caps_for_mode(). LAZYIO has to be mapped back as + * well: __ceph_get_caps() feeds the result to ceph_get_fmode() to bias + * i_nr_by_mode[] by FMODE_WAIT_BIAS while it waits, and dropping LAZY + * here would bias the open count without biasing the lazy count, making + * __ceph_all_opens_lazy() report a phantom non-lazy opener for as long + * as a lazy fd is waiting for caps. + */ static inline int get_used_fmode(int caps) { int fmode =3D 0; @@ -3055,6 +3177,8 @@ static inline int get_used_fmode(int caps) fmode |=3D CEPH_FILE_MODE_RD; if (caps & CEPH_CAP_FILE_WR) fmode |=3D CEPH_FILE_MODE_WR; + if (caps & CEPH_CAP_FILE_LAZYIO) + fmode |=3D CEPH_FILE_MODE_LAZY; return fmode; } =20 @@ -3590,13 +3714,29 @@ static void handle_cap_grant(struct inode *inode, =20 =20 /* - * If CACHE is being revoked, and we have no dirty buffers, - * try to invalidate (once). (If there are dirty buffers, we - * will invalidate _after_ writeback.) + * Check the revocation of *both* CACHE and LAZYIO, because + * CACHE may have been revoked earlier and cap->issued no + * longer contains it -- at that point only LAZYIO was + * covering us. If LAZYIO is now also being revoked and no + * cache cap remains, we must invalidate the page cache. + * Without this, a CACHE-revoked-then-LAZYIO-revoked sequence + * leaves stale pages in memory until the next periodic + * check_caps (up to 60s). Also invalidate when we have no + * dirty buffers (if dirty, invalidate after writeback). + * + * Keeping the page cache alive on a remaining LAZYIO is only + * safe if every opener has accepted the lazy semantics: those + * pages are retained for the lazy fds alone, and a non-lazy + * reader (mmap, or an fd that never asked for LAZYIO) must not + * be served from them. So drop the cache as well whenever a + * non-lazy opener is around. */ if (S_ISREG(inode->i_mode) && /* don't invalidate readdir cache */ - ((cap->issued & ~newcaps) & CEPH_CAP_FILE_CACHE) && - (newcaps & CEPH_CAP_FILE_LAZYIO) =3D=3D 0 && + ((cap->issued & ~newcaps) & + (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_LAZYIO)) && + !(newcaps & CEPH_CAP_FILE_CACHE) && + (!(newcaps & CEPH_CAP_FILE_LAZYIO) || + !__ceph_all_opens_lazy(ci)) && !(ci->i_wrbuffer_ref || ci->i_wb_ref)) { if (try_nonblocking_invalidate(inode)) { /* there were locked pages.. invalidate later @@ -3740,6 +3880,8 @@ static void handle_cap_grant(struct inode *inode, /* check cap bits */ wanted =3D __ceph_caps_wanted(ci); used =3D __ceph_caps_used(ci); + used =3D ceph_adjust_caps_used_for_lazyio(ci, used, cap->issued, + cap->implemented); dirty =3D __ceph_caps_dirty(ci); doutc(cl, " my wanted =3D %s, used =3D %s, dirty %s\n", ceph_cap_string(wanted), ceph_cap_string(used), @@ -3767,13 +3909,26 @@ static void handle_cap_grant(struct inode *inode, doutc(cl, "revocation: %s -> %s (revoking %s)\n", ceph_cap_string(cap->issued), ceph_cap_string(newcaps), ceph_cap_string(revoking)); + /* + * If BUFFER is being revoked and we have dirty data, + * trigger writeback before acking. When LAZYIO was + * covering for BUFFER (BUFFER not issued, dirty refs + * held), also trigger writeback. Clean cached pages + * under LAZYIO are handled by queue_invalidate below. + */ if (S_ISREG(inode->i_mode) && (revoking & used & CEPH_CAP_FILE_BUFFER)) { writeback =3D true; /* initiate writeback; will delay ack */ revoke_wait =3D true; + } else if (S_ISREG(inode->i_mode) && + (revoking & used & CEPH_CAP_FILE_LAZYIO) && + (ci->i_wrbuffer_ref || ci->i_wb_ref)) { + /* LAZYIO was covering for dirty data =E2=80=94 flush first */ + writeback =3D true; + revoke_wait =3D true; } else if (queue_invalidate && - revoking =3D=3D CEPH_CAP_FILE_CACHE && - (newcaps & CEPH_CAP_FILE_LAZYIO) =3D=3D 0) { + (revoking & (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_LAZYIO)) && + !(newcaps & (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_LAZYIO))) { revoke_wait =3D true; /* do nothing yet, invalidation will be queued */ } else if (cap =3D=3D ci->i_auth_cap) { check_caps =3D 1; /* check auth cap only */ diff --git a/fs/ceph/file.c b/fs/ceph/file.c index bd3e3f5c269e..b621930b59f0 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -347,10 +347,6 @@ int ceph_renew_caps(struct inode *inode, int fmode) flags =3D O_RDONLY; else if (wanted & CEPH_CAP_FILE_WR) flags =3D O_WRONLY; -#ifdef O_LAZY - if (wanted & CEPH_CAP_FILE_LAZYIO) - flags |=3D O_LAZY; -#endif =20 req =3D prepare_open_request(inode->i_sb, flags, 0); if (IS_ERR(req)) { @@ -358,6 +354,10 @@ int ceph_renew_caps(struct inode *inode, int fmode) goto out; } =20 + if (wanted & CEPH_CAP_FILE_LAZYIO) { + req->r_fmode |=3D CEPH_FILE_MODE_LAZY; + req->r_args.open.flags |=3D cpu_to_le32(CEPH_O_LAZY); + } req->r_inode =3D inode; ihold(inode); req->r_num_caps =3D 1; @@ -409,6 +409,19 @@ int ceph_open(struct inode *inode, struct file *file) doutc(cl, "%p %llx.%llx file %p flags %d (%d)\n", inode, ceph_vinop(inode), file, flags, file->f_flags); fmode =3D ceph_flags_to_mode(flags); + + /* + * If lazyio mount option is set, enable lazyio for all regular + * files. Skip snapped files: snap caps never include LAZYIO, + * so including it in wanted would force an unnecessary MDS + * round-trip for every open of a snapped file. + */ + if (S_ISREG(inode->i_mode) && + ceph_snap(inode) =3D=3D CEPH_NOSNAP && + (fsc->mount_options->flags & CEPH_MOUNT_OPT_LAZYIO)) { + fmode |=3D CEPH_FILE_MODE_LAZY; + } + wanted =3D ceph_caps_for_mode(fmode); =20 if (fmode & CEPH_FILE_MODE_WR) @@ -485,13 +498,16 @@ int ceph_open(struct inode *inode, struct file *file) err =3D PTR_ERR(req); goto out; } + req->r_fmode |=3D fmode & CEPH_FILE_MODE_LAZY; + if (fmode & CEPH_FILE_MODE_LAZY) + req->r_args.open.flags |=3D cpu_to_le32(CEPH_O_LAZY); req->r_inode =3D inode; ihold(inode); =20 req->r_num_caps =3D 1; err =3D ceph_mdsc_do_request(mdsc, NULL, req); if (!err) - err =3D ceph_init_file(inode, file, req->r_fmode); + err =3D ceph_init_file(inode, file, fmode); ceph_mdsc_put_request(req); doutc(cl, "open result=3D%d on %llx.%llx\n", err, ceph_vinop(inode)); out: @@ -836,6 +852,9 @@ int ceph_atomic_open(struct inode *dir, struct dentry *= dentry, } else { int fmode =3D ceph_flags_to_mode(flags); =20 + if (fsc->mount_options->flags & CEPH_MOUNT_OPT_LAZYIO) + fmode |=3D CEPH_FILE_MODE_LAZY; + mask =3D MAY_READ; if (fmode & CEPH_FILE_MODE_WR) mask |=3D MAY_WRITE; @@ -878,6 +897,10 @@ int ceph_atomic_open(struct inode *dir, struct dentry = *dentry, err =3D PTR_ERR(req); goto out_ctx; } + if (fsc->mount_options->flags & CEPH_MOUNT_OPT_LAZYIO) { + req->r_fmode |=3D CEPH_FILE_MODE_LAZY; + req->r_args.open.flags |=3D cpu_to_le32(CEPH_O_LAZY); + } req->r_dentry =3D dget(dentry); req->r_num_caps =3D 2; mask =3D CEPH_STAT_CAP_INODE | CEPH_CAP_AUTH_SHARED; diff --git a/fs/ceph/super.c b/fs/ceph/super.c index 6c6e9a1100db..1667f0be6d82 100644 --- a/fs/ceph/super.c +++ b/fs/ceph/super.c @@ -181,6 +181,7 @@ enum { Opt_pagecache, Opt_sparseread, Opt_nearfull_sync, + Opt_lazyio, }; =20 enum ceph_recover_session_mode { @@ -207,6 +208,7 @@ static const struct fs_parameter_spec ceph_mount_parame= ters[] =3D { fsparam_flag_no ("fsc", Opt_fscache), // fsc|nofsc fsparam_string ("fsc", Opt_fscache), // fsc=3D... fsparam_flag_no ("ino32", Opt_ino32), + fsparam_flag_no ("lazyio", Opt_lazyio), fsparam_string ("mds_namespace", Opt_mds_namespace), fsparam_string ("mon_addr", Opt_mon_addr), fsparam_flag_no ("nearfull_sync", Opt_nearfull_sync), @@ -604,6 +606,12 @@ static int ceph_parse_mount_param(struct fs_context *f= c, else fsopt->flags |=3D CEPH_MOUNT_OPT_NEARFULL_SYNC; break; + case Opt_lazyio: + if (result.negated) + fsopt->flags &=3D ~CEPH_MOUNT_OPT_LAZYIO; + else + fsopt->flags |=3D CEPH_MOUNT_OPT_LAZYIO; + break; case Opt_test_dummy_encryption: #ifdef CONFIG_FS_ENCRYPTION fscrypt_free_dummy_policy(&fsopt->dummy_enc_policy); @@ -762,6 +770,8 @@ static int ceph_show_options(struct seq_file *m, struct= dentry *root) seq_puts(m, ",sparseread"); if (fsopt->flags & CEPH_MOUNT_OPT_NEARFULL_SYNC) seq_puts(m, ",nearfull_sync"); + if (fsopt->flags & CEPH_MOUNT_OPT_LAZYIO) + seq_puts(m, ",lazyio"); =20 fscrypt_show_test_dummy_encryption(m, ',', root->d_sb); =20 @@ -1423,6 +1433,11 @@ static int ceph_reconfigure_fc(struct fs_context *fc) else ceph_clear_mount_opt(fsc, SPARSEREAD); =20 + if (fsopt->flags & CEPH_MOUNT_OPT_LAZYIO) + ceph_set_mount_opt(fsc, LAZYIO); + else + ceph_clear_mount_opt(fsc, LAZYIO); + if (strcmp_null(fsc->mount_options->mon_addr, fsopt->mon_addr)) { kfree(fsc->mount_options->mon_addr); fsc->mount_options->mon_addr =3D fsopt->mon_addr; diff --git a/fs/ceph/super.h b/fs/ceph/super.h index 72d4e30304dc..cbe898c56a49 100644 --- a/fs/ceph/super.h +++ b/fs/ceph/super.h @@ -46,6 +46,7 @@ #define CEPH_MOUNT_OPT_NOPAGECACHE (1<<16) /* bypass pagecache altoget= her */ #define CEPH_MOUNT_OPT_SPARSEREAD (1<<17) /* always do sparse reads */ #define CEPH_MOUNT_OPT_NEARFULL_SYNC (1<<18) /* sync writes when nearful= l */ +#define CEPH_MOUNT_OPT_LAZYIO (1<<19) /* force lazyio for all fil= e opens */ =20 #define CEPH_MOUNT_OPT_DEFAULT \ (CEPH_MOUNT_OPT_DCACHE | \ @@ -875,6 +876,22 @@ static inline bool __ceph_is_file_opened(struct ceph_i= node_info *ci) { return ci->i_nr_by_mode[0]; } + +/* + * True if the inode is open and every opener is a lazy one. Bit 0 of + * i_nr_by_mode[] is set for every open regardless of mode (see + * ceph_get_fmode()), so it doubles as the total open count. + * + * Callers must hold i_ceph_lock. + */ +static inline bool __ceph_all_opens_lazy(struct ceph_inode_info *ci) +{ + int nr_open =3D ci->i_nr_by_mode[ffs(CEPH_FILE_MODE_PIN)]; + int nr_lazy =3D ci->i_nr_by_mode[ffs(CEPH_FILE_MODE_LAZY)]; + + return nr_open > 0 && nr_lazy >=3D nr_open; +} + extern int __ceph_caps_file_wanted(struct ceph_inode_info *ci); extern int __ceph_caps_wanted(struct ceph_inode_info *ci); =20 diff --git a/fs/ceph/util.c b/fs/ceph/util.c index 2c34875675bf..be3db3f20344 100644 --- a/fs/ceph/util.c +++ b/fs/ceph/util.c @@ -73,10 +73,6 @@ int ceph_flags_to_mode(int flags) mode =3D CEPH_FILE_MODE_RDWR; break; } -#ifdef O_LAZY - if (flags & O_LAZY) - mode |=3D CEPH_FILE_MODE_LAZY; -#endif =20 return mode; } diff --git a/include/linux/ceph/ceph_fs.h b/include/linux/ceph/ceph_fs.h index 69ac3e55a3fe..01fd5f6647c8 100644 --- a/include/linux/ceph/ceph_fs.h +++ b/include/linux/ceph/ceph_fs.h @@ -414,6 +414,7 @@ extern const char *ceph_mds_op_name(int op); #define CEPH_O_CREAT 00000100 #define CEPH_O_EXCL 00000200 #define CEPH_O_TRUNC 00001000 +#define CEPH_O_LAZY 00020000 #define CEPH_O_DIRECTORY 00200000 #define CEPH_O_NOFOLLOW 00400000 =20 --=20 2.53.0