fs/ceph/file.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+)
From: Xiubo Li <xiubo.li@clyso.com>
For O_APPEND writes, ki_pos is set to the current EOF via
generic_write_checks() after fetching i_size from the MDS. However,
ceph_get_caps() may need to wait for Fwx exclusive caps if the write
extends the file (endoff > i_max_size). While waiting for Fwx, the
previous Fwx holder (another client) may have already extended the
file. When the MDS grants us Fwx, the cap grant message updates the
local i_size, but ki_pos remains at the old EOF, causing the append
write to land at a stale offset and overwrite data from the other
client.
Fix by re-reading i_size_read(inode) after ceph_get_caps() returns.
At this point we hold Fwx exclusive caps, no other client can modify
the file, and i_size reflects the true EOF from the MDS cap grant.
No extra MDS round-trip is needed. Only adjust ki_pos when the EOF
has actually changed.
After adjusting ki_pos forward, the write range [pos, pos+count) may
now exceed the i_max_size that was validated by ceph_get_caps() for
the old range. Re-check against i_max_size and truncate the write
if necessary to stay within the MDS-granted limit.
Fixes: 8e4473bb50a1 ("ceph: do not execute direct write in parallel if O_APPEND is specified")
Link: https://tracker.ceph.com/issues/7333
Signed-off-by: Xiubo Li <xiubo.li@clyso.com>
---
Changes in v3:
- Protect ci->i_max_size access with i_ceph_lock to avoid data race with handle_cap_grant() which updates i_max_size from the messenger thread.(Viacheslav Dubeyko)
- Link to v2: https://patch.msgid.link/20260718-ceph-append-fix-v2-1-88dcc4f8371f@clyso.com
Changes in v2:
- Re-check write range against i_max_size after adjusting ki_pos forward, and truncate if it exceeds the MDS-granted limit.(Viacheslav Dubeyko)
- Link to v1: https://patch.msgid.link/20260717-ceph-append-fix-v1-1-c51907ac8e36@clyso.com
---
fs/ceph/file.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/fs/ceph/file.c b/fs/ceph/file.c
index 3823671ab95a..c26bd4fef7d3 100644
--- a/fs/ceph/file.c
+++ b/fs/ceph/file.c
@@ -2489,6 +2489,54 @@ static ssize_t ceph_write_iter(struct kiocb *iocb, struct iov_iter *from)
if (err < 0)
goto out;
+ /*
+ * For O_APPEND writes we may have waited for Fwx exclusive caps
+ * while the previous Fwx holder (another client) extended the
+ * file. i_size has been updated via the cap grant message from
+ * the MDS, but ki_pos is still the old EOF. Re-read i_size here
+ * (no extra MDS round-trip needed) and adjust ki_pos to the true
+ * EOF. Since we hold Fwx, no other client can change the file.
+ */
+ if (iocb->ki_flags & IOCB_APPEND) {
+ loff_t cur_eof = i_size_read(inode);
+
+ if (cur_eof != pos) {
+ doutc(cl,
+ "%p %llx.%llx O_APPEND: pos adjusted %lld -> %lld\n",
+ inode, ceph_vinop(inode), pos, cur_eof);
+ iocb->ki_pos = cur_eof;
+ pos = cur_eof;
+ if (pos >= limit) {
+ err = -EFBIG;
+ goto out_caps;
+ }
+ iov_iter_truncate(from, limit - pos);
+ count = iov_iter_count(from);
+
+ /*
+ * ceph_get_caps() validated the old endoff
+ * against i_max_size; adjusting ki_pos forward
+ * may have shifted the write range beyond the
+ * granted max_size. Re-check and truncate if
+ * necessary.
+ */
+ spin_lock(&ci->i_ceph_lock);
+ if (pos + count > (loff_t)ci->i_max_size) {
+ loff_t max_size = ci->i_max_size;
+
+ spin_unlock(&ci->i_ceph_lock);
+ if (pos >= max_size) {
+ err = -EFBIG;
+ goto out_caps;
+ }
+ iov_iter_truncate(from, max_size - pos);
+ count = iov_iter_count(from);
+ } else {
+ spin_unlock(&ci->i_ceph_lock);
+ }
+ }
+ }
+
err = file_update_time(file);
if (err)
goto out_caps;
---
base-commit: afffcd98d2d22f2e7ebe42f71231d52d4eed8ef7
change-id: 20260717-ceph-append-fix-9820e3b1a78c
Best regards,
--
Xiubo Li <xiubo.li@clyso.com>On Tue, 2026-07-21 at 13:06 +0800, Xiubo Li via B4 Relay wrote:
> From: Xiubo Li <xiubo.li@clyso.com>
>
> For O_APPEND writes, ki_pos is set to the current EOF via
> generic_write_checks() after fetching i_size from the MDS. However,
> ceph_get_caps() may need to wait for Fwx exclusive caps if the write
> extends the file (endoff > i_max_size). While waiting for Fwx, the
> previous Fwx holder (another client) may have already extended the
> file. When the MDS grants us Fwx, the cap grant message updates the
> local i_size, but ki_pos remains at the old EOF, causing the append
> write to land at a stale offset and overwrite data from the other
> client.
>
> Fix by re-reading i_size_read(inode) after ceph_get_caps() returns.
> At this point we hold Fwx exclusive caps, no other client can modify
> the file, and i_size reflects the true EOF from the MDS cap grant.
> No extra MDS round-trip is needed. Only adjust ki_pos when the EOF
> has actually changed.
>
> After adjusting ki_pos forward, the write range [pos, pos+count) may
> now exceed the i_max_size that was validated by ceph_get_caps() for
> the old range. Re-check against i_max_size and truncate the write
> if necessary to stay within the MDS-granted limit.
>
> Fixes: 8e4473bb50a1 ("ceph: do not execute direct write in parallel
> if O_APPEND is specified")
> Link: https://tracker.ceph.com/issues/7333
> Signed-off-by: Xiubo Li <xiubo.li@clyso.com>
> ---
> Changes in v3:
> - Protect ci->i_max_size access with i_ceph_lock to avoid data race
> with handle_cap_grant() which updates i_max_size from the messenger
> thread.(Viacheslav Dubeyko)
> - Link to v2:
> https://patch.msgid.link/20260718-ceph-append-fix-v2-1-88dcc4f8371f@clyso.com
>
> Changes in v2:
> - Re-check write range against i_max_size after adjusting ki_pos
> forward, and truncate if it exceeds the MDS-granted limit.(Viacheslav
> Dubeyko)
> - Link to v1:
> https://patch.msgid.link/20260717-ceph-append-fix-v1-1-c51907ac8e36@clyso.com
> ---
> fs/ceph/file.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 48 insertions(+)
>
> diff --git a/fs/ceph/file.c b/fs/ceph/file.c
> index 3823671ab95a..c26bd4fef7d3 100644
> --- a/fs/ceph/file.c
> +++ b/fs/ceph/file.c
> @@ -2489,6 +2489,54 @@ static ssize_t ceph_write_iter(struct kiocb
> *iocb, struct iov_iter *from)
> if (err < 0)
> goto out;
>
> + /*
> + * For O_APPEND writes we may have waited for Fwx exclusive
> caps
> + * while the previous Fwx holder (another client) extended
> the
> + * file. i_size has been updated via the cap grant message
> from
> + * the MDS, but ki_pos is still the old EOF. Re-read i_size
> here
> + * (no extra MDS round-trip needed) and adjust ki_pos to the
> true
> + * EOF. Since we hold Fwx, no other client can change the
> file.
> + */
> + if (iocb->ki_flags & IOCB_APPEND) {
> + loff_t cur_eof = i_size_read(inode);
> +
> + if (cur_eof != pos) {
> + doutc(cl,
> + "%p %llx.%llx O_APPEND: pos adjusted
> %lld -> %lld\n",
> + inode, ceph_vinop(inode), pos,
> cur_eof);
> + iocb->ki_pos = cur_eof;
> + pos = cur_eof;
> + if (pos >= limit) {
> + err = -EFBIG;
> + goto out_caps;
> + }
> + iov_iter_truncate(from, limit - pos);
> + count = iov_iter_count(from);
> +
> + /*
> + * ceph_get_caps() validated the old endoff
> + * against i_max_size; adjusting ki_pos
> forward
> + * may have shifted the write range beyond
> the
> + * granted max_size. Re-check and truncate
> if
> + * necessary.
> + */
> + spin_lock(&ci->i_ceph_lock);
> + if (pos + count > (loff_t)ci->i_max_size) {
> + loff_t max_size = ci->i_max_size;
> +
> + spin_unlock(&ci->i_ceph_lock);
> + if (pos >= max_size) {
> + err = -EFBIG;
> + goto out_caps;
> + }
> + iov_iter_truncate(from, max_size -
> pos);
> + count = iov_iter_count(from);
> + } else {
> + spin_unlock(&ci->i_ceph_lock);
> + }
> + }
> + }
Just comment... :) For my taste, it is better to take the i_max_size
under the lock and then to use it:
spin_lock(&ci->i_ceph_lock);
loff_t max_size = ci->i_max_size;
spin_unlock(&ci->i_ceph_lock);
But it is not critical remarks and I don't see any issue with the
current implementation.
Looks good.
Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com>
Thanks,
Slava.
> +
> err = file_update_time(file);
> if (err)
> goto out_caps;
>
> ---
> base-commit: afffcd98d2d22f2e7ebe42f71231d52d4eed8ef7
> change-id: 20260717-ceph-append-fix-9820e3b1a78c
>
> Best regards,
> --
> Xiubo Li <xiubo.li@clyso.com>
>
Hi Slava,
Sounds good. I'll stick with the current implementation since you
agree it's not critical. Thanks for reviewing and adding the
Reviewed-by tag.
Thanks
Xiubo
On Wed, 22 Jul 2026 at 01:23, Viacheslav Dubeyko <slava@dubeyko.com> wrote:
>
> On Tue, 2026-07-21 at 13:06 +0800, Xiubo Li via B4 Relay wrote:
> > From: Xiubo Li <xiubo.li@clyso.com>
> >
> > For O_APPEND writes, ki_pos is set to the current EOF via
> > generic_write_checks() after fetching i_size from the MDS. However,
> > ceph_get_caps() may need to wait for Fwx exclusive caps if the write
> > extends the file (endoff > i_max_size). While waiting for Fwx, the
> > previous Fwx holder (another client) may have already extended the
> > file. When the MDS grants us Fwx, the cap grant message updates the
> > local i_size, but ki_pos remains at the old EOF, causing the append
> > write to land at a stale offset and overwrite data from the other
> > client.
> >
> > Fix by re-reading i_size_read(inode) after ceph_get_caps() returns.
> > At this point we hold Fwx exclusive caps, no other client can modify
> > the file, and i_size reflects the true EOF from the MDS cap grant.
> > No extra MDS round-trip is needed. Only adjust ki_pos when the EOF
> > has actually changed.
> >
> > After adjusting ki_pos forward, the write range [pos, pos+count) may
> > now exceed the i_max_size that was validated by ceph_get_caps() for
> > the old range. Re-check against i_max_size and truncate the write
> > if necessary to stay within the MDS-granted limit.
> >
> > Fixes: 8e4473bb50a1 ("ceph: do not execute direct write in parallel
> > if O_APPEND is specified")
> > Link: https://tracker.ceph.com/issues/7333
> > Signed-off-by: Xiubo Li <xiubo.li@clyso.com>
> > ---
> > Changes in v3:
> > - Protect ci->i_max_size access with i_ceph_lock to avoid data race
> > with handle_cap_grant() which updates i_max_size from the messenger
> > thread.(Viacheslav Dubeyko)
> > - Link to v2:
> > https://patch.msgid.link/20260718-ceph-append-fix-v2-1-88dcc4f8371f@clyso.com
> >
> > Changes in v2:
> > - Re-check write range against i_max_size after adjusting ki_pos
> > forward, and truncate if it exceeds the MDS-granted limit.(Viacheslav
> > Dubeyko)
> > - Link to v1:
> > https://patch.msgid.link/20260717-ceph-append-fix-v1-1-c51907ac8e36@clyso.com
> > ---
> > fs/ceph/file.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 48 insertions(+)
> >
> > diff --git a/fs/ceph/file.c b/fs/ceph/file.c
> > index 3823671ab95a..c26bd4fef7d3 100644
> > --- a/fs/ceph/file.c
> > +++ b/fs/ceph/file.c
> > @@ -2489,6 +2489,54 @@ static ssize_t ceph_write_iter(struct kiocb
> > *iocb, struct iov_iter *from)
> > if (err < 0)
> > goto out;
> >
> > + /*
> > + * For O_APPEND writes we may have waited for Fwx exclusive
> > caps
> > + * while the previous Fwx holder (another client) extended
> > the
> > + * file. i_size has been updated via the cap grant message
> > from
> > + * the MDS, but ki_pos is still the old EOF. Re-read i_size
> > here
> > + * (no extra MDS round-trip needed) and adjust ki_pos to the
> > true
> > + * EOF. Since we hold Fwx, no other client can change the
> > file.
> > + */
> > + if (iocb->ki_flags & IOCB_APPEND) {
> > + loff_t cur_eof = i_size_read(inode);
> > +
> > + if (cur_eof != pos) {
> > + doutc(cl,
> > + "%p %llx.%llx O_APPEND: pos adjusted
> > %lld -> %lld\n",
> > + inode, ceph_vinop(inode), pos,
> > cur_eof);
> > + iocb->ki_pos = cur_eof;
> > + pos = cur_eof;
> > + if (pos >= limit) {
> > + err = -EFBIG;
> > + goto out_caps;
> > + }
> > + iov_iter_truncate(from, limit - pos);
> > + count = iov_iter_count(from);
> > +
> > + /*
> > + * ceph_get_caps() validated the old endoff
> > + * against i_max_size; adjusting ki_pos
> > forward
> > + * may have shifted the write range beyond
> > the
> > + * granted max_size. Re-check and truncate
> > if
> > + * necessary.
> > + */
> > + spin_lock(&ci->i_ceph_lock);
> > + if (pos + count > (loff_t)ci->i_max_size) {
> > + loff_t max_size = ci->i_max_size;
> > +
> > + spin_unlock(&ci->i_ceph_lock);
> > + if (pos >= max_size) {
> > + err = -EFBIG;
> > + goto out_caps;
> > + }
> > + iov_iter_truncate(from, max_size -
> > pos);
> > + count = iov_iter_count(from);
> > + } else {
> > + spin_unlock(&ci->i_ceph_lock);
> > + }
> > + }
> > + }
>
> Just comment... :) For my taste, it is better to take the i_max_size
> under the lock and then to use it:
>
> spin_lock(&ci->i_ceph_lock);
> loff_t max_size = ci->i_max_size;
> spin_unlock(&ci->i_ceph_lock);
>
> But it is not critical remarks and I don't see any issue with the
> current implementation.
>
> Looks good.
>
> Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com>
>
> Thanks,
> Slava.
>
> > +
> > err = file_update_time(file);
> > if (err)
> > goto out_caps;
> >
> > ---
> > base-commit: afffcd98d2d22f2e7ebe42f71231d52d4eed8ef7
> > change-id: 20260717-ceph-append-fix-9820e3b1a78c
> >
> > Best regards,
> > --
> > Xiubo Li <xiubo.li@clyso.com>
> >
© 2016 - 2026 Red Hat, Inc.