Fast commit is meant to make fsync cheap, but on random-write workloads it
defeats itself.  ext4 still tracks a single coalesced [min,max] logical range
per inode (i_fc_lblk_start/len).  When an inode is dirtied at several disjoint
offsets between two commits, that span widens to cover them all, and at commit
time ext4_fc_snapshot_inode_data() walks the whole span through the extent
status tree -- emitting an ADD_RANGE per mapped segment and a DEL_RANGE per
hole inside it.  For scattered writes that is hundreds to thousands of ranges
even though only a handful of regions were actually modified.

The recently merged fast-commit snapshot work did not change this: it caps a
snapshot at EXT4_FC_SNAPSHOT_MAX_RANGES (2048) and fails over to a full commit
when the span exceeds it.  Measured on dev (7.1.0-rc4) with a sparse
random-write workload (1 GiB span, R disjoint 4 KiB writes per fsync, 300
fsyncs):

  R=16 regions/fsync:  ranges/commit 1095,  full-commit fallback 76%
                       (snap_fail_ranges_cap on 226 of 300 fsyncs)

Fast commit barely functions on this workload.

This series tracks the actually-modified disjoint ranges instead of one span,
and snapshots only those:

  1/2 replaces the single [min,max] range with a small, bounded set of sorted,
      disjoint ranges (up to EXT4_FC_MAX_RANGES = 16; the two closest are
      merged on overflow, so the worst case degrades to the old single-span
      behaviour).  ext4_fc_snapshot_inode_data() then walks only the tracked
      ranges.  The on-disk TLV format is unchanged.

  2/2 allocates the range array lazily: the first range stays inline, the
      array is allocated only when a second disjoint range appears, and on an
      allocation failure we fall back to the inline single range.  Per-inode
      fast-commit footprint stays ~20 bytes.

Result on the same workload (dev, patched):

  R=16:  ranges/commit 1095 -> 16,  fallback 76% -> 0.7%,
         snap_fail_ranges_cap 226 -> 0

i.e. the over-logging and the fallback storm are gone when the number of dirty
regions is within the bound.

Honest limitation: when the number of disjoint regions per fsync exceeds
EXT4_FC_MAX_RANGES (e.g. R=64 with the bound at 16), the overflow merge
re-coalesces and the win largely disappears (fallback 95% -> 87%).  A larger or
tunable bound recovers it -- see the open question below.

Testing (on dev, patched):
  - crash recovery: deterministic writes + fsync, kill -9 QEMU (power loss),
    reboot -> fast-commit replay -> verify every fsync'd block, e2fsck -fn.
    9600/9600 blocks verified, 0 mismatch, e2fsck clean.  Run with R=64, so the
    overflow-merge path is exercised.
  - ext4/generic fast-commit xfstests (ext4/044 ext4/045 generic/455 456 457
    470 482): ext4/044, ext4/045, generic/456 pass; generic/457, 470 skip
    (reflink/dax unsupported on ext4); generic/455 fails identically on
    unpatched dev (pre-existing, patch-unrelated); generic/482's single failure
    in a combined run did not reproduce (3/3 pass in isolation on the patched
    kernel).
  - on-disk format unchanged, so e2fsprogs needs no update.
  - both patches are checkpatch --strict clean.

Posted as RFC -- this sits right on top of the new snapshot code and I would
like direction before a non-RFC submission:

  - Is tracking disjoint ranges in-core (this series) the right shape, or would
    you rather the snapshot side skip holes/unmodified extents some other way?
  - EXT4_FC_MAX_RANGES is fixed at 16.  The R>bound case above wants it larger
    or tunable (sysfs/mount).  Preference?
  - The array is allocated under i_fc_lock with GFP_ATOMIC, falling back to the
    inline range on failure.  Acceptable, or move the allocation outside the
    lock?
  - Overflow merges the two ranges with the smallest gap -- acceptable
    heuristic?

Changes since v2 [1]:
  - Evaluate the range adjacency bounds in 64 bits to remove a latent
    ext4_lblk_t (u32) overflow.  The largest valid lblk is
    EXT_MAX_BLOCKS - 1 (the shrink path in ext4_setattr() passes it; the data
    path is bounded by the m_lblk >= EXT_MAX_BLOCKS check), so "end + 1" and a
    range's "start + len" can reach EXT_MAX_BLOCKS.  These were computed in u32
    in ext4_fc_range_add() (1/2) and __track_range() (2/2).  On reachable inputs
    they stop one short of wrapping, so there is no behavioural change, but the
    arithmetic is now done in u64 so it cannot wrap regardless of caller --
    matching ext4_fc_snapshot_lblk_range(), which already used 64-bit bounds.
    No other site in the series shares this pattern (audited).

Changes since v1 [2]:
  - Rebased from v6.17-rc3 onto ext4.git dev; re-implemented on top of the
    merged fast-commit snapshot model (v1 targeted the old
    ext4_fc_write_inode_data(), which no longer exists).
  - Motivation now uses the in-tree snap_fail_ranges_cap counter.
  - From and Signed-off-by use the same identity.
  - Reposted as RFC.

[1] v2: https://lore.kernel.org/linux-ext4/20260623082331epcms2p4798e9d26b06f9a005bcca7b2edf395d3@epcms2p4/
[2] v1: https://lore.kernel.org/linux-ext4/20260611044733epcms2p38013ae683a283555526f70e4eab6d2a9@epcms2p3/

Daejun Park (2):
  ext4: fast commit: track disjoint modified ranges per inode
  ext4: fast commit: allocate the range array lazily

 fs/ext4/ext4.h        |  42 ++++++--
 fs/ext4/fast_commit.c | 231 ++++++++++++++++++++++++++++++++++++------
 fs/ext4/super.c       |   1 +
 3 files changed, 234 insertions(+), 40 deletions(-)


base-commit: c143957520c6c9b5cd72e0de8b52b814f0c576fe
-- 
2.43.0

On Jul 21, 2026, at 18:59, Daejun Park <daejun7.park@samsung.com> wrote:
> 
> Fast commit is meant to make fsync cheap, but on random-write workloads it
> defeats itself.  ext4 still tracks a single coalesced [min,max] logical range
> per inode (i_fc_lblk_start/len).  When an inode is dirtied at several disjoint
> offsets between two commits, that span widens to cover them all, and at commit
> time ext4_fc_snapshot_inode_data() walks the whole span through the extent
> status tree -- emitting an ADD_RANGE per mapped segment and a DEL_RANGE per
> hole inside it.  For scattered writes that is hundreds to thousands of ranges
> even though only a handful of regions were actually modified.
> 
> The recently merged fast-commit snapshot work did not change this: it caps a
> snapshot at EXT4_FC_SNAPSHOT_MAX_RANGES (2048) and fails over to a full commit
> when the span exceeds it.  Measured on dev (7.1.0-rc4) with a sparse
> random-write workload (1 GiB span, R disjoint 4 KiB writes per fsync, 300
> fsyncs):
> 
>  R=16 regions/fsync:  ranges/commit 1095,  full-commit fallback 76%
>                       (snap_fail_ranges_cap on 226 of 300 fsyncs)
> 
> Fast commit barely functions on this workload.
> 
> This series tracks the actually-modified disjoint ranges instead of one span,
> and snapshots only those:
> 
>  1/2 replaces the single [min,max] range with a small, bounded set of sorted,
>      disjoint ranges (up to EXT4_FC_MAX_RANGES = 16; the two closest are
>      merged on overflow, so the worst case degrades to the old single-span
>      behaviour).  ext4_fc_snapshot_inode_data() then walks only the tracked
>      ranges.  The on-disk TLV format is unchanged.
> 
>  2/2 allocates the range array lazily: the first range stays inline, the
>      array is allocated only when a second disjoint range appears, and on an
>      allocation failure we fall back to the inline single range.  Per-inode
>      fast-commit footprint stays ~20 bytes.
> 
> Result on the same workload (dev, patched):
> 
>  R=16:  ranges/commit 1095 -> 16,  fallback 76% -> 0.7%,
>         snap_fail_ranges_cap 226 -> 0
> 
> i.e. the over-logging and the fallback storm are gone when the number of dirty
> regions is within the bound.
> 
> Honest limitation: when the number of disjoint regions per fsync exceeds
> EXT4_FC_MAX_RANGES (e.g. R=64 with the bound at 16), the overflow merge
> re-coalesces and the win largely disappears (fallback 95% -> 87%).  A larger or
> tunable bound recovers it -- see the open question below.
> 
> Testing (on dev, patched):
>  - crash recovery: deterministic writes + fsync, kill -9 QEMU (power loss),
>    reboot -> fast-commit replay -> verify every fsync'd block, e2fsck -fn.
>    9600/9600 blocks verified, 0 mismatch, e2fsck clean.  Run with R=64, so the
>    overflow-merge path is exercised.
>  - ext4/generic fast-commit xfstests (ext4/044 ext4/045 generic/455 456 457
>    470 482): ext4/044, ext4/045, generic/456 pass; generic/457, 470 skip
>    (reflink/dax unsupported on ext4); generic/455 fails identically on
>    unpatched dev (pre-existing, patch-unrelated); generic/482's single failure
>    in a combined run did not reproduce (3/3 pass in isolation on the patched
>    kernel).
>  - on-disk format unchanged, so e2fsprogs needs no update.
>  - both patches are checkpatch --strict clean.
> 
> Posted as RFC -- this sits right on top of the new snapshot code and I would
> like direction before a non-RFC submission:
> 
>  - Is tracking disjoint ranges in-core (this series) the right shape, or would
>    you rather the snapshot side skip holes/unmodified extents some other way?
>  - EXT4_FC_MAX_RANGES is fixed at 16.  The R>bound case above wants it larger
>    or tunable (sysfs/mount).  Preference?
>  - The array is allocated under i_fc_lock with GFP_ATOMIC, falling back to the
>    inline range on failure.  Acceptable, or move the allocation outside the
>    lock?
>  - Overflow merges the two ranges with the smallest gap -- acceptable
>    heuristic?

Have you looked at implementing a more dynamic structure like another
extent status tree, rbtree, xarray or or similar instead of having a
fixed-size array?  At *some* point the number of random blocks in the
fast commit will be so large that there is no value to tracking them
separately anymore, but the fixed 16-range limit is clearly too small
based on your testing.  It would be useful to do some testing of how
many separate ranges gets worse than the fallback.

Getting this tuning right is always difficult, and will depend on the
application workload, IO size, the storage bandwidth/latency, etc.
Having a dynamic limit is much better than a compile-time constant.

A runtime tunable is the minimum, possibly storing one range directly
in the inode, and using that as a union with the pointer to an allocated
array of ranges.  Something like the following that allows a range, with
an inode flag whether the union holds a single range or allocated array:

	union {
		struct ext4_fc_lblk_range i_fc_range;
		struct {
			unsigned int		   r_count;
			struct ext4_fc_lblk_range *r_ranges;
		} i_fc_range_array;
	};


Better would be a union with the root of a dynamic range tree.  It is
better to re-use some existing kernel functionality for this instead
of creating yet another tree implementation.

Cheers, Andreas

> Changes since v2 [1]:
>  - Evaluate the range adjacency bounds in 64 bits to remove a latent
>    ext4_lblk_t (u32) overflow.  The largest valid lblk is
>    EXT_MAX_BLOCKS - 1 (the shrink path in ext4_setattr() passes it; the data
>    path is bounded by the m_lblk >= EXT_MAX_BLOCKS check), so "end + 1" and a
>    range's "start + len" can reach EXT_MAX_BLOCKS.  These were computed in u32
>    in ext4_fc_range_add() (1/2) and __track_range() (2/2).  On reachable inputs
>    they stop one short of wrapping, so there is no behavioural change, but the
>    arithmetic is now done in u64 so it cannot wrap regardless of caller --
>    matching ext4_fc_snapshot_lblk_range(), which already used 64-bit bounds.
>    No other site in the series shares this pattern (audited).
> 
> Changes since v1 [2]:
>  - Rebased from v6.17-rc3 onto ext4.git dev; re-implemented on top of the
>    merged fast-commit snapshot model (v1 targeted the old
>    ext4_fc_write_inode_data(), which no longer exists).
>  - Motivation now uses the in-tree snap_fail_ranges_cap counter.
>  - From and Signed-off-by use the same identity.
>  - Reposted as RFC.
> 
> [1] v2: https://lore.kernel.org/linux-ext4/20260623082331epcms2p4798e9d26b06f9a005bcca7b2edf395d3@epcms2p4/
> [2] v1: https://lore.kernel.org/linux-ext4/20260611044733epcms2p38013ae683a283555526f70e4eab6d2a9@epcms2p3/
> 
> Daejun Park (2):
>  ext4: fast commit: track disjoint modified ranges per inode
>  ext4: fast commit: allocate the range array lazily
> 
> fs/ext4/ext4.h        |  42 ++++++--
> fs/ext4/fast_commit.c | 231 ++++++++++++++++++++++++++++++++++++------
> fs/ext4/super.c       |   1 +
> 3 files changed, 234 insertions(+), 40 deletions(-)
> 
> 
> base-commit: c143957520c6c9b5cd72e0de8b52b814f0c576fe
> -- 
> 2.43.0


Cheers, Andreas

On Jul 22, 2026, at 01:27, Andreas Dilger <adilger@dilger.ca> wrote:
> Have you looked at implementing a more dynamic structure like another
> extent status tree, rbtree, xarray or or similar instead of having a
> fixed-size array? [...] It would be useful to do some testing of how
> many separate ranges gets worse than the fallback.

Thanks -- I implemented the dynamic structure; it replaces the array and the
tunable entirely.  v4 tracks the modified ranges in a per-inode rbtree, which
answered both of your questions.

It can't be the shared extent status tree: I probed whether the shrinker
reclaims a mapped es entry for a modified-but-not-yet-committed range, and it
does (448 in one short run under mild pressure).  That "modified" mark is not
re-derivable, so marking the shared tree would silently lose modifications -- the
tracker has to own its structure.

So it is a private rbtree, reusing ext4's own machinery (generic rb_*, as the
extent status tree already does for per-inode ranges) -- idiomatic, no new
dependency.  The node is a dedicated 32-byte struct with its own slab cache, not
extent_status: extent_status carries an 8-byte es_pblk that fast commit never
uses, and its cache is SLAB_RECLAIM_ACCOUNT, which would misaccount these
non-reclaimable nodes.  The rb_root is inline in the inode, so only per-range
nodes are allocated.  (I also built and compared a maple-tree version:
behaviour-identical, within ~15% on memory, denser only at the range counts this
series already caps -- so the rbtree, with no dependency, is the choice.)

On "how many is too many": the cap is EXT4_FC_SNAPSHOT_MAX_RANGES (2048), the
limit the snapshot already enforces on emitted ranges -- reused directly, not a
new constant.  Each disjoint tracked range snapshots to at least one ADD_RANGE,
so a tracked set larger than that can never fit under the snapshot cap: the fast
commit would overrun it and fall back regardless.  That count is exactly where
tracking stops paying off, so at the cap the transaction falls back to a full
commit.  (Verified: R=4096 disjoint writes/fsync falls back there; R=1024 tracks
precisely.)

Thanks,
Daejun