[PATCH v1 00/12] f2fs: introduce metadata cache

Chao Yu posted 12 patches 1 month, 1 week ago
There is a newer version of this series
fs/f2fs/Makefile            |   2 +-
fs/f2fs/acl.c               |  26 +-
fs/f2fs/acl.h               |   8 +-
fs/f2fs/cache.c             | 690 +++++++++++++++++++++++++
fs/f2fs/cache.h             | 224 ++++++++
fs/f2fs/checkpoint.c        | 404 +++++++--------
fs/f2fs/compress.c          | 171 +++----
fs/f2fs/data.c              | 527 +++++++++++++------
fs/f2fs/debug.c             |  70 ++-
fs/f2fs/dir.c               | 168 +++---
fs/f2fs/extent_cache.c      |  14 +-
fs/f2fs/f2fs.h              | 333 ++++++------
fs/f2fs/file.c              |  78 ++-
fs/f2fs/gc.c                | 178 ++++---
fs/f2fs/inline.c            | 284 ++++++-----
fs/f2fs/inode.c             | 205 +++-----
fs/f2fs/iostat.h            |  11 +
fs/f2fs/namei.c             | 118 ++---
fs/f2fs/node.c              | 994 +++++++++++++++++-------------------
fs/f2fs/node.h              | 109 ++--
fs/f2fs/recovery.c          | 253 ++++-----
fs/f2fs/segment.c           | 263 +++++-----
fs/f2fs/segment.h           |  39 +-
fs/f2fs/shrinker.c          |  13 +
fs/f2fs/super.c             | 143 +++---
fs/f2fs/xattr.c             | 123 +++--
fs/f2fs/xattr.h             |  12 +-
include/linux/f2fs_fs.h     |   3 -
include/trace/events/f2fs.h |  71 +++
29 files changed, 3331 insertions(+), 2203 deletions(-)
create mode 100644 fs/f2fs/cache.c
create mode 100644 fs/f2fs/cache.h
[PATCH v1 00/12] f2fs: introduce metadata cache
Posted by Chao Yu 1 month, 1 week ago
This patchset introduces a self-managed metadata block cache in f2fs,
decoupling meta blocks, node blocks, and compressed data blocks from
the Linux VFS page cache and fake internal inodes.

=== 1. Background & Motivation ===

Currently, F2FS uses fake VFS inodes (meta_inode, node_inode, and
compress_inode) to manage internal block caching through the VFS page
cache. Because of this implementation, the f2fs block size was
historically coupled to the kernel page size.

We now want to unbind block size from page size to support configurations
where block size <= PAGE_SIZE (e.g., mounting a 4KB-block F2FS image on a
16KB or 64KB page system).

One possible approach is to continue using the VFS page cache to store
metadata blocks. However, doing so introduces three major architectural
issues (illustrated by a 4KB block on a 16KB page system):

1. Memory Overhead:
   Metadata access patterns are typically random and sparse. Caching a
   single 4KB metadata block inside a page cache folio forces the
   allocation of an entire 16KB folio, resulting in 4x memory waste.

2. Folio and Sub-block Conversion Complexity:
   Using larger folios requires tracking individual sub-block dirty/uptodate
   states within each folio and performing index-to-offset conversions across
   function boundaries. Because core metadata structures (e.g., f2fs_checkpoint,
   f2fs_sit_block, f2fs_nat_block, f2fs_summary_block, f2fs_node) are accessed
   extensively throughout the filesystem, this sub-block management and offset
   calculation complexity would spread across the entire F2FS codebase.

3. Lock Contention:
   Multiple independent node blocks (e.g., dnode blocks belonging to
   different files) can reside within the same folio. Concurrent fsync()
   calls on unrelated files would contend on the same folio_lock(),
   serializing metadata updates and degrading multi-threaded performance.

Decoupling metadata caching from PAGE_SIZE by allocating exact
block-sized cache entries is the critical first step toward supporting
4KB-block F2FS images on 16KB/64KB page systems.

=== 2. Metadata Cache Architecture & Design ===

This patchset introduces a dedicated, block-size-aligned caching

3. Lock Contention:
   Multiple independent node blocks (e.g., dnode blocks belonging to
   different files) can reside within the same folio. Concurrent fsync()
   calls on unrelated files would contend on the same folio_lock(),
   serializing metadata updates and degrading multi-threaded performance.

3. Lock Contention:
   Multiple independent node blocks (e.g., dnode blocks belonging to
   different files) can reside within the same folio. Concurrent fsync()
   calls on unrelated files would contend on the same folio_lock(),
   serializing metadata updates and degrading multi-threaded performance.

Decoupling metadata caching from PAGE_SIZE by allocating exact
block-sized cache entries is the critical first step toward supporting
4KB-block F2FS images on 16KB/64KB page systems.

=== 2. Metadata Cache Architecture & Design ===

This patchset introduces a dedicated, block-size-aligned caching
infrastructure with the following key components:

- Block-Size Aligned Allocation:
  Allocates memory buffers matching exactly the filesystem block size
  (4KB or 16KB) via kzalloc(), fully independent of the host
  architecture's PAGE_SIZE.

- Radix Tree Indexing with Fast Tag Scanning:
  Each cache instance (META_CACHE, NODE_CACHE, COMPRESS_CACHE) indexes
  cached blocks via a radix tree (keyed by Physical Block Address for meta/
  compress cache, and Node ID for node cache). Radix tree tags
  (F2FS_CACHE_TAG_DIRTY, F2FS_CACHE_TAG_WRITEBACK) provide O(1) batch gang
  lookups for flushing and writeback without dual-list shuffling.

- Lightweight Bit-Locking:
  Individual entries use atomic bit locks (F2FS_BLOCK_LOCKED via
  wait_on_bit_lock() / clear_and_wake_up_bit()) rather than heavyweight
  embedded mutexes/semaphores, minimizing memory footprint per entry.

- Direct BIO Read/Write & BIO Merging:
  Decouples metadata/node I/O from VFS address spaces by submitting direct
  BIOs (f2fs_submit_cache_read / f2fs_submit_cache_write) with chained
  adjacent vector merging (entry->next_entry) and dedicated completion
  handlers.

- Memory Reclamation Shrinker:
  Integrates with the kernel shrinker subsystem via a 3-phase isolation
  algorithm (isolate unreferenced clean entries -> truncate from radix tree
  under lock -> splice un-reclaimed entries back to LRU) to safely reclaim
  clean cached blocks under system memory pressure.

- Background Writeback Kthread & Checkpoint Integration:
  Provides a dedicated background kthread (f2fs_writeback-X:Y) for periodic
  dirty cache flushing, combined with synchronous flushing during checkpoint
  commit.

- Fault Injection, Tracepoints & Debugfs Observability:
  Integrates FAULT_KALLOC fault injection, tracepoints for cache state
  transitions and batch writeback, and per-cache memory breakdowns in debugfs.

=== 3. Patchset Organization ===

- Patch 01: Implement the core metadata cache infrastructure & direct BIO I/O.
- Patch 02: Initialize and teardown META_CACHE in sb_info.
- Patch 03: Integrate metadata cache into the memory shrinker subsystem.
- Patch 04: Introduce the background writeback kernel thread.
- Patch 05: Migrate metadata block caching (SIT, NAT, SSA, CP, recovery, GC)
            from meta_inode to META_CACHE.
- Patch 06: Initialize and teardown NODE_CACHE in sb_info.
- Patch 07: Migrate node and inode block caching from node_inode to NODE_CACHE.
- Patch 08: Initialize and teardown COMPRESS_CACHE in sb_info.
- Patch 09: Migrate compressed cluster caching from compress_inode to COMPRESS_CACHE.
- Patch 10: Add fault injection support for cache allocation paths.
- Patch 11: Introduce ftrace tracepoints for cache dirty and writeback events.
- Patch 12: Expose per-cache memory usage in debugfs.

Chao Yu (12):
  f2fs: cache: implement metadata cache
  f2fs: cache: initialize meta cache
  f2fs: cache: introduce shrinker
  f2fs: cache: introduce writeback thread
  f2fs: cache: use meta cache
  f2fs: cache: initialize node cache
  f2fs: cache: use node cache
  f2fs: cache: initialize compress cache
  f2fs: cache: use compress cache
  f2fs: cache: support fault injection
  f2fs: cache: introduce tracepoints
  f2fs: cache: show per-cache usage in debugfs

 fs/f2fs/Makefile            |   2 +-
 fs/f2fs/acl.c               |  26 +-
 fs/f2fs/acl.h               |   8 +-
 fs/f2fs/cache.c             | 690 +++++++++++++++++++++++++
 fs/f2fs/cache.h             | 224 ++++++++
 fs/f2fs/checkpoint.c        | 404 +++++++--------
 fs/f2fs/compress.c          | 171 +++----
 fs/f2fs/data.c              | 527 +++++++++++++------
 fs/f2fs/debug.c             |  70 ++-
 fs/f2fs/dir.c               | 168 +++---
 fs/f2fs/extent_cache.c      |  14 +-
 fs/f2fs/f2fs.h              | 333 ++++++------
 fs/f2fs/file.c              |  78 ++-
 fs/f2fs/gc.c                | 178 ++++---
 fs/f2fs/inline.c            | 284 ++++++-----
 fs/f2fs/inode.c             | 205 +++-----
 fs/f2fs/iostat.h            |  11 +
 fs/f2fs/namei.c             | 118 ++---
 fs/f2fs/node.c              | 994 +++++++++++++++++-------------------
 fs/f2fs/node.h              | 109 ++--
 fs/f2fs/recovery.c          | 253 ++++-----
 fs/f2fs/segment.c           | 263 +++++-----
 fs/f2fs/segment.h           |  39 +-
 fs/f2fs/shrinker.c          |  13 +
 fs/f2fs/super.c             | 143 +++---
 fs/f2fs/xattr.c             | 123 +++--
 fs/f2fs/xattr.h             |  12 +-
 include/linux/f2fs_fs.h     |   3 -
 include/trace/events/f2fs.h |  71 +++
 29 files changed, 3331 insertions(+), 2203 deletions(-)
 create mode 100644 fs/f2fs/cache.c
 create mode 100644 fs/f2fs/cache.h

-- 
2.49.0
Re: [PATCH v1 00/12] f2fs: introduce metadata cache
Posted by Wenjie Qi 1 month ago
Hi Chao,

At Xiaomi, we have an out-of-tree F2FS node-cache prototype for mobile
workloads that retains clean node contents in compressed memory after
page-cache reclaim.  This is different from this series:
F2FS_NODE_CACHE is the primary cache and can hold dirty authoritative
data, while our compressed contents are clean and disposable.

In our mobile workloads, clean F2FS node pages can represent roughly
100 MB of uncompressed working-set memory.  We have observed an LZ4
payload-to-original ratio of about 0.1 for these pages, excluding
zsmalloc fragmentation, entry metadata, and workspaces.  Node reads are
also on latency-sensitive paths, while memory pressure can repeatedly
reclaim and later reread the same node pages from storage.

These are observations from our out-of-tree mobile workloads, not
benchmark results from this series or current upstream Linux.

If the primary metadata-cache model moves forward, the direction we
would like to explore is one F2FS_NODE_CACHE slot per NID with mutually
exclusive clean representations:

    UNCOMPRESSED_CLEAN <-> COMPRESSED_CLEAN

Active, dirty, and writeback entries would stay uncompressed, and a
slot would not retain stable copies in both forms.  Compression would be
prepared outside the shrinker and direct-reclaim paths.  Shrinker reclaim
would continue to discard clean entries without compression, allocation,
or I/O.

If an optional compressed representation for clean, idle F2FS_NODE_CACHE
entries fits your intended metadata-cache roadmap, would it make sense to
discuss the clean-entry state and lifetime contract first, and prototype
it on top of v2 once the base cache is ready?
Re: [PATCH v1 00/12] f2fs: introduce metadata cache
Posted by Chao Yu 1 month ago
Wenjie,

Thanks for sharing the info, I guess we can discuss after we upstream
this serial. :)

On 8/24/26 18:07, Wenjie Qi wrote:
> Hi Chao,
> 
> At Xiaomi, we have an out-of-tree F2FS node-cache prototype for mobile
> workloads that retains clean node contents in compressed memory after
> page-cache reclaim.  This is different from this series:
> F2FS_NODE_CACHE is the primary cache and can hold dirty authoritative
> data, while our compressed contents are clean and disposable.
> 
> In our mobile workloads, clean F2FS node pages can represent roughly
> 100 MB of uncompressed working-set memory.  We have observed an LZ4
> payload-to-original ratio of about 0.1 for these pages, excluding
> zsmalloc fragmentation, entry metadata, and workspaces.  Node reads are
> also on latency-sensitive paths, while memory pressure can repeatedly
> reclaim and later reread the same node pages from storage.
> 
> These are observations from our out-of-tree mobile workloads, not
> benchmark results from this series or current upstream Linux.
> 
> If the primary metadata-cache model moves forward, the direction we
> would like to explore is one F2FS_NODE_CACHE slot per NID with mutually
> exclusive clean representations:
> 
>      UNCOMPRESSED_CLEAN <-> COMPRESSED_CLEAN
> 
> Active, dirty, and writeback entries would stay uncompressed, and a
> slot would not retain stable copies in both forms.  Compression would be
> prepared outside the shrinker and direct-reclaim paths.  Shrinker reclaim
> would continue to discard clean entries without compression, allocation,
> or I/O.
> 
> If an optional compressed representation for clean, idle F2FS_NODE_CACHE
> entries fits your intended metadata-cache roadmap, would it make sense to
> discuss the clean-entry state and lifetime contract first, and prototype
> it on top of v2 once the base cache is ready?