[PATCH v4 0/3] exfat: speed up file creation in large directories

Yang Wen posted 3 patches 1 week, 4 days ago
There is a newer version of this series
fs/exfat/dir.c      | 288 +++++++++++++++++++++++++++++++++++++++++++-
fs/exfat/exfat_fs.h |  26 +++-
fs/exfat/inode.c    |   1 +
fs/exfat/namei.c    |  78 +++++++++++-
fs/exfat/super.c    |   9 ++
5 files changed, 395 insertions(+), 7 deletions(-)
[PATCH v4 0/3] exfat: speed up file creation in large directories
Posted by Yang Wen 1 week, 4 days ago
Negative name lookups and empty-entry allocation can repeatedly scan a
directory from the beginning. Bulk creation in a large directory therefore
approaches O(N^2).

This series separates the optimization into three independently reviewable
steps. Patch 1 adds the Bloom filter used to reject definite name misses.
Patch 2 retains and correctly invalidates the next-empty-entry hint.
Patch 3 adds the LRU and shrinker used to reclaim filters under
memory pressure.

Test environment:
  QEMU TCG multi-thread, 4 vCPUs, 6 GiB RAM
  4 GiB exFAT image, 32 KiB clusters

The measured results were:

                         Before          After
  real                   589.48 s        15.94 s
  user                     4.72 s         4.20 s
  sys                    584.63 s        11.71 s

This is a 37.0-fold speedup.

Changes in v4:
- Publish the next-empty-entry hint only after the directory entry set is
  successfully committed, so post-allocation failures cannot skip an unused
  slot.
- Invalidate the destination name filter when rename or move fails because
  the new entry may already exist on disk.

Changes in v3:
- Split the change into Bloom filter, empty-entry hint, and
  shrinker patches.
- Accept filenames containing exactly 255 UTF-16 code units while building
  the Bloom filter.
- Invalidate the empty-entry hint in every path that can free directory
  entries, preventing stale hints from skipping earlier holes.

Changes in v2:
- Move exfat_name_filter_free() to exfat_evict_inode() because
  ->free_inode() may run from an RCU callback in softirq context.

Yang Wen (3):
  exfat: add a Bloom filter for negative name lookups
  exfat: retain the next empty directory entry hint
  exfat: reclaim name filters under memory pressure

 fs/exfat/dir.c      | 288 +++++++++++++++++++++++++++++++++++++++++++-
 fs/exfat/exfat_fs.h |  26 +++-
 fs/exfat/inode.c    |   1 +
 fs/exfat/namei.c    |  78 +++++++++++-
 fs/exfat/super.c    |   9 ++
 5 files changed, 395 insertions(+), 7 deletions(-)

-- 
2.34.1