fs/f2fs/data.c | 93 +++++++++++++++++++++++++- fs/f2fs/f2fs.h | 46 ++++++++++++- fs/f2fs/file.c | 85 +++++++++++++++++++++++- fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ fs/f2fs/inode.c | 6 ++ fs/f2fs/namei.c | 3 + fs/f2fs/node.c | 6 +- fs/f2fs/recovery.c | 9 ++- fs/f2fs/super.c | 25 +++++++ fs/f2fs/verity.c | 4 ++ 10 files changed, 409 insertions(+), 27 deletions(-)
The inode in F2FS occupies an entire 4k block. For many small files, this means
they consume much more space than their actual size. Therefore, there is
significant potential to better utilize the inode block space.
Currently, F2FS has two features to make use of the inode block space: inline
data and inline xattr.
Inline data stores file which size is smaller then 3.5k in inode block. However,
for slightly larger small files, there still have much waste.
For example, a 5k file requires 3 blocks, totaling 12k of space, which is
more than twice the size of the file itself!
Additionally, the end of a file often does not occupy an entire block. If we can
store the end of the file data within the inode block, we can save an entire
block for the file. This is particularly important for small files.
In fact, the current inline data is a special case of inline tail, and
inline tail is an extension of inline data.
To make it simple, inline tail only on small files(<64k). And for larger files,
inline tails don't provide any significant benefits.
The layout of an inline tail inode block is following:
| inode block | 4096 | inline tail enable |
| --------------- | ---- | --------------------------|
| inode info | 360 | |
| --------------- | ---- | --------------------------|
| | | extra info | 0~36 |
| | | **compact_addr[16] | 64 |
| addr table[923] | 3692 | reserved | 4 |
| | | **tail data | |
| | | inline_xattr | 200 |
| --------------- | ---- | --------------------------|
| nid table[5] | 20 |
| node footer | 24 |
F2fs-tools to support inline tail:
https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com
I tested inline tail by copying the source code of Linux 6.9.7. The storage
space was reduced by approximately 8%. Additionally, due to the reduced IO, the
copy time also reduced by around 10%.
This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs
-g quick' both with and without the patch; no regressions were observed.
The test result is:
f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds
Failures: generic/050 generic/064 generic/250 generic/252 generic/563
generic/735
Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s
---
v2:
- fix ARCH=arc build warning
---
Wu Bo (13):
f2fs: add inline tail mount option
f2fs: add inline tail disk layout definition
f2fs: implement inline tail write & truncate
f2fs: implement inline tail read & fiemap
f2fs: set inline tail flag when create inode
f2fs: fix address info has been truncated
f2fs: support seek for inline tail
f2fs: convert inline tail when inode expand
f2fs: fix data loss during inline tail writing
f2fs: avoid inlining quota files
f2fs: fix inline tail data lost
f2fs: convert inline tails to avoid potential issues
f2fs: implement inline tail forward recovery
fs/f2fs/data.c | 93 +++++++++++++++++++++++++-
fs/f2fs/f2fs.h | 46 ++++++++++++-
fs/f2fs/file.c | 85 +++++++++++++++++++++++-
fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------
fs/f2fs/inode.c | 6 ++
fs/f2fs/namei.c | 3 +
fs/f2fs/node.c | 6 +-
fs/f2fs/recovery.c | 9 ++-
fs/f2fs/super.c | 25 +++++++
fs/f2fs/verity.c | 4 ++
10 files changed, 409 insertions(+), 27 deletions(-)
base-commit: 67784a74e258a467225f0e68335df77acd67b7ab
--
2.35.3
On 2024/9/11 11:57, Wu Bo wrote: > The inode in F2FS occupies an entire 4k block. For many small files, this means > they consume much more space than their actual size. Therefore, there is > significant potential to better utilize the inode block space. > > Currently, F2FS has two features to make use of the inode block space: inline > data and inline xattr. > > Inline data stores file which size is smaller then 3.5k in inode block. However, > for slightly larger small files, there still have much waste. > For example, a 5k file requires 3 blocks, totaling 12k of space, which is > more than twice the size of the file itself! > > Additionally, the end of a file often does not occupy an entire block. If we can > store the end of the file data within the inode block, we can save an entire > block for the file. This is particularly important for small files. > > In fact, the current inline data is a special case of inline tail, and > inline tail is an extension of inline data. > > To make it simple, inline tail only on small files(<64k). And for larger files, > inline tails don't provide any significant benefits. > > The layout of an inline tail inode block is following: > > | inode block | 4096 | inline tail enable | > | --------------- | ---- | --------------------------| > | inode info | 360 | | > | --------------- | ---- | --------------------------| > | | | extra info | 0~36 | > | | | **compact_addr[16] | 64 | > | addr table[923] | 3692 | reserved | 4 | > | | | **tail data | | > | | | inline_xattr | 200 | > | --------------- | ---- | --------------------------| > | nid table[5] | 20 | > | node footer | 24 | > > F2fs-tools to support inline tail: > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com > > I tested inline tail by copying the source code of Linux 6.9.7. The storage > space was reduced by approximately 8%. Additionally, due to the reduced IO, the > copy time also reduced by around 10%. > > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs > -g quick' both with and without the patch; no regressions were observed. > The test result is: > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds > Failures: generic/050 generic/064 generic/250 generic/252 generic/563 > generic/735 > Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s MKFS_OPTIONS -- -O extra_attr,encrypt,inode_checksum,flexible_inline_xattr,inode_crtime,verity,compression -f /dev/vdc MOUNT_OPTIONS -- -o acl,user_xattr -o discard,inline_tail /dev/vdc /mnt/scratch_f2fs Before: Failures: generic/042 generic/050 generic/250 generic/252 generic/270 generic/389 generic/563 generic/700 generic/735 Failed 9 of 746 tests After: Failures: generic/042 generic/050 generic/125 generic/250 generic/252 generic/270 generic/389 generic/418 generic/551 generic/563 generic/700 generic/735 Failed 12 of 746 tests Failures: f2fs/004 Can you please check failed testcases? Thanks, > > --- > v2: > - fix ARCH=arc build warning > > --- > Wu Bo (13): > f2fs: add inline tail mount option > f2fs: add inline tail disk layout definition > f2fs: implement inline tail write & truncate > f2fs: implement inline tail read & fiemap > f2fs: set inline tail flag when create inode > f2fs: fix address info has been truncated > f2fs: support seek for inline tail > f2fs: convert inline tail when inode expand > f2fs: fix data loss during inline tail writing > f2fs: avoid inlining quota files > f2fs: fix inline tail data lost > f2fs: convert inline tails to avoid potential issues > f2fs: implement inline tail forward recovery > > fs/f2fs/data.c | 93 +++++++++++++++++++++++++- > fs/f2fs/f2fs.h | 46 ++++++++++++- > fs/f2fs/file.c | 85 +++++++++++++++++++++++- > fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ > fs/f2fs/inode.c | 6 ++ > fs/f2fs/namei.c | 3 + > fs/f2fs/node.c | 6 +- > fs/f2fs/recovery.c | 9 ++- > fs/f2fs/super.c | 25 +++++++ > fs/f2fs/verity.c | 4 ++ > 10 files changed, 409 insertions(+), 27 deletions(-) > > > base-commit: 67784a74e258a467225f0e68335df77acd67b7ab
On Thu, Sep 12, 2024 at 03:14:24PM +0800, Chao Yu via Linux-f2fs-devel wrote: > On 2024/9/11 11:57, Wu Bo wrote: > > The inode in F2FS occupies an entire 4k block. For many small files, this means > > they consume much more space than their actual size. Therefore, there is > > significant potential to better utilize the inode block space. > > > > Currently, F2FS has two features to make use of the inode block space: inline > > data and inline xattr. > > > > Inline data stores file which size is smaller then 3.5k in inode block. However, > > for slightly larger small files, there still have much waste. > > For example, a 5k file requires 3 blocks, totaling 12k of space, which is > > more than twice the size of the file itself! > > > > Additionally, the end of a file often does not occupy an entire block. If we can > > store the end of the file data within the inode block, we can save an entire > > block for the file. This is particularly important for small files. > > > > In fact, the current inline data is a special case of inline tail, and > > inline tail is an extension of inline data. > > > > To make it simple, inline tail only on small files(<64k). And for larger files, > > inline tails don't provide any significant benefits. > > > > The layout of an inline tail inode block is following: > > > > | inode block | 4096 | inline tail enable | > > | --------------- | ---- | --------------------------| > > | inode info | 360 | | > > | --------------- | ---- | --------------------------| > > | | | extra info | 0~36 | > > | | | **compact_addr[16] | 64 | > > | addr table[923] | 3692 | reserved | 4 | > > | | | **tail data | | > > | | | inline_xattr | 200 | > > | --------------- | ---- | --------------------------| > > | nid table[5] | 20 | > > | node footer | 24 | > > > > F2fs-tools to support inline tail: > > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com > > > > I tested inline tail by copying the source code of Linux 6.9.7. The storage > > space was reduced by approximately 8%. Additionally, due to the reduced IO, the > > copy time also reduced by around 10%. > > > > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs > > -g quick' both with and without the patch; no regressions were observed. > > The test result is: > > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds > > Failures: generic/050 generic/064 generic/250 generic/252 generic/563 > > generic/735 > > Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s > > MKFS_OPTIONS -- -O extra_attr,encrypt,inode_checksum,flexible_inline_xattr,inode_crtime,verity,compression -f /dev/vdc > MOUNT_OPTIONS -- -o acl,user_xattr -o discard,inline_tail /dev/vdc /mnt/scratch_f2fs Hi Chao, I used the default cfg to run xfstest-bld and didn't encounter the failed failures. This suggests the issue might be related to these additional options. However, I'm not sure how to include these options when running xfstest-bld. Could you let me know how to add them? Thanks > > Before: > Failures: generic/042 generic/050 generic/250 generic/252 generic/270 generic/389 generic/563 generic/700 generic/735 > Failed 9 of 746 tests > > After: > Failures: generic/042 generic/050 generic/125 generic/250 generic/252 generic/270 generic/389 generic/418 generic/551 generic/563 generic/700 generic/735 > Failed 12 of 746 tests > > Failures: f2fs/004 > > Can you please check failed testcases? > > Thanks, > > > > > --- > > v2: > > - fix ARCH=arc build warning > > > > --- > > Wu Bo (13): > > f2fs: add inline tail mount option > > f2fs: add inline tail disk layout definition > > f2fs: implement inline tail write & truncate > > f2fs: implement inline tail read & fiemap > > f2fs: set inline tail flag when create inode > > f2fs: fix address info has been truncated > > f2fs: support seek for inline tail > > f2fs: convert inline tail when inode expand > > f2fs: fix data loss during inline tail writing > > f2fs: avoid inlining quota files > > f2fs: fix inline tail data lost > > f2fs: convert inline tails to avoid potential issues > > f2fs: implement inline tail forward recovery > > > > fs/f2fs/data.c | 93 +++++++++++++++++++++++++- > > fs/f2fs/f2fs.h | 46 ++++++++++++- > > fs/f2fs/file.c | 85 +++++++++++++++++++++++- > > fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ > > fs/f2fs/inode.c | 6 ++ > > fs/f2fs/namei.c | 3 + > > fs/f2fs/node.c | 6 +- > > fs/f2fs/recovery.c | 9 ++- > > fs/f2fs/super.c | 25 +++++++ > > fs/f2fs/verity.c | 4 ++ > > 10 files changed, 409 insertions(+), 27 deletions(-) > > > > > > base-commit: 67784a74e258a467225f0e68335df77acd67b7ab > > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
On Fri, Sep 13, 2024 at 08:41:12PM -0600, Wu Bo via Linux-f2fs-devel wrote: > On Thu, Sep 12, 2024 at 03:14:24PM +0800, Chao Yu via Linux-f2fs-devel wrote: > > On 2024/9/11 11:57, Wu Bo wrote: > > > The inode in F2FS occupies an entire 4k block. For many small files, this means > > > they consume much more space than their actual size. Therefore, there is > > > significant potential to better utilize the inode block space. > > > > > > Currently, F2FS has two features to make use of the inode block space: inline > > > data and inline xattr. > > > > > > Inline data stores file which size is smaller then 3.5k in inode block. However, > > > for slightly larger small files, there still have much waste. > > > For example, a 5k file requires 3 blocks, totaling 12k of space, which is > > > more than twice the size of the file itself! > > > > > > Additionally, the end of a file often does not occupy an entire block. If we can > > > store the end of the file data within the inode block, we can save an entire > > > block for the file. This is particularly important for small files. > > > > > > In fact, the current inline data is a special case of inline tail, and > > > inline tail is an extension of inline data. > > > > > > To make it simple, inline tail only on small files(<64k). And for larger files, > > > inline tails don't provide any significant benefits. > > > > > > The layout of an inline tail inode block is following: > > > > > > | inode block | 4096 | inline tail enable | > > > | --------------- | ---- | --------------------------| > > > | inode info | 360 | | > > > | --------------- | ---- | --------------------------| > > > | | | extra info | 0~36 | > > > | | | **compact_addr[16] | 64 | > > > | addr table[923] | 3692 | reserved | 4 | > > > | | | **tail data | | > > > | | | inline_xattr | 200 | > > > | --------------- | ---- | --------------------------| > > > | nid table[5] | 20 | > > > | node footer | 24 | > > > > > > F2fs-tools to support inline tail: > > > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com > > > > > > I tested inline tail by copying the source code of Linux 6.9.7. The storage > > > space was reduced by approximately 8%. Additionally, due to the reduced IO, the > > > copy time also reduced by around 10%. > > > > > > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs > > > -g quick' both with and without the patch; no regressions were observed. > > > The test result is: > > > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds > > > Failures: generic/050 generic/064 generic/250 generic/252 generic/563 > > > generic/735 > > > Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s > > > > MKFS_OPTIONS -- -O extra_attr,encrypt,inode_checksum,flexible_inline_xattr,inode_crtime,verity,compression -f /dev/vdc > > MOUNT_OPTIONS -- -o acl,user_xattr -o discard,inline_tail /dev/vdc /mnt/scratch_f2fs > > Hi Chao, > > I used the default cfg to run xfstest-bld and didn't encounter the failed > failures. This suggests the issue might be related to these additional options. > However, I'm not sure how to include these options when running xfstest-bld. > Could you let me know how to add them? > > Thanks I found how to pass these options: 1. Add custom config file ``` # cat test-appliance/files/root/fs/f2fs/cfg/custom SIZE=small export MKFS_OPTIONS="-O extra_attr,encrypt,inode_checksum,flexible_inline_xattr,inode_crtime,verity,compression -f" export F2FS_MOUNT_OPTIONS="discard,inline_tail" TESTNAME="f2fs custom" ``` 2. Then run command as following: ``` kvm-xfstests -c f2fs/custom "generic/418" ``` However, I am only able to reproduce generic/418 and f2fs/004. I will look into these two cases first. > > > > > Before: > > Failures: generic/042 generic/050 generic/250 generic/252 generic/270 generic/389 generic/563 generic/700 generic/735 > > Failed 9 of 746 tests > > > > After: > > Failures: generic/042 generic/050 generic/125 generic/250 generic/252 generic/270 generic/389 generic/418 generic/551 generic/563 generic/700 generic/735 > > Failed 12 of 746 tests > > > > Failures: f2fs/004 > > > > Can you please check failed testcases? > > > > Thanks, > > > > > > > > --- > > > v2: > > > - fix ARCH=arc build warning > > > > > > --- > > > Wu Bo (13): > > > f2fs: add inline tail mount option > > > f2fs: add inline tail disk layout definition > > > f2fs: implement inline tail write & truncate > > > f2fs: implement inline tail read & fiemap > > > f2fs: set inline tail flag when create inode > > > f2fs: fix address info has been truncated > > > f2fs: support seek for inline tail > > > f2fs: convert inline tail when inode expand > > > f2fs: fix data loss during inline tail writing > > > f2fs: avoid inlining quota files > > > f2fs: fix inline tail data lost > > > f2fs: convert inline tails to avoid potential issues > > > f2fs: implement inline tail forward recovery > > > > > > fs/f2fs/data.c | 93 +++++++++++++++++++++++++- > > > fs/f2fs/f2fs.h | 46 ++++++++++++- > > > fs/f2fs/file.c | 85 +++++++++++++++++++++++- > > > fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ > > > fs/f2fs/inode.c | 6 ++ > > > fs/f2fs/namei.c | 3 + > > > fs/f2fs/node.c | 6 +- > > > fs/f2fs/recovery.c | 9 ++- > > > fs/f2fs/super.c | 25 +++++++ > > > fs/f2fs/verity.c | 4 ++ > > > 10 files changed, 409 insertions(+), 27 deletions(-) > > > > > > > > > base-commit: 67784a74e258a467225f0e68335df77acd67b7ab > > > > > > > > _______________________________________________ > > Linux-f2fs-devel mailing list > > Linux-f2fs-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
© 2016 - 2024 Red Hat, Inc.