[v2] More unit test for mballoc

[PATCH v2 0/5] More unit test for mballoc

Posted by Kemeng Shi 1 year, 11 months ago

This series covers more function to mark on-disk bitmap. Besides, some
code which is relevant to buddy cache is also tested.
Before more work is done, I want to be sure I'm not on a wrong
direction!

v1->v2:
-Fix unused variable warning which is reported at
https://lore.kernel.org/lkml/202311260042.kMxL6DnL-lkp@intel.com/T/

Kunit test result is as following:
# ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig --raw_output
[18:39:42] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[18:39:45] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=88
[18:39:57] Starting KUnit Kernel (1/1)...
KTAP version 1
1..2
    KTAP version 1
    # Subtest: ext4_mballoc_test
    # module: ext4
    1..6
        KTAP version 1
        # Subtest: test_new_blocks_simple
        ok 1 block_bits=10 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 2 block_bits=12 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 3 block_bits=16 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
    # test_new_blocks_simple: pass:3 fail:0 skip:0 total:3
    ok 1 test_new_blocks_simple
        KTAP version 1
        # Subtest: test_free_blocks_simple
        ok 1 block_bits=10 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 2 block_bits=12 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 3 block_bits=16 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
    # test_free_blocks_simple: pass:3 fail:0 skip:0 total:3
    ok 2 test_free_blocks_simple
        KTAP version 1
        # Subtest: test_mb_generate_buddy
        ok 1 block_bits=10 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 2 block_bits=12 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 3 block_bits=16 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
    # test_mb_generate_buddy: pass:3 fail:0 skip:0 total:3
    ok 3 test_mb_generate_buddy
        KTAP version 1
        # Subtest: test_mb_mark_used
        ok 1 block_bits=10 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 2 block_bits=12 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 3 block_bits=16 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
 # SKIP blocksize exceeds pagesize
    # test_mb_mark_used: pass:2 fail:0 skip:1 total:3
    ok 4 test_mb_mark_used
        KTAP version 1
        # Subtest: test_mb_free_blocks
        ok 1 block_bits=10 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 2 block_bits=12 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 3 block_bits=16 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
 # SKIP blocksize exceeds pagesize
    # test_mb_free_blocks: pass:2 fail:0 skip:1 total:3
    ok 5 test_mb_free_blocks
        KTAP version 1
        # Subtest: test_mark_diskspace_used
        ok 1 block_bits=10 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 2 block_bits=12 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
        ok 3 block_bits=16 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
    # test_mark_diskspace_used: pass:3 fail:0 skip:0 total:3
    ok 6 test_mark_diskspace_used
# ext4_mballoc_test: pass:6 fail:0 skip:0 total:6
# Totals: pass:16 fail:0 skip:2 total:18
ok 1 ext4_mballoc_test
    KTAP version 1
    # Subtest: ext4_inode_test
    # module: ext4_inode_test
    1..1
        KTAP version 1
        # Subtest: inode_test_xtimestamp_decoding
        ok 1 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits
        ok 2 1969-12-31 Upper bound of 32bit < 0 timestamp, no extra bits
        ok 3 1970-01-01 Lower bound of 32bit >=0 timestamp, no extra bits
        ok 4 2038-01-19 Upper bound of 32bit >=0 timestamp, no extra bits
        ok 5 2038-01-19 Lower bound of 32bit <0 timestamp, lo extra sec bit on
        ok 6 2106-02-07 Upper bound of 32bit <0 timestamp, lo extra sec bit on
        ok 7 2106-02-07 Lower bound of 32bit >=0 timestamp, lo extra sec bit on
        ok 8 2174-02-25 Upper bound of 32bit >=0 timestamp, lo extra sec bit on
        ok 9 2174-02-25 Lower bound of 32bit <0 timestamp, hi extra sec bit on
        ok 10 2242-03-16 Upper bound of 32bit <0 timestamp, hi extra sec bit on
        ok 11 2242-03-16 Lower bound of 32bit >=0 timestamp, hi extra sec bit on
        ok 12 2310-04-04 Upper bound of 32bit >=0 timestamp, hi extra sec bit on
        ok 13 2310-04-04 Upper bound of 32bit>=0 timestamp, hi extra sec bit 1. 1 ns
        ok 14 2378-04-22 Lower bound of 32bit>= timestamp. Extra sec bits 1. Max ns
        ok 15 2378-04-22 Lower bound of 32bit >=0 timestamp. All extra sec bits on
        ok 16 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra sec bits on
    # inode_test_xtimestamp_decoding: pass:16 fail:0 skip:0 total:16
    ok 1 inode_test_xtimestamp_decoding
# Totals: pass:16 fail:0 skip:0 total:16
ok 2 ext4_inode_test
[18:39:57] Elapsed time: 14.400s total, 2.940s configuring, 11.343s building, 0.074s running

Kemeng Shi (5):
  ext4: Add unit test for test_free_blocks_simple
  ext4: Add unit test of ext4_mb_generate_buddy
  ext4: Add unit test for mb_mark_used
  ext4: Add unit test for mb_free_blocks
  ext4: Add unit test for ext4_mb_mark_diskspace_used

 fs/ext4/mballoc-test.c | 502 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 502 insertions(+)

-- 
2.30.0

Re: [PATCH v2 0/5] More unit test for mballoc

Posted by Guenter Roeck 1 year, 9 months ago

On Wed, Jan 03, 2024 at 06:48:55PM +0800, Kemeng Shi wrote:
> This series covers more function to mark on-disk bitmap. Besides, some
> code which is relevant to buddy cache is also tested.
> Before more work is done, I want to be sure I'm not on a wrong
> direction!
> 
> v1->v2:
> -Fix unused variable warning which is reported at
> https://lore.kernel.org/lkml/202311260042.kMxL6DnL-lkp@intel.com/T/
> 

With this patch series in linux-text, and with various debug options
enabled, unit tests for ext4 fail widely and result in crashes.

[    4.798582]         # Subtest: test_new_blocks_simple
[    4.803166] BUG: key 00000000000000a8 has not been registered!
[    4.803443] ------------[ cut here ]------------
ILLOPC: ffffffff9050cbf4: 0f 0b
[    4.803521] DEBUG_LOCKS_WARN_ON(1)
[    4.803720] WARNING: CPU: 0 PID: 154 at kernel/locking/lockdep.c:4895 lockdep_init_map_type+0x224/0x250
...
[    4.833277]         # Subtest: test_mb_mark_used
[    4.835875] ------------[ cut here ]------------
[    4.836055] kernel BUG at fs/ext4/mballoc.c:2053!

Guenter

Re: [PATCH v2 0/5] More unit test for mballoc

Posted by Kemeng Shi 1 year, 9 months ago


on 2/26/2024 12:07 AM, Guenter Roeck wrote:
> On Wed, Jan 03, 2024 at 06:48:55PM +0800, Kemeng Shi wrote:
>> This series covers more function to mark on-disk bitmap. Besides, some
>> code which is relevant to buddy cache is also tested.
>> Before more work is done, I want to be sure I'm not on a wrong
>> direction!
>>
>> v1->v2:
>> -Fix unused variable warning which is reported at
>> https://lore.kernel.org/lkml/202311260042.kMxL6DnL-lkp@intel.com/T/
>>
> 
> With this patch series in linux-text, and with various debug options
> enabled, unit tests for ext4 fail widely and result in crashes.
> 
> [    4.798582]         # Subtest: test_new_blocks_simple
> [    4.803166] BUG: key 00000000000000a8 has not been registered!
> [    4.803443] ------------[ cut here ]------------
> ILLOPC: ffffffff9050cbf4: 0f 0b
> [    4.803521] DEBUG_LOCKS_WARN_ON(1)
> [    4.803720] WARNING: CPU: 0 PID: 154 at kernel/locking/lockdep.c:4895 lockdep_init_map_type+0x224/0x250
Not sure how this is triggerred.
> ...
> [    4.833277]         # Subtest: test_mb_mark_used
> [    4.835875] ------------[ cut here ]------------
> [    4.836055] kernel BUG at fs/ext4/mballoc.c:2053!
Internal functions mb_mark_used assumes group lock is held but unit
test code doesn't acquire it as there is no concurrent block
allocation/free in test code. Will add lock in unit test to fix this.
> 
> Guenter
> 
Hi Guenter, thanks for test and report this. I will appreciate it if
you could tell me how to reproduce this as I need to debug the first
issue and verify the fix. Thanks!

Re: [PATCH v2 0/5] More unit test for mballoc

Posted by Theodore Ts'o 1 year, 9 months ago

On Wed, 03 Jan 2024 18:48:55 +0800, Kemeng Shi wrote:
> This series covers more function to mark on-disk bitmap. Besides, some
> code which is relevant to buddy cache is also tested.
> Before more work is done, I want to be sure I'm not on a wrong
> direction!
> 
> v1->v2:
> -Fix unused variable warning which is reported at
> https://lore.kernel.org/lkml/202311260042.kMxL6DnL-lkp@intel.com/T/
> 
> [...]

Applied, thanks!

[1/5] ext4: Add unit test for test_free_blocks_simple
      commit: 6c5e0c9c21456fb561d0997fe2c4d3cf59745ba7
[2/5] ext4: Add unit test of ext4_mb_generate_buddy
      commit: 67d2a11b22b4d520072db62620851615df13183e
[3/5] ext4: Add unit test for mb_mark_used
      commit: ac96b56a2fbd9e05b28488bdc5d3bd8006b61d35
[4/5] ext4: Add unit test for mb_free_blocks
      commit: b7098e1fa7bcf350e089af9500c4f9992a1bd6cb
[5/5] ext4: Add unit test for ext4_mb_mark_diskspace_used
      commit: 2b81493f8eb6fc0c263dbca0064e10e4c00e0f91

Best regards,
-- 
Theodore Ts'o <tytso@mit.edu>