[PATCH v3 00/21] export/fuse: Use coroutines and multi-threading

Hanna Czenczek posted 21 patches 4 months, 2 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20250701114437.207419-1-hreitz@redhat.com
Maintainers: Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Xie Yongji <xieyongji@bytedance.com>, Coiby Xu <Coiby.Xu@gmail.com>, "Michael S. Tsirkin" <mst@redhat.com>, Stefano Garzarella <sgarzare@redhat.com>, Eric Blake <eblake@redhat.com>, Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>, Markus Armbruster <armbru@redhat.com>
qapi/block-export.json               |   39 +-
include/block/export.h               |   12 +-
include/qemu/osdep.h                 |    1 +
block/export/export.c                |   48 +-
block/export/fuse.c                  | 1181 ++++++++++++++++++++------
block/export/vduse-blk.c             |    7 +
block/export/vhost-user-blk-server.c |    8 +
block/file-posix.c                   |   17 +-
nbd/server.c                         |    6 +
util/osdep.c                         |   18 +
tests/qemu-iotests/307               |   47 +
tests/qemu-iotests/307.out           |   18 +
tests/qemu-iotests/308               |   55 +-
tests/qemu-iotests/308.out           |   61 +-
14 files changed, 1213 insertions(+), 305 deletions(-)
[PATCH v3 00/21] export/fuse: Use coroutines and multi-threading
Posted by Hanna Czenczek 4 months, 2 weeks ago
Hi,

This series:
- Fixes some bugs/minor inconveniences,
- Removes libfuse from the request processing path,
- Make the FUSE export use coroutines for request handling,
- Introduces multi-threading into the FUSE export.

More detail on the v1 cover letter:
https://lists.nongnu.org/archive/html/qemu-block/2025-03/msg00359.html

v2 cover letter:
https://lists.nongnu.org/archive/html/qemu-block/2025-06/msg00040.html


Changes from v2:
- Patch 12: Moved qemu_fcntl_addfl() into the #ifndef _WIN32 section
  where other fcntl() wrappers reside
- Patch 13: Fixed comment to say `export` instead of `exp`; kept
  Stefan’s R-b
- Patch 15: Same as patch 13
- Patch 18: In macro, use correct parameter `queue` instead of the
  non-existing `q`


git-backport-diff from v2:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/21:[----] [--] 'fuse: Copy write buffer content before polling'
002/21:[----] [--] 'fuse: Ensure init clean-up even with error_fatal'
003/21:[----] [--] 'fuse: Remove superfluous empty line'
004/21:[----] [--] 'fuse: Explicitly set inode ID to 1'
005/21:[----] [--] 'fuse: Change setup_... to mount_fuse_export()'
006/21:[----] [--] 'fuse: Fix mount options'
007/21:[----] [--] 'fuse: Set direct_io and parallel_direct_writes'
008/21:[----] [--] 'fuse: Introduce fuse_{at,de}tach_handlers()'
009/21:[----] [--] 'fuse: Introduce fuse_{inc,dec}_in_flight()'
010/21:[----] [--] 'fuse: Add halted flag'
011/21:[----] [--] 'fuse: Rename length to blk_len in fuse_write()'
012/21:[----] [-C] 'block: Move qemu_fcntl_addfl() into osdep.c'
013/21:[0008] [FC] 'fuse: Manually process requests (without libfuse)'
014/21:[----] [--] 'fuse: Reduce max read size'
015/21:[0008] [FC] 'fuse: Process requests in coroutines'
016/21:[----] [--] 'block/export: Add multi-threading interface'
017/21:[----] [--] 'iotests/307: Test multi-thread export interface'
018/21:[0016] [FC] 'fuse: Implement multi-threading'
019/21:[----] [--] 'qapi/block-export: Document FUSE's multi-threading'
020/21:[----] [--] 'iotests/308: Add multi-threading sanity test'
021/21:[----] [--] 'fuse: Increase MAX_WRITE_SIZE with a second buffer'


Hanna Czenczek (21):
  fuse: Copy write buffer content before polling
  fuse: Ensure init clean-up even with error_fatal
  fuse: Remove superfluous empty line
  fuse: Explicitly set inode ID to 1
  fuse: Change setup_... to mount_fuse_export()
  fuse: Fix mount options
  fuse: Set direct_io and parallel_direct_writes
  fuse: Introduce fuse_{at,de}tach_handlers()
  fuse: Introduce fuse_{inc,dec}_in_flight()
  fuse: Add halted flag
  fuse: Rename length to blk_len in fuse_write()
  block: Move qemu_fcntl_addfl() into osdep.c
  fuse: Manually process requests (without libfuse)
  fuse: Reduce max read size
  fuse: Process requests in coroutines
  block/export: Add multi-threading interface
  iotests/307: Test multi-thread export interface
  fuse: Implement multi-threading
  qapi/block-export: Document FUSE's multi-threading
  iotests/308: Add multi-threading sanity test
  fuse: Increase MAX_WRITE_SIZE with a second buffer

 qapi/block-export.json               |   39 +-
 include/block/export.h               |   12 +-
 include/qemu/osdep.h                 |    1 +
 block/export/export.c                |   48 +-
 block/export/fuse.c                  | 1181 ++++++++++++++++++++------
 block/export/vduse-blk.c             |    7 +
 block/export/vhost-user-blk-server.c |    8 +
 block/file-posix.c                   |   17 +-
 nbd/server.c                         |    6 +
 util/osdep.c                         |   18 +
 tests/qemu-iotests/307               |   47 +
 tests/qemu-iotests/307.out           |   18 +
 tests/qemu-iotests/308               |   55 +-
 tests/qemu-iotests/308.out           |   61 +-
 14 files changed, 1213 insertions(+), 305 deletions(-)

-- 
2.49.0


Re: [PATCH v3 00/21] export/fuse: Use coroutines and multi-threading
Posted by Kevin Wolf 3 weeks, 1 day ago
Am 01.07.2025 um 13:44 hat Hanna Czenczek geschrieben:
> Hi,
> 
> This series:
> - Fixes some bugs/minor inconveniences,
> - Removes libfuse from the request processing path,
> - Make the FUSE export use coroutines for request handling,
> - Introduces multi-threading into the FUSE export.
> 
> More detail on the v1 cover letter:
> https://lists.nongnu.org/archive/html/qemu-block/2025-03/msg00359.html
> 
> v2 cover letter:
> https://lists.nongnu.org/archive/html/qemu-block/2025-06/msg00040.html

I have finished review and while I did have comments here and there how
things could be nicer, I don't think I saw a real blocker. Please have a
look at my comments and let me know if you intend to respin the series
or if there are things I should fix up while applying the patches.

Kevin
Re: [PATCH v3 00/21] export/fuse: Use coroutines and multi-threading
Posted by Hanna Czenczek 2 weeks ago
On 23.10.25 17:15, Kevin Wolf wrote:
> Am 01.07.2025 um 13:44 hat Hanna Czenczek geschrieben:
>> Hi,
>>
>> This series:
>> - Fixes some bugs/minor inconveniences,
>> - Removes libfuse from the request processing path,
>> - Make the FUSE export use coroutines for request handling,
>> - Introduces multi-threading into the FUSE export.
>>
>> More detail on the v1 cover letter:
>> https://lists.nongnu.org/archive/html/qemu-block/2025-03/msg00359.html
>>
>> v2 cover letter:
>> https://lists.nongnu.org/archive/html/qemu-block/2025-06/msg00040.html
> I have finished review and while I did have comments here and there how
> things could be nicer, I don't think I saw a real blocker. Please have a
> look at my comments and let me know if you intend to respin the series
> or if there are things I should fix up while applying the patches.

Thanks for your suggestions!  They all sound good, I’ll revise.

Hanna


Re: [PATCH v3 00/21] export/fuse: Use coroutines and multi-threading
Posted by Stefan Hajnoczi 3 months, 2 weeks ago
On Tue, Jul 01, 2025 at 01:44:16PM +0200, Hanna Czenczek wrote:
> Hi,
> 
> This series:
> - Fixes some bugs/minor inconveniences,
> - Removes libfuse from the request processing path,
> - Make the FUSE export use coroutines for request handling,
> - Introduces multi-threading into the FUSE export.
> 
> More detail on the v1 cover letter:
> https://lists.nongnu.org/archive/html/qemu-block/2025-03/msg00359.html
> 
> v2 cover letter:
> https://lists.nongnu.org/archive/html/qemu-block/2025-06/msg00040.html
> 
> 
> Changes from v2:
> - Patch 12: Moved qemu_fcntl_addfl() into the #ifndef _WIN32 section
>   where other fcntl() wrappers reside
> - Patch 13: Fixed comment to say `export` instead of `exp`; kept
>   Stefan’s R-b
> - Patch 15: Same as patch 13
> - Patch 18: In macro, use correct parameter `queue` instead of the
>   non-existing `q`
> 
> 
> git-backport-diff from v2:
> 
> Key:
> [----] : patches are identical
> [####] : number of functional differences between upstream/downstream patch
> [down] : patch is downstream-only
> The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
> 
> 001/21:[----] [--] 'fuse: Copy write buffer content before polling'
> 002/21:[----] [--] 'fuse: Ensure init clean-up even with error_fatal'
> 003/21:[----] [--] 'fuse: Remove superfluous empty line'
> 004/21:[----] [--] 'fuse: Explicitly set inode ID to 1'
> 005/21:[----] [--] 'fuse: Change setup_... to mount_fuse_export()'
> 006/21:[----] [--] 'fuse: Fix mount options'
> 007/21:[----] [--] 'fuse: Set direct_io and parallel_direct_writes'
> 008/21:[----] [--] 'fuse: Introduce fuse_{at,de}tach_handlers()'
> 009/21:[----] [--] 'fuse: Introduce fuse_{inc,dec}_in_flight()'
> 010/21:[----] [--] 'fuse: Add halted flag'
> 011/21:[----] [--] 'fuse: Rename length to blk_len in fuse_write()'
> 012/21:[----] [-C] 'block: Move qemu_fcntl_addfl() into osdep.c'
> 013/21:[0008] [FC] 'fuse: Manually process requests (without libfuse)'
> 014/21:[----] [--] 'fuse: Reduce max read size'
> 015/21:[0008] [FC] 'fuse: Process requests in coroutines'
> 016/21:[----] [--] 'block/export: Add multi-threading interface'
> 017/21:[----] [--] 'iotests/307: Test multi-thread export interface'
> 018/21:[0016] [FC] 'fuse: Implement multi-threading'
> 019/21:[----] [--] 'qapi/block-export: Document FUSE's multi-threading'
> 020/21:[----] [--] 'iotests/308: Add multi-threading sanity test'
> 021/21:[----] [--] 'fuse: Increase MAX_WRITE_SIZE with a second buffer'
> 
> 
> Hanna Czenczek (21):
>   fuse: Copy write buffer content before polling
>   fuse: Ensure init clean-up even with error_fatal
>   fuse: Remove superfluous empty line
>   fuse: Explicitly set inode ID to 1
>   fuse: Change setup_... to mount_fuse_export()
>   fuse: Fix mount options
>   fuse: Set direct_io and parallel_direct_writes
>   fuse: Introduce fuse_{at,de}tach_handlers()
>   fuse: Introduce fuse_{inc,dec}_in_flight()
>   fuse: Add halted flag
>   fuse: Rename length to blk_len in fuse_write()
>   block: Move qemu_fcntl_addfl() into osdep.c
>   fuse: Manually process requests (without libfuse)
>   fuse: Reduce max read size
>   fuse: Process requests in coroutines
>   block/export: Add multi-threading interface
>   iotests/307: Test multi-thread export interface
>   fuse: Implement multi-threading
>   qapi/block-export: Document FUSE's multi-threading
>   iotests/308: Add multi-threading sanity test
>   fuse: Increase MAX_WRITE_SIZE with a second buffer
> 
>  qapi/block-export.json               |   39 +-
>  include/block/export.h               |   12 +-
>  include/qemu/osdep.h                 |    1 +
>  block/export/export.c                |   48 +-
>  block/export/fuse.c                  | 1181 ++++++++++++++++++++------
>  block/export/vduse-blk.c             |    7 +
>  block/export/vhost-user-blk-server.c |    8 +
>  block/file-posix.c                   |   17 +-
>  nbd/server.c                         |    6 +
>  util/osdep.c                         |   18 +
>  tests/qemu-iotests/307               |   47 +
>  tests/qemu-iotests/307.out           |   18 +
>  tests/qemu-iotests/308               |   55 +-
>  tests/qemu-iotests/308.out           |   61 +-
>  14 files changed, 1213 insertions(+), 305 deletions(-)
> 
> -- 
> 2.49.0
> 

Nice!

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>