[Qemu-devel] [PATCH v5 00/18] aio_context_acquire/release pushdown, part 2

Paolo Bonzini posted 18 patches 7 years, 2 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20170213135235.12274-1-pbonzini@redhat.com
Test checkpatch passed
Test docker passed
Test s390x passed
Makefile.objs                       |   4 -
block/blkdebug.c                    |   9 +-
block/blkreplay.c                   |   2 +-
block/block-backend.c               |  13 ++-
block/curl.c                        |  44 ++++++--
block/gluster.c                     |   9 +-
block/io.c                          |  38 ++-----
block/iscsi.c                       |  15 ++-
block/linux-aio.c                   |  10 +-
block/mirror.c                      |  12 +-
block/nbd-client.c                  | 117 +++++++++-----------
block/nbd-client.h                  |   2 +-
block/nfs.c                         |   9 +-
block/qed-cluster.c                 |   2 +
block/qed-table.c                   |  12 +-
block/qed.c                         |  58 +++++++---
block/qed.h                         |   3 +
block/sheepdog.c                    |  29 ++---
block/ssh.c                         |  29 ++---
block/throttle-groups.c             |   2 +
block/win32-aio.c                   |   9 +-
dma-helpers.c                       |   2 +
hw/block/virtio-blk.c               |  19 +++-
hw/scsi/scsi-bus.c                  |   2 +
hw/scsi/scsi-disk.c                 |  15 +++
hw/scsi/scsi-generic.c              |  20 +++-
hw/scsi/virtio-scsi.c               |   6 +
include/block/aio.h                 |  38 ++++++-
include/block/block_int.h           |  64 ++++++-----
include/io/channel.h                |  72 +++++++++++-
include/qemu/coroutine_int.h        |  11 +-
include/sysemu/block-backend.h      |  14 ++-
io/channel-command.c                |  13 +++
io/channel-file.c                   |  11 ++
io/channel-socket.c                 |  16 ++-
io/channel-tls.c                    |  12 ++
io/channel-watch.c                  |   6 +
io/channel.c                        |  97 ++++++++++++----
nbd/client.c                        |   2 +-
nbd/common.c                        |   9 +-
nbd/server.c                        |  94 +++++-----------
stubs/Makefile.objs                 |   1 +
stubs/linux-aio.c                   |  32 ++++++
stubs/set-fd-handler.c              |  11 --
tests/Makefile.include              |  19 ++--
tests/iothread.c                    |  91 +++++++++++++++
tests/iothread.h                    |  25 +++++
tests/test-aio-multithread.c        | 213 ++++++++++++++++++++++++++++++++++++
tests/test-thread-pool.c            |  12 +-
trace-events                        |  11 --
util/Makefile.objs                  |   6 +-
aio-posix.c => util/aio-posix.c     |  62 +++--------
aio-win32.c => util/aio-win32.c     |  30 ++---
util/aiocb.c                        |  55 ++++++++++
async.c => util/async.c             |  84 ++++++++++++--
iohandler.c => util/iohandler.c     |   0
main-loop.c => util/main-loop.c     |   0
util/qemu-coroutine-lock.c          |   5 +-
util/qemu-coroutine-sleep.c         |   2 +-
util/qemu-coroutine.c               |   8 ++
qemu-timer.c => util/qemu-timer.c   |   0
thread-pool.c => util/thread-pool.c |   8 +-
util/trace-events                   |  16 ++-
63 files changed, 1174 insertions(+), 468 deletions(-)
create mode 100644 stubs/linux-aio.c
create mode 100644 tests/iothread.c
create mode 100644 tests/iothread.h
create mode 100644 tests/test-aio-multithread.c
rename aio-posix.c => util/aio-posix.c (94%)
rename aio-win32.c => util/aio-win32.c (95%)
create mode 100644 util/aiocb.c
rename async.c => util/async.c (82%)
rename iohandler.c => util/iohandler.c (100%)
rename main-loop.c => util/main-loop.c (100%)
rename qemu-timer.c => util/qemu-timer.c (100%)
rename thread-pool.c => util/thread-pool.c (97%)
[Qemu-devel] [PATCH v5 00/18] aio_context_acquire/release pushdown, part 2
Posted by Paolo Bonzini 7 years, 2 months ago
Same as v3 except for tests/Makefile.include and trace-events rebase.

The next part is CoMutex thread-safety, which is relatively
small (6 patches).

Paolo

Paolo Bonzini (18):
  block: move AioContext, QEMUTimer, main-loop to libqemuutil
  aio: introduce aio_co_schedule and aio_co_wake
  block-backend: allow blk_prw from coroutine context
  test-thread-pool: use generic AioContext infrastructure
  io: add methods to set I/O handlers on AioContext
  io: make qio_channel_yield aware of AioContexts
  nbd: convert to use qio_channel_yield
  coroutine-lock: reschedule coroutine on the AioContext it was running on
  blkdebug: reschedule coroutine on the AioContext it is running on
  qed: introduce qed_aio_start_io and qed_aio_next_io_cb
  aio: push aio_context_acquire/release down to dispatching
  block: explicitly acquire aiocontext in timers that need it
  block: explicitly acquire aiocontext in callbacks that need it
  block: explicitly acquire aiocontext in bottom halves that need it
  block: explicitly acquire aiocontext in aio callbacks that need it
  aio-posix: partially inline aio_dispatch into aio_poll
  async: remove unnecessary inc/dec pairs
  block: document fields protected by AioContext lock

 Makefile.objs                       |   4 -
 block/blkdebug.c                    |   9 +-
 block/blkreplay.c                   |   2 +-
 block/block-backend.c               |  13 ++-
 block/curl.c                        |  44 ++++++--
 block/gluster.c                     |   9 +-
 block/io.c                          |  38 ++-----
 block/iscsi.c                       |  15 ++-
 block/linux-aio.c                   |  10 +-
 block/mirror.c                      |  12 +-
 block/nbd-client.c                  | 117 +++++++++-----------
 block/nbd-client.h                  |   2 +-
 block/nfs.c                         |   9 +-
 block/qed-cluster.c                 |   2 +
 block/qed-table.c                   |  12 +-
 block/qed.c                         |  58 +++++++---
 block/qed.h                         |   3 +
 block/sheepdog.c                    |  29 ++---
 block/ssh.c                         |  29 ++---
 block/throttle-groups.c             |   2 +
 block/win32-aio.c                   |   9 +-
 dma-helpers.c                       |   2 +
 hw/block/virtio-blk.c               |  19 +++-
 hw/scsi/scsi-bus.c                  |   2 +
 hw/scsi/scsi-disk.c                 |  15 +++
 hw/scsi/scsi-generic.c              |  20 +++-
 hw/scsi/virtio-scsi.c               |   6 +
 include/block/aio.h                 |  38 ++++++-
 include/block/block_int.h           |  64 ++++++-----
 include/io/channel.h                |  72 +++++++++++-
 include/qemu/coroutine_int.h        |  11 +-
 include/sysemu/block-backend.h      |  14 ++-
 io/channel-command.c                |  13 +++
 io/channel-file.c                   |  11 ++
 io/channel-socket.c                 |  16 ++-
 io/channel-tls.c                    |  12 ++
 io/channel-watch.c                  |   6 +
 io/channel.c                        |  97 ++++++++++++----
 nbd/client.c                        |   2 +-
 nbd/common.c                        |   9 +-
 nbd/server.c                        |  94 +++++-----------
 stubs/Makefile.objs                 |   1 +
 stubs/linux-aio.c                   |  32 ++++++
 stubs/set-fd-handler.c              |  11 --
 tests/Makefile.include              |  19 ++--
 tests/iothread.c                    |  91 +++++++++++++++
 tests/iothread.h                    |  25 +++++
 tests/test-aio-multithread.c        | 213 ++++++++++++++++++++++++++++++++++++
 tests/test-thread-pool.c            |  12 +-
 trace-events                        |  11 --
 util/Makefile.objs                  |   6 +-
 aio-posix.c => util/aio-posix.c     |  62 +++--------
 aio-win32.c => util/aio-win32.c     |  30 ++---
 util/aiocb.c                        |  55 ++++++++++
 async.c => util/async.c             |  84 ++++++++++++--
 iohandler.c => util/iohandler.c     |   0
 main-loop.c => util/main-loop.c     |   0
 util/qemu-coroutine-lock.c          |   5 +-
 util/qemu-coroutine-sleep.c         |   2 +-
 util/qemu-coroutine.c               |   8 ++
 qemu-timer.c => util/qemu-timer.c   |   0
 thread-pool.c => util/thread-pool.c |   8 +-
 util/trace-events                   |  16 ++-
 63 files changed, 1174 insertions(+), 468 deletions(-)
 create mode 100644 stubs/linux-aio.c
 create mode 100644 tests/iothread.c
 create mode 100644 tests/iothread.h
 create mode 100644 tests/test-aio-multithread.c
 rename aio-posix.c => util/aio-posix.c (94%)
 rename aio-win32.c => util/aio-win32.c (95%)
 create mode 100644 util/aiocb.c
 rename async.c => util/async.c (82%)
 rename iohandler.c => util/iohandler.c (100%)
 rename main-loop.c => util/main-loop.c (100%)
 rename qemu-timer.c => util/qemu-timer.c (100%)
 rename thread-pool.c => util/thread-pool.c (97%)

-- 
2.9.3


Re: [Qemu-devel] [PATCH v5 00/18] aio_context_acquire/release pushdown, part 2
Posted by Fam Zheng 7 years, 2 months ago
On Mon, 02/13 14:52, Paolo Bonzini wrote:
> Same as v3 except for tests/Makefile.include and trace-events rebase.
> 
> The next part is CoMutex thread-safety, which is relatively
> small (6 patches).
> 
> Paolo
> 
> Paolo Bonzini (18):
>   block: move AioContext, QEMUTimer, main-loop to libqemuutil
>   aio: introduce aio_co_schedule and aio_co_wake
>   block-backend: allow blk_prw from coroutine context
>   test-thread-pool: use generic AioContext infrastructure
>   io: add methods to set I/O handlers on AioContext
>   io: make qio_channel_yield aware of AioContexts
>   nbd: convert to use qio_channel_yield
>   coroutine-lock: reschedule coroutine on the AioContext it was running on
>   blkdebug: reschedule coroutine on the AioContext it is running on
>   qed: introduce qed_aio_start_io and qed_aio_next_io_cb
>   aio: push aio_context_acquire/release down to dispatching
>   block: explicitly acquire aiocontext in timers that need it
>   block: explicitly acquire aiocontext in callbacks that need it
>   block: explicitly acquire aiocontext in bottom halves that need it
>   block: explicitly acquire aiocontext in aio callbacks that need it
>   aio-posix: partially inline aio_dispatch into aio_poll
>   async: remove unnecessary inc/dec pairs
>   block: document fields protected by AioContext lock

Reviewed-by: Fam Zheng <famz@redhat.com>

Re: [Qemu-devel] [PATCH v5 00/18] aio_context_acquire/release pushdown, part 2
Posted by Stefan Hajnoczi 7 years, 2 months ago
On Mon, Feb 13, 2017 at 02:52:17PM +0100, Paolo Bonzini wrote:
> Same as v3 except for tests/Makefile.include and trace-events rebase.
> 
> The next part is CoMutex thread-safety, which is relatively
> small (6 patches).
> 
> Paolo
> 
> Paolo Bonzini (18):
>   block: move AioContext, QEMUTimer, main-loop to libqemuutil
>   aio: introduce aio_co_schedule and aio_co_wake
>   block-backend: allow blk_prw from coroutine context
>   test-thread-pool: use generic AioContext infrastructure
>   io: add methods to set I/O handlers on AioContext
>   io: make qio_channel_yield aware of AioContexts
>   nbd: convert to use qio_channel_yield
>   coroutine-lock: reschedule coroutine on the AioContext it was running on
>   blkdebug: reschedule coroutine on the AioContext it is running on
>   qed: introduce qed_aio_start_io and qed_aio_next_io_cb
>   aio: push aio_context_acquire/release down to dispatching
>   block: explicitly acquire aiocontext in timers that need it
>   block: explicitly acquire aiocontext in callbacks that need it
>   block: explicitly acquire aiocontext in bottom halves that need it
>   block: explicitly acquire aiocontext in aio callbacks that need it
>   aio-posix: partially inline aio_dispatch into aio_poll
>   async: remove unnecessary inc/dec pairs
>   block: document fields protected by AioContext lock
> 
>  Makefile.objs                       |   4 -
>  block/blkdebug.c                    |   9 +-
>  block/blkreplay.c                   |   2 +-
>  block/block-backend.c               |  13 ++-
>  block/curl.c                        |  44 ++++++--
>  block/gluster.c                     |   9 +-
>  block/io.c                          |  38 ++-----
>  block/iscsi.c                       |  15 ++-
>  block/linux-aio.c                   |  10 +-
>  block/mirror.c                      |  12 +-
>  block/nbd-client.c                  | 117 +++++++++-----------
>  block/nbd-client.h                  |   2 +-
>  block/nfs.c                         |   9 +-
>  block/qed-cluster.c                 |   2 +
>  block/qed-table.c                   |  12 +-
>  block/qed.c                         |  58 +++++++---
>  block/qed.h                         |   3 +
>  block/sheepdog.c                    |  29 ++---
>  block/ssh.c                         |  29 ++---
>  block/throttle-groups.c             |   2 +
>  block/win32-aio.c                   |   9 +-
>  dma-helpers.c                       |   2 +
>  hw/block/virtio-blk.c               |  19 +++-
>  hw/scsi/scsi-bus.c                  |   2 +
>  hw/scsi/scsi-disk.c                 |  15 +++
>  hw/scsi/scsi-generic.c              |  20 +++-
>  hw/scsi/virtio-scsi.c               |   6 +
>  include/block/aio.h                 |  38 ++++++-
>  include/block/block_int.h           |  64 ++++++-----
>  include/io/channel.h                |  72 +++++++++++-
>  include/qemu/coroutine_int.h        |  11 +-
>  include/sysemu/block-backend.h      |  14 ++-
>  io/channel-command.c                |  13 +++
>  io/channel-file.c                   |  11 ++
>  io/channel-socket.c                 |  16 ++-
>  io/channel-tls.c                    |  12 ++
>  io/channel-watch.c                  |   6 +
>  io/channel.c                        |  97 ++++++++++++----
>  nbd/client.c                        |   2 +-
>  nbd/common.c                        |   9 +-
>  nbd/server.c                        |  94 +++++-----------
>  stubs/Makefile.objs                 |   1 +
>  stubs/linux-aio.c                   |  32 ++++++
>  stubs/set-fd-handler.c              |  11 --
>  tests/Makefile.include              |  19 ++--
>  tests/iothread.c                    |  91 +++++++++++++++
>  tests/iothread.h                    |  25 +++++
>  tests/test-aio-multithread.c        | 213 ++++++++++++++++++++++++++++++++++++
>  tests/test-thread-pool.c            |  12 +-
>  trace-events                        |  11 --
>  util/Makefile.objs                  |   6 +-
>  aio-posix.c => util/aio-posix.c     |  62 +++--------
>  aio-win32.c => util/aio-win32.c     |  30 ++---
>  util/aiocb.c                        |  55 ++++++++++
>  async.c => util/async.c             |  84 ++++++++++++--
>  iohandler.c => util/iohandler.c     |   0
>  main-loop.c => util/main-loop.c     |   0
>  util/qemu-coroutine-lock.c          |   5 +-
>  util/qemu-coroutine-sleep.c         |   2 +-
>  util/qemu-coroutine.c               |   8 ++
>  qemu-timer.c => util/qemu-timer.c   |   0
>  thread-pool.c => util/thread-pool.c |   8 +-
>  util/trace-events                   |  16 ++-
>  63 files changed, 1174 insertions(+), 468 deletions(-)
>  create mode 100644 stubs/linux-aio.c
>  create mode 100644 tests/iothread.c
>  create mode 100644 tests/iothread.h
>  create mode 100644 tests/test-aio-multithread.c
>  rename aio-posix.c => util/aio-posix.c (94%)
>  rename aio-win32.c => util/aio-win32.c (95%)
>  create mode 100644 util/aiocb.c
>  rename async.c => util/async.c (82%)
>  rename iohandler.c => util/iohandler.c (100%)
>  rename main-loop.c => util/main-loop.c (100%)
>  rename qemu-timer.c => util/qemu-timer.c (100%)
>  rename thread-pool.c => util/thread-pool.c (97%)
> 
> -- 
> 2.9.3
> 
> 

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan