1. Patchew
  2. QEMU
  3. View series

[PULL 0/3] Block patches

Stefan Hajnoczi posted 3 patches 2 years ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20220505084208.2338968-1-stefanha@redhat.com
Maintainers: Stefan Hajnoczi <stefanha@redhat.com>, Fam Zheng <fam@euphon.net>, Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, Eduardo Habkost <eduardo@habkost.net>, Eric Blake <eblake@redhat.com>, Markus Armbruster <armbru@redhat.com>
There is a newer version of this series
qapi/qom.json                    |  43 ++++++++--
meson.build                      |  26 +++---
include/block/aio.h              |  10 +++
include/block/thread-pool.h      |   3 +
include/qemu/main-loop.h         |  10 +++
include/sysemu/event-loop-base.h |  41 +++++++++
include/sysemu/iothread.h        |   6 +-
event-loop-base.c                | 140 +++++++++++++++++++++++++++++++
iothread.c                       |  68 +++++----------
util/aio-posix.c                 |   1 +
util/async.c                     |  20 +++++
util/main-loop.c                 |  65 ++++++++++++++
util/thread-pool.c               |  55 +++++++++++-
13 files changed, 419 insertions(+), 69 deletions(-)
create mode 100644 include/sysemu/event-loop-base.h
create mode 100644 event-loop-base.c
[PULL 0/3] Block patches
Posted by Stefan Hajnoczi 2 years ago 
The following changes since commit 9cf289af47bcfae5c75de37d8e5d6fd23705322c:

  Merge tag 'qga-pull-request' of gitlab.com:marcandre.lureau/qemu into staging (2022-05-04 03:42:49 -0700)

are available in the Git repository at:

  https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to bef2e050d6a7feb865854c65570c496ac5a8cf53:

  util/event-loop-base: Introduce options to set the thread pool size (2022-05-04 17:02:19 +0100)

----------------------------------------------------------------
Pull request

Add new thread-pool-min/thread-pool-max parameters to control the thread pool
used for async I/O.

----------------------------------------------------------------

Nicolas Saenz Julienne (3):
  Introduce event-loop-base abstract class
  util/main-loop: Introduce the main loop into QOM
  util/event-loop-base: Introduce options to set the thread pool size

 qapi/qom.json                    |  43 ++++++++--
 meson.build                      |  26 +++---
 include/block/aio.h              |  10 +++
 include/block/thread-pool.h      |   3 +
 include/qemu/main-loop.h         |  10 +++
 include/sysemu/event-loop-base.h |  41 +++++++++
 include/sysemu/iothread.h        |   6 +-
 event-loop-base.c                | 140 +++++++++++++++++++++++++++++++
 iothread.c                       |  68 +++++----------
 util/aio-posix.c                 |   1 +
 util/async.c                     |  20 +++++
 util/main-loop.c                 |  65 ++++++++++++++
 util/thread-pool.c               |  55 +++++++++++-
 13 files changed, 419 insertions(+), 69 deletions(-)
 create mode 100644 include/sysemu/event-loop-base.h
 create mode 100644 event-loop-base.c

-- 
2.35.1
Re: [PULL 0/3] Block patches
Posted by Richard Henderson 2 years ago 
On 5/5/22 03:42, Stefan Hajnoczi wrote:
> The following changes since commit 9cf289af47bcfae5c75de37d8e5d6fd23705322c:
> 
>    Merge tag 'qga-pull-request' of gitlab.com:marcandre.lureau/qemu into staging (2022-05-04 03:42:49 -0700)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/stefanha/qemu.git tags/block-pull-request
> 
> for you to fetch changes up to bef2e050d6a7feb865854c65570c496ac5a8cf53:
> 
>    util/event-loop-base: Introduce options to set the thread pool size (2022-05-04 17:02:19 +0100)
> 
> ----------------------------------------------------------------
> Pull request
> 
> Add new thread-pool-min/thread-pool-max parameters to control the thread pool
> used for async I/O.
> 
> ----------------------------------------------------------------
> 
> Nicolas Saenz Julienne (3):
>    Introduce event-loop-base abstract class
>    util/main-loop: Introduce the main loop into QOM
>    util/event-loop-base: Introduce options to set the thread pool size
> 
>   qapi/qom.json                    |  43 ++++++++--
>   meson.build                      |  26 +++---
>   include/block/aio.h              |  10 +++
>   include/block/thread-pool.h      |   3 +
>   include/qemu/main-loop.h         |  10 +++
>   include/sysemu/event-loop-base.h |  41 +++++++++
>   include/sysemu/iothread.h        |   6 +-
>   event-loop-base.c                | 140 +++++++++++++++++++++++++++++++
>   iothread.c                       |  68 +++++----------
>   util/aio-posix.c                 |   1 +
>   util/async.c                     |  20 +++++
>   util/main-loop.c                 |  65 ++++++++++++++
>   util/thread-pool.c               |  55 +++++++++++-
>   13 files changed, 419 insertions(+), 69 deletions(-)
>   create mode 100644 include/sysemu/event-loop-base.h
>   create mode 100644 event-loop-base.c
> 

This appears to introduce a new error on msys2-64bit:


14/85 qemu:unit / test-aio                                          ERROR           2.14s 
   exit status 3
 >>> MALLOC_PERTURB_=82 G_TEST_SRCDIR=C:/GitLab-Runner/builds/qemu-project/qemu/tests/unit 
G_TEST_BUILDDIR=C:/GitLab-Runner/builds/qemu-project/qemu/build/tests/unit 
C:/GitLab-Runner/builds/qemu-project/qemu/build/tests/unit/test-aio.exe --tap -k
------------------------------------- 8< -------------------------------------
stderr:
(test program exited with status code 3)

https://gitlab.com/qemu-project/qemu/-/jobs/2418935125

Are you in a position to test this yourself locally?


r~
Re: [PULL 0/3] Block patches
Posted by Stefan Hajnoczi 1 year, 12 months ago 
On Thu, 5 May 2022 at 17:43, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 5/5/22 03:42, Stefan Hajnoczi wrote:
> > The following changes since commit 9cf289af47bcfae5c75de37d8e5d6fd23705322c:
> >
> >    Merge tag 'qga-pull-request' of gitlab.com:marcandre.lureau/qemu into staging (2022-05-04 03:42:49 -0700)
> >
> > are available in the Git repository at:
> >
> >    https://gitlab.com/stefanha/qemu.git tags/block-pull-request
> >
> > for you to fetch changes up to bef2e050d6a7feb865854c65570c496ac5a8cf53:
> >
> >    util/event-loop-base: Introduce options to set the thread pool size (2022-05-04 17:02:19 +0100)
> >
> > ----------------------------------------------------------------
> > Pull request
> >
> > Add new thread-pool-min/thread-pool-max parameters to control the thread pool
> > used for async I/O.
> >
> > ----------------------------------------------------------------
> >
> > Nicolas Saenz Julienne (3):
> >    Introduce event-loop-base abstract class
> >    util/main-loop: Introduce the main loop into QOM
> >    util/event-loop-base: Introduce options to set the thread pool size
> >
> >   qapi/qom.json                    |  43 ++++++++--
> >   meson.build                      |  26 +++---
> >   include/block/aio.h              |  10 +++
> >   include/block/thread-pool.h      |   3 +
> >   include/qemu/main-loop.h         |  10 +++
> >   include/sysemu/event-loop-base.h |  41 +++++++++
> >   include/sysemu/iothread.h        |   6 +-
> >   event-loop-base.c                | 140 +++++++++++++++++++++++++++++++
> >   iothread.c                       |  68 +++++----------
> >   util/aio-posix.c                 |   1 +
> >   util/async.c                     |  20 +++++
> >   util/main-loop.c                 |  65 ++++++++++++++
> >   util/thread-pool.c               |  55 +++++++++++-
> >   13 files changed, 419 insertions(+), 69 deletions(-)
> >   create mode 100644 include/sysemu/event-loop-base.h
> >   create mode 100644 event-loop-base.c
> >
>
> This appears to introduce a new error on msys2-64bit:
>
>
> 14/85 qemu:unit / test-aio                                          ERROR           2.14s
>    exit status 3
>  >>> MALLOC_PERTURB_=82 G_TEST_SRCDIR=C:/GitLab-Runner/builds/qemu-project/qemu/tests/unit
> G_TEST_BUILDDIR=C:/GitLab-Runner/builds/qemu-project/qemu/build/tests/unit
> C:/GitLab-Runner/builds/qemu-project/qemu/build/tests/unit/test-aio.exe --tap -k
> ------------------------------------- 8< -------------------------------------
> stderr:
> (test program exited with status code 3)
>
> https://gitlab.com/qemu-project/qemu/-/jobs/2418935125
>
> Are you in a position to test this yourself locally?

I haven't reproduced it yet but will dig a bit more.

test-aio.exe succeeds under Wine:
# random seed: R02S572ad8b9cfeac92bb23a64678114e66d
1..29
# Start of aio tests
ok 1 /aio/acquire
ok 2 /aio/external-client
# Start of bh tests
ok 3 /aio/bh/schedule
ok 4 /aio/bh/schedule10
ok 5 /aio/bh/cancel
ok 6 /aio/bh/delete
ok 7 /aio/bh/flush
# Start of callback-delete tests
ok 8 /aio/bh/callback-delete/one
ok 9 /aio/bh/callback-delete/many
# End of callback-delete tests
# End of bh tests
# Start of event tests
ok 10 /aio/event/add-remove
ok 11 /aio/event/wait
ok 12 /aio/event/flush
# Start of wait tests
ok 13 /aio/event/wait/no-flush-cb
# End of wait tests
# End of event tests
# Start of timer tests
ok 14 /aio/timer/schedule
# End of timer tests
# Start of coroutine tests
ok 15 /aio/coroutine/queue-chaining
ok 16 /aio/coroutine/worker-thread-co-enter
# End of coroutine tests
# End of aio tests
# Start of aio-gsource tests
ok 17 /aio-gsource/flush
# Start of bh tests
ok 18 /aio-gsource/bh/schedule
ok 19 /aio-gsource/bh/schedule10
ok 20 /aio-gsource/bh/cancel
ok 21 /aio-gsource/bh/delete
ok 22 /aio-gsource/bh/flush
# Start of callback-delete tests
ok 23 /aio-gsource/bh/callback-delete/one
ok 24 /aio-gsource/bh/callback-delete/many
# End of callback-delete tests
# End of bh tests
# Start of event tests
ok 25 /aio-gsource/event/add-remove
ok 26 /aio-gsource/event/wait
ok 27 /aio-gsource/event/flush
# Start of wait tests
ok 28 /aio-gsource/event/wait/no-flush-cb
# End of wait tests
# End of event tests
# Start of timer tests
ok 29 /aio-gsource/timer/schedule
# End of timer tests
# End of aio-gsource tests

Stefan
Re: [PULL 0/3] Block patches
Posted by Stefan Hajnoczi 1 year, 12 months ago 
It looks like a race condition in the test case. The failed test case
(tests/unit/test-aio.c:test_timer_schedule()) assumes the process will
be scheduled in a timely manner. A timer is scheduled for 750 ms and
the test waits for 1 second before running the event loop. At that
point a callback schedules the timer another 750 ms in the future. The
problem is that the exact sequence of event loop (aio_poll())
invocations can be unexpected on a heavily loaded CI runner leading to
spurious test failures.

I re-ran the job and it passed:
https://gitlab.com/qemu-project/qemu/-/jobs/2430529496

I also ran it locally under Wine and didn't see a failure.

FWIW I'm about to send another block pull request and it will include
these patches again. Feel free to wait for the next pull request.

Stefan

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.