[Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze

Eric Blake posted 36 patches 31 weeks ago
Failed in applying to current master (apply log)
Test checkpatch passed
Test docker-build@min-glib passed
Test docker-mingw@fedora passed
Test docker-quick@centos6 failed
Test s390x passed
docs/devel/qapi-code-gen.txt        |  87 ++++-
docs/interop/qmp-spec.txt           |  36 +-
qemu-doc.texi                       |   7 +
qapi/block-core.json                | 115 +++++-
qapi/introspect.json                |   6 +-
qapi/misc.json                      |  87 ++++-
scripts/qapi/commands.py            |  18 +-
scripts/qapi/common.py              |  15 +-
scripts/qapi/doc.py                 |   9 +-
scripts/qapi/introspect.py          |  83 +++--
include/block/accounting.h          |  35 ++
include/monitor/monitor.h           |   2 +-
include/qapi/qmp/dispatch.h         |   7 +-
include/qapi/qmp/qbool.h            |   1 -
include/qapi/qmp/qdict.h            |   1 -
include/qapi/qmp/qlist.h            |   1 -
include/qapi/qmp/qlit.h             |   4 +-
include/qapi/qmp/qnum.h             |   1 -
include/qapi/qmp/qobject.h          |  30 ++
include/qapi/qmp/qstring.h          |   3 +-
include/qemu/compiler.h             |  12 +-
vl.c                                |   7 +-
tests/libqtest.c                    |   6 +-
block.c                             |  13 +-
block/accounting.c                  |  91 +++++
block/qapi.c                        |  53 ++-
block/qcow2.c                       |   2 +-
block/rbd.c                         |   8 +-
block/sheepdog.c                    |   2 +-
blockdev.c                          |  64 +++-
hw/i386/acpi-build.c                |  14 +-
monitor.c                           | 701 +++++++++++++++++++++++++++++++-----
qapi/qmp-dispatch.c                 |  35 +-
qapi/qobject-input-visitor.c        |  24 +-
qapi/qobject-output-visitor.c       |   4 +-
qga/main.c                          |   2 +-
qmp.c                               |  18 +-
qobject/json-parser.c               |  13 +-
qobject/qbool.c                     |  15 +-
qobject/qdict.c                     |  65 ++--
qobject/qjson.c                     |  10 +-
qobject/qlist.c                     |  17 +-
qobject/qlit.c                      |  47 ++-
qobject/qnum.c                      |  17 +-
qobject/qstring.c                   |  38 +-
qom/object.c                        |  15 +-
target/i386/cpu.c                   |   2 +-
target/s390x/cpu_models.c           |   2 +-
tests/check-qdict.c                 |  20 +-
tests/check-qjson.c                 |  41 ++-
tests/check-qlist.c                 |   4 +-
tests/check-qlit.c                  |  30 +-
tests/check-qnum.c                  |   4 +-
tests/check-qobject.c               |   2 +-
tests/check-qstring.c               |   2 +-
tests/device-introspect-test.c      |  14 +-
tests/numa-test.c                   |   8 +-
tests/qmp-test.c                    |  97 ++++-
tests/qom-test.c                    |   4 +-
tests/test-char.c                   |   2 +-
tests/test-keyval.c                 |   8 +-
tests/test-qga.c                    |  19 +-
tests/test-qmp-cmds.c               |  12 +-
tests/test-qmp-event.c              |  16 +-
tests/test-qobject-input-visitor.c  |  21 +-
tests/test-qobject-output-visitor.c |  54 +--
tests/test-x86-cpuid-compat.c       |  17 +-
util/keyval.c                       |   4 +-
util/qemu-config.c                  |   2 +-
util/qemu-option.c                  |   6 +-
qemu-options.hx                     |   4 +-
tests/qapi-schema/test-qapi.py      |   2 +-
tests/qemu-iotests/089              |  20 +
tests/qemu-iotests/089.out          |   8 +
trace-events                        |   3 +
75 files changed, 1785 insertions(+), 484 deletions(-)

[Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze

Posted by Eric Blake 31 weeks ago
The following changes since commit 6ceb1b51f05f9e1892d082960ed602dca7b6696e:

  Merge remote-tracking branch 'remotes/kraxel/tags/audio-20180312-pull-request' into staging (2018-03-12 16:14:37 +0000)

are available in the Git repository at:

  git://repo.or.cz/qemu/ericb.git tags/pull-qapi-2018-03-12

for you to fetch changes up to a083c533b5a17c77ef164acdbf30eedfa9681fc6:

  qapi: add block latency histogram interface (2018-03-12 13:22:11 -0500)

This builds and passes 'make check', so even though the OOB portion
depends on chardev fixes that are still pending a pull request from
Paolo, that dependence can only be observed at runtime by clients
that use the new oob feature.  Given the timing of soft freeze, and
the fact that the chardev fixes do not form a build dependency, I
think it's okay if this pull request gets processed before Paolo's
(but it's also okay if Paolo's goes in first).

----------------------------------------------------------------
qapi patches for 2018-03-12, 2.12 softfreeze

- Marc-André Lureau: 0/4 qapi: generate a literal qobject for introspection
- Max Reitz: 0/7 block: Handle null backing link
- Peter Xu: 00/23 QMP: out-of-band (OOB) execution support
- Vladimir Sementsov-Ogievskiy: 0/2 block latency histogram

----------------------------------------------------------------
Marc-André Lureau (4):
      qapi2texi: minor python code simplification
      qlit: use QType instead of int
      qlit: add qobject_from_qlit()
      qapi: generate a literal qobject for introspection

Max Reitz (7):
      compiler: Add QEMU_BUILD_BUG_MSG() macro
      qapi: Add qobject_to()
      qapi: Replace qobject_to_X(o) by qobject_to(X, o)
      qapi: Remove qobject_to_X() functions
      qapi: Make more of qobject_to()
      block: Handle null backing link
      block: Deprecate "backing": ""

Peter Xu (23):
      docs: update QMP documents for OOB commands
      qobject: introduce qstring_get_try_str()
      qobject: introduce qobject_get_try_str()
      qobject: let object_property_get_str() use new API
      monitor: move skip_flush into monitor_data_init
      monitor: move the cur_mon hack deeper for QMP
      monitor: unify global init
      monitor: let mon_list be tail queue
      monitor: allow using IO thread for parsing
      qmp: introduce QMPCapability
      monitor: introduce monitor_qmp_respond()
      monitor: let suspend_cnt be thread safe
      monitor: let suspend/resume work even with QMPs
      monitor: separate QMP parser and dispatcher
      qmp: add new event "command-dropped"
      monitor: send event when command queue full
      qapi: introduce new cmd option "allow-oob"
      qmp: support out-of-band (oob) execution
      qmp: isolate responses into io thread
      monitor: enable IO thread for (qmp & !mux) typed
      qmp: add command "x-oob-test"
      tests: qmp-test: verify command batching
      tests: qmp-test: add oob test

Vladimir Sementsov-Ogievskiy (2):
      block/accounting: introduce latency histogram
      qapi: add block latency histogram interface

 docs/devel/qapi-code-gen.txt        |  87 ++++-
 docs/interop/qmp-spec.txt           |  36 +-
 qemu-doc.texi                       |   7 +
 qapi/block-core.json                | 115 +++++-
 qapi/introspect.json                |   6 +-
 qapi/misc.json                      |  87 ++++-
 scripts/qapi/commands.py            |  18 +-
 scripts/qapi/common.py              |  15 +-
 scripts/qapi/doc.py                 |   9 +-
 scripts/qapi/introspect.py          |  83 +++--
 include/block/accounting.h          |  35 ++
 include/monitor/monitor.h           |   2 +-
 include/qapi/qmp/dispatch.h         |   7 +-
 include/qapi/qmp/qbool.h            |   1 -
 include/qapi/qmp/qdict.h            |   1 -
 include/qapi/qmp/qlist.h            |   1 -
 include/qapi/qmp/qlit.h             |   4 +-
 include/qapi/qmp/qnum.h             |   1 -
 include/qapi/qmp/qobject.h          |  30 ++
 include/qapi/qmp/qstring.h          |   3 +-
 include/qemu/compiler.h             |  12 +-
 vl.c                                |   7 +-
 tests/libqtest.c                    |   6 +-
 block.c                             |  13 +-
 block/accounting.c                  |  91 +++++
 block/qapi.c                        |  53 ++-
 block/qcow2.c                       |   2 +-
 block/rbd.c                         |   8 +-
 block/sheepdog.c                    |   2 +-
 blockdev.c                          |  64 +++-
 hw/i386/acpi-build.c                |  14 +-
 monitor.c                           | 701 +++++++++++++++++++++++++++++++-----
 qapi/qmp-dispatch.c                 |  35 +-
 qapi/qobject-input-visitor.c        |  24 +-
 qapi/qobject-output-visitor.c       |   4 +-
 qga/main.c                          |   2 +-
 qmp.c                               |  18 +-
 qobject/json-parser.c               |  13 +-
 qobject/qbool.c                     |  15 +-
 qobject/qdict.c                     |  65 ++--
 qobject/qjson.c                     |  10 +-
 qobject/qlist.c                     |  17 +-
 qobject/qlit.c                      |  47 ++-
 qobject/qnum.c                      |  17 +-
 qobject/qstring.c                   |  38 +-
 qom/object.c                        |  15 +-
 target/i386/cpu.c                   |   2 +-
 target/s390x/cpu_models.c           |   2 +-
 tests/check-qdict.c                 |  20 +-
 tests/check-qjson.c                 |  41 ++-
 tests/check-qlist.c                 |   4 +-
 tests/check-qlit.c                  |  30 +-
 tests/check-qnum.c                  |   4 +-
 tests/check-qobject.c               |   2 +-
 tests/check-qstring.c               |   2 +-
 tests/device-introspect-test.c      |  14 +-
 tests/numa-test.c                   |   8 +-
 tests/qmp-test.c                    |  97 ++++-
 tests/qom-test.c                    |   4 +-
 tests/test-char.c                   |   2 +-
 tests/test-keyval.c                 |   8 +-
 tests/test-qga.c                    |  19 +-
 tests/test-qmp-cmds.c               |  12 +-
 tests/test-qmp-event.c              |  16 +-
 tests/test-qobject-input-visitor.c  |  21 +-
 tests/test-qobject-output-visitor.c |  54 +--
 tests/test-x86-cpuid-compat.c       |  17 +-
 util/keyval.c                       |   4 +-
 util/qemu-config.c                  |   2 +-
 util/qemu-option.c                  |   6 +-
 qemu-options.hx                     |   4 +-
 tests/qapi-schema/test-qapi.py      |   2 +-
 tests/qemu-iotests/089              |  20 +
 tests/qemu-iotests/089.out          |   8 +
 trace-events                        |   3 +
 75 files changed, 1785 insertions(+), 484 deletions(-)

-- 
2.14.3


Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze

Posted by Peter Maydell 31 weeks ago
On 12 March 2018 at 18:35, Eric Blake <eblake@redhat.com> wrote:
> The following changes since commit 6ceb1b51f05f9e1892d082960ed602dca7b6696e:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/audio-20180312-pull-request' into staging (2018-03-12 16:14:37 +0000)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/ericb.git tags/pull-qapi-2018-03-12
>
> for you to fetch changes up to a083c533b5a17c77ef164acdbf30eedfa9681fc6:
>
>   qapi: add block latency histogram interface (2018-03-12 13:22:11 -0500)
>
> This builds and passes 'make check', so even though the OOB portion
> depends on chardev fixes that are still pending a pull request from
> Paolo, that dependence can only be observed at runtime by clients
> that use the new oob feature.  Given the timing of soft freeze, and
> the fact that the chardev fixes do not form a build dependency, I
> think it's okay if this pull request gets processed before Paolo's
> (but it's also okay if Paolo's goes in first).
>
> ----------------------------------------------------------------
> qapi patches for 2018-03-12, 2.12 softfreeze
>
> - Marc-André Lureau: 0/4 qapi: generate a literal qobject for introspection
> - Max Reitz: 0/7 block: Handle null backing link
> - Peter Xu: 00/23 QMP: out-of-band (OOB) execution support
> - Vladimir Sementsov-Ogievskiy: 0/2 block latency histogram
>

Hi; I'm afraid this failed some of my build tests:

x86/Linux and x86 OpenBSD, compile failure (probably gcc-version-dependent):

/home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c: In
function ‘build_append_pci_bus_devices’:
/home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c:617:9:
error: ‘notify_method’ may be used uninitialized in this function
[-Werror=maybe-uninitialized]
         aml_append(parent_scope, notify_method);
         ^
/home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c:510:16:
note: ‘notify_method’ was declared here
     Aml *dev, *notify_method, *method;
                ^
cc1: all warnings being treated as errors


on PPC64 Linux, FreeBSD x86, OpenBSD x86, aarch64 Linux hosts, test fails;
looks like the same assert but in different tests:

ppc64:
TEST: tests/qmp-test... (pid=48041)
  /alpha/qmp/protocol:                                                 OK
  /alpha/qmp/oob:                                                      OK
  /alpha/qmp/query-status:                                             OK
  /alpha/qmp/query-block:
qemu-system-alpha: /home/pm215/qemu/
chardev/char-io.c:91: io_watch_poll_finalize: Assertion `iwp->src ==
((void *)0)' failed.
Broken pipe
FAIL
GTester: last random seed: R02S3e793887202ca8b099adb20531a072e6
(pid=48057)

aarch64:
  /aarch64/qmp/query-qmp-schema:                                       OK
  /aarch64/qmp/query-version:                                          OK
  /aarch64/qmp/query-commands:
Assertion failed: (iwp->src == NULL)
, function io_watch_poll_finalize, file /root/qemu/chardev/char-io.c, line 91.
Broken pipe
FAIL
GTester: last random seed: R02Sbd982a1e1da0c16d183f7725739b58af
(pid=58277)

and the others are the same but in
/mips64/device/introspect/abstract-interfaces
/alpha/qmp/query-named-block-nodes

thanks
-- PMM

Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze

Posted by Eric Blake 31 weeks ago
On 03/13/2018 09:02 AM, Peter Maydell wrote:
> On 12 March 2018 at 18:35, Eric Blake <eblake@redhat.com> wrote:
>> The following changes since commit 6ceb1b51f05f9e1892d082960ed602dca7b6696e:
>>
>>    Merge remote-tracking branch 'remotes/kraxel/tags/audio-20180312-pull-request' into staging (2018-03-12 16:14:37 +0000)
>>
>> are available in the Git repository at:
>>
>>    git://repo.or.cz/qemu/ericb.git tags/pull-qapi-2018-03-12
>>
>> for you to fetch changes up to a083c533b5a17c77ef164acdbf30eedfa9681fc6:
>>
>>    qapi: add block latency histogram interface (2018-03-12 13:22:11 -0500)
>>
>> This builds and passes 'make check', so even though the OOB portion
>> depends on chardev fixes that are still pending a pull request from
>> Paolo, that dependence can only be observed at runtime by clients
>> that use the new oob feature.  Given the timing of soft freeze, and
>> the fact that the chardev fixes do not form a build dependency, I
>> think it's okay if this pull request gets processed before Paolo's
>> (but it's also okay if Paolo's goes in first).

Based on the testsuite failures, it looks like Paolo's pull request with 
chardev fixes DOES have to go in first. More at [1] below.

> x86/Linux and x86 OpenBSD, compile failure (probably gcc-version-dependent):
> 
> /home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c: In
> function ‘build_append_pci_bus_devices’:
> /home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c:617:9:
> error: ‘notify_method’ may be used uninitialized in this function
> [-Werror=maybe-uninitialized]
>           aml_append(parent_scope, notify_method);
>           ^
> /home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c:510:16:
> note: ‘notify_method’ was declared here
>       Aml *dev, *notify_method, *method;
>                  ^
> cc1: all warnings being treated as errors

Odd - The only mention of notify_method in that file in my series is in 
the context:

$ git diff 6ceb1b51f..pull-qapi-2018-03-12 hw/i386/acpi-build.c \
    |grep notify_method
          notify_method = aml_method("DVNT", 2, AML_NOTSERIALIZED);

and where all the hunks look like:

@@ -154,21 +154,21 @@ static void acpi_get_pm_info(AcpiPmInfo *pm)
      /* Fill in optional s3/s4 related properties */
      o = object_property_get_qobject(obj, ACPI_PM_PROP_S3_DISABLED, NULL);
      if (o) {
-        pm->s3_disabled = qnum_get_uint(qobject_to_qnum(o));
+        pm->s3_disabled = qnum_get_uint(qobject_to(QNum, o));

which doesn't change control flow logic.  So all I can guess is that 
this has been a latent pre-existing problem, but the compiler is now 
flagging it.  At any rate, I can try and shut it up, and will spin a v2 
once the other issue is solved.


> 
> on PPC64 Linux, FreeBSD x86, OpenBSD x86, aarch64 Linux hosts, test fails;
> looks like the same assert but in different tests:
> 
> ppc64:
> TEST: tests/qmp-test... (pid=48041)
>    /alpha/qmp/protocol:                                                 OK
>    /alpha/qmp/oob:                                                      OK
>    /alpha/qmp/query-status:                                             OK
>    /alpha/qmp/query-block:
> qemu-system-alpha: /home/pm215/qemu/
> chardev/char-io.c:91: io_watch_poll_finalize: Assertion `iwp->src ==
> ((void *)0)' failed.
> Broken pipe
> FAIL
> GTester: last random seed: R02S3e793887202ca8b099adb20531a072e6
> (pid=48057)
> 

[1] this is probably the chardev fixes being tickled by oob.  (Weird 
that the change is not failing the oob test, though - or is the failure 
happening during cleanup of the oob test, AFTER it reported OK?)  Here's 
where I'm hoping Paolo's pull request with chardev fixes is the 
solution, otherwise, I may have to disable Peter's OOB patches.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze

Posted by Peter Xu 31 weeks ago
On Tue, Mar 13, 2018 at 09:17:36AM -0500, Eric Blake wrote:
> On 03/13/2018 09:02 AM, Peter Maydell wrote:
> > On 12 March 2018 at 18:35, Eric Blake <eblake@redhat.com> wrote:
> > > The following changes since commit 6ceb1b51f05f9e1892d082960ed602dca7b6696e:
> > > 
> > >    Merge remote-tracking branch 'remotes/kraxel/tags/audio-20180312-pull-request' into staging (2018-03-12 16:14:37 +0000)
> > > 
> > > are available in the Git repository at:
> > > 
> > >    git://repo.or.cz/qemu/ericb.git tags/pull-qapi-2018-03-12
> > > 
> > > for you to fetch changes up to a083c533b5a17c77ef164acdbf30eedfa9681fc6:
> > > 
> > >    qapi: add block latency histogram interface (2018-03-12 13:22:11 -0500)
> > > 
> > > This builds and passes 'make check', so even though the OOB portion
> > > depends on chardev fixes that are still pending a pull request from
> > > Paolo, that dependence can only be observed at runtime by clients
> > > that use the new oob feature.  Given the timing of soft freeze, and
> > > the fact that the chardev fixes do not form a build dependency, I
> > > think it's okay if this pull request gets processed before Paolo's
> > > (but it's also okay if Paolo's goes in first).
> 
> Based on the testsuite failures, it looks like Paolo's pull request with
> chardev fixes DOES have to go in first. More at [1] below.

[...]

> > 
> > on PPC64 Linux, FreeBSD x86, OpenBSD x86, aarch64 Linux hosts, test fails;
> > looks like the same assert but in different tests:
> > 
> > ppc64:
> > TEST: tests/qmp-test... (pid=48041)
> >    /alpha/qmp/protocol:                                                 OK
> >    /alpha/qmp/oob:                                                      OK
> >    /alpha/qmp/query-status:                                             OK
> >    /alpha/qmp/query-block:
> > qemu-system-alpha: /home/pm215/qemu/
> > chardev/char-io.c:91: io_watch_poll_finalize: Assertion `iwp->src ==
> > ((void *)0)' failed.
> > Broken pipe
> > FAIL
> > GTester: last random seed: R02S3e793887202ca8b099adb20531a072e6
> > (pid=48057)
> > 
> 
> [1] this is probably the chardev fixes being tickled by oob.  (Weird that
> the change is not failing the oob test, though - or is the failure happening
> during cleanup of the oob test, AFTER it reported OK?)  Here's where I'm
> hoping Paolo's pull request with chardev fixes is the solution, otherwise, I
> may have to disable Peter's OOB patches.

Yes it is.  The failure can possibly happen randomly on very random
tests if without the whole bunch of chardev patches.

I confirmed with Paolo offlist that all the chardev fixes will be in
Paolo's next chardev pull request for the softfreeze.  So with Paolo's
next pull request, all the tests should pass, with 100%. If it still
fails any, then please feel free to drop the whole OOB series so that
I'll rework after 2.12.

Sorry again for the troublesome.

-- 
Peter Xu

Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze

Posted by Eric Blake 31 weeks ago
On 03/13/2018 09:17 AM, Eric Blake wrote:
>>> This builds and passes 'make check', so even though the OOB portion
>>> depends on chardev fixes that are still pending a pull request from
>>> Paolo, that dependence can only be observed at runtime by clients
>>> that use the new oob feature.  Given the timing of soft freeze, and
>>> the fact that the chardev fixes do not form a build dependency, I
>>> think it's okay if this pull request gets processed before Paolo's
>>> (but it's also okay if Paolo's goes in first).
> 
> Based on the testsuite failures, it looks like Paolo's pull request with 
> chardev fixes DOES have to go in first.

Nearing the end of my workday; I'm not sure what the status is on 
Paolo's pull request with the chardev fixes, but hope that it doesn't 
invalidate this pull request from still being considered as soft freeze 
material.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze

Posted by Peter Maydell 31 weeks ago
On 13 March 2018 at 21:55, Eric Blake <eblake@redhat.com> wrote:
> On 03/13/2018 09:17 AM, Eric Blake wrote:
>>>>
>>>> This builds and passes 'make check', so even though the OOB portion
>>>> depends on chardev fixes that are still pending a pull request from
>>>> Paolo, that dependence can only be observed at runtime by clients
>>>> that use the new oob feature.  Given the timing of soft freeze, and
>>>> the fact that the chardev fixes do not form a build dependency, I
>>>> think it's okay if this pull request gets processed before Paolo's
>>>> (but it's also okay if Paolo's goes in first).
>>
>>
>> Based on the testsuite failures, it looks like Paolo's pull request with
>> chardev fixes DOES have to go in first.
>
>
> Nearing the end of my workday; I'm not sure what the status is on Paolo's
> pull request with the chardev fixes, but hope that it doesn't invalidate
> this pull request from still being considered as soft freeze material.

The freeze approach is that if you get v1 of the pull on the list
before the date, then the purpose of the following week before rc0
is to get the stragglers in and so that we can do re-spins if there
were minor issues with anything. So it's ok.

thanks
-- PMM