[PULL 00/13] Block layer patches

Test docker-mingw@fedora passed
Test docker-quick@centos7 passed
Test checkpatch passed
Test FreeBSD passed
Test asan passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200311154218.15532-1-kwolf@redhat.com
Maintainers: Max Reitz <mreitz@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, Laurent Vivier <lvivier@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Eric Blake <eblake@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>, Markus Armbruster <armbru@redhat.com>, Thomas Huth <thuth@redhat.com>
There is a newer version of this series
qapi/block-core.json          |  9 ++++-
include/block/block.h         |  1 +
include/block/block_int.h     |  7 ++++
block.c                       | 33 ++++++++++++++--
block/crypto.c                | 18 +++++++++
block/file-posix.c            | 23 +++++++++++
blockdev.c                    | 30 ++++-----------
qom/qom-qmp-cmds.c            | 16 +++-----
tests/qemu-iotests/iotests.py |  5 ++-
tests/Makefile.include        |  1 +
tests/qemu-iotests/085.out    |  4 +-
tests/qemu-iotests/155        | 88 ++++++++++++++++++++++++++++++++++++-------
tests/qemu-iotests/155.out    |  4 +-
tests/qemu-iotests/282        | 67 ++++++++++++++++++++++++++++++++
tests/qemu-iotests/282.out    | 11 ++++++
tests/qemu-iotests/group      |  1 +
tests/qtest/Makefile.include  |  1 -
17 files changed, 262 insertions(+), 57 deletions(-)
create mode 100755 tests/qemu-iotests/282
create mode 100644 tests/qemu-iotests/282.out
[PULL 00/13] Block layer patches
Posted by Kevin Wolf 5 years ago
The following changes since commit ba29883206d92a29ad5a466e679ccfc2ee6132ef:

  Merge remote-tracking branch 'remotes/borntraeger/tags/s390x-20200310' into staging (2020-03-10 16:50:28 +0000)

are available in the Git repository at:

  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to 8bb3b023f2055054ee119cb45b42d2b14be7fc8a:

  qemu-iotests: adding LUKS cleanup for non-UTF8 secret error (2020-03-11 15:54:38 +0100)

----------------------------------------------------------------
Block layer patches:

- Relax restrictions for blockdev-snapshot (allows libvirt to do live
  storage migration with blockdev-mirror)
- luks: Delete created files when block_crypto_co_create_opts_luks fails
- Fix memleaks in qmp_object_add

----------------------------------------------------------------
Daniel Henrique Barboza (4):
      block: introducing 'bdrv_co_delete_file' interface
      block.c: adding bdrv_co_delete_file
      crypto.c: cleanup created file when block_crypto_co_create_opts_luks fails
      qemu-iotests: adding LUKS cleanup for non-UTF8 secret error

Kevin Wolf (6):
      block: Make bdrv_get_cumulative_perm() public
      block: Relax restrictions for blockdev-snapshot
      iotests: Fix run_job() with use_log=False
      iotests: Test mirror with temporarily disabled target backing file
      block: Fix cross-AioContext blockdev-snapshot
      iotests: Add iothread cases to 155

Pan Nengyuan (1):
      qom-qmp-cmds: fix two memleaks in qmp_object_add

Peter Krempa (1):
      qapi: Add '@allow-write-only-overlay' feature for 'blockdev-snapshot'

Philippe Mathieu-Daudé (1):
      tests/qemu-iotests: Fix socket_scm_helper build path

 qapi/block-core.json          |  9 ++++-
 include/block/block.h         |  1 +
 include/block/block_int.h     |  7 ++++
 block.c                       | 33 ++++++++++++++--
 block/crypto.c                | 18 +++++++++
 block/file-posix.c            | 23 +++++++++++
 blockdev.c                    | 30 ++++-----------
 qom/qom-qmp-cmds.c            | 16 +++-----
 tests/qemu-iotests/iotests.py |  5 ++-
 tests/Makefile.include        |  1 +
 tests/qemu-iotests/085.out    |  4 +-
 tests/qemu-iotests/155        | 88 ++++++++++++++++++++++++++++++++++++-------
 tests/qemu-iotests/155.out    |  4 +-
 tests/qemu-iotests/282        | 67 ++++++++++++++++++++++++++++++++
 tests/qemu-iotests/282.out    | 11 ++++++
 tests/qemu-iotests/group      |  1 +
 tests/qtest/Makefile.include  |  1 -
 17 files changed, 262 insertions(+), 57 deletions(-)
 create mode 100755 tests/qemu-iotests/282
 create mode 100644 tests/qemu-iotests/282.out


Re: [PULL 00/13] Block layer patches
Posted by Peter Maydell 5 years ago
On Wed, 11 Mar 2020 at 15:42, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit ba29883206d92a29ad5a466e679ccfc2ee6132ef:
>
>   Merge remote-tracking branch 'remotes/borntraeger/tags/s390x-20200310' into staging (2020-03-10 16:50:28 +0000)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 8bb3b023f2055054ee119cb45b42d2b14be7fc8a:
>
>   qemu-iotests: adding LUKS cleanup for non-UTF8 secret error (2020-03-11 15:54:38 +0100)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - Relax restrictions for blockdev-snapshot (allows libvirt to do live
>   storage migration with blockdev-mirror)
> - luks: Delete created files when block_crypto_co_create_opts_luks fails
> - Fix memleaks in qmp_object_add
>
> ----------------------------------------------------------------


iotest 030 hung on x86-64 Linux (Ubuntu):

petmay01 11801  0.0  0.0  34668 26112 ?        S    11:24   0:03  |
                   \_ make --output-sync -C build/alldbg check V=1 -j8
petmay01 15277  0.0  0.0   4628   792 ?        S    11:25   0:00  |
                       \_ /bin/sh
/home/petmay01/linaro/qemu-for-merges/tests/check-block.sh
petmay01 15344  0.0  0.0  14172  3360 ?        S    11:25   0:00  |
                           \_ bash ./check -makecheck -qcow2 -g auto
petmay01 27902  0.0  0.0  14172  2128 ?        S    11:25   0:00  |
                               \_ bash ./check -makecheck -qcow2 -g
auto
petmay01 27903  0.0  0.0  52660 16400 ?        S    11:25   0:00  |
                                   \_ python3 -B 030
petmay01  1728  0.0  0.1 1011792 51604 ?       Sl   11:26   0:01  |
                                       \_
/home/petmay01/linaro/qemu-for-merges/build/alldbg/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64
-display none -vga none -chardev
socket,id=mon,path=/tmp/tmp.QBQTAAybTi/qemu-27903-monitor.sock -mon
chardev=mon,mode=control -qtest
unix:path=/tmp/tmp.QBQTAAybTi/qemu-27903-qtest.sock -accel qtest
-nodefaults -display none -accel qtest -drive
if=virtio,id=drive0,file=blkdebug::/home/petmay01/linaro/qemu-for-merges/build/alldbg/tests/qemu-iotests/scratch/test.img,format=qcow2,cache=writeback,aio=threads,backing.node-name=mid,backing.backing.node-name=base

I had to manually kill the offending QEMU process; resulting
output in the log:

--- /home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/030.out
 2019-07-15 17:18:35.251364738 +01
00
+++ /home/petmay01/linaro/qemu-for-merges/build/alldbg/tests/qemu-iotests/030.out.bad
  2020-03-12 13:44:
43.101182680 +0000
@@ -1,5 +1,27 @@
-...........................
+........................E..
+======================================================================
+ERROR: test_stream_pause (__main__.TestSingleDrive)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+  File "030", line 93, in test_stream_pause
+    self.pause_wait('drive0')
+  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/iotests.py",
line 927, in pause_wait
+    result = self.vm.qmp('query-block-jobs')
+  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/../../python/qemu/machine.py",
line 405, in qmp
+    return self._qmp.cmd(cmd, args=qmp_args)
+  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/../../python/qemu/qmp.py",
line 215, in cmd
+    return self.cmd_obj(qmp_cmd)
+  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/../../python/qemu/qmp.py",
line 198, in cmd_obj
+    resp = self.__json_read()
+  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/../../python/qemu/qmp.py",
line 89, in __json_read
+    data = self.__sockfile.readline()
+  File "/usr/lib/python3.6/socket.py", line 586, in readinto
+    return self._sock.recv_into(b)
+  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/iotests.py",
line 383, in timeout
+    raise Exception(self.errmsg)
+Exception: Timeout waiting for job to pause
+
 ----------------------------------------------------------------------


thanks
-- PMM

Re: [PULL 00/13] Block layer patches
Posted by Kevin Wolf 5 years ago
Am 12.03.2020 um 14:46 hat Peter Maydell geschrieben:
> On Wed, 11 Mar 2020 at 15:42, Kevin Wolf <kwolf@redhat.com> wrote:
> >
> > The following changes since commit ba29883206d92a29ad5a466e679ccfc2ee6132ef:
> >
> >   Merge remote-tracking branch 'remotes/borntraeger/tags/s390x-20200310' into staging (2020-03-10 16:50:28 +0000)
> >
> > are available in the Git repository at:
> >
> >   git://repo.or.cz/qemu/kevin.git tags/for-upstream
> >
> > for you to fetch changes up to 8bb3b023f2055054ee119cb45b42d2b14be7fc8a:
> >
> >   qemu-iotests: adding LUKS cleanup for non-UTF8 secret error (2020-03-11 15:54:38 +0100)
> >
> > ----------------------------------------------------------------
> > Block layer patches:
> >
> > - Relax restrictions for blockdev-snapshot (allows libvirt to do live
> >   storage migration with blockdev-mirror)
> > - luks: Delete created files when block_crypto_co_create_opts_luks fails
> > - Fix memleaks in qmp_object_add
> >
> > ----------------------------------------------------------------
> 
> 
> iotest 030 hung on x86-64 Linux (Ubuntu):
> 
> petmay01 11801  0.0  0.0  34668 26112 ?        S    11:24   0:03  |
>                    \_ make --output-sync -C build/alldbg check V=1 -j8
> petmay01 15277  0.0  0.0   4628   792 ?        S    11:25   0:00  |
>                        \_ /bin/sh
> /home/petmay01/linaro/qemu-for-merges/tests/check-block.sh
> petmay01 15344  0.0  0.0  14172  3360 ?        S    11:25   0:00  |
>                            \_ bash ./check -makecheck -qcow2 -g auto
> petmay01 27902  0.0  0.0  14172  2128 ?        S    11:25   0:00  |
>                                \_ bash ./check -makecheck -qcow2 -g
> auto
> petmay01 27903  0.0  0.0  52660 16400 ?        S    11:25   0:00  |
>                                    \_ python3 -B 030
> petmay01  1728  0.0  0.1 1011792 51604 ?       Sl   11:26   0:01  |
>                                        \_
> /home/petmay01/linaro/qemu-for-merges/build/alldbg/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64
> -display none -vga none -chardev
> socket,id=mon,path=/tmp/tmp.QBQTAAybTi/qemu-27903-monitor.sock -mon
> chardev=mon,mode=control -qtest
> unix:path=/tmp/tmp.QBQTAAybTi/qemu-27903-qtest.sock -accel qtest
> -nodefaults -display none -accel qtest -drive
> if=virtio,id=drive0,file=blkdebug::/home/petmay01/linaro/qemu-for-merges/build/alldbg/tests/qemu-iotests/scratch/test.img,format=qcow2,cache=writeback,aio=threads,backing.node-name=mid,backing.backing.node-name=base
> 
> I had to manually kill the offending QEMU process; resulting
> output in the log:
> 
> --- /home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/030.out
>  2019-07-15 17:18:35.251364738 +01
> 00
> +++ /home/petmay01/linaro/qemu-for-merges/build/alldbg/tests/qemu-iotests/030.out.bad
>   2020-03-12 13:44:
> 43.101182680 +0000
> @@ -1,5 +1,27 @@
> -...........................
> +........................E..
> +======================================================================
> +ERROR: test_stream_pause (__main__.TestSingleDrive)
> +----------------------------------------------------------------------
> +Traceback (most recent call last):
> +  File "030", line 93, in test_stream_pause
> +    self.pause_wait('drive0')
> +  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/iotests.py",
> line 927, in pause_wait
> +    result = self.vm.qmp('query-block-jobs')
> +  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/../../python/qemu/machine.py",
> line 405, in qmp
> +    return self._qmp.cmd(cmd, args=qmp_args)
> +  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/../../python/qemu/qmp.py",
> line 215, in cmd
> +    return self.cmd_obj(qmp_cmd)
> +  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/../../python/qemu/qmp.py",
> line 198, in cmd_obj
> +    resp = self.__json_read()
> +  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/../../python/qemu/qmp.py",
> line 89, in __json_read
> +    data = self.__sockfile.readline()
> +  File "/usr/lib/python3.6/socket.py", line 586, in readinto
> +    return self._sock.recv_into(b)
> +  File "/home/petmay01/linaro/qemu-for-merges/tests/qemu-iotests/iotests.py",
> line 383, in timeout
> +    raise Exception(self.errmsg)
> +Exception: Timeout waiting for job to pause
> +
>  ----------------------------------------------------------------------

For the record (discussed on IRC):

This seems to be intermittent failure where a short timeout (1 second)
might have been too short under heavy load. That this results in a hang
is a test case bug, but it already exists on master.

Peter will retry the test later.

Kevin


Re: [PULL 00/13] Block layer patches
Posted by Peter Maydell 5 years ago
On Wed, 11 Mar 2020 at 15:42, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit ba29883206d92a29ad5a466e679ccfc2ee6132ef:
>
>   Merge remote-tracking branch 'remotes/borntraeger/tags/s390x-20200310' into staging (2020-03-10 16:50:28 +0000)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 8bb3b023f2055054ee119cb45b42d2b14be7fc8a:
>
>   qemu-iotests: adding LUKS cleanup for non-UTF8 secret error (2020-03-11 15:54:38 +0100)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - Relax restrictions for blockdev-snapshot (allows libvirt to do live
>   storage migration with blockdev-mirror)
> - luks: Delete created files when block_crypto_co_create_opts_luks fails
> - Fix memleaks in qmp_object_add


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.0
for any user-visible changes.

-- PMM