[PATCH v2 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

Max Reitz posted 4 patches 4 years ago
Test docker-mingw@fedora failed
Test checkpatch passed
Test asan failed
Test docker-quick@centos7 failed
Test FreeBSD passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200429141126.85159-1-mreitz@redhat.com
Maintainers: Max Reitz <mreitz@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Xie Changlong <xiechanglong.d@gmail.com>, John Snow <jsnow@redhat.com>, Wen Congyang <wencongyang2@huawei.com>
include/block/block.h          |  1 +
include/sysemu/block-backend.h |  2 ++
block.c                        | 23 +++++++++++++++++++++++
block/block-backend.c          | 10 ++++++++++
block/commit.c                 | 16 +++++++++-------
block/replication.c            |  6 ++----
block/vvfat.c                  |  4 +---
qemu-img.c                     | 19 ++++++++++++++-----
8 files changed, 62 insertions(+), 19 deletions(-)
[PATCH v2 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly
Posted by Max Reitz 4 years ago
v1: https://lists.nongnu.org/archive/html/qemu-block/2020-04/msg01245.html

Branch: https://github.com/XanClic/qemu.git fix-bdrv_make_empty-v2
Branch: https://git.xanclic.moe/XanClic/qemu.git fix-bdrv_make_empty-v2

Based-on: <20200428192648.749066-1-eblake@redhat.com>
          (“qcow2: Allow resize of images with internal snapshots”)

Hi,

As described in v1’s cover letter (linked above), this series ensures
that all calls to BlockDriver.bdrv_make_empty() go through a wrapper
bdrv_make_empty() function that ensures the caller does have the
necessary permissions.


Changes in v2 (thanks for the quick reviews, I didn’t expect this series
to get attention so quickly :)):
- Added Based-on here in the cover letter [Eric]
- Patch 1: WRITE_UNCHANGED is sufficient [Kevin]
- Patch 3: Check whether blk->root is actually present with
           blk_is_available() [Kevin]
- Patch 4: Let bdrv_commit() only take the WRITE_UNCHANGED permission,
           and take it from the moment the @src BB is created and @bs is
           inserted [Kevin];
           then drop the drv->bdrv_make_empty check, just call
           blk_make_empty() and ignore -ENOTSUP [Eric]


git-backport-diff against v1:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/4:[0002] [FC] 'block: Add bdrv_make_empty()'
002/4:[----] [--] 'block: Use bdrv_make_empty() where possible'
003/4:[0005] [FC] 'block: Add blk_make_empty()'
004/4:[0020] [FC] 'block: Use blk_make_empty() after commits'


Max Reitz (4):
  block: Add bdrv_make_empty()
  block: Use bdrv_make_empty() where possible
  block: Add blk_make_empty()
  block: Use blk_make_empty() after commits

 include/block/block.h          |  1 +
 include/sysemu/block-backend.h |  2 ++
 block.c                        | 23 +++++++++++++++++++++++
 block/block-backend.c          | 10 ++++++++++
 block/commit.c                 | 16 +++++++++-------
 block/replication.c            |  6 ++----
 block/vvfat.c                  |  4 +---
 qemu-img.c                     | 19 ++++++++++++++-----
 8 files changed, 62 insertions(+), 19 deletions(-)

-- 
2.25.4


Re: [PATCH v2 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly
Posted by no-reply@patchew.org 4 years ago
Patchew URL: https://patchew.org/QEMU/20200429141126.85159-1-mreitz@redhat.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

  BUILD   pc-bios/optionrom/linuxboot_dma.raw
  BUILD   pc-bios/optionrom/kvmvapic.raw
  LINK    qemu-edid
/tmp/qemu-test/src/qemu-img.c:1071:27: error: implicit declaration of function 'blk_new_with_bs' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
                          ^
/tmp/qemu-test/src/qemu-img.c:1071:27: error: this function declaration is not a prototype [-Werror,-Wstrict-prototypes]
/tmp/qemu-test/src/qemu-img.c:1071:25: error: incompatible integer to pointer conversion assigning to 'BlockBackend *' (aka 'struct BlockBackend *') from 'int' [-Werror,-Wint-conversion]
        old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
                        ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 errors generated.
---
  SIGN    pc-bios/optionrom/linuxboot.bin
  SIGN    pc-bios/optionrom/linuxboot_dma.bin
  SIGN    pc-bios/optionrom/kvmvapic.bin
make: *** [/tmp/qemu-test/src/rules.mak:69: qemu-img.o] Error 1
make: *** Waiting for unfinished jobs....
  BUILD   pc-bios/optionrom/pvh.img
  BUILD   pc-bios/optionrom/pvh.raw
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=d13b024a2281417b9c8a863d2bbcfec3', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-j487_kuh/src/docker-src.2020-04-29-15.07.32.27231:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=d13b024a2281417b9c8a863d2bbcfec3
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-j487_kuh/src'
make: *** [docker-run-test-debug@fedora] Error 2

real    3m51.019s
user    0m7.947s


The full log is available at
http://patchew.org/logs/20200429141126.85159-1-mreitz@redhat.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v2 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly
Posted by no-reply@patchew.org 4 years ago
Patchew URL: https://patchew.org/QEMU/20200429141126.85159-1-mreitz@redhat.com/



Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

  LINK    qemu-edid.exe
  LINK    qemu-ga.exe
/tmp/qemu-test/src/qemu-img.c: In function 'img_commit':
/tmp/qemu-test/src/qemu-img.c:1071:27: error: implicit declaration of function 'blk_new_with_bs'; did you mean 'blk_get_stats'? [-Werror=implicit-function-declaration]
         old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
                           ^~~~~~~~~~~~~~~
                           blk_get_stats
/tmp/qemu-test/src/qemu-img.c:1071:27: error: nested extern declaration of 'blk_new_with_bs' [-Werror=nested-externs]
/tmp/qemu-test/src/qemu-img.c:1071:25: error: assignment to 'BlockBackend *' {aka 'struct BlockBackend *'} from 'int' makes pointer from integer without a cast [-Werror=int-conversion]
         old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
                         ^
cc1: all warnings being treated as errors
make: *** [/tmp/qemu-test/src/rules.mak:69: qemu-img.o] Error 1
make: *** Waiting for unfinished jobs....
  GEN     x86_64-softmmu/hmp-commands.h
  GEN     x86_64-softmmu/hmp-commands-info.h
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=e0a16b2399774bccbf7cf068081a9efd', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-etm7bld6/src/docker-src.2020-04-29-15.12.17.3885:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=e0a16b2399774bccbf7cf068081a9efd
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-etm7bld6/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real    2m47.092s
user    0m7.752s


The full log is available at
http://patchew.org/logs/20200429141126.85159-1-mreitz@redhat.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v2 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly
Posted by no-reply@patchew.org 4 years ago
Patchew URL: https://patchew.org/QEMU/20200429141126.85159-1-mreitz@redhat.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

  LINK    qemu-nbd
  LINK    qemu-storage-daemon
  LINK    qemu-io
/tmp/qemu-test/src/qemu-img.c:1071:27: error: implicit declaration of function 'blk_new_with_bs' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
                          ^
/tmp/qemu-test/src/qemu-img.c:1071:27: error: this function declaration is not a prototype [-Werror,-Wstrict-prototypes]
/tmp/qemu-test/src/qemu-img.c:1071:25: error: incompatible integer to pointer conversion assigning to 'BlockBackend *' (aka 'struct BlockBackend *') from 'int' [-Werror,-Wint-conversion]
        old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
                        ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 errors generated.
make: *** [/tmp/qemu-test/src/rules.mak:69: qemu-img.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 664, in <module>
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=0c317bc0372e46baa5e530cb5bffad05', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-mhzkjux4/src/docker-src.2020-04-29-19.38.18.14298:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=0c317bc0372e46baa5e530cb5bffad05
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-mhzkjux4/src'
make: *** [docker-run-test-debug@fedora] Error 2

real    3m19.927s
user    0m8.122s


The full log is available at
http://patchew.org/logs/20200429141126.85159-1-mreitz@redhat.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v2 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly
Posted by Kevin Wolf 3 years, 11 months ago
Am 29.04.2020 um 16:11 hat Max Reitz geschrieben:
> v1: https://lists.nongnu.org/archive/html/qemu-block/2020-04/msg01245.html
> 
> Branch: https://github.com/XanClic/qemu.git fix-bdrv_make_empty-v2
> Branch: https://git.xanclic.moe/XanClic/qemu.git fix-bdrv_make_empty-v2
> 
> Based-on: <20200428192648.749066-1-eblake@redhat.com>
>           (“qcow2: Allow resize of images with internal snapshots”)
> 
> Hi,
> 
> As described in v1’s cover letter (linked above), this series ensures
> that all calls to BlockDriver.bdrv_make_empty() go through a wrapper
> bdrv_make_empty() function that ensures the caller does have the
> necessary permissions.

Thanks, fixed up the test output in patch 4 and applied to the block
branch.

Kevin


Re: [PATCH v2 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly
Posted by Kevin Wolf 3 years, 11 months ago
Am 14.05.2020 um 15:08 hat Kevin Wolf geschrieben:
> Am 29.04.2020 um 16:11 hat Max Reitz geschrieben:
> > v1: https://lists.nongnu.org/archive/html/qemu-block/2020-04/msg01245.html
> > 
> > Branch: https://github.com/XanClic/qemu.git fix-bdrv_make_empty-v2
> > Branch: https://git.xanclic.moe/XanClic/qemu.git fix-bdrv_make_empty-v2
> > 
> > Based-on: <20200428192648.749066-1-eblake@redhat.com>
> >           (“qcow2: Allow resize of images with internal snapshots”)
> > 
> > Hi,
> > 
> > As described in v1’s cover letter (linked above), this series ensures
> > that all calls to BlockDriver.bdrv_make_empty() go through a wrapper
> > bdrv_make_empty() function that ensures the caller does have the
> > necessary permissions.
> 
> Thanks, fixed up the test output in patch 4 and applied to the block
> branch.

Hmm, replication is doing criminal things and this results in:

test-replication: block.c:6899: bdrv_make_empty: Assertion `c->perm & (BLK_PERM_WRITE | BLK_PERM_WRITE_UNCHANGED)' failed.

Not your bug, but it breaks 'make check', so it needs to be fixed before
I can send a pull request. I'll see what I can do...

Kevin


Re: [PATCH v2 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly
Posted by no-reply@patchew.org 4 years ago
Patchew URL: https://patchew.org/QEMU/20200429141126.85159-1-mreitz@redhat.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

  BUILD   pc-bios/optionrom/pvh.raw
  SIGN    pc-bios/optionrom/pvh.bin
/tmp/qemu-test/src/qemu-img.c: In function 'img_commit':
/tmp/qemu-test/src/qemu-img.c:1071:9: error: implicit declaration of function 'blk_new_with_bs' [-Werror=implicit-function-declaration]
         old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
         ^
/tmp/qemu-test/src/qemu-img.c:1071:9: error: nested extern declaration of 'blk_new_with_bs' [-Werror=nested-externs]
/tmp/qemu-test/src/qemu-img.c:1071:25: error: assignment makes pointer from integer without a cast [-Werror]
         old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
                         ^
cc1: all warnings being treated as errors
make: *** [qemu-img.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** wait: No child processes.  Stop.
Traceback (most recent call last):
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=70ea0da52ca941ecafb5efd2e18dfe6a', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-2faaaxl7/src/docker-src.2020-04-29-19.35.24.5998:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=70ea0da52ca941ecafb5efd2e18dfe6a
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-2faaaxl7/src'
make: *** [docker-run-test-quick@centos7] Error 2

real    2m24.697s
user    0m7.953s


The full log is available at
http://patchew.org/logs/20200429141126.85159-1-mreitz@redhat.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v2 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly
Posted by no-reply@patchew.org 4 years ago
Patchew URL: https://patchew.org/QEMU/20200429141126.85159-1-mreitz@redhat.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

  LINK    qemu-storage-daemon
  LINK    qemu-io
/tmp/qemu-test/src/qemu-img.c: In function 'img_commit':
/tmp/qemu-test/src/qemu-img.c:1071:9: error: implicit declaration of function 'blk_new_with_bs' [-Werror=implicit-function-declaration]
         old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
         ^
/tmp/qemu-test/src/qemu-img.c:1071:9: error: nested extern declaration of 'blk_new_with_bs' [-Werror=nested-externs]
/tmp/qemu-test/src/qemu-img.c:1071:25: error: assignment makes pointer from integer without a cast [-Werror]
         old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
                         ^
cc1: all warnings being treated as errors
make: *** [qemu-img.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** wait: No child processes.  Stop.
Traceback (most recent call last):
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=54c1e8d7f38d491c879a9f7fd70b93fd', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-j98lpee6/src/docker-src.2020-04-29-15.04.21.16547:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=54c1e8d7f38d491c879a9f7fd70b93fd
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-j98lpee6/src'
make: *** [docker-run-test-quick@centos7] Error 2

real    2m41.192s
user    0m8.095s


The full log is available at
http://patchew.org/logs/20200429141126.85159-1-mreitz@redhat.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com