The following changes since commit ba29883206d92a29ad5a466e679ccfc2ee6132ef:

Merge remote-tracking branch 'remotes/borntraeger/tags/s390x-20200310' into staging (2020-03-10 16:50:28 +0000)

git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to 8bb3b023f2055054ee119cb45b42d2b14be7fc8a:

qemu-iotests: adding LUKS cleanup for non-UTF8 secret error (2020-03-11 15:54:38 +0100)

Block layer patches:

- Relax restrictions for blockdev-snapshot (allows libvirt to do live

storage migration with blockdev-mirror)

- luks: Delete created files when block_crypto_co_create_opts_luks fails

- Fix memleaks in qmp_object_add

Daniel Henrique Barboza (4):

block: introducing 'bdrv_co_delete_file' interface

block.c: adding bdrv_co_delete_file

crypto.c: cleanup created file when block_crypto_co_create_opts_luks fails

qemu-iotests: adding LUKS cleanup for non-UTF8 secret error

Kevin Wolf (6):

block: Make bdrv_get_cumulative_perm() public

block: Relax restrictions for blockdev-snapshot

iotests: Fix run_job() with use_log=False

iotests: Test mirror with temporarily disabled target backing file

block: Fix cross-AioContext blockdev-snapshot

iotests: Add iothread cases to 155

Pan Nengyuan (1):

qom-qmp-cmds: fix two memleaks in qmp_object_add

Peter Krempa (1):

qapi: Add '@allow-write-only-overlay' feature for 'blockdev-snapshot'

Philippe Mathieu-Daudé (1):

tests/qemu-iotests: Fix socket_scm_helper build path

qapi/block-core.json | 9 ++++-

include/block/block.h | 1 +

include/block/block_int.h | 7 ++++

block.c | 33 ++++++++++++++--

block/crypto.c | 18 +++++++++

block/file-posix.c | 23 +++++++++++

blockdev.c | 30 ++++-----------

qom/qom-qmp-cmds.c | 16 +++-----

tests/qemu-iotests/iotests.py | 5 ++-

tests/Makefile.include | 1 +

tests/qemu-iotests/085.out | 4 +-

tests/qemu-iotests/155 | 88 ++++++++++++++++++++++++++++++++++++-------

tests/qemu-iotests/155.out | 4 +-

tests/qemu-iotests/282 | 67 ++++++++++++++++++++++++++++++++

tests/qemu-iotests/282.out | 11 ++++++

tests/qemu-iotests/group | 1 +

tests/qtest/Makefile.include | 1 -

17 files changed, 262 insertions(+), 57 deletions(-)

create mode 100755 tests/qemu-iotests/282

create mode 100644 tests/qemu-iotests/282.out

From: Daniel Henrique Barboza <danielhb413@gmail.com>

When using a non-UTF8 secret to create a volume using qemu-img, the

following error happens:

$ qemu-img create -f luks --object secret,id=vol_1_encrypt0,file=vol_resize_pool.vol_1.secret.qzVQrI -o key-secret=vol_1_encrypt0 /var/tmp/pool_target/vol_1 10240K

Formatting '/var/tmp/pool_target/vol_1', fmt=luks size=10485760 key-secret=vol_1_encrypt0

qemu-img: /var/tmp/pool_target/vol_1: Data from secret vol_1_encrypt0 is not valid UTF-8

However, the created file '/var/tmp/pool_target/vol_1' is left behind in the

file system after the failure. This behavior can be observed when creating

the volume using Libvirt, via 'virsh vol-create', and then getting "volume

target path already exist" errors when trying to re-create the volume.

The volume file is created inside block_crypto_co_create_opts_luks(), in

block/crypto.c. If the bdrv_create_file() call is successful but any

succeeding step fails*, the existing 'fail' label does not take into

account the created file, leaving it behind.

This patch changes block_crypto_co_create_opts_luks() to delete

'filename' in case of failure. A failure in this point means that

the volume is now truncated/corrupted, so even if 'filename' was an

existing volume before calling qemu-img, it is now unusable. Deleting

the file it is not much worse than leaving it in the filesystem in

this scenario, and we don't have to deal with checking the file

pre-existence in the code.

* in our case, block_crypto_co_create_generic calls qcrypto_block_create,

which calls qcrypto_block_luks_create, and this function fails when

calling qcrypto_secret_lookup_as_utf8.

Reported-by: Srikanth Aithal <bssrikanth@in.ibm.com>

Suggested-by: Kevin Wolf <kwolf@redhat.com>

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>

Message-Id: <20200130213907.2830642-4-danielhb413@gmail.com>

block/crypto.c | 18 ++++++++++++++++++

1 file changed, 18 insertions(+)

diff --git a/block/crypto.c b/block/crypto.c

--- a/block/crypto.c

+++ b/block/crypto.c

@@ -XXX,XX +XXX,XX @@

#include "qapi/error.h"

#include "qemu/module.h"

#include "qemu/option.h"

+#include "qemu/cutils.h"

#include "crypto.h"

typedef struct BlockCrypto BlockCrypto;

@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_crypto_co_create_opts_luks(const char *filename,

ret = 0;

fail:

+ /*

+ * If an error occurred, delete 'filename'. Even if the file existed

+ * beforehand, it has been truncated and corrupted in the process.

+ */

+ if (ret && bs) {

+ Error *local_delete_err = NULL;

+ int r_del = bdrv_co_delete_file(bs, &local_delete_err);

+ /*

+ * ENOTSUP will happen if the block driver doesn't support

+ * the 'bdrv_co_delete_file' interface. This is a predictable

+ * scenario and shouldn't be reported back to the user.

+ */

+ if ((r_del < 0) && (r_del != -ENOTSUP)) {

+ error_report_err(local_delete_err);

+ }

+

bdrv_unref(bs);

qapi_free_QCryptoBlockCreateOptions(create_opts);

qobject_unref(cryptoopts);

2.20.1

From: Daniel Henrique Barboza <danielhb413@gmail.com>

This patch adds a new test file to exercise the case where

qemu-img fails to complete for the LUKS format when a non-UTF8

secret is used.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>

Message-Id: <20200130213907.2830642-5-danielhb413@gmail.com>

tests/qemu-iotests/282 | 67 ++++++++++++++++++++++++++++++++++++++

tests/qemu-iotests/282.out | 11 +++++++

tests/qemu-iotests/group | 1 +

3 files changed, 79 insertions(+)

create mode 100755 tests/qemu-iotests/282

create mode 100644 tests/qemu-iotests/282.out

diff --git a/tests/qemu-iotests/282 b/tests/qemu-iotests/282

+++ b/tests/qemu-iotests/282

+# Test qemu-img file cleanup for LUKS when using a non-UTF8 secret

+# Copyright (C) 2020, IBM Corporation.

+seq=`basename $0`

+ _cleanup_test_img

+ rm non_utf8_secret

+ rm -f $TEST_IMAGE_FILE

+. ./common.rc

+. ./common.filter

+_supported_fmt luks

+_supported_proto generic

+_unsupported_proto vxhs

+echo "== Create non-UTF8 secret =="

+echo -n -e '\x3a\x3c\x3b\xff' > non_utf8_secret

+SECRET="secret,id=sec0,file=non_utf8_secret"

+echo "== Throws an error because of invalid UTF-8 secret =="

+$QEMU_IMG create -f $IMGFMT --object $SECRET -o "key-secret=sec0" $TEST_IMAGE_FILE 4M

+echo "== Image file should not exist after the error =="

+if test -f "$TEST_IMAGE_FILE"; then

+ exit 1

+fi

+echo "== Create a stub image file and run qemu-img again =="

+touch $TEST_IMAGE_FILE

+$QEMU_IMG create -f $IMGFMT --object $SECRET -o "key-secret=sec0" $TEST_IMAGE_FILE 4M

+echo "== Pre-existing image file should also be deleted after the error =="

+if test -f "$TEST_IMAGE_FILE"; then

+ exit 1

+fi

diff --git a/tests/qemu-iotests/282.out b/tests/qemu-iotests/282.out

+++ b/tests/qemu-iotests/282.out

+QA output created by 282

+== Create non-UTF8 secret ==

+== Throws an error because of invalid UTF-8 secret ==

+qemu-img: vol.img: Data from secret sec0 is not valid UTF-8

+Formatting 'vol.img', fmt=luks size=4194304 key-secret=sec0

+== Image file should not exist after the error ==

+== Create a stub image file and run qemu-img again ==

+qemu-img: vol.img: Data from secret sec0 is not valid UTF-8

+Formatting 'vol.img', fmt=luks size=4194304 key-secret=sec0

+== Pre-existing image file should also be deleted after the error ==

+ *** done

diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/group

+++ b/tests/qemu-iotests/group

@@ -XXX,XX +XXX,XX @@

279 rw backing quick

280 rw migration quick

281 rw quick

+282 rw img quick

283 auto quick

284 rw

286 rw quick

2.20.1

From: Daniel Henrique Barboza <danielhb413@gmail.com>

Using the new 'bdrv_co_delete_file' interface, a pure co_routine function

'bdrv_co_delete_file' inside block.c can can be used in a way similar of

the existing bdrv_create_file to to clean up a created file.

We're creating a pure co_routine because the only caller of

'bdrv_co_delete_file' will be already in co_routine context, thus there

is no need to add all the machinery to check for qemu_in_coroutine() and

create a separated co_routine to do the job.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>

Message-Id: <20200130213907.2830642-3-danielhb413@gmail.com>

include/block/block.h | 1 +

block.c | 26 ++++++++++++++++++++++++++

2 files changed, 27 insertions(+)

diff --git a/include/block/block.h b/include/block/block.h

--- a/include/block/block.h

+++ b/include/block/block.h

@@ -XXX,XX +XXX,XX @@ bool bdrv_is_backing_chain_frozen(BlockDriverState *bs, BlockDriverState *base,

int bdrv_freeze_backing_chain(BlockDriverState *bs, BlockDriverState *base,

Error **errp);

void bdrv_unfreeze_backing_chain(BlockDriverState *bs, BlockDriverState *base);

+int coroutine_fn bdrv_co_delete_file(BlockDriverState *bs, Error **errp);

typedef struct BdrvCheckResult {

diff --git a/block.c b/block.c

index XXXXXXX..XXXXXXX 100644

--- a/block.c

+++ b/block.c

@@ -XXX,XX +XXX,XX @@ int bdrv_create_file(const char *filename, QemuOpts *opts, Error **errp)

}

}

+int coroutine_fn bdrv_co_delete_file(BlockDriverState *bs, Error **errp)

+{

+ Error *local_err = NULL;

+ int ret;

+

+ assert(bs != NULL);

+

+ if (!bs->drv) {

+ error_setg(errp, "Block node '%s' is not opened", bs->filename);

+ return -ENOMEDIUM;

+ if (!bs->drv->bdrv_co_delete_file) {

+ error_setg(errp, "Driver '%s' does not support image deletion",

+ bs->drv->format_name);

+ return -ENOTSUP;

+ }

+

+ ret = bs->drv->bdrv_co_delete_file(bs, &local_err);

+ if (ret < 0) {

+ error_propagate(errp, local_err);

+ }

+

+ return ret;

+}

+

/**

* Try to get @bs's logical and physical block size.

* On success, store them in @bsz struct and return 0.

2.20.1

From: Daniel Henrique Barboza <danielhb413@gmail.com>

Adding to Block Drivers the capability of being able to clean up

its created files can be useful in certain situations. For the

LUKS driver, for instance, a failure in one of its authentication

steps can leave files in the host that weren't there before.

This patch adds the 'bdrv_co_delete_file' interface to block

drivers and add it to the 'file' driver in file-posix.c. The

implementation is given by 'raw_co_delete_file'.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>

Message-Id: <20200130213907.2830642-2-danielhb413@gmail.com>

include/block/block_int.h | 4 ++++

block/file-posix.c | 23 +++++++++++++++++++++++

2 files changed, 27 insertions(+)

diff --git a/include/block/block_int.h b/include/block/block_int.h

--- a/include/block/block_int.h

+++ b/include/block/block_int.h

@@ -XXX,XX +XXX,XX @@ struct BlockDriver {

*/

int coroutine_fn (*bdrv_co_flush)(BlockDriverState *bs);

+ /* Delete a created file. */

+ int coroutine_fn (*bdrv_co_delete_file)(BlockDriverState *bs,

+ Error **errp);

/*

* Flushes all data that was already written to the OS all the way down to

* the disk (for example file-posix.c calls fsync()).

diff --git a/block/file-posix.c b/block/file-posix.c

--- a/block/file-posix.c

+++ b/block/file-posix.c

@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_create_opts(const char *filename, QemuOpts *opts,

return raw_co_create(&options, errp);

+static int coroutine_fn raw_co_delete_file(BlockDriverState *bs,

+ Error **errp)

+ struct stat st;

+ int ret;

+

+ if (!(stat(bs->filename, &st) == 0) || !S_ISREG(st.st_mode)) {

+ error_setg_errno(errp, ENOENT, "%s is not a regular file",

+ bs->filename);

+ return -ENOENT;

+ }

+

+ ret = unlink(bs->filename);

+ if (ret < 0) {

+ ret = -errno;

+ error_setg_errno(errp, -ret, "Error when deleting file %s",

+ bs->filename);

+ }

+

+ return ret;

/*

* Find allocation range in @bs around offset @start.

* May change underlying file descriptor's file offset.

@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_file = {