The following changes since commit 9964e96dc9999cf7f7c936ee854a795415d19b60:

Merge remote-tracking branch 'jasowang/tags/net-pull-request' into staging (2017-05-23 15:01:31 +0100)

for you to fetch changes up to 42a48128417b3bfade93d1a4721348cc480e9e50:

Merge remote-tracking branch 'mreitz/tags/pull-block-2017-05-29-v3' into queue-block (2017-05-29 16:34:27 +0200)

Alberto Garcia (2):

stream: fix crash in stream_start() when block_job_create() fails

qcow2: remove extra local_error variable

Daniel P. Berrange (4):

qemu-img: add support for --object with 'dd' command

qemu-img: fix --image-opts usage with dd command

qemu-img: introduce --target-image-opts for 'convert' command

qemu-img: copy *key-secret opts when opening newly created files

Eric Blake (1):

block: Tweak error message related to qemu-img amend

Fam Zheng (3):

iotests: 147: Don't test inet6 if not available

qemu-img: Fix documentation of convert

qemu-img: Fix leakage of options on error

Kevin Wolf (3):

qemu-iotests: Test streaming with missing job ID

mirror: Drop permissions on s->target on completion

Merge remote-tracking branch 'mreitz/tags/pull-block-2017-05-29-v3' into queue-block

Max Reitz (2):

block: Fix backing paths for filenames with colons

block/file-*: *_parse_filename() and colons

Stephen Bates (1):

nvme: Add support for Controller Memory Buffers

This fixes an assertion failure that was triggered by qemu-iotests 129

on some CI host, while the same test case didn't seem to fail on other

hosts.

Essentially the problem is that the blk_unref(s->target) in

mirror_exit() doesn't necessarily mean that the BlockBackend goes away

immediately. It is possible that the job completion was triggered nested

in mirror_drain(), which looks like this:

BlockBackend *target = s->target;

blk_ref(target);

blk_drain(target);

blk_unref(target);

In this case, the write permissions for s->target are retained until

after blk_drain(), which makes removing mirror_top_bs fail for the

active commit case (can't have a writable backing file in the chain

without the filter driver).

Explicitly dropping the permissions first means that the additional

reference doesn't hurt and the job can complete successfully even if

called from the nested blk_drain().

Acked-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Max Reitz <mreitz@redhat.com>

block/mirror.c | 7 ++++++-

1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/block/mirror.c b/block/mirror.c

--- a/block/mirror.c

+++ b/block/mirror.c

@@ -XXX,XX +XXX,XX @@ static void mirror_exit(BlockJob *job, void *opaque)

/* Remove target parent that still uses BLK_PERM_WRITE/RESIZE before

* inserting target_bs at s->to_replace, where we might not be able to get

- * these permissions. */

+ * these permissions.

+ *

+ * Note that blk_unref() alone doesn't necessarily drop permissions because

+ * we might be running nested inside mirror_drain(), which takes an extra

+ * reference, so use an explicit blk_set_perm() first. */

+ blk_set_perm(s->target, 0, BLK_PERM_ALL, &error_abort);

blk_unref(s->target);

s->target = NULL;

From: "Daniel P. Berrange" <berrange@redhat.com>

The qemu-img dd/convert commands will create an image file and

then try to open it. Historically it has been possible to open

new files without passing any options. With encrypted files

though, the *key-secret options are mandatory, so we need to

provide those options when opening the newly created file.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

Message-id: 20170515164712.6643-5-berrange@redhat.com

Reviewed-by: Max Reitz <mreitz@redhat.com>

Signed-off-by: Max Reitz <mreitz@redhat.com>

qemu-img.c | 42 +++++++++++++++++++++++++++++++++++++-----

1 file changed, 37 insertions(+), 5 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c

--- a/qemu-img.c

+++ b/qemu-img.c

@@ -XXX,XX +XXX,XX @@ static BlockBackend *img_open_opts(const char *optstr,

static BlockBackend *img_open_file(const char *filename,

+ QDict *options,

const char *fmt, int flags,

bool writethrough, bool quiet,

bool force_share)

BlockBackend *blk;

Error *local_err = NULL;

- QDict *options = qdict_new();

+ if (!options) {

+ options = qdict_new();

+ }

if (fmt) {

qdict_put_str(options, "driver", fmt);

@@ -XXX,XX +XXX,XX @@ static BlockBackend *img_open_file(const char *filename,

+static int img_add_key_secrets(void *opaque,

+ const char *name, const char *value,

+ Error **errp)

+{

+ QDict *options = opaque;

+

+ if (g_str_has_suffix(name, "key-secret")) {

+ qdict_put(options, name, qstring_from_str(value));

+ }

+

+ return 0;

+}

+

+static BlockBackend *img_open_new_file(const char *filename,

+ QemuOpts *create_opts,

+ const char *fmt, int flags,

+ bool writethrough, bool quiet,

+ bool force_share)

+{

+ QDict *options = NULL;

+

+ options = qdict_new();

+ qemu_opt_foreach(create_opts, img_add_key_secrets, options, &error_abort);

+

+ return img_open_file(filename, options, fmt, flags, writethrough, quiet,

+ force_share);

+}

+

+

static BlockBackend *img_open(bool image_opts,

const char *filename,

const char *fmt, int flags, bool writethrough,

@@ -XXX,XX +XXX,XX @@ static BlockBackend *img_open(bool image_opts,

blk = img_open_opts(filename, opts, flags, writethrough, quiet,

force_share);

} else {

- blk = img_open_file(filename, fmt, flags, writethrough, quiet,

+ blk = img_open_file(filename, NULL, fmt, flags, writethrough, quiet,

force_share);

}

return blk;

@@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv)

* That has to wait for bdrv_create to be improved

* to allow filenames in option syntax

*/

- s.target = img_open_file(out_filename, out_fmt, flags,

- writethrough, quiet, false);

+ s.target = img_open_new_file(out_filename, opts, out_fmt,

+ flags, writethrough, quiet, false);

}

if (!s.target) {

ret = -1;

@@ -XXX,XX +XXX,XX @@ static int img_dd(int argc, char **argv)

* with the bdrv_create() call above which does not

* support image-opts style.

*/

- blk2 = img_open_file(out.filename, out_fmt, BDRV_O_RDWR,

+ blk2 = img_open_file(out.filename, NULL, out_fmt, BDRV_O_RDWR,

false, false, false);

if (!blk2) {

From: Alberto Garcia <berto@igalia.com>

The code that tries to reopen a BlockDriverState in stream_start()

when the creation of a new block job fails crashes because it attempts

to dereference a pointer that is known to be NULL.

This is a regression introduced in a170a91fd3eab6155da39e740381867e,

likely because the code was copied from stream_complete().

Cc: qemu-stable@nongnu.org

Reported-by: Kashyap Chamarthy <kchamart@redhat.com>

Signed-off-by: Alberto Garcia <berto@igalia.com>

Tested-by: Kashyap Chamarthy <kchamart@redhat.com>

block/stream.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/stream.c b/block/stream.c

--- a/block/stream.c

+++ b/block/stream.c

@@ -XXX,XX +XXX,XX @@ void stream_start(const char *job_id, BlockDriverState *bs,

if (orig_bs_flags != bdrv_get_flags(bs)) {

- bdrv_reopen(bs, s->bs_flags, NULL);

+ bdrv_reopen(bs, orig_bs_flags, NULL);

}

}

From: Eric Blake <eblake@redhat.com>

When converting a 1.1 image down to 0.10, qemu-iotests 060 forces

a contrived failure where allocating a cluster used to replace a

zero cluster reads unaligned data. Since it is a zero cluster

rather than a data cluster being converted, changing the error

message to match our earlier change in 'qcow2: Make distinction

between zero cluster types obvious' is worthwhile.

Suggested-by: Max Reitz <mreitz@redhat.com>

Signed-off-by: Eric Blake <eblake@redhat.com>

Message-id: 20170508171302.17805-1-eblake@redhat.com

[mreitz: Commit message fixes]

Signed-off-by: Max Reitz <mreitz@redhat.com>

block/qcow2-cluster.c | 3 ++-

tests/qemu-iotests/060.out | 2 +-

2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c

--- a/block/qcow2-cluster.c

+++ b/block/qcow2-cluster.c

@@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,

}

if (offset_into_cluster(s, offset)) {

- qcow2_signal_corruption(bs, true, -1, -1, "Data cluster offset "

+ qcow2_signal_corruption(bs, true, -1, -1,

+ "Cluster allocation offset "

"%#" PRIx64 " unaligned (L2 offset: %#"

PRIx64 ", L2 index: %#x)", offset,

l2_offset, j);

diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out

--- a/tests/qemu-iotests/060.out

+++ b/tests/qemu-iotests/060.out

@@ -XXX,XX +XXX,XX @@ read failed: Input/output error

Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864

wrote 65536/65536 bytes at offset 0

64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

-qcow2: Marking image as corrupt: Data cluster offset 0x52a00 unaligned (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed

+qcow2: Marking image as corrupt: Cluster allocation offset 0x52a00 unaligned (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed

qemu-img: Error while amending options: Input/output error

=== Testing unaligned reftable entry ===

From: "Daniel P. Berrange" <berrange@redhat.com>

The qemu-img dd command added --image-opts support, but missed

the corresponding --object support. This prevented passing

secrets (eg auth passwords) needed by certain disk images.

Reviewed-by: Fam Zheng <famz@redhat.com>

Reviewed-by: Max Reitz <mreitz@redhat.com>

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

Message-id: 20170515164712.6643-2-berrange@redhat.com

Signed-off-by: Max Reitz <mreitz@redhat.com>

qemu-img.c | 18 ++++++++++++++++++

1 file changed, 18 insertions(+)

diff --git a/qemu-img.c b/qemu-img.c

--- a/qemu-img.c

+++ b/qemu-img.c

@@ -XXX,XX +XXX,XX @@ static int img_dd(int argc, char **argv)

};

const struct option long_options[] = {

{ "help", no_argument, 0, 'h'},

+ { "object", required_argument, 0, OPTION_OBJECT},

{ "image-opts", no_argument, 0, OPTION_IMAGE_OPTS},

{ "force-share", no_argument, 0, 'U'},

{ 0, 0, 0, 0 }

@@ -XXX,XX +XXX,XX @@ static int img_dd(int argc, char **argv)

case 'U':

force_share = true;

break;

+ case OPTION_OBJECT: {

+ QemuOpts *opts;

+ opts = qemu_opts_parse_noisily(&qemu_object_opts,

+ optarg, true);

+ if (!opts) {

+ ret = -1;

+ goto out;

+ }

+ } break;

case OPTION_IMAGE_OPTS:

image_opts = true;

break;

@@ -XXX,XX +XXX,XX @@ static int img_dd(int argc, char **argv)

ret = -1;

goto out;

From: "Daniel P. Berrange" <berrange@redhat.com>

The --image-opts flag can only be used to affect the parsing

of the source image. The target image has to be specified in

the traditional style regardless, since it needs to be passed

to the bdrv_create() API which does not support the new style

opts.

Reviewed-by: Fam Zheng <famz@redhat.com>

Reviewed-by: Max Reitz <mreitz@redhat.com>

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

Message-id: 20170515164712.6643-3-berrange@redhat.com

Signed-off-by: Max Reitz <mreitz@redhat.com>

qemu-img.c | 9 +++++++--

1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c

--- a/qemu-img.c

+++ b/qemu-img.c

@@ -XXX,XX +XXX,XX @@ static int img_dd(int argc, char **argv)

goto out;

- blk2 = img_open(image_opts, out.filename, out_fmt, BDRV_O_RDWR,

- false, false, false);

+ /* TODO, we can't honour --image-opts for the target,

+ * since it needs to be given in a format compatible

+ * with the bdrv_create() call above which does not

+ * support image-opts style.

+ */

+ blk2 = img_open_file(out.filename, out_fmt, BDRV_O_RDWR,

+ false, false, false);

if (!blk2) {

ret = -1;

This adds a small test for the image streaming error path for failing

block_job_create(), which would have found the null pointer dereference

in commit a170a91f.

Reviewed-by: Alberto Garcia <berto@igalia.com>

Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

Reviewed-by: Jeff Cody <jcody@redhat.com>

tests/qemu-iotests/030 | 4 ++++

tests/qemu-iotests/030.out | 4 ++--

2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/030

+++ b/tests/qemu-iotests/030

@@ -XXX,XX +XXX,XX @@ class TestSingleDrive(iotests.QMPTestCase):

result = self.vm.qmp('block-stream', device='nonexistent')

self.assert_qmp(result, 'error/class', 'GenericError')

+ def test_job_id_missing(self):

+ result = self.vm.qmp('block-stream', device='mid')

+ self.assert_qmp(result, 'error/class', 'GenericError')

+

class TestParallelOps(iotests.QMPTestCase):

num_ops = 4 # Number of parallel block-stream operations

diff --git a/tests/qemu-iotests/030.out b/tests/qemu-iotests/030.out