The following changes since commit cfe6c547690b06fbce54a6d0f7b05dd7f18e36ea:

Merge remote-tracking branch 'remotes/xanclic/tags/pull-block-2019-01-31' into staging (2019-01-31 19:26:09 +0000)

git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to 7471a649fc3a391dd497297013fb2525ca9821ba:

scsi-disk: Add device_id property (2019-02-01 13:48:11 +0100)

Block layer patches:

- vmdk: Support for blockdev-create

- block: Apply auto-read-only for ro-whitelist drivers

- virtio-scsi: Fixes related to attaching/detaching iothreads

- scsi-disk: Fixed erroneously detected multipath setup with multiple

disks created with node-names. Added device_id property.

- block: Fix hangs in synchronous APIs with iothreads

- block: Fix invalidate_cache error path for parent activation

- block-backend, mirror, qcow2, vpc, vdi, qemu-iotests:

Minor fixes and code improvements

Alberto Garcia (7):

mirror: Release the dirty bitmap if mirror_start_job() fails

mirror: Block the source BlockDriverState in mirror_start_job()

qcow2: Assert that refcount block offsets fit in the refcount table

virtio-scsi: Move BlockBackend back to the main AioContext on unplug

scsi-disk: Acquire the AioContext in scsi_*_realize()

virtio-scsi: Forbid devices with different iothreads sharing a blockdev

qtest.py: Wait for the result of qtest commands

Fam Zheng (3):

vmdk: Refactor vmdk_create_extent

vmdk: Implement .bdrv_co_create callback

iotests: Filter cid numbers in VMDK extent info

John Snow (1):

iotests/236: fix transaction kwarg order

Kevin Wolf (7):

block: Fix hangs in synchronous APIs with iothreads

iotests: Add VMDK tests for blockdev-create

vmdk: Reject excess extents in blockdev-create

block: Apply auto-read-only for ro-whitelist drivers

block: Fix invalidate_cache error path for parent activation

scsi-disk: Don't use empty string as device id

scsi-disk: Add device_id property

Markus Armbruster (2):

block: Replace qdict_put() by qdict_put_obj() where appropriate

block: Eliminate the S_1KiB, S_2KiB, ... macros

Max Reitz (2):

iotests: Make 234 stable

iotests: Filter second BLOCK_JOB_ERROR from 229

Peter Maydell (3):

block/vpc: Don't take address of fields in packed structs

block/vdi: Don't take address of fields in packed structs

uuid: Make qemu_uuid_bswap() take and return a QemuUUID

Thomas Huth (1):

block: Remove blk_attach_dev_legacy() / legacy_dev code

yuchenlin (1):

qemu-iotests: add test case for dmg

qapi/block-core.json | 71 +++

qapi/qapi-schema.json | 16 +-

block/qcow2.h | 10 +-

include/qemu/units.h | 73 ---

include/qemu/uuid.h | 2 +-

include/sysemu/block-backend.h | 5 +-

block.c | 27 +-

block/blklogwrites.c | 5 +-

block/block-backend.c | 59 +--

block/io.c | 8 +-

block/mirror.c | 11 +

block/nbd-client.c | 1 +

block/nvme.c | 1 +

block/qcow2-refcount.c | 3 +

block/qcow2.c | 1 +

block/qed.c | 1 +

block/vdi.c | 57 ++-

block/vmdk.c | 532 +++++++++++++++------

block/vpc.c | 4 +-

hw/acpi/vmgenid.c | 6 +-

hw/scsi/scsi-disk.c | 59 ++-

hw/scsi/virtio-scsi.c | 13 +

tests/test-block-iothread.c | 372 ++++++++++++++

tests/vmgenid-test.c | 2 +-

util/uuid.c | 10 +-

scripts/qtest.py | 6 +

tests/Makefile.include | 2 +

tests/qemu-iotests/141.out | 4 +-

tests/qemu-iotests/229 | 6 +-

tests/qemu-iotests/229.out | 1 -

tests/qemu-iotests/234 | 56 ++-

tests/qemu-iotests/234.out | 10 +-

tests/qemu-iotests/236.out | 56 +--

tests/qemu-iotests/237 | 237 +++++++++

tests/qemu-iotests/237.out | 348 ++++++++++++++

tests/qemu-iotests/239 | 53 ++

tests/qemu-iotests/239.out | 4 +

tests/qemu-iotests/240 | 129 +++++

tests/qemu-iotests/240.out | 54 +++

tests/qemu-iotests/check | 7 +

tests/qemu-iotests/common.filter | 1 +

tests/qemu-iotests/group | 3 +

tests/qemu-iotests/iotests.py | 22 +-

.../qemu-iotests/sample_images/simple-dmg.dmg.bz2 | Bin 0 -> 3479 bytes

44 files changed, 1931 insertions(+), 417 deletions(-)

create mode 100644 tests/test-block-iothread.c

create mode 100755 tests/qemu-iotests/237

create mode 100644 tests/qemu-iotests/237.out

create mode 100755 tests/qemu-iotests/239

create mode 100644 tests/qemu-iotests/239.out

create mode 100755 tests/qemu-iotests/240

create mode 100644 tests/qemu-iotests/240.out

create mode 100644 tests/qemu-iotests/sample_images/simple-dmg.dmg.bz2

From: Fam Zheng <famz@redhat.com>

Signed-off-by: Fam Zheng <famz@redhat.com>

tests/qemu-iotests/common.filter | 1 +

tests/qemu-iotests/iotests.py | 1 +

2 files changed, 2 insertions(+)

diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter

--- a/tests/qemu-iotests/common.filter

+++ b/tests/qemu-iotests/common.filter

@@ -XXX,XX +XXX,XX @@ _filter_img_info()

-e "/table_size: [0-9]\\+/d" \

-e "/compat: '[^']*'/d" \

-e "/compat6: \\(on\\|off\\)/d" \

+ -e "s/cid: [0-9]\+/cid: XXXXXXXXXX/" \

-e "/static: \\(on\\|off\\)/d" \

-e "/zeroed_grain: \\(on\\|off\\)/d" \

-e "/subformat: '[^']*'/d" \

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py

--- a/tests/qemu-iotests/iotests.py

+++ b/tests/qemu-iotests/iotests.py

@@ -XXX,XX +XXX,XX @@ def filter_img_info(output, filename):

.replace(imgfmt, 'IMGFMT')

line = re.sub('iters: [0-9]+', 'iters: XXX', line)

line = re.sub('uuid: [-a-f0-9]+', 'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', line)

+ line = re.sub('cid: [0-9]+', 'cid: XXXXXXXXXX', line)

lines.append(line)

return '\n'.join(lines)

2.20.1

From: Alberto Garcia <berto@igalia.com>

The mirror_start_job() function used for the commit-active job blocks

the source, target and all intermediate nodes for the duration of the

job.

target <- intermediate <- source

Since 4ef85a9c2339 this function creates a dummy mirror_top_bs that

goes on top of the source node, and it is this dummy node that gets

blocked instead. The source node is never blocked or added to the

job's list of nodes.

target <- intermediate <- source <- mirror_top

At the moment I don't think it is possible to exploit this problem

because any additional job on 'source' would either be forbidden for

other reasons or it would need to involve an additional node that is

blocked, causing an error.

This can be seen in the error messages, however, because they never

refer to the source node being blocked:

$ qemu-img create -f qcow2 hd0.qcow2 1M

$ qemu-img create -f qcow2 -b hd0.qcow2 hd1.qcow2

$ qemu-io -c 'write 0 1M' hd0.qcow2

$ $QEMU -drive if=none,file=hd1.qcow2,node-name=hd1

{ "execute": "qmp_capabilities" }

{ "execute": "block-commit", "arguments": {"device": "hd1", "speed": 256}}

{ "execute": "block-stream", "arguments": {"device": "hd1"}}

{ "error": {"class": "GenericError",

"desc": "Node 'hd0' is busy: block device is in use by block job: commit"}}

After this patch the error message refers to 'hd1', as it should.

The expected output of iotest 141 also needs to be updated for the

same reason.

Signed-off-by: Alberto Garcia <berto@igalia.com>

block/mirror.c | 8 ++++++++

tests/qemu-iotests/141.out | 4 ++--

2 files changed, 10 insertions(+), 2 deletions(-)

@@ -XXX,XX +XXX,XX @@ static void mirror_start_job(const char *job_id, BlockDriverState *bs,

+ ret = block_job_add_bdrv(&s->common, "source", bs, 0,

+ BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE |

+ BLK_PERM_CONSISTENT_READ,

+ errp);

+ if (ret < 0) {

+ goto fail;

+ }

+

/* Required permissions are already taken with blk_new() */

block_job_add_bdrv(&s->common, "target", target, 0, BLK_PERM_ALL,

&error_abort);

diff --git a/tests/qemu-iotests/141.out b/tests/qemu-iotests/141.out

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/141.out

+++ b/tests/qemu-iotests/141.out

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "mirror"}}

{"return": {}}

-{"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: node is used as backing hd of 'NODE_NAME'"}}

+{"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: mirror"}}

{"return": {}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job0"}}

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}}

{"return": {}}

-{"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: node is used as backing hd of 'NODE_NAME'"}}

+{"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: commit"}}

{"return": {}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job0"}}

2.20.1

From: Alberto Garcia <berto@igalia.com>

At the moment I don't see how to make this function fail after the

dirty bitmap has been created, but if that was possible then we would

hit the assert(QLIST_EMPTY(&bs->dirty_bitmaps)) in bdrv_close().

Signed-off-by: Alberto Garcia <berto@igalia.com>

block/mirror.c | 3 +++

1 file changed, 3 insertions(+)

@@ -XXX,XX +XXX,XX @@ fail:

g_free(s->replaces);

blk_unref(s->target);

bs_opaque->job = NULL;

+ if (s->dirty_bitmap) {

+ bdrv_release_dirty_bitmap(bs, s->dirty_bitmap);

job_early_fail(&s->common.job);

}

2.20.1

From: Peter Maydell <peter.maydell@linaro.org>

Taking the address of a field in a packed struct is a bad idea, because

it might not be actually aligned enough for that pointer type (and

thus cause a crash on dereference on some host architectures). Newer

versions of clang warn about this. Avoid the bug by generating the

UUID into a local variable which is definitely safely aligned and

then copying it into place.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

block/vpc.c | 4 +++-

diff --git a/block/vpc.c b/block/vpc.c

--- a/block/vpc.c

+++ b/block/vpc.c

@@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create(BlockdevCreateOptions *opts,

int64_t total_size;

int disk_type;

int ret = -EIO;

+ QemuUUID uuid;

assert(opts->driver == BLOCKDEV_DRIVER_VPC);

vpc_opts = &opts->u.vpc;

@@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create(BlockdevCreateOptions *opts,

footer->type = cpu_to_be32(disk_type);

- qemu_uuid_generate(&footer->uuid);

+ qemu_uuid_generate(&uuid);

+ footer->uuid = uuid;

footer->checksum = cpu_to_be32(vpc_checksum(buf, HEADER_SIZE));

2.20.1

From: Fam Zheng <famz@redhat.com>

The extracted vmdk_init_extent takes a BlockBackend object and

initializes the format metadata. It is the common part between "qemu-img

create" and "blockdev-create".

Add a "BlockBackend *pbb" parameter to vmdk_create_extent, to return the

opened BB to the caller in the next patch.

Signed-off-by: Fam Zheng <famz@redhat.com>

Reviewed-by: Markus Armbruster <armbru@redhat.com>

block/vmdk.c | 69 ++++++++++++++++++++++++++++++++--------------------

1 file changed, 43 insertions(+), 26 deletions(-)