The following changes since commit ad88e4252f09c2956b99c90de39e95bab2e8e7af:

Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-1-2019' into staging (2019-06-03 10:25:12 +0100)

for you to fetch changes up to 9593db8ccd27800ce4a17f1d5b735b9130c541a2:

iotests: Fix duplicated diff output on failure (2019-06-03 16:33:20 +0200)

- block: AioContext management, part 2

- Avoid recursive block_status call (i.e. lseek() calls) if possible

- linux-aio: Drop unused BlockAIOCB submission method

- nvme: add Get/Set Feature Timestamp support

- Fix crash on commit job start with active I/O on base node

- Fix crash in bdrv_drained_end

- Fix integer overflow in qcow2 discard

John Snow (1):

blockdev: fix missed target unref for drive-backup

Julia Suvorova (1):

block/linux-aio: Drop unused BlockAIOCB submission method

Kenneth Heitke (1):

nvme: add Get/Set Feature Timestamp support

Kevin Wolf (19):

block: Drain source node in bdrv_replace_node()

iotests: Test commit job start with concurrent I/O

test-block-iothread: Check filter node in test_propagate_mirror

nbd-server: Call blk_set_allow_aio_context_change()

block: Add Error to blk_set_aio_context()

block: Add BlockBackend.ctx

block: Add qdev_prop_drive_iothread property type

scsi-disk: Use qdev_prop_drive_iothread

block: Adjust AioContexts when attaching nodes

test-block-iothread: Test adding parent to iothread node

test-block-iothread: BlockBackend AioContext across root node change

block: Move node without parents to main AioContext

blockdev: Use bdrv_try_set_aio_context() for monitor commands

block: Remove wrong bdrv_set_aio_context() calls

virtio-scsi-test: Test attaching new overlay with iothreads

iotests: Attach new devices to node in non-default iothread

test-bdrv-drain: Use bdrv_try_set_aio_context()

block: Remove bdrv_set_aio_context()

iotests: Fix duplicated diff output on failure

Max Reitz (2):

block/io: Delay decrementing the quiesce_counter

iotests: Test cancelling a job and closing the VM

Vladimir Sementsov-Ogievskiy (4):

tests/perf: Test lseek influence on qcow2 block-status

block: avoid recursive block_status call if possible

block/qcow2-refcount: add trace-point to qcow2_process_discards

block/io: bdrv_pdiscard: support int64_t bytes parameter

This tests that concurrent requests are correctly drained before making

graph modifications instead of running into assertions in

bdrv_replace_node().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

Reviewed-by: Max Reitz <mreitz@redhat.com>

---

tests/qemu-iotests/255 | 83 +++++++++++++++++++++++++++++++++++

tests/qemu-iotests/255.out | 16 +++++++

tests/qemu-iotests/group | 1 +

tests/qemu-iotests/iotests.py | 10 ++++-

4 files changed, 109 insertions(+), 1 deletion(-)

create mode 100755 tests/qemu-iotests/255

create mode 100644 tests/qemu-iotests/255.out

diff --git a/tests/qemu-iotests/255 b/tests/qemu-iotests/255

+++ b/tests/qemu-iotests/255

+#!/usr/bin/env python

+# Test commit job graph modifications while requests are active

+# Copyright (C) 2019 Red Hat, Inc.

+#

+# Creator/Owner: Kevin Wolf <kwolf@redhat.com>

+import iotests

+from iotests import imgfmt

+iotests.verify_image_format(supported_fmts=['qcow2'])

+def blockdev_create(vm, options):

+ result = vm.qmp_log('blockdev-create',

+ filters=[iotests.filter_qmp_testfiles],

+ job_id='job0', options=options)

+ if 'return' in result:

+ assert result['return'] == {}

+ vm.run_job('job0')

+ iotests.log("")

+with iotests.FilePath('t.qcow2') as disk_path, \

+ iotests.FilePath('t.qcow2.mid') as mid_path, \

+ iotests.FilePath('t.qcow2.base') as base_path, \

+ iotests.VM() as vm:

+ iotests.log("=== Create backing chain and start VM ===")

+ iotests.log("")

+ size = 128 * 1024 * 1024

+ size_str = str(size)

+ iotests.create_image(base_path, size)

+ iotests.qemu_img_log('create', '-f', iotests.imgfmt, mid_path, size_str)

+ iotests.qemu_img_log('create', '-f', iotests.imgfmt, disk_path, size_str)

+ # Create a backing chain like this:

+ # base <- [throttled: bps-read=4096] <- mid <- overlay

+ vm.add_object('throttle-group,x-bps-read=4096,id=throttle0')

+ vm.add_blockdev('file,filename=%s,node-name=base' % (base_path))

+ vm.add_blockdev('throttle,throttle-group=throttle0,file=base,node-name=throttled')

+ vm.add_blockdev('file,filename=%s,node-name=mid-file' % (mid_path))

+ vm.add_blockdev('qcow2,file=mid-file,node-name=mid,backing=throttled')

+ vm.add_drive_raw('if=none,id=overlay,driver=qcow2,file=%s,backing=mid' % (disk_path))

+ vm.launch()

+ iotests.log("=== Start background read requests ===")

+ iotests.log("")

+ def start_requests():

+ vm.hmp_qemu_io('overlay', 'aio_read 0 4k')

+ vm.hmp_qemu_io('overlay', 'aio_read 0 4k')

+

+ start_requests()

+

+ iotests.log("=== Run a commit job ===")

+ iotests.log("")

+

+ result = vm.qmp_log('block-commit', job_id='job0', auto_finalize=False,

+ device='overlay', top_node='mid')

+

+ vm.run_job('job0', auto_finalize=False, pre_finalize=start_requests,

+ auto_dismiss=True)

+

+ vm.shutdown()

diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out

+++ b/tests/qemu-iotests/255.out

+=== Create backing chain and start VM ===

+Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16

+

+Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16

+

+=== Start background read requests ===

+

+=== Run a commit job ===

+

+{"execute": "block-commit", "arguments": {"auto-finalize": false, "device": "overlay", "job-id": "job0", "top-node": "mid"}}

+{"return": {}}

+{"execute": "job-finalize", "arguments": {"id": "job0"}}

+{"return": {}}

+{"data": {"id": "job0", "type": "commit"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}

+{"data": {"device": "job0", "len": 134217728, "offset": 134217728, "speed": 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}

252 rw auto backing quick

253 rw auto quick

254 rw auto backing quick

+255 rw auto quick

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/iotests.py

+++ b/tests/qemu-iotests/iotests.py

@@ -XXX,XX +XXX,XX @@ def qemu_img_pipe(*args):

sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args))))

return subp.communicate()[0]

+def qemu_img_log(*args):

+ result = qemu_img_pipe(*args)

+ log(result, filters=[filter_testfiles])

+ return result

+

def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]):

args = [ 'info' ]

if imgopts:

@@ -XXX,XX +XXX,XX @@ class VM(qtest.QEMUQtestMachine):

return result

# Returns None on success, and an error string on failure

- def run_job(self, job, auto_finalize=True, auto_dismiss=False):

+ def run_job(self, job, auto_finalize=True, auto_dismiss=False,

+ pre_finalize=None):

error = None

while True:

for ev in self.get_qmp_events_filtered(wait=True):

@@ -XXX,XX +XXX,XX @@ class VM(qtest.QEMUQtestMachine):

error = j['error']

log('Job failed: %s' % (j['error']))

elif status == 'pending' and not auto_finalize:

+ if pre_finalize:

+ pre_finalize()

self.qmp_log('job-finalize', id=job)

elif status == 'concluded' and not auto_dismiss:

self.qmp_log('job-dismiss', id=job)

2.20.1

From: John Snow <jsnow@redhat.com>

If the bitmap can't be used for whatever reason, we skip putting down

the reference. Fix that.

In practice, this means that if you attempt to gracefully exit QEMU

after a backup command being rejected, bdrv_close_all will fail and

tell you some unpleasant things via assert().

Reported-by: aihua liang <aliang@redhat.com>

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1703916

Signed-off-by: John Snow <jsnow@redhat.com>

Reviewed-by: Kevin Wolf <kwolf@redhat.com>

blockdev.c | 13 ++++++-------

1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/blockdev.c b/blockdev.c

--- a/blockdev.c

+++ b/blockdev.c

@@ -XXX,XX +XXX,XX @@ static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn,

if (set_backing_hd) {

bdrv_set_backing_hd(target_bs, source, &local_err);

if (local_err) {

- bdrv_unref(target_bs);

- goto out;

+ goto unref;

@@ -XXX,XX +XXX,XX @@ static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn,

bmap = bdrv_find_dirty_bitmap(bs, backup->bitmap);

if (!bmap) {

error_setg(errp, "Bitmap '%s' could not be found", backup->bitmap);

- bdrv_unref(target_bs);

- goto out;

+ goto unref;

}

if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) {

- goto out;

+ goto unref;

}

}

if (!backup->auto_finalize) {

@@ -XXX,XX +XXX,XX @@ static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn,

backup->sync, bmap, backup->compress,

backup->on_source_error, backup->on_target_error,

job_flags, NULL, NULL, txn, &local_err);

- bdrv_unref(target_bs);

if (local_err != NULL) {

error_propagate(errp, local_err);

- goto out;

+ goto unref;

}

+unref:

+ bdrv_unref(target_bs);

out:

aio_context_release(aio_context);

return job;

2.20.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

drv_co_block_status digs bs->file for additional, more accurate search

for hole inside region, reported as DATA by bs since 5daa74a6ebc.

This accuracy is not free: assume we have qcow2 disk. Actually, qcow2

knows, where are holes and where is data. But every block_status

request calls lseek additionally. Assume a big disk, full of

data, in any iterative copying block job (or img convert) we'll call

lseek(HOLE) on every iteration, and each of these lseeks will have to

iterate through all metadata up to the end of file. It's obviously

ineffective behavior. And for many scenarios we don't need this lseek

at all.

However, lseek is needed when we have metadata-preallocated image.

So, let's detect metadata-preallocation case and don't dig qcow2's

protocol file in other cases.

The idea is to compare allocation size in POV of filesystem with

allocations size in POV of Qcow2 (by refcounts). If allocation in fs is

significantly lower, consider it as metadata-preallocation case.

102 iotest changed, as our detector can't detect shrinked file as

metadata-preallocation, which don't seem to be wrong, as with metadata

preallocation we always have valid file length.

Two other iotests have a slight change in their QMP output sequence:

Active 'block-commit' returns earlier because the job coroutine yields

earlier on a blocking operation. This operation is loading the refcount

blocks in qcow2_detect_metadata_preallocation().

Suggested-by: Denis V. Lunev <den@openvz.org>

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

---

block/qcow2.h | 4 ++++

include/block/block.h | 8 +++++++-

block/io.c | 9 ++++++++-

block/qcow2-refcount.c | 32 ++++++++++++++++++++++++++++++++

block/qcow2.c | 11 +++++++++++

tests/qemu-iotests/102 | 2 +-

tests/qemu-iotests/102.out | 3 ++-

tests/qemu-iotests/141.out | 2 +-

tests/qemu-iotests/144.out | 2 +-

9 files changed, 67 insertions(+), 6 deletions(-)

diff --git a/block/qcow2.h b/block/qcow2.h

index XXXXXXX..XXXXXXX 100644

--- a/block/qcow2.h

+++ b/block/qcow2.h

@@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcow2State {

int nb_threads;

BdrvChild *data_file;

+

+ bool metadata_preallocation_checked;

+ bool metadata_preallocation;

} BDRVQcow2State;

typedef struct Qcow2COWRegion {

@@ -XXX,XX +XXX,XX @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,

void *cb_opaque, Error **errp);

int qcow2_shrink_reftable(BlockDriverState *bs);

int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size);

+int qcow2_detect_metadata_preallocation(BlockDriverState *bs);

/* qcow2-cluster.c functions */

int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size,

diff --git a/include/block/block.h b/include/block/block.h

index XXXXXXX..XXXXXXX 100644

--- a/include/block/block.h

+++ b/include/block/block.h

@@ -XXX,XX +XXX,XX @@ typedef struct HDGeometry {

* BDRV_BLOCK_EOF: the returned pnum covers through end of file for this

* layer, set by block layer

*

- * Internal flag:

+ * Internal flags:

* BDRV_BLOCK_RAW: for use by passthrough drivers, such as raw, to request

* that the block layer recompute the answer from the returned

* BDS; must be accompanied by just BDRV_BLOCK_OFFSET_VALID.

+ * BDRV_BLOCK_RECURSE: request that the block layer will recursively search for

+ * zeroes in file child of current block node inside

+ * returned region. Only valid together with both

+ * BDRV_BLOCK_DATA and BDRV_BLOCK_OFFSET_VALID. Should not

+ * appear with BDRV_BLOCK_ZERO.

*

* If BDRV_BLOCK_OFFSET_VALID is set, the map parameter represents the

* host offset within the returned BDS that is allocated for the

@@ -XXX,XX +XXX,XX @@ typedef struct HDGeometry {

#define BDRV_BLOCK_RAW 0x08

#define BDRV_BLOCK_ALLOCATED 0x10

#define BDRV_BLOCK_EOF 0x20

+#define BDRV_BLOCK_RECURSE 0x40

#define BDRV_BLOCK_OFFSET_MASK BDRV_SECTOR_MASK

typedef QSIMPLEQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry) BlockReopenQueue;

diff --git a/block/io.c b/block/io.c

index XXXXXXX..XXXXXXX 100644

--- a/block/io.c

+++ b/block/io.c

@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,

*/

assert(*pnum && QEMU_IS_ALIGNED(*pnum, align) &&

align > offset - aligned_offset);

+ if (ret & BDRV_BLOCK_RECURSE) {

+ assert(ret & BDRV_BLOCK_DATA);

+ assert(ret & BDRV_BLOCK_OFFSET_VALID);

+ assert(!(ret & BDRV_BLOCK_ZERO));

+ }

+

*pnum -= offset - aligned_offset;

if (*pnum > bytes) {

*pnum = bytes;

@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,

}

}

- if (want_zero && local_file && local_file != bs &&

+ if (want_zero && ret & BDRV_BLOCK_RECURSE &&

+ local_file && local_file != bs &&

(ret & BDRV_BLOCK_DATA) && !(ret & BDRV_BLOCK_ZERO) &&

(ret & BDRV_BLOCK_OFFSET_VALID)) {

int64_t file_pnum;

diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c

index XXXXXXX..XXXXXXX 100644

--- a/block/qcow2-refcount.c

+++ b/block/qcow2-refcount.c

@@ -XXX,XX +XXX,XX @@ int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size)

"There are no references in the refcount table.");

return -EIO;

}

+

+int qcow2_detect_metadata_preallocation(BlockDriverState *bs)

+{

+ BDRVQcow2State *s = bs->opaque;

+ int64_t i, end_cluster, cluster_count = 0, threshold;

+ int64_t file_length, real_allocation, real_clusters;

+

+ file_length = bdrv_getlength(bs->file->bs);

+ if (file_length < 0) {

+ return file_length;

+ }

+

+ real_allocation = bdrv_get_allocated_file_size(bs->file->bs);

+ if (real_allocation < 0) {

+ return real_allocation;

+ }

+

+ real_clusters = real_allocation / s->cluster_size;

+ threshold = MAX(real_clusters * 10 / 9, real_clusters + 2);

+

+ end_cluster = size_to_clusters(s, file_length);

+ for (i = 0; i < end_cluster && cluster_count < threshold; i++) {

+ uint64_t refcount;

+ int ret = qcow2_get_refcount(bs, i, &refcount);

+ if (ret < 0) {

+ return ret;

+ }

+ cluster_count += !!refcount;

+ }

+

+ return cluster_count >= threshold;

+}

diff --git a/block/qcow2.c b/block/qcow2.c

index XXXXXXX..XXXXXXX 100644

--- a/block/qcow2.c

+++ b/block/qcow2.c

@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,

unsigned int bytes;

int status = 0;

+ if (!s->metadata_preallocation_checked) {

+ ret = qcow2_detect_metadata_preallocation(bs);

+ s->metadata_preallocation = (ret == 1);

+ s->metadata_preallocation_checked = true;

+ }

+

bytes = MIN(INT_MAX, count);

qemu_co_mutex_lock(&s->lock);

ret = qcow2_get_cluster_offset(bs, offset, &bytes, &cluster_offset);

@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,

} else if (ret != QCOW2_CLUSTER_UNALLOCATED) {

status |= BDRV_BLOCK_DATA;

}

+ if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) &&

+ (status & BDRV_BLOCK_OFFSET_VALID))

+ {

+ status |= BDRV_BLOCK_RECURSE;

+ }

return status;

}

diff --git a/tests/qemu-iotests/102 b/tests/qemu-iotests/102

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/102

+++ b/tests/qemu-iotests/102

@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c 'write 0 64k' "$TEST_IMG" | _filter_qemu_io

$QEMU_IMG resize -f raw --shrink "$TEST_IMG" $((5 * 64 * 1024))

$QEMU_IO -c map "$TEST_IMG"

-$QEMU_IMG map "$TEST_IMG"

+$QEMU_IMG map "$TEST_IMG" | _filter_qemu_img_map

echo

echo '=== Testing map on an image file truncated outside of qemu ==='

diff --git a/tests/qemu-iotests/102.out b/tests/qemu-iotests/102.out

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/102.out

+++ b/tests/qemu-iotests/102.out

@@ -XXX,XX +XXX,XX @@ wrote 65536/65536 bytes at offset 0

64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

Image resized.

64 KiB (0x10000) bytes allocated at offset 0 bytes (0x0)

-Offset Length Mapped to File

+Offset Length File

+0 0x10000 TEST_DIR/t.IMGFMT

=== Testing map on an image file truncated outside of qemu ===

diff --git a/tests/qemu-iotests/141.out b/tests/qemu-iotests/141.out

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/141.out

+++ b/tests/qemu-iotests/141.out

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.

{"return": {}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}

+{"return": {}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}}

-{"return": {}}

{"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: commit"}}

{"return": {}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}}

diff --git a/tests/qemu-iotests/144.out b/tests/qemu-iotests/144.out

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/144.out

+++ b/tests/qemu-iotests/144.out

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "virtio0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "virtio0"}}

+{"return": {}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "virtio0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "virtio0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}}

{"return": {}}

-{"return": {}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "virtio0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "virtio0"}}

{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "virtio0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}}

--

2.20.1

This patch adds a test where we cancel a throttled mirror job and

immediately close the VM before it can be cancelled. Doing so will

invoke bdrv_drain_all() while the mirror job tries to drain the

throttled node. When bdrv_drain_all_end() tries to lift its drain on

the throttle node, the job will exit and replace the current root node

of the BB drive0 (which is the job's filter node) by the throttle node.

Before the previous patch, this replacement did not increase drive0's

quiesce_counter by a sufficient amount, so when

bdrv_parent_drained_end() (invoked by bdrv_do_drained_end(), invoked by

bdrv_drain_all_end()) tried to end the drain on all of the throttle

node's parents, it decreased drive0's quiesce_counter below 0 -- which

fails an assertion.

tests/qemu-iotests/255 | 52 ++++++++++++++++++++++++++++++++++++++

tests/qemu-iotests/255.out | 24 ++++++++++++++++++

2 files changed, 76 insertions(+)

diff --git a/tests/qemu-iotests/255 b/tests/qemu-iotests/255

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/255

+++ b/tests/qemu-iotests/255

@@ -XXX,XX +XXX,XX @@ def blockdev_create(vm, options):

vm.run_job('job0')

iotests.log("")

+iotests.log('Finishing a commit job with background reads')

+iotests.log('============================================')

+iotests.log('')

+

with iotests.FilePath('t.qcow2') as disk_path, \

iotests.FilePath('t.qcow2.mid') as mid_path, \

iotests.FilePath('t.qcow2.base') as base_path, \

@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.qcow2') as disk_path, \

auto_dismiss=True)

vm.shutdown()

+

+iotests.log('')

+iotests.log('Closing the VM while a job is being cancelled')

+iotests.log('=============================================')

+iotests.log('')

+

+with iotests.FilePath('src.qcow2') as src_path, \

+ iotests.FilePath('dst.qcow2') as dst_path, \

+ iotests.VM() as vm:

+

+ iotests.log('=== Create images and start VM ===')

+ iotests.log('')

+

+ size = 128 * 1024 * 1024

+ size_str = str(size)

+

+ iotests.qemu_img_log('create', '-f', iotests.imgfmt, src_path, size_str)

+ iotests.qemu_img_log('create', '-f', iotests.imgfmt, dst_path, size_str)

+

+ iotests.log(iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write 0 1M',

+ src_path),

+ filters=[iotests.filter_test_dir, iotests.filter_qemu_io])

+

+ vm.add_object('throttle-group,x-bps-read=4096,id=throttle0')

+

+ vm.add_blockdev('file,node-name=src-file,filename=%s' % (src_path))

+ vm.add_blockdev('%s,node-name=src,file=src-file' % (iotests.imgfmt))

+

+ vm.add_blockdev('file,node-name=dst-file,filename=%s' % (dst_path))

+ vm.add_blockdev('%s,node-name=dst,file=dst-file' % (iotests.imgfmt))

+

+ vm.add_blockdev('throttle,node-name=src-throttled,' +

+ 'throttle-group=throttle0,file=src')

+

+ vm.add_device('virtio-blk,drive=src-throttled')

+

+ vm.launch()

+

+ iotests.log('=== Start a mirror job ===')

+ iotests.log('')

+

+ vm.qmp_log('blockdev-mirror', job_id='job0', device='src-throttled',

+ target='dst', sync='full')

+

+ vm.qmp_log('block-job-cancel', device='job0')

+ vm.qmp_log('quit')

+

+ vm.shutdown()

diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out

--- a/tests/qemu-iotests/255.out

+++ b/tests/qemu-iotests/255.out

@@ -XXX,XX +XXX,XX @@

+Finishing a commit job with background reads

+============================================

+

=== Create backing chain and start VM ===

Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 l

{"return": {}}

{"data": {"id": "job0", "type": "commit"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}

{"data": {"device": "job0", "len": 134217728, "offset": 134217728, "speed": 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}

+

+Closing the VM while a job is being cancelled

+=============================================

+

+=== Create images and start VM ===

+

+Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16

+

+Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16

+

+wrote 1048576/1048576 bytes at offset 0

+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

+

+=== Start a mirror job ===

+

+{"execute": "blockdev-mirror", "arguments": {"device": "src-throttled", "job-id": "job0", "sync": "full", "target": "dst"}}

+{"return": {}}

+{"execute": "block-job-cancel", "arguments": {"device": "job0"}}

+{"return": {}}

+{"execute": "quit", "arguments": {}}

+{"return": {}}

2.20.1

From: Kenneth Heitke <kenneth.heitke@intel.com>

Signed-off-by: Kenneth Heitke <kenneth.heitke@intel.com>

Reviewed-by: Klaus Birkelund Jensen <klaus.jensen@cnexlabs.com>

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

---

hw/block/nvme.h | 2 +

include/block/nvme.h | 2 +

hw/block/nvme.c | 106 +++++++++++++++++++++++++++++++++++++++++-

hw/block/trace-events | 2 +

4 files changed, 110 insertions(+), 2 deletions(-)

diff --git a/hw/block/nvme.h b/hw/block/nvme.h

index XXXXXXX..XXXXXXX 100644

--- a/hw/block/nvme.h

+++ b/hw/block/nvme.h

@@ -XXX,XX +XXX,XX @@ typedef struct NvmeCtrl {

uint32_t cmbloc;

uint8_t *cmbuf;

uint64_t irq_status;

+ uint64_t host_timestamp; /* Timestamp sent by the host */

+ uint64_t timestamp_set_qemu_clock_ms; /* QEMU clock time */

char *serial;

NvmeNamespace *namespaces;

diff --git a/include/block/nvme.h b/include/block/nvme.h

index XXXXXXX..XXXXXXX 100644

--- a/include/block/nvme.h

+++ b/include/block/nvme.h

@@ -XXX,XX +XXX,XX @@ enum NvmeIdCtrlOncs {

NVME_ONCS_WRITE_ZEROS = 1 << 3,

NVME_ONCS_FEATURES = 1 << 4,

NVME_ONCS_RESRVATIONS = 1 << 5,

+ NVME_ONCS_TIMESTAMP = 1 << 6,