The following changes since commit 96662996eda78c48aadddd4e76d8615c7eb72d80:

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20210513a' into staging (2021-05-14 12:03:47 +0100)

are available in the Git repository at:

https://github.com/XanClic/qemu.git tags/pull-block-2021-05-14

for you to fetch changes up to c61ebf362d0abf288ce266845519d5a550a1d89f:

write-threshold: deal with includes (2021-05-14 16:14:10 +0200)

Block patches:

- drop block/io write notifiers

- qemu-iotests enhancements to make debugging easier

- rbd parsing fix

- HMP qemu-io fix (for iothreads)

- mirror job cancel relaxation (do not cancel in-flight requests when a

READY mirror job is canceled with force=false)

- document qcow2's data_file and data_file_raw features

- fix iotest 297 for pylint 2.8

- block/copy-on-read refactoring

Emanuele Giuseppe Esposito (1):

qemu-iotests: fix pylint 2.8 consider-using-with error

Paolo Bonzini (5):

qemu-iotests: do not buffer the test output

qemu-iotests: allow passing unittest.main arguments to the test

scripts

qemu-iotests: move command line and environment handling from

TestRunner to TestEnv

qemu-iotests: let "check" spawn an arbitrary test command

qemu-iotests: fix case of SOCK_DIR already in the environment

Vladimir Sementsov-Ogievskiy (10):

monitor: hmp_qemu_io: acquire aio contex, fix crash

mirror: stop cancelling in-flight requests on non-force cancel in

READY

block/copy-on-read: use bdrv_drop_filter() and drop s->active

block/write-threshold: don't use write notifiers

block: drop write notifiers

test-write-threshold: rewrite test_threshold_(not_)trigger tests

block/write-threshold: drop extra APIs

test-write-threshold: drop extra tests

test-write-threshold: drop extra TestStruct structure

write-threshold: deal with includes

docs/tools/qemu-img.rst | 31 +++++++

include/block/block_int.h | 15 +---

include/block/write-threshold.h | 27 ++----

include/qemu/job.h | 2 +-

block.c | 1 -

block/backup.c | 2 +-

block/copy-on-read.c | 33 +------

block/io.c | 11 +--

block/mirror.c | 6 +-

block/monitor/block-hmp-cmds.c | 31 ++++---

block/rbd.c | 32 ++++---

block/write-threshold.c | 91 ++++---------------

job.c | 2 +-

qemu-io-cmds.c | 8 +-

qemu-io.c | 17 +++-

tests/unit/test-write-threshold.c | 90 ++-----------------

tests/qemu-iotests/231 | 4 +

tests/qemu-iotests/231.out | 7 +-

tests/qemu-iotests/240.out | 8 +-

tests/qemu-iotests/245.out | 8 +-

tests/qemu-iotests/264 | 2 +-

tests/qemu-iotests/295.out | 6 +-

tests/qemu-iotests/296.out | 8 +-

tests/qemu-iotests/check | 19 +++-

tests/qemu-iotests/iotests.py | 145 +++++++++++++++++-------------

tests/qemu-iotests/pylintrc | 3 +

tests/qemu-iotests/testenv.py | 22 ++++-

tests/qemu-iotests/testrunner.py | 37 +++-----

28 files changed, 289 insertions(+), 379 deletions(-)

2.31.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

write-notifiers are used only for write-threshold. New code for such

purpose should create filters.

Let's better special-case write-threshold and drop write notifiers at

all. (Actually, write-threshold is special-cased anyway, as the only

user of write-notifiers)

So, create a new direct interface for bdrv_co_write_req_prepare() and

drop all write-notifier related logic from write-threshold.c.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Reviewed-by: Max Reitz <mreitz@redhat.com>

Message-Id: <20210506090621.11848-2-vsementsov@virtuozzo.com>

Reviewed-by: Eric Blake <eblake@redhat.com>

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

[mreitz: Adjusted comment as per Eric's suggestion]

Signed-off-by: Max Reitz <mreitz@redhat.com>

include/block/block_int.h | 1 -

include/block/write-threshold.h | 9 +++++

block/io.c | 5 ++-

block/write-threshold.c | 70 +++++++--------------------------

4 files changed, 27 insertions(+), 58 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h

index XXXXXXX..XXXXXXX 100644

--- a/include/block/block_int.h

+++ b/include/block/block_int.h

@@ -XXX,XX +XXX,XX @@ struct BlockDriverState {

/* threshold limit for writes, in bytes. "High water mark". */

uint64_t write_threshold_offset;

- NotifierWithReturn write_threshold_notifier;

/* Writing to the list requires the BQL _and_ the dirty_bitmap_mutex.

* Reading from the list can be done with either the BQL or the

diff --git a/include/block/write-threshold.h b/include/block/write-threshold.h

index XXXXXXX..XXXXXXX 100644

--- a/include/block/write-threshold.h

+++ b/include/block/write-threshold.h

@@ -XXX,XX +XXX,XX @@ bool bdrv_write_threshold_is_set(const BlockDriverState *bs);

uint64_t bdrv_write_threshold_exceeded(const BlockDriverState *bs,

const BdrvTrackedRequest *req);

+/*

+ * bdrv_write_threshold_check_write

+ *

+ * Check whether the specified request exceeds the write threshold.

+ * If so, send a corresponding event and disable write threshold checking.

+ */

+void bdrv_write_threshold_check_write(BlockDriverState *bs, int64_t offset,

+ int64_t bytes);

+

#endif

@@ -XXX,XX +XXX,XX @@

#include "block/blockjob_int.h"

#include "block/block_int.h"

#include "block/coroutines.h"

+#include "block/write-threshold.h"

#include "qemu/cutils.h"

#include "qapi/error.h"

#include "qemu/error-report.h"

@@ -XXX,XX +XXX,XX @@ bdrv_co_write_req_prepare(BdrvChild *child, int64_t offset, int64_t bytes,

} else {

assert(child->perm & BLK_PERM_WRITE);

}

- return notifier_with_return_list_notify(&bs->before_write_notifiers,

- req);

+ bdrv_write_threshold_check_write(bs, offset, bytes);

+ return 0;

case BDRV_TRACKED_TRUNCATE:

assert(child->perm & BLK_PERM_RESIZE);

return 0;

diff --git a/block/write-threshold.c b/block/write-threshold.c

index XXXXXXX..XXXXXXX 100644

--- a/block/write-threshold.c

+++ b/block/write-threshold.c

@@ -XXX,XX +XXX,XX @@ bool bdrv_write_threshold_is_set(const BlockDriverState *bs)

return bs->write_threshold_offset > 0;

-static void write_threshold_disable(BlockDriverState *bs)

- if (bdrv_write_threshold_is_set(bs)) {

- notifier_with_return_remove(&bs->write_threshold_notifier);

- bs->write_threshold_offset = 0;

uint64_t bdrv_write_threshold_exceeded(const BlockDriverState *bs,

const BdrvTrackedRequest *req)

@@ -XXX,XX +XXX,XX @@ uint64_t bdrv_write_threshold_exceeded(const BlockDriverState *bs,

return 0;

-static int coroutine_fn before_write_notify(NotifierWithReturn *notifier,

- void *opaque)

- BdrvTrackedRequest *req = opaque;

- BlockDriverState *bs = req->bs;

- uint64_t amount = 0;

-

- amount = bdrv_write_threshold_exceeded(bs, req);

- if (amount > 0) {

- qapi_event_send_block_write_threshold(

- bs->node_name,

- amount,

- bs->write_threshold_offset);

-

- /* autodisable to avoid flooding the monitor */

- write_threshold_disable(bs);

- }

-

- return 0; /* should always let other notifiers run */

-

-static void write_threshold_register_notifier(BlockDriverState *bs)

-{

- bs->write_threshold_notifier.notify = before_write_notify;

- bdrv_add_before_write_notifier(bs, &bs->write_threshold_notifier);

-}

-

-static void write_threshold_update(BlockDriverState *bs,

- int64_t threshold_bytes)

-{

- bs->write_threshold_offset = threshold_bytes;

-}

-

void bdrv_write_threshold_set(BlockDriverState *bs, uint64_t threshold_bytes)

{

- if (bdrv_write_threshold_is_set(bs)) {

- if (threshold_bytes > 0) {

- write_threshold_update(bs, threshold_bytes);

- } else {

- write_threshold_disable(bs);

- }

- } else {

- if (threshold_bytes > 0) {

- /* avoid multiple registration */

- write_threshold_register_notifier(bs);

- write_threshold_update(bs, threshold_bytes);

- }

- /* discard bogus disable request */

- }

+ bs->write_threshold_offset = threshold_bytes;

}

void qmp_block_set_write_threshold(const char *node_name,

@@ -XXX,XX +XXX,XX @@ void qmp_block_set_write_threshold(const char *node_name,

aio_context_release(aio_context);

}

+

+void bdrv_write_threshold_check_write(BlockDriverState *bs, int64_t offset,

+ int64_t bytes)

+{

+ int64_t end = offset + bytes;

+ uint64_t wtr = bs->write_threshold_offset;

+

+ if (wtr > 0 && end > wtr) {

+ qapi_event_send_block_write_threshold(bs->node_name, end - wtr, wtr);

+

+ /* autodisable to avoid flooding the monitor */

+ bdrv_write_threshold_set(bs, 0);

2.31.1

From: Emanuele Giuseppe Esposito <eesposit@redhat.com>

pylint 2.8 introduces consider-using-with error, suggesting

to use the 'with' block statement when possible.

Modify all subprocess.Popen call to use the 'with' statement,

except the one in __init__ of QemuIoInteractive class, since

it is assigned to a class field and used in other methods.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>

Message-Id: <20210510190449.65948-1-eesposit@redhat.com>

[mreitz: Disable bad-option-value warning in the iotests' pylintrc, so

that disabling consider-using-with in QemuIoInteractive will

not produce a warning in pre-2.8 pylint versions]

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

tests/qemu-iotests/iotests.py | 65 ++++++++++++++++----------------

tests/qemu-iotests/pylintrc | 3 ++

tests/qemu-iotests/testrunner.py | 22 +++++------

3 files changed, 47 insertions(+), 43 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py

--- a/tests/qemu-iotests/iotests.py

+++ b/tests/qemu-iotests/iotests.py

@@ -XXX,XX +XXX,XX @@ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],