Block patches:

- Minor fixes and tests from the freeze period (too minor to be included

in 4.2)

- Allow many bash iotests to test qcow2's external data file feature

- Add compress filter driver

- Fix Python iotests after 6f6e1698a6

- Fix for the backup job

We need some way to correlate QAPI BlockPermission values with

BLK_PERM_* flags. We could:

(1) have the same order in the QAPI definition as the the BLK_PERM_*

flags are in LSb-first order. However, then there is no guarantee

that they actually match (e.g. when someone modifies the QAPI schema

without thinking of the BLK_PERM_* definitions).

We could add static assertions, but these would break what’s good

about this solution, namely its simplicity.

(2) define the BLK_PERM_* flags based on the BlockPermission values.

But this way whenever someone were to modify the QAPI order

(perfectly sensible in theory), the BLK_PERM_* values would change.

Because these values are used for file locking, this might break

file locking between different qemu versions.

Therefore, go the slightly more cumbersome way: Add a function to

translate from the QAPI constants to the BLK_PERM_* flags.

Signed-off-by: Max Reitz <mreitz@redhat.com>

Message-id: 20191108123455.39445-2-mreitz@redhat.com

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

block.c | 18 ++++++++++++++++++

include/block/block.h | 1 +

2 files changed, 19 insertions(+)

diff --git a/block.c b/block.c

index XXXXXXX..XXXXXXX 100644

--- a/block.c

+++ b/block.c

@@ -XXX,XX +XXX,XX @@ void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c,

*nshared = shared;

}

+uint64_t bdrv_qapi_perm_to_blk_perm(BlockPermission qapi_perm)

+{

+ static const uint64_t permissions[] = {

+ [BLOCK_PERMISSION_CONSISTENT_READ] = BLK_PERM_CONSISTENT_READ,

+ [BLOCK_PERMISSION_WRITE] = BLK_PERM_WRITE,

+ [BLOCK_PERMISSION_WRITE_UNCHANGED] = BLK_PERM_WRITE_UNCHANGED,

+ [BLOCK_PERMISSION_RESIZE] = BLK_PERM_RESIZE,

+ [BLOCK_PERMISSION_GRAPH_MOD] = BLK_PERM_GRAPH_MOD,

+ };

+

+ QEMU_BUILD_BUG_ON(ARRAY_SIZE(permissions) != BLOCK_PERMISSION__MAX);

+ QEMU_BUILD_BUG_ON(1UL << ARRAY_SIZE(permissions) != BLK_PERM_ALL + 1);

+

+ assert(qapi_perm < BLOCK_PERMISSION__MAX);

+

+ return permissions[qapi_perm];

+}

+

static void bdrv_replace_child_noperm(BdrvChild *child,

BlockDriverState *new_bs)

{

diff --git a/include/block/block.h b/include/block/block.h

index XXXXXXX..XXXXXXX 100644

--- a/include/block/block.h

+++ b/include/block/block.h

@@ -XXX,XX +XXX,XX @@ enum {

};

char *bdrv_perm_names(uint64_t perm);

+uint64_t bdrv_qapi_perm_to_blk_perm(BlockPermission qapi_perm);

/* disk I/O throttling */

void bdrv_init(void);

--

2.24.1

We can save some LoC in xdbg_graph_add_edge() by using

bdrv_qapi_perm_to_blk_perm().

Signed-off-by: Max Reitz <mreitz@redhat.com>

Message-id: 20191108123455.39445-3-mreitz@redhat.com

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

block.c | 29 ++++++++---------------------

1 file changed, 8 insertions(+), 21 deletions(-)

diff --git a/block.c b/block.c

index XXXXXXX..XXXXXXX 100644

--- a/block.c

+++ b/block.c

@@ -XXX,XX +XXX,XX @@ static void xdbg_graph_add_node(XDbgBlockGraphConstructor *gr, void *node,

static void xdbg_graph_add_edge(XDbgBlockGraphConstructor *gr, void *parent,

const BdrvChild *child)

{

- typedef struct {

- unsigned int flag;

- BlockPermission num;

- } PermissionMap;

-

- static const PermissionMap permissions[] = {

- { BLK_PERM_CONSISTENT_READ, BLOCK_PERMISSION_CONSISTENT_READ },

- { BLK_PERM_WRITE, BLOCK_PERMISSION_WRITE },

- { BLK_PERM_WRITE_UNCHANGED, BLOCK_PERMISSION_WRITE_UNCHANGED },

- { BLK_PERM_RESIZE, BLOCK_PERMISSION_RESIZE },

- { BLK_PERM_GRAPH_MOD, BLOCK_PERMISSION_GRAPH_MOD },

- { 0, 0 }

- };

- const PermissionMap *p;

+ BlockPermission qapi_perm;

XDbgBlockGraphEdge *edge;

- QEMU_BUILD_BUG_ON(1UL << (ARRAY_SIZE(permissions) - 1) != BLK_PERM_ALL + 1);

-

edge = g_new0(XDbgBlockGraphEdge, 1);

edge->parent = xdbg_graph_node_num(gr, parent);

edge->child = xdbg_graph_node_num(gr, child->bs);

edge->name = g_strdup(child->name);

- for (p = permissions; p->flag; p++) {

- if (p->flag & child->perm) {

- QAPI_LIST_ADD(edge->perm, p->num);

+ for (qapi_perm = 0; qapi_perm < BLOCK_PERMISSION__MAX; qapi_perm++) {

+ uint64_t flag = bdrv_qapi_perm_to_blk_perm(qapi_perm);

+

+ if (flag & child->perm) {

+ QAPI_LIST_ADD(edge->perm, qapi_perm);

}

- if (p->flag & child->shared_perm) {

- QAPI_LIST_ADD(edge->shared_perm, p->num);

+ if (flag & child->shared_perm) {

+ QAPI_LIST_ADD(edge->shared_perm, qapi_perm);

}

}

--

2.24.1

Sometimes it is useful to be able to add a node to the block graph that

takes or unshare a certain set of permissions for debugging purposes.

This patch adds this capability to blkdebug.

(Note that you cannot make blkdebug release or share permissions that it

needs to take or cannot share, because this might result in assertion

failures in the block layer. But if the blkdebug node has no parents,

it will not take any permissions and share everything by default, so you

can then freely choose what permissions to take and share.)

Signed-off-by: Max Reitz <mreitz@redhat.com>

Message-id: 20191108123455.39445-4-mreitz@redhat.com

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

block/blkdebug.c | 93 +++++++++++++++++++++++++++++++++++++++++++-

qapi/block-core.json | 14 ++++++-

2 files changed, 105 insertions(+), 2 deletions(-)

diff --git a/block/blkdebug.c b/block/blkdebug.c

index XXXXXXX..XXXXXXX 100644

--- a/block/blkdebug.c

+++ b/block/blkdebug.c

@@ -XXX,XX +XXX,XX @@

#include "qemu/cutils.h"

#include "qemu/config-file.h"

#include "block/block_int.h"

+#include "block/qdict.h"

#include "qemu/module.h"

#include "qemu/option.h"

+#include "qapi/qapi-visit-block-core.h"

#include "qapi/qmp/qdict.h"

+#include "qapi/qmp/qlist.h"

#include "qapi/qmp/qstring.h"

+#include "qapi/qobject-input-visitor.h"

#include "sysemu/qtest.h"

typedef struct BDRVBlkdebugState {

@@ -XXX,XX +XXX,XX @@ typedef struct BDRVBlkdebugState {

uint64_t opt_discard;

uint64_t max_discard;

+ uint64_t take_child_perms;

+ uint64_t unshare_child_perms;

+

/* For blkdebug_refresh_filename() */

char *config_file;

@@ -XXX,XX +XXX,XX @@ static void blkdebug_parse_filename(const char *filename, QDict *options,

qdict_put_str(options, "x-image", filename);

}

+static int blkdebug_parse_perm_list(uint64_t *dest, QDict *options,

+ const char *prefix, Error **errp)

+{

+ int ret = 0;

+ QDict *subqdict = NULL;

+ QObject *crumpled_subqdict = NULL;

+ Visitor *v = NULL;

+ BlockPermissionList *perm_list = NULL, *element;

+ Error *local_err = NULL;

+

+ *dest = 0;

+

+ qdict_extract_subqdict(options, &subqdict, prefix);

+ if (!qdict_size(subqdict)) {

+ goto out;

+ }

+

+ crumpled_subqdict = qdict_crumple(subqdict, errp);

+ if (!crumpled_subqdict) {

+ ret = -EINVAL;

+ goto out;

+ }

+

+ v = qobject_input_visitor_new(crumpled_subqdict);

+ visit_type_BlockPermissionList(v, NULL, &perm_list, &local_err);

+ if (local_err) {

+ error_propagate(errp, local_err);

+ ret = -EINVAL;

+ goto out;

+ }

+

+ for (element = perm_list; element; element = element->next) {

+ *dest |= bdrv_qapi_perm_to_blk_perm(element->value);

+ }

+

+out:

+ qapi_free_BlockPermissionList(perm_list);

+ visit_free(v);

+ qobject_unref(subqdict);

+ qobject_unref(crumpled_subqdict);

+ return ret;

+}

+

+static int blkdebug_parse_perms(BDRVBlkdebugState *s, QDict *options,

+ Error **errp)

+{

+ int ret;

+

+ ret = blkdebug_parse_perm_list(&s->take_child_perms, options,

+ "take-child-perms.", errp);

+ if (ret < 0) {

+ return ret;

+ }

+

+ ret = blkdebug_parse_perm_list(&s->unshare_child_perms, options,

+ "unshare-child-perms.", errp);

+ if (ret < 0) {

+ return ret;

+ }

+

+ return 0;

+}

+

static QemuOptsList runtime_opts = {

.name = "blkdebug",

.head = QTAILQ_HEAD_INITIALIZER(runtime_opts.head),

@@ -XXX,XX +XXX,XX @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,

/* Set initial state */

s->state = 1;

+ /* Parse permissions modifiers before opening the image file */

+ ret = blkdebug_parse_perms(s, options, errp);

+ if (ret < 0) {

+ goto out;

+ }

+

/* Open the image file */

bs->file = bdrv_open_child(qemu_opt_get(opts, "x-image"), options, "image",

bs, &child_file, false, &local_err);

@@ -XXX,XX +XXX,XX @@ static int blkdebug_reopen_prepare(BDRVReopenState *reopen_state,

return 0;

}

+static void blkdebug_child_perm(BlockDriverState *bs, BdrvChild *c,

+ const BdrvChildRole *role,

+ BlockReopenQueue *reopen_queue,

+ uint64_t perm, uint64_t shared,

+ uint64_t *nperm, uint64_t *nshared)

+{

+ BDRVBlkdebugState *s = bs->opaque;

+

+ bdrv_filter_default_perms(bs, c, role, reopen_queue, perm, shared,

+ nperm, nshared);

+

+ *nperm |= s->take_child_perms;

+ *nshared &= ~s->unshare_child_perms;

+}

+

static const char *const blkdebug_strong_runtime_opts[] = {

"config",

"inject-error.",

@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_blkdebug = {

.bdrv_file_open = blkdebug_open,

.bdrv_close = blkdebug_close,

.bdrv_reopen_prepare = blkdebug_reopen_prepare,

- .bdrv_child_perm = bdrv_filter_default_perms,

+ .bdrv_child_perm = blkdebug_child_perm,

.bdrv_getlength = blkdebug_getlength,

.bdrv_refresh_filename = blkdebug_refresh_filename,

diff --git a/qapi/block-core.json b/qapi/block-core.json

index XXXXXXX..XXXXXXX 100644

--- a/qapi/block-core.json

+++ b/qapi/block-core.json

@@ -XXX,XX +XXX,XX @@

#

# @set-state: array of state-change descriptions

#

+# @take-child-perms: Permissions to take on @image in addition to what

+# is necessary anyway (which depends on how the

+# blkdebug node is used). Defaults to none.

+# (since 5.0)

+#

+# @unshare-child-perms: Permissions not to share on @image in addition

+# to what cannot be shared anyway (which depends

+# on how the blkdebug node is used). Defaults

+# to none. (since 5.0)

+#

# Since: 2.9

##

{ 'struct': 'BlockdevOptionsBlkdebug',

@@ -XXX,XX +XXX,XX @@

'*opt-write-zero': 'int32', '*max-write-zero': 'int32',

'*opt-discard': 'int32', '*max-discard': 'int32',

'*inject-error': ['BlkdebugInjectErrorOptions'],

- '*set-state': ['BlkdebugSetStateOptions'] } }

+ '*set-state': ['BlkdebugSetStateOptions'],

+ '*take-child-perms': ['BlockPermission'],

+ '*unshare-child-perms': ['BlockPermission'] } }

##

# @BlockdevOptionsBlklogwrites:

--

2.24.1

Callers can use this new parameter to expect failure during the

completion process.

Signed-off-by: Max Reitz <mreitz@redhat.com>

Reviewed-by: John Snow <jsnow@redhat.com>

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Message-id: 20191108123455.39445-5-mreitz@redhat.com

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

tests/qemu-iotests/iotests.py | 18 ++++++++++++------

1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/iotests.py

+++ b/tests/qemu-iotests/iotests.py

@@ -XXX,XX +XXX,XX @@ class QMPTestCase(unittest.TestCase):

self.assert_no_active_block_jobs()

return result

- def wait_until_completed(self, drive='drive0', check_offset=True, wait=60.0):

+ def wait_until_completed(self, drive='drive0', check_offset=True, wait=60.0,

+ error=None):

'''Wait for a block job to finish, returning the event'''

while True:

for event in self.vm.get_qmp_events(wait=wait):

if event['event'] == 'BLOCK_JOB_COMPLETED':

self.assert_qmp(event, 'data/device', drive)

- self.assert_qmp_absent(event, 'data/error')

- if check_offset:

- self.assert_qmp(event, 'data/offset', event['data']['len'])

+ if error is None:

+ self.assert_qmp_absent(event, 'data/error')

+ if check_offset:

+ self.assert_qmp(event, 'data/offset',

+ event['data']['len'])

+ else:

+ self.assert_qmp(event, 'data/error', error)

self.assert_no_active_block_jobs()

return event

elif event['event'] == 'JOB_STATUS_CHANGE':

@@ -XXX,XX +XXX,XX @@ class QMPTestCase(unittest.TestCase):

self.assert_qmp(event, 'data/type', 'mirror')

self.assert_qmp(event, 'data/offset', event['data']['len'])

- def complete_and_wait(self, drive='drive0', wait_ready=True):

+ def complete_and_wait(self, drive='drive0', wait_ready=True,

+ completion_error=None):

'''Complete a block job and wait for it to finish'''

if wait_ready:

self.wait_ready(drive=drive)

@@ -XXX,XX +XXX,XX @@ class QMPTestCase(unittest.TestCase):

result = self.vm.qmp('block-job-complete', device=drive)

self.assert_qmp(result, 'return', {})

- event = self.wait_until_completed(drive=drive)

+ event = self.wait_until_completed(drive=drive, error=completion_error)

self.assert_qmp(event, 'data/type', 'mirror')

def pause_wait(self, job_id='job0'):

--

2.24.1

Signed-off-by: Max Reitz <mreitz@redhat.com>

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Reviewed-by: John Snow <jsnow@redhat.com>

Message-id: 20191108123455.39445-6-mreitz@redhat.com

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

tests/qemu-iotests/041 | 44 ++++++++++++++++++++++++++++++++++++++

tests/qemu-iotests/041.out | 4 ++--

2 files changed, 46 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/041

+++ b/tests/qemu-iotests/041

@@ -XXX,XX +XXX,XX @@ class TestOrphanedSource(iotests.QMPTestCase):

target='dest-ro')

self.assert_qmp(result, 'error/class', 'GenericError')

+ def test_failing_permission_in_complete(self):

+ self.assert_no_active_block_jobs()

+

+ # Unshare consistent-read on the target

+ # (The mirror job does not care)

+ result = self.vm.qmp('blockdev-add',

+ driver='blkdebug',

+ node_name='dest-perm',

+ image='dest',

+ unshare_child_perms=['consistent-read'])

+ self.assert_qmp(result, 'return', {})

+

+ result = self.vm.qmp('blockdev-mirror', job_id='job', device='src',

+ sync='full', target='dest',

+ filter_node_name='mirror-filter')

+ self.assert_qmp(result, 'return', {})

+

+ # Require consistent-read on the source

+ # (We can only add this node once the job has started, or it

+ # will complain that it does not want to run on non-root nodes)

+ result = self.vm.qmp('blockdev-add',

+ driver='blkdebug',

+ node_name='src-perm',

+ image='src',

+ take_child_perms=['consistent-read'])

+ self.assert_qmp(result, 'return', {})

+

+ # While completing, mirror will attempt to replace src by

+ # dest, which must fail because src-perm requires

+ # consistent-read but dest-perm does not share it; thus

+ # aborting the job when it is supposed to complete

+ self.complete_and_wait('job',

+ completion_error='Operation not permitted')

+

+ # Assert that all of our nodes are still there (except for the

+ # mirror filter, which should be gone despite the failure)

+ nodes = self.vm.qmp('query-named-block-nodes')['return']

+ nodes = [node['node-name'] for node in nodes]

+

+ for expect in ('src', 'src-perm', 'dest', 'dest-perm'):

+ self.assertTrue(expect in nodes, '%s disappeared' % expect)

+ self.assertFalse('mirror-filter' in nodes,

+ 'Mirror filter node did not disappear')

+

if __name__ == '__main__':

iotests.main(supported_fmts=['qcow2', 'qed'],

supported_protocols=['file'])

diff --git a/tests/qemu-iotests/041.out b/tests/qemu-iotests/041.out

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/041.out

+++ b/tests/qemu-iotests/041.out

@@ -XXX,XX +XXX,XX @@

-..........................................................................................

+...........................................................................................

----------------------------------------------------------------------

-Ran 90 tests

+Ran 91 tests

OK

--

2.24.1

From: PanNengyuan <pannengyuan@huawei.com>

This avoid a memory leak when qom-set is called to set throttle_group

limits, here is an easy way to reproduce:

1. run qemu-iotests as follow and check the result with asan:

./check -qcow2 184

Following is the asan output backtrack:

Direct leak of 912 byte(s) in 3 object(s) allocated from:

#0 0xffff8d7ab3c3 in __interceptor_calloc (/lib64/libasan.so.4+0xd33c3)

#1 0xffff8d4c31cb in g_malloc0 (/lib64/libglib-2.0.so.0+0x571cb)

#2 0x190c857 in qobject_input_start_struct /mnt/sdc/qemu-master/qemu-4.2.0-rc0/qapi/qobject-input-visitor.c:295

#3 0x19070df in visit_start_struct /mnt/sdc/qemu-master/qemu-4.2.0-rc0/qapi/qapi-visit-core.c:49

#4 0x1948b87 in visit_type_ThrottleLimits qapi/qapi-visit-block-core.c:3759

#5 0x17e4aa3 in throttle_group_set_limits /mnt/sdc/qemu-master/qemu-4.2.0-rc0/block/throttle-groups.c:900

#6 0x1650eff in object_property_set /mnt/sdc/qemu-master/qemu-4.2.0-rc0/qom/object.c:1272

#7 0x1658517 in object_property_set_qobject /mnt/sdc/qemu-master/qemu-4.2.0-rc0/qom/qom-qobject.c:26

#8 0x15880bb in qmp_qom_set /mnt/sdc/qemu-master/qemu-4.2.0-rc0/qom/qom-qmp-cmds.c:74

#9 0x157e3e3 in qmp_marshal_qom_set qapi/qapi-commands-qom.c:154

Reported-by: Euler Robot <euler.robot@huawei.com>

Signed-off-by: PanNengyuan <pannengyuan@huawei.com>

Message-id: 1574835614-42028-1-git-send-email-pannengyuan@huawei.com

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

block/throttle-groups.c | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/block/throttle-groups.c b/block/throttle-groups.c

index XXXXXXX..XXXXXXX 100644

--- a/block/throttle-groups.c

+++ b/block/throttle-groups.c

@@ -XXX,XX +XXX,XX @@ static void throttle_group_set_limits(Object *obj, Visitor *v,

{

ThrottleGroup *tg = THROTTLE_GROUP(obj);

ThrottleConfig cfg;

- ThrottleLimits arg = { 0 };

- ThrottleLimits *argp = &arg;

+ ThrottleLimits *argp;

Error *local_err = NULL;

visit_type_ThrottleLimits(v, name, &argp, &local_err);

@@ -XXX,XX +XXX,XX @@ static void throttle_group_set_limits(Object *obj, Visitor *v,

unlock:

qemu_mutex_unlock(&tg->lock);

ret:

+ qapi_free_ThrottleLimits(argp);

error_propagate(errp, local_err);

return;

}

--

2.24.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

qcow2_can_store_new_dirty_bitmap works wrong, as it considers only

bitmaps already stored in the qcow2 image and ignores persistent

BdrvDirtyBitmap objects.

So, let's instead count persistent BdrvDirtyBitmaps. We load all qcow2

bitmaps on open, so there should not be any bitmap in the image for

which we don't have BdrvDirtyBitmaps version. If it is - it's a kind of

corruption, and no reason to check for corruptions here (open() and

close() are better places for it).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Message-id: 20191014115126.15360-2-vsementsov@virtuozzo.com

Reviewed-by: Max Reitz <mreitz@redhat.com>

Cc: qemu-stable@nongnu.org

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

block/qcow2-bitmap.c | 41 ++++++++++++++++++-----------------------

1 file changed, 18 insertions(+), 23 deletions(-)

diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c

index XXXXXXX..XXXXXXX 100644

--- a/block/qcow2-bitmap.c

+++ b/block/qcow2-bitmap.c

@@ -XXX,XX +XXX,XX @@ bool coroutine_fn qcow2_co_can_store_new_dirty_bitmap(BlockDriverState *bs,

Error **errp)

{

BDRVQcow2State *s = bs->opaque;

- bool found;

- Qcow2BitmapList *bm_list;

+ BdrvDirtyBitmap *bitmap;

+ uint64_t bitmap_directory_size = 0;

+ uint32_t nb_bitmaps = 0;

+

+ if (bdrv_find_dirty_bitmap(bs, name)) {

+ error_setg(errp, "Bitmap already exists: %s", name);

+ return false;

+ }

if (s->qcow_version < 3) {

/* Without autoclear_features, we would always have to assume

@@ -XXX,XX +XXX,XX @@ bool coroutine_fn qcow2_co_can_store_new_dirty_bitmap(BlockDriverState *bs,

goto fail;

}

- if (s->nb_bitmaps == 0) {

- return true;

+ FOR_EACH_DIRTY_BITMAP(bs, bitmap) {

+ if (bdrv_dirty_bitmap_get_persistence(bitmap)) {

+ nb_bitmaps++;

+ bitmap_directory_size +=

+ calc_dir_entry_size(strlen(bdrv_dirty_bitmap_name(bitmap)), 0);

+ }

}

+ nb_bitmaps++;

+ bitmap_directory_size += calc_dir_entry_size(strlen(name), 0);

- if (s->nb_bitmaps >= QCOW2_MAX_BITMAPS) {

+ if (nb_bitmaps > QCOW2_MAX_BITMAPS) {

error_setg(errp,

"Maximum number of persistent bitmaps is already reached");

goto fail;

}

- if (s->bitmap_directory_size + calc_dir_entry_size(strlen(name), 0) >

- QCOW2_MAX_BITMAP_DIRECTORY_SIZE)

- {

+ if (bitmap_directory_size > QCOW2_MAX_BITMAP_DIRECTORY_SIZE) {

error_setg(errp, "Not enough space in the bitmap directory");

goto fail;

}

- qemu_co_mutex_lock(&s->lock);

- bm_list = bitmap_list_load(bs, s->bitmap_directory_offset,

- s->bitmap_directory_size, errp);

- qemu_co_mutex_unlock(&s->lock);

- if (bm_list == NULL) {

- goto fail;

- }

-

- found = find_bitmap_by_name(bm_list, name);

- bitmap_list_free(bm_list);

- if (found) {

- error_setg(errp, "Bitmap with the same name is already stored");

- goto fail;

- }

-

return true;

fail:

--

2.24.1

Probably due to blind copy-pasting, we have several instances of "qocw2"

in our iotests. Fix them.

Reported-by: Maxim Levitsky <mlevitsk@redhat.com>

Signed-off-by: Max Reitz <mreitz@redhat.com>

Message-id: 20191107163708.833192-2-mreitz@redhat.com

Reviewed-by: Eric Blake <eblake@redhat.com>

Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

tests/qemu-iotests/060 | 2 +-

tests/qemu-iotests/061 | 2 +-

tests/qemu-iotests/062 | 2 +-

tests/qemu-iotests/066 | 2 +-

tests/qemu-iotests/068 | 2 +-

tests/qemu-iotests/108 | 2 +-

tests/qemu-iotests/138 | 2 +-

tests/qemu-iotests/261 | 2 +-

8 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/060

+++ b/tests/qemu-iotests/060

@@ -XXX,XX +XXX,XX @@ _filter_io_error()

. ./common.rc

. ./common.filter

-# This tests qocw2-specific low-level functionality

+# This tests qcow2-specific low-level functionality

_supported_fmt qcow2

_supported_proto file

_supported_os Linux

diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/061

+++ b/tests/qemu-iotests/061

@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15

. ./common.rc

. ./common.filter

-# This tests qocw2-specific low-level functionality

+# This tests qcow2-specific low-level functionality

_supported_fmt qcow2

_supported_proto file

_supported_os Linux

diff --git a/tests/qemu-iotests/062 b/tests/qemu-iotests/062

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/062

+++ b/tests/qemu-iotests/062

@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15

. ./common.rc

. ./common.filter

-# This tests qocw2-specific low-level functionality

+# This tests qcow2-specific low-level functionality

_supported_fmt qcow2

_supported_proto generic

diff --git a/tests/qemu-iotests/066 b/tests/qemu-iotests/066

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/066

+++ b/tests/qemu-iotests/066

@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15

. ./common.rc

. ./common.filter

-# This tests qocw2-specific low-level functionality

+# This tests qcow2-specific low-level functionality

_supported_fmt qcow2

_supported_proto generic

diff --git a/tests/qemu-iotests/068 b/tests/qemu-iotests/068

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/068

+++ b/tests/qemu-iotests/068

@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15

. ./common.rc

. ./common.filter

-# This tests qocw2-specific low-level functionality

+# This tests qcow2-specific low-level functionality

_supported_fmt qcow2

_supported_proto generic

diff --git a/tests/qemu-iotests/108 b/tests/qemu-iotests/108

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/108

+++ b/tests/qemu-iotests/108

@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15

. ./common.rc

. ./common.filter

-# This tests qocw2-specific low-level functionality

+# This tests qcow2-specific low-level functionality

_supported_fmt qcow2

_supported_proto file

_supported_os Linux

diff --git a/tests/qemu-iotests/138 b/tests/qemu-iotests/138

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/138

+++ b/tests/qemu-iotests/138

@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15

. ./common.rc

. ./common.filter

-# This tests qocw2-specific low-level functionality

+# This tests qcow2-specific low-level functionality

_supported_fmt qcow2

_supported_proto file

_supported_os Linux

diff --git a/tests/qemu-iotests/261 b/tests/qemu-iotests/261

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/261

+++ b/tests/qemu-iotests/261

@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15

. ./common.rc

. ./common.filter

-# This tests qocw2-specific low-level functionality

+# This tests qcow2-specific low-level functionality

_supported_fmt qcow2

_supported_proto file

_supported_os Linux

--

2.24.1

This is useful for tests that want to whitelist fields from dump-header

(with grep) but still print all header extensions.

Signed-off-by: Max Reitz <mreitz@redhat.com>

Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>

Message-id: 20191107163708.833192-3-mreitz@redhat.com

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

tests/qemu-iotests/qcow2.py | 5 +++++

1 file changed, 5 insertions(+)

diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py

index XXXXXXX..XXXXXXX 100755

--- a/tests/qemu-iotests/qcow2.py

+++ b/tests/qemu-iotests/qcow2.py

@@ -XXX,XX +XXX,XX @@ def cmd_dump_header(fd):

h.dump()

h.dump_extensions()

+def cmd_dump_header_exts(fd):

+ h = QcowHeader(fd)

+ h.dump_extensions()

+

def cmd_set_header(fd, name, value):

try:

value = int(value, 0)

@@ -XXX,XX +XXX,XX @@ def cmd_set_feature_bit(fd, group, bit):

cmds = [

[ 'dump-header', cmd_dump_header, 0, 'Dump image header and header extensions' ],

+ [ 'dump-header-exts', cmd_dump_header_exts, 0, 'Dump image header extensions' ],

[ 'set-header', cmd_set_header, 2, 'Set a field in the header'],

[ 'add-header-ext', cmd_add_header_ext, 2, 'Add a header extension' ],

[ 'add-header-ext-stdio', cmd_add_header_ext_stdio, 1, 'Add a header extension, data from stdin' ],

--

2.24.1

Print the feature fields as a set of bits so that filtering is easier.

Signed-off-by: Max Reitz <mreitz@redhat.com>

Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>

Message-id: 20191107163708.833192-4-mreitz@redhat.com

Signed-off-by: Max Reitz <mreitz@redhat.com>

---

tests/qemu-iotests/031.out | 36 +++++++++----------

tests/qemu-iotests/036.out | 18 +++++-----

tests/qemu-iotests/039.out | 22 ++++++------

tests/qemu-iotests/060.out | 20 +++++------

tests/qemu-iotests/061.out | 72 ++++++++++++++++++-------------------

tests/qemu-iotests/137.out | 2 +-

tests/qemu-iotests/qcow2.py | 18 +++++++---

7 files changed, 99 insertions(+), 89 deletions(-)

diff --git a/tests/qemu-iotests/031.out b/tests/qemu-iotests/031.out

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/031.out

+++ b/tests/qemu-iotests/031.out

@@ -XXX,XX +XXX,XX @@ refcount_table_offset 0x10000

refcount_table_clusters 1

nb_snapshots 0

snapshot_offset 0x0

-incompatible_features 0x0

-compatible_features 0x0

-autoclear_features 0x0

+incompatible_features []

+compatible_features []

+autoclear_features []

refcount_order 4

header_length 72

@@ -XXX,XX +XXX,XX @@ refcount_table_offset 0x10000

refcount_table_clusters 1

nb_snapshots 0

snapshot_offset 0x0

-incompatible_features 0x0

-compatible_features 0x0

-autoclear_features 0x0

+incompatible_features []

+compatible_features []

+autoclear_features []

refcount_order 4

header_length 72

@@ -XXX,XX +XXX,XX @@ refcount_table_offset 0x10000

refcount_table_clusters 1

nb_snapshots 0

snapshot_offset 0x0

-incompatible_features 0x0

-compatible_features 0x0

-autoclear_features 0x0

+incompatible_features []

+compatible_features []

+autoclear_features []

refcount_order 4

header_length 72

@@ -XXX,XX +XXX,XX @@ refcount_table_offset 0x10000

refcount_table_clusters 1

nb_snapshots 0

snapshot_offset 0x0

-incompatible_features 0x0

-compatible_features 0x0

-autoclear_features 0x0

+incompatible_features []

+compatible_features []

+autoclear_features []

refcount_order 4

header_length 104

@@ -XXX,XX +XXX,XX @@ refcount_table_offset 0x10000

refcount_table_clusters 1

nb_snapshots 0

snapshot_offset 0x0

-incompatible_features 0x0

-compatible_features 0x0

-autoclear_features 0x0

+incompatible_features []

+compatible_features []

+autoclear_features []

refcount_order 4

header_length 104

@@ -XXX,XX +XXX,XX @@ refcount_table_offset 0x10000

refcount_table_clusters 1

nb_snapshots 0

snapshot_offset 0x0

-incompatible_features 0x0

-compatible_features 0x0

-autoclear_features 0x0

+incompatible_features []

+compatible_features []

+autoclear_features []

refcount_order 4

header_length 104

diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/036.out

+++ b/tests/qemu-iotests/036.out

@@ -XXX,XX +XXX,XX @@ refcount_table_offset 0x10000

refcount_table_clusters 1

nb_snapshots 0

snapshot_offset 0x0

-incompatible_features 0x8000000000000000

-compatible_features 0x0

-autoclear_features 0x0

+incompatible_features [63]

+compatible_features []

+autoclear_features []

refcount_order 4

header_length 104

@@ -XXX,XX +XXX,XX @@ refcount_table_offset 0x10000

refcount_table_clusters 1

nb_snapshots 0

snapshot_offset 0x0

-incompatible_features 0x0

-compatible_features 0x0

-autoclear_features 0x8000000000000000

+incompatible_features []

+compatible_features []

+autoclear_features [63]

refcount_order 4

header_length 104

@@ -XXX,XX +XXX,XX @@ refcount_table_offset 0x10000

refcount_table_clusters 1

nb_snapshots 0

snapshot_offset 0x0

-incompatible_features 0x0

-compatible_features 0x0

-autoclear_features 0x0

+incompatible_features []

+compatible_features []

+autoclear_features []

refcount_order 4

header_length 104

diff --git a/tests/qemu-iotests/039.out b/tests/qemu-iotests/039.out

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/039.out

+++ b/tests/qemu-iotests/039.out

@@ -XXX,XX +XXX,XX @@ QA output created by 039

Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728

wrote 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

-incompatible_features 0x0

+incompatible_features []

No errors were found on the image.

== Creating a dirty image file ==

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728

wrote 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

./common.rc: Killed ( VALGRIND_QEMU="${VALGRIND_QEMU_IO}" _qemu_proc_exec "${VALGRIND_LOGFILE}" "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" )

-incompatible_features 0x1

+incompatible_features [0]

ERROR cluster 5 refcount=0 reference=1

ERROR OFLAG_COPIED data cluster: l2_entry=8000000000050000 refcount=0

@@ -XXX,XX +XXX,XX @@ Data may be corrupted, or further writes to the image may corrupt it.

== Read-only access must still work ==

read 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

-incompatible_features 0x1

+incompatible_features [0]

== Repairing the image file must succeed ==

ERROR cluster 5 refcount=0 reference=1

@@ -XXX,XX +XXX,XX @@ The following inconsistencies were found and repaired:

Double checking the fixed image now...

No errors were found on the image.

-incompatible_features 0x0

+incompatible_features []

== Data should still be accessible after repair ==

read 512/512 bytes at offset 0

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728

wrote 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

./common.rc: Killed ( VALGRIND_QEMU="${VALGRIND_QEMU_IO}" _qemu_proc_exec "${VALGRIND_LOGFILE}" "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" )

-incompatible_features 0x1

+incompatible_features [0]

ERROR cluster 5 refcount=0 reference=1

Rebuilding refcount structure

Repairing cluster 1 refcount=1 reference=0

Repairing cluster 2 refcount=1 reference=0

wrote 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

-incompatible_features 0x0

+incompatible_features []

== Creating an image file with lazy_refcounts=off ==

Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728

wrote 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

./common.rc: Killed ( VALGRIND_QEMU="${VALGRIND_QEMU_IO}" _qemu_proc_exec "${VALGRIND_LOGFILE}" "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" )

-incompatible_features 0x0

+incompatible_features []

No errors were found on the image.

== Committing to a backing file with lazy_refcounts=on ==

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/

wrote 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

Image committed.

-incompatible_features 0x0

-incompatible_features 0x0

+incompatible_features []

+incompatible_features []

No errors were found on the image.

No errors were found on the image.

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728

wrote 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

./common.rc: Killed ( VALGRIND_QEMU="${VALGRIND_QEMU_IO}" _qemu_proc_exec "${VALGRIND_LOGFILE}" "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" )

-incompatible_features 0x1

+incompatible_features [0]

ERROR cluster 5 refcount=0 reference=1

ERROR OFLAG_COPIED data cluster: l2_entry=8000000000050000 refcount=0

@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728

wrote 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

./common.rc: Killed ( VALGRIND_QEMU="${VALGRIND_QEMU_IO}" _qemu_proc_exec "${VALGRIND_LOGFILE}" "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" )

-incompatible_features 0x0

+incompatible_features []

No errors were found on the image.

*** done

diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out

index XXXXXXX..XXXXXXX 100644

--- a/tests/qemu-iotests/060.out

+++ b/tests/qemu-iotests/060.out

@@ -XXX,XX +XXX,XX @@ ERROR cluster 3 refcount=1 reference=3

1 errors were found on the image.

Data may be corrupted, or further writes to the image may corrupt it.

-incompatible_features 0x0

+incompatible_features []

qcow2: Marking image as corrupt: Preventing invalid write on metadata (overlaps with active L1 table); further corruption events will be suppressed

write failed: Input/output error

-incompatible_features 0x2

+incompatible_features [1]

image: TEST_DIR/t.IMGFMT

file format: IMGFMT

virtual size: 64 MiB (67108864 bytes)

@@ -XXX,XX +XXX,XX @@ ERROR cluster 2 refcount=1 reference=2

2 errors were found on the image.

Data may be corrupted, or further writes to the image may corrupt it.

-incompatible_features 0x0

+incompatible_features []

qcow2: Marking image as corrupt: Preventing invalid write on metadata (overlaps with refcount block); further corruption events will be suppressed

write failed: Input/output error

-incompatible_features 0x2

+incompatible_features [1]

ERROR refcount block 0 refcount=2

ERROR cluster 2 refcount=1 reference=2

Rebuilding refcount structure

@@ -XXX,XX +XXX,XX @@ The following inconsistencies were found and repaired:

Double checking the fixed image now...

No errors were found on the image.

-incompatible_features 0x0

+incompatible_features []

wrote 512/512 bytes at offset 0

512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

-incompatible_features 0x0

+incompatible_features []

=== Testing cluster data reference into inactive L2 table ===

@@ -XXX,XX +XXX,XX @@ Data may be corrupted, or further writes to the image may corrupt it.

1 leaked clusters were found on the image.

This means waste of disk space, but no harm to data.

-incompatible_features 0x0

+incompatible_features []

qcow2: Marking image as corrupt: Preventing invalid write on metadata (overlaps with inactive L2 table); further corruption events will be suppressed

write failed: Input/output error

-incompatible_features 0x2

+incompatible_features [1]

ERROR cluster 4 refcount=1 reference=2

Leaked cluster 9 refcount=1 reference=0

Repairing cluster 4 refcount=1 reference=2

@@ -XXX,XX +XXX,XX @@ The following inconsistencies were found and repaired:

Double checking the fixed image now...

No errors were found on the image.

-incompatible_features 0x0

+incompatible_features []