The following changes since commit 91fe7a376ad46e3cc5e82d418aad22173c948a3c: Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2018-06-15 11:41:44 +0100) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream for you to fetch changes up to 6266e900b8083945cb766b45c124fb3c42932cb3: block: Remove dead deprecation warning code (2018-06-15 14:49:44 +0200) ---------------------------------------------------------------- Block layer patches: - Fix options that work only with -drive or -blockdev, but not with both, because of QDict type confusion - rbd: Add options 'auth-client-required' and 'key-secret' - Remove deprecated -drive options serial/addr/cyls/heads/secs/trans - rbd, iscsi: Remove deprecated 'filename' option - Fix 'qemu-img map' crash with unaligned image size - Improve QMP documentation for jobs ---------------------------------------------------------------- Eric Blake (2): qemu-img: Fix assert when mapping unaligned raw file iotests: Add test 221 to catch qemu-img map regression John Snow (2): jobs: fix stale wording jobs: fix verb references in docs Kevin Wolf (4): block: Remove deprecated -drive geometry options block: Remove deprecated -drive option addr block: Remove deprecated -drive option serial block: Remove dead deprecation warning code Markus Armbruster (17): rbd: Drop deprecated -drive parameter "filename" iscsi: Drop deprecated -drive parameter "filename" qobject: Move block-specific qdict code to block-qdict.c block: Fix -blockdev for certain non-string scalars block: Fix -drive for certain non-string scalars block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts() block: Factor out qobject_input_visitor_new_flat_confused() block: Make remaining uses of qobject input visitor more robust block-qdict: Simplify qdict_flatten_qdict() block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist() block-qdict: Clean up qdict_crumple() a bit block-qdict: Simplify qdict_is_list() some check-block-qdict: Rename qdict_flatten()'s variables for clarity check-block-qdict: Cover flattening of empty lists and dictionaries block: Fix -blockdev / blockdev-add for empty objects and arrays rbd: New parameter auth-client-required rbd: New parameter key-secret Max Reitz (1): block: Add block-specific QDict header qapi/block-core.json | 19 ++ qapi/job.json | 23 +- include/block/qdict.h | 34 +++ include/hw/block/block.h | 1 - include/qapi/qmp/qdict.h | 17 -- include/sysemu/blockdev.h | 3 - block.c | 1 + block/block-backend.c | 1 - block/crypto.c | 12 +- block/gluster.c | 1 + block/iscsi.c | 24 +- block/nbd.c | 16 +- block/nfs.c | 8 +- block/parallels.c | 11 +- block/qcow.c | 11 +- block/qcow2.c | 11 +- block/qed.c | 11 +- block/quorum.c | 1 + block/rbd.c | 85 +++--- block/sheepdog.c | 23 +- block/snapshot.c | 1 + block/ssh.c | 16 +- block/vdi.c | 8 +- block/vhdx.c | 11 +- block/vpc.c | 11 +- block/vvfat.c | 1 + block/vxhs.c | 1 + blockdev.c | 111 +------ device-hotplug.c | 4 - hw/block/block.c | 27 -- hw/block/nvme.c | 1 - hw/block/virtio-blk.c | 1 - hw/ide/qdev.c | 1 - hw/scsi/scsi-disk.c | 1 - hw/usb/dev-storage.c | 1 - qemu-img.c | 2 +- qobject/block-qdict.c | 722 +++++++++++++++++++++++++++++++++++++++++++++ qobject/qdict.c | 628 --------------------------------------- tests/ahci-test.c | 6 +- tests/check-block-qdict.c | 690 +++++++++++++++++++++++++++++++++++++++++++ tests/check-qdict.c | 641 ---------------------------------------- tests/check-qobject.c | 1 + tests/hd-geo-test.c | 37 +-- tests/ide-test.c | 8 +- tests/test-replication.c | 1 + util/qemu-config.c | 1 + MAINTAINERS | 2 + hmp-commands.hx | 1 - qemu-doc.texi | 15 - qemu-options.hx | 14 +- qobject/Makefile.objs | 1 + tests/Makefile.include | 4 + tests/qemu-iotests/221 | 60 ++++ tests/qemu-iotests/221.out | 16 + tests/qemu-iotests/group | 1 + 55 files changed, 1692 insertions(+), 1668 deletions(-) create mode 100644 include/block/qdict.h create mode 100644 qobject/block-qdict.c create mode 100644 tests/check-block-qdict.c create mode 100755 tests/qemu-iotests/221 create mode 100644 tests/qemu-iotests/221.out From: Eric Blake <eblake@redhat.com> Commit a290f085 exposed a latent bug in qemu-img map introduced during the conversion of block status to be byte-based. Earlier in commit 5e344dd8, the internal interface get_block_status() switched to take byte-based parameters, but still called a sector-based block layer function; as such, rounding was added in the lone caller to obey the contract. However, commit 237d78f8 changed get_block_status() to truly be byte-based, at which point rounding to sector boundaries can result in calling bdrv_block_status() with 'bytes == 0' (a coding error) when the boundary between data and a hole falls mid-sector (true for the past-EOF implicit hole present in POSIX files). Fix things by removing the rounding that is now no longer necessary. See also https://bugzilla.redhat.com/1589738 Fixes: 237d78f8 Reported-by: Dan Kenigsberg <danken@redhat.com> Reported-by: Nir Soffer <nsoffer@redhat.com> Reported-by: Maor Lipchuk <mlipchuk@redhat.com> CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qemu-img.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static int img_map(int argc, char **argv) int64_t n; /* Probe up to 1 GiB at a time. */ - n = QEMU_ALIGN_DOWN(MIN(1 << 30, length - offset), BDRV_SECTOR_SIZE); + n = MIN(1 << 30, length - offset); ret = get_block_status(bs, offset, n, &next); if (ret < 0) { -- 2.13.6 From: Eric Blake <eblake@redhat.com> Although qemu-img creates aligned files (by rounding up), it must also gracefully handle files that are not sector-aligned. Test that the bug fixed in the previous patch does not recur. It's a bit annoying that we can see the (implicit) hole past the end of the file on to the next sector boundary, so if we ever reach the point where we report a byte-accurate size rather than our current behavior of always rounding up, this test will probably need a slight modification. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/221 | 60 ++++++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/221.out | 16 +++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 77 insertions(+) create mode 100755 tests/qemu-iotests/221 create mode 100644 tests/qemu-iotests/221.out diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221 new file mode 100755 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/221 @@ -XXX,XX +XXX,XX @@ +#!/bin/bash +# +# Test qemu-img vs. unaligned images +# +# Copyright (C) 2018 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +seq="$(basename $0)" +echo "QA output created by $seq" + +here="$PWD" +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt raw +_supported_proto file +_supported_os Linux + +echo +echo "=== Check mapping of unaligned raw image ===" +echo + +_make_test_img 43009 # qemu-img create rounds size up +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +truncate --size=43009 "$TEST_IMG" # so we resize it and check again +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +$QEMU_IO -c 'w 43008 1' "$TEST_IMG" | _filter_qemu_io # writing also rounds up +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +truncate --size=43009 "$TEST_IMG" # so we resize it and check again +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +# success, all done +echo '*** done' +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/221.out @@ -XXX,XX +XXX,XX @@ +QA output created by 221 + +=== Check mapping of unaligned raw image === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=43009 +[{ "start": 0, "length": 43520, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 43520, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +wrote 1/1 bytes at offset 43008 +1 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 40960, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, +{ "start": 40960, "length": 2049, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, +{ "start": 43009, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 40960, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, +{ "start": 40960, "length": 2049, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, +{ "start": 43009, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -XXX,XX +XXX,XX @@ 217 rw auto quick 218 rw auto quick 219 rw auto +221 rw auto quick -- 2.13.6 From: John Snow <jsnow@redhat.com> During the design for manual completion, we decided not to use the "manual" property as a shorthand for both auto-dismiss and auto-finalize. Fix the wording. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/job.json | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/qapi/job.json b/qapi/job.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/job.json +++ b/qapi/job.json @@ -XXX,XX +XXX,XX @@ # the last job in a transaction. # # @pending: The job has finished its work, but has finalization steps that it -# needs to make prior to completing. These changes may require -# manual intervention by the management process if manual was set -# to true. These changes may still fail. +# needs to make prior to completing. These changes will require +# manual intervention via @job-finalize if auto-finalize was set to +# false. These pending changes may still fail. # # @aborting: The job is in the process of being aborted, and will finish with # an error. The job will afterwards report that it is @concluded. # This status may not be visible to the management process. # -# @concluded: The job has finished all work. If manual was set to true, the job -# will remain in the query list until it is dismissed. +# @concluded: The job has finished all work. If auto-dismiss was set to false, +# the job will remain in the query list until it is dismissed via +# @job-dismiss. # # @null: The job is in the process of being dismantled. This state should not # ever be visible externally. -- 2.13.6 From: John Snow <jsnow@redhat.com> These point to the job versions now, not the blockjob versions which don't really exist anymore. Except set-speed, which does. It sticks out like a sore thumb. This patch doesn't fix that, but it doesn't make it any worse, either. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/job.json | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/qapi/job.json b/qapi/job.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/job.json +++ b/qapi/job.json @@ -XXX,XX +XXX,XX @@ # # Represents command verbs that can be applied to a job. # -# @cancel: see @block-job-cancel +# @cancel: see @job-cancel # -# @pause: see @block-job-pause +# @pause: see @job-pause # -# @resume: see @block-job-resume +# @resume: see @job-resume # # @set-speed: see @block-job-set-speed # -# @complete: see @block-job-complete +# @complete: see @job-complete # -# @dismiss: see @block-job-dismiss +# @dismiss: see @job-dismiss # -# @finalize: see @block-job-finalize +# @finalize: see @job-finalize # # Since: 2.12 ## -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Parameter "filename" is deprecated since commit 91589d9e5ca, v2.10.0. Time to get rid of it. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/rbd.c | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, QObject *crumpled = NULL; const QDictEntry *e; Error *local_err = NULL; - const char *filename; char *keypairs, *secretid; int r; - /* If we are given a filename, parse the filename, with precedence given to - * filename encoded options */ - filename = qdict_get_try_str(options, "filename"); - if (filename) { - warn_report("'filename' option specified. " - "This is an unsupported option, and may be deprecated " - "in the future"); - qemu_rbd_parse_filename(filename, options, &local_err); - qdict_del(options, "filename"); - if (local_err) { - error_propagate(errp, local_err); - return -EINVAL; - } - } - keypairs = g_strdup(qdict_get_try_str(options, "=keyvalue-pairs")); if (keypairs) { qdict_del(options, "=keyvalue-pairs"); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Parameter "filename" is deprecated since commit 5c3ad1a6a8f, v2.10.0. Time to get rid of it. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/iscsi.c | 23 ++--------------------- 1 file changed, 2 insertions(+), 21 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index XXXXXXX..XXXXXXX 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -XXX,XX +XXX,XX @@ static QemuOptsList runtime_opts = { .name = "timeout", .type = QEMU_OPT_NUMBER, }, - { - .name = "filename", - .type = QEMU_OPT_STRING, - }, { /* end of list */ } }, }; @@ -XXX,XX +XXX,XX @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags, char *initiator_name = NULL; QemuOpts *opts; Error *local_err = NULL; - const char *transport_name, *portal, *target, *filename; + const char *transport_name, *portal, *target; #if LIBISCSI_API_VERSION >= (20160603) enum iscsi_transport_type transport; #endif int i, ret = 0, timeout = 0, lun; - /* If we are given a filename, parse the filename, with precedence given to - * filename encoded options */ - filename = qdict_get_try_str(options, "filename"); - if (filename) { - warn_report("'filename' option specified. " - "This is an unsupported option, and may be deprecated " - "in the future"); - iscsi_parse_filename(filename, options, &local_err); - if (local_err) { - ret = -EINVAL; - error_propagate(errp, local_err); - goto exit; - } - } - opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); qemu_opts_absorb_qdict(opts, options, &local_err); if (local_err) { @@ -XXX,XX +XXX,XX @@ out: } memset(iscsilun, 0, sizeof(IscsiLun)); } -exit: + return ret; } -- 2.13.6 From: Max Reitz <mreitz@redhat.com> There are numerous QDict functions that have been introduced for and are used only by the block layer. Move their declarations into an own header file to reflect that. While qdict_extract_subqdict() is in fact used outside of the block layer (in util/qemu-config.c), it is still a function related very closely to how the block layer works with nested QDicts, namely by sometimes flattening them. Therefore, its declaration is put into this header as well and util/qemu-config.c includes it with a comment stating exactly which function it needs. Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20180509165530.29561-7-mreitz@redhat.com> [Copyright note tweaked, superfluous includes dropped] Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/qdict.h | 32 ++++++++++++++++++++++++++++++++ include/qapi/qmp/qdict.h | 17 ----------------- block.c | 1 + block/gluster.c | 1 + block/iscsi.c | 1 + block/nbd.c | 1 + block/nfs.c | 1 + block/parallels.c | 1 + block/qcow.c | 1 + block/qcow2.c | 1 + block/qed.c | 1 + block/quorum.c | 1 + block/rbd.c | 1 + block/sheepdog.c | 1 + block/snapshot.c | 1 + block/ssh.c | 1 + block/vhdx.c | 1 + block/vpc.c | 1 + block/vvfat.c | 1 + block/vxhs.c | 1 + blockdev.c | 1 + qobject/qdict.c | 1 + tests/check-qdict.c | 1 + tests/check-qobject.c | 1 + tests/test-replication.c | 1 + util/qemu-config.c | 1 + 26 files changed, 56 insertions(+), 17 deletions(-) create mode 100644 include/block/qdict.h diff --git a/include/block/qdict.h b/include/block/qdict.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/block/qdict.h @@ -XXX,XX +XXX,XX @@ +/* + * Special QDict functions used by the block layer + * + * Copyright (c) 2013-2018 Red Hat, Inc. + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + */ + +#ifndef BLOCK_QDICT_H +#define BLOCK_QDICT_H + +#include "qapi/qmp/qdict.h" + +void qdict_copy_default(QDict *dst, QDict *src, const char *key); +void qdict_set_default_str(QDict *dst, const char *key, const char *val); + +void qdict_join(QDict *dest, QDict *src, bool overwrite); + +void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); +void qdict_array_split(QDict *src, QList **dst); +int qdict_array_entries(QDict *src, const char *subqdict); +QObject *qdict_crumple(const QDict *src, Error **errp); +void qdict_flatten(QDict *qdict); + +typedef struct QDictRenames { + const char *from; + const char *to; +} QDictRenames; +bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp); + +#endif diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h index XXXXXXX..XXXXXXX 100644 --- a/include/qapi/qmp/qdict.h +++ b/include/qapi/qmp/qdict.h @@ -XXX,XX +XXX,XX @@ int64_t qdict_get_try_int(const QDict *qdict, const char *key, bool qdict_get_try_bool(const QDict *qdict, const char *key, bool def_value); const char *qdict_get_try_str(const QDict *qdict, const char *key); -void qdict_copy_default(QDict *dst, QDict *src, const char *key); -void qdict_set_default_str(QDict *dst, const char *key, const char *val); - QDict *qdict_clone_shallow(const QDict *src); -void qdict_flatten(QDict *qdict); - -void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); -void qdict_array_split(QDict *src, QList **dst); -int qdict_array_entries(QDict *src, const char *subqdict); -QObject *qdict_crumple(const QDict *src, Error **errp); - -void qdict_join(QDict *dest, QDict *src, bool overwrite); - -typedef struct QDictRenames { - const char *from; - const char *to; -} QDictRenames; -bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp); #endif /* QDICT_H */ diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ #include "block/block_int.h" #include "block/blockjob.h" #include "block/nbd.h" +#include "block/qdict.h" #include "qemu/error-report.h" #include "module_block.h" #include "qemu/module.h" diff --git a/block/gluster.c b/block/gluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/gluster.c +++ b/block/gluster.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include <glusterfs/api/glfs.h> #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qerror.h" diff --git a/block/iscsi.c b/block/iscsi.c index XXXXXXX..XXXXXXX 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -XXX,XX +XXX,XX @@ #include "qemu/bitops.h" #include "qemu/bitmap.h" #include "block/block_int.h" +#include "block/qdict.h" #include "scsi/constants.h" #include "qemu/iov.h" #include "qemu/option.h" diff --git a/block/nbd.c b/block/nbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "nbd-client.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qemu/uri.h" #include "block/block_int.h" diff --git a/block/nfs.c b/block/nfs.c index XXXXXXX..XXXXXXX 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -XXX,XX +XXX,XX @@ #include "qemu/error-report.h" #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "trace.h" #include "qemu/iov.h" #include "qemu/option.h" diff --git a/block/parallels.c b/block/parallels.c index XXXXXXX..XXXXXXX 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ #include "qapi/error.h" #include "qemu/error-report.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include <zlib.h> diff --git a/block/qed.c b/block/qed.c index XXXXXXX..XXXXXXX 100644 --- a/block/qed.c +++ b/block/qed.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qemu/timer.h" #include "qemu/bswap.h" diff --git a/block/quorum.c b/block/quorum.c index XXXXXXX..XXXXXXX 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -XXX,XX +XXX,XX @@ #include "qemu/cutils.h" #include "qemu/option.h" #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qapi/qapi-events-block.h" #include "qapi/qmp/qdict.h" diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ #include "qemu/error-report.h" #include "qemu/option.h" #include "block/block_int.h" +#include "block/qdict.h" #include "crypto/secret.h" #include "qemu/cutils.h" #include "qapi/qmp/qstring.h" diff --git a/block/sheepdog.c b/block/sheepdog.c index XXXXXXX..XXXXXXX 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -XXX,XX +XXX,XX @@ #include "qemu/option.h" #include "qemu/sockets.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/bitops.h" #include "qemu/cutils.h" diff --git a/block/snapshot.c b/block/snapshot.c index XXXXXXX..XXXXXXX 100644 --- a/block/snapshot.c +++ b/block/snapshot.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "block/snapshot.h" #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qerror.h" diff --git a/block/ssh.c b/block/ssh.c index XXXXXXX..XXXXXXX 100644 --- a/block/ssh.c +++ b/block/ssh.c @@ -XXX,XX +XXX,XX @@ #include <libssh2_sftp.h> #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/option.h" diff --git a/block/vhdx.c b/block/vhdx.c index XXXXXXX..XXXXXXX 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" diff --git a/block/vpc.c b/block/vpc.c index XXXXXXX..XXXXXXX 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ #include <dirent.h> #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "qemu/module.h" #include "qemu/option.h" #include "qemu/bswap.h" diff --git a/block/vxhs.c b/block/vxhs.c index XXXXXXX..XXXXXXX 100644 --- a/block/vxhs.c +++ b/block/vxhs.c @@ -XXX,XX +XXX,XX @@ #include <qnio/qnio_api.h> #include <sys/param.h> #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/qmp/qerror.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qstring.h" diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ #include "sysemu/blockdev.h" #include "hw/block/block.h" #include "block/blockjob.h" +#include "block/qdict.h" #include "block/throttle-groups.h" #include "monitor/monitor.h" #include "qemu/error-report.h" diff --git a/qobject/qdict.c b/qobject/qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/qdict.c +++ b/qobject/qdict.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" +#include "block/qdict.h" #include "qapi/qmp/qnum.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qbool.h" diff --git a/tests/check-qdict.c b/tests/check-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-qdict.c +++ b/tests/check-qdict.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" +#include "block/qdict.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qlist.h" #include "qapi/qmp/qnum.h" diff --git a/tests/check-qobject.c b/tests/check-qobject.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-qobject.c +++ b/tests/check-qobject.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" +#include "block/qdict.h" #include "qapi/qmp/qbool.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qlist.h" diff --git a/tests/test-replication.c b/tests/test-replication.c index XXXXXXX..XXXXXXX 100644 --- a/tests/test-replication.c +++ b/tests/test-replication.c @@ -XXX,XX +XXX,XX @@ #include "qemu/option.h" #include "replication.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #define IMG_SIZE (64 * 1024 * 1024) diff --git a/util/qemu-config.c b/util/qemu-config.c index XXXXXXX..XXXXXXX 100644 --- a/util/qemu-config.c +++ b/util/qemu-config.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" +#include "block/qdict.h" /* for qdict_extract_subqdict() */ #include "qapi/error.h" #include "qapi/qapi-commands-misc.h" #include "qapi/qmp/qdict.h" -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Pure code motion, except for two brace placements and a comment tweaked to appease checkpatch. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 640 ++++++++++++++++++++++++++++++++++++++++++++ qobject/qdict.c | 629 -------------------------------------------- tests/check-block-qdict.c | 655 ++++++++++++++++++++++++++++++++++++++++++++++ tests/check-qdict.c | 642 --------------------------------------------- MAINTAINERS | 2 + qobject/Makefile.objs | 1 + tests/Makefile.include | 4 + 7 files changed, 1302 insertions(+), 1271 deletions(-) create mode 100644 qobject/block-qdict.c create mode 100644 tests/check-block-qdict.c diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ +/* + * Special QDict functions used by the block layer + * + * Copyright (c) 2013-2018 Red Hat, Inc. + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "block/qdict.h" +#include "qapi/qmp/qlist.h" +#include "qemu/cutils.h" +#include "qapi/error.h" + +/** + * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the + * value of 'key' in 'src' is copied there (and the refcount increased + * accordingly). + */ +void qdict_copy_default(QDict *dst, QDict *src, const char *key) +{ + QObject *val; + + if (qdict_haskey(dst, key)) { + return; + } + + val = qdict_get(src, key); + if (val) { + qdict_put_obj(dst, key, qobject_ref(val)); + } +} + +/** + * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a + * new QString initialised by 'val' is put there. + */ +void qdict_set_default_str(QDict *dst, const char *key, const char *val) +{ + if (qdict_haskey(dst, key)) { + return; + } + + qdict_put_str(dst, key, val); +} + +static void qdict_flatten_qdict(QDict *qdict, QDict *target, + const char *prefix); + +static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) +{ + QObject *value; + const QListEntry *entry; + char *new_key; + int i; + + /* This function is never called with prefix == NULL, i.e., it is always + * called from within qdict_flatten_q(list|dict)(). Therefore, it does not + * need to remove list entries during the iteration (the whole list will be + * deleted eventually anyway from qdict_flatten_qdict()). */ + assert(prefix); + + entry = qlist_first(qlist); + + for (i = 0; entry; entry = qlist_next(entry), i++) { + value = qlist_entry_obj(entry); + new_key = g_strdup_printf("%s.%i", prefix, i); + + if (qobject_type(value) == QTYPE_QDICT) { + qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); + } else if (qobject_type(value) == QTYPE_QLIST) { + qdict_flatten_qlist(qobject_to(QList, value), target, new_key); + } else { + /* All other types are moved to the target unchanged. */ + qdict_put_obj(target, new_key, qobject_ref(value)); + } + + g_free(new_key); + } +} + +static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) +{ + QObject *value; + const QDictEntry *entry, *next; + char *new_key; + bool delete; + + entry = qdict_first(qdict); + + while (entry != NULL) { + + next = qdict_next(qdict, entry); + value = qdict_entry_value(entry); + new_key = NULL; + delete = false; + + if (prefix) { + new_key = g_strdup_printf("%s.%s", prefix, entry->key); + } + + if (qobject_type(value) == QTYPE_QDICT) { + /* Entries of QDicts are processed recursively, the QDict object + * itself disappears. */ + qdict_flatten_qdict(qobject_to(QDict, value), target, + new_key ? new_key : entry->key); + delete = true; + } else if (qobject_type(value) == QTYPE_QLIST) { + qdict_flatten_qlist(qobject_to(QList, value), target, + new_key ? new_key : entry->key); + delete = true; + } else if (prefix) { + /* All other objects are moved to the target unchanged. */ + qdict_put_obj(target, new_key, qobject_ref(value)); + delete = true; + } + + g_free(new_key); + + if (delete) { + qdict_del(qdict, entry->key); + + /* Restart loop after modifying the iterated QDict */ + entry = qdict_first(qdict); + continue; + } + + entry = next; + } +} + +/** + * qdict_flatten(): For each nested QDict with key x, all fields with key y + * are moved to this QDict and their key is renamed to "x.y". For each nested + * QList with key x, the field at index y is moved to this QDict with the key + * "x.y" (i.e., the reverse of what qdict_array_split() does). + * This operation is applied recursively for nested QDicts and QLists. + */ +void qdict_flatten(QDict *qdict) +{ + qdict_flatten_qdict(qdict, qdict, NULL); +} + +/* extract all the src QDict entries starting by start into dst */ +void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start) + +{ + const QDictEntry *entry, *next; + const char *p; + + *dst = qdict_new(); + entry = qdict_first(src); + + while (entry != NULL) { + next = qdict_next(src, entry); + if (strstart(entry->key, start, &p)) { + qdict_put_obj(*dst, p, qobject_ref(entry->value)); + qdict_del(src, entry->key); + } + entry = next; + } +} + +static int qdict_count_prefixed_entries(const QDict *src, const char *start) +{ + const QDictEntry *entry; + int count = 0; + + for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { + if (strstart(entry->key, start, NULL)) { + if (count == INT_MAX) { + return -ERANGE; + } + count++; + } + } + + return count; +} + +/** + * qdict_array_split(): This function moves array-like elements of a QDict into + * a new QList. Every entry in the original QDict with a key "%u" or one + * prefixed "%u.", where %u designates an unsigned integer starting at 0 and + * incrementally counting up, will be moved to a new QDict at index %u in the + * output QList with the key prefix removed, if that prefix is "%u.". If the + * whole key is just "%u", the whole QObject will be moved unchanged without + * creating a new QDict. The function terminates when there is no entry in the + * QDict with a prefix directly (incrementally) following the last one; it also + * returns if there are both entries with "%u" and "%u." for the same index %u. + * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66} + * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66}) + * => [{"a": 42, "b": 23}, {"x": 0}, 66] + * and {"4.y": 1, "o.o": 7} (remainder of the old QDict) + */ +void qdict_array_split(QDict *src, QList **dst) +{ + unsigned i; + + *dst = qlist_new(); + + for (i = 0; i < UINT_MAX; i++) { + QObject *subqobj; + bool is_subqdict; + QDict *subqdict; + char indexstr[32], prefix[32]; + size_t snprintf_ret; + + snprintf_ret = snprintf(indexstr, 32, "%u", i); + assert(snprintf_ret < 32); + + subqobj = qdict_get(src, indexstr); + + snprintf_ret = snprintf(prefix, 32, "%u.", i); + assert(snprintf_ret < 32); + + /* Overflow is the same as positive non-zero results */ + is_subqdict = qdict_count_prefixed_entries(src, prefix); + + /* + * There may be either a single subordinate object (named + * "%u") or multiple objects (each with a key prefixed "%u."), + * but not both. + */ + if (!subqobj == !is_subqdict) { + break; + } + + if (is_subqdict) { + qdict_extract_subqdict(src, &subqdict, prefix); + assert(qdict_size(subqdict) > 0); + } else { + qobject_ref(subqobj); + qdict_del(src, indexstr); + } + + qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict)); + } +} + +/** + * qdict_split_flat_key: + * @key: the key string to split + * @prefix: non-NULL pointer to hold extracted prefix + * @suffix: non-NULL pointer to remaining suffix + * + * Given a flattened key such as 'foo.0.bar', split it into two parts + * at the first '.' separator. Allows double dot ('..') to escape the + * normal separator. + * + * e.g. + * 'foo.0.bar' -> prefix='foo' and suffix='0.bar' + * 'foo..0.bar' -> prefix='foo.0' and suffix='bar' + * + * The '..' sequence will be unescaped in the returned 'prefix' + * string. The 'suffix' string will be left in escaped format, so it + * can be fed back into the qdict_split_flat_key() key as the input + * later. + * + * The caller is responsible for freeing the string returned in @prefix + * using g_free(). + */ +static void qdict_split_flat_key(const char *key, char **prefix, + const char **suffix) +{ + const char *separator; + size_t i, j; + + /* Find first '.' separator, but if there is a pair '..' + * that acts as an escape, so skip over '..' */ + separator = NULL; + do { + if (separator) { + separator += 2; + } else { + separator = key; + } + separator = strchr(separator, '.'); + } while (separator && separator[1] == '.'); + + if (separator) { + *prefix = g_strndup(key, separator - key); + *suffix = separator + 1; + } else { + *prefix = g_strdup(key); + *suffix = NULL; + } + + /* Unescape the '..' sequence into '.' */ + for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) { + if ((*prefix)[i] == '.') { + assert((*prefix)[i + 1] == '.'); + i++; + } + (*prefix)[j] = (*prefix)[i]; + } + (*prefix)[j] = '\0'; +} + +/** + * qdict_is_list: + * @maybe_list: dict to check if keys represent list elements. + * + * Determine whether all keys in @maybe_list are valid list elements. + * If @maybe_list is non-zero in length and all the keys look like + * valid list indexes, this will return 1. If @maybe_list is zero + * length or all keys are non-numeric then it will return 0 to indicate + * it is a normal qdict. If there is a mix of numeric and non-numeric + * keys, or the list indexes are non-contiguous, an error is reported. + * + * Returns: 1 if a valid list, 0 if a dict, -1 on error + */ +static int qdict_is_list(QDict *maybe_list, Error **errp) +{ + const QDictEntry *ent; + ssize_t len = 0; + ssize_t max = -1; + int is_list = -1; + int64_t val; + + for (ent = qdict_first(maybe_list); ent != NULL; + ent = qdict_next(maybe_list, ent)) { + + if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) { + if (is_list == -1) { + is_list = 1; + } else if (!is_list) { + error_setg(errp, + "Cannot mix list and non-list keys"); + return -1; + } + len++; + if (val > max) { + max = val; + } + } else { + if (is_list == -1) { + is_list = 0; + } else if (is_list) { + error_setg(errp, + "Cannot mix list and non-list keys"); + return -1; + } + } + } + + if (is_list == -1) { + assert(!qdict_size(maybe_list)); + is_list = 0; + } + + /* NB this isn't a perfect check - e.g. it won't catch + * a list containing '1', '+1', '01', '3', but that + * does not matter - we've still proved that the + * input is a list. It is up the caller to do a + * stricter check if desired */ + if (len != (max + 1)) { + error_setg(errp, "List indices are not contiguous, " + "saw %zd elements but %zd largest index", + len, max); + return -1; + } + + return is_list; +} + +/** + * qdict_crumple: + * @src: the original flat dictionary (only scalar values) to crumple + * + * Takes a flat dictionary whose keys use '.' separator to indicate + * nesting, and values are scalars, and crumples it into a nested + * structure. + * + * To include a literal '.' in a key name, it must be escaped as '..' + * + * For example, an input of: + * + * { 'foo.0.bar': 'one', 'foo.0.wizz': '1', + * 'foo.1.bar': 'two', 'foo.1.wizz': '2' } + * + * will result in an output of: + * + * { + * 'foo': [ + * { 'bar': 'one', 'wizz': '1' }, + * { 'bar': 'two', 'wizz': '2' } + * ], + * } + * + * The following scenarios in the input dict will result in an + * error being returned: + * + * - Any values in @src are non-scalar types + * - If keys in @src imply that a particular level is both a + * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar". + * - If keys in @src imply that a particular level is a list, + * but the indices are non-contiguous. e.g. "foo.0.bar" and + * "foo.2.bar" without any "foo.1.bar" present. + * - If keys in @src represent list indexes, but are not in + * the "%zu" format. e.g. "foo.+0.bar" + * + * Returns: either a QDict or QList for the nested data structure, or NULL + * on error + */ +QObject *qdict_crumple(const QDict *src, Error **errp) +{ + const QDictEntry *ent; + QDict *two_level, *multi_level = NULL; + QObject *dst = NULL, *child; + size_t i; + char *prefix = NULL; + const char *suffix = NULL; + int is_list; + + two_level = qdict_new(); + + /* Step 1: split our totally flat dict into a two level dict */ + for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) { + if (qobject_type(ent->value) == QTYPE_QDICT || + qobject_type(ent->value) == QTYPE_QLIST) { + error_setg(errp, "Value %s is not a scalar", + ent->key); + goto error; + } + + qdict_split_flat_key(ent->key, &prefix, &suffix); + + child = qdict_get(two_level, prefix); + if (suffix) { + QDict *child_dict = qobject_to(QDict, child); + if (!child_dict) { + if (child) { + error_setg(errp, "Key %s prefix is already set as a scalar", + prefix); + goto error; + } + + child_dict = qdict_new(); + qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); + } + + qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); + } else { + if (child) { + error_setg(errp, "Key %s prefix is already set as a dict", + prefix); + goto error; + } + qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); + } + + g_free(prefix); + prefix = NULL; + } + + /* Step 2: optionally process the two level dict recursively + * into a multi-level dict */ + multi_level = qdict_new(); + for (ent = qdict_first(two_level); ent != NULL; + ent = qdict_next(two_level, ent)) { + QDict *dict = qobject_to(QDict, ent->value); + if (dict) { + child = qdict_crumple(dict, errp); + if (!child) { + goto error; + } + + qdict_put_obj(multi_level, ent->key, child); + } else { + qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value)); + } + } + qobject_unref(two_level); + two_level = NULL; + + /* Step 3: detect if we need to turn our dict into list */ + is_list = qdict_is_list(multi_level, errp); + if (is_list < 0) { + goto error; + } + + if (is_list) { + dst = QOBJECT(qlist_new()); + + for (i = 0; i < qdict_size(multi_level); i++) { + char *key = g_strdup_printf("%zu", i); + + child = qdict_get(multi_level, key); + g_free(key); + + if (!child) { + error_setg(errp, "Missing list index %zu", i); + goto error; + } + + qlist_append_obj(qobject_to(QList, dst), qobject_ref(child)); + } + qobject_unref(multi_level); + multi_level = NULL; + } else { + dst = QOBJECT(multi_level); + } + + return dst; + + error: + g_free(prefix); + qobject_unref(multi_level); + qobject_unref(two_level); + qobject_unref(dst); + return NULL; +} + +/** + * qdict_array_entries(): Returns the number of direct array entries if the + * sub-QDict of src specified by the prefix in subqdict (or src itself for + * prefix == "") is valid as an array, i.e. the length of the created list if + * the sub-QDict would become empty after calling qdict_array_split() on it. If + * the array is not valid, -EINVAL is returned. + */ +int qdict_array_entries(QDict *src, const char *subqdict) +{ + const QDictEntry *entry; + unsigned i; + unsigned entries = 0; + size_t subqdict_len = strlen(subqdict); + + assert(!subqdict_len || subqdict[subqdict_len - 1] == '.'); + + /* qdict_array_split() loops until UINT_MAX, but as we want to return + * negative errors, we only have a signed return value here. Any additional + * entries will lead to -EINVAL. */ + for (i = 0; i < INT_MAX; i++) { + QObject *subqobj; + int subqdict_entries; + char *prefix = g_strdup_printf("%s%u.", subqdict, i); + + subqdict_entries = qdict_count_prefixed_entries(src, prefix); + + /* Remove ending "." */ + prefix[strlen(prefix) - 1] = 0; + subqobj = qdict_get(src, prefix); + + g_free(prefix); + + if (subqdict_entries < 0) { + return subqdict_entries; + } + + /* There may be either a single subordinate object (named "%u") or + * multiple objects (each with a key prefixed "%u."), but not both. */ + if (subqobj && subqdict_entries) { + return -EINVAL; + } else if (!subqobj && !subqdict_entries) { + break; + } + + entries += subqdict_entries ? subqdict_entries : 1; + } + + /* Consider everything handled that isn't part of the given sub-QDict */ + for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { + if (!strstart(qdict_entry_key(entry), subqdict, NULL)) { + entries++; + } + } + + /* Anything left in the sub-QDict that wasn't handled? */ + if (qdict_size(src) != entries) { + return -EINVAL; + } + + return i; +} + +/** + * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all + * elements from src to dest. + * + * If an element from src has a key already present in dest, it will not be + * moved unless overwrite is true. + * + * If overwrite is true, the conflicting values in dest will be discarded and + * replaced by the corresponding values from src. + * + * Therefore, with overwrite being true, the src QDict will always be empty when + * this function returns. If overwrite is false, the src QDict will be empty + * iff there were no conflicts. + */ +void qdict_join(QDict *dest, QDict *src, bool overwrite) +{ + const QDictEntry *entry, *next; + + entry = qdict_first(src); + while (entry) { + next = qdict_next(src, entry); + + if (overwrite || !qdict_haskey(dest, entry->key)) { + qdict_put_obj(dest, entry->key, qobject_ref(entry->value)); + qdict_del(src, entry->key); + } + + entry = next; + } +} + +/** + * qdict_rename_keys(): Rename keys in qdict according to the replacements + * specified in the array renames. The array must be terminated by an entry + * with from = NULL. + * + * The renames are performed individually in the order of the array, so entries + * may be renamed multiple times and may or may not conflict depending on the + * order of the renames array. + * + * Returns true for success, false in error cases. + */ +bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) +{ + QObject *qobj; + + while (renames->from) { + if (qdict_haskey(qdict, renames->from)) { + if (qdict_haskey(qdict, renames->to)) { + error_setg(errp, "'%s' and its alias '%s' can't be used at the " + "same time", renames->to, renames->from); + return false; + } + + qobj = qdict_get(qdict, renames->from); + qdict_put_obj(qdict, renames->to, qobject_ref(qobj)); + qdict_del(qdict, renames->from); + } + + renames++; + } + return true; +} diff --git a/qobject/qdict.c b/qobject/qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/qdict.c +++ b/qobject/qdict.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" -#include "block/qdict.h" #include "qapi/qmp/qnum.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qbool.h" -#include "qapi/qmp/qlist.h" #include "qapi/qmp/qnull.h" #include "qapi/qmp/qstring.h" -#include "qapi/error.h" -#include "qemu/queue.h" -#include "qemu-common.h" -#include "qemu/cutils.h" /** * qdict_new(): Create a new QDict @@ -XXX,XX +XXX,XX @@ void qdict_destroy_obj(QObject *obj) g_free(qdict); } - -/** - * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the - * value of 'key' in 'src' is copied there (and the refcount increased - * accordingly). - */ -void qdict_copy_default(QDict *dst, QDict *src, const char *key) -{ - QObject *val; - - if (qdict_haskey(dst, key)) { - return; - } - - val = qdict_get(src, key); - if (val) { - qdict_put_obj(dst, key, qobject_ref(val)); - } -} - -/** - * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a - * new QString initialised by 'val' is put there. - */ -void qdict_set_default_str(QDict *dst, const char *key, const char *val) -{ - if (qdict_haskey(dst, key)) { - return; - } - - qdict_put_str(dst, key, val); -} - -static void qdict_flatten_qdict(QDict *qdict, QDict *target, - const char *prefix); - -static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) -{ - QObject *value; - const QListEntry *entry; - char *new_key; - int i; - - /* This function is never called with prefix == NULL, i.e., it is always - * called from within qdict_flatten_q(list|dict)(). Therefore, it does not - * need to remove list entries during the iteration (the whole list will be - * deleted eventually anyway from qdict_flatten_qdict()). */ - assert(prefix); - - entry = qlist_first(qlist); - - for (i = 0; entry; entry = qlist_next(entry), i++) { - value = qlist_entry_obj(entry); - new_key = g_strdup_printf("%s.%i", prefix, i); - - if (qobject_type(value) == QTYPE_QDICT) { - qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); - } else if (qobject_type(value) == QTYPE_QLIST) { - qdict_flatten_qlist(qobject_to(QList, value), target, new_key); - } else { - /* All other types are moved to the target unchanged. */ - qdict_put_obj(target, new_key, qobject_ref(value)); - } - - g_free(new_key); - } -} - -static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) -{ - QObject *value; - const QDictEntry *entry, *next; - char *new_key; - bool delete; - - entry = qdict_first(qdict); - - while (entry != NULL) { - - next = qdict_next(qdict, entry); - value = qdict_entry_value(entry); - new_key = NULL; - delete = false; - - if (prefix) { - new_key = g_strdup_printf("%s.%s", prefix, entry->key); - } - - if (qobject_type(value) == QTYPE_QDICT) { - /* Entries of QDicts are processed recursively, the QDict object - * itself disappears. */ - qdict_flatten_qdict(qobject_to(QDict, value), target, - new_key ? new_key : entry->key); - delete = true; - } else if (qobject_type(value) == QTYPE_QLIST) { - qdict_flatten_qlist(qobject_to(QList, value), target, - new_key ? new_key : entry->key); - delete = true; - } else if (prefix) { - /* All other objects are moved to the target unchanged. */ - qdict_put_obj(target, new_key, qobject_ref(value)); - delete = true; - } - - g_free(new_key); - - if (delete) { - qdict_del(qdict, entry->key); - - /* Restart loop after modifying the iterated QDict */ - entry = qdict_first(qdict); - continue; - } - - entry = next; - } -} - -/** - * qdict_flatten(): For each nested QDict with key x, all fields with key y - * are moved to this QDict and their key is renamed to "x.y". For each nested - * QList with key x, the field at index y is moved to this QDict with the key - * "x.y" (i.e., the reverse of what qdict_array_split() does). - * This operation is applied recursively for nested QDicts and QLists. - */ -void qdict_flatten(QDict *qdict) -{ - qdict_flatten_qdict(qdict, qdict, NULL); -} - -/* extract all the src QDict entries starting by start into dst */ -void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start) - -{ - const QDictEntry *entry, *next; - const char *p; - - *dst = qdict_new(); - entry = qdict_first(src); - - while (entry != NULL) { - next = qdict_next(src, entry); - if (strstart(entry->key, start, &p)) { - qdict_put_obj(*dst, p, qobject_ref(entry->value)); - qdict_del(src, entry->key); - } - entry = next; - } -} - -static int qdict_count_prefixed_entries(const QDict *src, const char *start) -{ - const QDictEntry *entry; - int count = 0; - - for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { - if (strstart(entry->key, start, NULL)) { - if (count == INT_MAX) { - return -ERANGE; - } - count++; - } - } - - return count; -} - -/** - * qdict_array_split(): This function moves array-like elements of a QDict into - * a new QList. Every entry in the original QDict with a key "%u" or one - * prefixed "%u.", where %u designates an unsigned integer starting at 0 and - * incrementally counting up, will be moved to a new QDict at index %u in the - * output QList with the key prefix removed, if that prefix is "%u.". If the - * whole key is just "%u", the whole QObject will be moved unchanged without - * creating a new QDict. The function terminates when there is no entry in the - * QDict with a prefix directly (incrementally) following the last one; it also - * returns if there are both entries with "%u" and "%u." for the same index %u. - * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66} - * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66}) - * => [{"a": 42, "b": 23}, {"x": 0}, 66] - * and {"4.y": 1, "o.o": 7} (remainder of the old QDict) - */ -void qdict_array_split(QDict *src, QList **dst) -{ - unsigned i; - - *dst = qlist_new(); - - for (i = 0; i < UINT_MAX; i++) { - QObject *subqobj; - bool is_subqdict; - QDict *subqdict; - char indexstr[32], prefix[32]; - size_t snprintf_ret; - - snprintf_ret = snprintf(indexstr, 32, "%u", i); - assert(snprintf_ret < 32); - - subqobj = qdict_get(src, indexstr); - - snprintf_ret = snprintf(prefix, 32, "%u.", i); - assert(snprintf_ret < 32); - - /* Overflow is the same as positive non-zero results */ - is_subqdict = qdict_count_prefixed_entries(src, prefix); - - // There may be either a single subordinate object (named "%u") or - // multiple objects (each with a key prefixed "%u."), but not both. - if (!subqobj == !is_subqdict) { - break; - } - - if (is_subqdict) { - qdict_extract_subqdict(src, &subqdict, prefix); - assert(qdict_size(subqdict) > 0); - } else { - qobject_ref(subqobj); - qdict_del(src, indexstr); - } - - qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict)); - } -} - -/** - * qdict_split_flat_key: - * @key: the key string to split - * @prefix: non-NULL pointer to hold extracted prefix - * @suffix: non-NULL pointer to remaining suffix - * - * Given a flattened key such as 'foo.0.bar', split it into two parts - * at the first '.' separator. Allows double dot ('..') to escape the - * normal separator. - * - * e.g. - * 'foo.0.bar' -> prefix='foo' and suffix='0.bar' - * 'foo..0.bar' -> prefix='foo.0' and suffix='bar' - * - * The '..' sequence will be unescaped in the returned 'prefix' - * string. The 'suffix' string will be left in escaped format, so it - * can be fed back into the qdict_split_flat_key() key as the input - * later. - * - * The caller is responsible for freeing the string returned in @prefix - * using g_free(). - */ -static void qdict_split_flat_key(const char *key, char **prefix, - const char **suffix) -{ - const char *separator; - size_t i, j; - - /* Find first '.' separator, but if there is a pair '..' - * that acts as an escape, so skip over '..' */ - separator = NULL; - do { - if (separator) { - separator += 2; - } else { - separator = key; - } - separator = strchr(separator, '.'); - } while (separator && separator[1] == '.'); - - if (separator) { - *prefix = g_strndup(key, separator - key); - *suffix = separator + 1; - } else { - *prefix = g_strdup(key); - *suffix = NULL; - } - - /* Unescape the '..' sequence into '.' */ - for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) { - if ((*prefix)[i] == '.') { - assert((*prefix)[i + 1] == '.'); - i++; - } - (*prefix)[j] = (*prefix)[i]; - } - (*prefix)[j] = '\0'; -} - -/** - * qdict_is_list: - * @maybe_list: dict to check if keys represent list elements. - * - * Determine whether all keys in @maybe_list are valid list elements. - * If @maybe_list is non-zero in length and all the keys look like - * valid list indexes, this will return 1. If @maybe_list is zero - * length or all keys are non-numeric then it will return 0 to indicate - * it is a normal qdict. If there is a mix of numeric and non-numeric - * keys, or the list indexes are non-contiguous, an error is reported. - * - * Returns: 1 if a valid list, 0 if a dict, -1 on error - */ -static int qdict_is_list(QDict *maybe_list, Error **errp) -{ - const QDictEntry *ent; - ssize_t len = 0; - ssize_t max = -1; - int is_list = -1; - int64_t val; - - for (ent = qdict_first(maybe_list); ent != NULL; - ent = qdict_next(maybe_list, ent)) { - - if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) { - if (is_list == -1) { - is_list = 1; - } else if (!is_list) { - error_setg(errp, - "Cannot mix list and non-list keys"); - return -1; - } - len++; - if (val > max) { - max = val; - } - } else { - if (is_list == -1) { - is_list = 0; - } else if (is_list) { - error_setg(errp, - "Cannot mix list and non-list keys"); - return -1; - } - } - } - - if (is_list == -1) { - assert(!qdict_size(maybe_list)); - is_list = 0; - } - - /* NB this isn't a perfect check - e.g. it won't catch - * a list containing '1', '+1', '01', '3', but that - * does not matter - we've still proved that the - * input is a list. It is up the caller to do a - * stricter check if desired */ - if (len != (max + 1)) { - error_setg(errp, "List indices are not contiguous, " - "saw %zd elements but %zd largest index", - len, max); - return -1; - } - - return is_list; -} - -/** - * qdict_crumple: - * @src: the original flat dictionary (only scalar values) to crumple - * - * Takes a flat dictionary whose keys use '.' separator to indicate - * nesting, and values are scalars, and crumples it into a nested - * structure. - * - * To include a literal '.' in a key name, it must be escaped as '..' - * - * For example, an input of: - * - * { 'foo.0.bar': 'one', 'foo.0.wizz': '1', - * 'foo.1.bar': 'two', 'foo.1.wizz': '2' } - * - * will result in an output of: - * - * { - * 'foo': [ - * { 'bar': 'one', 'wizz': '1' }, - * { 'bar': 'two', 'wizz': '2' } - * ], - * } - * - * The following scenarios in the input dict will result in an - * error being returned: - * - * - Any values in @src are non-scalar types - * - If keys in @src imply that a particular level is both a - * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar". - * - If keys in @src imply that a particular level is a list, - * but the indices are non-contiguous. e.g. "foo.0.bar" and - * "foo.2.bar" without any "foo.1.bar" present. - * - If keys in @src represent list indexes, but are not in - * the "%zu" format. e.g. "foo.+0.bar" - * - * Returns: either a QDict or QList for the nested data structure, or NULL - * on error - */ -QObject *qdict_crumple(const QDict *src, Error **errp) -{ - const QDictEntry *ent; - QDict *two_level, *multi_level = NULL; - QObject *dst = NULL, *child; - size_t i; - char *prefix = NULL; - const char *suffix = NULL; - int is_list; - - two_level = qdict_new(); - - /* Step 1: split our totally flat dict into a two level dict */ - for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) { - if (qobject_type(ent->value) == QTYPE_QDICT || - qobject_type(ent->value) == QTYPE_QLIST) { - error_setg(errp, "Value %s is not a scalar", - ent->key); - goto error; - } - - qdict_split_flat_key(ent->key, &prefix, &suffix); - - child = qdict_get(two_level, prefix); - if (suffix) { - QDict *child_dict = qobject_to(QDict, child); - if (!child_dict) { - if (child) { - error_setg(errp, "Key %s prefix is already set as a scalar", - prefix); - goto error; - } - - child_dict = qdict_new(); - qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); - } - - qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); - } else { - if (child) { - error_setg(errp, "Key %s prefix is already set as a dict", - prefix); - goto error; - } - qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); - } - - g_free(prefix); - prefix = NULL; - } - - /* Step 2: optionally process the two level dict recursively - * into a multi-level dict */ - multi_level = qdict_new(); - for (ent = qdict_first(two_level); ent != NULL; - ent = qdict_next(two_level, ent)) { - QDict *dict = qobject_to(QDict, ent->value); - if (dict) { - child = qdict_crumple(dict, errp); - if (!child) { - goto error; - } - - qdict_put_obj(multi_level, ent->key, child); - } else { - qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value)); - } - } - qobject_unref(two_level); - two_level = NULL; - - /* Step 3: detect if we need to turn our dict into list */ - is_list = qdict_is_list(multi_level, errp); - if (is_list < 0) { - goto error; - } - - if (is_list) { - dst = QOBJECT(qlist_new()); - - for (i = 0; i < qdict_size(multi_level); i++) { - char *key = g_strdup_printf("%zu", i); - - child = qdict_get(multi_level, key); - g_free(key); - - if (!child) { - error_setg(errp, "Missing list index %zu", i); - goto error; - } - - qlist_append_obj(qobject_to(QList, dst), qobject_ref(child)); - } - qobject_unref(multi_level); - multi_level = NULL; - } else { - dst = QOBJECT(multi_level); - } - - return dst; - - error: - g_free(prefix); - qobject_unref(multi_level); - qobject_unref(two_level); - qobject_unref(dst); - return NULL; -} - -/** - * qdict_array_entries(): Returns the number of direct array entries if the - * sub-QDict of src specified by the prefix in subqdict (or src itself for - * prefix == "") is valid as an array, i.e. the length of the created list if - * the sub-QDict would become empty after calling qdict_array_split() on it. If - * the array is not valid, -EINVAL is returned. - */ -int qdict_array_entries(QDict *src, const char *subqdict) -{ - const QDictEntry *entry; - unsigned i; - unsigned entries = 0; - size_t subqdict_len = strlen(subqdict); - - assert(!subqdict_len || subqdict[subqdict_len - 1] == '.'); - - /* qdict_array_split() loops until UINT_MAX, but as we want to return - * negative errors, we only have a signed return value here. Any additional - * entries will lead to -EINVAL. */ - for (i = 0; i < INT_MAX; i++) { - QObject *subqobj; - int subqdict_entries; - char *prefix = g_strdup_printf("%s%u.", subqdict, i); - - subqdict_entries = qdict_count_prefixed_entries(src, prefix); - - /* Remove ending "." */ - prefix[strlen(prefix) - 1] = 0; - subqobj = qdict_get(src, prefix); - - g_free(prefix); - - if (subqdict_entries < 0) { - return subqdict_entries; - } - - /* There may be either a single subordinate object (named "%u") or - * multiple objects (each with a key prefixed "%u."), but not both. */ - if (subqobj && subqdict_entries) { - return -EINVAL; - } else if (!subqobj && !subqdict_entries) { - break; - } - - entries += subqdict_entries ? subqdict_entries : 1; - } - - /* Consider everything handled that isn't part of the given sub-QDict */ - for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { - if (!strstart(qdict_entry_key(entry), subqdict, NULL)) { - entries++; - } - } - - /* Anything left in the sub-QDict that wasn't handled? */ - if (qdict_size(src) != entries) { - return -EINVAL; - } - - return i; -} - -/** - * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all - * elements from src to dest. - * - * If an element from src has a key already present in dest, it will not be - * moved unless overwrite is true. - * - * If overwrite is true, the conflicting values in dest will be discarded and - * replaced by the corresponding values from src. - * - * Therefore, with overwrite being true, the src QDict will always be empty when - * this function returns. If overwrite is false, the src QDict will be empty - * iff there were no conflicts. - */ -void qdict_join(QDict *dest, QDict *src, bool overwrite) -{ - const QDictEntry *entry, *next; - - entry = qdict_first(src); - while (entry) { - next = qdict_next(src, entry); - - if (overwrite || !qdict_haskey(dest, entry->key)) { - qdict_put_obj(dest, entry->key, qobject_ref(entry->value)); - qdict_del(src, entry->key); - } - - entry = next; - } -} - -/** - * qdict_rename_keys(): Rename keys in qdict according to the replacements - * specified in the array renames. The array must be terminated by an entry - * with from = NULL. - * - * The renames are performed individually in the order of the array, so entries - * may be renamed multiple times and may or may not conflict depending on the - * order of the renames array. - * - * Returns true for success, false in error cases. - */ -bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) -{ - QObject *qobj; - - while (renames->from) { - if (qdict_haskey(qdict, renames->from)) { - if (qdict_haskey(qdict, renames->to)) { - error_setg(errp, "'%s' and its alias '%s' can't be used at the " - "same time", renames->to, renames->from); - return false; - } - - qobj = qdict_get(qdict, renames->from); - qdict_put_obj(qdict, renames->to, qobject_ref(qobj)); - qdict_del(qdict, renames->from); - } - - renames++; - } - return true; -} diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/check-block-qdict.c @@ -XXX,XX +XXX,XX @@ +/* + * Unit-tests for Block layer QDict extras + * + * Copyright (c) 2013-2018 Red Hat, Inc. + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "block/qdict.h" +#include "qapi/qmp/qlist.h" +#include "qapi/qmp/qnum.h" +#include "qapi/error.h" + +static void qdict_defaults_test(void) +{ + QDict *dict, *copy; + + dict = qdict_new(); + copy = qdict_new(); + + qdict_set_default_str(dict, "foo", "abc"); + qdict_set_default_str(dict, "foo", "def"); + g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc"); + qdict_set_default_str(dict, "bar", "ghi"); + + qdict_copy_default(copy, dict, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc"); + qdict_set_default_str(copy, "bar", "xyz"); + qdict_copy_default(copy, dict, "bar"); + g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz"); + + qobject_unref(copy); + qobject_unref(dict); +} + +static void qdict_flatten_test(void) +{ + QList *list1 = qlist_new(); + QList *list2 = qlist_new(); + QDict *dict1 = qdict_new(); + QDict *dict2 = qdict_new(); + QDict *dict3 = qdict_new(); + + /* + * Test the flattening of + * + * { + * "e": [ + * 42, + * [ + * 23, + * 66, + * { + * "a": 0, + * "b": 1 + * } + * ] + * ], + * "f": { + * "c": 2, + * "d": 3, + * }, + * "g": 4 + * } + * + * to + * + * { + * "e.0": 42, + * "e.1.0": 23, + * "e.1.1": 66, + * "e.1.2.a": 0, + * "e.1.2.b": 1, + * "f.c": 2, + * "f.d": 3, + * "g": 4 + * } + */ + + qdict_put_int(dict1, "a", 0); + qdict_put_int(dict1, "b", 1); + + qlist_append_int(list1, 23); + qlist_append_int(list1, 66); + qlist_append(list1, dict1); + qlist_append_int(list2, 42); + qlist_append(list2, list1); + + qdict_put_int(dict2, "c", 2); + qdict_put_int(dict2, "d", 3); + qdict_put(dict3, "e", list2); + qdict_put(dict3, "f", dict2); + qdict_put_int(dict3, "g", 4); + + qdict_flatten(dict3); + + g_assert(qdict_get_int(dict3, "e.0") == 42); + g_assert(qdict_get_int(dict3, "e.1.0") == 23); + g_assert(qdict_get_int(dict3, "e.1.1") == 66); + g_assert(qdict_get_int(dict3, "e.1.2.a") == 0); + g_assert(qdict_get_int(dict3, "e.1.2.b") == 1); + g_assert(qdict_get_int(dict3, "f.c") == 2); + g_assert(qdict_get_int(dict3, "f.d") == 3); + g_assert(qdict_get_int(dict3, "g") == 4); + + g_assert(qdict_size(dict3) == 8); + + qobject_unref(dict3); +} + +static void qdict_array_split_test(void) +{ + QDict *test_dict = qdict_new(); + QDict *dict1, *dict2; + QNum *int1; + QList *test_list; + + /* + * Test the split of + * + * { + * "1.x": 0, + * "4.y": 1, + * "0.a": 42, + * "o.o": 7, + * "0.b": 23, + * "2": 66 + * } + * + * to + * + * [ + * { + * "a": 42, + * "b": 23 + * }, + * { + * "x": 0 + * }, + * 66 + * ] + * + * and + * + * { + * "4.y": 1, + * "o.o": 7 + * } + * + * (remaining in the old QDict) + * + * This example is given in the comment of qdict_array_split(). + */ + + qdict_put_int(test_dict, "1.x", 0); + qdict_put_int(test_dict, "4.y", 1); + qdict_put_int(test_dict, "0.a", 42); + qdict_put_int(test_dict, "o.o", 7); + qdict_put_int(test_dict, "0.b", 23); + qdict_put_int(test_dict, "2", 66); + + qdict_array_split(test_dict, &test_list); + + dict1 = qobject_to(QDict, qlist_pop(test_list)); + dict2 = qobject_to(QDict, qlist_pop(test_list)); + int1 = qobject_to(QNum, qlist_pop(test_list)); + + g_assert(dict1); + g_assert(dict2); + g_assert(int1); + g_assert(qlist_empty(test_list)); + + qobject_unref(test_list); + + g_assert(qdict_get_int(dict1, "a") == 42); + g_assert(qdict_get_int(dict1, "b") == 23); + + g_assert(qdict_size(dict1) == 2); + + qobject_unref(dict1); + + g_assert(qdict_get_int(dict2, "x") == 0); + + g_assert(qdict_size(dict2) == 1); + + qobject_unref(dict2); + + g_assert_cmpint(qnum_get_int(int1), ==, 66); + + qobject_unref(int1); + + g_assert(qdict_get_int(test_dict, "4.y") == 1); + g_assert(qdict_get_int(test_dict, "o.o") == 7); + + g_assert(qdict_size(test_dict) == 2); + + qobject_unref(test_dict); + + /* + * Test the split of + * + * { + * "0": 42, + * "1": 23, + * "1.x": 84 + * } + * + * to + * + * [ + * 42 + * ] + * + * and + * + * { + * "1": 23, + * "1.x": 84 + * } + * + * That is, test whether splitting stops if there is both an entry with key + * of "%u" and other entries with keys prefixed "%u." for the same index. + */ + + test_dict = qdict_new(); + + qdict_put_int(test_dict, "0", 42); + qdict_put_int(test_dict, "1", 23); + qdict_put_int(test_dict, "1.x", 84); + + qdict_array_split(test_dict, &test_list); + + int1 = qobject_to(QNum, qlist_pop(test_list)); + + g_assert(int1); + g_assert(qlist_empty(test_list)); + + qobject_unref(test_list); + + g_assert_cmpint(qnum_get_int(int1), ==, 42); + + qobject_unref(int1); + + g_assert(qdict_get_int(test_dict, "1") == 23); + g_assert(qdict_get_int(test_dict, "1.x") == 84); + + g_assert(qdict_size(test_dict) == 2); + + qobject_unref(test_dict); +} + +static void qdict_array_entries_test(void) +{ + QDict *dict = qdict_new(); + + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); + + qdict_put_int(dict, "bar", 0); + qdict_put_int(dict, "baz.0", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); + + qdict_put_int(dict, "foo.1", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); + qdict_put_int(dict, "foo.0", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2); + qdict_put_int(dict, "foo.bar", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); + qdict_del(dict, "foo.bar"); + + qdict_put_int(dict, "foo.2.a", 0); + qdict_put_int(dict, "foo.2.b", 0); + qdict_put_int(dict, "foo.2.c", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); + + qobject_unref(dict); + + dict = qdict_new(); + qdict_put_int(dict, "1", 0); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); + qdict_put_int(dict, "0", 0); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2); + qdict_put_int(dict, "bar", 0); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); + qdict_del(dict, "bar"); + + qdict_put_int(dict, "2.a", 0); + qdict_put_int(dict, "2.b", 0); + qdict_put_int(dict, "2.c", 0); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3); + + qobject_unref(dict); +} + +static void qdict_join_test(void) +{ + QDict *dict1, *dict2; + bool overwrite = false; + int i; + + dict1 = qdict_new(); + dict2 = qdict_new(); + + /* Test everything once without overwrite and once with */ + do { + /* Test empty dicts */ + qdict_join(dict1, dict2, overwrite); + + g_assert(qdict_size(dict1) == 0); + g_assert(qdict_size(dict2) == 0); + + /* First iteration: Test movement */ + /* Second iteration: Test empty source and non-empty destination */ + qdict_put_int(dict2, "foo", 42); + + for (i = 0; i < 2; i++) { + qdict_join(dict1, dict2, overwrite); + + g_assert(qdict_size(dict1) == 1); + g_assert(qdict_size(dict2) == 0); + + g_assert(qdict_get_int(dict1, "foo") == 42); + } + + /* Test non-empty source and destination without conflict */ + qdict_put_int(dict2, "bar", 23); + + qdict_join(dict1, dict2, overwrite); + + g_assert(qdict_size(dict1) == 2); + g_assert(qdict_size(dict2) == 0); + + g_assert(qdict_get_int(dict1, "foo") == 42); + g_assert(qdict_get_int(dict1, "bar") == 23); + + /* Test conflict */ + qdict_put_int(dict2, "foo", 84); + + qdict_join(dict1, dict2, overwrite); + + g_assert(qdict_size(dict1) == 2); + g_assert(qdict_size(dict2) == !overwrite); + + g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42)); + g_assert(qdict_get_int(dict1, "bar") == 23); + + if (!overwrite) { + g_assert(qdict_get_int(dict2, "foo") == 84); + } + + /* Check the references */ + g_assert(qdict_get(dict1, "foo")->base.refcnt == 1); + g_assert(qdict_get(dict1, "bar")->base.refcnt == 1); + + if (!overwrite) { + g_assert(qdict_get(dict2, "foo")->base.refcnt == 1); + } + + /* Clean up */ + qdict_del(dict1, "foo"); + qdict_del(dict1, "bar"); + + if (!overwrite) { + qdict_del(dict2, "foo"); + } + } while (overwrite ^= true); + + qobject_unref(dict1); + qobject_unref(dict2); +} + +static void qdict_crumple_test_recursive(void) +{ + QDict *src, *dst, *rule, *vnc, *acl, *listen; + QList *rules; + + src = qdict_new(); + qdict_put_str(src, "vnc.listen.addr", "127.0.0.1"); + qdict_put_str(src, "vnc.listen.port", "5901"); + qdict_put_str(src, "vnc.acl.rules.0.match", "fred"); + qdict_put_str(src, "vnc.acl.rules.0.policy", "allow"); + qdict_put_str(src, "vnc.acl.rules.1.match", "bob"); + qdict_put_str(src, "vnc.acl.rules.1.policy", "deny"); + qdict_put_str(src, "vnc.acl.default", "deny"); + qdict_put_str(src, "vnc.acl..name", "acl0"); + qdict_put_str(src, "vnc.acl.rule..name", "acl0"); + + dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); + g_assert(dst); + g_assert_cmpint(qdict_size(dst), ==, 1); + + vnc = qdict_get_qdict(dst, "vnc"); + g_assert(vnc); + g_assert_cmpint(qdict_size(vnc), ==, 3); + + listen = qdict_get_qdict(vnc, "listen"); + g_assert(listen); + g_assert_cmpint(qdict_size(listen), ==, 2); + g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr")); + g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port")); + + acl = qdict_get_qdict(vnc, "acl"); + g_assert(acl); + g_assert_cmpint(qdict_size(acl), ==, 3); + + rules = qdict_get_qlist(acl, "rules"); + g_assert(rules); + g_assert_cmpint(qlist_size(rules), ==, 2); + + rule = qobject_to(QDict, qlist_pop(rules)); + g_assert(rule); + g_assert_cmpint(qdict_size(rule), ==, 2); + g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match")); + g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy")); + qobject_unref(rule); + + rule = qobject_to(QDict, qlist_pop(rules)); + g_assert(rule); + g_assert_cmpint(qdict_size(rule), ==, 2); + g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match")); + g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy")); + qobject_unref(rule); + + /* With recursive crumpling, we should see all names unescaped */ + g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); + g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); + + qobject_unref(src); + qobject_unref(dst); +} + +static void qdict_crumple_test_empty(void) +{ + QDict *src, *dst; + + src = qdict_new(); + + dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); + + g_assert_cmpint(qdict_size(dst), ==, 0); + + qobject_unref(src); + qobject_unref(dst); +} + +static int qdict_count_entries(QDict *dict) +{ + const QDictEntry *e; + int count = 0; + + for (e = qdict_first(dict); e; e = qdict_next(dict, e)) { + count++; + } + + return count; +} + +static void qdict_rename_keys_test(void) +{ + QDict *dict = qdict_new(); + QDict *copy; + QDictRenames *renames; + Error *local_err = NULL; + + qdict_put_str(dict, "abc", "foo"); + qdict_put_str(dict, "abcdef", "bar"); + qdict_put_int(dict, "number", 42); + qdict_put_bool(dict, "flag", true); + qdict_put_null(dict, "nothing"); + + /* Empty rename list */ + renames = (QDictRenames[]) { + { NULL, "this can be anything" } + }; + copy = qdict_clone_shallow(dict); + qdict_rename_keys(copy, renames, &error_abort); + + g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); + g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); + g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); + g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + + qobject_unref(copy); + + /* Simple rename of all entries */ + renames = (QDictRenames[]) { + { "abc", "str1" }, + { "abcdef", "str2" }, + { "number", "int" }, + { "flag", "bool" }, + { "nothing", "null" }, + { NULL , NULL } + }; + copy = qdict_clone_shallow(dict); + qdict_rename_keys(copy, renames, &error_abort); + + g_assert(!qdict_haskey(copy, "abc")); + g_assert(!qdict_haskey(copy, "abcdef")); + g_assert(!qdict_haskey(copy, "number")); + g_assert(!qdict_haskey(copy, "flag")); + g_assert(!qdict_haskey(copy, "nothing")); + + g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar"); + g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42); + g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true); + g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + + qobject_unref(copy); + + /* Renames are processed top to bottom */ + renames = (QDictRenames[]) { + { "abc", "tmp" }, + { "abcdef", "abc" }, + { "number", "abcdef" }, + { "flag", "number" }, + { "nothing", "flag" }, + { "tmp", "nothing" }, + { NULL , NULL } + }; + copy = qdict_clone_shallow(dict); + qdict_rename_keys(copy, renames, &error_abort); + + g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar"); + g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42); + g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true); + g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL); + g_assert(!qdict_haskey(copy, "tmp")); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + + qobject_unref(copy); + + /* Conflicting rename */ + renames = (QDictRenames[]) { + { "abcdef", "abc" }, + { NULL , NULL } + }; + copy = qdict_clone_shallow(dict); + qdict_rename_keys(copy, renames, &local_err); + + g_assert(local_err != NULL); + error_free(local_err); + local_err = NULL; + + g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); + g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); + g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); + g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + + qobject_unref(copy); + + /* Renames in an empty dict */ + renames = (QDictRenames[]) { + { "abcdef", "abc" }, + { NULL , NULL } + }; + + qobject_unref(dict); + dict = qdict_new(); + + qdict_rename_keys(dict, renames, &error_abort); + g_assert(qdict_first(dict) == NULL); + + qobject_unref(dict); +} + +static void qdict_crumple_test_bad_inputs(void) +{ + QDict *src; + Error *error = NULL; + + src = qdict_new(); + /* rule.0 can't be both a string and a dict */ + qdict_put_str(src, "rule.0", "fred"); + qdict_put_str(src, "rule.0.policy", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); + + src = qdict_new(); + /* rule can't be both a list and a dict */ + qdict_put_str(src, "rule.0", "fred"); + qdict_put_str(src, "rule.a", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); + + src = qdict_new(); + /* The input should be flat, ie no dicts or lists */ + qdict_put(src, "rule.a", qdict_new()); + qdict_put_str(src, "rule.b", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); + + src = qdict_new(); + /* List indexes must not have gaps */ + qdict_put_str(src, "rule.0", "deny"); + qdict_put_str(src, "rule.3", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); + + src = qdict_new(); + /* List indexes must be in %zu format */ + qdict_put_str(src, "rule.0", "deny"); + qdict_put_str(src, "rule.+1", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); +} + +int main(int argc, char **argv) +{ + g_test_init(&argc, &argv, NULL); + + g_test_add_func("/public/defaults", qdict_defaults_test); + g_test_add_func("/public/flatten", qdict_flatten_test); + g_test_add_func("/public/array_split", qdict_array_split_test); + g_test_add_func("/public/array_entries", qdict_array_entries_test); + g_test_add_func("/public/join", qdict_join_test); + g_test_add_func("/public/crumple/recursive", + qdict_crumple_test_recursive); + g_test_add_func("/public/crumple/empty", + qdict_crumple_test_empty); + g_test_add_func("/public/crumple/bad_inputs", + qdict_crumple_test_bad_inputs); + + g_test_add_func("/public/rename_keys", qdict_rename_keys_test); + + return g_test_run(); +} diff --git a/tests/check-qdict.c b/tests/check-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-qdict.c +++ b/tests/check-qdict.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" -#include "block/qdict.h" #include "qapi/qmp/qdict.h" -#include "qapi/qmp/qlist.h" -#include "qapi/qmp/qnum.h" -#include "qapi/qmp/qstring.h" -#include "qapi/error.h" -#include "qemu-common.h" /* * Public Interface test-cases @@ -XXX,XX +XXX,XX @@ static void qdict_get_try_str_test(void) qobject_unref(tests_dict); } -static void qdict_defaults_test(void) -{ - QDict *dict, *copy; - - dict = qdict_new(); - copy = qdict_new(); - - qdict_set_default_str(dict, "foo", "abc"); - qdict_set_default_str(dict, "foo", "def"); - g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc"); - qdict_set_default_str(dict, "bar", "ghi"); - - qdict_copy_default(copy, dict, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc"); - qdict_set_default_str(copy, "bar", "xyz"); - qdict_copy_default(copy, dict, "bar"); - g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz"); - - qobject_unref(copy); - qobject_unref(dict); -} - static void qdict_haskey_not_test(void) { QDict *tests_dict = qdict_new(); @@ -XXX,XX +XXX,XX @@ static void qdict_iterapi_test(void) qobject_unref(tests_dict); } -static void qdict_flatten_test(void) -{ - QList *list1 = qlist_new(); - QList *list2 = qlist_new(); - QDict *dict1 = qdict_new(); - QDict *dict2 = qdict_new(); - QDict *dict3 = qdict_new(); - - /* - * Test the flattening of - * - * { - * "e": [ - * 42, - * [ - * 23, - * 66, - * { - * "a": 0, - * "b": 1 - * } - * ] - * ], - * "f": { - * "c": 2, - * "d": 3, - * }, - * "g": 4 - * } - * - * to - * - * { - * "e.0": 42, - * "e.1.0": 23, - * "e.1.1": 66, - * "e.1.2.a": 0, - * "e.1.2.b": 1, - * "f.c": 2, - * "f.d": 3, - * "g": 4 - * } - */ - - qdict_put_int(dict1, "a", 0); - qdict_put_int(dict1, "b", 1); - - qlist_append_int(list1, 23); - qlist_append_int(list1, 66); - qlist_append(list1, dict1); - qlist_append_int(list2, 42); - qlist_append(list2, list1); - - qdict_put_int(dict2, "c", 2); - qdict_put_int(dict2, "d", 3); - qdict_put(dict3, "e", list2); - qdict_put(dict3, "f", dict2); - qdict_put_int(dict3, "g", 4); - - qdict_flatten(dict3); - - g_assert(qdict_get_int(dict3, "e.0") == 42); - g_assert(qdict_get_int(dict3, "e.1.0") == 23); - g_assert(qdict_get_int(dict3, "e.1.1") == 66); - g_assert(qdict_get_int(dict3, "e.1.2.a") == 0); - g_assert(qdict_get_int(dict3, "e.1.2.b") == 1); - g_assert(qdict_get_int(dict3, "f.c") == 2); - g_assert(qdict_get_int(dict3, "f.d") == 3); - g_assert(qdict_get_int(dict3, "g") == 4); - - g_assert(qdict_size(dict3) == 8); - - qobject_unref(dict3); -} - -static void qdict_array_split_test(void) -{ - QDict *test_dict = qdict_new(); - QDict *dict1, *dict2; - QNum *int1; - QList *test_list; - - /* - * Test the split of - * - * { - * "1.x": 0, - * "4.y": 1, - * "0.a": 42, - * "o.o": 7, - * "0.b": 23, - * "2": 66 - * } - * - * to - * - * [ - * { - * "a": 42, - * "b": 23 - * }, - * { - * "x": 0 - * }, - * 66 - * ] - * - * and - * - * { - * "4.y": 1, - * "o.o": 7 - * } - * - * (remaining in the old QDict) - * - * This example is given in the comment of qdict_array_split(). - */ - - qdict_put_int(test_dict, "1.x", 0); - qdict_put_int(test_dict, "4.y", 1); - qdict_put_int(test_dict, "0.a", 42); - qdict_put_int(test_dict, "o.o", 7); - qdict_put_int(test_dict, "0.b", 23); - qdict_put_int(test_dict, "2", 66); - - qdict_array_split(test_dict, &test_list); - - dict1 = qobject_to(QDict, qlist_pop(test_list)); - dict2 = qobject_to(QDict, qlist_pop(test_list)); - int1 = qobject_to(QNum, qlist_pop(test_list)); - - g_assert(dict1); - g_assert(dict2); - g_assert(int1); - g_assert(qlist_empty(test_list)); - - qobject_unref(test_list); - - g_assert(qdict_get_int(dict1, "a") == 42); - g_assert(qdict_get_int(dict1, "b") == 23); - - g_assert(qdict_size(dict1) == 2); - - qobject_unref(dict1); - - g_assert(qdict_get_int(dict2, "x") == 0); - - g_assert(qdict_size(dict2) == 1); - - qobject_unref(dict2); - - g_assert_cmpint(qnum_get_int(int1), ==, 66); - - qobject_unref(int1); - - g_assert(qdict_get_int(test_dict, "4.y") == 1); - g_assert(qdict_get_int(test_dict, "o.o") == 7); - - g_assert(qdict_size(test_dict) == 2); - - qobject_unref(test_dict); - - /* - * Test the split of - * - * { - * "0": 42, - * "1": 23, - * "1.x": 84 - * } - * - * to - * - * [ - * 42 - * ] - * - * and - * - * { - * "1": 23, - * "1.x": 84 - * } - * - * That is, test whether splitting stops if there is both an entry with key - * of "%u" and other entries with keys prefixed "%u." for the same index. - */ - - test_dict = qdict_new(); - - qdict_put_int(test_dict, "0", 42); - qdict_put_int(test_dict, "1", 23); - qdict_put_int(test_dict, "1.x", 84); - - qdict_array_split(test_dict, &test_list); - - int1 = qobject_to(QNum, qlist_pop(test_list)); - - g_assert(int1); - g_assert(qlist_empty(test_list)); - - qobject_unref(test_list); - - g_assert_cmpint(qnum_get_int(int1), ==, 42); - - qobject_unref(int1); - - g_assert(qdict_get_int(test_dict, "1") == 23); - g_assert(qdict_get_int(test_dict, "1.x") == 84); - - g_assert(qdict_size(test_dict) == 2); - - qobject_unref(test_dict); -} - -static void qdict_array_entries_test(void) -{ - QDict *dict = qdict_new(); - - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); - - qdict_put_int(dict, "bar", 0); - qdict_put_int(dict, "baz.0", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); - - qdict_put_int(dict, "foo.1", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); - qdict_put_int(dict, "foo.0", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2); - qdict_put_int(dict, "foo.bar", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); - qdict_del(dict, "foo.bar"); - - qdict_put_int(dict, "foo.2.a", 0); - qdict_put_int(dict, "foo.2.b", 0); - qdict_put_int(dict, "foo.2.c", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); - - qobject_unref(dict); - - dict = qdict_new(); - qdict_put_int(dict, "1", 0); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); - qdict_put_int(dict, "0", 0); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2); - qdict_put_int(dict, "bar", 0); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); - qdict_del(dict, "bar"); - - qdict_put_int(dict, "2.a", 0); - qdict_put_int(dict, "2.b", 0); - qdict_put_int(dict, "2.c", 0); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3); - - qobject_unref(dict); -} - -static void qdict_join_test(void) -{ - QDict *dict1, *dict2; - bool overwrite = false; - int i; - - dict1 = qdict_new(); - dict2 = qdict_new(); - - /* Test everything once without overwrite and once with */ - do - { - /* Test empty dicts */ - qdict_join(dict1, dict2, overwrite); - - g_assert(qdict_size(dict1) == 0); - g_assert(qdict_size(dict2) == 0); - - /* First iteration: Test movement */ - /* Second iteration: Test empty source and non-empty destination */ - qdict_put_int(dict2, "foo", 42); - - for (i = 0; i < 2; i++) { - qdict_join(dict1, dict2, overwrite); - - g_assert(qdict_size(dict1) == 1); - g_assert(qdict_size(dict2) == 0); - - g_assert(qdict_get_int(dict1, "foo") == 42); - } - - /* Test non-empty source and destination without conflict */ - qdict_put_int(dict2, "bar", 23); - - qdict_join(dict1, dict2, overwrite); - - g_assert(qdict_size(dict1) == 2); - g_assert(qdict_size(dict2) == 0); - - g_assert(qdict_get_int(dict1, "foo") == 42); - g_assert(qdict_get_int(dict1, "bar") == 23); - - /* Test conflict */ - qdict_put_int(dict2, "foo", 84); - - qdict_join(dict1, dict2, overwrite); - - g_assert(qdict_size(dict1) == 2); - g_assert(qdict_size(dict2) == !overwrite); - - g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42)); - g_assert(qdict_get_int(dict1, "bar") == 23); - - if (!overwrite) { - g_assert(qdict_get_int(dict2, "foo") == 84); - } - - /* Check the references */ - g_assert(qdict_get(dict1, "foo")->base.refcnt == 1); - g_assert(qdict_get(dict1, "bar")->base.refcnt == 1); - - if (!overwrite) { - g_assert(qdict_get(dict2, "foo")->base.refcnt == 1); - } - - /* Clean up */ - qdict_del(dict1, "foo"); - qdict_del(dict1, "bar"); - - if (!overwrite) { - qdict_del(dict2, "foo"); - } - } - while (overwrite ^= true); - - qobject_unref(dict1); - qobject_unref(dict2); -} - -static void qdict_crumple_test_recursive(void) -{ - QDict *src, *dst, *rule, *vnc, *acl, *listen; - QList *rules; - - src = qdict_new(); - qdict_put_str(src, "vnc.listen.addr", "127.0.0.1"); - qdict_put_str(src, "vnc.listen.port", "5901"); - qdict_put_str(src, "vnc.acl.rules.0.match", "fred"); - qdict_put_str(src, "vnc.acl.rules.0.policy", "allow"); - qdict_put_str(src, "vnc.acl.rules.1.match", "bob"); - qdict_put_str(src, "vnc.acl.rules.1.policy", "deny"); - qdict_put_str(src, "vnc.acl.default", "deny"); - qdict_put_str(src, "vnc.acl..name", "acl0"); - qdict_put_str(src, "vnc.acl.rule..name", "acl0"); - - dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); - g_assert(dst); - g_assert_cmpint(qdict_size(dst), ==, 1); - - vnc = qdict_get_qdict(dst, "vnc"); - g_assert(vnc); - g_assert_cmpint(qdict_size(vnc), ==, 3); - - listen = qdict_get_qdict(vnc, "listen"); - g_assert(listen); - g_assert_cmpint(qdict_size(listen), ==, 2); - g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr")); - g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port")); - - acl = qdict_get_qdict(vnc, "acl"); - g_assert(acl); - g_assert_cmpint(qdict_size(acl), ==, 3); - - rules = qdict_get_qlist(acl, "rules"); - g_assert(rules); - g_assert_cmpint(qlist_size(rules), ==, 2); - - rule = qobject_to(QDict, qlist_pop(rules)); - g_assert(rule); - g_assert_cmpint(qdict_size(rule), ==, 2); - g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match")); - g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy")); - qobject_unref(rule); - - rule = qobject_to(QDict, qlist_pop(rules)); - g_assert(rule); - g_assert_cmpint(qdict_size(rule), ==, 2); - g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match")); - g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy")); - qobject_unref(rule); - - /* With recursive crumpling, we should see all names unescaped */ - g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); - g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); - - qobject_unref(src); - qobject_unref(dst); -} - -static void qdict_crumple_test_empty(void) -{ - QDict *src, *dst; - - src = qdict_new(); - - dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); - - g_assert_cmpint(qdict_size(dst), ==, 0); - - qobject_unref(src); - qobject_unref(dst); -} - -static int qdict_count_entries(QDict *dict) -{ - const QDictEntry *e; - int count = 0; - - for (e = qdict_first(dict); e; e = qdict_next(dict, e)) { - count++; - } - - return count; -} - -static void qdict_rename_keys_test(void) -{ - QDict *dict = qdict_new(); - QDict *copy; - QDictRenames *renames; - Error *local_err = NULL; - - qdict_put_str(dict, "abc", "foo"); - qdict_put_str(dict, "abcdef", "bar"); - qdict_put_int(dict, "number", 42); - qdict_put_bool(dict, "flag", true); - qdict_put_null(dict, "nothing"); - - /* Empty rename list */ - renames = (QDictRenames[]) { - { NULL, "this can be anything" } - }; - copy = qdict_clone_shallow(dict); - qdict_rename_keys(copy, renames, &error_abort); - - g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); - g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); - g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); - g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); - g_assert_cmpint(qdict_count_entries(copy), ==, 5); - - qobject_unref(copy); - - /* Simple rename of all entries */ - renames = (QDictRenames[]) { - { "abc", "str1" }, - { "abcdef", "str2" }, - { "number", "int" }, - { "flag", "bool" }, - { "nothing", "null" }, - { NULL , NULL } - }; - copy = qdict_clone_shallow(dict); - qdict_rename_keys(copy, renames, &error_abort); - - g_assert(!qdict_haskey(copy, "abc")); - g_assert(!qdict_haskey(copy, "abcdef")); - g_assert(!qdict_haskey(copy, "number")); - g_assert(!qdict_haskey(copy, "flag")); - g_assert(!qdict_haskey(copy, "nothing")); - - g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar"); - g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42); - g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true); - g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL); - g_assert_cmpint(qdict_count_entries(copy), ==, 5); - - qobject_unref(copy); - - /* Renames are processed top to bottom */ - renames = (QDictRenames[]) { - { "abc", "tmp" }, - { "abcdef", "abc" }, - { "number", "abcdef" }, - { "flag", "number" }, - { "nothing", "flag" }, - { "tmp", "nothing" }, - { NULL , NULL } - }; - copy = qdict_clone_shallow(dict); - qdict_rename_keys(copy, renames, &error_abort); - - g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar"); - g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42); - g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true); - g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL); - g_assert(!qdict_haskey(copy, "tmp")); - g_assert_cmpint(qdict_count_entries(copy), ==, 5); - - qobject_unref(copy); - - /* Conflicting rename */ - renames = (QDictRenames[]) { - { "abcdef", "abc" }, - { NULL , NULL } - }; - copy = qdict_clone_shallow(dict); - qdict_rename_keys(copy, renames, &local_err); - - g_assert(local_err != NULL); - error_free(local_err); - local_err = NULL; - - g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); - g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); - g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); - g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); - g_assert_cmpint(qdict_count_entries(copy), ==, 5); - - qobject_unref(copy); - - /* Renames in an empty dict */ - renames = (QDictRenames[]) { - { "abcdef", "abc" }, - { NULL , NULL } - }; - - qobject_unref(dict); - dict = qdict_new(); - - qdict_rename_keys(dict, renames, &error_abort); - g_assert(qdict_first(dict) == NULL); - - qobject_unref(dict); -} - -static void qdict_crumple_test_bad_inputs(void) -{ - QDict *src; - Error *error = NULL; - - src = qdict_new(); - /* rule.0 can't be both a string and a dict */ - qdict_put_str(src, "rule.0", "fred"); - qdict_put_str(src, "rule.0.policy", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); - - src = qdict_new(); - /* rule can't be both a list and a dict */ - qdict_put_str(src, "rule.0", "fred"); - qdict_put_str(src, "rule.a", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); - - src = qdict_new(); - /* The input should be flat, ie no dicts or lists */ - qdict_put(src, "rule.a", qdict_new()); - qdict_put_str(src, "rule.b", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); - - src = qdict_new(); - /* List indexes must not have gaps */ - qdict_put_str(src, "rule.0", "deny"); - qdict_put_str(src, "rule.3", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); - - src = qdict_new(); - /* List indexes must be in %zu format */ - qdict_put_str(src, "rule.0", "deny"); - qdict_put_str(src, "rule.+1", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); -} - /* * Errors test-cases */ @@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv) g_test_add_func("/public/get_try_int", qdict_get_try_int_test); g_test_add_func("/public/get_str", qdict_get_str_test); g_test_add_func("/public/get_try_str", qdict_get_try_str_test); - g_test_add_func("/public/defaults", qdict_defaults_test); g_test_add_func("/public/haskey_not", qdict_haskey_not_test); g_test_add_func("/public/haskey", qdict_haskey_test); g_test_add_func("/public/del", qdict_del_test); g_test_add_func("/public/to_qdict", qobject_to_qdict_test); g_test_add_func("/public/iterapi", qdict_iterapi_test); - g_test_add_func("/public/flatten", qdict_flatten_test); - g_test_add_func("/public/array_split", qdict_array_split_test); - g_test_add_func("/public/array_entries", qdict_array_entries_test); - g_test_add_func("/public/join", qdict_join_test); g_test_add_func("/errors/put_exists", qdict_put_exists_test); g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test); - g_test_add_func("/public/crumple/recursive", - qdict_crumple_test_recursive); - g_test_add_func("/public/crumple/empty", - qdict_crumple_test_empty); - g_test_add_func("/public/crumple/bad_inputs", - qdict_crumple_test_bad_inputs); - - g_test_add_func("/public/rename_keys", qdict_rename_keys_test); - /* The Big one */ if (g_test_slow()) { g_test_add_func("/stress/test", qdict_stress_test); diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: qemu-img* F: qemu-io* F: tests/qemu-iotests/ F: util/qemu-progress.c +F: qobject/block-qdict.c +F: test/check-block-qdict.c T: git git://repo.or.cz/qemu/kevin.git block Block I/O path diff --git a/qobject/Makefile.objs b/qobject/Makefile.objs index XXXXXXX..XXXXXXX 100644 --- a/qobject/Makefile.objs +++ b/qobject/Makefile.objs @@ -XXX,XX +XXX,XX @@ util-obj-y = qnull.o qnum.o qstring.o qdict.o qlist.o qbool.o qlit.o util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.o +util-obj-y += block-qdict.o diff --git a/tests/Makefile.include b/tests/Makefile.include index XXXXXXX..XXXXXXX 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -XXX,XX +XXX,XX @@ SYSEMU_TARGET_LIST := $(subst -softmmu.mak,,$(notdir \ check-unit-y = tests/check-qdict$(EXESUF) gcov-files-check-qdict-y = qobject/qdict.c +check-unit-y = tests/check-block-qdict$(EXESUF) +gcov-files-check-block-qdict-y = qobject/block-qdict.c check-unit-y += tests/test-char$(EXESUF) gcov-files-check-qdict-y = chardev/char.c check-unit-y += tests/check-qnum$(EXESUF) @@ -XXX,XX +XXX,XX @@ GENERATED_FILES += tests/test-qapi-types.h tests/test-qapi-visit.h \ test-obj-y = tests/check-qnum.o tests/check-qstring.o tests/check-qdict.o \ tests/check-qlist.o tests/check-qnull.o tests/check-qobject.o \ tests/check-qjson.o tests/check-qlit.o \ + tests/check-block-qtest.o \ tests/test-coroutine.o tests/test-string-output-visitor.o \ tests/test-string-input-visitor.o tests/test-qobject-output-visitor.o \ tests/test-clone-visitor.o \ @@ -XXX,XX +XXX,XX @@ test-block-obj-y = $(block-obj-y) $(test-io-obj-y) tests/iothread.o tests/check-qnum$(EXESUF): tests/check-qnum.o $(test-util-obj-y) tests/check-qstring$(EXESUF): tests/check-qstring.o $(test-util-obj-y) tests/check-qdict$(EXESUF): tests/check-qdict.o $(test-util-obj-y) +tests/check-block-qdict$(EXESUF): tests/check-block-qdict.o $(test-util-obj-y) tests/check-qlist$(EXESUF): tests/check-qlist.o $(test-util-obj-y) tests/check-qnull$(EXESUF): tests/check-qnull.o $(test-util-obj-y) tests/check-qobject$(EXESUF): tests/check-qobject.o $(test-util-obj-y) -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Configuration flows through the block subsystem in a rather peculiar way. Configuration made with -drive enters it as QemuOpts. Configuration made with -blockdev / blockdev-add enters it as QAPI type BlockdevOptions. The block subsystem uses QDict, QemuOpts and QAPI types internally. The precise flow is next to impossible to explain (I tried for this commit message, but gave up after wasting several hours). What I can explain is a flaw in the BlockDriver interface that leads to this bug: $ qemu-system-x86_64 -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234 qemu-system-x86_64: -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234: Internal error: parameter user invalid QMP blockdev-add is broken the same way. Here's what happens. The block layer passes configuration represented as flat QDict (with dotted keys) to BlockDriver methods .bdrv_file_open(). The QDict's members are typed according to the QAPI schema. nfs_file_open() converts it to QAPI type BlockdevOptionsNfs, with qdict_crumple() and a qobject input visitor. This visitor comes in two flavors. The plain flavor requires scalars to be typed according to the QAPI schema. That's the case here. The keyval flavor requires string scalars. That's not the case here. nfs_file_open() uses the latter, and promptly falls apart for members @user, @group, @tcp-syn-count, @readahead-size, @page-cache-size, @debug. Switching to the plain flavor would fix -blockdev, but break -drive, because there the scalars arrive in nfs_file_open() as strings. The proper fix would be to replace the QDict by QAPI type BlockdevOptions in the BlockDriver interface. Sadly, that's beyond my reach right now. Next best would be to fix the block layer to always pass correctly typed QDicts to the BlockDriver methods. Also beyond my reach. What I can do is throw another hack onto the pile: have nfs_file_open() convert all members to string, so use of the keyval flavor actually works, by replacing qdict_crumple() by new function qdict_crumple_for_keyval_qiv(). The pattern "pass result of qdict_crumple() to qobject_input_visitor_new_keyval()" occurs several times more: * qemu_rbd_open() Same issue as nfs_file_open(), but since BlockdevOptionsRbd has only string members, its only a latent bug. Fix it anyway. * parallels_co_create_opts(), qcow_co_create_opts(), qcow2_co_create_opts(), bdrv_qed_co_create_opts(), sd_co_create_opts(), vhdx_co_create_opts(), vpc_co_create_opts() These work, because they create the QDict with qemu_opts_to_qdict_filtered(), which creates only string scalars. The function sports a TODO comment asking for better typing; that's going to be fun. Use qdict_crumple_for_keyval_qiv() to be safe. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/qdict.h | 1 + block/nfs.c | 2 +- block/parallels.c | 2 +- block/qcow.c | 2 +- block/qcow2.c | 2 +- block/qed.c | 2 +- block/rbd.c | 2 +- block/sheepdog.c | 2 +- block/vhdx.c | 2 +- block/vpc.c | 2 +- qobject/block-qdict.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++ 11 files changed, 67 insertions(+), 9 deletions(-) diff --git a/include/block/qdict.h b/include/block/qdict.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/qdict.h +++ b/include/block/qdict.h @@ -XXX,XX +XXX,XX @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); void qdict_array_split(QDict *src, QList **dst); int qdict_array_entries(QDict *src, const char *subqdict); QObject *qdict_crumple(const QDict *src, Error **errp); +QObject *qdict_crumple_for_keyval_qiv(QDict *qdict, Error **errp); void qdict_flatten(QDict *qdict); typedef struct QDictRenames { diff --git a/block/nfs.c b/block/nfs.c index XXXXXXX..XXXXXXX 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, const QDictEntry *e; Error *local_err = NULL; - crumpled = qdict_crumple(options, errp); + crumpled = qdict_crumple_for_keyval_qiv(options, errp); if (crumpled == NULL) { return NULL; } diff --git a/block/parallels.c b/block/parallels.c index XXXXXXX..XXXXXXX 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "parallels"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "qcow"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt qdict_put_str(qdict, "file", bs->node_name); /* Now get the QAPI type BlockdevCreateOptions */ - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/qed.c b/block/qed.c index XXXXXXX..XXXXXXX 100644 --- a/block/qed.c +++ b/block/qed.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "qed"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, } /* Convert the remaining options into a QAPI object */ - crumpled = qdict_crumple(options, errp); + crumpled = qdict_crumple_for_keyval_qiv(options, errp); if (crumpled == NULL) { r = -EINVAL; goto out; diff --git a/block/sheepdog.c b/block/sheepdog.c index XXXXXXX..XXXXXXX 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, } /* Get the QAPI object */ - crumpled = qdict_crumple(qdict, errp); + crumpled = qdict_crumple_for_keyval_qiv(qdict, errp); if (crumpled == NULL) { ret = -EINVAL; goto fail; diff --git a/block/vhdx.c b/block/vhdx.c index XXXXXXX..XXXXXXX 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "vhdx"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/vpc.c b/block/vpc.c index XXXXXXX..XXXXXXX 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "vpc"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "block/qdict.h" +#include "qapi/qmp/qbool.h" #include "qapi/qmp/qlist.h" +#include "qapi/qmp/qnum.h" +#include "qapi/qmp/qstring.h" #include "qemu/cutils.h" #include "qapi/error.h" @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) } /** + * qdict_crumple_for_keyval_qiv: + * @src: the flat dictionary (only scalar values) to crumple + * @errp: location to store error + * + * Like qdict_crumple(), but additionally transforms scalar values so + * the result can be passed to qobject_input_visitor_new_keyval(). + * + * The block subsystem uses this function to prepare its flat QDict + * with possibly confused scalar types for a visit. It should not be + * used for anything else, and it should go away once the block + * subsystem has been cleaned up. + */ +QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp) +{ + QDict *tmp = NULL; + char *buf; + const char *s; + const QDictEntry *ent; + QObject *dst; + + for (ent = qdict_first(src); ent; ent = qdict_next(src, ent)) { + buf = NULL; + switch (qobject_type(ent->value)) { + case QTYPE_QNULL: + case QTYPE_QSTRING: + continue; + case QTYPE_QNUM: + s = buf = qnum_to_string(qobject_to(QNum, ent->value)); + break; + case QTYPE_QDICT: + case QTYPE_QLIST: + /* @src isn't flat; qdict_crumple() will fail */ + continue; + case QTYPE_QBOOL: + s = qbool_get_bool(qobject_to(QBool, ent->value)) + ? "on" : "off"; + break; + default: + abort(); + } + + if (!tmp) { + tmp = qdict_clone_shallow(src); + } + qdict_put(tmp, ent->key, qstring_from_str(s)); + g_free(buf); + } + + dst = qdict_crumple(tmp ?: src, errp); + qobject_unref(tmp); + return dst; +} + +/** * qdict_array_entries(): Returns the number of direct array entries if the * sub-QDict of src specified by the prefix in subqdict (or src itself for * prefix == "") is valid as an array, i.e. the length of the created list if -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> The previous commit fixed -blockdev breakage due to misuse of the qobject input visitor's keyval flavor in bdrv_file_open(). The commit message explain why using the plain flavor would be just as wrong; it would break -drive. Turns out we break it in three places: nbd_open(), sd_open() and ssh_file_open(). They are even marked FIXME. Example breakage: $ qemu-system-x86 -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off qemu-system-x86: -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off: Invalid parameter type for 'numeric', expected: boolean Fix it the same way: replace qdict_crumple() by qdict_crumple_for_keyval_qiv(), and switch from plain to the keyval flavor. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/nbd.c | 12 ++---------- block/sheepdog.c | 12 ++---------- block/ssh.c | 12 ++---------- 3 files changed, 6 insertions(+), 30 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -XXX,XX +XXX,XX @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, goto done; } - crumpled_addr = qdict_crumple(addr, errp); + crumpled_addr = qdict_crumple_for_keyval_qiv(addr, errp); if (!crumpled_addr) { goto done; } - /* - * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive - * server.type=inet. .to doesn't matter, it's ignored anyway. - * That's because when @options come from -blockdev or - * blockdev_add, members are typed according to the QAPI schema, - * but when they come from -drive, they're all QString. The - * visitor expects the former. - */ - iv = qobject_input_visitor_new(crumpled_addr); + iv = qobject_input_visitor_new_keyval(crumpled_addr); visit_type_SocketAddress(iv, NULL, &saddr, &local_err); if (local_err) { error_propagate(errp, local_err); diff --git a/block/sheepdog.c b/block/sheepdog.c index XXXXXXX..XXXXXXX 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -XXX,XX +XXX,XX @@ static SocketAddress *sd_server_config(QDict *options, Error **errp) qdict_extract_subqdict(options, &server, "server."); - crumpled_server = qdict_crumple(server, errp); + crumpled_server = qdict_crumple_for_keyval_qiv(server, errp); if (!crumpled_server) { goto done; } - /* - * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive - * server.type=inet. .to doesn't matter, it's ignored anyway. - * That's because when @options come from -blockdev or - * blockdev_add, members are typed according to the QAPI schema, - * but when they come from -drive, they're all QString. The - * visitor expects the former. - */ - iv = qobject_input_visitor_new(crumpled_server); + iv = qobject_input_visitor_new_keyval(crumpled_server); visit_type_SocketAddress(iv, NULL, &saddr, &local_err); if (local_err) { error_propagate(errp, local_err); diff --git a/block/ssh.c b/block/ssh.c index XXXXXXX..XXXXXXX 100644 --- a/block/ssh.c +++ b/block/ssh.c @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) } /* Create the QAPI object */ - crumpled = qdict_crumple(options, errp); + crumpled = qdict_crumple_for_keyval_qiv(options, errp); if (crumpled == NULL) { goto fail; } - /* - * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive. - * .to doesn't matter, it's ignored anyway. - * That's because when @options come from -blockdev or - * blockdev_add, members are typed according to the QAPI schema, - * but when they come from -drive, they're all QString. The - * visitor expects the former. - */ - v = qobject_input_visitor_new(crumpled); + v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevOptionsSsh(v, NULL, &result, &local_err); visit_free(v); qobject_unref(crumpled); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> The following pattern occurs in the .bdrv_co_create_opts() methods of parallels, qcow, qcow2, qed, vhdx and vpc: qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { ret = -EINVAL; goto done; } v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); [...] ret = 0; done: qobject_unref(qdict); [...] return ret; If qobject_to() fails, we return failure without setting errp. That's wrong. As far as I can tell, it cannot fail here. Clean it up anyway, by removing the useless conversion. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/parallels.c | 9 ++++----- block/qcow.c | 9 ++++----- block/qcow2.c | 9 ++++----- block/qed.c | 9 ++++----- block/vhdx.c | 9 ++++----- block/vpc.c | 9 ++++----- 6 files changed, 24 insertions(+), 30 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index XXXXXXX..XXXXXXX 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, BlockdevCreateOptions *create_options = NULL; Error *local_err = NULL; BlockDriverState *bs = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; int ret; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto done; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, { BlockdevCreateOptions *create_options = NULL; BlockDriverState *bs = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; const char *val; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt Error **errp) { BlockdevCreateOptions *create_options = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt /* Now get the QAPI type BlockdevCreateOptions */ qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto finish; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qed.c b/block/qed.c index XXXXXXX..XXXXXXX 100644 --- a/block/qed.c +++ b/block/qed.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, Error **errp) { BlockdevCreateOptions *create_options = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/vhdx.c b/block/vhdx.c index XXXXXXX..XXXXXXX 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, Error **errp) { BlockdevCreateOptions *create_options = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/vpc.c b/block/vpc.c index XXXXXXX..XXXXXXX 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, QemuOpts *opts, Error **errp) { BlockdevCreateOptions *create_options = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/qdict.h | 3 ++- block/nbd.c | 7 ++----- block/nfs.c | 7 ++----- block/parallels.c | 7 ++----- block/qcow.c | 7 ++----- block/qcow2.c | 7 ++----- block/qed.c | 7 ++----- block/rbd.c | 7 ++----- block/sheepdog.c | 14 ++++---------- block/ssh.c | 7 ++----- block/vhdx.c | 7 ++----- block/vpc.c | 7 ++----- qobject/block-qdict.c | 28 +++++++++++++++++++++++++++- 13 files changed, 53 insertions(+), 62 deletions(-) diff --git a/include/block/qdict.h b/include/block/qdict.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/qdict.h +++ b/include/block/qdict.h @@ -XXX,XX +XXX,XX @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); void qdict_array_split(QDict *src, QList **dst); int qdict_array_entries(QDict *src, const char *subqdict); QObject *qdict_crumple(const QDict *src, Error **errp); -QObject *qdict_crumple_for_keyval_qiv(QDict *qdict, Error **errp); void qdict_flatten(QDict *qdict); typedef struct QDictRenames { @@ -XXX,XX +XXX,XX @@ typedef struct QDictRenames { } QDictRenames; bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp); +Visitor *qobject_input_visitor_new_flat_confused(QDict *qdict, + Error **errp); #endif diff --git a/block/nbd.c b/block/nbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -XXX,XX +XXX,XX @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, { SocketAddress *saddr = NULL; QDict *addr = NULL; - QObject *crumpled_addr = NULL; Visitor *iv = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, goto done; } - crumpled_addr = qdict_crumple_for_keyval_qiv(addr, errp); - if (!crumpled_addr) { + iv = qobject_input_visitor_new_flat_confused(addr, errp); + if (!iv) { goto done; } - iv = qobject_input_visitor_new_keyval(crumpled_addr); visit_type_SocketAddress(iv, NULL, &saddr, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -XXX,XX +XXX,XX @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, done: qobject_unref(addr); - qobject_unref(crumpled_addr); visit_free(iv); return saddr; } diff --git a/block/nfs.c b/block/nfs.c index XXXXXXX..XXXXXXX 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, Error **errp) { BlockdevOptionsNfs *opts = NULL; - QObject *crumpled = NULL; Visitor *v; const QDictEntry *e; Error *local_err = NULL; - crumpled = qdict_crumple_for_keyval_qiv(options, errp); - if (crumpled == NULL) { + v = qobject_input_visitor_new_flat_confused(options, errp); + if (!v) { return NULL; } - v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevOptionsNfs(v, NULL, &opts, &local_err); visit_free(v); - qobject_unref(crumpled); if (local_err) { error_propagate(errp, local_err); diff --git a/block/parallels.c b/block/parallels.c index XXXXXXX..XXXXXXX 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, Error *local_err = NULL; BlockDriverState *bs = NULL; QDict *qdict; - QObject *qobj; Visitor *v; int ret; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "parallels"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto done; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, BlockdevCreateOptions *create_options = NULL; BlockDriverState *bs = NULL; QDict *qdict; - QObject *qobj; Visitor *v; const char *val; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "qcow"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt { BlockdevCreateOptions *create_options = NULL; QDict *qdict; - QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt qdict_put_str(qdict, "file", bs->node_name); /* Now get the QAPI type BlockdevCreateOptions */ - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto finish; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qed.c b/block/qed.c index XXXXXXX..XXXXXXX 100644 --- a/block/qed.c +++ b/block/qed.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, { BlockdevCreateOptions *create_options = NULL; QDict *qdict; - QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "qed"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, BDRVRBDState *s = bs->opaque; BlockdevOptionsRbd *opts = NULL; Visitor *v; - QObject *crumpled = NULL; const QDictEntry *e; Error *local_err = NULL; char *keypairs, *secretid; @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, } /* Convert the remaining options into a QAPI object */ - crumpled = qdict_crumple_for_keyval_qiv(options, errp); - if (crumpled == NULL) { + v = qobject_input_visitor_new_flat_confused(options, errp); + if (!v) { r = -EINVAL; goto out; } - v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevOptionsRbd(v, NULL, &opts, &local_err); visit_free(v); - qobject_unref(crumpled); if (local_err) { error_propagate(errp, local_err); diff --git a/block/sheepdog.c b/block/sheepdog.c index XXXXXXX..XXXXXXX 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -XXX,XX +XXX,XX @@ static void sd_aio_setup(SheepdogAIOCB *acb, BDRVSheepdogState *s, static SocketAddress *sd_server_config(QDict *options, Error **errp) { QDict *server = NULL; - QObject *crumpled_server = NULL; Visitor *iv = NULL; SocketAddress *saddr = NULL; Error *local_err = NULL; qdict_extract_subqdict(options, &server, "server."); - crumpled_server = qdict_crumple_for_keyval_qiv(server, errp); - if (!crumpled_server) { + iv = qobject_input_visitor_new_flat_confused(server, errp); + if (!iv) { goto done; } - iv = qobject_input_visitor_new_keyval(crumpled_server); visit_type_SocketAddress(iv, NULL, &saddr, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -XXX,XX +XXX,XX @@ static SocketAddress *sd_server_config(QDict *options, Error **errp) done: visit_free(iv); - qobject_unref(crumpled_server); qobject_unref(server); return saddr; } @@ -XXX,XX +XXX,XX @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, { BlockdevCreateOptions *create_options = NULL; QDict *qdict, *location_qdict; - QObject *crumpled; Visitor *v; char *redundancy; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, } /* Get the QAPI object */ - crumpled = qdict_crumple_for_keyval_qiv(qdict, errp); - if (crumpled == NULL) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); - qobject_unref(crumpled); if (local_err) { error_propagate(errp, local_err); diff --git a/block/ssh.c b/block/ssh.c index XXXXXXX..XXXXXXX 100644 --- a/block/ssh.c +++ b/block/ssh.c @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) BlockdevOptionsSsh *result = NULL; QemuOpts *opts = NULL; Error *local_err = NULL; - QObject *crumpled; const QDictEntry *e; Visitor *v; @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) } /* Create the QAPI object */ - crumpled = qdict_crumple_for_keyval_qiv(options, errp); - if (crumpled == NULL) { + v = qobject_input_visitor_new_flat_confused(options, errp); + if (!v) { goto fail; } - v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevOptionsSsh(v, NULL, &result, &local_err); visit_free(v); - qobject_unref(crumpled); if (local_err) { error_propagate(errp, local_err); diff --git a/block/vhdx.c b/block/vhdx.c index XXXXXXX..XXXXXXX 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, { BlockdevCreateOptions *create_options = NULL; QDict *qdict; - QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "vhdx"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/vpc.c b/block/vpc.c index XXXXXXX..XXXXXXX 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, { BlockdevCreateOptions *create_options = NULL; QDict *qdict; - QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "vpc"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ #include "qapi/qmp/qlist.h" #include "qapi/qmp/qnum.h" #include "qapi/qmp/qstring.h" +#include "qapi/qobject-input-visitor.h" #include "qemu/cutils.h" #include "qapi/error.h" @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) * used for anything else, and it should go away once the block * subsystem has been cleaned up. */ -QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp) +static QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp) { QDict *tmp = NULL; char *buf; @@ -XXX,XX +XXX,XX @@ bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) } return true; } + +/* + * Create a QObject input visitor for flat @qdict with possibly + * confused scalar types. + * + * The block subsystem uses this function to visit its flat QDict with + * possibly confused scalar types. It should not be used for anything + * else, and it should go away once the block subsystem has been + * cleaned up. + */ +Visitor *qobject_input_visitor_new_flat_confused(QDict *qdict, + Error **errp) +{ + QObject *crumpled; + Visitor *v; + + crumpled = qdict_crumple_for_keyval_qiv(qdict, errp); + if (!crumpled) { + return NULL; + } + + v = qobject_input_visitor_new_keyval(crumpled); + qobject_unref(crumpled); + return v; +} -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Remaining uses of qobject_input_visitor_new_keyval() in the block subsystem: * block_crypto_open_opts_init() Currently doesn't visit any non-string scalars, thus safe. It's called from - block_crypto_open_luks() Creates the QDict with qemu_opts_to_qdict_filtered(), which creates only string scalars, but has a TODO asking for other types. - qcow_open() - qcow2_open(), qcow2_co_invalidate_cache(), qcow2_reopen_prepare() * block_crypto_create_opts_init(), called from - block_crypto_co_create_opts_luks() Also creates the QDict with qemu_opts_to_qdict_filtered(). * vdi_co_create_opts() Also creates the QDict with qemu_opts_to_qdict_filtered(). Replace these uses by qobject_input_visitor_new_flat_confused() for robustness. This adds crumpling. Right now, that's a no-op, but if we ever extend these things in non-flat ways, crumpling will be needed. Signed-off-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/crypto.c | 12 +++++++++--- block/vdi.c | 8 ++++++-- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "crypto/block.h" #include "qapi/opts-visitor.h" #include "qapi/qapi-visit-crypto.h" -#include "qapi/qmp/qdict.h" #include "qapi/qobject-input-visitor.h" #include "qapi/error.h" #include "qemu/option.h" @@ -XXX,XX +XXX,XX @@ block_crypto_open_opts_init(QCryptoBlockFormat format, ret = g_new0(QCryptoBlockOpenOptions, 1); ret->format = format; - v = qobject_input_visitor_new_keyval(QOBJECT(opts)); + v = qobject_input_visitor_new_flat_confused(opts, &local_err); + if (local_err) { + goto out; + } visit_start_struct(v, NULL, NULL, 0, &local_err); if (local_err) { @@ -XXX,XX +XXX,XX @@ block_crypto_create_opts_init(QCryptoBlockFormat format, ret = g_new0(QCryptoBlockCreateOptions, 1); ret->format = format; - v = qobject_input_visitor_new_keyval(QOBJECT(opts)); + v = qobject_input_visitor_new_flat_confused(opts, &local_err); + if (local_err) { + goto out; + } visit_start_struct(v, NULL, NULL, 0, &local_err); if (local_err) { diff --git a/block/vdi.c b/block/vdi.c index XXXXXXX..XXXXXXX 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qapi/error.h" -#include "qapi/qmp/qdict.h" #include "qapi/qobject-input-visitor.h" #include "qapi/qapi-visit-block-core.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vdi_co_create_opts(const char *filename, QemuOpts *opts, } /* Get the QAPI object */ - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { + ret = -EINVAL; + goto done; + } visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> There's no need to restart the loop. We don't elsewhere, e.g. in qdict_extract_subqdict(), qdict_join() and qemu_opts_absorb_qdict(). Simplify accordingly. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) QObject *value; const QDictEntry *entry, *next; char *new_key; - bool delete; entry = qdict_first(qdict); while (entry != NULL) { - next = qdict_next(qdict, entry); value = qdict_entry_value(entry); new_key = NULL; - delete = false; if (prefix) { new_key = g_strdup_printf("%s.%s", prefix, entry->key); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) * itself disappears. */ qdict_flatten_qdict(qobject_to(QDict, value), target, new_key ? new_key : entry->key); - delete = true; + qdict_del(qdict, entry->key); } else if (qobject_type(value) == QTYPE_QLIST) { qdict_flatten_qlist(qobject_to(QList, value), target, new_key ? new_key : entry->key); - delete = true; + qdict_del(qdict, entry->key); } else if (prefix) { /* All other objects are moved to the target unchanged. */ qdict_put_obj(target, new_key, qobject_ref(value)); - delete = true; - } - - g_free(new_key); - - if (delete) { qdict_del(qdict, entry->key); - - /* Restart loop after modifying the iterated QDict */ - entry = qdict_first(qdict); - continue; } + g_free(new_key); entry = next; } } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> qdict_flatten_qdict() skips copying scalars from @qdict to @target when the two are the same. Fair enough, but it uses a non-obvious test for "same". Replace it by the obvious one. While there, improve comments. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) value = qlist_entry_obj(entry); new_key = g_strdup_printf("%s.%i", prefix, i); + /* + * Flatten non-empty QDict and QList recursively into @target, + * copy other objects to @target + */ if (qobject_type(value) == QTYPE_QDICT) { qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); } else if (qobject_type(value) == QTYPE_QLIST) { qdict_flatten_qlist(qobject_to(QList, value), target, new_key); } else { - /* All other types are moved to the target unchanged. */ qdict_put_obj(target, new_key, qobject_ref(value)); } @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) new_key = g_strdup_printf("%s.%s", prefix, entry->key); } + /* + * Flatten non-empty QDict and QList recursively into @target, + * copy other objects to @target + */ if (qobject_type(value) == QTYPE_QDICT) { - /* Entries of QDicts are processed recursively, the QDict object - * itself disappears. */ qdict_flatten_qdict(qobject_to(QDict, value), target, new_key ? new_key : entry->key); qdict_del(qdict, entry->key); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) qdict_flatten_qlist(qobject_to(QList, value), target, new_key ? new_key : entry->key); qdict_del(qdict, entry->key); - } else if (prefix) { - /* All other objects are moved to the target unchanged. */ + } else if (target != qdict) { qdict_put_obj(target, new_key, qobject_ref(value)); qdict_del(qdict, entry->key); } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> When you mix scalar and non-scalar keys, whether you get an "already set as scalar" or an "already set as dict" error depends on qdict iteration order. Neither message makes much sense. Replace by ""Cannot mix scalar and non-scalar keys". This is similar to the message we get for mixing list and non-list keys. I find qdict_crumple()'s first loop hard to understand. Rearrange it and add a comment. Signed-off-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static int qdict_is_list(QDict *maybe_list, Error **errp) QObject *qdict_crumple(const QDict *src, Error **errp) { const QDictEntry *ent; - QDict *two_level, *multi_level = NULL; + QDict *two_level, *multi_level = NULL, *child_dict; QObject *dst = NULL, *child; size_t i; char *prefix = NULL; @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) } qdict_split_flat_key(ent->key, &prefix, &suffix); - child = qdict_get(two_level, prefix); + child_dict = qobject_to(QDict, child); + + if (child) { + /* + * If @child_dict, then all previous keys with this prefix + * had a suffix. If @suffix, this one has one as well, + * and we're good, else there's a clash. + */ + if (!child_dict || !suffix) { + error_setg(errp, "Cannot mix scalar and non-scalar keys"); + goto error; + } + } + if (suffix) { - QDict *child_dict = qobject_to(QDict, child); if (!child_dict) { - if (child) { - error_setg(errp, "Key %s prefix is already set as a scalar", - prefix); - goto error; - } - child_dict = qdict_new(); - qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); + qdict_put(two_level, prefix, child_dict); } - qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); } else { - if (child) { - error_setg(errp, "Key %s prefix is already set as a dict", - prefix); - goto error; - } qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 27 +++++++++++---------------- 1 file changed, 11 insertions(+), 16 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static int qdict_is_list(QDict *maybe_list, Error **errp) for (ent = qdict_first(maybe_list); ent != NULL; ent = qdict_next(maybe_list, ent)) { + int is_index = !qemu_strtoi64(ent->key, NULL, 10, &val); - if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) { - if (is_list == -1) { - is_list = 1; - } else if (!is_list) { - error_setg(errp, - "Cannot mix list and non-list keys"); - return -1; - } + if (is_list == -1) { + is_list = is_index; + } + + if (is_index != is_list) { + error_setg(errp, "Cannot mix list and non-list keys"); + return -1; + } + + if (is_index) { len++; if (val > max) { max = val; } - } else { - if (is_list == -1) { - is_list = 0; - } else if (is_list) { - error_setg(errp, - "Cannot mix list and non-list keys"); - return -1; - } } } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/check-block-qdict.c | 57 ++++++++++++++++++++++++----------------------- 1 file changed, 29 insertions(+), 28 deletions(-) diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-block-qdict.c +++ b/tests/check-block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_defaults_test(void) static void qdict_flatten_test(void) { - QList *list1 = qlist_new(); - QList *list2 = qlist_new(); - QDict *dict1 = qdict_new(); - QDict *dict2 = qdict_new(); - QDict *dict3 = qdict_new(); + QList *e_1 = qlist_new(); + QList *e = qlist_new(); + QDict *e_1_2 = qdict_new(); + QDict *f = qdict_new(); + QDict *root = qdict_new(); /* * Test the flattening of @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) * } */ - qdict_put_int(dict1, "a", 0); - qdict_put_int(dict1, "b", 1); + qdict_put_int(e_1_2, "a", 0); + qdict_put_int(e_1_2, "b", 1); - qlist_append_int(list1, 23); - qlist_append_int(list1, 66); - qlist_append(list1, dict1); - qlist_append_int(list2, 42); - qlist_append(list2, list1); + qlist_append_int(e_1, 23); + qlist_append_int(e_1, 66); + qlist_append(e_1, e_1_2); + qlist_append_int(e, 42); + qlist_append(e, e_1); - qdict_put_int(dict2, "c", 2); - qdict_put_int(dict2, "d", 3); - qdict_put(dict3, "e", list2); - qdict_put(dict3, "f", dict2); - qdict_put_int(dict3, "g", 4); + qdict_put_int(f, "c", 2); + qdict_put_int(f, "d", 3); - qdict_flatten(dict3); + qdict_put(root, "e", e); + qdict_put(root, "f", f); + qdict_put_int(root, "g", 4); - g_assert(qdict_get_int(dict3, "e.0") == 42); - g_assert(qdict_get_int(dict3, "e.1.0") == 23); - g_assert(qdict_get_int(dict3, "e.1.1") == 66); - g_assert(qdict_get_int(dict3, "e.1.2.a") == 0); - g_assert(qdict_get_int(dict3, "e.1.2.b") == 1); - g_assert(qdict_get_int(dict3, "f.c") == 2); - g_assert(qdict_get_int(dict3, "f.d") == 3); - g_assert(qdict_get_int(dict3, "g") == 4); + qdict_flatten(root); - g_assert(qdict_size(dict3) == 8); + g_assert(qdict_get_int(root, "e.0") == 42); + g_assert(qdict_get_int(root, "e.1.0") == 23); + g_assert(qdict_get_int(root, "e.1.1") == 66); + g_assert(qdict_get_int(root, "e.1.2.a") == 0); + g_assert(qdict_get_int(root, "e.1.2.b") == 1); + g_assert(qdict_get_int(root, "f.c") == 2); + g_assert(qdict_get_int(root, "f.d") == 3); + g_assert(qdict_get_int(root, "g") == 4); - qobject_unref(dict3); + g_assert(qdict_size(root) == 8); + + qobject_unref(root); } static void qdict_array_split_test(void) -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/check-block-qdict.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-block-qdict.c +++ b/tests/check-block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) QList *e = qlist_new(); QDict *e_1_2 = qdict_new(); QDict *f = qdict_new(); + QList *y = qlist_new(); + QDict *z = qdict_new(); QDict *root = qdict_new(); /* @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) * "c": 2, * "d": 3, * }, - * "g": 4 + * "g": 4, + * "y": [{}], + * "z": {"a": []} * } * * to @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) * "f.d": 3, * "g": 4 * } + * + * Note that "y" and "z" get eaten. */ qdict_put_int(e_1_2, "a", 0); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) qdict_put_int(f, "c", 2); qdict_put_int(f, "d", 3); + qlist_append(y, qdict_new()); + + qdict_put(z, "a", qlist_new()); + qdict_put(root, "e", e); qdict_put(root, "f", f); qdict_put_int(root, "g", 4); + qdict_put(root, "y", y); + qdict_put(root, "z", z); qdict_flatten(root); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> -blockdev and blockdev-add silently ignore empty objects and arrays in their argument. That's because qmp_blockdev_add() converts the argument to a flat QDict, and qdict_flatten() eats empty QDict and QList members. For instance, we ignore an empty BlockdevOptions member @cache. No real harm, as absent means the same as empty there. Thus, the flaw puts an artificial restriction on the QAPI schema: we can't have potentially empty objects and arrays within BlockdevOptions, except when they're optional and "empty" has the same meaning as "absent". Our QAPI schema satisfies this restriction (I checked), but it's a trap for the unwary, and a temptation to employ awkward workarounds for the wary. Let's get rid of it. Change qdict_flatten() and qdict_crumple() to treat empty dictionaries and lists exactly like scalars. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 54 +++++++++++++++++++++++++++++------------------ tests/check-block-qdict.c | 38 ++++++++++++++++++++++++++------- 2 files changed, 63 insertions(+), 29 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) { QObject *value; const QListEntry *entry; + QDict *dict_val; + QList *list_val; char *new_key; int i; @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) for (i = 0; entry; entry = qlist_next(entry), i++) { value = qlist_entry_obj(entry); + dict_val = qobject_to(QDict, value); + list_val = qobject_to(QList, value); new_key = g_strdup_printf("%s.%i", prefix, i); /* * Flatten non-empty QDict and QList recursively into @target, * copy other objects to @target */ - if (qobject_type(value) == QTYPE_QDICT) { - qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); - } else if (qobject_type(value) == QTYPE_QLIST) { - qdict_flatten_qlist(qobject_to(QList, value), target, new_key); + if (dict_val && qdict_size(dict_val)) { + qdict_flatten_qdict(dict_val, target, new_key); + } else if (list_val && !qlist_empty(list_val)) { + qdict_flatten_qlist(list_val, target, new_key); } else { qdict_put_obj(target, new_key, qobject_ref(value)); } @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) { QObject *value; const QDictEntry *entry, *next; + QDict *dict_val; + QList *list_val; char *new_key; entry = qdict_first(qdict); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) while (entry != NULL) { next = qdict_next(qdict, entry); value = qdict_entry_value(entry); + dict_val = qobject_to(QDict, value); + list_val = qobject_to(QList, value); new_key = NULL; if (prefix) { @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) * Flatten non-empty QDict and QList recursively into @target, * copy other objects to @target */ - if (qobject_type(value) == QTYPE_QDICT) { - qdict_flatten_qdict(qobject_to(QDict, value), target, + if (dict_val && qdict_size(dict_val)) { + qdict_flatten_qdict(dict_val, target, new_key ? new_key : entry->key); qdict_del(qdict, entry->key); - } else if (qobject_type(value) == QTYPE_QLIST) { - qdict_flatten_qlist(qobject_to(QList, value), target, + } else if (list_val && !qlist_empty(list_val)) { + qdict_flatten_qlist(list_val, target, new_key ? new_key : entry->key); qdict_del(qdict, entry->key); } else if (target != qdict) { @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) } /** - * qdict_flatten(): For each nested QDict with key x, all fields with key y - * are moved to this QDict and their key is renamed to "x.y". For each nested - * QList with key x, the field at index y is moved to this QDict with the key - * "x.y" (i.e., the reverse of what qdict_array_split() does). + * qdict_flatten(): For each nested non-empty QDict with key x, all + * fields with key y are moved to this QDict and their key is renamed + * to "x.y". For each nested non-empty QList with key x, the field at + * index y is moved to this QDict with the key "x.y" (i.e., the + * reverse of what qdict_array_split() does). * This operation is applied recursively for nested QDicts and QLists. */ void qdict_flatten(QDict *qdict) @@ -XXX,XX +XXX,XX @@ static int qdict_is_list(QDict *maybe_list, Error **errp) * @src: the original flat dictionary (only scalar values) to crumple * * Takes a flat dictionary whose keys use '.' separator to indicate - * nesting, and values are scalars, and crumples it into a nested - * structure. + * nesting, and values are scalars, empty dictionaries or empty lists, + * and crumples it into a nested structure. * * To include a literal '.' in a key name, it must be escaped as '..' * @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) { const QDictEntry *ent; QDict *two_level, *multi_level = NULL, *child_dict; + QDict *dict_val; + QList *list_val; QObject *dst = NULL, *child; size_t i; char *prefix = NULL; @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) /* Step 1: split our totally flat dict into a two level dict */ for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) { - if (qobject_type(ent->value) == QTYPE_QDICT || - qobject_type(ent->value) == QTYPE_QLIST) { - error_setg(errp, "Value %s is not a scalar", - ent->key); + dict_val = qobject_to(QDict, ent->value); + list_val = qobject_to(QList, ent->value); + if ((dict_val && qdict_size(dict_val)) + || (list_val && !qlist_empty(list_val))) { + error_setg(errp, "Value %s is not flat", ent->key); goto error; } @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) multi_level = qdict_new(); for (ent = qdict_first(two_level); ent != NULL; ent = qdict_next(two_level, ent)) { - QDict *dict = qobject_to(QDict, ent->value); - if (dict) { - child = qdict_crumple(dict, errp); + dict_val = qobject_to(QDict, ent->value); + if (dict_val && qdict_size(dict_val)) { + child = qdict_crumple(dict_val, errp); if (!child) { goto error; } diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-block-qdict.c +++ b/tests/check-block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) * "e.1.2.b": 1, * "f.c": 2, * "f.d": 3, - * "g": 4 + * "g": 4, + * "y.0": {}, + * "z.a": [] * } - * - * Note that "y" and "z" get eaten. */ qdict_put_int(e_1_2, "a", 0); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) g_assert(qdict_get_int(root, "f.c") == 2); g_assert(qdict_get_int(root, "f.d") == 3); g_assert(qdict_get_int(root, "g") == 4); + g_assert(!qdict_size(qdict_get_qdict(root, "y.0"))); + g_assert(qlist_empty(qdict_get_qlist(root, "z.a"))); - g_assert(qdict_size(root) == 8); + g_assert(qdict_size(root) == 10); qobject_unref(root); } @@ -XXX,XX +XXX,XX @@ static void qdict_join_test(void) static void qdict_crumple_test_recursive(void) { QDict *src, *dst, *rule, *vnc, *acl, *listen; - QList *rules; + QDict *empty, *empty_dict, *empty_list_0; + QList *rules, *empty_list, *empty_dict_a; src = qdict_new(); qdict_put_str(src, "vnc.listen.addr", "127.0.0.1"); @@ -XXX,XX +XXX,XX @@ static void qdict_crumple_test_recursive(void) qdict_put_str(src, "vnc.acl.default", "deny"); qdict_put_str(src, "vnc.acl..name", "acl0"); qdict_put_str(src, "vnc.acl.rule..name", "acl0"); + qdict_put(src, "empty.dict.a", qlist_new()); + qdict_put(src, "empty.list.0", qdict_new()); dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); g_assert(dst); - g_assert_cmpint(qdict_size(dst), ==, 1); + g_assert_cmpint(qdict_size(dst), ==, 2); vnc = qdict_get_qdict(dst, "vnc"); g_assert(vnc); @@ -XXX,XX +XXX,XX @@ static void qdict_crumple_test_recursive(void) g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); + empty = qdict_get_qdict(dst, "empty"); + g_assert(empty); + g_assert_cmpint(qdict_size(empty), ==, 2); + empty_dict = qdict_get_qdict(empty, "dict"); + g_assert(empty_dict); + g_assert_cmpint(qdict_size(empty_dict), ==, 1); + empty_dict_a = qdict_get_qlist(empty_dict, "a"); + g_assert(empty_dict_a && qlist_empty(empty_dict_a)); + empty_list = qdict_get_qlist(empty, "list"); + g_assert(empty_list); + g_assert_cmpint(qlist_size(empty_list), ==, 1); + empty_list_0 = qobject_to(QDict, qlist_pop(empty_list)); + g_assert(empty_list_0); + g_assert_cmpint(qdict_size(empty_list_0), ==, 0); + qobject_unref(src); qobject_unref(dst); } @@ -XXX,XX +XXX,XX @@ static void qdict_rename_keys_test(void) static void qdict_crumple_test_bad_inputs(void) { - QDict *src; + QDict *src, *nested; Error *error = NULL; src = qdict_new(); @@ -XXX,XX +XXX,XX @@ static void qdict_crumple_test_bad_inputs(void) src = qdict_new(); /* The input should be flat, ie no dicts or lists */ - qdict_put(src, "rule.a", qdict_new()); + nested = qdict_new(); + qdict_put(nested, "x", qdict_new()); + qdict_put(src, "rule.a", nested); qdict_put_str(src, "rule.b", "allow"); g_assert(qdict_crumple(src, &error) == NULL); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Parameter auth-client-required lets you configure authentication methods. We tried to provide that in v2.9.0, but backed out due to interface design doubts (commit 464444fcc16). This commit is similar to what we backed out, but simpler: we use a list of enumeration values instead of a list of objects with a member of enumeration type. Let's review our reasons for backing out the first try, as stated in the commit message: * The implementation uses deprecated rados_conf_set() key "auth_supported". No biggie. Fixed: we use "auth-client-required". * The implementation makes -drive silently ignore invalid parameters "auth" and "auth-supported.*.X" where X isn't "auth". Fixable (in fact I'm going to fix similar bugs around parameter server), so again no biggie. That fix is commit 2836284db60. This commit doesn't bring the bugs back. * BlockdevOptionsRbd member @password-secret applies only to authentication method cephx. Should it be a variant member of RbdAuthMethod? We've had time to ponder, and we decided to stick to the way Ceph configuration works: the key configured separately, and silently ignored if the authentication method doesn't use it. * BlockdevOptionsRbd member @user could apply to both methods cephx and none, but I'm not sure it's actually used with none. If it isn't, should it be a variant member of RbdAuthMethod? Likewise. * The client offers a *set* of authentication methods, not a list. Should the methods be optional members of BlockdevOptionsRbd instead of members of list @auth-supported? The latter begs the question what multiple entries for the same method mean. Trivial question now that RbdAuthMethod contains nothing but @type, but less so when RbdAuthMethod acquires other members, such the ones discussed above. Again, we decided to stick to the way Ceph configuration works, except we make auth-client-required a list of enumeration values instead of a string containing keywords separated by delimiters. * How BlockdevOptionsRbd member @auth-supported interacts with settings from a configuration file specified with @conf is undocumented. I suspect it's untested, too. Not actually true, the documentation for @conf says "Values in the configuration file will be overridden by options specified via QAPI", and we've tested this. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 13 +++++++++++++ block/rbd.c | 42 ++++++++++++++++++++++++++++++++---------- 2 files changed, 45 insertions(+), 10 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ ## +# @RbdAuthMode: +# +# Since: 3.0 +## +{ 'enum': 'RbdAuthMode', + 'data': [ 'cephx', 'none' ] } + +## # @BlockdevOptionsRbd: # # @pool: Ceph pool name. @@ -XXX,XX +XXX,XX @@ # # @user: Ceph id name. # +# @auth-client-required: Acceptable authentication modes. +# This maps to Ceph configuration option +# "auth_client_required". (Since 3.0) +# # @server: Monitor host address and port. This maps # to the "mon_host" Ceph option. # @@ -XXX,XX +XXX,XX @@ '*conf': 'str', '*snapshot': 'str', '*user': 'str', + '*auth-client-required': ['RbdAuthMode'], '*server': ['InetSocketAddressBase'] } } ## diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp) static int qemu_rbd_set_auth(rados_t cluster, const char *secretid, + BlockdevOptionsRbd *opts, Error **errp) { - if (secretid == 0) { - return 0; - } + char *acr; + int r; + GString *accu; + RbdAuthModeList *auth; + + if (secretid) { + gchar *secret = qcrypto_secret_lookup_as_base64(secretid, + errp); + if (!secret) { + return -1; + } - gchar *secret = qcrypto_secret_lookup_as_base64(secretid, - errp); - if (!secret) { - return -1; + rados_conf_set(cluster, "key", secret); + g_free(secret); } - rados_conf_set(cluster, "key", secret); - g_free(secret); + if (opts->has_auth_client_required) { + accu = g_string_new(""); + for (auth = opts->auth_client_required; auth; auth = auth->next) { + if (accu->str[0]) { + g_string_append_c(accu, ';'); + } + g_string_append(accu, RbdAuthMode_str(auth->value)); + } + acr = g_string_free(accu, FALSE); + r = rados_conf_set(cluster, "auth_client_required", acr); + g_free(acr); + if (r < 0) { + error_setg_errno(errp, -r, + "Could not set 'auth_client_required'"); + return r; + } + } return 0; } @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, } } - if (qemu_rbd_set_auth(*cluster, secretid, errp) < 0) { + if (qemu_rbd_set_auth(*cluster, secretid, opts, errp) < 0) { r = -EIO; goto failed_shutdown; } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Legacy -drive supports "password-secret" parameter that isn't available with -blockdev / blockdev-add. That's because we backed out our first try to provide it there due to interface design doubts, in commit 577d8c9a811, v2.9.0. This is the second try. It brings back the parameter, except it's named "key-secret" now. Let's review our reasons for backing out the first try, as stated in the commit message: * BlockdevOptionsRbd member @password-secret isn't actually a password, it's a key generated by Ceph. Addressed by the rename. * We're not sure where member @password-secret belongs (see the previous commit). See previous commit. * How @password-secret interacts with settings from a configuration file specified with @conf is undocumented. Not actually true, the documentation for @conf says "Values in the configuration file will be overridden by options specified via QAPI", and we've tested this. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 6 ++++++ block/rbd.c | 41 +++++++++++++++++++++++++---------------- 2 files changed, 31 insertions(+), 16 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # This maps to Ceph configuration option # "auth_client_required". (Since 3.0) # +# @key-secret: ID of a QCryptoSecret object providing a key +# for cephx authentication. +# This maps to Ceph configuration option +# "key". (Since 3.0) +# # @server: Monitor host address and port. This maps # to the "mon_host" Ceph option. # @@ -XXX,XX +XXX,XX @@ '*snapshot': 'str', '*user': 'str', '*auth-client-required': ['RbdAuthMode'], + '*key-secret': 'str', '*server': ['InetSocketAddressBase'] } } ## diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp) } -static int qemu_rbd_set_auth(rados_t cluster, const char *secretid, - BlockdevOptionsRbd *opts, +static int qemu_rbd_set_auth(rados_t cluster, BlockdevOptionsRbd *opts, Error **errp) { - char *acr; + char *key, *acr; int r; GString *accu; RbdAuthModeList *auth; - if (secretid) { - gchar *secret = qcrypto_secret_lookup_as_base64(secretid, - errp); - if (!secret) { - return -1; + if (opts->key_secret) { + key = qcrypto_secret_lookup_as_base64(opts->key_secret, errp); + if (!key) { + return -EIO; + } + r = rados_conf_set(cluster, "key", key); + g_free(key); + if (r < 0) { + error_setg_errno(errp, -r, "Could not set 'key'"); + return r; } - - rados_conf_set(cluster, "key", secret); - g_free(secret); } if (opts->has_auth_client_required) { @@ -XXX,XX +XXX,XX @@ static QemuOptsList runtime_opts = { }, }; -/* FIXME Deprecate and remove keypairs or make it available in QMP. - * password_secret should eventually be configurable in opts->location. Support - * for it in .bdrv_open will make it work here as well. */ +/* FIXME Deprecate and remove keypairs or make it available in QMP. */ static int qemu_rbd_do_create(BlockdevCreateOptions *options, const char *keypairs, const char *password_secret, Error **errp) @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, Error *local_err = NULL; int r; + if (secretid) { + if (opts->key_secret) { + error_setg(errp, + "Legacy 'password-secret' clashes with 'key-secret'"); + return -EINVAL; + } + opts->key_secret = g_strdup(secretid); + opts->has_key_secret = true; + } + mon_host = qemu_rbd_mon_host(opts, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, } } - if (qemu_rbd_set_auth(*cluster, secretid, opts, errp) < 0) { - r = -EIO; + r = qemu_rbd_set_auth(*cluster, opts, errp); + if (r < 0) { goto failed_shutdown; } -- 2.13.6 The -drive options cyls, heads, secs and trans were deprecated in QEMU 2.10. It's time to remove them. hd-geo-test tested both the old version with geometry options in -drive and the new one with -device. Therefore the code using -drive doesn't have to be replaced there, we just need to remove the -drive test cases. This in turn allows some simplification of the code. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> --- include/sysemu/blockdev.h | 1 - blockdev.c | 75 +---------------------------------------------- hw/block/block.c | 14 --------- tests/hd-geo-test.c | 37 +++++------------------ hmp-commands.hx | 1 - qemu-doc.texi | 5 ---- qemu-options.hx | 7 +---- 7 files changed, 9 insertions(+), 131 deletions(-) diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/blockdev.h +++ b/include/sysemu/blockdev.h @@ -XXX,XX +XXX,XX @@ struct DriveInfo { int auto_del; /* see blockdev_mark_auto_del() */ bool is_default; /* Added by default_drive() ? */ int media_cd; - int cyls, heads, secs, trans; QemuOpts *opts; char *serial; QTAILQ_ENTRY(DriveInfo) next; diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ QemuOptsList qemu_legacy_drive_opts = { .type = QEMU_OPT_STRING, .help = "interface (ide, scsi, sd, mtd, floppy, pflash, virtio)", },{ - .name = "cyls", - .type = QEMU_OPT_NUMBER, - .help = "number of cylinders (ide disk geometry)", - },{ - .name = "heads", - .type = QEMU_OPT_NUMBER, - .help = "number of heads (ide disk geometry)", - },{ - .name = "secs", - .type = QEMU_OPT_NUMBER, - .help = "number of sectors (ide disk geometry)", - },{ - .name = "trans", - .type = QEMU_OPT_STRING, - .help = "chs translation (auto, lba, none)", - },{ .name = "addr", .type = QEMU_OPT_STRING, .help = "pci address (virtio only)", @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) QemuOpts *legacy_opts; DriveMediaType media = MEDIA_DISK; BlockInterfaceType type; - int cyls, heads, secs, translation; int max_devs, bus_id, unit_id, index; const char *devaddr; const char *werror, *rerror; @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) Error *local_err = NULL; int i; const char *deprecated[] = { - "serial", "trans", "secs", "heads", "cyls", "addr" + "serial", "addr" }; /* Change legacy command line options into QMP ones */ @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) type = block_default_type; } - /* Geometry */ - cyls = qemu_opt_get_number(legacy_opts, "cyls", 0); - heads = qemu_opt_get_number(legacy_opts, "heads", 0); - secs = qemu_opt_get_number(legacy_opts, "secs", 0); - - if (cyls || heads || secs) { - if (cyls < 1) { - error_report("invalid physical cyls number"); - goto fail; - } - if (heads < 1) { - error_report("invalid physical heads number"); - goto fail; - } - if (secs < 1) { - error_report("invalid physical secs number"); - goto fail; - } - } - - translation = BIOS_ATA_TRANSLATION_AUTO; - value = qemu_opt_get(legacy_opts, "trans"); - if (value != NULL) { - if (!cyls) { - error_report("'%s' trans must be used with cyls, heads and secs", - value); - goto fail; - } - if (!strcmp(value, "none")) { - translation = BIOS_ATA_TRANSLATION_NONE; - } else if (!strcmp(value, "lba")) { - translation = BIOS_ATA_TRANSLATION_LBA; - } else if (!strcmp(value, "large")) { - translation = BIOS_ATA_TRANSLATION_LARGE; - } else if (!strcmp(value, "rechs")) { - translation = BIOS_ATA_TRANSLATION_RECHS; - } else if (!strcmp(value, "auto")) { - translation = BIOS_ATA_TRANSLATION_AUTO; - } else { - error_report("'%s' invalid translation type", value); - goto fail; - } - } - - if (media == MEDIA_CDROM) { - if (cyls || secs || heads) { - error_report("CHS can't be set with media=cdrom"); - goto fail; - } - } - /* Device address specified by bus/unit or index. * If none was specified, try to find the first free one. */ bus_id = qemu_opt_get_number(legacy_opts, "bus", 0); @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) dinfo = g_malloc0(sizeof(*dinfo)); dinfo->opts = all_opts; - dinfo->cyls = cyls; - dinfo->heads = heads; - dinfo->secs = secs; - dinfo->trans = translation; - dinfo->type = type; dinfo->bus = bus_id; dinfo->unit = unit_id; diff --git a/hw/block/block.c b/hw/block/block.c index XXXXXXX..XXXXXXX 100644 --- a/hw/block/block.c +++ b/hw/block/block.c @@ -XXX,XX +XXX,XX @@ bool blkconf_geometry(BlockConf *conf, int *ptrans, unsigned cyls_max, unsigned heads_max, unsigned secs_max, Error **errp) { - DriveInfo *dinfo; - - if (!conf->cyls && !conf->heads && !conf->secs) { - /* try to fall back to value set with legacy -drive cyls=... */ - dinfo = blk_legacy_dinfo(conf->blk); - if (dinfo) { - conf->cyls = dinfo->cyls; - conf->heads = dinfo->heads; - conf->secs = dinfo->secs; - if (ptrans) { - *ptrans = dinfo->trans; - } - } - } if (!conf->cyls && !conf->heads && !conf->secs) { hd_geometry_guess(conf->blk, &conf->cyls, &conf->heads, &conf->secs, diff --git a/tests/hd-geo-test.c b/tests/hd-geo-test.c index XXXXXXX..XXXXXXX 100644 --- a/tests/hd-geo-test.c +++ b/tests/hd-geo-test.c @@ -XXX,XX +XXX,XX @@ static void setup_mbr(int img_idx, MBRcontents mbr) static int setup_ide(int argc, char *argv[], int argv_sz, int ide_idx, const char *dev, int img_idx, - MBRcontents mbr, const char *opts) + MBRcontents mbr) { char *s1, *s2, *s3; @@ -XXX,XX +XXX,XX @@ static int setup_ide(int argc, char *argv[], int argv_sz, s3 = g_strdup(",media=cdrom"); } argc = append_arg(argc, argv, argv_sz, - g_strdup_printf("%s%s%s%s", s1, s2, s3, opts)); + g_strdup_printf("%s%s%s", s1, s2, s3)); g_free(s1); g_free(s2); g_free(s3); @@ -XXX,XX +XXX,XX @@ static void test_ide_mbr(bool use_device, MBRcontents mbr) for (i = 0; i < backend_last; i++) { cur_ide[i] = &hd_chst[i][mbr]; dev = use_device ? (is_hd(cur_ide[i]) ? "ide-hd" : "ide-cd") : NULL; - argc = setup_ide(argc, argv, ARGV_SIZE, i, dev, i, mbr, ""); + argc = setup_ide(argc, argv, ARGV_SIZE, i, dev, i, mbr); } args = g_strjoinv(" ", argv); qtest_start(args); @@ -XXX,XX +XXX,XX @@ static void test_ide_drive_user(const char *dev, bool trans) const CHST expected_chst = { secs / (4 * 32) , 4, 32, trans }; argc = setup_common(argv, ARGV_SIZE); - opts = g_strdup_printf("%s,%s%scyls=%d,heads=%d,secs=%d", - dev ?: "", - trans && dev ? "bios-chs-" : "", - trans ? "trans=lba," : "", + opts = g_strdup_printf("%s,%scyls=%d,heads=%d,secs=%d", + dev, trans ? "bios-chs-trans=lba," : "", expected_chst.cyls, expected_chst.heads, expected_chst.secs); cur_ide[0] = &expected_chst; - argc = setup_ide(argc, argv, ARGV_SIZE, - 0, dev ? opts : NULL, backend_small, mbr_chs, - dev ? "" : opts); + argc = setup_ide(argc, argv, ARGV_SIZE, 0, opts, backend_small, mbr_chs); g_free(opts); args = g_strjoinv(" ", argv); qtest_start(args); @@ -XXX,XX +XXX,XX @@ static void test_ide_drive_user(const char *dev, bool trans) } /* - * Test case: IDE device (if=ide) with explicit CHS - */ -static void test_ide_drive_user_chs(void) -{ - test_ide_drive_user(NULL, false); -} - -/* - * Test case: IDE device (if=ide) with explicit CHS and translation - */ -static void test_ide_drive_user_chst(void) -{ - test_ide_drive_user(NULL, true); -} - -/* * Test case: IDE device (if=none) with explicit CHS */ static void test_ide_device_user_chs(void) @@ -XXX,XX +XXX,XX @@ static void test_ide_drive_cd_0(void) for (i = 0; i <= backend_empty; i++) { ide_idx = backend_empty - i; cur_ide[ide_idx] = &hd_chst[i][mbr_blank]; - argc = setup_ide(argc, argv, ARGV_SIZE, - ide_idx, NULL, i, mbr_blank, ""); + argc = setup_ide(argc, argv, ARGV_SIZE, ide_idx, NULL, i, mbr_blank); } args = g_strjoinv(" ", argv); qtest_start(args); @@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv) qtest_add_func("hd-geo/ide/drive/mbr/blank", test_ide_drive_mbr_blank); qtest_add_func("hd-geo/ide/drive/mbr/lba", test_ide_drive_mbr_lba); qtest_add_func("hd-geo/ide/drive/mbr/chs", test_ide_drive_mbr_chs); - qtest_add_func("hd-geo/ide/drive/user/chs", test_ide_drive_user_chs); - qtest_add_func("hd-geo/ide/drive/user/chst", test_ide_drive_user_chst); qtest_add_func("hd-geo/ide/drive/cd_0", test_ide_drive_cd_0); qtest_add_func("hd-geo/ide/device/mbr/blank", test_ide_device_mbr_blank); qtest_add_func("hd-geo/ide/device/mbr/lba", test_ide_device_mbr_lba); diff --git a/hmp-commands.hx b/hmp-commands.hx index XXXXXXX..XXXXXXX 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -XXX,XX +XXX,XX @@ ETEXI .params = "[-n] [[<domain>:]<bus>:]<slot>\n" "[file=file][,if=type][,bus=n]\n" "[,unit=m][,media=d][,index=i]\n" - "[,cyls=c,heads=h,secs=s[,trans=t]]\n" "[,snapshot=on|off][,cache=on|off]\n" "[,readonly=on|off][,copy-on-read=on|off]", .help = "add drive to PCI storage controller", diff --git a/qemu-doc.texi b/qemu-doc.texi index XXXXXXX..XXXXXXX 100644 --- a/qemu-doc.texi +++ b/qemu-doc.texi @@ -XXX,XX +XXX,XX @@ with ``-device ...,netdev=x''), or ``-nic user,smb=/some/dir'' (for embedded NICs). The new syntax allows different settings to be provided per NIC. -@subsection -drive cyls=...,heads=...,secs=...,trans=... (since 2.10.0) - -The drive geometry arguments are replaced by the the geometry arguments -that can be specified with the ``-device'' parameter. - @subsection -drive serial=... (since 2.10.0) The drive serial argument is replaced by the the serial argument diff --git a/qemu-options.hx b/qemu-options.hx index XXXXXXX..XXXXXXX 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -XXX,XX +XXX,XX @@ ETEXI DEF("drive", HAS_ARG, QEMU_OPTION_drive, "-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n" - " [,cyls=c,heads=h,secs=s[,trans=t]][,snapshot=on|off]\n" " [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n" - " [,serial=s][,addr=A][,rerror=ignore|stop|report]\n" + " [,snapshot=on|off][,serial=s][,addr=A][,rerror=ignore|stop|report]\n" " [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n" " [,readonly=on|off][,copy-on-read=on|off]\n" " [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n" @@ -XXX,XX +XXX,XX @@ This option defines where is connected the drive by using an index in the list of available connectors of a given interface type. @item media=@var{media} This option defines the type of the media: disk or cdrom. -@item cyls=@var{c},heads=@var{h},secs=@var{s}[,trans=@var{t}] -Force disk physical geometry and the optional BIOS translation (trans=none or -lba). These parameters are deprecated, use the corresponding parameters -of @code{-device} instead. @item snapshot=@var{snapshot} @var{snapshot} is "on" or "off" and controls snapshot mode for the given drive (see @option{-snapshot}). -- 2.13.6 The -drive option addr was deprecated in QEMU 2.10. It's time to remove it. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> --- include/sysemu/blockdev.h | 1 - blockdev.c | 17 +---------------- device-hotplug.c | 4 ---- qemu-doc.texi | 5 ----- qemu-options.hx | 5 +---- 5 files changed, 2 insertions(+), 30 deletions(-) diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/blockdev.h +++ b/include/sysemu/blockdev.h @@ -XXX,XX +XXX,XX @@ typedef enum { } BlockInterfaceType; struct DriveInfo { - const char *devaddr; BlockInterfaceType type; int bus; int unit; diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ QemuOptsList qemu_legacy_drive_opts = { .type = QEMU_OPT_STRING, .help = "interface (ide, scsi, sd, mtd, floppy, pflash, virtio)", },{ - .name = "addr", - .type = QEMU_OPT_STRING, - .help = "pci address (virtio only)", - },{ .name = "serial", .type = QEMU_OPT_STRING, .help = "disk serial number", @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) DriveMediaType media = MEDIA_DISK; BlockInterfaceType type; int max_devs, bus_id, unit_id, index; - const char *devaddr; const char *werror, *rerror; bool read_only = false; bool copy_on_read; @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) Error *local_err = NULL; int i; const char *deprecated[] = { - "serial", "addr" + "serial" }; /* Change legacy command line options into QMP ones */ @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) } /* Add virtio block device */ - devaddr = qemu_opt_get(legacy_opts, "addr"); - if (devaddr && type != IF_VIRTIO) { - error_report("addr is not supported by this bus type"); - goto fail; - } - if (type == IF_VIRTIO) { QemuOpts *devopts; devopts = qemu_opts_create(qemu_find_opts("device"), NULL, 0, @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) } qemu_opt_set(devopts, "drive", qdict_get_str(bs_opts, "id"), &error_abort); - if (devaddr) { - qemu_opt_set(devopts, "addr", devaddr, &error_abort); - } } filename = qemu_opt_get(legacy_opts, "file"); @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) dinfo->type = type; dinfo->bus = bus_id; dinfo->unit = unit_id; - dinfo->devaddr = devaddr; dinfo->serial = g_strdup(serial); blk_set_legacy_dinfo(blk, dinfo); diff --git a/device-hotplug.c b/device-hotplug.c index XXXXXXX..XXXXXXX 100644 --- a/device-hotplug.c +++ b/device-hotplug.c @@ -XXX,XX +XXX,XX @@ void hmp_drive_add(Monitor *mon, const QDict *qdict) if (!dinfo) { goto err; } - if (dinfo->devaddr) { - monitor_printf(mon, "Parameter addr not supported\n"); - goto err; - } switch (dinfo->type) { case IF_NONE: diff --git a/qemu-doc.texi b/qemu-doc.texi index XXXXXXX..XXXXXXX 100644 --- a/qemu-doc.texi +++ b/qemu-doc.texi @@ -XXX,XX +XXX,XX @@ provided per NIC. The drive serial argument is replaced by the the serial argument that can be specified with the ``-device'' parameter. -@subsection -drive addr=... (since 2.10.0) - -The drive addr argument is replaced by the the addr argument -that can be specified with the ``-device'' parameter. - @subsection -usbdevice (since 2.10.0) The ``-usbdevice DEV'' argument is now a synonym for setting diff --git a/qemu-options.hx b/qemu-options.hx index XXXXXXX..XXXXXXX 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -XXX,XX +XXX,XX @@ ETEXI DEF("drive", HAS_ARG, QEMU_OPTION_drive, "-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n" " [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n" - " [,snapshot=on|off][,serial=s][,addr=A][,rerror=ignore|stop|report]\n" + " [,snapshot=on|off][,serial=s][,rerror=ignore|stop|report]\n" " [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n" " [,readonly=on|off][,copy-on-read=on|off]\n" " [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n" @@ -XXX,XX +XXX,XX @@ an untrusted format header. This option specifies the serial number to assign to the device. This parameter is deprecated, use the corresponding parameter of @code{-device} instead. -@item addr=@var{addr} -Specify the controller's PCI address (if=virtio only). This parameter is -deprecated, use the corresponding parameter of @code{-device} instead. @item werror=@var{action},rerror=@var{action} Specify which @var{action} to take on write and read errors. Valid actions are: "ignore" (ignore the error and try to continue), "stop" (pause QEMU), -- 2.13.6 The -drive option serial was deprecated in QEMU 2.10. It's time to remove it. Tests need to be updated to set the serial number with -global instead of using the -drive option. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> --- include/hw/block/block.h | 1 - include/sysemu/blockdev.h | 1 - block/block-backend.c | 1 - blockdev.c | 10 ---------- hw/block/block.c | 13 ------------- hw/block/nvme.c | 1 - hw/block/virtio-blk.c | 1 - hw/ide/qdev.c | 1 - hw/scsi/scsi-disk.c | 1 - hw/usb/dev-storage.c | 1 - tests/ahci-test.c | 6 +++--- tests/ide-test.c | 8 ++++---- qemu-doc.texi | 5 ----- qemu-options.hx | 6 +----- 14 files changed, 8 insertions(+), 48 deletions(-) diff --git a/include/hw/block/block.h b/include/hw/block/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/block/block.h +++ b/include/hw/block/block.h @@ -XXX,XX +XXX,XX @@ static inline unsigned int get_physical_block_exp(BlockConf *conf) /* Configuration helpers */ -void blkconf_serial(BlockConf *conf, char **serial); bool blkconf_geometry(BlockConf *conf, int *trans, unsigned cyls_max, unsigned heads_max, unsigned secs_max, Error **errp); diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/blockdev.h +++ b/include/sysemu/blockdev.h @@ -XXX,XX +XXX,XX @@ struct DriveInfo { bool is_default; /* Added by default_drive() ? */ int media_cd; QemuOpts *opts; - char *serial; QTAILQ_ENTRY(DriveInfo) next; }; diff --git a/block/block-backend.c b/block/block-backend.c index XXXXXXX..XXXXXXX 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -XXX,XX +XXX,XX @@ static void drive_info_del(DriveInfo *dinfo) return; } qemu_opts_del(dinfo->opts); - g_free(dinfo->serial); g_free(dinfo); } diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ QemuOptsList qemu_legacy_drive_opts = { .type = QEMU_OPT_STRING, .help = "interface (ide, scsi, sd, mtd, floppy, pflash, virtio)", },{ - .name = "serial", - .type = QEMU_OPT_STRING, - .help = "disk serial number", - },{ .name = "file", .type = QEMU_OPT_STRING, .help = "file name", @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) const char *werror, *rerror; bool read_only = false; bool copy_on_read; - const char *serial; const char *filename; Error *local_err = NULL; int i; const char *deprecated[] = { - "serial" }; /* Change legacy command line options into QMP ones */ @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) goto fail; } - /* Serial number */ - serial = qemu_opt_get(legacy_opts, "serial"); - /* no id supplied -> create one */ if (qemu_opts_id(all_opts) == NULL) { char *new_id; @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) dinfo->type = type; dinfo->bus = bus_id; dinfo->unit = unit_id; - dinfo->serial = g_strdup(serial); blk_set_legacy_dinfo(blk, dinfo); diff --git a/hw/block/block.c b/hw/block/block.c index XXXXXXX..XXXXXXX 100644 --- a/hw/block/block.c +++ b/hw/block/block.c @@ -XXX,XX +XXX,XX @@ #include "qapi/qapi-types-block.h" #include "qemu/error-report.h" -void blkconf_serial(BlockConf *conf, char **serial) -{ - DriveInfo *dinfo; - - if (!*serial) { - /* try to fall back to value set with legacy -drive serial=... */ - dinfo = blk_legacy_dinfo(conf->blk); - if (dinfo) { - *serial = g_strdup(dinfo->serial); - } - } -} - void blkconf_blocksizes(BlockConf *conf) { BlockBackend *blk = conf->blk; diff --git a/hw/block/nvme.c b/hw/block/nvme.c index XXXXXXX..XXXXXXX 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -XXX,XX +XXX,XX @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp) return; } - blkconf_serial(&n->conf, &n->serial); if (!n->serial) { error_setg(errp, "serial property not set"); return; diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index XXXXXXX..XXXXXXX 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -XXX,XX +XXX,XX @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp) return; } - blkconf_serial(&conf->conf, &conf->serial); if (!blkconf_apply_backend_options(&conf->conf, blk_is_read_only(conf->conf.blk), true, errp)) { diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c index XXXXXXX..XXXXXXX 100644 --- a/hw/ide/qdev.c +++ b/hw/ide/qdev.c @@ -XXX,XX +XXX,XX @@ static void ide_dev_initfn(IDEDevice *dev, IDEDriveKind kind, Error **errp) return; } - blkconf_serial(&dev->conf, &dev->serial); if (kind != IDE_CD) { if (!blkconf_geometry(&dev->conf, &dev->chs_trans, 65535, 16, 255, errp)) { diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index XXXXXXX..XXXXXXX 100644 --- a/hw/scsi/scsi-disk.c +++ b/hw/scsi/scsi-disk.c @@ -XXX,XX +XXX,XX @@ static void scsi_realize(SCSIDevice *dev, Error **errp) return; } - blkconf_serial(&s->qdev.conf, &s->serial); blkconf_blocksizes(&s->qdev.conf); if (s->qdev.conf.logical_block_size > diff --git a/hw/usb/dev-storage.c b/hw/usb/dev-storage.c index XXXXXXX..XXXXXXX 100644 --- a/hw/usb/dev-storage.c +++ b/hw/usb/dev-storage.c @@ -XXX,XX +XXX,XX @@ static void usb_msd_storage_realize(USBDevice *dev, Error **errp) return; } - blkconf_serial(&s->conf, &dev->serial); blkconf_blocksizes(&s->conf); if (!blkconf_apply_backend_options(&s->conf, blk_is_read_only(blk), true, errp)) { diff --git a/tests/ahci-test.c b/tests/ahci-test.c index XXXXXXX..XXXXXXX 100644 --- a/tests/ahci-test.c +++ b/tests/ahci-test.c @@ -XXX,XX +XXX,XX @@ static AHCIQState *ahci_boot(const char *cli, ...) s = ahci_vboot(cli, ap); va_end(ap); } else { - cli = "-drive if=none,id=drive0,file=%s,cache=writeback,serial=%s" - ",format=%s" + cli = "-drive if=none,id=drive0,file=%s,cache=writeback,format=%s" " -M q35 " "-device ide-hd,drive=drive0 " + "-global ide-hd.serial=%s " "-global ide-hd.ver=%s"; - s = ahci_boot(cli, tmp_path, "testdisk", imgfmt, "version"); + s = ahci_boot(cli, tmp_path, imgfmt, "testdisk", "version"); } return s; diff --git a/tests/ide-test.c b/tests/ide-test.c index XXXXXXX..XXXXXXX 100644 --- a/tests/ide-test.c +++ b/tests/ide-test.c @@ -XXX,XX +XXX,XX @@ static void test_bmdma_no_busmaster(void) static void test_bmdma_setup(void) { ide_test_start( - "-drive file=%s,if=ide,serial=%s,cache=writeback,format=raw " - "-global ide-hd.ver=%s", + "-drive file=%s,if=ide,cache=writeback,format=raw " + "-global ide-hd.serial=%s -global ide-hd.ver=%s", tmp_path, "testdisk", "version"); qtest_irq_intercept_in(global_qtest, "ioapic"); } @@ -XXX,XX +XXX,XX @@ static void test_identify(void) int ret; ide_test_start( - "-drive file=%s,if=ide,serial=%s,cache=writeback,format=raw " - "-global ide-hd.ver=%s", + "-drive file=%s,if=ide,cache=writeback,format=raw " + "-global ide-hd.serial=%s -global ide-hd.ver=%s", tmp_path, "testdisk", "version"); dev = get_pci_device(&bmdma_bar, &ide_bar); diff --git a/qemu-doc.texi b/qemu-doc.texi index XXXXXXX..XXXXXXX 100644 --- a/qemu-doc.texi +++ b/qemu-doc.texi @@ -XXX,XX +XXX,XX @@ with ``-device ...,netdev=x''), or ``-nic user,smb=/some/dir'' (for embedded NICs). The new syntax allows different settings to be provided per NIC. -@subsection -drive serial=... (since 2.10.0) - -The drive serial argument is replaced by the the serial argument -that can be specified with the ``-device'' parameter. - @subsection -usbdevice (since 2.10.0) The ``-usbdevice DEV'' argument is now a synonym for setting diff --git a/qemu-options.hx b/qemu-options.hx index XXXXXXX..XXXXXXX 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -XXX,XX +XXX,XX @@ ETEXI DEF("drive", HAS_ARG, QEMU_OPTION_drive, "-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n" " [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n" - " [,snapshot=on|off][,serial=s][,rerror=ignore|stop|report]\n" + " [,snapshot=on|off][,rerror=ignore|stop|report]\n" " [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n" " [,readonly=on|off][,copy-on-read=on|off]\n" " [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n" @@ -XXX,XX +XXX,XX @@ The default mode is @option{cache=writeback}. Specify which disk @var{format} will be used rather than detecting the format. Can be used to specify format=raw to avoid interpreting an untrusted format header. -@item serial=@var{serial} -This option specifies the serial number to assign to the device. This -parameter is deprecated, use the corresponding parameter of @code{-device} -instead. @item werror=@var{action},rerror=@var{action} Specify which @var{action} to take on write and read errors. Valid actions are: "ignore" (ignore the error and try to continue), "stop" (pause QEMU), -- 2.13.6 We removed all options from the 'deprecated' array, so the code is dead and can be removed as well. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> --- blockdev.c | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) const char *filename; Error *local_err = NULL; int i; - const char *deprecated[] = { - }; /* Change legacy command line options into QMP ones */ static const struct { @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) goto fail; } - /* Other deprecated options */ - if (!qtest_enabled()) { - for (i = 0; i < ARRAY_SIZE(deprecated); i++) { - if (qemu_opt_get(legacy_opts, deprecated[i]) != NULL) { - error_report("'%s' is deprecated, please use the corresponding " - "option of '-device' instead", deprecated[i]); - } - } - } - /* Media type */ value = qemu_opt_get(legacy_opts, "media"); if (value) { -- 2.13.6 The following changes since commit 005ad32358f12fe9313a4a01918a55e60d4f39e5: Merge tag 'pull-tpm-2023-09-12-3' of https://github.com/stefanberger/qemu-tpm into staging (2023-09-13 13:41:57 -0400) are available in the Git repository at: https://repo.or.cz/qemu/kevin.git tags/for-upstream for you to fetch changes up to 5d96864b73225ee61b0dad7e928f0cddf14270fc: block-coroutine-wrapper: use qemu_get_current_aio_context() (2023-09-15 15:49:14 +0200) ---------------------------------------------------------------- Block layer patches - Graph locking part 4 (node management) - qemu-img map: report compressed data blocks - block-backend: process I/O in the current AioContext ---------------------------------------------------------------- Andrey Drobyshev via (2): block: add BDRV_BLOCK_COMPRESSED flag for bdrv_block_status() qemu-img: map: report compressed data blocks Kevin Wolf (21): block: Remove unused BlockReopenQueueEntry.perms_checked preallocate: Factor out preallocate_truncate_to_real_size() preallocate: Don't poll during permission updates block: Take AioContext lock for bdrv_append() more consistently block: Introduce bdrv_schedule_unref() block-coroutine-wrapper: Add no_co_wrapper_bdrv_wrlock functions block-coroutine-wrapper: Allow arbitrary parameter names block: Mark bdrv_replace_child_noperm() GRAPH_WRLOCK block: Mark bdrv_replace_child_tran() GRAPH_WRLOCK block: Mark bdrv_attach_child_common() GRAPH_WRLOCK block: Call transaction callbacks with lock held block: Mark bdrv_attach_child() GRAPH_WRLOCK block: Mark bdrv_parent_perms_conflict() and callers GRAPH_RDLOCK block: Mark bdrv_get_cumulative_perm() and callers GRAPH_RDLOCK block: Mark bdrv_child_perm() GRAPH_RDLOCK block: Mark bdrv_parent_cb_change_media() GRAPH_RDLOCK block: Take graph rdlock in bdrv_drop_intermediate() block: Take graph rdlock in bdrv_change_aio_context() block: Mark bdrv_root_unref_child() GRAPH_WRLOCK block: Mark bdrv_unref_child() GRAPH_WRLOCK block: Mark bdrv_add/del_child() and caller GRAPH_WRLOCK Stefan Hajnoczi (5): block: remove AIOCBInfo->get_aio_context() test-bdrv-drain: avoid race with BH in IOThread drain test block-backend: process I/O in the current AioContext block-backend: process zoned requests in the current AioContext block-coroutine-wrapper: use qemu_get_current_aio_context() qapi/block-core.json | 6 +- include/block/aio.h | 1 - include/block/block-common.h | 7 + include/block/block-global-state.h | 32 +- include/block/block-io.h | 1 - include/block/block_int-common.h | 34 +- include/block/block_int-global-state.h | 14 +- include/sysemu/block-backend-global-state.h | 4 +- block.c | 348 +++++++--- block/blklogwrites.c | 4 + block/blkverify.c | 2 + block/block-backend.c | 64 +- block/copy-before-write.c | 10 +- block/crypto.c | 6 +- block/graph-lock.c | 26 +- block/io.c | 23 +- block/mirror.c | 8 + block/preallocate.c | 133 ++-- block/qcow.c | 5 +- block/qcow2.c | 7 +- block/quorum.c | 23 +- block/replication.c | 9 + block/snapshot.c | 2 + block/stream.c | 20 +- block/vmdk.c | 15 + blockdev.c | 23 +- blockjob.c | 2 + hw/nvme/ctrl.c | 7 - qemu-img.c | 8 +- softmmu/dma-helpers.c | 8 - tests/unit/test-bdrv-drain.c | 31 +- tests/unit/test-bdrv-graph-mod.c | 20 + tests/unit/test-block-iothread.c | 3 + util/thread-pool.c | 8 - scripts/block-coroutine-wrapper.py | 24 +- tests/qemu-iotests/051.pc.out | 6 +- tests/qemu-iotests/122.out | 84 +-- tests/qemu-iotests/146.out | 780 +++++++++++------------ tests/qemu-iotests/154.out | 194 +++--- tests/qemu-iotests/179.out | 178 +++--- tests/qemu-iotests/209.out | 4 +- tests/qemu-iotests/221.out | 16 +- tests/qemu-iotests/223.out | 60 +- tests/qemu-iotests/241.out | 10 +- tests/qemu-iotests/244.out | 24 +- tests/qemu-iotests/252.out | 10 +- tests/qemu-iotests/253.out | 20 +- tests/qemu-iotests/274.out | 48 +- tests/qemu-iotests/tests/nbd-qemu-allocation.out | 16 +- tests/qemu-iotests/tests/qemu-img-bitmaps.out | 24 +- 50 files changed, 1376 insertions(+), 1036 deletions(-) This field has been unused since commit 72373e40fbc ('block: bdrv_reopen_multiple: refresh permissions on updated graph'). Remove it. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-2-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static int bdrv_fill_options(QDict **options, const char *filename, typedef struct BlockReopenQueueEntry { bool prepared; - bool perms_checked; BDRVReopenState state; QTAILQ_ENTRY(BlockReopenQueueEntry) entry; } BlockReopenQueueEntry; -- 2.41.0 It's essentially the same code in preallocate_check_perm() and preallocate_close(), except that the latter ignores errors. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-3-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/preallocate.c | 48 +++++++++++++++++++++------------------------ 1 file changed, 22 insertions(+), 26 deletions(-) diff --git a/block/preallocate.c b/block/preallocate.c index XXXXXXX..XXXXXXX 100644 --- a/block/preallocate.c +++ b/block/preallocate.c @@ -XXX,XX +XXX,XX @@ static int preallocate_open(BlockDriverState *bs, QDict *options, int flags, return 0; } -static void preallocate_close(BlockDriverState *bs) +static int preallocate_truncate_to_real_size(BlockDriverState *bs, Error **errp) { - int ret; BDRVPreallocateState *s = bs->opaque; - - if (s->data_end < 0) { - return; - } + int ret; if (s->file_end < 0) { s->file_end = bdrv_getlength(bs->file->bs); if (s->file_end < 0) { - return; + error_setg_errno(errp, -s->file_end, "Failed to get file length"); + return s->file_end; } } if (s->data_end < s->file_end) { ret = bdrv_truncate(bs->file, s->data_end, true, PREALLOC_MODE_OFF, 0, NULL); - s->file_end = ret < 0 ? ret : s->data_end; + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to drop preallocation"); + s->file_end = ret; + return ret; + } + s->file_end = s->data_end; + } + + return 0; +} + +static void preallocate_close(BlockDriverState *bs) +{ + BDRVPreallocateState *s = bs->opaque; + + if (s->data_end >= 0) { + preallocate_truncate_to_real_size(bs, NULL); } } @@ -XXX,XX +XXX,XX @@ static int preallocate_check_perm(BlockDriverState *bs, * We should truncate in check_perm, as in set_perm bs->file->perm will * be already changed, and we should not violate it. */ - if (s->file_end < 0) { - s->file_end = bdrv_getlength(bs->file->bs); - if (s->file_end < 0) { - error_setg(errp, "Failed to get file length"); - return s->file_end; - } - } - - if (s->data_end < s->file_end) { - int ret = bdrv_truncate(bs->file, s->data_end, true, - PREALLOC_MODE_OFF, 0, NULL); - if (ret < 0) { - error_setg(errp, "Failed to drop preallocation"); - s->file_end = ret; - return ret; - } - s->file_end = s->data_end; - } + return preallocate_truncate_to_real_size(bs, errp); } return 0; -- 2.41.0 When the permission related BlockDriver callbacks are called, we are in the middle of an operation traversing the block graph. Polling in such a place is a very bad idea because the graph could change in unexpected ways. In the future, callers will also hold the graph lock, which is likely to turn polling into a deadlock. So we need to get rid of calls to functions like bdrv_getlength() or bdrv_truncate() there as these functions poll internally. They are currently used so that when no parent has write/resize permissions on the image any more, the preallocate filter drops the extra preallocated area in the image file and gives up write/resize permissions itself. In order to achieve this without polling in .bdrv_check_perm, don't immediately truncate the image, but only schedule a BH to do so. The filter keeps the write/resize permissions a bit longer now until the BH has executed. There is one case in which delaying doesn't work: Reopening the image read-only. In this case, bs->file will likely be reopened read-only, too, so keeping write permissions a bit longer on it doesn't work. But we can already cover this case in preallocate_reopen_prepare() and not rely on the permission updates for it. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-4-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/preallocate.c | 89 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 69 insertions(+), 20 deletions(-) diff --git a/block/preallocate.c b/block/preallocate.c index XXXXXXX..XXXXXXX 100644 --- a/block/preallocate.c +++ b/block/preallocate.c @@ -XXX,XX +XXX,XX @@ typedef struct BDRVPreallocateState { * be invalid (< 0) when we don't have both exclusive BLK_PERM_RESIZE and * BLK_PERM_WRITE permissions on file child. */ + + /* Gives up the resize permission on children when parents don't need it */ + QEMUBH *drop_resize_bh; } BDRVPreallocateState; +static int preallocate_drop_resize(BlockDriverState *bs, Error **errp); +static void preallocate_drop_resize_bh(void *opaque); + #define PREALLOCATE_OPT_PREALLOC_ALIGN "prealloc-align" #define PREALLOCATE_OPT_PREALLOC_SIZE "prealloc-size" static QemuOptsList runtime_opts = { @@ -XXX,XX +XXX,XX @@ static int preallocate_open(BlockDriverState *bs, QDict *options, int flags, * For this to work, mark them invalid. */ s->file_end = s->zero_start = s->data_end = -EINVAL; + s->drop_resize_bh = qemu_bh_new(preallocate_drop_resize_bh, bs); ret = bdrv_open_file_child(NULL, options, "file", bs, errp); if (ret < 0) { @@ -XXX,XX +XXX,XX @@ static void preallocate_close(BlockDriverState *bs) { BDRVPreallocateState *s = bs->opaque; + qemu_bh_cancel(s->drop_resize_bh); + qemu_bh_delete(s->drop_resize_bh); + if (s->data_end >= 0) { preallocate_truncate_to_real_size(bs, NULL); } @@ -XXX,XX +XXX,XX @@ static int preallocate_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue *queue, Error **errp) { PreallocateOpts *opts = g_new0(PreallocateOpts, 1); + int ret; if (!preallocate_absorb_opts(opts, reopen_state->options, reopen_state->bs->file->bs, errp)) { @@ -XXX,XX +XXX,XX @@ static int preallocate_reopen_prepare(BDRVReopenState *reopen_state, return -EINVAL; } + /* + * Drop the preallocation already here if reopening read-only. The child + * might also be reopened read-only and then scheduling a BH during the + * permission update is too late. + */ + if ((reopen_state->flags & BDRV_O_RDWR) == 0) { + ret = preallocate_drop_resize(reopen_state->bs, errp); + if (ret < 0) { + g_free(opts); + return ret; + } + } + reopen_state->opaque = opts; return 0; @@ -XXX,XX +XXX,XX @@ preallocate_co_getlength(BlockDriverState *bs) return ret; } -static int preallocate_check_perm(BlockDriverState *bs, - uint64_t perm, uint64_t shared, Error **errp) +static int preallocate_drop_resize(BlockDriverState *bs, Error **errp) { BDRVPreallocateState *s = bs->opaque; + int ret; - if (s->data_end >= 0 && !can_write_resize(perm)) { - /* - * Lose permissions. - * We should truncate in check_perm, as in set_perm bs->file->perm will - * be already changed, and we should not violate it. - */ - return preallocate_truncate_to_real_size(bs, errp); + if (s->data_end < 0) { + return 0; + } + + /* + * Before switching children to be read-only, truncate them to remove + * the preallocation and let them have the real size. + */ + ret = preallocate_truncate_to_real_size(bs, errp); + if (ret < 0) { + return ret; } + /* + * We'll drop our permissions and will allow other users to take write and + * resize permissions (see preallocate_child_perm). Anyone will be able to + * change the child, so mark all states invalid. We'll regain control if a + * parent requests write access again. + */ + s->data_end = s->file_end = s->zero_start = -EINVAL; + + bdrv_graph_rdlock_main_loop(); + bdrv_child_refresh_perms(bs, bs->file, NULL); + bdrv_graph_rdunlock_main_loop(); + return 0; } +static void preallocate_drop_resize_bh(void *opaque) +{ + /* + * In case of errors, we'll simply keep the exclusive lock on the image + * indefinitely. + */ + preallocate_drop_resize(opaque, NULL); +} + static void preallocate_set_perm(BlockDriverState *bs, uint64_t perm, uint64_t shared) { BDRVPreallocateState *s = bs->opaque; if (can_write_resize(perm)) { + qemu_bh_cancel(s->drop_resize_bh); if (s->data_end < 0) { s->data_end = s->file_end = s->zero_start = - bdrv_getlength(bs->file->bs); + bs->file->bs->total_sectors * BDRV_SECTOR_SIZE; } } else { - /* - * We drop our permissions, as well as allow shared - * permissions (see preallocate_child_perm), anyone will be able to - * change the child, so mark all states invalid. We'll regain control if - * get good permissions back. - */ - s->data_end = s->file_end = s->zero_start = -EINVAL; + qemu_bh_schedule(s->drop_resize_bh); } } @@ -XXX,XX +XXX,XX @@ static void preallocate_child_perm(BlockDriverState *bs, BdrvChild *c, BdrvChildRole role, BlockReopenQueue *reopen_queue, uint64_t perm, uint64_t shared, uint64_t *nperm, uint64_t *nshared) { + BDRVPreallocateState *s = bs->opaque; + bdrv_default_perms(bs, c, role, reopen_queue, perm, shared, nperm, nshared); - if (can_write_resize(perm)) { - /* This should come by default, but let's enforce: */ + /* + * We need exclusive write and resize permissions on the child not only when + * the parent can write to it, but also after the parent gave up write + * permissions until preallocate_drop_resize() has completed. + */ + if (can_write_resize(perm) || s->data_end != -EINVAL) { *nperm |= BLK_PERM_WRITE | BLK_PERM_RESIZE; /* @@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_preallocate_filter = { .bdrv_co_flush = preallocate_co_flush, .bdrv_co_truncate = preallocate_co_truncate, - .bdrv_check_perm = preallocate_check_perm, .bdrv_set_perm = preallocate_set_perm, .bdrv_child_perm = preallocate_child_perm, -- 2.41.0 The documentation for bdrv_append() says that the caller must hold the AioContext lock for bs_top. Change all callers to actually adhere to the contract. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-5-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/unit/test-bdrv-drain.c | 3 +++ tests/unit/test-bdrv-graph-mod.c | 6 ++++++ tests/unit/test-block-iothread.c | 3 +++ 3 files changed, 12 insertions(+) diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-bdrv-drain.c +++ b/tests/unit/test-bdrv-drain.c @@ -XXX,XX +XXX,XX @@ static void test_append_to_drained(void) g_assert_cmpint(base_s->drain_count, ==, 1); g_assert_cmpint(base->in_flight, ==, 0); + aio_context_acquire(qemu_get_aio_context()); bdrv_append(overlay, base, &error_abort); + aio_context_release(qemu_get_aio_context()); + g_assert_cmpint(base->in_flight, ==, 0); g_assert_cmpint(overlay->in_flight, ==, 0); diff --git a/tests/unit/test-bdrv-graph-mod.c b/tests/unit/test-bdrv-graph-mod.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-bdrv-graph-mod.c +++ b/tests/unit/test-bdrv-graph-mod.c @@ -XXX,XX +XXX,XX @@ static void test_update_perm_tree(void) bdrv_attach_child(filter, bs, "child", &child_of_bds, BDRV_CHILD_DATA, &error_abort); + aio_context_acquire(qemu_get_aio_context()); ret = bdrv_append(filter, bs, NULL); g_assert_cmpint(ret, <, 0); + aio_context_release(qemu_get_aio_context()); bdrv_unref(filter); blk_unref(root); @@ -XXX,XX +XXX,XX @@ static void test_should_update_child(void) g_assert(target->backing->bs == bs); bdrv_attach_child(filter, target, "target", &child_of_bds, BDRV_CHILD_DATA, &error_abort); + aio_context_acquire(qemu_get_aio_context()); bdrv_append(filter, bs, &error_abort); + aio_context_release(qemu_get_aio_context()); g_assert(target->backing->bs == bs); bdrv_unref(filter); @@ -XXX,XX +XXX,XX @@ static void test_append_greedy_filter(void) BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, &error_abort); + aio_context_acquire(qemu_get_aio_context()); bdrv_append(fl, base, &error_abort); + aio_context_release(qemu_get_aio_context()); bdrv_unref(fl); bdrv_unref(top); } diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothread.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-block-iothread.c +++ b/tests/unit/test-block-iothread.c @@ -XXX,XX +XXX,XX @@ static void test_propagate_mirror(void) &error_abort); /* Start a mirror job */ + aio_context_acquire(main_ctx); mirror_start("job0", src, target, NULL, JOB_DEFAULT, 0, 0, 0, MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN, false, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, false, "filter_node", MIRROR_COPY_MODE_BACKGROUND, &error_abort); + aio_context_release(main_ctx); + WITH_JOB_LOCK_GUARD() { job = job_get_locked("job0"); } -- 2.41.0 bdrv_unref() is called by a lot of places that need to hold the graph lock (it naturally happens in the context of operations that change the graph). However, bdrv_unref() takes the graph writer lock internally, so it can't actually be called while already holding a graph lock without causing a deadlock. bdrv_unref() also can't just become GRAPH_WRLOCK because it drains the node before closing it, and draining requires that the graph is unlocked. The solution is to defer deleting the node until we don't hold the lock any more and draining is possible again. Note that keeping images open for longer than necessary can create problems, too: You can't open an image again before it is really closed (if image locking didn't prevent it, it would cause corruption). Reopening an image immediately happens at least during bdrv_open() and bdrv_co_create(). In order to solve this problem, make sure to run the deferred unref in bdrv_graph_wrunlock(), i.e. the first possible place where we can drain again. This is also why bdrv_schedule_unref() is marked GRAPH_WRLOCK. The output of iotest 051 is updated because the additional polling changes the order of HMP output, resulting in a new "(qemu)" prompt in the test output that was previously on a separate line and filtered out. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-ID: <20230911094620.45040-6-kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block-global-state.h | 1 + block.c | 17 +++++++++++++++++ block/graph-lock.c | 26 +++++++++++++++++++------- tests/qemu-iotests/051.pc.out | 6 +++--- 4 files changed, 40 insertions(+), 10 deletions(-) diff --git a/include/block/block-global-state.h b/include/block/block-global-state.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block-global-state.h +++ b/include/block/block-global-state.h @@ -XXX,XX +XXX,XX @@ void bdrv_img_create(const char *filename, const char *fmt, void bdrv_ref(BlockDriverState *bs); void no_coroutine_fn bdrv_unref(BlockDriverState *bs); void coroutine_fn no_co_wrapper bdrv_co_unref(BlockDriverState *bs); +void GRAPH_WRLOCK bdrv_schedule_unref(BlockDriverState *bs); void bdrv_unref_child(BlockDriverState *parent, BdrvChild *child); BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs, BlockDriverState *child_bs, diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ void bdrv_unref(BlockDriverState *bs) } } +/* + * Release a BlockDriverState reference while holding the graph write lock. + * + * Calling bdrv_unref() directly is forbidden while holding the graph lock + * because bdrv_close() both involves polling and taking the graph lock + * internally. bdrv_schedule_unref() instead delays decreasing the refcount and + * possibly closing @bs until the graph lock is released. + */ +void bdrv_schedule_unref(BlockDriverState *bs) +{ + if (!bs) { + return; + } + aio_bh_schedule_oneshot(qemu_get_aio_context(), + (QEMUBHFunc *) bdrv_unref, bs); +} + struct BdrvOpBlocker { Error *reason; QLIST_ENTRY(BdrvOpBlocker) list; diff --git a/block/graph-lock.c b/block/graph-lock.c index XXXXXXX..XXXXXXX 100644 --- a/block/graph-lock.c +++ b/block/graph-lock.c @@ -XXX,XX +XXX,XX @@ void bdrv_graph_wrlock(BlockDriverState *bs) void bdrv_graph_wrunlock(void) { GLOBAL_STATE_CODE(); - QEMU_LOCK_GUARD(&aio_context_list_lock); assert(qatomic_read(&has_writer)); + WITH_QEMU_LOCK_GUARD(&aio_context_list_lock) { + /* + * No need for memory barriers, this works in pair with + * the slow path of rdlock() and both take the lock. + */ + qatomic_store_release(&has_writer, 0); + + /* Wake up all coroutines that are waiting to read the graph */ + qemu_co_enter_all(&reader_queue, &aio_context_list_lock); + } + /* - * No need for memory barriers, this works in pair with - * the slow path of rdlock() and both take the lock. + * Run any BHs that were scheduled during the wrlock section and that + * callers might expect to have finished (in particular, this is important + * for bdrv_schedule_unref()). + * + * Do this only after restarting coroutines so that nested event loops in + * BHs don't deadlock if their condition relies on the coroutine making + * progress. */ - qatomic_store_release(&has_writer, 0); - - /* Wake up all coroutine that are waiting to read the graph */ - qemu_co_enter_all(&reader_queue, &aio_context_list_lock); + aio_bh_poll(qemu_get_aio_context()); } void coroutine_fn bdrv_graph_co_rdlock(void) diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/051.pc.out +++ b/tests/qemu-iotests/051.pc.out @@ -XXX,XX +XXX,XX @@ QEMU_PROG: -device scsi-hd,drive=disk: Device needs media, but drive is empty Testing: -drive file=TEST_DIR/t.qcow2,if=none,node-name=disk -object iothread,id=thread0 -device virtio-scsi,iothread=thread0,id=virtio-scsi0 -device scsi-hd,bus=virtio-scsi0.0,drive=disk,share-rw=on -device ide-hd,drive=disk,share-rw=on QEMU X.Y.Z monitor - type 'help' for more information -QEMU_PROG: -device ide-hd,drive=disk,share-rw=on: Cannot change iothread of active block backend +(qemu) QEMU_PROG: -device ide-hd,drive=disk,share-rw=on: Cannot change iothread of active block backend Testing: -drive file=TEST_DIR/t.qcow2,if=none,node-name=disk -object iothread,id=thread0 -device virtio-scsi,iothread=thread0,id=virtio-scsi0 -device scsi-hd,bus=virtio-scsi0.0,drive=disk,share-rw=on -device virtio-blk-pci,drive=disk,share-rw=on QEMU X.Y.Z monitor - type 'help' for more information -QEMU_PROG: -device virtio-blk-pci,drive=disk,share-rw=on: Cannot change iothread of active block backend +(qemu) QEMU_PROG: -device virtio-blk-pci,drive=disk,share-rw=on: Cannot change iothread of active block backend Testing: -drive file=TEST_DIR/t.qcow2,if=none,node-name=disk -object iothread,id=thread0 -device virtio-scsi,iothread=thread0,id=virtio-scsi0 -device scsi-hd,bus=virtio-scsi0.0,drive=disk,share-rw=on -device lsi53c895a,id=lsi0 -device scsi-hd,bus=lsi0.0,drive=disk,share-rw=on QEMU X.Y.Z monitor - type 'help' for more information @@ -XXX,XX +XXX,XX @@ QEMU X.Y.Z monitor - type 'help' for more information Testing: -drive file=TEST_DIR/t.qcow2,if=none,node-name=disk -object iothread,id=thread0 -device virtio-scsi,iothread=thread0,id=virtio-scsi0 -device scsi-hd,bus=virtio-scsi0.0,drive=disk,share-rw=on -device virtio-blk-pci,drive=disk,iothread=thread0,share-rw=on QEMU X.Y.Z monitor - type 'help' for more information -QEMU_PROG: -device virtio-blk-pci,drive=disk,iothread=thread0,share-rw=on: Cannot change iothread of active block backend +(qemu) QEMU_PROG: -device virtio-blk-pci,drive=disk,iothread=thread0,share-rw=on: Cannot change iothread of active block backend Testing: -drive file=TEST_DIR/t.qcow2,if=none,node-name=disk -object iothread,id=thread0 -device virtio-scsi,iothread=thread0,id=virtio-scsi0 -device scsi-hd,bus=virtio-scsi0.0,drive=disk,share-rw=on -device virtio-scsi,id=virtio-scsi1,iothread=thread0 -device scsi-hd,bus=virtio-scsi1.0,drive=disk,share-rw=on QEMU X.Y.Z monitor - type 'help' for more information -- 2.41.0 Add a new wrapper type for GRAPH_WRLOCK functions that should be called from coroutine context. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-7-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block-common.h | 4 ++++ scripts/block-coroutine-wrapper.py | 11 +++++++++++ 2 files changed, 15 insertions(+) diff --git a/include/block/block-common.h b/include/block/block-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block-common.h +++ b/include/block/block-common.h @@ -XXX,XX +XXX,XX @@ * function. The coroutine yields after scheduling the BH and is reentered when * the wrapped function returns. * + * A no_co_wrapper_bdrv_wrlock function is a no_co_wrapper function that + * automatically takes the graph wrlock when calling the wrapped function. + * * If the first parameter of the function is a BlockDriverState, BdrvChild or * BlockBackend pointer, the AioContext lock for it is taken in the wrapper. */ #define no_co_wrapper +#define no_co_wrapper_bdrv_wrlock #include "block/blockjob.h" diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py index XXXXXXX..XXXXXXX 100644 --- a/scripts/block-coroutine-wrapper.py +++ b/scripts/block-coroutine-wrapper.py @@ -XXX,XX +XXX,XX @@ def __init__(self, wrapper_type: str, return_type: str, name: str, self.args = [ParamDecl(arg.strip()) for arg in args.split(',')] self.create_only_co = 'mixed' not in variant self.graph_rdlock = 'bdrv_rdlock' in variant + self.graph_wrlock = 'bdrv_wrlock' in variant self.wrapper_type = wrapper_type if wrapper_type == 'co': + if self.graph_wrlock: + raise ValueError(f"co function can't be wrlock: {self.name}") subsystem, subname = self.name.split('_', 1) self.target_name = f'{subsystem}_co_{subname}' else: @@ -XXX,XX +XXX,XX @@ def gen_no_co_wrapper(func: FuncDecl) -> str: name = func.target_name struct_name = func.struct_name + graph_lock='' + graph_unlock='' + if func.graph_wrlock: + graph_lock=' bdrv_graph_wrlock(NULL);' + graph_unlock=' bdrv_graph_wrunlock();' + return f"""\ /* * Wrappers for {name} @@ -XXX,XX +XXX,XX @@ def gen_no_co_wrapper(func: FuncDecl) -> str: {struct_name} *s = opaque; AioContext *ctx = {func.gen_ctx('s->')}; +{graph_lock} aio_context_acquire(ctx); {func.get_result}{name}({ func.gen_list('s->{name}') }); aio_context_release(ctx); +{graph_unlock} aio_co_wake(s->co); }} -- 2.41.0 Don't assume specific parameter names like 'bs' or 'blk' in the generated code, but use the actual name. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-8-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- scripts/block-coroutine-wrapper.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py index XXXXXXX..XXXXXXX 100644 --- a/scripts/block-coroutine-wrapper.py +++ b/scripts/block-coroutine-wrapper.py @@ -XXX,XX +XXX,XX @@ def __init__(self, wrapper_type: str, return_type: str, name: str, def gen_ctx(self, prefix: str = '') -> str: t = self.args[0].type + name = self.args[0].name if t == 'BlockDriverState *': - return f'bdrv_get_aio_context({prefix}bs)' + return f'bdrv_get_aio_context({prefix}{name})' elif t == 'BdrvChild *': - return f'bdrv_get_aio_context({prefix}child->bs)' + return f'bdrv_get_aio_context({prefix}{name}->bs)' elif t == 'BlockBackend *': - return f'blk_get_aio_context({prefix}blk)' + return f'blk_get_aio_context({prefix}{name})' else: return 'qemu_get_aio_context()' -- 2.41.0 Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_replace_child_noperm(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-9-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static BlockDriverState *bdrv_open_inherit(const char *filename, static bool bdrv_recurse_has_child(BlockDriverState *bs, BlockDriverState *child); -static void bdrv_replace_child_noperm(BdrvChild *child, - BlockDriverState *new_bs); +static void GRAPH_WRLOCK +bdrv_replace_child_noperm(BdrvChild *child, BlockDriverState *new_bs); + static void bdrv_remove_child(BdrvChild *child, Transaction *tran); static int bdrv_reopen_prepare(BDRVReopenState *reopen_state, @@ -XXX,XX +XXX,XX @@ static void bdrv_replace_child_abort(void *opaque) BlockDriverState *new_bs = s->child->bs; GLOBAL_STATE_CODE(); + bdrv_graph_wrlock(s->old_bs); + /* old_bs reference is transparently moved from @s to @s->child */ if (!s->child->bs) { /* @@ -XXX,XX +XXX,XX @@ static void bdrv_replace_child_abort(void *opaque) } assert(s->child->quiesced_parent); bdrv_replace_child_noperm(s->child, s->old_bs); + + bdrv_graph_wrunlock(); bdrv_unref(new_bs); } @@ -XXX,XX +XXX,XX @@ static void bdrv_replace_child_tran(BdrvChild *child, BlockDriverState *new_bs, if (new_bs) { bdrv_ref(new_bs); } + + bdrv_graph_wrlock(new_bs); bdrv_replace_child_noperm(child, new_bs); + bdrv_graph_wrunlock(); /* old_bs reference is transparently moved from @child to @s */ } @@ -XXX,XX +XXX,XX @@ uint64_t bdrv_qapi_perm_to_blk_perm(BlockPermission qapi_perm) * If @new_bs is non-NULL, the parent of @child must already be drained through * @child and the caller must hold the AioContext lock for @new_bs. */ -static void bdrv_replace_child_noperm(BdrvChild *child, - BlockDriverState *new_bs) +static void GRAPH_WRLOCK +bdrv_replace_child_noperm(BdrvChild *child, BlockDriverState *new_bs) { BlockDriverState *old_bs = child->bs; int new_bs_quiesce_counter; @@ -XXX,XX +XXX,XX @@ static void bdrv_replace_child_noperm(BdrvChild *child, assert(bdrv_get_aio_context(old_bs) == bdrv_get_aio_context(new_bs)); } - /* TODO Pull this up into the callers to avoid polling here */ - bdrv_graph_wrlock(new_bs); if (old_bs) { if (child->klass->detach) { child->klass->detach(child); @@ -XXX,XX +XXX,XX @@ static void bdrv_replace_child_noperm(BdrvChild *child, child->klass->attach(child); } } - bdrv_graph_wrunlock(); /* * If the parent was drained through this BdrvChild previously, but new_bs @@ -XXX,XX +XXX,XX @@ static void bdrv_attach_child_common_abort(void *opaque) BlockDriverState *bs = s->child->bs; GLOBAL_STATE_CODE(); + + bdrv_graph_wrlock(NULL); bdrv_replace_child_noperm(s->child, NULL); + bdrv_graph_wrunlock(); if (bdrv_get_aio_context(bs) != s->old_child_ctx) { bdrv_try_change_aio_context(bs, s->old_child_ctx, NULL, &error_abort); @@ -XXX,XX +XXX,XX @@ static BdrvChild *bdrv_attach_child_common(BlockDriverState *child_bs, * a problem, we already did this), but it will still poll until the parent * is fully quiesced, so it will not be negatively affected either. */ + bdrv_graph_wrlock(child_bs); bdrv_parent_drained_begin_single(new_child); bdrv_replace_child_noperm(new_child, child_bs); + bdrv_graph_wrunlock(); BdrvAttachChildCommonState *s = g_new(BdrvAttachChildCommonState, 1); *s = (BdrvAttachChildCommonState) { @@ -XXX,XX +XXX,XX @@ void bdrv_root_unref_child(BdrvChild *child) BlockDriverState *child_bs = child->bs; GLOBAL_STATE_CODE(); + bdrv_graph_wrlock(NULL); bdrv_replace_child_noperm(child, NULL); + bdrv_graph_wrunlock(); bdrv_child_free(child); if (child_bs) { -- 2.41.0 Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_replace_child_tran(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. While a graph lock is held, polling is not allowed. Therefore draining the necessary nodes can no longer be done in bdrv_remove_child() and bdrv_replace_node_noperm(), but the callers must already make sure that they are drained. Note that the transaction callbacks still take the lock internally, so tran_finalize() must be called without the lock held. This is because bdrv_append() also calls bdrv_attach_child_noperm(), which currently requires to be called unlocked. Once it changes, the transaction callbacks can be changed, too. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-10-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block.c | 78 ++++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 50 insertions(+), 28 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static bool bdrv_recurse_has_child(BlockDriverState *bs, static void GRAPH_WRLOCK bdrv_replace_child_noperm(BdrvChild *child, BlockDriverState *new_bs); -static void bdrv_remove_child(BdrvChild *child, Transaction *tran); +static void GRAPH_WRLOCK +bdrv_remove_child(BdrvChild *child, Transaction *tran); static int bdrv_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue *queue, @@ -XXX,XX +XXX,XX @@ static TransactionActionDrv bdrv_replace_child_drv = { * * The function doesn't update permissions, caller is responsible for this. */ -static void bdrv_replace_child_tran(BdrvChild *child, BlockDriverState *new_bs, - Transaction *tran) +static void GRAPH_WRLOCK +bdrv_replace_child_tran(BdrvChild *child, BlockDriverState *new_bs, + Transaction *tran) { BdrvReplaceChildState *s = g_new(BdrvReplaceChildState, 1); @@ -XXX,XX +XXX,XX @@ static void bdrv_replace_child_tran(BdrvChild *child, BlockDriverState *new_bs, bdrv_ref(new_bs); } - bdrv_graph_wrlock(new_bs); bdrv_replace_child_noperm(child, new_bs); - bdrv_graph_wrunlock(); /* old_bs reference is transparently moved from @child to @s */ } @@ -XXX,XX +XXX,XX @@ static int bdrv_set_file_or_backing_noperm(BlockDriverState *parent_bs, } if (child) { + bdrv_drained_begin(child->bs); + bdrv_graph_wrlock(NULL); + bdrv_unset_inherits_from(parent_bs, child, tran); bdrv_remove_child(child, tran); + + bdrv_graph_wrunlock(); + bdrv_drained_end(child->bs); } if (!child_bs) { @@ -XXX,XX +XXX,XX @@ void bdrv_close_all(void) assert(QTAILQ_EMPTY(&all_bdrv_states)); } -static bool should_update_child(BdrvChild *c, BlockDriverState *to) +static bool GRAPH_RDLOCK should_update_child(BdrvChild *c, BlockDriverState *to) { GQueue *queue; GHashTable *found; @@ -XXX,XX +XXX,XX @@ static TransactionActionDrv bdrv_remove_child_drv = { .commit = bdrv_remove_child_commit, }; -/* Function doesn't update permissions, caller is responsible for this. */ -static void bdrv_remove_child(BdrvChild *child, Transaction *tran) +/* + * Function doesn't update permissions, caller is responsible for this. + * + * @child->bs (if non-NULL) must be drained. + */ +static void GRAPH_WRLOCK bdrv_remove_child(BdrvChild *child, Transaction *tran) { if (!child) { return; } if (child->bs) { - BlockDriverState *bs = child->bs; - bdrv_drained_begin(bs); + assert(child->quiesced_parent); bdrv_replace_child_tran(child, NULL, tran); - bdrv_drained_end(bs); } tran_add(tran, &bdrv_remove_child_drv, child); } -static void undrain_on_clean_cb(void *opaque) -{ - bdrv_drained_end(opaque); -} - -static TransactionActionDrv undrain_on_clean = { - .clean = undrain_on_clean_cb, -}; - -static int bdrv_replace_node_noperm(BlockDriverState *from, - BlockDriverState *to, - bool auto_skip, Transaction *tran, - Error **errp) +/* + * Both @from and @to (if non-NULL) must be drained. @to must be kept drained + * until the transaction is completed. + */ +static int GRAPH_WRLOCK +bdrv_replace_node_noperm(BlockDriverState *from, + BlockDriverState *to, + bool auto_skip, Transaction *tran, + Error **errp) { BdrvChild *c, *next; GLOBAL_STATE_CODE(); - bdrv_drained_begin(from); - bdrv_drained_begin(to); - tran_add(tran, &undrain_on_clean, from); - tran_add(tran, &undrain_on_clean, to); + assert(from->quiesce_counter); + assert(to->quiesce_counter); QLIST_FOREACH_SAFE(c, &from->parents, next_parent, next) { assert(c->bs == from); @@ -XXX,XX +XXX,XX @@ static int bdrv_replace_node_common(BlockDriverState *from, assert(qemu_get_current_aio_context() == qemu_get_aio_context()); assert(bdrv_get_aio_context(from) == bdrv_get_aio_context(to)); bdrv_drained_begin(from); + bdrv_drained_begin(to); + + bdrv_graph_wrlock(to); /* * Do the replacement without permission update. @@ -XXX,XX +XXX,XX @@ static int bdrv_replace_node_common(BlockDriverState *from, } if (detach_subchain) { + /* to_cow_parent is already drained because from is drained */ bdrv_remove_child(bdrv_filter_or_cow_child(to_cow_parent), tran); } @@ -XXX,XX +XXX,XX @@ static int bdrv_replace_node_common(BlockDriverState *from, ret = 0; out: + bdrv_graph_wrunlock(); tran_finalize(tran, ret); + bdrv_drained_end(to); bdrv_drained_end(from); bdrv_unref(from); @@ -XXX,XX +XXX,XX @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, BdrvChild *child; Transaction *tran = tran_new(); AioContext *old_context, *new_context = NULL; + bool drained = false; GLOBAL_STATE_CODE(); @@ -XXX,XX +XXX,XX @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, aio_context_acquire(new_context); } + bdrv_drained_begin(bs_new); + bdrv_drained_begin(bs_top); + drained = true; + + bdrv_graph_wrlock(bs_new); ret = bdrv_replace_node_noperm(bs_top, bs_new, true, tran, errp); + bdrv_graph_wrunlock(); if (ret < 0) { goto out; } @@ -XXX,XX +XXX,XX @@ out: bdrv_refresh_limits(bs_top, NULL, NULL); bdrv_graph_rdunlock_main_loop(); + if (drained) { + bdrv_drained_end(bs_top); + bdrv_drained_end(bs_new); + } + if (new_context && old_context != new_context) { aio_context_release(new_context); aio_context_acquire(old_context); @@ -XXX,XX +XXX,XX @@ int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs, bdrv_ref(old_bs); bdrv_drained_begin(old_bs); bdrv_drained_begin(new_bs); + bdrv_graph_wrlock(new_bs); bdrv_replace_child_tran(child, new_bs, tran); @@ -XXX,XX +XXX,XX @@ int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs, refresh_list = g_slist_prepend(refresh_list, new_bs); ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp); + bdrv_graph_wrunlock(); tran_finalize(tran, ret); -- 2.41.0 Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_attach_child_common(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. Note that the transaction callbacks still take the lock internally, so tran_finalize() must be called without the lock held. This is because bdrv_append() also calls bdrv_replace_node_noperm(), which currently requires the transaction callbacks to be called unlocked. In the next step, both of them can be switched to locked tran_finalize() calls together. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-11-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block.c | 133 +++++++++++++++++++++++++++++++------------------ block/stream.c | 20 ++++++-- 2 files changed, 100 insertions(+), 53 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static TransactionActionDrv bdrv_attach_child_common_drv = { * @child_bs can move to a different AioContext in this function. Callers must * make sure that their AioContext locking is still correct after this. */ -static BdrvChild *bdrv_attach_child_common(BlockDriverState *child_bs, - const char *child_name, - const BdrvChildClass *child_class, - BdrvChildRole child_role, - uint64_t perm, uint64_t shared_perm, - void *opaque, - Transaction *tran, Error **errp) +static BdrvChild * GRAPH_WRLOCK +bdrv_attach_child_common(BlockDriverState *child_bs, + const char *child_name, + const BdrvChildClass *child_class, + BdrvChildRole child_role, + uint64_t perm, uint64_t shared_perm, + void *opaque, + Transaction *tran, Error **errp) { BdrvChild *new_child; AioContext *parent_ctx, *new_child_ctx; @@ -XXX,XX +XXX,XX @@ static BdrvChild *bdrv_attach_child_common(BlockDriverState *child_bs, * a problem, we already did this), but it will still poll until the parent * is fully quiesced, so it will not be negatively affected either. */ - bdrv_graph_wrlock(child_bs); bdrv_parent_drained_begin_single(new_child); bdrv_replace_child_noperm(new_child, child_bs); - bdrv_graph_wrunlock(); BdrvAttachChildCommonState *s = g_new(BdrvAttachChildCommonState, 1); *s = (BdrvAttachChildCommonState) { @@ -XXX,XX +XXX,XX @@ static BdrvChild *bdrv_attach_child_common(BlockDriverState *child_bs, * @child_bs can move to a different AioContext in this function. Callers must * make sure that their AioContext locking is still correct after this. */ -static BdrvChild *bdrv_attach_child_noperm(BlockDriverState *parent_bs, - BlockDriverState *child_bs, - const char *child_name, - const BdrvChildClass *child_class, - BdrvChildRole child_role, - Transaction *tran, - Error **errp) +static BdrvChild * GRAPH_WRLOCK +bdrv_attach_child_noperm(BlockDriverState *parent_bs, + BlockDriverState *child_bs, + const char *child_name, + const BdrvChildClass *child_class, + BdrvChildRole child_role, + Transaction *tran, + Error **errp) { uint64_t perm, shared_perm; @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs, GLOBAL_STATE_CODE(); + bdrv_graph_wrlock(child_bs); + child = bdrv_attach_child_common(child_bs, child_name, child_class, child_role, perm, shared_perm, opaque, tran, errp); @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs, ret = bdrv_refresh_perms(child_bs, tran, errp); out: + bdrv_graph_wrunlock(); tran_finalize(tran, ret); bdrv_unref(child_bs); @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs, GLOBAL_STATE_CODE(); + bdrv_graph_wrlock(child_bs); + child = bdrv_attach_child_noperm(parent_bs, child_bs, child_name, child_class, child_role, tran, errp); if (!child) { @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs, } out: + bdrv_graph_wrunlock(); tran_finalize(tran, ret); bdrv_unref(child_bs); @@ -XXX,XX +XXX,XX @@ static BdrvChildRole bdrv_backing_role(BlockDriverState *bs) * Sets the bs->backing or bs->file link of a BDS. A new reference is created; * callers which don't need their own reference any more must call bdrv_unref(). * + * If the respective child is already present (i.e. we're detaching a node), + * that child node must be drained. + * * Function doesn't update permissions, caller is responsible for this. * * The caller must hold the AioContext lock for @child_bs. Both @parent_bs and * @child_bs can move to a different AioContext in this function. Callers must * make sure that their AioContext locking is still correct after this. */ -static int bdrv_set_file_or_backing_noperm(BlockDriverState *parent_bs, - BlockDriverState *child_bs, - bool is_backing, - Transaction *tran, Error **errp) +static int GRAPH_WRLOCK +bdrv_set_file_or_backing_noperm(BlockDriverState *parent_bs, + BlockDriverState *child_bs, + bool is_backing, + Transaction *tran, Error **errp) { bool update_inherits_from = bdrv_inherits_from_recursive(child_bs, parent_bs); @@ -XXX,XX +XXX,XX @@ static int bdrv_set_file_or_backing_noperm(BlockDriverState *parent_bs, } if (child) { - bdrv_drained_begin(child->bs); - bdrv_graph_wrlock(NULL); - + assert(child->bs->quiesce_counter); bdrv_unset_inherits_from(parent_bs, child, tran); bdrv_remove_child(child, tran); - - bdrv_graph_wrunlock(); - bdrv_drained_end(child->bs); } if (!child_bs) { @@ -XXX,XX +XXX,XX @@ static int bdrv_set_file_or_backing_noperm(BlockDriverState *parent_bs, } out: - bdrv_graph_rdlock_main_loop(); bdrv_refresh_limits(parent_bs, tran, NULL); - bdrv_graph_rdunlock_main_loop(); return 0; } @@ -XXX,XX +XXX,XX @@ out: * The caller must hold the AioContext lock for @backing_hd. Both @bs and * @backing_hd can move to a different AioContext in this function. Callers must * make sure that their AioContext locking is still correct after this. + * + * If a backing child is already present (i.e. we're detaching a node), that + * child node must be drained. */ -static int bdrv_set_backing_noperm(BlockDriverState *bs, - BlockDriverState *backing_hd, - Transaction *tran, Error **errp) +static int GRAPH_WRLOCK +bdrv_set_backing_noperm(BlockDriverState *bs, + BlockDriverState *backing_hd, + Transaction *tran, Error **errp) { GLOBAL_STATE_CODE(); return bdrv_set_file_or_backing_noperm(bs, backing_hd, true, tran, errp); @@ -XXX,XX +XXX,XX @@ int bdrv_set_backing_hd_drained(BlockDriverState *bs, GLOBAL_STATE_CODE(); assert(bs->quiesce_counter > 0); + if (bs->backing) { + assert(bs->backing->bs->quiesce_counter > 0); + } + bdrv_graph_wrlock(backing_hd); ret = bdrv_set_backing_noperm(bs, backing_hd, tran, errp); if (ret < 0) { @@ -XXX,XX +XXX,XX @@ int bdrv_set_backing_hd_drained(BlockDriverState *bs, ret = bdrv_refresh_perms(bs, tran, errp); out: + bdrv_graph_wrunlock(); tran_finalize(tran, ret); return ret; } @@ -XXX,XX +XXX,XX @@ out: int bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd, Error **errp) { + BlockDriverState *drain_bs = bs->backing ? bs->backing->bs : bs; int ret; GLOBAL_STATE_CODE(); - bdrv_drained_begin(bs); + bdrv_ref(drain_bs); + bdrv_drained_begin(drain_bs); ret = bdrv_set_backing_hd_drained(bs, backing_hd, errp); - bdrv_drained_end(bs); + bdrv_drained_end(drain_bs); + bdrv_unref(drain_bs); return ret; } @@ -XXX,XX +XXX,XX @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, Error **errp) abort: tran_abort(tran); + QTAILQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) { if (bs_entry->prepared) { ctx = bdrv_get_aio_context(bs_entry->state.bs); @@ -XXX,XX +XXX,XX @@ static int bdrv_reopen_parse_file_or_backing(BDRVReopenState *reopen_state, reopen_state->old_file_bs = old_child_bs; } + if (old_child_bs) { + bdrv_ref(old_child_bs); + bdrv_drained_begin(old_child_bs); + } + old_ctx = bdrv_get_aio_context(bs); ctx = bdrv_get_aio_context(new_child_bs); if (old_ctx != ctx) { @@ -XXX,XX +XXX,XX @@ static int bdrv_reopen_parse_file_or_backing(BDRVReopenState *reopen_state, aio_context_acquire(ctx); } + bdrv_graph_wrlock(new_child_bs); + ret = bdrv_set_file_or_backing_noperm(bs, new_child_bs, is_backing, tran, errp); + bdrv_graph_wrunlock(); + if (old_ctx != ctx) { aio_context_release(ctx); aio_context_acquire(old_ctx); } + if (old_child_bs) { + bdrv_drained_end(old_child_bs); + bdrv_unref(old_child_bs); + } + return ret; } @@ -XXX,XX +XXX,XX @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, BdrvChild *child; Transaction *tran = tran_new(); AioContext *old_context, *new_context = NULL; - bool drained = false; GLOBAL_STATE_CODE(); assert(!bs_new->backing); old_context = bdrv_get_aio_context(bs_top); + bdrv_drained_begin(bs_top); + + /* + * bdrv_drained_begin() requires that only the AioContext of the drained + * node is locked, and at this point it can still differ from the AioContext + * of bs_top. + */ + new_context = bdrv_get_aio_context(bs_new); + aio_context_release(old_context); + aio_context_acquire(new_context); + bdrv_drained_begin(bs_new); + aio_context_release(new_context); + aio_context_acquire(old_context); + new_context = NULL; + + bdrv_graph_wrlock(bs_top); child = bdrv_attach_child_noperm(bs_new, bs_top, "backing", &child_of_bds, bdrv_backing_role(bs_new), @@ -XXX,XX +XXX,XX @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, } /* - * bdrv_attach_child_noperm could change the AioContext of bs_top. - * bdrv_replace_node_noperm calls bdrv_drained_begin, so let's temporarily - * hold the new AioContext, since bdrv_drained_begin calls BDRV_POLL_WHILE - * that assumes the new lock is taken. + * bdrv_attach_child_noperm could change the AioContext of bs_top and + * bs_new, but at least they are in the same AioContext now. This is the + * AioContext that we need to lock for the rest of the function. */ new_context = bdrv_get_aio_context(bs_top); @@ -XXX,XX +XXX,XX @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, aio_context_acquire(new_context); } - bdrv_drained_begin(bs_new); - bdrv_drained_begin(bs_top); - drained = true; - - bdrv_graph_wrlock(bs_new); ret = bdrv_replace_node_noperm(bs_top, bs_new, true, tran, errp); - bdrv_graph_wrunlock(); if (ret < 0) { goto out; } ret = bdrv_refresh_perms(bs_new, tran, errp); out: + bdrv_graph_wrunlock(); tran_finalize(tran, ret); bdrv_graph_rdlock_main_loop(); bdrv_refresh_limits(bs_top, NULL, NULL); bdrv_graph_rdunlock_main_loop(); - if (drained) { - bdrv_drained_end(bs_top); - bdrv_drained_end(bs_new); - } + bdrv_drained_end(bs_top); + bdrv_drained_end(bs_new); if (new_context && old_context != new_context) { aio_context_release(new_context); diff --git a/block/stream.c b/block/stream.c index XXXXXXX..XXXXXXX 100644 --- a/block/stream.c +++ b/block/stream.c @@ -XXX,XX +XXX,XX @@ static int stream_prepare(Job *job) { StreamBlockJob *s = container_of(job, StreamBlockJob, common.job); BlockDriverState *unfiltered_bs = bdrv_skip_filters(s->target_bs); + BlockDriverState *unfiltered_bs_cow = bdrv_cow_bs(unfiltered_bs); BlockDriverState *base; BlockDriverState *unfiltered_base; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int stream_prepare(Job *job) s->cor_filter_bs = NULL; /* - * bdrv_set_backing_hd() requires that unfiltered_bs is drained. Drain - * already here and use bdrv_set_backing_hd_drained() instead because - * the polling during drained_begin() might change the graph, and if we do - * this only later, we may end up working with the wrong base node (or it - * might even have gone away by the time we want to use it). + * bdrv_set_backing_hd() requires that the unfiltered_bs and the COW child + * of unfiltered_bs is drained. Drain already here and use + * bdrv_set_backing_hd_drained() instead because the polling during + * drained_begin() might change the graph, and if we do this only later, we + * may end up working with the wrong base node (or it might even have gone + * away by the time we want to use it). */ bdrv_drained_begin(unfiltered_bs); + if (unfiltered_bs_cow) { + bdrv_ref(unfiltered_bs_cow); + bdrv_drained_begin(unfiltered_bs_cow); + } base = bdrv_filter_or_cow_bs(s->above_base); unfiltered_base = bdrv_skip_filters(base); @@ -XXX,XX +XXX,XX @@ static int stream_prepare(Job *job) } out: + if (unfiltered_bs_cow) { + bdrv_drained_end(unfiltered_bs_cow); + bdrv_unref(unfiltered_bs_cow); + } bdrv_drained_end(unfiltered_bs); return ret; } -- 2.41.0 In previous patches, we changed some transactionable functions to be marked as GRAPH_WRLOCK, but required that tran_finalize() is still called without the lock. This was because all callbacks that can be in the same transaction need to follow the same convention. Now that we don't have conflicting requirements any more, we can switch all of the transaction callbacks to be declared GRAPH_WRLOCK, too, and call tran_finalize() with the lock held. Document for each of these transactionable functions that the lock needs to be held when completing the transaction, and make sure that all callers down to the place where the transaction is finalised actually have the writer lock. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-12-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block.c | 61 +++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 17 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ typedef struct BdrvReplaceChildState { BlockDriverState *old_bs; } BdrvReplaceChildState; -static void bdrv_replace_child_commit(void *opaque) +static void GRAPH_WRLOCK bdrv_replace_child_commit(void *opaque) { BdrvReplaceChildState *s = opaque; GLOBAL_STATE_CODE(); - bdrv_unref(s->old_bs); + bdrv_schedule_unref(s->old_bs); } -static void bdrv_replace_child_abort(void *opaque) +static void GRAPH_WRLOCK bdrv_replace_child_abort(void *opaque) { BdrvReplaceChildState *s = opaque; BlockDriverState *new_bs = s->child->bs; GLOBAL_STATE_CODE(); - bdrv_graph_wrlock(s->old_bs); + assert_bdrv_graph_writable(); /* old_bs reference is transparently moved from @s to @s->child */ if (!s->child->bs) { @@ -XXX,XX +XXX,XX @@ static void bdrv_replace_child_abort(void *opaque) assert(s->child->quiesced_parent); bdrv_replace_child_noperm(s->child, s->old_bs); - bdrv_graph_wrunlock(); bdrv_unref(new_bs); } @@ -XXX,XX +XXX,XX @@ static TransactionActionDrv bdrv_replace_child_drv = { * Both @child->bs and @new_bs (if non-NULL) must be drained. @new_bs must be * kept drained until the transaction is completed. * + * After calling this function, the transaction @tran may only be completed + * while holding a writer lock for the graph. + * * The function doesn't update permissions, caller is responsible for this. */ static void GRAPH_WRLOCK @@ -XXX,XX +XXX,XX @@ typedef struct BdrvAttachChildCommonState { AioContext *old_child_ctx; } BdrvAttachChildCommonState; -static void bdrv_attach_child_common_abort(void *opaque) +static void GRAPH_WRLOCK bdrv_attach_child_common_abort(void *opaque) { BdrvAttachChildCommonState *s = opaque; BlockDriverState *bs = s->child->bs; GLOBAL_STATE_CODE(); + assert_bdrv_graph_writable(); - bdrv_graph_wrlock(NULL); bdrv_replace_child_noperm(s->child, NULL); - bdrv_graph_wrunlock(); if (bdrv_get_aio_context(bs) != s->old_child_ctx) { bdrv_try_change_aio_context(bs, s->old_child_ctx, NULL, &error_abort); @@ -XXX,XX +XXX,XX @@ static void bdrv_attach_child_common_abort(void *opaque) tran_commit(tran); } - bdrv_unref(bs); + bdrv_schedule_unref(bs); bdrv_child_free(s->child); } @@ -XXX,XX +XXX,XX @@ static TransactionActionDrv bdrv_attach_child_common_drv = { * * Function doesn't update permissions, caller is responsible for this. * + * After calling this function, the transaction @tran may only be completed + * while holding a writer lock for the graph. + * * Returns new created child. * * The caller must hold the AioContext lock for @child_bs. Both @parent_bs and @@ -XXX,XX +XXX,XX @@ bdrv_attach_child_common(BlockDriverState *child_bs, * The caller must hold the AioContext lock for @child_bs. Both @parent_bs and * @child_bs can move to a different AioContext in this function. Callers must * make sure that their AioContext locking is still correct after this. + * + * After calling this function, the transaction @tran may only be completed + * while holding a writer lock for the graph. */ static BdrvChild * GRAPH_WRLOCK bdrv_attach_child_noperm(BlockDriverState *parent_bs, @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs, ret = bdrv_refresh_perms(child_bs, tran, errp); out: - bdrv_graph_wrunlock(); tran_finalize(tran, ret); + bdrv_graph_wrunlock(); bdrv_unref(child_bs); @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs, } out: - bdrv_graph_wrunlock(); tran_finalize(tran, ret); + bdrv_graph_wrunlock(); bdrv_unref(child_bs); @@ -XXX,XX +XXX,XX @@ static BdrvChildRole bdrv_backing_role(BlockDriverState *bs) * The caller must hold the AioContext lock for @child_bs. Both @parent_bs and * @child_bs can move to a different AioContext in this function. Callers must * make sure that their AioContext locking is still correct after this. + * + * After calling this function, the transaction @tran may only be completed + * while holding a writer lock for the graph. */ static int GRAPH_WRLOCK bdrv_set_file_or_backing_noperm(BlockDriverState *parent_bs, @@ -XXX,XX +XXX,XX @@ out: * * If a backing child is already present (i.e. we're detaching a node), that * child node must be drained. + * + * After calling this function, the transaction @tran may only be completed + * while holding a writer lock for the graph. */ static int GRAPH_WRLOCK bdrv_set_backing_noperm(BlockDriverState *bs, @@ -XXX,XX +XXX,XX @@ int bdrv_set_backing_hd_drained(BlockDriverState *bs, ret = bdrv_refresh_perms(bs, tran, errp); out: - bdrv_graph_wrunlock(); tran_finalize(tran, ret); + bdrv_graph_wrunlock(); return ret; } @@ -XXX,XX +XXX,XX @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, Error **errp) aio_context_release(ctx); } + bdrv_graph_wrlock(NULL); tran_commit(tran); + bdrv_graph_wrunlock(); QTAILQ_FOREACH_REVERSE(bs_entry, bs_queue, entry) { BlockDriverState *bs = bs_entry->state.bs; @@ -XXX,XX +XXX,XX @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, Error **errp) goto cleanup; abort: + bdrv_graph_wrlock(NULL); tran_abort(tran); + bdrv_graph_wrunlock(); QTAILQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) { if (bs_entry->prepared) { @@ -XXX,XX +XXX,XX @@ int bdrv_reopen_set_read_only(BlockDriverState *bs, bool read_only, * true and reopen_state->new_backing_bs contains a pointer to the new * backing BlockDriverState (or NULL). * + * After calling this function, the transaction @tran may only be completed + * while holding a writer lock for the graph. + * * Return 0 on success, otherwise return < 0 and set @errp. * * The caller must hold the AioContext lock of @reopen_state->bs. @@ -XXX,XX +XXX,XX @@ static int bdrv_reopen_parse_file_or_backing(BDRVReopenState *reopen_state, * commit() for any other BDS that have been left in a prepare() state * * The caller must hold the AioContext lock of @reopen_state->bs. + * + * After calling this function, the transaction @change_child_tran may only be + * completed while holding a writer lock for the graph. */ static int bdrv_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue *queue, @@ -XXX,XX +XXX,XX @@ static TransactionActionDrv bdrv_remove_child_drv = { * Function doesn't update permissions, caller is responsible for this. * * @child->bs (if non-NULL) must be drained. + * + * After calling this function, the transaction @tran may only be completed + * while holding a writer lock for the graph. */ static void GRAPH_WRLOCK bdrv_remove_child(BdrvChild *child, Transaction *tran) { @@ -XXX,XX +XXX,XX @@ static void GRAPH_WRLOCK bdrv_remove_child(BdrvChild *child, Transaction *tran) /* * Both @from and @to (if non-NULL) must be drained. @to must be kept drained * until the transaction is completed. + * + * After calling this function, the transaction @tran may only be completed + * while holding a writer lock for the graph. */ static int GRAPH_WRLOCK bdrv_replace_node_noperm(BlockDriverState *from, @@ -XXX,XX +XXX,XX @@ static int bdrv_replace_node_common(BlockDriverState *from, ret = 0; out: - bdrv_graph_wrunlock(); tran_finalize(tran, ret); + bdrv_graph_wrunlock(); bdrv_drained_end(to); bdrv_drained_end(from); @@ -XXX,XX +XXX,XX @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, ret = bdrv_refresh_perms(bs_new, tran, errp); out: - bdrv_graph_wrunlock(); tran_finalize(tran, ret); - bdrv_graph_rdlock_main_loop(); bdrv_refresh_limits(bs_top, NULL, NULL); - bdrv_graph_rdunlock_main_loop(); + bdrv_graph_wrunlock(); bdrv_drained_end(bs_top); bdrv_drained_end(bs_new); @@ -XXX,XX +XXX,XX @@ int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs, refresh_list = g_slist_prepend(refresh_list, new_bs); ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp); - bdrv_graph_wrunlock(); tran_finalize(tran, ret); + bdrv_graph_wrunlock(); bdrv_drained_end(old_bs); bdrv_drained_end(new_bs); bdrv_unref(old_bs); -- 2.41.0 Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_attach_child_common(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-13-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block-global-state.h | 14 ++++++++------ block.c | 7 +++---- block/quorum.c | 2 ++ block/replication.c | 6 ++++++ tests/unit/test-bdrv-drain.c | 14 ++++++++++++++ tests/unit/test-bdrv-graph-mod.c | 10 ++++++++++ 6 files changed, 43 insertions(+), 10 deletions(-) diff --git a/include/block/block-global-state.h b/include/block/block-global-state.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block-global-state.h +++ b/include/block/block-global-state.h @@ -XXX,XX +XXX,XX @@ void no_coroutine_fn bdrv_unref(BlockDriverState *bs); void coroutine_fn no_co_wrapper bdrv_co_unref(BlockDriverState *bs); void GRAPH_WRLOCK bdrv_schedule_unref(BlockDriverState *bs); void bdrv_unref_child(BlockDriverState *parent, BdrvChild *child); -BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs, - BlockDriverState *child_bs, - const char *child_name, - const BdrvChildClass *child_class, - BdrvChildRole child_role, - Error **errp); + +BdrvChild * GRAPH_WRLOCK +bdrv_attach_child(BlockDriverState *parent_bs, + BlockDriverState *child_bs, + const char *child_name, + const BdrvChildClass *child_class, + BdrvChildRole child_role, + Error **errp); bool bdrv_op_is_blocked(BlockDriverState *bs, BlockOpType op, Error **errp); void bdrv_op_block(BlockDriverState *bs, BlockOpType op, Error *reason); diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs, GLOBAL_STATE_CODE(); - bdrv_graph_wrlock(child_bs); - child = bdrv_attach_child_noperm(parent_bs, child_bs, child_name, child_class, child_role, tran, errp); if (!child) { @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs, out: tran_finalize(tran, ret); - bdrv_graph_wrunlock(); - bdrv_unref(child_bs); + bdrv_schedule_unref(child_bs); return ret < 0 ? NULL : child; } @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_open_child(const char *filename, return NULL; } + bdrv_graph_wrlock(NULL); ctx = bdrv_get_aio_context(bs); aio_context_acquire(ctx); child = bdrv_attach_child(parent, bs, bdref_key, child_class, child_role, errp); aio_context_release(ctx); + bdrv_graph_wrunlock(); return child; } diff --git a/block/quorum.c b/block/quorum.c index XXXXXXX..XXXXXXX 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -XXX,XX +XXX,XX @@ static void quorum_add_child(BlockDriverState *bs, BlockDriverState *child_bs, /* We can safely add the child now */ bdrv_ref(child_bs); + bdrv_graph_wrlock(child_bs); child = bdrv_attach_child(bs, child_bs, indexstr, &child_of_bds, BDRV_CHILD_DATA, errp); + bdrv_graph_wrunlock(); if (child == NULL) { s->next_child_index--; goto out; diff --git a/block/replication.c b/block/replication.c index XXXXXXX..XXXXXXX 100644 --- a/block/replication.c +++ b/block/replication.c @@ -XXX,XX +XXX,XX @@ static void replication_start(ReplicationState *rs, ReplicationMode mode, return; } + bdrv_graph_wrlock(bs); + bdrv_ref(hidden_disk->bs); s->hidden_disk = bdrv_attach_child(bs, hidden_disk->bs, "hidden disk", &child_of_bds, BDRV_CHILD_DATA, &local_err); if (local_err) { error_propagate(errp, local_err); + bdrv_graph_wrunlock(); aio_context_release(aio_context); return; } @@ -XXX,XX +XXX,XX @@ static void replication_start(ReplicationState *rs, ReplicationMode mode, BDRV_CHILD_DATA, &local_err); if (local_err) { error_propagate(errp, local_err); + bdrv_graph_wrunlock(); aio_context_release(aio_context); return; } + bdrv_graph_wrunlock(); + /* start backup job now */ error_setg(&s->blocker, "Block device is in use by internal backup job"); diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-bdrv-drain.c +++ b/tests/unit/test-bdrv-drain.c @@ -XXX,XX +XXX,XX @@ static void do_test_delete_by_drain(bool detach_instead_of_delete, null_bs = bdrv_open("null-co://", NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL, &error_abort); + bdrv_graph_wrlock(NULL); bdrv_attach_child(bs, null_bs, "null-child", &child_of_bds, BDRV_CHILD_DATA, &error_abort); + bdrv_graph_wrunlock(); /* This child will be the one to pass to requests through to, and * it will stall until a drain occurs */ @@ -XXX,XX +XXX,XX @@ static void do_test_delete_by_drain(bool detach_instead_of_delete, &error_abort); child_bs->total_sectors = 65536 >> BDRV_SECTOR_BITS; /* Takes our reference to child_bs */ + bdrv_graph_wrlock(NULL); tts->wait_child = bdrv_attach_child(bs, child_bs, "wait-child", &child_of_bds, BDRV_CHILD_DATA | BDRV_CHILD_PRIMARY, &error_abort); + bdrv_graph_wrunlock(); /* This child is just there to be deleted * (for detach_instead_of_delete == true) */ null_bs = bdrv_open("null-co://", NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL, &error_abort); + bdrv_graph_wrlock(NULL); bdrv_attach_child(bs, null_bs, "null-child", &child_of_bds, BDRV_CHILD_DATA, &error_abort); + bdrv_graph_wrunlock(); blk = blk_new(qemu_get_aio_context(), BLK_PERM_ALL, BLK_PERM_ALL); blk_insert_bs(blk, bs, &error_abort); @@ -XXX,XX +XXX,XX @@ static void detach_indirect_bh(void *opaque) bdrv_unref_child(data->parent_b, data->child_b); bdrv_ref(data->c); + bdrv_graph_wrlock(NULL); data->child_c = bdrv_attach_child(data->parent_b, data->c, "PB-C", &child_of_bds, BDRV_CHILD_DATA, &error_abort); + bdrv_graph_wrunlock(); } static void detach_by_parent_aio_cb(void *opaque, int ret) @@ -XXX,XX +XXX,XX @@ static void test_detach_indirect(bool by_parent_cb) /* Set child relationships */ bdrv_ref(b); bdrv_ref(a); + bdrv_graph_wrlock(NULL); child_b = bdrv_attach_child(parent_b, b, "PB-B", &child_of_bds, BDRV_CHILD_DATA, &error_abort); child_a = bdrv_attach_child(parent_b, a, "PB-A", &child_of_bds, @@ -XXX,XX +XXX,XX @@ static void test_detach_indirect(bool by_parent_cb) bdrv_attach_child(parent_a, a, "PA-A", by_parent_cb ? &child_of_bds : &detach_by_driver_cb_class, BDRV_CHILD_DATA, &error_abort); + bdrv_graph_wrunlock(); g_assert_cmpint(parent_a->refcnt, ==, 1); g_assert_cmpint(parent_b->refcnt, ==, 1); @@ -XXX,XX +XXX,XX @@ static void test_drop_intermediate_poll(void) * Establish the chain last, so the chain links are the first * elements in the BDS.parents lists */ + bdrv_graph_wrlock(NULL); for (i = 0; i < 3; i++) { if (i) { /* Takes the reference to chain[i - 1] */ @@ -XXX,XX +XXX,XX @@ static void test_drop_intermediate_poll(void) &chain_child_class, BDRV_CHILD_COW, &error_abort); } } + bdrv_graph_wrunlock(); job = block_job_create("job", &test_simple_job_driver, NULL, job_node, 0, BLK_PERM_ALL, 0, 0, NULL, NULL, &error_abort); @@ -XXX,XX +XXX,XX @@ static void do_test_replace_child_mid_drain(int old_drain_count, new_child_bs->total_sectors = 1; bdrv_ref(old_child_bs); + bdrv_graph_wrlock(NULL); bdrv_attach_child(parent_bs, old_child_bs, "child", &child_of_bds, BDRV_CHILD_COW, &error_abort); + bdrv_graph_wrunlock(); parent_s->setup_completed = true; for (i = 0; i < old_drain_count; i++) { diff --git a/tests/unit/test-bdrv-graph-mod.c b/tests/unit/test-bdrv-graph-mod.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-bdrv-graph-mod.c +++ b/tests/unit/test-bdrv-graph-mod.c @@ -XXX,XX +XXX,XX @@ static void test_update_perm_tree(void) blk_insert_bs(root, bs, &error_abort); + bdrv_graph_wrlock(NULL); bdrv_attach_child(filter, bs, "child", &child_of_bds, BDRV_CHILD_DATA, &error_abort); + bdrv_graph_wrunlock(); aio_context_acquire(qemu_get_aio_context()); ret = bdrv_append(filter, bs, NULL); @@ -XXX,XX +XXX,XX @@ static void test_should_update_child(void) bdrv_set_backing_hd(target, bs, &error_abort); g_assert(target->backing->bs == bs); + bdrv_graph_wrlock(NULL); bdrv_attach_child(filter, target, "target", &child_of_bds, BDRV_CHILD_DATA, &error_abort); + bdrv_graph_wrunlock(); aio_context_acquire(qemu_get_aio_context()); bdrv_append(filter, bs, &error_abort); aio_context_release(qemu_get_aio_context()); @@ -XXX,XX +XXX,XX @@ static void test_parallel_exclusive_write(void) */ bdrv_ref(base); + bdrv_graph_wrlock(NULL); bdrv_attach_child(top, fl1, "backing", &child_of_bds, BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, &error_abort); @@ -XXX,XX +XXX,XX @@ static void test_parallel_exclusive_write(void) bdrv_attach_child(fl2, base, "backing", &child_of_bds, BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, &error_abort); + bdrv_graph_wrunlock(); bdrv_replace_node(fl1, fl2, &error_abort); @@ -XXX,XX +XXX,XX @@ static void test_parallel_perm_update(void) */ bdrv_ref(base); + bdrv_graph_wrlock(NULL); bdrv_attach_child(top, ws, "file", &child_of_bds, BDRV_CHILD_DATA, &error_abort); c_fl1 = bdrv_attach_child(ws, fl1, "first", &child_of_bds, @@ -XXX,XX +XXX,XX @@ static void test_parallel_perm_update(void) bdrv_attach_child(fl2, base, "backing", &child_of_bds, BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, &error_abort); + bdrv_graph_wrunlock(); /* Select fl1 as first child to be active */ s->selected = c_fl1; @@ -XXX,XX +XXX,XX @@ static void test_append_greedy_filter(void) BlockDriverState *base = no_perm_node("base"); BlockDriverState *fl = exclusive_writer_node("fl1"); + bdrv_graph_wrlock(NULL); bdrv_attach_child(top, base, "backing", &child_of_bds, BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, &error_abort); + bdrv_graph_wrunlock(); aio_context_acquire(qemu_get_aio_context()); bdrv_append(fl, base, &error_abort); -- 2.41.0 The function reads the parents list, so it needs to hold the graph lock. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-14-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block_int-common.h | 6 ++--- include/block/block_int-global-state.h | 8 +++--- include/sysemu/block-backend-global-state.h | 4 +-- block.c | 28 +++++++++++++-------- block/block-backend.c | 26 ++++++++++++++----- block/crypto.c | 6 +++-- block/mirror.c | 8 ++++++ block/vmdk.c | 2 ++ tests/unit/test-bdrv-graph-mod.c | 4 +++ 9 files changed, 66 insertions(+), 26 deletions(-) diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int-common.h +++ b/include/block/block_int-common.h @@ -XXX,XX +XXX,XX @@ struct BlockDriver { */ void (*bdrv_cancel_in_flight)(BlockDriverState *bs); - int (*bdrv_inactivate)(BlockDriverState *bs); + int GRAPH_RDLOCK_PTR (*bdrv_inactivate)(BlockDriverState *bs); int (*bdrv_snapshot_create)(BlockDriverState *bs, QEMUSnapshotInfo *sn_info); @@ -XXX,XX +XXX,XX @@ struct BdrvChildClass { * when migration is completing) and it can start/stop requesting * permissions and doing I/O on it. */ - void (*activate)(BdrvChild *child, Error **errp); - int (*inactivate)(BdrvChild *child); + void GRAPH_RDLOCK_PTR (*activate)(BdrvChild *child, Error **errp); + int GRAPH_RDLOCK_PTR (*inactivate)(BdrvChild *child); void GRAPH_WRLOCK_PTR (*attach)(BdrvChild *child); void GRAPH_WRLOCK_PTR (*detach)(BdrvChild *child); diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -XXX,XX +XXX,XX @@ void bdrv_get_cumulative_perm(BlockDriverState *bs, uint64_t *perm, * bdrv_child_refresh_perms() instead and make the parent's * .bdrv_child_perm() implementation return the correct values. */ -int bdrv_child_try_set_perm(BdrvChild *c, uint64_t perm, uint64_t shared, - Error **errp); +int GRAPH_RDLOCK +bdrv_child_try_set_perm(BdrvChild *c, uint64_t perm, uint64_t shared, + Error **errp); /** * Calls bs->drv->bdrv_child_perm() and updates the child's permission @@ -XXX,XX +XXX,XX @@ int bdrv_child_try_set_perm(BdrvChild *c, uint64_t perm, uint64_t shared, * values than before, but which will not result in the block layer * automatically refreshing the permissions. */ -int bdrv_child_refresh_perms(BlockDriverState *bs, BdrvChild *c, Error **errp); +int GRAPH_RDLOCK +bdrv_child_refresh_perms(BlockDriverState *bs, BdrvChild *c, Error **errp); bool GRAPH_RDLOCK bdrv_recurse_can_replace(BlockDriverState *bs, BlockDriverState *to_replace); diff --git a/include/sysemu/block-backend-global-state.h b/include/sysemu/block-backend-global-state.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/block-backend-global-state.h +++ b/include/sysemu/block-backend-global-state.h @@ -XXX,XX +XXX,XX @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp); int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp); bool bdrv_has_blk(BlockDriverState *bs); bool bdrv_is_root_node(BlockDriverState *bs); -int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm, - Error **errp); +int GRAPH_UNLOCKED blk_set_perm(BlockBackend *blk, uint64_t perm, + uint64_t shared_perm, Error **errp); void blk_get_perm(BlockBackend *blk, uint64_t *perm, uint64_t *shared_perm); void blk_iostatus_enable(BlockBackend *blk); diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static bool bdrv_a_allow_b(BdrvChild *a, BdrvChild *b, Error **errp) return false; } -static bool bdrv_parent_perms_conflict(BlockDriverState *bs, Error **errp) +static bool GRAPH_RDLOCK +bdrv_parent_perms_conflict(BlockDriverState *bs, Error **errp) { BdrvChild *a, *b; GLOBAL_STATE_CODE(); @@ -XXX,XX +XXX,XX @@ static void bdrv_child_perm(BlockDriverState *bs, BlockDriverState *child_bs, * simplest way to satisfy this criteria: use only result of * bdrv_topological_dfs() or NULL as @list parameter. */ -static GSList *bdrv_topological_dfs(GSList *list, GHashTable *found, - BlockDriverState *bs) +static GSList * GRAPH_RDLOCK +bdrv_topological_dfs(GSList *list, GHashTable *found, BlockDriverState *bs) { BdrvChild *child; g_autoptr(GHashTable) local_found = NULL; @@ -XXX,XX +XXX,XX @@ static int bdrv_node_refresh_perm(BlockDriverState *bs, BlockReopenQueue *q, * @list is a product of bdrv_topological_dfs() (may be called several times) - * a topologically sorted subgraph. */ -static int bdrv_do_refresh_perms(GSList *list, BlockReopenQueue *q, - Transaction *tran, Error **errp) +static int GRAPH_RDLOCK +bdrv_do_refresh_perms(GSList *list, BlockReopenQueue *q, Transaction *tran, + Error **errp) { int ret; BlockDriverState *bs; @@ -XXX,XX +XXX,XX @@ static int bdrv_do_refresh_perms(GSList *list, BlockReopenQueue *q, * topologically sorted. It's not a problem if some node occurs in the @list * several times. */ -static int bdrv_list_refresh_perms(GSList *list, BlockReopenQueue *q, - Transaction *tran, Error **errp) +static int GRAPH_RDLOCK +bdrv_list_refresh_perms(GSList *list, BlockReopenQueue *q, Transaction *tran, + Error **errp) { g_autoptr(GHashTable) found = g_hash_table_new(NULL, NULL); g_autoptr(GSList) refresh_list = NULL; @@ -XXX,XX +XXX,XX @@ char *bdrv_perm_names(uint64_t perm) /* @tran is allowed to be NULL. In this case no rollback is possible */ -static int bdrv_refresh_perms(BlockDriverState *bs, Transaction *tran, - Error **errp) +static int GRAPH_RDLOCK +bdrv_refresh_perms(BlockDriverState *bs, Transaction *tran, Error **errp) { int ret; Transaction *local_tran = NULL; @@ -XXX,XX +XXX,XX @@ void bdrv_root_unref_child(BdrvChild *child) GLOBAL_STATE_CODE(); bdrv_graph_wrlock(NULL); bdrv_replace_child_noperm(child, NULL); - bdrv_graph_wrunlock(); bdrv_child_free(child); if (child_bs) { @@ -XXX,XX +XXX,XX @@ void bdrv_root_unref_child(BdrvChild *child) NULL); } + bdrv_graph_wrunlock(); bdrv_unref(child_bs); } @@ -XXX,XX +XXX,XX @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, Error **errp) * reconfiguring the fd and that's why it does it in raw_check_perm(), not * in raw_reopen_prepare() which is called with "old" permissions. */ + bdrv_graph_rdlock_main_loop(); ret = bdrv_list_refresh_perms(refresh_list, bs_queue, tran, errp); + bdrv_graph_rdunlock_main_loop(); + if (ret < 0) { goto abort; } @@ -XXX,XX +XXX,XX @@ int bdrv_activate(BlockDriverState *bs, Error **errp) BdrvDirtyBitmap *bm; GLOBAL_STATE_CODE(); + GRAPH_RDLOCK_GUARD_MAINLOOP(); if (!bs->drv) { return -ENOMEDIUM; @@ -XXX,XX +XXX,XX @@ static int bdrv_inactivate_recurse(BlockDriverState *bs) uint64_t cumulative_perms, cumulative_shared_perms; GLOBAL_STATE_CODE(); + GRAPH_RDLOCK_GUARD_MAINLOOP(); if (!bs->drv) { return -ENOMEDIUM; diff --git a/block/block-backend.c b/block/block-backend.c index XXXXXXX..XXXXXXX 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -XXX,XX +XXX,XX @@ static QTAILQ_HEAD(, BlockBackend) block_backends = static QTAILQ_HEAD(, BlockBackend) monitor_block_backends = QTAILQ_HEAD_INITIALIZER(monitor_block_backends); +static int coroutine_mixed_fn GRAPH_RDLOCK +blk_set_perm_locked(BlockBackend *blk, uint64_t perm, uint64_t shared_perm, + Error **errp); + static void blk_root_inherit_options(BdrvChildRole role, bool parent_is_format, int *child_flags, QDict *child_options, int parent_flags, QDict *parent_options) @@ -XXX,XX +XXX,XX @@ static void blk_vm_state_changed(void *opaque, bool running, RunState state) * * If an error is returned, the VM cannot be allowed to be resumed. */ -static void blk_root_activate(BdrvChild *child, Error **errp) +static void GRAPH_RDLOCK blk_root_activate(BdrvChild *child, Error **errp) { BlockBackend *blk = child->opaque; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static void blk_root_activate(BdrvChild *child, Error **errp) */ saved_shared_perm = blk->shared_perm; - blk_set_perm(blk, blk->perm, BLK_PERM_ALL, &local_err); + blk_set_perm_locked(blk, blk->perm, BLK_PERM_ALL, &local_err); if (local_err) { error_propagate(errp, local_err); blk->disable_perm = true; @@ -XXX,XX +XXX,XX @@ static void blk_root_activate(BdrvChild *child, Error **errp) return; } - blk_set_perm(blk, blk->perm, blk->shared_perm, &local_err); + blk_set_perm_locked(blk, blk->perm, blk->shared_perm, &local_err); if (local_err) { error_propagate(errp, local_err); blk->disable_perm = true; @@ -XXX,XX +XXX,XX @@ static bool blk_can_inactivate(BlockBackend *blk) return blk->force_allow_inactivate; } -static int blk_root_inactivate(BdrvChild *child) +static int GRAPH_RDLOCK blk_root_inactivate(BdrvChild *child) { BlockBackend *blk = child->opaque; @@ -XXX,XX +XXX,XX @@ int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp) /* * Sets the permission bitmasks that the user of the BlockBackend needs. */ -int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm, - Error **errp) +static int coroutine_mixed_fn GRAPH_RDLOCK +blk_set_perm_locked(BlockBackend *blk, uint64_t perm, uint64_t shared_perm, + Error **errp) { int ret; GLOBAL_STATE_CODE(); @@ -XXX,XX +XXX,XX @@ int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm, return 0; } +int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm, + Error **errp) +{ + GLOBAL_STATE_CODE(); + GRAPH_RDLOCK_GUARD_MAINLOOP(); + + return blk_set_perm_locked(blk, perm, shared_perm, errp); +} + void blk_get_perm(BlockBackend *blk, uint64_t *perm, uint64_t *shared_perm) { GLOBAL_STATE_CODE(); diff --git a/block/crypto.c b/block/crypto.c index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -XXX,XX +XXX,XX @@ block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp) return spec_info; } -static int +static int GRAPH_RDLOCK block_crypto_amend_prepare(BlockDriverState *bs, Error **errp) { BlockCrypto *crypto = bs->opaque; @@ -XXX,XX +XXX,XX @@ block_crypto_amend_prepare(BlockDriverState *bs, Error **errp) return ret; } -static void +static void GRAPH_RDLOCK block_crypto_amend_cleanup(BlockDriverState *bs) { BlockCrypto *crypto = bs->opaque; @@ -XXX,XX +XXX,XX @@ block_crypto_amend_options_luks(BlockDriverState *bs, QCryptoBlockAmendOptions *amend_options = NULL; int ret = -EINVAL; + assume_graph_lock(); /* FIXME */ + assert(crypto); assert(crypto->block); diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static int mirror_exit_common(Job *job) * mirror_top_bs from now on, so keep it drained. */ bdrv_drained_begin(mirror_top_bs); bs_opaque->stop = true; + + bdrv_graph_rdlock_main_loop(); bdrv_child_refresh_perms(mirror_top_bs, mirror_top_bs->backing, &error_abort); + bdrv_graph_rdunlock_main_loop(); + if (!abort && s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) { BlockDriverState *backing = s->is_none_mode ? src : s->base; BlockDriverState *unfiltered_target = bdrv_skip_filters(target_bs); @@ -XXX,XX +XXX,XX @@ static BlockJob *mirror_start_job( uint64_t target_perms, target_shared_perms; int ret; + GLOBAL_STATE_CODE(); + if (granularity == 0) { granularity = bdrv_get_default_bitmap_granularity(target); } @@ -XXX,XX +XXX,XX @@ fail: } bs_opaque->stop = true; + bdrv_graph_rdlock_main_loop(); bdrv_child_refresh_perms(mirror_top_bs, mirror_top_bs->backing, &error_abort); + bdrv_graph_rdunlock_main_loop(); bdrv_replace_node(mirror_top_bs, mirror_top_bs->backing->bs, &error_abort); bdrv_unref(mirror_top_bs); diff --git a/block/vmdk.c b/block/vmdk.c index XXXXXXX..XXXXXXX 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -XXX,XX +XXX,XX @@ static int vmdk_open(BlockDriverState *bs, QDict *options, int flags, BDRVVmdkState *s = bs->opaque; uint32_t magic; + GRAPH_RDLOCK_GUARD_MAINLOOP(); + ret = bdrv_open_file_child(NULL, options, "file", bs, errp); if (ret < 0) { return ret; diff --git a/tests/unit/test-bdrv-graph-mod.c b/tests/unit/test-bdrv-graph-mod.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-bdrv-graph-mod.c +++ b/tests/unit/test-bdrv-graph-mod.c @@ -XXX,XX +XXX,XX @@ static void test_parallel_perm_update(void) /* Select fl1 as first child to be active */ s->selected = c_fl1; + + bdrv_graph_rdlock_main_loop(); + bdrv_child_refresh_perms(top, top->children.lh_first, &error_abort); assert(c_fl1->perm & BLK_PERM_WRITE); @@ -XXX,XX +XXX,XX @@ static void test_parallel_perm_update(void) assert(c_fl1->perm & BLK_PERM_WRITE); assert(!(c_fl2->perm & BLK_PERM_WRITE)); + bdrv_graph_rdunlock_main_loop(); bdrv_unref(top); } -- 2.41.0 The function reads the parents list, so it needs to hold the graph lock. This happens to result in BlockDriver.bdrv_set_perm() to be called with the graph lock held. For consistency, make it the same for all of the BlockDriver callbacks for updating permissions and annotate the function pointers with GRAPH_RDLOCK_PTR. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-15-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block_int-common.h | 9 ++++--- include/block/block_int-global-state.h | 4 +-- block.c | 35 ++++++++++++++++++++------ blockdev.c | 6 +++++ 4 files changed, 40 insertions(+), 14 deletions(-) diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int-common.h +++ b/include/block/block_int-common.h @@ -XXX,XX +XXX,XX @@ struct BlockDriver { * If both conditions are met, 0 is returned. Otherwise, -errno is returned * and errp is set to an error describing the conflict. */ - int (*bdrv_check_perm)(BlockDriverState *bs, uint64_t perm, - uint64_t shared, Error **errp); + int GRAPH_RDLOCK_PTR (*bdrv_check_perm)(BlockDriverState *bs, uint64_t perm, + uint64_t shared, Error **errp); /** * Called to inform the driver that the set of cumulative set of used @@ -XXX,XX +XXX,XX @@ struct BlockDriver { * This function is only invoked after bdrv_check_perm(), so block drivers * may rely on preparations made in their .bdrv_check_perm implementation. */ - void (*bdrv_set_perm)(BlockDriverState *bs, uint64_t perm, uint64_t shared); + void GRAPH_RDLOCK_PTR (*bdrv_set_perm)( + BlockDriverState *bs, uint64_t perm, uint64_t shared); /* * Called to inform the driver that after a previous bdrv_check_perm() @@ -XXX,XX +XXX,XX @@ struct BlockDriver { * This function can be called even for nodes that never saw a * bdrv_check_perm() call. It is a no-op then. */ - void (*bdrv_abort_perm_update)(BlockDriverState *bs); + void GRAPH_RDLOCK_PTR (*bdrv_abort_perm_update)(BlockDriverState *bs); /** * Returns in @nperm and @nshared the permissions that the driver for @bs diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs, void *opaque, Error **errp); void bdrv_root_unref_child(BdrvChild *child); -void bdrv_get_cumulative_perm(BlockDriverState *bs, uint64_t *perm, - uint64_t *shared_perm); +void GRAPH_RDLOCK bdrv_get_cumulative_perm(BlockDriverState *bs, uint64_t *perm, + uint64_t *shared_perm); /** * Sets a BdrvChild's permissions. Avoid if the parent is a BDS; use diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static void bdrv_child_set_perm(BdrvChild *c, uint64_t perm, tran_add(tran, &bdrv_child_set_pem_drv, s); } -static void bdrv_drv_set_perm_commit(void *opaque) +static void GRAPH_RDLOCK bdrv_drv_set_perm_commit(void *opaque) { BlockDriverState *bs = opaque; uint64_t cumulative_perms, cumulative_shared_perms; @@ -XXX,XX +XXX,XX @@ static void bdrv_drv_set_perm_commit(void *opaque) } } -static void bdrv_drv_set_perm_abort(void *opaque) +static void GRAPH_RDLOCK bdrv_drv_set_perm_abort(void *opaque) { BlockDriverState *bs = opaque; GLOBAL_STATE_CODE(); @@ -XXX,XX +XXX,XX @@ TransactionActionDrv bdrv_drv_set_perm_drv = { .commit = bdrv_drv_set_perm_commit, }; -static int bdrv_drv_set_perm(BlockDriverState *bs, uint64_t perm, - uint64_t shared_perm, Transaction *tran, - Error **errp) +/* + * After calling this function, the transaction @tran may only be completed + * while holding a reader lock for the graph. + */ +static int GRAPH_RDLOCK +bdrv_drv_set_perm(BlockDriverState *bs, uint64_t perm, uint64_t shared_perm, + Transaction *tran, Error **errp) { GLOBAL_STATE_CODE(); if (!bs->drv) { @@ -XXX,XX +XXX,XX @@ bdrv_replace_child_tran(BdrvChild *child, BlockDriverState *new_bs, /* * Refresh permissions in @bs subtree. The function is intended to be called * after some graph modification that was done without permission update. + * + * After calling this function, the transaction @tran may only be completed + * while holding a reader lock for the graph. */ -static int bdrv_node_refresh_perm(BlockDriverState *bs, BlockReopenQueue *q, - Transaction *tran, Error **errp) +static int GRAPH_RDLOCK +bdrv_node_refresh_perm(BlockDriverState *bs, BlockReopenQueue *q, + Transaction *tran, Error **errp) { BlockDriver *drv = bs->drv; BdrvChild *c; @@ -XXX,XX +XXX,XX @@ static int bdrv_node_refresh_perm(BlockDriverState *bs, BlockReopenQueue *q, /* * @list is a product of bdrv_topological_dfs() (may be called several times) - * a topologically sorted subgraph. + * + * After calling this function, the transaction @tran may only be completed + * while holding a reader lock for the graph. */ static int GRAPH_RDLOCK bdrv_do_refresh_perms(GSList *list, BlockReopenQueue *q, Transaction *tran, @@ -XXX,XX +XXX,XX @@ bdrv_do_refresh_perms(GSList *list, BlockReopenQueue *q, Transaction *tran, * @list is any list of nodes. List is completed by all subtrees and * topologically sorted. It's not a problem if some node occurs in the @list * several times. + * + * After calling this function, the transaction @tran may only be completed + * while holding a reader lock for the graph. */ static int GRAPH_RDLOCK bdrv_list_refresh_perms(GSList *list, BlockReopenQueue *q, Transaction *tran, @@ -XXX,XX +XXX,XX @@ char *bdrv_perm_names(uint64_t perm) } -/* @tran is allowed to be NULL. In this case no rollback is possible */ +/* + * @tran is allowed to be NULL. In this case no rollback is possible. + * + * After calling this function, the transaction @tran may only be completed + * while holding a reader lock for the graph. + */ static int GRAPH_RDLOCK bdrv_refresh_perms(BlockDriverState *bs, Transaction *tran, Error **errp) { diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ static void external_snapshot_action(TransactionAction *action, AioContext *aio_context; uint64_t perm, shared; + /* TODO We'll eventually have to take a writer lock in this function */ + GRAPH_RDLOCK_GUARD_MAINLOOP(); + tran_add(tran, &external_snapshot_drv, state); /* 'blockdev-snapshot' and 'blockdev-snapshot-sync' have similar @@ -XXX,XX +XXX,XX @@ void qmp_block_commit(const char *job_id, const char *device, int job_flags = JOB_DEFAULT; uint64_t top_perm, top_shared; + /* TODO We'll eventually have to take a writer lock in this function */ + GRAPH_RDLOCK_GUARD_MAINLOOP(); + if (!has_speed) { speed = 0; } -- 2.41.0 This adds GRAPH_RDLOCK annotations to declare that callers of bdrv_child_perm() need to hold a reader lock for the graph because some implementations access the children list of a node. The callers of bdrv_child_perm() conveniently already hold the lock. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-16-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block_int-common.h | 10 +++++----- block.c | 11 ++++++----- block/copy-before-write.c | 10 +++++----- 3 files changed, 16 insertions(+), 15 deletions(-) diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int-common.h +++ b/include/block/block_int-common.h @@ -XXX,XX +XXX,XX @@ struct BlockDriver { * permissions, but those that will be needed after applying the * @reopen_queue. */ - void (*bdrv_child_perm)(BlockDriverState *bs, BdrvChild *c, - BdrvChildRole role, - BlockReopenQueue *reopen_queue, - uint64_t parent_perm, uint64_t parent_shared, - uint64_t *nperm, uint64_t *nshared); + void GRAPH_RDLOCK_PTR (*bdrv_child_perm)( + BlockDriverState *bs, BdrvChild *c, BdrvChildRole role, + BlockReopenQueue *reopen_queue, + uint64_t parent_perm, uint64_t parent_shared, + uint64_t *nperm, uint64_t *nshared); /** * Register/unregister a buffer for I/O. For example, when the driver is diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ bdrv_parent_perms_conflict(BlockDriverState *bs, Error **errp) return false; } -static void bdrv_child_perm(BlockDriverState *bs, BlockDriverState *child_bs, - BdrvChild *c, BdrvChildRole role, - BlockReopenQueue *reopen_queue, - uint64_t parent_perm, uint64_t parent_shared, - uint64_t *nperm, uint64_t *nshared) +static void GRAPH_RDLOCK +bdrv_child_perm(BlockDriverState *bs, BlockDriverState *child_bs, + BdrvChild *c, BdrvChildRole role, + BlockReopenQueue *reopen_queue, + uint64_t parent_perm, uint64_t parent_shared, + uint64_t *nperm, uint64_t *nshared) { assert(bs->drv && bs->drv->bdrv_child_perm); GLOBAL_STATE_CODE(); diff --git a/block/copy-before-write.c b/block/copy-before-write.c index XXXXXXX..XXXXXXX 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -XXX,XX +XXX,XX @@ static void cbw_refresh_filename(BlockDriverState *bs) bs->file->bs->filename); } -static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, - BdrvChildRole role, - BlockReopenQueue *reopen_queue, - uint64_t perm, uint64_t shared, - uint64_t *nperm, uint64_t *nshared) +static void GRAPH_RDLOCK +cbw_child_perm(BlockDriverState *bs, BdrvChild *c, BdrvChildRole role, + BlockReopenQueue *reopen_queue, + uint64_t perm, uint64_t shared, + uint64_t *nperm, uint64_t *nshared) { if (!(role & BDRV_CHILD_FILTERED)) { /* -- 2.41.0 The function reads the parents list, so it needs to hold the graph lock. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Message-ID: <20230911094620.45040-18-kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ int bdrv_drop_intermediate(BlockDriverState *top, BlockDriverState *base, backing_file_str = base->filename; } + bdrv_graph_rdlock_main_loop(); QLIST_FOREACH(c, &top->parents, next_parent) { updated_children = g_slist_prepend(updated_children, c); } + bdrv_graph_rdunlock_main_loop(); /* * It seems correct to pass detach_subchain=true here, but it triggers -- 2.41.0 The function reads the parents list, so it needs to hold the graph lock. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Message-ID: <20230911094620.45040-19-kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static bool bdrv_change_aio_context(BlockDriverState *bs, AioContext *ctx, return true; } + bdrv_graph_rdlock_main_loop(); QLIST_FOREACH(c, &bs->parents, next_parent) { if (!bdrv_parent_change_aio_context(c, ctx, visited, tran, errp)) { + bdrv_graph_rdunlock_main_loop(); return false; } } QLIST_FOREACH(c, &bs->children, next) { if (!bdrv_child_change_aio_context(c, ctx, visited, tran, errp)) { + bdrv_graph_rdunlock_main_loop(); return false; } } + bdrv_graph_rdunlock_main_loop(); state = g_new(BdrvStateSetAioContext, 1); *state = (BdrvStateSetAioContext) { -- 2.41.0 Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_root_unref_child(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-20-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block_int-global-state.h | 2 +- block.c | 6 +++--- block/block-backend.c | 3 +++ blockjob.c | 2 ++ 4 files changed, 9 insertions(+), 4 deletions(-) diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -XXX,XX +XXX,XX @@ BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs, BdrvChildRole child_role, uint64_t perm, uint64_t shared_perm, void *opaque, Error **errp); -void bdrv_root_unref_child(BdrvChild *child); +void GRAPH_WRLOCK bdrv_root_unref_child(BdrvChild *child); void GRAPH_RDLOCK bdrv_get_cumulative_perm(BlockDriverState *bs, uint64_t *perm, uint64_t *shared_perm); diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ void bdrv_root_unref_child(BdrvChild *child) BlockDriverState *child_bs = child->bs; GLOBAL_STATE_CODE(); - bdrv_graph_wrlock(NULL); bdrv_replace_child_noperm(child, NULL); bdrv_child_free(child); @@ -XXX,XX +XXX,XX @@ void bdrv_root_unref_child(BdrvChild *child) NULL); } - bdrv_graph_wrunlock(); - bdrv_unref(child_bs); + bdrv_schedule_unref(child_bs); } typedef struct BdrvSetInheritsFrom { @@ -XXX,XX +XXX,XX @@ void bdrv_unref_child(BlockDriverState *parent, BdrvChild *child) return; } + bdrv_graph_wrlock(NULL); bdrv_unset_inherits_from(parent, child, NULL); bdrv_root_unref_child(child); + bdrv_graph_wrunlock(); } diff --git a/block/block-backend.c b/block/block-backend.c index XXXXXXX..XXXXXXX 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -XXX,XX +XXX,XX @@ void blk_remove_bs(BlockBackend *blk) blk_drain(blk); root = blk->root; blk->root = NULL; + + bdrv_graph_wrlock(NULL); bdrv_root_unref_child(root); + bdrv_graph_wrunlock(); } /* diff --git a/blockjob.c b/blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/blockjob.c +++ b/blockjob.c @@ -XXX,XX +XXX,XX @@ void block_job_remove_all_bdrv(BlockJob *job) * one to make sure that such a concurrent access does not attempt * to process an already freed BdrvChild. */ + bdrv_graph_wrlock(NULL); while (job->nodes) { GSList *l = job->nodes; BdrvChild *c = l->data; @@ -XXX,XX +XXX,XX @@ void block_job_remove_all_bdrv(BlockJob *job) g_slist_free_1(l); } + bdrv_graph_wrunlock(); } bool block_job_has_bdrv(BlockJob *job, BlockDriverState *bs) -- 2.41.0 Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_unref_child(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-21-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block-global-state.h | 7 ++++++- block.c | 11 +++++++---- block/blklogwrites.c | 4 ++++ block/blkverify.c | 2 ++ block/qcow2.c | 4 +++- block/quorum.c | 6 ++++++ block/replication.c | 3 +++ block/snapshot.c | 2 ++ block/vmdk.c | 11 +++++++++++ tests/unit/test-bdrv-drain.c | 8 ++++++-- 10 files changed, 50 insertions(+), 8 deletions(-) diff --git a/include/block/block-global-state.h b/include/block/block-global-state.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block-global-state.h +++ b/include/block/block-global-state.h @@ -XXX,XX +XXX,XX @@ void bdrv_ref(BlockDriverState *bs); void no_coroutine_fn bdrv_unref(BlockDriverState *bs); void coroutine_fn no_co_wrapper bdrv_co_unref(BlockDriverState *bs); void GRAPH_WRLOCK bdrv_schedule_unref(BlockDriverState *bs); -void bdrv_unref_child(BlockDriverState *parent, BdrvChild *child); + +void GRAPH_WRLOCK +bdrv_unref_child(BlockDriverState *parent, BdrvChild *child); + +void coroutine_fn no_co_wrapper_bdrv_wrlock +bdrv_co_unref_child(BlockDriverState *parent, BdrvChild *child); BdrvChild * GRAPH_WRLOCK bdrv_attach_child(BlockDriverState *parent_bs, diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ bdrv_open_driver(BlockDriverState *bs, BlockDriver *drv, const char *node_name, open_failed: bs->drv = NULL; if (bs->file != NULL) { + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, bs->file); + bdrv_graph_wrunlock(); assert(!bs->file); } g_free(bs->opaque); @@ -XXX,XX +XXX,XX @@ static void bdrv_set_inherits_from(BlockDriverState *bs, * @root that point to @root, where necessary. * @tran is allowed to be NULL. In this case no rollback is possible */ -static void bdrv_unset_inherits_from(BlockDriverState *root, BdrvChild *child, - Transaction *tran) +static void GRAPH_WRLOCK +bdrv_unset_inherits_from(BlockDriverState *root, BdrvChild *child, + Transaction *tran) { BdrvChild *c; @@ -XXX,XX +XXX,XX @@ void bdrv_unref_child(BlockDriverState *parent, BdrvChild *child) return; } - bdrv_graph_wrlock(NULL); bdrv_unset_inherits_from(parent, child, NULL); bdrv_root_unref_child(child); - bdrv_graph_wrunlock(); } @@ -XXX,XX +XXX,XX @@ static void bdrv_close(BlockDriverState *bs) bs->drv = NULL; } + bdrv_graph_wrlock(NULL); QLIST_FOREACH_SAFE(child, &bs->children, next, next) { bdrv_unref_child(bs, child); } + bdrv_graph_wrunlock(); assert(!bs->backing); assert(!bs->file); diff --git a/block/blklogwrites.c b/block/blklogwrites.c index XXXXXXX..XXXXXXX 100644 --- a/block/blklogwrites.c +++ b/block/blklogwrites.c @@ -XXX,XX +XXX,XX @@ static int blk_log_writes_open(BlockDriverState *bs, QDict *options, int flags, ret = 0; fail_log: if (ret < 0) { + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, s->log_file); + bdrv_graph_wrunlock(); s->log_file = NULL; } fail: @@ -XXX,XX +XXX,XX @@ static void blk_log_writes_close(BlockDriverState *bs) { BDRVBlkLogWritesState *s = bs->opaque; + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, s->log_file); s->log_file = NULL; + bdrv_graph_wrunlock(); } static int64_t coroutine_fn GRAPH_RDLOCK diff --git a/block/blkverify.c b/block/blkverify.c index XXXXXXX..XXXXXXX 100644 --- a/block/blkverify.c +++ b/block/blkverify.c @@ -XXX,XX +XXX,XX @@ static void blkverify_close(BlockDriverState *bs) { BDRVBlkverifyState *s = bs->opaque; + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, s->test_file); s->test_file = NULL; + bdrv_graph_wrunlock(); } static int64_t coroutine_fn GRAPH_RDLOCK diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, g_free(s->image_data_file); if (open_data_file && has_data_file(bs)) { bdrv_graph_co_rdunlock(); - bdrv_unref_child(bs, s->data_file); + bdrv_co_unref_child(bs, s->data_file); bdrv_graph_co_rdlock(); s->data_file = NULL; } @@ -XXX,XX +XXX,XX @@ static void qcow2_do_close(BlockDriverState *bs, bool close_data_file) g_free(s->image_backing_format); if (close_data_file && has_data_file(bs)) { + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, s->data_file); + bdrv_graph_wrunlock(); s->data_file = NULL; } diff --git a/block/quorum.c b/block/quorum.c index XXXXXXX..XXXXXXX 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -XXX,XX +XXX,XX @@ static int quorum_open(BlockDriverState *bs, QDict *options, int flags, close_exit: /* cleanup on error */ + bdrv_graph_wrlock(NULL); for (i = 0; i < s->num_children; i++) { if (!opened[i]) { continue; } bdrv_unref_child(bs, s->children[i]); } + bdrv_graph_wrunlock(); g_free(s->children); g_free(opened); exit: @@ -XXX,XX +XXX,XX @@ static void quorum_close(BlockDriverState *bs) BDRVQuorumState *s = bs->opaque; int i; + bdrv_graph_wrlock(NULL); for (i = 0; i < s->num_children; i++) { bdrv_unref_child(bs, s->children[i]); } + bdrv_graph_wrunlock(); g_free(s->children); } @@ -XXX,XX +XXX,XX @@ static void quorum_del_child(BlockDriverState *bs, BdrvChild *child, memmove(&s->children[i], &s->children[i + 1], (s->num_children - i - 1) * sizeof(BdrvChild *)); s->children = g_renew(BdrvChild *, s->children, --s->num_children); + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, child); + bdrv_graph_wrunlock(); quorum_refresh_flags(bs); bdrv_drained_end(bs); diff --git a/block/replication.c b/block/replication.c index XXXXXXX..XXXXXXX 100644 --- a/block/replication.c +++ b/block/replication.c @@ -XXX,XX +XXX,XX @@ static void replication_done(void *opaque, int ret) if (ret == 0) { s->stage = BLOCK_REPLICATION_DONE; + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, s->secondary_disk); s->secondary_disk = NULL; bdrv_unref_child(bs, s->hidden_disk); s->hidden_disk = NULL; + bdrv_graph_wrunlock(); + s->error = 0; } else { s->stage = BLOCK_REPLICATION_FAILOVER_FAILED; diff --git a/block/snapshot.c b/block/snapshot.c index XXXXXXX..XXXXXXX 100644 --- a/block/snapshot.c +++ b/block/snapshot.c @@ -XXX,XX +XXX,XX @@ int bdrv_snapshot_goto(BlockDriverState *bs, } /* .bdrv_open() will re-attach it */ + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, fallback); + bdrv_graph_wrunlock(); ret = bdrv_snapshot_goto(fallback_bs, snapshot_id, errp); open_ret = drv->bdrv_open(bs, options, bs->open_flags, &local_err); diff --git a/block/vmdk.c b/block/vmdk.c index XXXXXXX..XXXXXXX 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -XXX,XX +XXX,XX @@ static void vmdk_free_extents(BlockDriverState *bs) BDRVVmdkState *s = bs->opaque; VmdkExtent *e; + bdrv_graph_wrlock(NULL); for (i = 0; i < s->num_extents; i++) { e = &s->extents[i]; g_free(e->l1_table); @@ -XXX,XX +XXX,XX @@ static void vmdk_free_extents(BlockDriverState *bs) bdrv_unref_child(bs, e->file); } } + bdrv_graph_wrunlock(); + g_free(s->extents); } @@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs, ret = vmdk_add_extent(bs, extent_file, true, sectors, 0, 0, 0, 0, 0, &extent, errp); if (ret < 0) { + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, extent_file); + bdrv_graph_wrunlock(); goto out; } extent->flat_start_offset = flat_offset << 9; @@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs, } g_free(buf); if (ret) { + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, extent_file); + bdrv_graph_wrunlock(); goto out; } extent = &s->extents[s->num_extents - 1]; } else if (!strcmp(type, "SESPARSE")) { ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp); if (ret) { + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, extent_file); + bdrv_graph_wrunlock(); goto out; } extent = &s->extents[s->num_extents - 1]; } else { error_setg(errp, "Unsupported extent type '%s'", type); + bdrv_graph_wrlock(NULL); bdrv_unref_child(bs, extent_file); + bdrv_graph_wrunlock(); ret = -ENOTSUP; goto out; } diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-bdrv-drain.c +++ b/tests/unit/test-bdrv-drain.c @@ -XXX,XX +XXX,XX @@ typedef struct BDRVTestTopState { static void bdrv_test_top_close(BlockDriverState *bs) { BdrvChild *c, *next_c; + + bdrv_graph_wrlock(NULL); QLIST_FOREACH_SAFE(c, &bs->children, next, next_c) { bdrv_unref_child(bs, c); } + bdrv_graph_wrunlock(); } static int coroutine_fn GRAPH_RDLOCK @@ -XXX,XX +XXX,XX @@ static void coroutine_fn test_co_delete_by_drain(void *opaque) } else { BdrvChild *c, *next_c; QLIST_FOREACH_SAFE(c, &bs->children, next, next_c) { - bdrv_unref_child(bs, c); + bdrv_co_unref_child(bs, c); } } @@ -XXX,XX +XXX,XX @@ static void detach_indirect_bh(void *opaque) struct detach_by_parent_data *data = opaque; bdrv_dec_in_flight(data->child_b->bs); + + bdrv_graph_wrlock(NULL); bdrv_unref_child(data->parent_b, data->child_b); bdrv_ref(data->c); - bdrv_graph_wrlock(NULL); data->child_c = bdrv_attach_child(data->parent_b, data->c, "PB-C", &child_of_bds, BDRV_CHILD_DATA, &error_abort); -- 2.41.0 The functions read the parents list in the generic block layer, so we need to hold the graph lock already there. The BlockDriver implementations actually modify the graph, so it has to be a writer lock. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-22-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block-global-state.h | 8 +++++--- include/block/block_int-common.h | 9 +++++---- block/quorum.c | 23 ++++++----------------- blockdev.c | 17 +++++++++++------ 4 files changed, 27 insertions(+), 30 deletions(-) diff --git a/include/block/block-global-state.h b/include/block/block-global-state.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block-global-state.h +++ b/include/block/block-global-state.h @@ -XXX,XX +XXX,XX @@ int bdrv_try_change_aio_context(BlockDriverState *bs, AioContext *ctx, int bdrv_probe_blocksizes(BlockDriverState *bs, BlockSizes *bsz); int bdrv_probe_geometry(BlockDriverState *bs, HDGeometry *geo); -void bdrv_add_child(BlockDriverState *parent, BlockDriverState *child, - Error **errp); -void bdrv_del_child(BlockDriverState *parent, BdrvChild *child, Error **errp); +void GRAPH_WRLOCK +bdrv_add_child(BlockDriverState *parent, BlockDriverState *child, Error **errp); + +void GRAPH_WRLOCK +bdrv_del_child(BlockDriverState *parent, BdrvChild *child, Error **errp); /** * diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int-common.h +++ b/include/block/block_int-common.h @@ -XXX,XX +XXX,XX @@ struct BlockDriver { */ int (*bdrv_probe_geometry)(BlockDriverState *bs, HDGeometry *geo); - void (*bdrv_add_child)(BlockDriverState *parent, BlockDriverState *child, - Error **errp); - void (*bdrv_del_child)(BlockDriverState *parent, BdrvChild *child, - Error **errp); + void GRAPH_WRLOCK_PTR (*bdrv_add_child)( + BlockDriverState *parent, BlockDriverState *child, Error **errp); + + void GRAPH_WRLOCK_PTR (*bdrv_del_child)( + BlockDriverState *parent, BdrvChild *child, Error **errp); /** * Informs the block driver that a permission change is intended. The diff --git a/block/quorum.c b/block/quorum.c index XXXXXXX..XXXXXXX 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -XXX,XX +XXX,XX @@ static void quorum_close(BlockDriverState *bs) g_free(s->children); } -static void quorum_add_child(BlockDriverState *bs, BlockDriverState *child_bs, - Error **errp) +static void GRAPH_WRLOCK +quorum_add_child(BlockDriverState *bs, BlockDriverState *child_bs, Error **errp) { BDRVQuorumState *s = bs->opaque; BdrvChild *child; @@ -XXX,XX +XXX,XX @@ static void quorum_add_child(BlockDriverState *bs, BlockDriverState *child_bs, } s->next_child_index++; - bdrv_drained_begin(bs); - /* We can safely add the child now */ bdrv_ref(child_bs); - bdrv_graph_wrlock(child_bs); child = bdrv_attach_child(bs, child_bs, indexstr, &child_of_bds, BDRV_CHILD_DATA, errp); - bdrv_graph_wrunlock(); if (child == NULL) { s->next_child_index--; - goto out; + return; } s->children = g_renew(BdrvChild *, s->children, s->num_children + 1); s->children[s->num_children++] = child; quorum_refresh_flags(bs); - -out: - bdrv_drained_end(bs); } -static void quorum_del_child(BlockDriverState *bs, BdrvChild *child, - Error **errp) +static void GRAPH_WRLOCK +quorum_del_child(BlockDriverState *bs, BdrvChild *child, Error **errp) { BDRVQuorumState *s = bs->opaque; char indexstr[INDEXSTR_LEN]; @@ -XXX,XX +XXX,XX @@ static void quorum_del_child(BlockDriverState *bs, BdrvChild *child, s->next_child_index--; } - bdrv_drained_begin(bs); - /* We can safely remove this child now */ memmove(&s->children[i], &s->children[i + 1], (s->num_children - i - 1) * sizeof(BdrvChild *)); s->children = g_renew(BdrvChild *, s->children, --s->num_children); - bdrv_graph_wrlock(NULL); + bdrv_unref_child(bs, child); - bdrv_graph_wrunlock(); quorum_refresh_flags(bs); - bdrv_drained_end(bs); } static void quorum_gather_child_options(BlockDriverState *bs, QDict *target, diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ out: aio_context_release(aio_context); } -static BdrvChild *bdrv_find_child(BlockDriverState *parent_bs, - const char *child_name) +static BdrvChild * GRAPH_RDLOCK +bdrv_find_child(BlockDriverState *parent_bs, const char *child_name) { BdrvChild *child; @@ -XXX,XX +XXX,XX @@ void qmp_x_blockdev_change(const char *parent, const char *child, BlockDriverState *parent_bs, *new_bs = NULL; BdrvChild *p_child; + bdrv_graph_wrlock(NULL); + parent_bs = bdrv_lookup_bs(parent, parent, errp); if (!parent_bs) { - return; + goto out; } if (!child == !node) { @@ -XXX,XX +XXX,XX @@ void qmp_x_blockdev_change(const char *parent, const char *child, } else { error_setg(errp, "Either child or node must be specified"); } - return; + goto out; } if (child) { @@ -XXX,XX +XXX,XX @@ void qmp_x_blockdev_change(const char *parent, const char *child, if (!p_child) { error_setg(errp, "Node '%s' does not have child '%s'", parent, child); - return; + goto out; } bdrv_del_child(parent_bs, p_child, errp); } @@ -XXX,XX +XXX,XX @@ void qmp_x_blockdev_change(const char *parent, const char *child, new_bs = bdrv_find_node(node); if (!new_bs) { error_setg(errp, "Node '%s' not found", node); - return; + goto out; } bdrv_add_child(parent_bs, new_bs, errp); } + +out: + bdrv_graph_wrunlock(); } BlockJobInfoList *qmp_query_block_jobs(Error **errp) -- 2.41.0 From: Andrey Drobyshev via <qemu-block@nongnu.org> Functions qcow2_get_host_offset(), get_cluster_offset(), vmdk_co_block_status() explicitly report compressed cluster types when data is compressed. However, this information is never passed further. Let's make use of it by adding new BDRV_BLOCK_COMPRESSED flag for bdrv_block_status(), so that caller may know that the data range is compressed. In particular, we're going to use this flag to tweak "qemu-img map" output. This new flag is only being utilized by qcow, qcow2 and vmdk formats, as only those support compression. Reviewed-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Hanna Czenczek <hreitz@redhat.com> Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> Message-ID: <20230907210226.953821-2-andrey.drobyshev@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block-common.h | 3 +++ block/qcow.c | 5 ++++- block/qcow2.c | 3 +++ block/vmdk.c | 2 ++ 4 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/block/block-common.h b/include/block/block-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block-common.h +++ b/include/block/block-common.h @@ -XXX,XX +XXX,XX @@ typedef enum { * layer rather than any backing, set by block layer * BDRV_BLOCK_EOF: the returned pnum covers through end of file for this * layer, set by block layer + * BDRV_BLOCK_COMPRESSED: the underlying data is compressed; only valid for + * the formats supporting compression: qcow, qcow2 * * Internal flags: * BDRV_BLOCK_RAW: for use by passthrough drivers, such as raw, to request @@ -XXX,XX +XXX,XX @@ typedef enum { #define BDRV_BLOCK_ALLOCATED 0x10 #define BDRV_BLOCK_EOF 0x20 #define BDRV_BLOCK_RECURSE 0x40 +#define BDRV_BLOCK_COMPRESSED 0x80 typedef QTAILQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry) BlockReopenQueue; diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ qcow_co_block_status(BlockDriverState *bs, bool want_zero, if (!cluster_offset) { return 0; } - if ((cluster_offset & QCOW_OFLAG_COMPRESSED) || s->crypto) { + if (cluster_offset & QCOW_OFLAG_COMPRESSED) { + return BDRV_BLOCK_DATA | BDRV_BLOCK_COMPRESSED; + } + if (s->crypto) { return BDRV_BLOCK_DATA; } *map = cluster_offset | index_in_cluster; diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ qcow2_co_block_status(BlockDriverState *bs, bool want_zero, int64_t offset, { status |= BDRV_BLOCK_RECURSE; } + if (type == QCOW2_SUBCLUSTER_COMPRESSED) { + status |= BDRV_BLOCK_COMPRESSED; + } return status; } diff --git a/block/vmdk.c b/block/vmdk.c index XXXXXXX..XXXXXXX 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -XXX,XX +XXX,XX @@ vmdk_co_block_status(BlockDriverState *bs, bool want_zero, if (extent->flat) { ret |= BDRV_BLOCK_RECURSE; } + } else { + ret |= BDRV_BLOCK_COMPRESSED; } *file = extent->file->bs; break; -- 2.41.0 From: Andrey Drobyshev via <qemu-block@nongnu.org> Right now "qemu-img map" reports compressed blocks as containing data but having no host offset. This is not very informative. Instead, let's add another boolean field named "compressed" in case JSON output mode is specified. This is achieved by utilizing new allocation status flag BDRV_BLOCK_COMPRESSED for bdrv_block_status(). Also update the expected qemu-iotests outputs to contain the new field. Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> Message-ID: <20230907210226.953821-3-andrey.drobyshev@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 6 +- qemu-img.c | 8 +- tests/qemu-iotests/122.out | 84 +- tests/qemu-iotests/146.out | 780 +++++++++--------- tests/qemu-iotests/154.out | 194 ++--- tests/qemu-iotests/179.out | 178 ++-- tests/qemu-iotests/209.out | 4 +- tests/qemu-iotests/221.out | 16 +- tests/qemu-iotests/223.out | 60 +- tests/qemu-iotests/241.out | 10 +- tests/qemu-iotests/244.out | 24 +- tests/qemu-iotests/252.out | 10 +- tests/qemu-iotests/253.out | 20 +- tests/qemu-iotests/274.out | 48 +- .../tests/nbd-qemu-allocation.out | 16 +- tests/qemu-iotests/tests/qemu-img-bitmaps.out | 24 +- 16 files changed, 744 insertions(+), 738 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # # @zero: whether the virtual blocks read as zeroes # +# @compressed: true if the data is stored compressed (since 8.2) +# # @depth: number of layers (0 = top image, 1 = top image's backing # file, ..., n - 1 = bottom image (where n is the number of images # in the chain)) before reaching one for which the range is @@ -XXX,XX +XXX,XX @@ ## { 'struct': 'MapEntry', 'data': {'start': 'int', 'length': 'int', 'data': 'bool', - 'zero': 'bool', 'depth': 'int', 'present': 'bool', - '*offset': 'int', '*filename': 'str' } } + 'zero': 'bool', 'compressed': 'bool', 'depth': 'int', + 'present': 'bool', '*offset': 'int', '*filename': 'str' } } ## # @BlockdevCacheInfo: diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static int dump_map_entry(OutputFormat output_format, MapEntry *e, case OFORMAT_JSON: printf("{ \"start\": %"PRId64", \"length\": %"PRId64"," " \"depth\": %"PRId64", \"present\": %s, \"zero\": %s," - " \"data\": %s", e->start, e->length, e->depth, + " \"data\": %s, \"compressed\": %s", + e->start, e->length, e->depth, e->present ? "true" : "false", e->zero ? "true" : "false", - e->data ? "true" : "false"); + e->data ? "true" : "false", + e->compressed ? "true" : "false"); if (e->has_offset) { printf(", \"offset\": %"PRId64"", e->offset); } @@ -XXX,XX +XXX,XX @@ static int get_block_status(BlockDriverState *bs, int64_t offset, .length = bytes, .data = !!(ret & BDRV_BLOCK_DATA), .zero = !!(ret & BDRV_BLOCK_ZERO), + .compressed = !!(ret & BDRV_BLOCK_COMPRESSED), .offset = map, .has_offset = has_offset, .depth = depth, @@ -XXX,XX +XXX,XX @@ static inline bool entry_mergeable(const MapEntry *curr, const MapEntry *next) } if (curr->zero != next->zero || curr->data != next->data || + curr->compressed != next->compressed || curr->depth != next->depth || curr->present != next->present || !curr->filename != !next->filename || diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/122.out +++ b/tests/qemu-iotests/122.out @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 4194304 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 65536/65536 bytes at offset 8388608 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 65536, "length": 4128768, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 4194304, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 4259840, "length": 4128768, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 8388608, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 8454144, "length": 4128768, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 65536, "length": 4128768, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 4194304, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 4259840, "length": 4128768, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 8388608, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 8454144, "length": 4128768, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 65536/65536 bytes at offset 4194304 @@ -XXX,XX +XXX,XX @@ wrote 1024/1024 bytes at offset 1046528 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 1024/1024 bytes at offset 0 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 65536, "length": 65536, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 131072, "length": 196608, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 327680, "length": 655360, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 983040, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 1048576, "length": 1046528, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 65536, "length": 65536, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 131072, "length": 196608, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 327680, "length": 655360, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 983040, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 1048576, "length": 1046528, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] read 16384/16384 bytes at offset 0 16 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 16384/16384 bytes at offset 16384 @@ -XXX,XX +XXX,XX @@ read 3145728/3145728 bytes at offset 0 3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 63963136/63963136 bytes at offset 3145728 61 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] convert -c -S 0: read 3145728/3145728 bytes at offset 0 3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 63963136/63963136 bytes at offset 3145728 61 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}] Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 wrote 33554432/33554432 bytes at offset 0 32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ read 30408704/30408704 bytes at offset 3145728 29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 33554432/33554432 bytes at offset 33554432 32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] convert -c -S 0 with source backing file: read 3145728/3145728 bytes at offset 0 @@ -XXX,XX +XXX,XX @@ read 30408704/30408704 bytes at offset 3145728 29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 33554432/33554432 bytes at offset 33554432 32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}] convert -S 0 -B ... read 3145728/3145728 bytes at offset 0 @@ -XXX,XX +XXX,XX @@ read 30408704/30408704 bytes at offset 3145728 29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 33554432/33554432 bytes at offset 33554432 32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] convert -c -S 0 -B ... read 3145728/3145728 bytes at offset 0 @@ -XXX,XX +XXX,XX @@ read 30408704/30408704 bytes at offset 3145728 29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 33554432/33554432 bytes at offset 33554432 32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}] === Non-zero -S === @@ -XXX,XX +XXX,XX @@ wrote 1024/1024 bytes at offset 66560 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) convert -S 4k -[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 4096, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 8192, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 12288, "length": 4096, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 16384, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 20480, "length": 67088384, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4096, "length": 4096, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 8192, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 12288, "length": 4096, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 16384, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 20480, "length": 67088384, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] convert -c -S 4k -[{ "start": 0, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 1024, "length": 7168, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 8192, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 9216, "length": 8192, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 17408, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 18432, "length": 67090432, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 1024, "length": 7168, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 8192, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 9216, "length": 8192, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 17408, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 18432, "length": 67090432, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] convert -S 8k -[{ "start": 0, "length": 24576, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 24576, "length": 67084288, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 24576, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 24576, "length": 67084288, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] convert -c -S 8k -[{ "start": 0, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 1024, "length": 7168, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 8192, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 9216, "length": 8192, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 17408, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true}, -{ "start": 18432, "length": 67090432, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 1024, "length": 7168, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 8192, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 9216, "length": 8192, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 17408, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "compressed": true}, +{ "start": 18432, "length": 67090432, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] === -n to a non-zero image === @@ -XXX,XX +XXX,XX @@ Images are identical. Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 -[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 -[{ "start": 0, "length": 67108864, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] === -n to an empty image with a backing file === Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864 Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT -[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT -[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}] +[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 327680}] === -n -B to an image without a backing file === diff --git a/tests/qemu-iotests/146.out b/tests/qemu-iotests/146.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/146.out +++ b/tests/qemu-iotests/146.out @@ -XXX,XX +XXX,XX @@ QA output created by 146 === Testing VPC Autodetect === -[{ "start": 0, "length": 136363130880, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 136363130880, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Testing VPC with current_size force === -[{ "start": 0, "length": 136365211648, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 136365211648, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Testing VPC with chs force === -[{ "start": 0, "length": 136363130880, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 136363130880, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Testing Hyper-V Autodetect === -[{ "start": 0, "length": 136365211648, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 136365211648, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Testing Hyper-V with current_size force === -[{ "start": 0, "length": 136365211648, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 136365211648, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Testing Hyper-V with chs force === -[{ "start": 0, "length": 136363130880, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 136363130880, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Testing d2v Autodetect === -[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4194304, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 8388608, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 12582912, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 16777216, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 33554432, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 35651584, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 37748736, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 39845888, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 41943040, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 44040192, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 46137344, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 48234496, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 50331648, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 52428800, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 54525952, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 56623104, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 58720256, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 60817408, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 62914560, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 65011712, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 67108864, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 69206016, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 71303168, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 73400320, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 75497472, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 77594624, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 79691776, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 81788928, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 83886080, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 85983232, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 88080384, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 90177536, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 92274688, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 94371840, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 96468992, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 98566144, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 100663296, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 102760448, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 104857600, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 106954752, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 109051904, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 111149056, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 113246208, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 115343360, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 117440512, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 119537664, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 121634816, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 123731968, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 125829120, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 127926272, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 130023424, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 132120576, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 134217728, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 136314880, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 138412032, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 140509184, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 142606336, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 144703488, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 146800640, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 148897792, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 150994944, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 153092096, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 155189248, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 157286400, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 159383552, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 161480704, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 163577856, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 165675008, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 167772160, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 169869312, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 171966464, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 174063616, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 176160768, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 178257920, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 180355072, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 182452224, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 184549376, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 186646528, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 188743680, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 190840832, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 192937984, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 195035136, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 197132288, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 199229440, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 201326592, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 203423744, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 205520896, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 207618048, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 209715200, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 211812352, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 213909504, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 216006656, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 218103808, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 220200960, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 222298112, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 224395264, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 226492416, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 228589568, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 230686720, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 232783872, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 234881024, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 236978176, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 239075328, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 241172480, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 243269632, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 245366784, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 247463936, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 249561088, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 251658240, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 253755392, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 255852544, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 257949696, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 260046848, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 262144000, "length": 1310720, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4194304, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 8388608, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 12582912, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 16777216, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 33554432, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 35651584, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 37748736, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 39845888, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 41943040, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 44040192, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 46137344, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 48234496, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 50331648, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 52428800, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 54525952, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 56623104, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 58720256, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 60817408, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 62914560, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 65011712, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 67108864, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 69206016, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 71303168, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 73400320, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 75497472, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 77594624, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 79691776, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 81788928, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 83886080, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 85983232, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 88080384, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 90177536, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 92274688, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 94371840, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 96468992, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 98566144, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 100663296, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 102760448, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 104857600, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 106954752, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 109051904, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 111149056, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 113246208, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 115343360, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 117440512, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 119537664, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 121634816, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 123731968, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 125829120, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 127926272, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 130023424, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 132120576, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 134217728, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 136314880, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 138412032, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 140509184, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 142606336, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 144703488, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 146800640, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 148897792, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 150994944, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 153092096, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 155189248, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 157286400, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 159383552, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 161480704, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 163577856, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 165675008, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 167772160, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 169869312, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 171966464, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 174063616, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 176160768, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 178257920, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 180355072, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 182452224, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 184549376, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 186646528, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 188743680, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 190840832, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 192937984, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 195035136, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 197132288, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 199229440, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 201326592, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 203423744, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 205520896, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 207618048, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 209715200, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 211812352, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 213909504, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 216006656, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 218103808, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 220200960, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 222298112, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 224395264, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 226492416, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 228589568, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 230686720, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 232783872, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 234881024, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 236978176, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 239075328, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 241172480, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 243269632, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 245366784, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 247463936, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 249561088, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 251658240, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 253755392, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 255852544, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 257949696, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 260046848, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 262144000, "length": 1310720, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] === Testing d2v with current_size force === -[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4194304, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 8388608, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 12582912, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 16777216, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 33554432, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 35651584, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 37748736, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 39845888, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 41943040, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 44040192, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 46137344, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 48234496, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 50331648, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 52428800, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 54525952, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 56623104, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 58720256, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 60817408, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 62914560, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 65011712, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 67108864, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 69206016, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 71303168, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 73400320, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 75497472, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 77594624, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 79691776, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 81788928, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 83886080, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 85983232, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 88080384, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 90177536, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 92274688, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 94371840, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 96468992, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 98566144, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 100663296, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 102760448, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 104857600, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 106954752, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 109051904, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 111149056, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 113246208, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 115343360, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 117440512, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 119537664, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 121634816, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 123731968, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 125829120, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 127926272, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 130023424, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 132120576, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 134217728, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 136314880, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 138412032, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 140509184, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 142606336, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 144703488, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 146800640, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 148897792, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 150994944, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 153092096, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 155189248, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 157286400, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 159383552, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 161480704, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 163577856, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 165675008, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 167772160, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 169869312, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 171966464, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 174063616, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 176160768, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 178257920, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 180355072, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 182452224, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 184549376, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 186646528, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 188743680, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 190840832, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 192937984, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 195035136, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 197132288, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 199229440, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 201326592, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 203423744, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 205520896, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 207618048, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 209715200, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 211812352, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 213909504, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 216006656, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 218103808, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 220200960, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 222298112, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 224395264, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 226492416, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 228589568, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 230686720, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 232783872, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 234881024, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 236978176, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 239075328, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 241172480, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 243269632, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 245366784, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 247463936, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 249561088, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 251658240, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 253755392, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 255852544, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 257949696, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 260046848, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 262144000, "length": 1310720, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4194304, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 8388608, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 12582912, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 16777216, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 33554432, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 35651584, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 37748736, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 39845888, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 41943040, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 44040192, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 46137344, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 48234496, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 50331648, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 52428800, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 54525952, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 56623104, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 58720256, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 60817408, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 62914560, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 65011712, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 67108864, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 69206016, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 71303168, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 73400320, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 75497472, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 77594624, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 79691776, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 81788928, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 83886080, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 85983232, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 88080384, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 90177536, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 92274688, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 94371840, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 96468992, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 98566144, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 100663296, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 102760448, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 104857600, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 106954752, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 109051904, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 111149056, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 113246208, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 115343360, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 117440512, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 119537664, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 121634816, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 123731968, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 125829120, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 127926272, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 130023424, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 132120576, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 134217728, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 136314880, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 138412032, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 140509184, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 142606336, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 144703488, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 146800640, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 148897792, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 150994944, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 153092096, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 155189248, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 157286400, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 159383552, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 161480704, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 163577856, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 165675008, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 167772160, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 169869312, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 171966464, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 174063616, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 176160768, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 178257920, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 180355072, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 182452224, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 184549376, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 186646528, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 188743680, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 190840832, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 192937984, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 195035136, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 197132288, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 199229440, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 201326592, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 203423744, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 205520896, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 207618048, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 209715200, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 211812352, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 213909504, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 216006656, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 218103808, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 220200960, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 222298112, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 224395264, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 226492416, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 228589568, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 230686720, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 232783872, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 234881024, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 236978176, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 239075328, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 241172480, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 243269632, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 245366784, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 247463936, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 249561088, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 251658240, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 253755392, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 255852544, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 257949696, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 260046848, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 262144000, "length": 1310720, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] === Testing d2v with chs force === -[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4194304, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 8388608, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 12582912, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 16777216, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 33554432, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 35651584, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 37748736, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 39845888, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 41943040, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 44040192, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 46137344, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 48234496, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 50331648, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 52428800, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 54525952, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 56623104, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 58720256, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 60817408, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 62914560, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 65011712, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 67108864, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 69206016, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 71303168, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 73400320, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 75497472, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 77594624, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 79691776, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 81788928, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 83886080, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 85983232, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 88080384, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 90177536, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 92274688, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 94371840, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 96468992, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 98566144, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 100663296, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 102760448, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 104857600, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 106954752, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 109051904, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 111149056, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 113246208, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 115343360, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 117440512, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 119537664, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 121634816, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 123731968, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 125829120, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 127926272, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 130023424, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 132120576, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 134217728, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 136314880, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 138412032, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 140509184, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 142606336, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 144703488, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 146800640, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 148897792, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 150994944, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 153092096, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 155189248, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 157286400, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 159383552, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 161480704, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 163577856, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 165675008, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 167772160, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 169869312, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 171966464, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 174063616, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 176160768, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 178257920, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 180355072, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 182452224, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 184549376, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 186646528, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 188743680, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 190840832, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 192937984, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 195035136, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 197132288, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 199229440, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 201326592, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 203423744, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 205520896, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 207618048, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 209715200, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 211812352, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 213909504, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 216006656, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 218103808, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 220200960, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 222298112, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 224395264, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 226492416, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 228589568, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 230686720, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 232783872, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 234881024, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 236978176, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 239075328, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 241172480, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 243269632, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 245366784, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 247463936, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 249561088, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 251658240, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 253755392, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 255852544, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 257949696, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 260046848, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 262144000, "length": 1310720, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4194304, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 8388608, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 12582912, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 16777216, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 33554432, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 35651584, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 37748736, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 39845888, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 41943040, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 44040192, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 46137344, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 48234496, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 50331648, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 52428800, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 54525952, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 56623104, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 58720256, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 60817408, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 62914560, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 65011712, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 67108864, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 69206016, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 71303168, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 73400320, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 75497472, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 77594624, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 79691776, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 81788928, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 83886080, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 85983232, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 88080384, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 90177536, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 92274688, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 94371840, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 96468992, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 98566144, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 100663296, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 102760448, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 104857600, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 106954752, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 109051904, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 111149056, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 113246208, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 115343360, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 117440512, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 119537664, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 121634816, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 123731968, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 125829120, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 127926272, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 130023424, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 132120576, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 134217728, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 136314880, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 138412032, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 140509184, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 142606336, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 144703488, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 146800640, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 148897792, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 150994944, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 153092096, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 155189248, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 157286400, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 159383552, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 161480704, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 163577856, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 165675008, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 167772160, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 169869312, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 171966464, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 174063616, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 176160768, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 178257920, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 180355072, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 182452224, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 184549376, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 186646528, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 188743680, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 190840832, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 192937984, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 195035136, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 197132288, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 199229440, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 201326592, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 203423744, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 205520896, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 207618048, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 209715200, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 211812352, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 213909504, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 216006656, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 218103808, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 220200960, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 222298112, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 224395264, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 226492416, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 228589568, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 230686720, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 232783872, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 234881024, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 236978176, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 239075328, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 241172480, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 243269632, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 245366784, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 247463936, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 249561088, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 251658240, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 253755392, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 255852544, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 257949696, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 260046848, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 262144000, "length": 1310720, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] === Testing Image create, default === @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/IMGFMT-create-test.IMGFMT', fmt=IMGFMT size=4294967296 === Read created image, default opts ==== -[{ "start": 0, "length": 4295467008, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 4295467008, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Read created image, force_size_calc=chs ==== -[{ "start": 0, "length": 4295467008, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 4295467008, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Read created image, force_size_calc=current_size ==== -[{ "start": 0, "length": 4295467008, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 4295467008, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Testing Image create, force_size === @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/IMGFMT-create-test.IMGFMT', fmt=IMGFMT size=4294967296 === Read created image, default opts ==== -[{ "start": 0, "length": 4294967296, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 4294967296, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Read created image, force_size_calc=chs ==== -[{ "start": 0, "length": 4294967296, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 4294967296, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === Read created image, force_size_calc=current_size ==== -[{ "start": 0, "length": 4294967296, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 4294967296, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] *** done diff --git a/tests/qemu-iotests/154.out b/tests/qemu-iotests/154.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/154.out +++ b/tests/qemu-iotests/154.out @@ -XXX,XX +XXX,XX @@ wrote 2048/2048 bytes at offset 17408 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 2048/2048 bytes at offset 27648 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 4096, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 8192, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 12288, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 16384, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 20480, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 24576, "length": 8192, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 32768, "length": 134184960, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 4096, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 8192, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 12288, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 16384, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 20480, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 24576, "length": 8192, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 32768, "length": 134184960, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == backing file contains non-zero data before write_zeroes == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 65536 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 2048/2048 bytes at offset 67584 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 32768, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 32768, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 36864, "length": 28672, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 69632, "length": 134148096, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 32768, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 32768, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 36864, "length": 28672, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 69632, "length": 134148096, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == backing file contains non-zero data after write_zeroes == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 44032 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 3072/3072 bytes at offset 40960 3 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 32768, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 32768, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 36864, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 40960, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 45056, "length": 134172672, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 32768, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 32768, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 36864, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 40960, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 45056, "length": 134172672, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == write_zeroes covers non-zero data == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ wrote 2048/2048 bytes at offset 29696 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 4096/4096 bytes at offset 28672 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 4096, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 8192, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 12288, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 16384, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 20480, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 24576, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 28672, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 32768, "length": 134184960, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 4096, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 8192, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 12288, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 16384, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 20480, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 24576, "length": 4096, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 28672, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 32768, "length": 134184960, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == spanning two clusters, non-zero before request == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 67584 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 5120/5120 bytes at offset 68608 5 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 32768, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 32768, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 36864, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 40960, "length": 8192, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 49152, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 53248, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 57344, "length": 8192, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 69632, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 73728, "length": 134144000, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 32768, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 32768, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 36864, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 40960, "length": 8192, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 49152, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 53248, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 57344, "length": 8192, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 69632, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 73728, "length": 134144000, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == spanning two clusters, non-zero after request == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ read 7168/7168 bytes at offset 65536 7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 1024/1024 bytes at offset 72704 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 32768, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 32768, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 36864, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 40960, "length": 8192, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 49152, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 53248, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 57344, "length": 8192, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 69632, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 73728, "length": 134144000, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 32768, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 32768, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 36864, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 40960, "length": 8192, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 49152, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 53248, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 57344, "length": 8192, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 69632, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 73728, "length": 134144000, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == spanning two clusters, partially overwriting backing file == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 5120 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 2048/2048 bytes at offset 6144 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 8192, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 8192, "length": 134209536, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 8192, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 8192, "length": 134209536, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == spanning multiple clusters, non-zero in first cluster == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ read 2048/2048 bytes at offset 65536 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 10240/10240 bytes at offset 67584 10 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 69632, "length": 8192, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 77824, "length": 134139904, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 69632, "length": 8192, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 77824, "length": 134139904, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == spanning multiple clusters, non-zero in intermediate cluster == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ wrote 7168/7168 bytes at offset 67584 7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 12288/12288 bytes at offset 65536 12 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 65536, "length": 12288, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 77824, "length": 134139904, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 65536, "length": 12288, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 77824, "length": 134139904, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == spanning multiple clusters, non-zero in final cluster == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ read 10240/10240 bytes at offset 65536 10 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 2048/2048 bytes at offset 75776 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 65536, "length": 8192, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 73728, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 77824, "length": 134139904, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 65536, "length": 8192, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 73728, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 77824, "length": 134139904, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == spanning multiple clusters, partially overwriting backing file == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 @@ -XXX,XX +XXX,XX @@ read 2048/2048 bytes at offset 74752 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 1024/1024 bytes at offset 76800 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 69632, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 73728, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 77824, "length": 134139904, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 65536, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 69632, "length": 4096, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 73728, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 77824, "length": 134139904, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] == unaligned image tail cluster, no allocation needed == Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 wrote 512/512 bytes at offset 134217728 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 wrote 512/512 bytes at offset 134219264 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 wrote 1024/1024 bytes at offset 134218240 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 wrote 2048/2048 bytes at offset 134217728 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134218752 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 512/512 bytes at offset 134217728 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 512/512 bytes at offset 134219264 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 1024/1024 bytes at offset 134218240 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 2048/2048 bytes at offset 134217728 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] wrote 512/512 bytes at offset 134217728 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 512/512 bytes at offset 134217728 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 512/512 bytes at offset 134219264 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 1024/1024 bytes at offset 134218240 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 2048/2048 bytes at offset 134217728 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134218752 wrote 1024/1024 bytes at offset 134217728 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 134217728 read 512/512 bytes at offset 134218240 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 1024/1024 bytes allocated at offset 128 MiB -[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] wrote 1024/1024 bytes at offset 134217728 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 1024/1024 bytes allocated at offset 128 MiB read 1024/1024 bytes at offset 134217728 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 1024, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 134217728, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 1024, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] == unaligned image tail cluster, allocation required == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134218752 @@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 134217728 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 1536/1536 bytes at offset 134218240 1.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134218752 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134219776 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 512/512 bytes at offset 134218240 @@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 134218240 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 1024/1024 bytes at offset 134218752 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 134217728, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 134217728, "length": 2048, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] *** done diff --git a/tests/qemu-iotests/179.out b/tests/qemu-iotests/179.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/179.out +++ b/tests/qemu-iotests/179.out @@ -XXX,XX +XXX,XX @@ wrote 2097152/2097152 bytes at offset 6291456 2 MiB (0x200000) bytes not allocated at offset 4 MiB (0x400000) 2 MiB (0x200000) bytes allocated at offset 6 MiB (0x600000) 56 MiB (0x3800000) bytes not allocated at offset 8 MiB (0x800000) -[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 8388608, "length": 58720256, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 8388608, "length": 58720256, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] wrote 2097150/2097150 bytes at offset 10485761 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 2097150/2097150 bytes at offset 14680065 @@ -XXX,XX +XXX,XX @@ wrote 2097150/2097150 bytes at offset 14680065 2 MiB (0x200000) bytes not allocated at offset 12 MiB (0xc00000) 2 MiB (0x200000) bytes allocated at offset 14 MiB (0xe00000) 48 MiB (0x3000000) bytes not allocated at offset 16 MiB (0x1000000) -[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 8388608, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 12582912, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 16777216, "length": 50331648, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 8388608, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 12582912, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 16777216, "length": 50331648, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] wrote 14680064/14680064 bytes at offset 18874368 14 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 2097152/2097152 bytes at offset 20971520 @@ -XXX,XX +XXX,XX @@ wrote 6291456/6291456 bytes at offset 25165824 2 MiB (0x200000) bytes not allocated at offset 16 MiB (0x1000000) 14 MiB (0xe00000) bytes allocated at offset 18 MiB (0x1200000) 32 MiB (0x2000000) bytes not allocated at offset 32 MiB (0x2000000) -[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 8388608, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 12582912, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 16777216, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 25165824, "length": 6291456, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 33554432, "length": 33554432, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 8388608, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 12582912, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 16777216, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 25165824, "length": 6291456, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 33554432, "length": 33554432, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] wrote 2097152/2097152 bytes at offset 27262976 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 2097152/2097152 bytes at offset 29360128 @@ -XXX,XX +XXX,XX @@ wrote 2097152/2097152 bytes at offset 29360128 2 MiB (0x200000) bytes not allocated at offset 16 MiB (0x1000000) 14 MiB (0xe00000) bytes allocated at offset 18 MiB (0x1200000) 32 MiB (0x2000000) bytes not allocated at offset 32 MiB (0x2000000) -[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 8388608, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 12582912, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 16777216, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 33554432, "length": 33554432, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 8388608, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 12582912, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 16777216, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 33554432, "length": 33554432, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] wrote 8388608/8388608 bytes at offset 33554432 8 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 2097152/2097152 bytes at offset 35651584 @@ -XXX,XX +XXX,XX @@ wrote 2097152/2097152 bytes at offset 37748736 2 MiB (0x200000) bytes not allocated at offset 16 MiB (0x1000000) 22 MiB (0x1600000) bytes allocated at offset 18 MiB (0x1200000) 24 MiB (0x1800000) bytes not allocated at offset 40 MiB (0x2800000) -[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 8388608, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 12582912, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 16777216, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 33554432, "length": 8388608, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 41943040, "length": 25165824, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 4194304, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 6291456, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 8388608, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 10485760, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 12582912, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 14680064, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 16777216, "length": 2097152, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 18874368, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 20971520, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 23068672, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 25165824, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 27262976, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 29360128, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 31457280, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 33554432, "length": 8388608, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 41943040, "length": 25165824, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] wrote 8388608/8388608 bytes at offset 41943040 8 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 8388608/8388608 bytes at offset 50331648 @@ -XXX,XX +XXX,XX @@ wrote 2097152/2097152 bytes at offset 62914560 4 MiB (0x400000) bytes not allocated at offset 54 MiB (0x3600000) 4 MiB (0x400000) bytes allocated at offset 58 MiB (0x3a00000) 2 MiB (0x200000) bytes not allocated at offset 62 MiB (0x3e00000) -[{ "start": 0, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 2097152, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false}, -{ "start": 4194304, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 6291456, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false}, -{ "start": 8388608, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 10485760, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false}, -{ "start": 12582912, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 14680064, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false}, -{ "start": 16777216, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 18874368, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 20971520, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false}, -{ "start": 23068672, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 25165824, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 27262976, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false}, -{ "start": 29360128, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 31457280, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 33554432, "length": 10485760, "depth": 1, "present": true, "zero": true, "data": false}, -{ "start": 44040192, "length": 4194304, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 48234496, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false}, -{ "start": 50331648, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 52428800, "length": 4194304, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 56623104, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 58720256, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 60817408, "length": 4194304, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 65011712, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 2097152, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 4194304, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 6291456, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 8388608, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 10485760, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 12582912, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 14680064, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 16777216, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 18874368, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 20971520, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 23068672, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 25165824, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 27262976, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 29360128, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 31457280, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 33554432, "length": 10485760, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 44040192, "length": 4194304, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 48234496, "length": 2097152, "depth": 1, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 50331648, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 52428800, "length": 4194304, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 56623104, "length": 2097152, "depth": 1, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 58720256, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 60817408, "length": 4194304, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 65011712, "length": 2097152, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] No errors were found on the image. No errors were found on the image. diff --git a/tests/qemu-iotests/209.out b/tests/qemu-iotests/209.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/209.out +++ b/tests/qemu-iotests/209.out @@ -XXX,XX +XXX,XX @@ -[{ "start": 0, "length": 524288, "depth": 0, "present": true, "zero": false, "data": true, "offset": 0}, -{ "start": 524288, "length": 524288, "depth": 0, "present": true, "zero": true, "data": false, "offset": 524288}] +[{ "start": 0, "length": 524288, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 0}, +{ "start": 524288, "length": 524288, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": 524288}] done. diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/221.out +++ b/tests/qemu-iotests/221.out @@ -XXX,XX +XXX,XX @@ QA output created by 221 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65537 discard 65537/65537 bytes at offset 0 64.001 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 66048, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] -[{ "start": 0, "length": 66048, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 66048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 66048, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] wrote 1/1 bytes at offset 65536 1 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 65536, "length": 1, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 65537, "length": 511, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] -[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 65536, "length": 1, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 65537, "length": 511, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 65536, "length": 1, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 65537, "length": 511, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 65536, "length": 1, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 65537, "length": 511, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] *** done diff --git a/tests/qemu-iotests/223.out b/tests/qemu-iotests/223.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/223.out +++ b/tests/qemu-iotests/223.out @@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 2097152/2097152 bytes at offset 2097152 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 1044480, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 1048576, "length": 3145728, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] -[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 65536, "length": 2031616, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false}] +[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4096, "length": 1044480, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 1048576, "length": 3145728, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 65536, "length": 2031616, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}] === Contrast to small granularity dirty-bitmap === -[{ "start": 0, "length": 512, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 512, "length": 512, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 1024, "length": 2096128, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false}] +[{ "start": 0, "length": 512, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 512, "length": 512, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 1024, "length": 2096128, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}] === Check bitmap taken from another node === -[{ "start": 0, "length": 4194304, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 4194304, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] === End qemu NBD server === @@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 2097152/2097152 bytes at offset 2097152 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 1044480, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 1048576, "length": 3145728, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] -[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 65536, "length": 2031616, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false}] +[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4096, "length": 1044480, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 1048576, "length": 3145728, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 65536, "length": 2031616, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}] === Contrast to small granularity dirty-bitmap === -[{ "start": 0, "length": 512, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 512, "length": 512, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 1024, "length": 2096128, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false}] +[{ "start": 0, "length": 512, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 512, "length": 512, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 1024, "length": 2096128, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}] === Check bitmap taken from another node === -[{ "start": 0, "length": 4194304, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 4194304, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] === End qemu NBD server === @@ -XXX,XX +XXX,XX @@ read 2097152/2097152 bytes at offset 2097152 === Use qemu-nbd as server === -[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 65536, "length": 2031616, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false}] -[{ "start": 0, "length": 512, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 512, "length": 512, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 1024, "length": 11321, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] -[{ "start": 12345, "length": 2084807, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false}] +[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 65536, "length": 2031616, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}] +[{ "start": 0, "length": 512, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 512, "length": 512, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 1024, "length": 11321, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] +[{ "start": 12345, "length": 2084807, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}] *** done diff --git a/tests/qemu-iotests/241.out b/tests/qemu-iotests/241.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/241.out +++ b/tests/qemu-iotests/241.out @@ -XXX,XX +XXX,XX @@ exports available: 1 export: '' size: 1024 min block: 1 -[{ "start": 0, "length": 1000, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 1000, "length": 24, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 1000, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 1000, "length": 24, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] 1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) === Exporting unaligned raw image, forced server sector alignment === @@ -XXX,XX +XXX,XX @@ exports available: 1 export: '' size: 1024 min block: 512 -[{ "start": 0, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] 1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -XXX,XX +XXX,XX @@ exports available: 1 export: '' size: 1024 min block: 1 -[{ "start": 0, "length": 1000, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 1000, "length": 24, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 1000, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 1000, "length": 24, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] 1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) *** done diff --git a/tests/qemu-iotests/244.out b/tests/qemu-iotests/244.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/244.out +++ b/tests/qemu-iotests/244.out @@ -XXX,XX +XXX,XX @@ wrote 3145728/3145728 bytes at offset 3145728 3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) No errors were found on the image. -[{ "start": 0, "length": 1048576, "depth": 0, "present": false, "zero": true, "data": false}, -{ "start": 1048576, "length": 1048576, "depth": 0, "present": true, "zero": false, "data": true, "offset": 1048576}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 4194304, "length": 1048576, "depth": 0, "present": true, "zero": true, "data": false, "offset": 4194304}, -{ "start": 5242880, "length": 1048576, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 6291456, "length": 60817408, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 1048576, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 1048576, "length": 1048576, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 1048576}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 4194304, "length": 1048576, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": 4194304}, +{ "start": 5242880, "length": 1048576, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 6291456, "length": 60817408, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 3145728/3145728 bytes at offset 3145728 3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) No errors were found on the image. -[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": 0}, -{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 4194304, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "offset": 4194304}, -{ "start": 6291456, "length": 60817408, "depth": 0, "present": true, "zero": false, "data": true, "offset": 6291456}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 0}, +{ "start": 2097152, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 4194304, "length": 2097152, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": 4194304}, +{ "start": 6291456, "length": 60817408, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 6291456}] read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Offset Length Mapped to File 0 0x100000 0 TEST_DIR/t.qcow2.data -[{ "start": 0, "length": 1048576, "depth": 0, "present": true, "zero": false, "data": true, "offset": 0}, -{ "start": 1048576, "length": 66060288, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 1048576, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 0}, +{ "start": 1048576, "length": 66060288, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] === Copy offloading === diff --git a/tests/qemu-iotests/252.out b/tests/qemu-iotests/252.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/252.out +++ b/tests/qemu-iotests/252.out @@ -XXX,XX +XXX,XX @@ read 131072/131072 bytes at offset 131072 read 131072/131072 bytes at offset 262144 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 262144, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 262144, "length": 131072, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 262144, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 262144, "length": 131072, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] read 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ read 131072/131072 bytes at offset 131072 read 131072/131072 bytes at offset 262144 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 262144, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 262144, "length": 65536, "depth": 0, "present": true, "zero": true, "data": false}, -{ "start": 327680, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 262144, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 262144, "length": 65536, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}, +{ "start": 327680, "length": 65536, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] *** done diff --git a/tests/qemu-iotests/253.out b/tests/qemu-iotests/253.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/253.out +++ b/tests/qemu-iotests/253.out @@ -XXX,XX +XXX,XX @@ QA output created by 253 === Check mapping of unaligned raw image === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048575 -[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 1044480, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] -[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 1044480, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4096, "length": 1044480, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4096, "length": 1044480, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] wrote 65535/65535 bytes at offset 983040 63.999 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 978944, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 983040, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] -[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 978944, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}, -{ "start": 983040, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4096, "length": 978944, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 983040, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 4096, "length": 978944, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}, +{ "start": 983040, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] *** done diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/274.out +++ b/tests/qemu-iotests/274.out @@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576 0/1048576 bytes allocated at offset 1 MiB === Checking map === -[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 327680}] Offset Length Mapped to File 0 0x200000 0x50000 TEST_DIR/PID-base -[{ "start": 0, "length": 1048576, "depth": 1, "present": true, "zero": false, "data": true, "offset": 327680}] +[{ "start": 0, "length": 1048576, "depth": 1, "present": true, "zero": false, "data": true, "compressed": false, "offset": 327680}] Offset Length Mapped to File 0 0x100000 0x50000 TEST_DIR/PID-base -[{ "start": 0, "length": 1048576, "depth": 2, "present": true, "zero": false, "data": true, "offset": 327680}, -{ "start": 1048576, "length": 1048576, "depth": 0, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 1048576, "depth": 2, "present": true, "zero": false, "data": true, "compressed": false, "offset": 327680}, +{ "start": 1048576, "length": 1048576, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false}] Offset Length Mapped to File 0 0x100000 0x50000 TEST_DIR/PID-base @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 5368709120 1 GiB (0x40000000) bytes not allocated at offset 0 bytes (0x0) 7 GiB (0x1c0000000) bytes allocated at offset 1 GiB (0x40000000) -[{ "start": 0, "length": 1073741824, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 1073741824, "length": 7516192768, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 1073741824, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 1073741824, "length": 7516192768, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === preallocation=metadata === wrote 65536/65536 bytes at offset 33285996544 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 33285996544 30 GiB (0x780000000) bytes not allocated at offset 0 bytes (0x0) 3 GiB (0xc0000000) bytes allocated at offset 30 GiB (0x780000000) -[{ "start": 0, "length": 32212254720, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 32212254720, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "offset": 327680}, -{ "start": 32749125632, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "offset": 537264128}, -{ "start": 33285996544, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "offset": 1074200576}, -{ "start": 33822867456, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "offset": 1611137024}, -{ "start": 34359738368, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "offset": 2148139008}, -{ "start": 34896609280, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "offset": 2685075456}] +[{ "start": 0, "length": 32212254720, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 32212254720, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": 327680}, +{ "start": 32749125632, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": 537264128}, +{ "start": 33285996544, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": 1074200576}, +{ "start": 33822867456, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": 1611137024}, +{ "start": 34359738368, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": 2148139008}, +{ "start": 34896609280, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": 2685075456}] === preallocation=falloc === wrote 65536/65536 bytes at offset 9437184 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 9437184 5 MiB (0x500000) bytes not allocated at offset 0 bytes (0x0) 10 MiB (0xa00000) bytes allocated at offset 5 MiB (0x500000) -[{ "start": 0, "length": 5242880, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 5242880, "length": 10485760, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}] +[{ "start": 0, "length": 5242880, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 5242880, "length": 10485760, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 327680}] === preallocation=full === wrote 65536/65536 bytes at offset 11534336 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 11534336 8 MiB (0x800000) bytes not allocated at offset 0 bytes (0x0) 4 MiB (0x400000) bytes allocated at offset 8 MiB (0x800000) -[{ "start": 0, "length": 8388608, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 8388608, "length": 4194304, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}] +[{ "start": 0, "length": 8388608, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 8388608, "length": 4194304, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 327680}] === preallocation=off === wrote 65536/65536 bytes at offset 259072 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 259072 192 KiB (0x30000) bytes not allocated at offset 0 bytes (0x0) 320 KiB (0x50000) bytes allocated at offset 192 KiB (0x30000) -[{ "start": 0, "length": 196608, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 196608, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}, -{ "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 196608, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 196608, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 327680}, +{ "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === preallocation=off === wrote 65536/65536 bytes at offset 344064 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 344064 256 KiB (0x40000) bytes not allocated at offset 0 bytes (0x0) 256 KiB (0x40000) bytes allocated at offset 256 KiB (0x40000) -[{ "start": 0, "length": 262144, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 262144, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] === preallocation=off === wrote 65536/65536 bytes at offset 446464 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 446464 256 KiB (0x40000) bytes not allocated at offset 0 bytes (0x0) 244 KiB (0x3d000) bytes allocated at offset 256 KiB (0x40000) -[{ "start": 0, "length": 262144, "depth": 1, "present": false, "zero": true, "data": false}, -{ "start": 262144, "length": 249856, "depth": 0, "present": true, "zero": true, "data": false}] +[{ "start": 0, "length": 262144, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}, +{ "start": 262144, "length": 249856, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}] diff --git a/tests/qemu-iotests/tests/nbd-qemu-allocation.out b/tests/qemu-iotests/tests/nbd-qemu-allocation.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/tests/nbd-qemu-allocation.out +++ b/tests/qemu-iotests/tests/nbd-qemu-allocation.out @@ -XXX,XX +XXX,XX @@ wrote 2097152/2097152 bytes at offset 1048576 === Check allocation over NBD === -[{ "start": 0, "length": 1048576, "depth": 1, "present": true, "zero": false, "data": true, "offset": 327680}, -{ "start": 1048576, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}, -{ "start": 3145728, "length": 1048576, "depth": 1, "present": false, "zero": true, "data": false}] +[{ "start": 0, "length": 1048576, "depth": 1, "present": true, "zero": false, "data": true, "compressed": false, "offset": 327680}, +{ "start": 1048576, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 327680}, +{ "start": 3145728, "length": 1048576, "depth": 1, "present": false, "zero": true, "data": false, "compressed": false}] exports available: 1 export: '' size: 4194304 @@ -XXX,XX +XXX,XX @@ exports available: 1 available meta contexts: 2 base:allocation qemu:allocation-depth -[{ "start": 0, "length": 3145728, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 3145728, "length": 1048576, "depth": 0, "present": true, "zero": true, "data": false, "offset": OFFSET}] -[{ "start": 0, "length": 1048576, "depth": 0, "present": true, "zero": true, "data": true, "offset": OFFSET}, -{ "start": 1048576, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 3145728, "length": 1048576, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 3145728, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 3145728, "length": 1048576, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 1048576, "depth": 0, "present": true, "zero": true, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 1048576, "length": 2097152, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 3145728, "length": 1048576, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] *** done diff --git a/tests/qemu-iotests/tests/qemu-img-bitmaps.out b/tests/qemu-iotests/tests/qemu-img-bitmaps.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/tests/qemu-img-bitmaps.out +++ b/tests/qemu-iotests/tests/qemu-img-bitmaps.out @@ -XXX,XX +XXX,XX @@ Format specific information: === Check bitmap contents === -[{ "start": 0, "length": 3145728, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 3145728, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 4194304, "length": 6291456, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] -[{ "start": 0, "length": 1048576, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 1048576, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 2097152, "length": 8388608, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] -[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 3145728, "length": 7340032, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] -[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 2097152, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false}, -{ "start": 3145728, "length": 7340032, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}] +[{ "start": 0, "length": 3145728, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 3145728, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 4194304, "length": 6291456, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 1048576, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 1048576, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 2097152, "length": 8388608, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 3145728, "length": 7340032, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] +[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 2097152, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false, "compressed": false}, +{ "start": 3145728, "length": 7340032, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": OFFSET}] === Check handling of inconsistent bitmap === -- 2.41.0 From: Stefan Hajnoczi <stefanha@redhat.com> The synchronous bdrv_aio_cancel() function needs the acb's AioContext so it can call aio_poll() to wait for cancellation. It turns out that all users run under the BQL in the main AioContext, so this callback is not needed. Remove the callback, mark bdrv_aio_cancel() GLOBAL_STATE_CODE just like its blk_aio_cancel() caller, and poll the main loop AioContext. The purpose of this cleanup is to identify bdrv_aio_cancel() as an API that does not work with the multi-queue block layer. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230912231037.826804-2-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Klaus Jensen <k.jensen@samsung.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/aio.h | 1 - include/block/block-global-state.h | 2 ++ include/block/block-io.h | 1 - block/block-backend.c | 17 ----------------- block/io.c | 23 ++++++++--------------- hw/nvme/ctrl.c | 7 ------- softmmu/dma-helpers.c | 8 -------- util/thread-pool.c | 8 -------- 8 files changed, 10 insertions(+), 57 deletions(-) diff --git a/include/block/aio.h b/include/block/aio.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -XXX,XX +XXX,XX @@ typedef void BlockCompletionFunc(void *opaque, int ret); typedef struct AIOCBInfo { void (*cancel_async)(BlockAIOCB *acb); - AioContext *(*get_aio_context)(BlockAIOCB *acb); size_t aiocb_size; } AIOCBInfo; diff --git a/include/block/block-global-state.h b/include/block/block-global-state.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block-global-state.h +++ b/include/block/block-global-state.h @@ -XXX,XX +XXX,XX @@ void bdrv_drain_all_begin_nopoll(void); void bdrv_drain_all_end(void); void bdrv_drain_all(void); +void bdrv_aio_cancel(BlockAIOCB *acb); + int bdrv_has_zero_init_1(BlockDriverState *bs); int bdrv_has_zero_init(BlockDriverState *bs); BlockDriverState *bdrv_find_node(const char *node_name); diff --git a/include/block/block-io.h b/include/block/block-io.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block-io.h +++ b/include/block/block-io.h @@ -XXX,XX +XXX,XX @@ bdrv_co_delete_file_noerr(BlockDriverState *bs); /* async block I/O */ -void bdrv_aio_cancel(BlockAIOCB *acb); void bdrv_aio_cancel_async(BlockAIOCB *acb); /* sg packet commands */ diff --git a/block/block-backend.c b/block/block-backend.c index XXXXXXX..XXXXXXX 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -XXX,XX +XXX,XX @@ #define NOT_DONE 0x7fffffff /* used while emulated sync operation in progress */ -static AioContext *blk_aiocb_get_aio_context(BlockAIOCB *acb); - typedef struct BlockBackendAioNotifier { void (*attached_aio_context)(AioContext *new_context, void *opaque); void (*detach_aio_context)(void *opaque); @@ -XXX,XX +XXX,XX @@ typedef struct BlockBackendAIOCB { } BlockBackendAIOCB; static const AIOCBInfo block_backend_aiocb_info = { - .get_aio_context = blk_aiocb_get_aio_context, .aiocb_size = sizeof(BlockBackendAIOCB), }; @@ -XXX,XX +XXX,XX @@ typedef struct BlkAioEmAIOCB { bool has_returned; } BlkAioEmAIOCB; -static AioContext *blk_aio_em_aiocb_get_aio_context(BlockAIOCB *acb_) -{ - BlkAioEmAIOCB *acb = container_of(acb_, BlkAioEmAIOCB, common); - - return blk_get_aio_context(acb->rwco.blk); -} - static const AIOCBInfo blk_aio_em_aiocb_info = { .aiocb_size = sizeof(BlkAioEmAIOCB), - .get_aio_context = blk_aio_em_aiocb_get_aio_context, }; static void blk_aio_complete(BlkAioEmAIOCB *acb) @@ -XXX,XX +XXX,XX @@ AioContext *blk_get_aio_context(BlockBackend *blk) return blk->ctx; } -static AioContext *blk_aiocb_get_aio_context(BlockAIOCB *acb) -{ - BlockBackendAIOCB *blk_acb = DO_UPCAST(BlockBackendAIOCB, common, acb); - return blk_get_aio_context(blk_acb->blk); -} - int blk_set_aio_context(BlockBackend *blk, AioContext *new_context, Error **errp) { diff --git a/block/io.c b/block/io.c index XXXXXXX..XXXXXXX 100644 --- a/block/io.c +++ b/block/io.c @@ -XXX,XX +XXX,XX @@ int bdrv_load_vmstate(BlockDriverState *bs, uint8_t *buf, /**************************************************************/ /* async I/Os */ +/** + * Synchronously cancels an acb. Must be called with the BQL held and the acb + * must be processed with the BQL held too (IOThreads are not allowed). + * + * Use bdrv_aio_cancel_async() instead when possible. + */ void bdrv_aio_cancel(BlockAIOCB *acb) { - IO_CODE(); + GLOBAL_STATE_CODE(); qemu_aio_ref(acb); bdrv_aio_cancel_async(acb); - while (acb->refcnt > 1) { - if (acb->aiocb_info->get_aio_context) { - aio_poll(acb->aiocb_info->get_aio_context(acb), true); - } else if (acb->bs) { - /* qemu_aio_ref and qemu_aio_unref are not thread-safe, so - * assert that we're not using an I/O thread. Thread-safe - * code should use bdrv_aio_cancel_async exclusively. - */ - assert(bdrv_get_aio_context(acb->bs) == qemu_get_aio_context()); - aio_poll(bdrv_get_aio_context(acb->bs), true); - } else { - abort(); - } - } + AIO_WAIT_WHILE_UNLOCKED(NULL, acb->refcnt > 1); qemu_aio_unref(acb); } diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index XXXXXXX..XXXXXXX 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -XXX,XX +XXX,XX @@ static inline bool nvme_is_write(NvmeRequest *req) rw->opcode == NVME_CMD_WRITE_ZEROES; } -static AioContext *nvme_get_aio_context(BlockAIOCB *acb) -{ - return qemu_get_aio_context(); -} - static void nvme_misc_cb(void *opaque, int ret) { NvmeRequest *req = opaque; @@ -XXX,XX +XXX,XX @@ static void nvme_flush_cancel(BlockAIOCB *acb) static const AIOCBInfo nvme_flush_aiocb_info = { .aiocb_size = sizeof(NvmeFlushAIOCB), .cancel_async = nvme_flush_cancel, - .get_aio_context = nvme_get_aio_context, }; static void nvme_do_flush(NvmeFlushAIOCB *iocb); @@ -XXX,XX +XXX,XX @@ static void nvme_format_cancel(BlockAIOCB *aiocb) static const AIOCBInfo nvme_format_aiocb_info = { .aiocb_size = sizeof(NvmeFormatAIOCB), .cancel_async = nvme_format_cancel, - .get_aio_context = nvme_get_aio_context, }; static void nvme_format_set(NvmeNamespace *ns, uint8_t lbaf, uint8_t mset, diff --git a/softmmu/dma-helpers.c b/softmmu/dma-helpers.c index XXXXXXX..XXXXXXX 100644 --- a/softmmu/dma-helpers.c +++ b/softmmu/dma-helpers.c @@ -XXX,XX +XXX,XX @@ static void dma_aio_cancel(BlockAIOCB *acb) } } -static AioContext *dma_get_aio_context(BlockAIOCB *acb) -{ - DMAAIOCB *dbs = container_of(acb, DMAAIOCB, common); - - return dbs->ctx; -} - static const AIOCBInfo dma_aiocb_info = { .aiocb_size = sizeof(DMAAIOCB), .cancel_async = dma_aio_cancel, - .get_aio_context = dma_get_aio_context, }; BlockAIOCB *dma_blk_io(AioContext *ctx, diff --git a/util/thread-pool.c b/util/thread-pool.c index XXXXXXX..XXXXXXX 100644 --- a/util/thread-pool.c +++ b/util/thread-pool.c @@ -XXX,XX +XXX,XX @@ static void thread_pool_cancel(BlockAIOCB *acb) } -static AioContext *thread_pool_get_aio_context(BlockAIOCB *acb) -{ - ThreadPoolElement *elem = (ThreadPoolElement *)acb; - ThreadPool *pool = elem->pool; - return pool->ctx; -} - static const AIOCBInfo thread_pool_aiocb_info = { .aiocb_size = sizeof(ThreadPoolElement), .cancel_async = thread_pool_cancel, - .get_aio_context = thread_pool_get_aio_context, }; BlockAIOCB *thread_pool_submit_aio(ThreadPoolFunc *func, void *arg, -- 2.41.0 From: Stefan Hajnoczi <stefanha@redhat.com> This patch fixes a race condition in test-bdrv-drain that is difficult to reproduce. test-bdrv-drain sometimes fails without an error message on the block pull request sent by Kevin Wolf on Sep 4, 2023. I was able to reproduce it locally and found that "block-backend: process I/O in the current AioContext" (in this patch series) is the first commit where it reproduces. I do not know why "block-backend: process I/O in the current AioContext" exposes this bug. It might be related to the fact that the test's preadv request runs in the main thread instead of IOThread a after my commit. That might simply change the timing of the test. Now on to the race condition in test-bdrv-drain. The main thread schedules a BH in IOThread a and then drains the BDS: aio_bh_schedule_oneshot(ctx_a, test_iothread_main_thread_bh, &data); /* The request is running on the IOThread a. Draining its block device * will make sure that it has completed as far as the BDS is concerned, * but the drain in this thread can continue immediately after * bdrv_dec_in_flight() and aio_ret might be assigned only slightly * later. */ do_drain_begin(drain_type, bs); If the BH completes before do_drain_begin() then there is nothing to worry about. If the BH invokes bdrv_flush() before do_drain_begin(), then do_drain_begin() waits for it to complete. The problematic case is when do_drain_begin() runs before the BH enters bdrv_flush(). Then do_drain_begin() misses the BH and the drain mechanism has failed in quiescing I/O. Fix this by incrementing the in_flight counter so that do_drain_begin() waits for test_iothread_main_thread_bh(). Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230912231037.826804-3-stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/unit/test-bdrv-drain.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-bdrv-drain.c +++ b/tests/unit/test-bdrv-drain.c @@ -XXX,XX +XXX,XX @@ static void test_iothread_main_thread_bh(void *opaque) * executed during drain, otherwise this would deadlock. */ aio_context_acquire(bdrv_get_aio_context(data->bs)); bdrv_flush(data->bs); + bdrv_dec_in_flight(data->bs); /* incremented by test_iothread_common() */ aio_context_release(bdrv_get_aio_context(data->bs)); } @@ -XXX,XX +XXX,XX @@ static void test_iothread_common(enum drain_type drain_type, int drain_thread) aio_context_acquire(ctx_a); } + /* + * Increment in_flight so that do_drain_begin() waits for + * test_iothread_main_thread_bh(). This prevents the race between + * test_iothread_main_thread_bh() in IOThread a and do_drain_begin() in + * this thread. test_iothread_main_thread_bh() decrements in_flight. + */ + bdrv_inc_in_flight(bs); aio_bh_schedule_oneshot(ctx_a, test_iothread_main_thread_bh, &data); /* The request is running on the IOThread a. Draining its block device -- 2.41.0 From: Stefan Hajnoczi <stefanha@redhat.com> Switch blk_aio_*() APIs over to multi-queue by using qemu_get_current_aio_context() instead of blk_get_aio_context(). This change will allow devices to process I/O in multiple IOThreads in the future. I audited existing blk_aio_*() callers: - migration/block.c: blk_mig_lock() protects the data accessed by the completion callback. - The remaining emulated devices and exports run with qemu_get_aio_context() == blk_get_aio_context(). Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230912231037.826804-4-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/block-backend.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index XXXXXXX..XXXXXXX 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_abort_aio_request(BlockBackend *blk, acb->blk = blk; acb->ret = ret; - replay_bh_schedule_oneshot_event(blk_get_aio_context(blk), + replay_bh_schedule_oneshot_event(qemu_get_current_aio_context(), error_callback_bh, acb); return &acb->common; } @@ -XXX,XX +XXX,XX @@ static BlockAIOCB *blk_aio_prwv(BlockBackend *blk, int64_t offset, acb->has_returned = false; co = qemu_coroutine_create(co_entry, acb); - aio_co_enter(blk_get_aio_context(blk), co); + aio_co_enter(qemu_get_current_aio_context(), co); acb->has_returned = true; if (acb->rwco.ret != NOT_DONE) { - replay_bh_schedule_oneshot_event(blk_get_aio_context(blk), + replay_bh_schedule_oneshot_event(qemu_get_current_aio_context(), blk_aio_complete_bh, acb); } -- 2.41.0 From: Stefan Hajnoczi <stefanha@redhat.com> Process zoned requests in the current thread's AioContext instead of in the BlockBackend's AioContext. There is no need to use the BlockBackend's AioContext thanks to CoMutex bs->wps->colock, which protects zone metadata. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230912231037.826804-5-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/block-backend.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index XXXXXXX..XXXXXXX 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_aio_zone_report(BlockBackend *blk, int64_t offset, acb->has_returned = false; co = qemu_coroutine_create(blk_aio_zone_report_entry, acb); - aio_co_enter(blk_get_aio_context(blk), co); + aio_co_enter(qemu_get_current_aio_context(), co); acb->has_returned = true; if (acb->rwco.ret != NOT_DONE) { - replay_bh_schedule_oneshot_event(blk_get_aio_context(blk), + replay_bh_schedule_oneshot_event(qemu_get_current_aio_context(), blk_aio_complete_bh, acb); } @@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_aio_zone_mgmt(BlockBackend *blk, BlockZoneOp op, acb->has_returned = false; co = qemu_coroutine_create(blk_aio_zone_mgmt_entry, acb); - aio_co_enter(blk_get_aio_context(blk), co); + aio_co_enter(qemu_get_current_aio_context(), co); acb->has_returned = true; if (acb->rwco.ret != NOT_DONE) { - replay_bh_schedule_oneshot_event(blk_get_aio_context(blk), + replay_bh_schedule_oneshot_event(qemu_get_current_aio_context(), blk_aio_complete_bh, acb); } @@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_aio_zone_append(BlockBackend *blk, int64_t *offset, acb->has_returned = false; co = qemu_coroutine_create(blk_aio_zone_append_entry, acb); - aio_co_enter(blk_get_aio_context(blk), co); + aio_co_enter(qemu_get_current_aio_context(), co); acb->has_returned = true; if (acb->rwco.ret != NOT_DONE) { - replay_bh_schedule_oneshot_event(blk_get_aio_context(blk), + replay_bh_schedule_oneshot_event(qemu_get_current_aio_context(), blk_aio_complete_bh, acb); } -- 2.41.0 From: Stefan Hajnoczi <stefanha@redhat.com> Use qemu_get_current_aio_context() in mixed wrappers and coroutine wrappers so that code runs in the caller's AioContext instead of moving to the BlockDriverState's AioContext. This change is necessary for the multi-queue block layer where any thread can call into the block layer. Most wrappers are IO_CODE where it's safe to use the current AioContext nowadays. BlockDrivers and the core block layer use their own locks and no longer depend on the AioContext lock for thread-safety. The bdrv_create() wrapper invokes GLOBAL_STATE code. Using the current AioContext is safe because this code is only called with the BQL held from the main loop thread. The output of qemu-iotests 051 is sensitive to event loop activity. Update the output because the monitor BH runs at a different time, causing prompts to be printed differently in the output. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230912231037.826804-6-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- scripts/block-coroutine-wrapper.py | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py index XXXXXXX..XXXXXXX 100644 --- a/scripts/block-coroutine-wrapper.py +++ b/scripts/block-coroutine-wrapper.py @@ -XXX,XX +XXX,XX @@ def __init__(self, wrapper_type: str, return_type: str, name: str, raise ValueError(f"no_co function can't be rdlock: {self.name}") self.target_name = f'{subsystem}_{subname}' - self.ctx = self.gen_ctx() - self.get_result = 's->ret = ' self.ret = 'return s.ret;' self.co_ret = 'return ' @@ -XXX,XX +XXX,XX @@ def create_mixed_wrapper(func: FuncDecl) -> str: {func.co_ret}{name}({ func.gen_list('{name}') }); }} else {{ {struct_name} s = {{ - .poll_state.ctx = {func.ctx}, + .poll_state.ctx = qemu_get_current_aio_context(), .poll_state.in_progress = true, { func.gen_block(' .{name} = {name},') } @@ -XXX,XX +XXX,XX @@ def create_co_wrapper(func: FuncDecl) -> str: {func.return_type} {func.name}({ func.gen_list('{decl}') }) {{ {struct_name} s = {{ - .poll_state.ctx = {func.ctx}, + .poll_state.ctx = qemu_get_current_aio_context(), .poll_state.in_progress = true, { func.gen_block(' .{name} = {name},') } -- 2.41.0