The following changes since commit 91fe7a376ad46e3cc5e82d418aad22173c948a3c: Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2018-06-15 11:41:44 +0100) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream for you to fetch changes up to 6266e900b8083945cb766b45c124fb3c42932cb3: block: Remove dead deprecation warning code (2018-06-15 14:49:44 +0200) ---------------------------------------------------------------- Block layer patches: - Fix options that work only with -drive or -blockdev, but not with both, because of QDict type confusion - rbd: Add options 'auth-client-required' and 'key-secret' - Remove deprecated -drive options serial/addr/cyls/heads/secs/trans - rbd, iscsi: Remove deprecated 'filename' option - Fix 'qemu-img map' crash with unaligned image size - Improve QMP documentation for jobs ---------------------------------------------------------------- Eric Blake (2): qemu-img: Fix assert when mapping unaligned raw file iotests: Add test 221 to catch qemu-img map regression John Snow (2): jobs: fix stale wording jobs: fix verb references in docs Kevin Wolf (4): block: Remove deprecated -drive geometry options block: Remove deprecated -drive option addr block: Remove deprecated -drive option serial block: Remove dead deprecation warning code Markus Armbruster (17): rbd: Drop deprecated -drive parameter "filename" iscsi: Drop deprecated -drive parameter "filename" qobject: Move block-specific qdict code to block-qdict.c block: Fix -blockdev for certain non-string scalars block: Fix -drive for certain non-string scalars block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts() block: Factor out qobject_input_visitor_new_flat_confused() block: Make remaining uses of qobject input visitor more robust block-qdict: Simplify qdict_flatten_qdict() block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist() block-qdict: Clean up qdict_crumple() a bit block-qdict: Simplify qdict_is_list() some check-block-qdict: Rename qdict_flatten()'s variables for clarity check-block-qdict: Cover flattening of empty lists and dictionaries block: Fix -blockdev / blockdev-add for empty objects and arrays rbd: New parameter auth-client-required rbd: New parameter key-secret Max Reitz (1): block: Add block-specific QDict header qapi/block-core.json | 19 ++ qapi/job.json | 23 +- include/block/qdict.h | 34 +++ include/hw/block/block.h | 1 - include/qapi/qmp/qdict.h | 17 -- include/sysemu/blockdev.h | 3 - block.c | 1 + block/block-backend.c | 1 - block/crypto.c | 12 +- block/gluster.c | 1 + block/iscsi.c | 24 +- block/nbd.c | 16 +- block/nfs.c | 8 +- block/parallels.c | 11 +- block/qcow.c | 11 +- block/qcow2.c | 11 +- block/qed.c | 11 +- block/quorum.c | 1 + block/rbd.c | 85 +++--- block/sheepdog.c | 23 +- block/snapshot.c | 1 + block/ssh.c | 16 +- block/vdi.c | 8 +- block/vhdx.c | 11 +- block/vpc.c | 11 +- block/vvfat.c | 1 + block/vxhs.c | 1 + blockdev.c | 111 +------ device-hotplug.c | 4 - hw/block/block.c | 27 -- hw/block/nvme.c | 1 - hw/block/virtio-blk.c | 1 - hw/ide/qdev.c | 1 - hw/scsi/scsi-disk.c | 1 - hw/usb/dev-storage.c | 1 - qemu-img.c | 2 +- qobject/block-qdict.c | 722 +++++++++++++++++++++++++++++++++++++++++++++ qobject/qdict.c | 628 --------------------------------------- tests/ahci-test.c | 6 +- tests/check-block-qdict.c | 690 +++++++++++++++++++++++++++++++++++++++++++ tests/check-qdict.c | 641 ---------------------------------------- tests/check-qobject.c | 1 + tests/hd-geo-test.c | 37 +-- tests/ide-test.c | 8 +- tests/test-replication.c | 1 + util/qemu-config.c | 1 + MAINTAINERS | 2 + hmp-commands.hx | 1 - qemu-doc.texi | 15 - qemu-options.hx | 14 +- qobject/Makefile.objs | 1 + tests/Makefile.include | 4 + tests/qemu-iotests/221 | 60 ++++ tests/qemu-iotests/221.out | 16 + tests/qemu-iotests/group | 1 + 55 files changed, 1692 insertions(+), 1668 deletions(-) create mode 100644 include/block/qdict.h create mode 100644 qobject/block-qdict.c create mode 100644 tests/check-block-qdict.c create mode 100755 tests/qemu-iotests/221 create mode 100644 tests/qemu-iotests/221.out From: Eric Blake <eblake@redhat.com> Commit a290f085 exposed a latent bug in qemu-img map introduced during the conversion of block status to be byte-based. Earlier in commit 5e344dd8, the internal interface get_block_status() switched to take byte-based parameters, but still called a sector-based block layer function; as such, rounding was added in the lone caller to obey the contract. However, commit 237d78f8 changed get_block_status() to truly be byte-based, at which point rounding to sector boundaries can result in calling bdrv_block_status() with 'bytes == 0' (a coding error) when the boundary between data and a hole falls mid-sector (true for the past-EOF implicit hole present in POSIX files). Fix things by removing the rounding that is now no longer necessary. See also https://bugzilla.redhat.com/1589738 Fixes: 237d78f8 Reported-by: Dan Kenigsberg <danken@redhat.com> Reported-by: Nir Soffer <nsoffer@redhat.com> Reported-by: Maor Lipchuk <mlipchuk@redhat.com> CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qemu-img.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static int img_map(int argc, char **argv) int64_t n; /* Probe up to 1 GiB at a time. */ - n = QEMU_ALIGN_DOWN(MIN(1 << 30, length - offset), BDRV_SECTOR_SIZE); + n = MIN(1 << 30, length - offset); ret = get_block_status(bs, offset, n, &next); if (ret < 0) { -- 2.13.6 From: Eric Blake <eblake@redhat.com> Although qemu-img creates aligned files (by rounding up), it must also gracefully handle files that are not sector-aligned. Test that the bug fixed in the previous patch does not recur. It's a bit annoying that we can see the (implicit) hole past the end of the file on to the next sector boundary, so if we ever reach the point where we report a byte-accurate size rather than our current behavior of always rounding up, this test will probably need a slight modification. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/221 | 60 ++++++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/221.out | 16 +++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 77 insertions(+) create mode 100755 tests/qemu-iotests/221 create mode 100644 tests/qemu-iotests/221.out diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221 new file mode 100755 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/221 @@ -XXX,XX +XXX,XX @@ +#!/bin/bash +# +# Test qemu-img vs. unaligned images +# +# Copyright (C) 2018 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +seq="$(basename $0)" +echo "QA output created by $seq" + +here="$PWD" +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt raw +_supported_proto file +_supported_os Linux + +echo +echo "=== Check mapping of unaligned raw image ===" +echo + +_make_test_img 43009 # qemu-img create rounds size up +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +truncate --size=43009 "$TEST_IMG" # so we resize it and check again +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +$QEMU_IO -c 'w 43008 1' "$TEST_IMG" | _filter_qemu_io # writing also rounds up +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +truncate --size=43009 "$TEST_IMG" # so we resize it and check again +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +# success, all done +echo '*** done' +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/221.out @@ -XXX,XX +XXX,XX @@ +QA output created by 221 + +=== Check mapping of unaligned raw image === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=43009 +[{ "start": 0, "length": 43520, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 43520, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +wrote 1/1 bytes at offset 43008 +1 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 40960, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, +{ "start": 40960, "length": 2049, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, +{ "start": 43009, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 40960, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, +{ "start": 40960, "length": 2049, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, +{ "start": 43009, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -XXX,XX +XXX,XX @@ 217 rw auto quick 218 rw auto quick 219 rw auto +221 rw auto quick -- 2.13.6 From: John Snow <jsnow@redhat.com> During the design for manual completion, we decided not to use the "manual" property as a shorthand for both auto-dismiss and auto-finalize. Fix the wording. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/job.json | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/qapi/job.json b/qapi/job.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/job.json +++ b/qapi/job.json @@ -XXX,XX +XXX,XX @@ # the last job in a transaction. # # @pending: The job has finished its work, but has finalization steps that it -# needs to make prior to completing. These changes may require -# manual intervention by the management process if manual was set -# to true. These changes may still fail. +# needs to make prior to completing. These changes will require +# manual intervention via @job-finalize if auto-finalize was set to +# false. These pending changes may still fail. # # @aborting: The job is in the process of being aborted, and will finish with # an error. The job will afterwards report that it is @concluded. # This status may not be visible to the management process. # -# @concluded: The job has finished all work. If manual was set to true, the job -# will remain in the query list until it is dismissed. +# @concluded: The job has finished all work. If auto-dismiss was set to false, +# the job will remain in the query list until it is dismissed via +# @job-dismiss. # # @null: The job is in the process of being dismantled. This state should not # ever be visible externally. -- 2.13.6 From: John Snow <jsnow@redhat.com> These point to the job versions now, not the blockjob versions which don't really exist anymore. Except set-speed, which does. It sticks out like a sore thumb. This patch doesn't fix that, but it doesn't make it any worse, either. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/job.json | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/qapi/job.json b/qapi/job.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/job.json +++ b/qapi/job.json @@ -XXX,XX +XXX,XX @@ # # Represents command verbs that can be applied to a job. # -# @cancel: see @block-job-cancel +# @cancel: see @job-cancel # -# @pause: see @block-job-pause +# @pause: see @job-pause # -# @resume: see @block-job-resume +# @resume: see @job-resume # # @set-speed: see @block-job-set-speed # -# @complete: see @block-job-complete +# @complete: see @job-complete # -# @dismiss: see @block-job-dismiss +# @dismiss: see @job-dismiss # -# @finalize: see @block-job-finalize +# @finalize: see @job-finalize # # Since: 2.12 ## -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Parameter "filename" is deprecated since commit 91589d9e5ca, v2.10.0. Time to get rid of it. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/rbd.c | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, QObject *crumpled = NULL; const QDictEntry *e; Error *local_err = NULL; - const char *filename; char *keypairs, *secretid; int r; - /* If we are given a filename, parse the filename, with precedence given to - * filename encoded options */ - filename = qdict_get_try_str(options, "filename"); - if (filename) { - warn_report("'filename' option specified. " - "This is an unsupported option, and may be deprecated " - "in the future"); - qemu_rbd_parse_filename(filename, options, &local_err); - qdict_del(options, "filename"); - if (local_err) { - error_propagate(errp, local_err); - return -EINVAL; - } - } - keypairs = g_strdup(qdict_get_try_str(options, "=keyvalue-pairs")); if (keypairs) { qdict_del(options, "=keyvalue-pairs"); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Parameter "filename" is deprecated since commit 5c3ad1a6a8f, v2.10.0. Time to get rid of it. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/iscsi.c | 23 ++--------------------- 1 file changed, 2 insertions(+), 21 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index XXXXXXX..XXXXXXX 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -XXX,XX +XXX,XX @@ static QemuOptsList runtime_opts = { .name = "timeout", .type = QEMU_OPT_NUMBER, }, - { - .name = "filename", - .type = QEMU_OPT_STRING, - }, { /* end of list */ } }, }; @@ -XXX,XX +XXX,XX @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags, char *initiator_name = NULL; QemuOpts *opts; Error *local_err = NULL; - const char *transport_name, *portal, *target, *filename; + const char *transport_name, *portal, *target; #if LIBISCSI_API_VERSION >= (20160603) enum iscsi_transport_type transport; #endif int i, ret = 0, timeout = 0, lun; - /* If we are given a filename, parse the filename, with precedence given to - * filename encoded options */ - filename = qdict_get_try_str(options, "filename"); - if (filename) { - warn_report("'filename' option specified. " - "This is an unsupported option, and may be deprecated " - "in the future"); - iscsi_parse_filename(filename, options, &local_err); - if (local_err) { - ret = -EINVAL; - error_propagate(errp, local_err); - goto exit; - } - } - opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); qemu_opts_absorb_qdict(opts, options, &local_err); if (local_err) { @@ -XXX,XX +XXX,XX @@ out: } memset(iscsilun, 0, sizeof(IscsiLun)); } -exit: + return ret; } -- 2.13.6 From: Max Reitz <mreitz@redhat.com> There are numerous QDict functions that have been introduced for and are used only by the block layer. Move their declarations into an own header file to reflect that. While qdict_extract_subqdict() is in fact used outside of the block layer (in util/qemu-config.c), it is still a function related very closely to how the block layer works with nested QDicts, namely by sometimes flattening them. Therefore, its declaration is put into this header as well and util/qemu-config.c includes it with a comment stating exactly which function it needs. Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20180509165530.29561-7-mreitz@redhat.com> [Copyright note tweaked, superfluous includes dropped] Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/qdict.h | 32 ++++++++++++++++++++++++++++++++ include/qapi/qmp/qdict.h | 17 ----------------- block.c | 1 + block/gluster.c | 1 + block/iscsi.c | 1 + block/nbd.c | 1 + block/nfs.c | 1 + block/parallels.c | 1 + block/qcow.c | 1 + block/qcow2.c | 1 + block/qed.c | 1 + block/quorum.c | 1 + block/rbd.c | 1 + block/sheepdog.c | 1 + block/snapshot.c | 1 + block/ssh.c | 1 + block/vhdx.c | 1 + block/vpc.c | 1 + block/vvfat.c | 1 + block/vxhs.c | 1 + blockdev.c | 1 + qobject/qdict.c | 1 + tests/check-qdict.c | 1 + tests/check-qobject.c | 1 + tests/test-replication.c | 1 + util/qemu-config.c | 1 + 26 files changed, 56 insertions(+), 17 deletions(-) create mode 100644 include/block/qdict.h diff --git a/include/block/qdict.h b/include/block/qdict.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/block/qdict.h @@ -XXX,XX +XXX,XX @@ +/* + * Special QDict functions used by the block layer + * + * Copyright (c) 2013-2018 Red Hat, Inc. + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + */ + +#ifndef BLOCK_QDICT_H +#define BLOCK_QDICT_H + +#include "qapi/qmp/qdict.h" + +void qdict_copy_default(QDict *dst, QDict *src, const char *key); +void qdict_set_default_str(QDict *dst, const char *key, const char *val); + +void qdict_join(QDict *dest, QDict *src, bool overwrite); + +void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); +void qdict_array_split(QDict *src, QList **dst); +int qdict_array_entries(QDict *src, const char *subqdict); +QObject *qdict_crumple(const QDict *src, Error **errp); +void qdict_flatten(QDict *qdict); + +typedef struct QDictRenames { + const char *from; + const char *to; +} QDictRenames; +bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp); + +#endif diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h index XXXXXXX..XXXXXXX 100644 --- a/include/qapi/qmp/qdict.h +++ b/include/qapi/qmp/qdict.h @@ -XXX,XX +XXX,XX @@ int64_t qdict_get_try_int(const QDict *qdict, const char *key, bool qdict_get_try_bool(const QDict *qdict, const char *key, bool def_value); const char *qdict_get_try_str(const QDict *qdict, const char *key); -void qdict_copy_default(QDict *dst, QDict *src, const char *key); -void qdict_set_default_str(QDict *dst, const char *key, const char *val); - QDict *qdict_clone_shallow(const QDict *src); -void qdict_flatten(QDict *qdict); - -void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); -void qdict_array_split(QDict *src, QList **dst); -int qdict_array_entries(QDict *src, const char *subqdict); -QObject *qdict_crumple(const QDict *src, Error **errp); - -void qdict_join(QDict *dest, QDict *src, bool overwrite); - -typedef struct QDictRenames { - const char *from; - const char *to; -} QDictRenames; -bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp); #endif /* QDICT_H */ diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ #include "block/block_int.h" #include "block/blockjob.h" #include "block/nbd.h" +#include "block/qdict.h" #include "qemu/error-report.h" #include "module_block.h" #include "qemu/module.h" diff --git a/block/gluster.c b/block/gluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/gluster.c +++ b/block/gluster.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include <glusterfs/api/glfs.h> #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qerror.h" diff --git a/block/iscsi.c b/block/iscsi.c index XXXXXXX..XXXXXXX 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -XXX,XX +XXX,XX @@ #include "qemu/bitops.h" #include "qemu/bitmap.h" #include "block/block_int.h" +#include "block/qdict.h" #include "scsi/constants.h" #include "qemu/iov.h" #include "qemu/option.h" diff --git a/block/nbd.c b/block/nbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "nbd-client.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qemu/uri.h" #include "block/block_int.h" diff --git a/block/nfs.c b/block/nfs.c index XXXXXXX..XXXXXXX 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -XXX,XX +XXX,XX @@ #include "qemu/error-report.h" #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "trace.h" #include "qemu/iov.h" #include "qemu/option.h" diff --git a/block/parallels.c b/block/parallels.c index XXXXXXX..XXXXXXX 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ #include "qapi/error.h" #include "qemu/error-report.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include <zlib.h> diff --git a/block/qed.c b/block/qed.c index XXXXXXX..XXXXXXX 100644 --- a/block/qed.c +++ b/block/qed.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qemu/timer.h" #include "qemu/bswap.h" diff --git a/block/quorum.c b/block/quorum.c index XXXXXXX..XXXXXXX 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -XXX,XX +XXX,XX @@ #include "qemu/cutils.h" #include "qemu/option.h" #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qapi/qapi-events-block.h" #include "qapi/qmp/qdict.h" diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ #include "qemu/error-report.h" #include "qemu/option.h" #include "block/block_int.h" +#include "block/qdict.h" #include "crypto/secret.h" #include "qemu/cutils.h" #include "qapi/qmp/qstring.h" diff --git a/block/sheepdog.c b/block/sheepdog.c index XXXXXXX..XXXXXXX 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -XXX,XX +XXX,XX @@ #include "qemu/option.h" #include "qemu/sockets.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/bitops.h" #include "qemu/cutils.h" diff --git a/block/snapshot.c b/block/snapshot.c index XXXXXXX..XXXXXXX 100644 --- a/block/snapshot.c +++ b/block/snapshot.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "block/snapshot.h" #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qerror.h" diff --git a/block/ssh.c b/block/ssh.c index XXXXXXX..XXXXXXX 100644 --- a/block/ssh.c +++ b/block/ssh.c @@ -XXX,XX +XXX,XX @@ #include <libssh2_sftp.h> #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/option.h" diff --git a/block/vhdx.c b/block/vhdx.c index XXXXXXX..XXXXXXX 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" diff --git a/block/vpc.c b/block/vpc.c index XXXXXXX..XXXXXXX 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ #include <dirent.h> #include "qapi/error.h" #include "block/block_int.h" +#include "block/qdict.h" #include "qemu/module.h" #include "qemu/option.h" #include "qemu/bswap.h" diff --git a/block/vxhs.c b/block/vxhs.c index XXXXXXX..XXXXXXX 100644 --- a/block/vxhs.c +++ b/block/vxhs.c @@ -XXX,XX +XXX,XX @@ #include <qnio/qnio_api.h> #include <sys/param.h> #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/qmp/qerror.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qstring.h" diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ #include "sysemu/blockdev.h" #include "hw/block/block.h" #include "block/blockjob.h" +#include "block/qdict.h" #include "block/throttle-groups.h" #include "monitor/monitor.h" #include "qemu/error-report.h" diff --git a/qobject/qdict.c b/qobject/qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/qdict.c +++ b/qobject/qdict.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" +#include "block/qdict.h" #include "qapi/qmp/qnum.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qbool.h" diff --git a/tests/check-qdict.c b/tests/check-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-qdict.c +++ b/tests/check-qdict.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" +#include "block/qdict.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qlist.h" #include "qapi/qmp/qnum.h" diff --git a/tests/check-qobject.c b/tests/check-qobject.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-qobject.c +++ b/tests/check-qobject.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" +#include "block/qdict.h" #include "qapi/qmp/qbool.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qlist.h" diff --git a/tests/test-replication.c b/tests/test-replication.c index XXXXXXX..XXXXXXX 100644 --- a/tests/test-replication.c +++ b/tests/test-replication.c @@ -XXX,XX +XXX,XX @@ #include "qemu/option.h" #include "replication.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #define IMG_SIZE (64 * 1024 * 1024) diff --git a/util/qemu-config.c b/util/qemu-config.c index XXXXXXX..XXXXXXX 100644 --- a/util/qemu-config.c +++ b/util/qemu-config.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" +#include "block/qdict.h" /* for qdict_extract_subqdict() */ #include "qapi/error.h" #include "qapi/qapi-commands-misc.h" #include "qapi/qmp/qdict.h" -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Pure code motion, except for two brace placements and a comment tweaked to appease checkpatch. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 640 ++++++++++++++++++++++++++++++++++++++++++++ qobject/qdict.c | 629 -------------------------------------------- tests/check-block-qdict.c | 655 ++++++++++++++++++++++++++++++++++++++++++++++ tests/check-qdict.c | 642 --------------------------------------------- MAINTAINERS | 2 + qobject/Makefile.objs | 1 + tests/Makefile.include | 4 + 7 files changed, 1302 insertions(+), 1271 deletions(-) create mode 100644 qobject/block-qdict.c create mode 100644 tests/check-block-qdict.c diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ +/* + * Special QDict functions used by the block layer + * + * Copyright (c) 2013-2018 Red Hat, Inc. + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "block/qdict.h" +#include "qapi/qmp/qlist.h" +#include "qemu/cutils.h" +#include "qapi/error.h" + +/** + * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the + * value of 'key' in 'src' is copied there (and the refcount increased + * accordingly). + */ +void qdict_copy_default(QDict *dst, QDict *src, const char *key) +{ + QObject *val; + + if (qdict_haskey(dst, key)) { + return; + } + + val = qdict_get(src, key); + if (val) { + qdict_put_obj(dst, key, qobject_ref(val)); + } +} + +/** + * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a + * new QString initialised by 'val' is put there. + */ +void qdict_set_default_str(QDict *dst, const char *key, const char *val) +{ + if (qdict_haskey(dst, key)) { + return; + } + + qdict_put_str(dst, key, val); +} + +static void qdict_flatten_qdict(QDict *qdict, QDict *target, + const char *prefix); + +static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) +{ + QObject *value; + const QListEntry *entry; + char *new_key; + int i; + + /* This function is never called with prefix == NULL, i.e., it is always + * called from within qdict_flatten_q(list|dict)(). Therefore, it does not + * need to remove list entries during the iteration (the whole list will be + * deleted eventually anyway from qdict_flatten_qdict()). */ + assert(prefix); + + entry = qlist_first(qlist); + + for (i = 0; entry; entry = qlist_next(entry), i++) { + value = qlist_entry_obj(entry); + new_key = g_strdup_printf("%s.%i", prefix, i); + + if (qobject_type(value) == QTYPE_QDICT) { + qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); + } else if (qobject_type(value) == QTYPE_QLIST) { + qdict_flatten_qlist(qobject_to(QList, value), target, new_key); + } else { + /* All other types are moved to the target unchanged. */ + qdict_put_obj(target, new_key, qobject_ref(value)); + } + + g_free(new_key); + } +} + +static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) +{ + QObject *value; + const QDictEntry *entry, *next; + char *new_key; + bool delete; + + entry = qdict_first(qdict); + + while (entry != NULL) { + + next = qdict_next(qdict, entry); + value = qdict_entry_value(entry); + new_key = NULL; + delete = false; + + if (prefix) { + new_key = g_strdup_printf("%s.%s", prefix, entry->key); + } + + if (qobject_type(value) == QTYPE_QDICT) { + /* Entries of QDicts are processed recursively, the QDict object + * itself disappears. */ + qdict_flatten_qdict(qobject_to(QDict, value), target, + new_key ? new_key : entry->key); + delete = true; + } else if (qobject_type(value) == QTYPE_QLIST) { + qdict_flatten_qlist(qobject_to(QList, value), target, + new_key ? new_key : entry->key); + delete = true; + } else if (prefix) { + /* All other objects are moved to the target unchanged. */ + qdict_put_obj(target, new_key, qobject_ref(value)); + delete = true; + } + + g_free(new_key); + + if (delete) { + qdict_del(qdict, entry->key); + + /* Restart loop after modifying the iterated QDict */ + entry = qdict_first(qdict); + continue; + } + + entry = next; + } +} + +/** + * qdict_flatten(): For each nested QDict with key x, all fields with key y + * are moved to this QDict and their key is renamed to "x.y". For each nested + * QList with key x, the field at index y is moved to this QDict with the key + * "x.y" (i.e., the reverse of what qdict_array_split() does). + * This operation is applied recursively for nested QDicts and QLists. + */ +void qdict_flatten(QDict *qdict) +{ + qdict_flatten_qdict(qdict, qdict, NULL); +} + +/* extract all the src QDict entries starting by start into dst */ +void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start) + +{ + const QDictEntry *entry, *next; + const char *p; + + *dst = qdict_new(); + entry = qdict_first(src); + + while (entry != NULL) { + next = qdict_next(src, entry); + if (strstart(entry->key, start, &p)) { + qdict_put_obj(*dst, p, qobject_ref(entry->value)); + qdict_del(src, entry->key); + } + entry = next; + } +} + +static int qdict_count_prefixed_entries(const QDict *src, const char *start) +{ + const QDictEntry *entry; + int count = 0; + + for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { + if (strstart(entry->key, start, NULL)) { + if (count == INT_MAX) { + return -ERANGE; + } + count++; + } + } + + return count; +} + +/** + * qdict_array_split(): This function moves array-like elements of a QDict into + * a new QList. Every entry in the original QDict with a key "%u" or one + * prefixed "%u.", where %u designates an unsigned integer starting at 0 and + * incrementally counting up, will be moved to a new QDict at index %u in the + * output QList with the key prefix removed, if that prefix is "%u.". If the + * whole key is just "%u", the whole QObject will be moved unchanged without + * creating a new QDict. The function terminates when there is no entry in the + * QDict with a prefix directly (incrementally) following the last one; it also + * returns if there are both entries with "%u" and "%u." for the same index %u. + * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66} + * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66}) + * => [{"a": 42, "b": 23}, {"x": 0}, 66] + * and {"4.y": 1, "o.o": 7} (remainder of the old QDict) + */ +void qdict_array_split(QDict *src, QList **dst) +{ + unsigned i; + + *dst = qlist_new(); + + for (i = 0; i < UINT_MAX; i++) { + QObject *subqobj; + bool is_subqdict; + QDict *subqdict; + char indexstr[32], prefix[32]; + size_t snprintf_ret; + + snprintf_ret = snprintf(indexstr, 32, "%u", i); + assert(snprintf_ret < 32); + + subqobj = qdict_get(src, indexstr); + + snprintf_ret = snprintf(prefix, 32, "%u.", i); + assert(snprintf_ret < 32); + + /* Overflow is the same as positive non-zero results */ + is_subqdict = qdict_count_prefixed_entries(src, prefix); + + /* + * There may be either a single subordinate object (named + * "%u") or multiple objects (each with a key prefixed "%u."), + * but not both. + */ + if (!subqobj == !is_subqdict) { + break; + } + + if (is_subqdict) { + qdict_extract_subqdict(src, &subqdict, prefix); + assert(qdict_size(subqdict) > 0); + } else { + qobject_ref(subqobj); + qdict_del(src, indexstr); + } + + qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict)); + } +} + +/** + * qdict_split_flat_key: + * @key: the key string to split + * @prefix: non-NULL pointer to hold extracted prefix + * @suffix: non-NULL pointer to remaining suffix + * + * Given a flattened key such as 'foo.0.bar', split it into two parts + * at the first '.' separator. Allows double dot ('..') to escape the + * normal separator. + * + * e.g. + * 'foo.0.bar' -> prefix='foo' and suffix='0.bar' + * 'foo..0.bar' -> prefix='foo.0' and suffix='bar' + * + * The '..' sequence will be unescaped in the returned 'prefix' + * string. The 'suffix' string will be left in escaped format, so it + * can be fed back into the qdict_split_flat_key() key as the input + * later. + * + * The caller is responsible for freeing the string returned in @prefix + * using g_free(). + */ +static void qdict_split_flat_key(const char *key, char **prefix, + const char **suffix) +{ + const char *separator; + size_t i, j; + + /* Find first '.' separator, but if there is a pair '..' + * that acts as an escape, so skip over '..' */ + separator = NULL; + do { + if (separator) { + separator += 2; + } else { + separator = key; + } + separator = strchr(separator, '.'); + } while (separator && separator[1] == '.'); + + if (separator) { + *prefix = g_strndup(key, separator - key); + *suffix = separator + 1; + } else { + *prefix = g_strdup(key); + *suffix = NULL; + } + + /* Unescape the '..' sequence into '.' */ + for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) { + if ((*prefix)[i] == '.') { + assert((*prefix)[i + 1] == '.'); + i++; + } + (*prefix)[j] = (*prefix)[i]; + } + (*prefix)[j] = '\0'; +} + +/** + * qdict_is_list: + * @maybe_list: dict to check if keys represent list elements. + * + * Determine whether all keys in @maybe_list are valid list elements. + * If @maybe_list is non-zero in length and all the keys look like + * valid list indexes, this will return 1. If @maybe_list is zero + * length or all keys are non-numeric then it will return 0 to indicate + * it is a normal qdict. If there is a mix of numeric and non-numeric + * keys, or the list indexes are non-contiguous, an error is reported. + * + * Returns: 1 if a valid list, 0 if a dict, -1 on error + */ +static int qdict_is_list(QDict *maybe_list, Error **errp) +{ + const QDictEntry *ent; + ssize_t len = 0; + ssize_t max = -1; + int is_list = -1; + int64_t val; + + for (ent = qdict_first(maybe_list); ent != NULL; + ent = qdict_next(maybe_list, ent)) { + + if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) { + if (is_list == -1) { + is_list = 1; + } else if (!is_list) { + error_setg(errp, + "Cannot mix list and non-list keys"); + return -1; + } + len++; + if (val > max) { + max = val; + } + } else { + if (is_list == -1) { + is_list = 0; + } else if (is_list) { + error_setg(errp, + "Cannot mix list and non-list keys"); + return -1; + } + } + } + + if (is_list == -1) { + assert(!qdict_size(maybe_list)); + is_list = 0; + } + + /* NB this isn't a perfect check - e.g. it won't catch + * a list containing '1', '+1', '01', '3', but that + * does not matter - we've still proved that the + * input is a list. It is up the caller to do a + * stricter check if desired */ + if (len != (max + 1)) { + error_setg(errp, "List indices are not contiguous, " + "saw %zd elements but %zd largest index", + len, max); + return -1; + } + + return is_list; +} + +/** + * qdict_crumple: + * @src: the original flat dictionary (only scalar values) to crumple + * + * Takes a flat dictionary whose keys use '.' separator to indicate + * nesting, and values are scalars, and crumples it into a nested + * structure. + * + * To include a literal '.' in a key name, it must be escaped as '..' + * + * For example, an input of: + * + * { 'foo.0.bar': 'one', 'foo.0.wizz': '1', + * 'foo.1.bar': 'two', 'foo.1.wizz': '2' } + * + * will result in an output of: + * + * { + * 'foo': [ + * { 'bar': 'one', 'wizz': '1' }, + * { 'bar': 'two', 'wizz': '2' } + * ], + * } + * + * The following scenarios in the input dict will result in an + * error being returned: + * + * - Any values in @src are non-scalar types + * - If keys in @src imply that a particular level is both a + * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar". + * - If keys in @src imply that a particular level is a list, + * but the indices are non-contiguous. e.g. "foo.0.bar" and + * "foo.2.bar" without any "foo.1.bar" present. + * - If keys in @src represent list indexes, but are not in + * the "%zu" format. e.g. "foo.+0.bar" + * + * Returns: either a QDict or QList for the nested data structure, or NULL + * on error + */ +QObject *qdict_crumple(const QDict *src, Error **errp) +{ + const QDictEntry *ent; + QDict *two_level, *multi_level = NULL; + QObject *dst = NULL, *child; + size_t i; + char *prefix = NULL; + const char *suffix = NULL; + int is_list; + + two_level = qdict_new(); + + /* Step 1: split our totally flat dict into a two level dict */ + for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) { + if (qobject_type(ent->value) == QTYPE_QDICT || + qobject_type(ent->value) == QTYPE_QLIST) { + error_setg(errp, "Value %s is not a scalar", + ent->key); + goto error; + } + + qdict_split_flat_key(ent->key, &prefix, &suffix); + + child = qdict_get(two_level, prefix); + if (suffix) { + QDict *child_dict = qobject_to(QDict, child); + if (!child_dict) { + if (child) { + error_setg(errp, "Key %s prefix is already set as a scalar", + prefix); + goto error; + } + + child_dict = qdict_new(); + qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); + } + + qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); + } else { + if (child) { + error_setg(errp, "Key %s prefix is already set as a dict", + prefix); + goto error; + } + qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); + } + + g_free(prefix); + prefix = NULL; + } + + /* Step 2: optionally process the two level dict recursively + * into a multi-level dict */ + multi_level = qdict_new(); + for (ent = qdict_first(two_level); ent != NULL; + ent = qdict_next(two_level, ent)) { + QDict *dict = qobject_to(QDict, ent->value); + if (dict) { + child = qdict_crumple(dict, errp); + if (!child) { + goto error; + } + + qdict_put_obj(multi_level, ent->key, child); + } else { + qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value)); + } + } + qobject_unref(two_level); + two_level = NULL; + + /* Step 3: detect if we need to turn our dict into list */ + is_list = qdict_is_list(multi_level, errp); + if (is_list < 0) { + goto error; + } + + if (is_list) { + dst = QOBJECT(qlist_new()); + + for (i = 0; i < qdict_size(multi_level); i++) { + char *key = g_strdup_printf("%zu", i); + + child = qdict_get(multi_level, key); + g_free(key); + + if (!child) { + error_setg(errp, "Missing list index %zu", i); + goto error; + } + + qlist_append_obj(qobject_to(QList, dst), qobject_ref(child)); + } + qobject_unref(multi_level); + multi_level = NULL; + } else { + dst = QOBJECT(multi_level); + } + + return dst; + + error: + g_free(prefix); + qobject_unref(multi_level); + qobject_unref(two_level); + qobject_unref(dst); + return NULL; +} + +/** + * qdict_array_entries(): Returns the number of direct array entries if the + * sub-QDict of src specified by the prefix in subqdict (or src itself for + * prefix == "") is valid as an array, i.e. the length of the created list if + * the sub-QDict would become empty after calling qdict_array_split() on it. If + * the array is not valid, -EINVAL is returned. + */ +int qdict_array_entries(QDict *src, const char *subqdict) +{ + const QDictEntry *entry; + unsigned i; + unsigned entries = 0; + size_t subqdict_len = strlen(subqdict); + + assert(!subqdict_len || subqdict[subqdict_len - 1] == '.'); + + /* qdict_array_split() loops until UINT_MAX, but as we want to return + * negative errors, we only have a signed return value here. Any additional + * entries will lead to -EINVAL. */ + for (i = 0; i < INT_MAX; i++) { + QObject *subqobj; + int subqdict_entries; + char *prefix = g_strdup_printf("%s%u.", subqdict, i); + + subqdict_entries = qdict_count_prefixed_entries(src, prefix); + + /* Remove ending "." */ + prefix[strlen(prefix) - 1] = 0; + subqobj = qdict_get(src, prefix); + + g_free(prefix); + + if (subqdict_entries < 0) { + return subqdict_entries; + } + + /* There may be either a single subordinate object (named "%u") or + * multiple objects (each with a key prefixed "%u."), but not both. */ + if (subqobj && subqdict_entries) { + return -EINVAL; + } else if (!subqobj && !subqdict_entries) { + break; + } + + entries += subqdict_entries ? subqdict_entries : 1; + } + + /* Consider everything handled that isn't part of the given sub-QDict */ + for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { + if (!strstart(qdict_entry_key(entry), subqdict, NULL)) { + entries++; + } + } + + /* Anything left in the sub-QDict that wasn't handled? */ + if (qdict_size(src) != entries) { + return -EINVAL; + } + + return i; +} + +/** + * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all + * elements from src to dest. + * + * If an element from src has a key already present in dest, it will not be + * moved unless overwrite is true. + * + * If overwrite is true, the conflicting values in dest will be discarded and + * replaced by the corresponding values from src. + * + * Therefore, with overwrite being true, the src QDict will always be empty when + * this function returns. If overwrite is false, the src QDict will be empty + * iff there were no conflicts. + */ +void qdict_join(QDict *dest, QDict *src, bool overwrite) +{ + const QDictEntry *entry, *next; + + entry = qdict_first(src); + while (entry) { + next = qdict_next(src, entry); + + if (overwrite || !qdict_haskey(dest, entry->key)) { + qdict_put_obj(dest, entry->key, qobject_ref(entry->value)); + qdict_del(src, entry->key); + } + + entry = next; + } +} + +/** + * qdict_rename_keys(): Rename keys in qdict according to the replacements + * specified in the array renames. The array must be terminated by an entry + * with from = NULL. + * + * The renames are performed individually in the order of the array, so entries + * may be renamed multiple times and may or may not conflict depending on the + * order of the renames array. + * + * Returns true for success, false in error cases. + */ +bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) +{ + QObject *qobj; + + while (renames->from) { + if (qdict_haskey(qdict, renames->from)) { + if (qdict_haskey(qdict, renames->to)) { + error_setg(errp, "'%s' and its alias '%s' can't be used at the " + "same time", renames->to, renames->from); + return false; + } + + qobj = qdict_get(qdict, renames->from); + qdict_put_obj(qdict, renames->to, qobject_ref(qobj)); + qdict_del(qdict, renames->from); + } + + renames++; + } + return true; +} diff --git a/qobject/qdict.c b/qobject/qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/qdict.c +++ b/qobject/qdict.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" -#include "block/qdict.h" #include "qapi/qmp/qnum.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/qbool.h" -#include "qapi/qmp/qlist.h" #include "qapi/qmp/qnull.h" #include "qapi/qmp/qstring.h" -#include "qapi/error.h" -#include "qemu/queue.h" -#include "qemu-common.h" -#include "qemu/cutils.h" /** * qdict_new(): Create a new QDict @@ -XXX,XX +XXX,XX @@ void qdict_destroy_obj(QObject *obj) g_free(qdict); } - -/** - * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the - * value of 'key' in 'src' is copied there (and the refcount increased - * accordingly). - */ -void qdict_copy_default(QDict *dst, QDict *src, const char *key) -{ - QObject *val; - - if (qdict_haskey(dst, key)) { - return; - } - - val = qdict_get(src, key); - if (val) { - qdict_put_obj(dst, key, qobject_ref(val)); - } -} - -/** - * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a - * new QString initialised by 'val' is put there. - */ -void qdict_set_default_str(QDict *dst, const char *key, const char *val) -{ - if (qdict_haskey(dst, key)) { - return; - } - - qdict_put_str(dst, key, val); -} - -static void qdict_flatten_qdict(QDict *qdict, QDict *target, - const char *prefix); - -static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) -{ - QObject *value; - const QListEntry *entry; - char *new_key; - int i; - - /* This function is never called with prefix == NULL, i.e., it is always - * called from within qdict_flatten_q(list|dict)(). Therefore, it does not - * need to remove list entries during the iteration (the whole list will be - * deleted eventually anyway from qdict_flatten_qdict()). */ - assert(prefix); - - entry = qlist_first(qlist); - - for (i = 0; entry; entry = qlist_next(entry), i++) { - value = qlist_entry_obj(entry); - new_key = g_strdup_printf("%s.%i", prefix, i); - - if (qobject_type(value) == QTYPE_QDICT) { - qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); - } else if (qobject_type(value) == QTYPE_QLIST) { - qdict_flatten_qlist(qobject_to(QList, value), target, new_key); - } else { - /* All other types are moved to the target unchanged. */ - qdict_put_obj(target, new_key, qobject_ref(value)); - } - - g_free(new_key); - } -} - -static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) -{ - QObject *value; - const QDictEntry *entry, *next; - char *new_key; - bool delete; - - entry = qdict_first(qdict); - - while (entry != NULL) { - - next = qdict_next(qdict, entry); - value = qdict_entry_value(entry); - new_key = NULL; - delete = false; - - if (prefix) { - new_key = g_strdup_printf("%s.%s", prefix, entry->key); - } - - if (qobject_type(value) == QTYPE_QDICT) { - /* Entries of QDicts are processed recursively, the QDict object - * itself disappears. */ - qdict_flatten_qdict(qobject_to(QDict, value), target, - new_key ? new_key : entry->key); - delete = true; - } else if (qobject_type(value) == QTYPE_QLIST) { - qdict_flatten_qlist(qobject_to(QList, value), target, - new_key ? new_key : entry->key); - delete = true; - } else if (prefix) { - /* All other objects are moved to the target unchanged. */ - qdict_put_obj(target, new_key, qobject_ref(value)); - delete = true; - } - - g_free(new_key); - - if (delete) { - qdict_del(qdict, entry->key); - - /* Restart loop after modifying the iterated QDict */ - entry = qdict_first(qdict); - continue; - } - - entry = next; - } -} - -/** - * qdict_flatten(): For each nested QDict with key x, all fields with key y - * are moved to this QDict and their key is renamed to "x.y". For each nested - * QList with key x, the field at index y is moved to this QDict with the key - * "x.y" (i.e., the reverse of what qdict_array_split() does). - * This operation is applied recursively for nested QDicts and QLists. - */ -void qdict_flatten(QDict *qdict) -{ - qdict_flatten_qdict(qdict, qdict, NULL); -} - -/* extract all the src QDict entries starting by start into dst */ -void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start) - -{ - const QDictEntry *entry, *next; - const char *p; - - *dst = qdict_new(); - entry = qdict_first(src); - - while (entry != NULL) { - next = qdict_next(src, entry); - if (strstart(entry->key, start, &p)) { - qdict_put_obj(*dst, p, qobject_ref(entry->value)); - qdict_del(src, entry->key); - } - entry = next; - } -} - -static int qdict_count_prefixed_entries(const QDict *src, const char *start) -{ - const QDictEntry *entry; - int count = 0; - - for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { - if (strstart(entry->key, start, NULL)) { - if (count == INT_MAX) { - return -ERANGE; - } - count++; - } - } - - return count; -} - -/** - * qdict_array_split(): This function moves array-like elements of a QDict into - * a new QList. Every entry in the original QDict with a key "%u" or one - * prefixed "%u.", where %u designates an unsigned integer starting at 0 and - * incrementally counting up, will be moved to a new QDict at index %u in the - * output QList with the key prefix removed, if that prefix is "%u.". If the - * whole key is just "%u", the whole QObject will be moved unchanged without - * creating a new QDict. The function terminates when there is no entry in the - * QDict with a prefix directly (incrementally) following the last one; it also - * returns if there are both entries with "%u" and "%u." for the same index %u. - * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66} - * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66}) - * => [{"a": 42, "b": 23}, {"x": 0}, 66] - * and {"4.y": 1, "o.o": 7} (remainder of the old QDict) - */ -void qdict_array_split(QDict *src, QList **dst) -{ - unsigned i; - - *dst = qlist_new(); - - for (i = 0; i < UINT_MAX; i++) { - QObject *subqobj; - bool is_subqdict; - QDict *subqdict; - char indexstr[32], prefix[32]; - size_t snprintf_ret; - - snprintf_ret = snprintf(indexstr, 32, "%u", i); - assert(snprintf_ret < 32); - - subqobj = qdict_get(src, indexstr); - - snprintf_ret = snprintf(prefix, 32, "%u.", i); - assert(snprintf_ret < 32); - - /* Overflow is the same as positive non-zero results */ - is_subqdict = qdict_count_prefixed_entries(src, prefix); - - // There may be either a single subordinate object (named "%u") or - // multiple objects (each with a key prefixed "%u."), but not both. - if (!subqobj == !is_subqdict) { - break; - } - - if (is_subqdict) { - qdict_extract_subqdict(src, &subqdict, prefix); - assert(qdict_size(subqdict) > 0); - } else { - qobject_ref(subqobj); - qdict_del(src, indexstr); - } - - qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict)); - } -} - -/** - * qdict_split_flat_key: - * @key: the key string to split - * @prefix: non-NULL pointer to hold extracted prefix - * @suffix: non-NULL pointer to remaining suffix - * - * Given a flattened key such as 'foo.0.bar', split it into two parts - * at the first '.' separator. Allows double dot ('..') to escape the - * normal separator. - * - * e.g. - * 'foo.0.bar' -> prefix='foo' and suffix='0.bar' - * 'foo..0.bar' -> prefix='foo.0' and suffix='bar' - * - * The '..' sequence will be unescaped in the returned 'prefix' - * string. The 'suffix' string will be left in escaped format, so it - * can be fed back into the qdict_split_flat_key() key as the input - * later. - * - * The caller is responsible for freeing the string returned in @prefix - * using g_free(). - */ -static void qdict_split_flat_key(const char *key, char **prefix, - const char **suffix) -{ - const char *separator; - size_t i, j; - - /* Find first '.' separator, but if there is a pair '..' - * that acts as an escape, so skip over '..' */ - separator = NULL; - do { - if (separator) { - separator += 2; - } else { - separator = key; - } - separator = strchr(separator, '.'); - } while (separator && separator[1] == '.'); - - if (separator) { - *prefix = g_strndup(key, separator - key); - *suffix = separator + 1; - } else { - *prefix = g_strdup(key); - *suffix = NULL; - } - - /* Unescape the '..' sequence into '.' */ - for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) { - if ((*prefix)[i] == '.') { - assert((*prefix)[i + 1] == '.'); - i++; - } - (*prefix)[j] = (*prefix)[i]; - } - (*prefix)[j] = '\0'; -} - -/** - * qdict_is_list: - * @maybe_list: dict to check if keys represent list elements. - * - * Determine whether all keys in @maybe_list are valid list elements. - * If @maybe_list is non-zero in length and all the keys look like - * valid list indexes, this will return 1. If @maybe_list is zero - * length or all keys are non-numeric then it will return 0 to indicate - * it is a normal qdict. If there is a mix of numeric and non-numeric - * keys, or the list indexes are non-contiguous, an error is reported. - * - * Returns: 1 if a valid list, 0 if a dict, -1 on error - */ -static int qdict_is_list(QDict *maybe_list, Error **errp) -{ - const QDictEntry *ent; - ssize_t len = 0; - ssize_t max = -1; - int is_list = -1; - int64_t val; - - for (ent = qdict_first(maybe_list); ent != NULL; - ent = qdict_next(maybe_list, ent)) { - - if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) { - if (is_list == -1) { - is_list = 1; - } else if (!is_list) { - error_setg(errp, - "Cannot mix list and non-list keys"); - return -1; - } - len++; - if (val > max) { - max = val; - } - } else { - if (is_list == -1) { - is_list = 0; - } else if (is_list) { - error_setg(errp, - "Cannot mix list and non-list keys"); - return -1; - } - } - } - - if (is_list == -1) { - assert(!qdict_size(maybe_list)); - is_list = 0; - } - - /* NB this isn't a perfect check - e.g. it won't catch - * a list containing '1', '+1', '01', '3', but that - * does not matter - we've still proved that the - * input is a list. It is up the caller to do a - * stricter check if desired */ - if (len != (max + 1)) { - error_setg(errp, "List indices are not contiguous, " - "saw %zd elements but %zd largest index", - len, max); - return -1; - } - - return is_list; -} - -/** - * qdict_crumple: - * @src: the original flat dictionary (only scalar values) to crumple - * - * Takes a flat dictionary whose keys use '.' separator to indicate - * nesting, and values are scalars, and crumples it into a nested - * structure. - * - * To include a literal '.' in a key name, it must be escaped as '..' - * - * For example, an input of: - * - * { 'foo.0.bar': 'one', 'foo.0.wizz': '1', - * 'foo.1.bar': 'two', 'foo.1.wizz': '2' } - * - * will result in an output of: - * - * { - * 'foo': [ - * { 'bar': 'one', 'wizz': '1' }, - * { 'bar': 'two', 'wizz': '2' } - * ], - * } - * - * The following scenarios in the input dict will result in an - * error being returned: - * - * - Any values in @src are non-scalar types - * - If keys in @src imply that a particular level is both a - * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar". - * - If keys in @src imply that a particular level is a list, - * but the indices are non-contiguous. e.g. "foo.0.bar" and - * "foo.2.bar" without any "foo.1.bar" present. - * - If keys in @src represent list indexes, but are not in - * the "%zu" format. e.g. "foo.+0.bar" - * - * Returns: either a QDict or QList for the nested data structure, or NULL - * on error - */ -QObject *qdict_crumple(const QDict *src, Error **errp) -{ - const QDictEntry *ent; - QDict *two_level, *multi_level = NULL; - QObject *dst = NULL, *child; - size_t i; - char *prefix = NULL; - const char *suffix = NULL; - int is_list; - - two_level = qdict_new(); - - /* Step 1: split our totally flat dict into a two level dict */ - for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) { - if (qobject_type(ent->value) == QTYPE_QDICT || - qobject_type(ent->value) == QTYPE_QLIST) { - error_setg(errp, "Value %s is not a scalar", - ent->key); - goto error; - } - - qdict_split_flat_key(ent->key, &prefix, &suffix); - - child = qdict_get(two_level, prefix); - if (suffix) { - QDict *child_dict = qobject_to(QDict, child); - if (!child_dict) { - if (child) { - error_setg(errp, "Key %s prefix is already set as a scalar", - prefix); - goto error; - } - - child_dict = qdict_new(); - qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); - } - - qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); - } else { - if (child) { - error_setg(errp, "Key %s prefix is already set as a dict", - prefix); - goto error; - } - qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); - } - - g_free(prefix); - prefix = NULL; - } - - /* Step 2: optionally process the two level dict recursively - * into a multi-level dict */ - multi_level = qdict_new(); - for (ent = qdict_first(two_level); ent != NULL; - ent = qdict_next(two_level, ent)) { - QDict *dict = qobject_to(QDict, ent->value); - if (dict) { - child = qdict_crumple(dict, errp); - if (!child) { - goto error; - } - - qdict_put_obj(multi_level, ent->key, child); - } else { - qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value)); - } - } - qobject_unref(two_level); - two_level = NULL; - - /* Step 3: detect if we need to turn our dict into list */ - is_list = qdict_is_list(multi_level, errp); - if (is_list < 0) { - goto error; - } - - if (is_list) { - dst = QOBJECT(qlist_new()); - - for (i = 0; i < qdict_size(multi_level); i++) { - char *key = g_strdup_printf("%zu", i); - - child = qdict_get(multi_level, key); - g_free(key); - - if (!child) { - error_setg(errp, "Missing list index %zu", i); - goto error; - } - - qlist_append_obj(qobject_to(QList, dst), qobject_ref(child)); - } - qobject_unref(multi_level); - multi_level = NULL; - } else { - dst = QOBJECT(multi_level); - } - - return dst; - - error: - g_free(prefix); - qobject_unref(multi_level); - qobject_unref(two_level); - qobject_unref(dst); - return NULL; -} - -/** - * qdict_array_entries(): Returns the number of direct array entries if the - * sub-QDict of src specified by the prefix in subqdict (or src itself for - * prefix == "") is valid as an array, i.e. the length of the created list if - * the sub-QDict would become empty after calling qdict_array_split() on it. If - * the array is not valid, -EINVAL is returned. - */ -int qdict_array_entries(QDict *src, const char *subqdict) -{ - const QDictEntry *entry; - unsigned i; - unsigned entries = 0; - size_t subqdict_len = strlen(subqdict); - - assert(!subqdict_len || subqdict[subqdict_len - 1] == '.'); - - /* qdict_array_split() loops until UINT_MAX, but as we want to return - * negative errors, we only have a signed return value here. Any additional - * entries will lead to -EINVAL. */ - for (i = 0; i < INT_MAX; i++) { - QObject *subqobj; - int subqdict_entries; - char *prefix = g_strdup_printf("%s%u.", subqdict, i); - - subqdict_entries = qdict_count_prefixed_entries(src, prefix); - - /* Remove ending "." */ - prefix[strlen(prefix) - 1] = 0; - subqobj = qdict_get(src, prefix); - - g_free(prefix); - - if (subqdict_entries < 0) { - return subqdict_entries; - } - - /* There may be either a single subordinate object (named "%u") or - * multiple objects (each with a key prefixed "%u."), but not both. */ - if (subqobj && subqdict_entries) { - return -EINVAL; - } else if (!subqobj && !subqdict_entries) { - break; - } - - entries += subqdict_entries ? subqdict_entries : 1; - } - - /* Consider everything handled that isn't part of the given sub-QDict */ - for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { - if (!strstart(qdict_entry_key(entry), subqdict, NULL)) { - entries++; - } - } - - /* Anything left in the sub-QDict that wasn't handled? */ - if (qdict_size(src) != entries) { - return -EINVAL; - } - - return i; -} - -/** - * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all - * elements from src to dest. - * - * If an element from src has a key already present in dest, it will not be - * moved unless overwrite is true. - * - * If overwrite is true, the conflicting values in dest will be discarded and - * replaced by the corresponding values from src. - * - * Therefore, with overwrite being true, the src QDict will always be empty when - * this function returns. If overwrite is false, the src QDict will be empty - * iff there were no conflicts. - */ -void qdict_join(QDict *dest, QDict *src, bool overwrite) -{ - const QDictEntry *entry, *next; - - entry = qdict_first(src); - while (entry) { - next = qdict_next(src, entry); - - if (overwrite || !qdict_haskey(dest, entry->key)) { - qdict_put_obj(dest, entry->key, qobject_ref(entry->value)); - qdict_del(src, entry->key); - } - - entry = next; - } -} - -/** - * qdict_rename_keys(): Rename keys in qdict according to the replacements - * specified in the array renames. The array must be terminated by an entry - * with from = NULL. - * - * The renames are performed individually in the order of the array, so entries - * may be renamed multiple times and may or may not conflict depending on the - * order of the renames array. - * - * Returns true for success, false in error cases. - */ -bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) -{ - QObject *qobj; - - while (renames->from) { - if (qdict_haskey(qdict, renames->from)) { - if (qdict_haskey(qdict, renames->to)) { - error_setg(errp, "'%s' and its alias '%s' can't be used at the " - "same time", renames->to, renames->from); - return false; - } - - qobj = qdict_get(qdict, renames->from); - qdict_put_obj(qdict, renames->to, qobject_ref(qobj)); - qdict_del(qdict, renames->from); - } - - renames++; - } - return true; -} diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/check-block-qdict.c @@ -XXX,XX +XXX,XX @@ +/* + * Unit-tests for Block layer QDict extras + * + * Copyright (c) 2013-2018 Red Hat, Inc. + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "block/qdict.h" +#include "qapi/qmp/qlist.h" +#include "qapi/qmp/qnum.h" +#include "qapi/error.h" + +static void qdict_defaults_test(void) +{ + QDict *dict, *copy; + + dict = qdict_new(); + copy = qdict_new(); + + qdict_set_default_str(dict, "foo", "abc"); + qdict_set_default_str(dict, "foo", "def"); + g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc"); + qdict_set_default_str(dict, "bar", "ghi"); + + qdict_copy_default(copy, dict, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc"); + qdict_set_default_str(copy, "bar", "xyz"); + qdict_copy_default(copy, dict, "bar"); + g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz"); + + qobject_unref(copy); + qobject_unref(dict); +} + +static void qdict_flatten_test(void) +{ + QList *list1 = qlist_new(); + QList *list2 = qlist_new(); + QDict *dict1 = qdict_new(); + QDict *dict2 = qdict_new(); + QDict *dict3 = qdict_new(); + + /* + * Test the flattening of + * + * { + * "e": [ + * 42, + * [ + * 23, + * 66, + * { + * "a": 0, + * "b": 1 + * } + * ] + * ], + * "f": { + * "c": 2, + * "d": 3, + * }, + * "g": 4 + * } + * + * to + * + * { + * "e.0": 42, + * "e.1.0": 23, + * "e.1.1": 66, + * "e.1.2.a": 0, + * "e.1.2.b": 1, + * "f.c": 2, + * "f.d": 3, + * "g": 4 + * } + */ + + qdict_put_int(dict1, "a", 0); + qdict_put_int(dict1, "b", 1); + + qlist_append_int(list1, 23); + qlist_append_int(list1, 66); + qlist_append(list1, dict1); + qlist_append_int(list2, 42); + qlist_append(list2, list1); + + qdict_put_int(dict2, "c", 2); + qdict_put_int(dict2, "d", 3); + qdict_put(dict3, "e", list2); + qdict_put(dict3, "f", dict2); + qdict_put_int(dict3, "g", 4); + + qdict_flatten(dict3); + + g_assert(qdict_get_int(dict3, "e.0") == 42); + g_assert(qdict_get_int(dict3, "e.1.0") == 23); + g_assert(qdict_get_int(dict3, "e.1.1") == 66); + g_assert(qdict_get_int(dict3, "e.1.2.a") == 0); + g_assert(qdict_get_int(dict3, "e.1.2.b") == 1); + g_assert(qdict_get_int(dict3, "f.c") == 2); + g_assert(qdict_get_int(dict3, "f.d") == 3); + g_assert(qdict_get_int(dict3, "g") == 4); + + g_assert(qdict_size(dict3) == 8); + + qobject_unref(dict3); +} + +static void qdict_array_split_test(void) +{ + QDict *test_dict = qdict_new(); + QDict *dict1, *dict2; + QNum *int1; + QList *test_list; + + /* + * Test the split of + * + * { + * "1.x": 0, + * "4.y": 1, + * "0.a": 42, + * "o.o": 7, + * "0.b": 23, + * "2": 66 + * } + * + * to + * + * [ + * { + * "a": 42, + * "b": 23 + * }, + * { + * "x": 0 + * }, + * 66 + * ] + * + * and + * + * { + * "4.y": 1, + * "o.o": 7 + * } + * + * (remaining in the old QDict) + * + * This example is given in the comment of qdict_array_split(). + */ + + qdict_put_int(test_dict, "1.x", 0); + qdict_put_int(test_dict, "4.y", 1); + qdict_put_int(test_dict, "0.a", 42); + qdict_put_int(test_dict, "o.o", 7); + qdict_put_int(test_dict, "0.b", 23); + qdict_put_int(test_dict, "2", 66); + + qdict_array_split(test_dict, &test_list); + + dict1 = qobject_to(QDict, qlist_pop(test_list)); + dict2 = qobject_to(QDict, qlist_pop(test_list)); + int1 = qobject_to(QNum, qlist_pop(test_list)); + + g_assert(dict1); + g_assert(dict2); + g_assert(int1); + g_assert(qlist_empty(test_list)); + + qobject_unref(test_list); + + g_assert(qdict_get_int(dict1, "a") == 42); + g_assert(qdict_get_int(dict1, "b") == 23); + + g_assert(qdict_size(dict1) == 2); + + qobject_unref(dict1); + + g_assert(qdict_get_int(dict2, "x") == 0); + + g_assert(qdict_size(dict2) == 1); + + qobject_unref(dict2); + + g_assert_cmpint(qnum_get_int(int1), ==, 66); + + qobject_unref(int1); + + g_assert(qdict_get_int(test_dict, "4.y") == 1); + g_assert(qdict_get_int(test_dict, "o.o") == 7); + + g_assert(qdict_size(test_dict) == 2); + + qobject_unref(test_dict); + + /* + * Test the split of + * + * { + * "0": 42, + * "1": 23, + * "1.x": 84 + * } + * + * to + * + * [ + * 42 + * ] + * + * and + * + * { + * "1": 23, + * "1.x": 84 + * } + * + * That is, test whether splitting stops if there is both an entry with key + * of "%u" and other entries with keys prefixed "%u." for the same index. + */ + + test_dict = qdict_new(); + + qdict_put_int(test_dict, "0", 42); + qdict_put_int(test_dict, "1", 23); + qdict_put_int(test_dict, "1.x", 84); + + qdict_array_split(test_dict, &test_list); + + int1 = qobject_to(QNum, qlist_pop(test_list)); + + g_assert(int1); + g_assert(qlist_empty(test_list)); + + qobject_unref(test_list); + + g_assert_cmpint(qnum_get_int(int1), ==, 42); + + qobject_unref(int1); + + g_assert(qdict_get_int(test_dict, "1") == 23); + g_assert(qdict_get_int(test_dict, "1.x") == 84); + + g_assert(qdict_size(test_dict) == 2); + + qobject_unref(test_dict); +} + +static void qdict_array_entries_test(void) +{ + QDict *dict = qdict_new(); + + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); + + qdict_put_int(dict, "bar", 0); + qdict_put_int(dict, "baz.0", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); + + qdict_put_int(dict, "foo.1", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); + qdict_put_int(dict, "foo.0", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2); + qdict_put_int(dict, "foo.bar", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); + qdict_del(dict, "foo.bar"); + + qdict_put_int(dict, "foo.2.a", 0); + qdict_put_int(dict, "foo.2.b", 0); + qdict_put_int(dict, "foo.2.c", 0); + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); + + qobject_unref(dict); + + dict = qdict_new(); + qdict_put_int(dict, "1", 0); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); + qdict_put_int(dict, "0", 0); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2); + qdict_put_int(dict, "bar", 0); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); + qdict_del(dict, "bar"); + + qdict_put_int(dict, "2.a", 0); + qdict_put_int(dict, "2.b", 0); + qdict_put_int(dict, "2.c", 0); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3); + + qobject_unref(dict); +} + +static void qdict_join_test(void) +{ + QDict *dict1, *dict2; + bool overwrite = false; + int i; + + dict1 = qdict_new(); + dict2 = qdict_new(); + + /* Test everything once without overwrite and once with */ + do { + /* Test empty dicts */ + qdict_join(dict1, dict2, overwrite); + + g_assert(qdict_size(dict1) == 0); + g_assert(qdict_size(dict2) == 0); + + /* First iteration: Test movement */ + /* Second iteration: Test empty source and non-empty destination */ + qdict_put_int(dict2, "foo", 42); + + for (i = 0; i < 2; i++) { + qdict_join(dict1, dict2, overwrite); + + g_assert(qdict_size(dict1) == 1); + g_assert(qdict_size(dict2) == 0); + + g_assert(qdict_get_int(dict1, "foo") == 42); + } + + /* Test non-empty source and destination without conflict */ + qdict_put_int(dict2, "bar", 23); + + qdict_join(dict1, dict2, overwrite); + + g_assert(qdict_size(dict1) == 2); + g_assert(qdict_size(dict2) == 0); + + g_assert(qdict_get_int(dict1, "foo") == 42); + g_assert(qdict_get_int(dict1, "bar") == 23); + + /* Test conflict */ + qdict_put_int(dict2, "foo", 84); + + qdict_join(dict1, dict2, overwrite); + + g_assert(qdict_size(dict1) == 2); + g_assert(qdict_size(dict2) == !overwrite); + + g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42)); + g_assert(qdict_get_int(dict1, "bar") == 23); + + if (!overwrite) { + g_assert(qdict_get_int(dict2, "foo") == 84); + } + + /* Check the references */ + g_assert(qdict_get(dict1, "foo")->base.refcnt == 1); + g_assert(qdict_get(dict1, "bar")->base.refcnt == 1); + + if (!overwrite) { + g_assert(qdict_get(dict2, "foo")->base.refcnt == 1); + } + + /* Clean up */ + qdict_del(dict1, "foo"); + qdict_del(dict1, "bar"); + + if (!overwrite) { + qdict_del(dict2, "foo"); + } + } while (overwrite ^= true); + + qobject_unref(dict1); + qobject_unref(dict2); +} + +static void qdict_crumple_test_recursive(void) +{ + QDict *src, *dst, *rule, *vnc, *acl, *listen; + QList *rules; + + src = qdict_new(); + qdict_put_str(src, "vnc.listen.addr", "127.0.0.1"); + qdict_put_str(src, "vnc.listen.port", "5901"); + qdict_put_str(src, "vnc.acl.rules.0.match", "fred"); + qdict_put_str(src, "vnc.acl.rules.0.policy", "allow"); + qdict_put_str(src, "vnc.acl.rules.1.match", "bob"); + qdict_put_str(src, "vnc.acl.rules.1.policy", "deny"); + qdict_put_str(src, "vnc.acl.default", "deny"); + qdict_put_str(src, "vnc.acl..name", "acl0"); + qdict_put_str(src, "vnc.acl.rule..name", "acl0"); + + dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); + g_assert(dst); + g_assert_cmpint(qdict_size(dst), ==, 1); + + vnc = qdict_get_qdict(dst, "vnc"); + g_assert(vnc); + g_assert_cmpint(qdict_size(vnc), ==, 3); + + listen = qdict_get_qdict(vnc, "listen"); + g_assert(listen); + g_assert_cmpint(qdict_size(listen), ==, 2); + g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr")); + g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port")); + + acl = qdict_get_qdict(vnc, "acl"); + g_assert(acl); + g_assert_cmpint(qdict_size(acl), ==, 3); + + rules = qdict_get_qlist(acl, "rules"); + g_assert(rules); + g_assert_cmpint(qlist_size(rules), ==, 2); + + rule = qobject_to(QDict, qlist_pop(rules)); + g_assert(rule); + g_assert_cmpint(qdict_size(rule), ==, 2); + g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match")); + g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy")); + qobject_unref(rule); + + rule = qobject_to(QDict, qlist_pop(rules)); + g_assert(rule); + g_assert_cmpint(qdict_size(rule), ==, 2); + g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match")); + g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy")); + qobject_unref(rule); + + /* With recursive crumpling, we should see all names unescaped */ + g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); + g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); + + qobject_unref(src); + qobject_unref(dst); +} + +static void qdict_crumple_test_empty(void) +{ + QDict *src, *dst; + + src = qdict_new(); + + dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); + + g_assert_cmpint(qdict_size(dst), ==, 0); + + qobject_unref(src); + qobject_unref(dst); +} + +static int qdict_count_entries(QDict *dict) +{ + const QDictEntry *e; + int count = 0; + + for (e = qdict_first(dict); e; e = qdict_next(dict, e)) { + count++; + } + + return count; +} + +static void qdict_rename_keys_test(void) +{ + QDict *dict = qdict_new(); + QDict *copy; + QDictRenames *renames; + Error *local_err = NULL; + + qdict_put_str(dict, "abc", "foo"); + qdict_put_str(dict, "abcdef", "bar"); + qdict_put_int(dict, "number", 42); + qdict_put_bool(dict, "flag", true); + qdict_put_null(dict, "nothing"); + + /* Empty rename list */ + renames = (QDictRenames[]) { + { NULL, "this can be anything" } + }; + copy = qdict_clone_shallow(dict); + qdict_rename_keys(copy, renames, &error_abort); + + g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); + g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); + g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); + g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + + qobject_unref(copy); + + /* Simple rename of all entries */ + renames = (QDictRenames[]) { + { "abc", "str1" }, + { "abcdef", "str2" }, + { "number", "int" }, + { "flag", "bool" }, + { "nothing", "null" }, + { NULL , NULL } + }; + copy = qdict_clone_shallow(dict); + qdict_rename_keys(copy, renames, &error_abort); + + g_assert(!qdict_haskey(copy, "abc")); + g_assert(!qdict_haskey(copy, "abcdef")); + g_assert(!qdict_haskey(copy, "number")); + g_assert(!qdict_haskey(copy, "flag")); + g_assert(!qdict_haskey(copy, "nothing")); + + g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar"); + g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42); + g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true); + g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + + qobject_unref(copy); + + /* Renames are processed top to bottom */ + renames = (QDictRenames[]) { + { "abc", "tmp" }, + { "abcdef", "abc" }, + { "number", "abcdef" }, + { "flag", "number" }, + { "nothing", "flag" }, + { "tmp", "nothing" }, + { NULL , NULL } + }; + copy = qdict_clone_shallow(dict); + qdict_rename_keys(copy, renames, &error_abort); + + g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar"); + g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42); + g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true); + g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL); + g_assert(!qdict_haskey(copy, "tmp")); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + + qobject_unref(copy); + + /* Conflicting rename */ + renames = (QDictRenames[]) { + { "abcdef", "abc" }, + { NULL , NULL } + }; + copy = qdict_clone_shallow(dict); + qdict_rename_keys(copy, renames, &local_err); + + g_assert(local_err != NULL); + error_free(local_err); + local_err = NULL; + + g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); + g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); + g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); + g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); + g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + + qobject_unref(copy); + + /* Renames in an empty dict */ + renames = (QDictRenames[]) { + { "abcdef", "abc" }, + { NULL , NULL } + }; + + qobject_unref(dict); + dict = qdict_new(); + + qdict_rename_keys(dict, renames, &error_abort); + g_assert(qdict_first(dict) == NULL); + + qobject_unref(dict); +} + +static void qdict_crumple_test_bad_inputs(void) +{ + QDict *src; + Error *error = NULL; + + src = qdict_new(); + /* rule.0 can't be both a string and a dict */ + qdict_put_str(src, "rule.0", "fred"); + qdict_put_str(src, "rule.0.policy", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); + + src = qdict_new(); + /* rule can't be both a list and a dict */ + qdict_put_str(src, "rule.0", "fred"); + qdict_put_str(src, "rule.a", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); + + src = qdict_new(); + /* The input should be flat, ie no dicts or lists */ + qdict_put(src, "rule.a", qdict_new()); + qdict_put_str(src, "rule.b", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); + + src = qdict_new(); + /* List indexes must not have gaps */ + qdict_put_str(src, "rule.0", "deny"); + qdict_put_str(src, "rule.3", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); + + src = qdict_new(); + /* List indexes must be in %zu format */ + qdict_put_str(src, "rule.0", "deny"); + qdict_put_str(src, "rule.+1", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); + g_assert(error != NULL); + error_free(error); + error = NULL; + qobject_unref(src); +} + +int main(int argc, char **argv) +{ + g_test_init(&argc, &argv, NULL); + + g_test_add_func("/public/defaults", qdict_defaults_test); + g_test_add_func("/public/flatten", qdict_flatten_test); + g_test_add_func("/public/array_split", qdict_array_split_test); + g_test_add_func("/public/array_entries", qdict_array_entries_test); + g_test_add_func("/public/join", qdict_join_test); + g_test_add_func("/public/crumple/recursive", + qdict_crumple_test_recursive); + g_test_add_func("/public/crumple/empty", + qdict_crumple_test_empty); + g_test_add_func("/public/crumple/bad_inputs", + qdict_crumple_test_bad_inputs); + + g_test_add_func("/public/rename_keys", qdict_rename_keys_test); + + return g_test_run(); +} diff --git a/tests/check-qdict.c b/tests/check-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-qdict.c +++ b/tests/check-qdict.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" -#include "block/qdict.h" #include "qapi/qmp/qdict.h" -#include "qapi/qmp/qlist.h" -#include "qapi/qmp/qnum.h" -#include "qapi/qmp/qstring.h" -#include "qapi/error.h" -#include "qemu-common.h" /* * Public Interface test-cases @@ -XXX,XX +XXX,XX @@ static void qdict_get_try_str_test(void) qobject_unref(tests_dict); } -static void qdict_defaults_test(void) -{ - QDict *dict, *copy; - - dict = qdict_new(); - copy = qdict_new(); - - qdict_set_default_str(dict, "foo", "abc"); - qdict_set_default_str(dict, "foo", "def"); - g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc"); - qdict_set_default_str(dict, "bar", "ghi"); - - qdict_copy_default(copy, dict, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc"); - qdict_set_default_str(copy, "bar", "xyz"); - qdict_copy_default(copy, dict, "bar"); - g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz"); - - qobject_unref(copy); - qobject_unref(dict); -} - static void qdict_haskey_not_test(void) { QDict *tests_dict = qdict_new(); @@ -XXX,XX +XXX,XX @@ static void qdict_iterapi_test(void) qobject_unref(tests_dict); } -static void qdict_flatten_test(void) -{ - QList *list1 = qlist_new(); - QList *list2 = qlist_new(); - QDict *dict1 = qdict_new(); - QDict *dict2 = qdict_new(); - QDict *dict3 = qdict_new(); - - /* - * Test the flattening of - * - * { - * "e": [ - * 42, - * [ - * 23, - * 66, - * { - * "a": 0, - * "b": 1 - * } - * ] - * ], - * "f": { - * "c": 2, - * "d": 3, - * }, - * "g": 4 - * } - * - * to - * - * { - * "e.0": 42, - * "e.1.0": 23, - * "e.1.1": 66, - * "e.1.2.a": 0, - * "e.1.2.b": 1, - * "f.c": 2, - * "f.d": 3, - * "g": 4 - * } - */ - - qdict_put_int(dict1, "a", 0); - qdict_put_int(dict1, "b", 1); - - qlist_append_int(list1, 23); - qlist_append_int(list1, 66); - qlist_append(list1, dict1); - qlist_append_int(list2, 42); - qlist_append(list2, list1); - - qdict_put_int(dict2, "c", 2); - qdict_put_int(dict2, "d", 3); - qdict_put(dict3, "e", list2); - qdict_put(dict3, "f", dict2); - qdict_put_int(dict3, "g", 4); - - qdict_flatten(dict3); - - g_assert(qdict_get_int(dict3, "e.0") == 42); - g_assert(qdict_get_int(dict3, "e.1.0") == 23); - g_assert(qdict_get_int(dict3, "e.1.1") == 66); - g_assert(qdict_get_int(dict3, "e.1.2.a") == 0); - g_assert(qdict_get_int(dict3, "e.1.2.b") == 1); - g_assert(qdict_get_int(dict3, "f.c") == 2); - g_assert(qdict_get_int(dict3, "f.d") == 3); - g_assert(qdict_get_int(dict3, "g") == 4); - - g_assert(qdict_size(dict3) == 8); - - qobject_unref(dict3); -} - -static void qdict_array_split_test(void) -{ - QDict *test_dict = qdict_new(); - QDict *dict1, *dict2; - QNum *int1; - QList *test_list; - - /* - * Test the split of - * - * { - * "1.x": 0, - * "4.y": 1, - * "0.a": 42, - * "o.o": 7, - * "0.b": 23, - * "2": 66 - * } - * - * to - * - * [ - * { - * "a": 42, - * "b": 23 - * }, - * { - * "x": 0 - * }, - * 66 - * ] - * - * and - * - * { - * "4.y": 1, - * "o.o": 7 - * } - * - * (remaining in the old QDict) - * - * This example is given in the comment of qdict_array_split(). - */ - - qdict_put_int(test_dict, "1.x", 0); - qdict_put_int(test_dict, "4.y", 1); - qdict_put_int(test_dict, "0.a", 42); - qdict_put_int(test_dict, "o.o", 7); - qdict_put_int(test_dict, "0.b", 23); - qdict_put_int(test_dict, "2", 66); - - qdict_array_split(test_dict, &test_list); - - dict1 = qobject_to(QDict, qlist_pop(test_list)); - dict2 = qobject_to(QDict, qlist_pop(test_list)); - int1 = qobject_to(QNum, qlist_pop(test_list)); - - g_assert(dict1); - g_assert(dict2); - g_assert(int1); - g_assert(qlist_empty(test_list)); - - qobject_unref(test_list); - - g_assert(qdict_get_int(dict1, "a") == 42); - g_assert(qdict_get_int(dict1, "b") == 23); - - g_assert(qdict_size(dict1) == 2); - - qobject_unref(dict1); - - g_assert(qdict_get_int(dict2, "x") == 0); - - g_assert(qdict_size(dict2) == 1); - - qobject_unref(dict2); - - g_assert_cmpint(qnum_get_int(int1), ==, 66); - - qobject_unref(int1); - - g_assert(qdict_get_int(test_dict, "4.y") == 1); - g_assert(qdict_get_int(test_dict, "o.o") == 7); - - g_assert(qdict_size(test_dict) == 2); - - qobject_unref(test_dict); - - /* - * Test the split of - * - * { - * "0": 42, - * "1": 23, - * "1.x": 84 - * } - * - * to - * - * [ - * 42 - * ] - * - * and - * - * { - * "1": 23, - * "1.x": 84 - * } - * - * That is, test whether splitting stops if there is both an entry with key - * of "%u" and other entries with keys prefixed "%u." for the same index. - */ - - test_dict = qdict_new(); - - qdict_put_int(test_dict, "0", 42); - qdict_put_int(test_dict, "1", 23); - qdict_put_int(test_dict, "1.x", 84); - - qdict_array_split(test_dict, &test_list); - - int1 = qobject_to(QNum, qlist_pop(test_list)); - - g_assert(int1); - g_assert(qlist_empty(test_list)); - - qobject_unref(test_list); - - g_assert_cmpint(qnum_get_int(int1), ==, 42); - - qobject_unref(int1); - - g_assert(qdict_get_int(test_dict, "1") == 23); - g_assert(qdict_get_int(test_dict, "1.x") == 84); - - g_assert(qdict_size(test_dict) == 2); - - qobject_unref(test_dict); -} - -static void qdict_array_entries_test(void) -{ - QDict *dict = qdict_new(); - - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); - - qdict_put_int(dict, "bar", 0); - qdict_put_int(dict, "baz.0", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); - - qdict_put_int(dict, "foo.1", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); - qdict_put_int(dict, "foo.0", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2); - qdict_put_int(dict, "foo.bar", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); - qdict_del(dict, "foo.bar"); - - qdict_put_int(dict, "foo.2.a", 0); - qdict_put_int(dict, "foo.2.b", 0); - qdict_put_int(dict, "foo.2.c", 0); - g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); - - qobject_unref(dict); - - dict = qdict_new(); - qdict_put_int(dict, "1", 0); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); - qdict_put_int(dict, "0", 0); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2); - qdict_put_int(dict, "bar", 0); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); - qdict_del(dict, "bar"); - - qdict_put_int(dict, "2.a", 0); - qdict_put_int(dict, "2.b", 0); - qdict_put_int(dict, "2.c", 0); - g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3); - - qobject_unref(dict); -} - -static void qdict_join_test(void) -{ - QDict *dict1, *dict2; - bool overwrite = false; - int i; - - dict1 = qdict_new(); - dict2 = qdict_new(); - - /* Test everything once without overwrite and once with */ - do - { - /* Test empty dicts */ - qdict_join(dict1, dict2, overwrite); - - g_assert(qdict_size(dict1) == 0); - g_assert(qdict_size(dict2) == 0); - - /* First iteration: Test movement */ - /* Second iteration: Test empty source and non-empty destination */ - qdict_put_int(dict2, "foo", 42); - - for (i = 0; i < 2; i++) { - qdict_join(dict1, dict2, overwrite); - - g_assert(qdict_size(dict1) == 1); - g_assert(qdict_size(dict2) == 0); - - g_assert(qdict_get_int(dict1, "foo") == 42); - } - - /* Test non-empty source and destination without conflict */ - qdict_put_int(dict2, "bar", 23); - - qdict_join(dict1, dict2, overwrite); - - g_assert(qdict_size(dict1) == 2); - g_assert(qdict_size(dict2) == 0); - - g_assert(qdict_get_int(dict1, "foo") == 42); - g_assert(qdict_get_int(dict1, "bar") == 23); - - /* Test conflict */ - qdict_put_int(dict2, "foo", 84); - - qdict_join(dict1, dict2, overwrite); - - g_assert(qdict_size(dict1) == 2); - g_assert(qdict_size(dict2) == !overwrite); - - g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42)); - g_assert(qdict_get_int(dict1, "bar") == 23); - - if (!overwrite) { - g_assert(qdict_get_int(dict2, "foo") == 84); - } - - /* Check the references */ - g_assert(qdict_get(dict1, "foo")->base.refcnt == 1); - g_assert(qdict_get(dict1, "bar")->base.refcnt == 1); - - if (!overwrite) { - g_assert(qdict_get(dict2, "foo")->base.refcnt == 1); - } - - /* Clean up */ - qdict_del(dict1, "foo"); - qdict_del(dict1, "bar"); - - if (!overwrite) { - qdict_del(dict2, "foo"); - } - } - while (overwrite ^= true); - - qobject_unref(dict1); - qobject_unref(dict2); -} - -static void qdict_crumple_test_recursive(void) -{ - QDict *src, *dst, *rule, *vnc, *acl, *listen; - QList *rules; - - src = qdict_new(); - qdict_put_str(src, "vnc.listen.addr", "127.0.0.1"); - qdict_put_str(src, "vnc.listen.port", "5901"); - qdict_put_str(src, "vnc.acl.rules.0.match", "fred"); - qdict_put_str(src, "vnc.acl.rules.0.policy", "allow"); - qdict_put_str(src, "vnc.acl.rules.1.match", "bob"); - qdict_put_str(src, "vnc.acl.rules.1.policy", "deny"); - qdict_put_str(src, "vnc.acl.default", "deny"); - qdict_put_str(src, "vnc.acl..name", "acl0"); - qdict_put_str(src, "vnc.acl.rule..name", "acl0"); - - dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); - g_assert(dst); - g_assert_cmpint(qdict_size(dst), ==, 1); - - vnc = qdict_get_qdict(dst, "vnc"); - g_assert(vnc); - g_assert_cmpint(qdict_size(vnc), ==, 3); - - listen = qdict_get_qdict(vnc, "listen"); - g_assert(listen); - g_assert_cmpint(qdict_size(listen), ==, 2); - g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr")); - g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port")); - - acl = qdict_get_qdict(vnc, "acl"); - g_assert(acl); - g_assert_cmpint(qdict_size(acl), ==, 3); - - rules = qdict_get_qlist(acl, "rules"); - g_assert(rules); - g_assert_cmpint(qlist_size(rules), ==, 2); - - rule = qobject_to(QDict, qlist_pop(rules)); - g_assert(rule); - g_assert_cmpint(qdict_size(rule), ==, 2); - g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match")); - g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy")); - qobject_unref(rule); - - rule = qobject_to(QDict, qlist_pop(rules)); - g_assert(rule); - g_assert_cmpint(qdict_size(rule), ==, 2); - g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match")); - g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy")); - qobject_unref(rule); - - /* With recursive crumpling, we should see all names unescaped */ - g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); - g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); - - qobject_unref(src); - qobject_unref(dst); -} - -static void qdict_crumple_test_empty(void) -{ - QDict *src, *dst; - - src = qdict_new(); - - dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); - - g_assert_cmpint(qdict_size(dst), ==, 0); - - qobject_unref(src); - qobject_unref(dst); -} - -static int qdict_count_entries(QDict *dict) -{ - const QDictEntry *e; - int count = 0; - - for (e = qdict_first(dict); e; e = qdict_next(dict, e)) { - count++; - } - - return count; -} - -static void qdict_rename_keys_test(void) -{ - QDict *dict = qdict_new(); - QDict *copy; - QDictRenames *renames; - Error *local_err = NULL; - - qdict_put_str(dict, "abc", "foo"); - qdict_put_str(dict, "abcdef", "bar"); - qdict_put_int(dict, "number", 42); - qdict_put_bool(dict, "flag", true); - qdict_put_null(dict, "nothing"); - - /* Empty rename list */ - renames = (QDictRenames[]) { - { NULL, "this can be anything" } - }; - copy = qdict_clone_shallow(dict); - qdict_rename_keys(copy, renames, &error_abort); - - g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); - g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); - g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); - g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); - g_assert_cmpint(qdict_count_entries(copy), ==, 5); - - qobject_unref(copy); - - /* Simple rename of all entries */ - renames = (QDictRenames[]) { - { "abc", "str1" }, - { "abcdef", "str2" }, - { "number", "int" }, - { "flag", "bool" }, - { "nothing", "null" }, - { NULL , NULL } - }; - copy = qdict_clone_shallow(dict); - qdict_rename_keys(copy, renames, &error_abort); - - g_assert(!qdict_haskey(copy, "abc")); - g_assert(!qdict_haskey(copy, "abcdef")); - g_assert(!qdict_haskey(copy, "number")); - g_assert(!qdict_haskey(copy, "flag")); - g_assert(!qdict_haskey(copy, "nothing")); - - g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar"); - g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42); - g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true); - g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL); - g_assert_cmpint(qdict_count_entries(copy), ==, 5); - - qobject_unref(copy); - - /* Renames are processed top to bottom */ - renames = (QDictRenames[]) { - { "abc", "tmp" }, - { "abcdef", "abc" }, - { "number", "abcdef" }, - { "flag", "number" }, - { "nothing", "flag" }, - { "tmp", "nothing" }, - { NULL , NULL } - }; - copy = qdict_clone_shallow(dict); - qdict_rename_keys(copy, renames, &error_abort); - - g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar"); - g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42); - g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true); - g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL); - g_assert(!qdict_haskey(copy, "tmp")); - g_assert_cmpint(qdict_count_entries(copy), ==, 5); - - qobject_unref(copy); - - /* Conflicting rename */ - renames = (QDictRenames[]) { - { "abcdef", "abc" }, - { NULL , NULL } - }; - copy = qdict_clone_shallow(dict); - qdict_rename_keys(copy, renames, &local_err); - - g_assert(local_err != NULL); - error_free(local_err); - local_err = NULL; - - g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); - g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); - g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); - g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); - g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); - g_assert_cmpint(qdict_count_entries(copy), ==, 5); - - qobject_unref(copy); - - /* Renames in an empty dict */ - renames = (QDictRenames[]) { - { "abcdef", "abc" }, - { NULL , NULL } - }; - - qobject_unref(dict); - dict = qdict_new(); - - qdict_rename_keys(dict, renames, &error_abort); - g_assert(qdict_first(dict) == NULL); - - qobject_unref(dict); -} - -static void qdict_crumple_test_bad_inputs(void) -{ - QDict *src; - Error *error = NULL; - - src = qdict_new(); - /* rule.0 can't be both a string and a dict */ - qdict_put_str(src, "rule.0", "fred"); - qdict_put_str(src, "rule.0.policy", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); - - src = qdict_new(); - /* rule can't be both a list and a dict */ - qdict_put_str(src, "rule.0", "fred"); - qdict_put_str(src, "rule.a", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); - - src = qdict_new(); - /* The input should be flat, ie no dicts or lists */ - qdict_put(src, "rule.a", qdict_new()); - qdict_put_str(src, "rule.b", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); - - src = qdict_new(); - /* List indexes must not have gaps */ - qdict_put_str(src, "rule.0", "deny"); - qdict_put_str(src, "rule.3", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); - - src = qdict_new(); - /* List indexes must be in %zu format */ - qdict_put_str(src, "rule.0", "deny"); - qdict_put_str(src, "rule.+1", "allow"); - - g_assert(qdict_crumple(src, &error) == NULL); - g_assert(error != NULL); - error_free(error); - error = NULL; - qobject_unref(src); -} - /* * Errors test-cases */ @@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv) g_test_add_func("/public/get_try_int", qdict_get_try_int_test); g_test_add_func("/public/get_str", qdict_get_str_test); g_test_add_func("/public/get_try_str", qdict_get_try_str_test); - g_test_add_func("/public/defaults", qdict_defaults_test); g_test_add_func("/public/haskey_not", qdict_haskey_not_test); g_test_add_func("/public/haskey", qdict_haskey_test); g_test_add_func("/public/del", qdict_del_test); g_test_add_func("/public/to_qdict", qobject_to_qdict_test); g_test_add_func("/public/iterapi", qdict_iterapi_test); - g_test_add_func("/public/flatten", qdict_flatten_test); - g_test_add_func("/public/array_split", qdict_array_split_test); - g_test_add_func("/public/array_entries", qdict_array_entries_test); - g_test_add_func("/public/join", qdict_join_test); g_test_add_func("/errors/put_exists", qdict_put_exists_test); g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test); - g_test_add_func("/public/crumple/recursive", - qdict_crumple_test_recursive); - g_test_add_func("/public/crumple/empty", - qdict_crumple_test_empty); - g_test_add_func("/public/crumple/bad_inputs", - qdict_crumple_test_bad_inputs); - - g_test_add_func("/public/rename_keys", qdict_rename_keys_test); - /* The Big one */ if (g_test_slow()) { g_test_add_func("/stress/test", qdict_stress_test); diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: qemu-img* F: qemu-io* F: tests/qemu-iotests/ F: util/qemu-progress.c +F: qobject/block-qdict.c +F: test/check-block-qdict.c T: git git://repo.or.cz/qemu/kevin.git block Block I/O path diff --git a/qobject/Makefile.objs b/qobject/Makefile.objs index XXXXXXX..XXXXXXX 100644 --- a/qobject/Makefile.objs +++ b/qobject/Makefile.objs @@ -XXX,XX +XXX,XX @@ util-obj-y = qnull.o qnum.o qstring.o qdict.o qlist.o qbool.o qlit.o util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.o +util-obj-y += block-qdict.o diff --git a/tests/Makefile.include b/tests/Makefile.include index XXXXXXX..XXXXXXX 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -XXX,XX +XXX,XX @@ SYSEMU_TARGET_LIST := $(subst -softmmu.mak,,$(notdir \ check-unit-y = tests/check-qdict$(EXESUF) gcov-files-check-qdict-y = qobject/qdict.c +check-unit-y = tests/check-block-qdict$(EXESUF) +gcov-files-check-block-qdict-y = qobject/block-qdict.c check-unit-y += tests/test-char$(EXESUF) gcov-files-check-qdict-y = chardev/char.c check-unit-y += tests/check-qnum$(EXESUF) @@ -XXX,XX +XXX,XX @@ GENERATED_FILES += tests/test-qapi-types.h tests/test-qapi-visit.h \ test-obj-y = tests/check-qnum.o tests/check-qstring.o tests/check-qdict.o \ tests/check-qlist.o tests/check-qnull.o tests/check-qobject.o \ tests/check-qjson.o tests/check-qlit.o \ + tests/check-block-qtest.o \ tests/test-coroutine.o tests/test-string-output-visitor.o \ tests/test-string-input-visitor.o tests/test-qobject-output-visitor.o \ tests/test-clone-visitor.o \ @@ -XXX,XX +XXX,XX @@ test-block-obj-y = $(block-obj-y) $(test-io-obj-y) tests/iothread.o tests/check-qnum$(EXESUF): tests/check-qnum.o $(test-util-obj-y) tests/check-qstring$(EXESUF): tests/check-qstring.o $(test-util-obj-y) tests/check-qdict$(EXESUF): tests/check-qdict.o $(test-util-obj-y) +tests/check-block-qdict$(EXESUF): tests/check-block-qdict.o $(test-util-obj-y) tests/check-qlist$(EXESUF): tests/check-qlist.o $(test-util-obj-y) tests/check-qnull$(EXESUF): tests/check-qnull.o $(test-util-obj-y) tests/check-qobject$(EXESUF): tests/check-qobject.o $(test-util-obj-y) -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Configuration flows through the block subsystem in a rather peculiar way. Configuration made with -drive enters it as QemuOpts. Configuration made with -blockdev / blockdev-add enters it as QAPI type BlockdevOptions. The block subsystem uses QDict, QemuOpts and QAPI types internally. The precise flow is next to impossible to explain (I tried for this commit message, but gave up after wasting several hours). What I can explain is a flaw in the BlockDriver interface that leads to this bug: $ qemu-system-x86_64 -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234 qemu-system-x86_64: -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234: Internal error: parameter user invalid QMP blockdev-add is broken the same way. Here's what happens. The block layer passes configuration represented as flat QDict (with dotted keys) to BlockDriver methods .bdrv_file_open(). The QDict's members are typed according to the QAPI schema. nfs_file_open() converts it to QAPI type BlockdevOptionsNfs, with qdict_crumple() and a qobject input visitor. This visitor comes in two flavors. The plain flavor requires scalars to be typed according to the QAPI schema. That's the case here. The keyval flavor requires string scalars. That's not the case here. nfs_file_open() uses the latter, and promptly falls apart for members @user, @group, @tcp-syn-count, @readahead-size, @page-cache-size, @debug. Switching to the plain flavor would fix -blockdev, but break -drive, because there the scalars arrive in nfs_file_open() as strings. The proper fix would be to replace the QDict by QAPI type BlockdevOptions in the BlockDriver interface. Sadly, that's beyond my reach right now. Next best would be to fix the block layer to always pass correctly typed QDicts to the BlockDriver methods. Also beyond my reach. What I can do is throw another hack onto the pile: have nfs_file_open() convert all members to string, so use of the keyval flavor actually works, by replacing qdict_crumple() by new function qdict_crumple_for_keyval_qiv(). The pattern "pass result of qdict_crumple() to qobject_input_visitor_new_keyval()" occurs several times more: * qemu_rbd_open() Same issue as nfs_file_open(), but since BlockdevOptionsRbd has only string members, its only a latent bug. Fix it anyway. * parallels_co_create_opts(), qcow_co_create_opts(), qcow2_co_create_opts(), bdrv_qed_co_create_opts(), sd_co_create_opts(), vhdx_co_create_opts(), vpc_co_create_opts() These work, because they create the QDict with qemu_opts_to_qdict_filtered(), which creates only string scalars. The function sports a TODO comment asking for better typing; that's going to be fun. Use qdict_crumple_for_keyval_qiv() to be safe. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/qdict.h | 1 + block/nfs.c | 2 +- block/parallels.c | 2 +- block/qcow.c | 2 +- block/qcow2.c | 2 +- block/qed.c | 2 +- block/rbd.c | 2 +- block/sheepdog.c | 2 +- block/vhdx.c | 2 +- block/vpc.c | 2 +- qobject/block-qdict.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++ 11 files changed, 67 insertions(+), 9 deletions(-) diff --git a/include/block/qdict.h b/include/block/qdict.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/qdict.h +++ b/include/block/qdict.h @@ -XXX,XX +XXX,XX @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); void qdict_array_split(QDict *src, QList **dst); int qdict_array_entries(QDict *src, const char *subqdict); QObject *qdict_crumple(const QDict *src, Error **errp); +QObject *qdict_crumple_for_keyval_qiv(QDict *qdict, Error **errp); void qdict_flatten(QDict *qdict); typedef struct QDictRenames { diff --git a/block/nfs.c b/block/nfs.c index XXXXXXX..XXXXXXX 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, const QDictEntry *e; Error *local_err = NULL; - crumpled = qdict_crumple(options, errp); + crumpled = qdict_crumple_for_keyval_qiv(options, errp); if (crumpled == NULL) { return NULL; } diff --git a/block/parallels.c b/block/parallels.c index XXXXXXX..XXXXXXX 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "parallels"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "qcow"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt qdict_put_str(qdict, "file", bs->node_name); /* Now get the QAPI type BlockdevCreateOptions */ - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/qed.c b/block/qed.c index XXXXXXX..XXXXXXX 100644 --- a/block/qed.c +++ b/block/qed.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "qed"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, } /* Convert the remaining options into a QAPI object */ - crumpled = qdict_crumple(options, errp); + crumpled = qdict_crumple_for_keyval_qiv(options, errp); if (crumpled == NULL) { r = -EINVAL; goto out; diff --git a/block/sheepdog.c b/block/sheepdog.c index XXXXXXX..XXXXXXX 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, } /* Get the QAPI object */ - crumpled = qdict_crumple(qdict, errp); + crumpled = qdict_crumple_for_keyval_qiv(qdict, errp); if (crumpled == NULL) { ret = -EINVAL; goto fail; diff --git a/block/vhdx.c b/block/vhdx.c index XXXXXXX..XXXXXXX 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "vhdx"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/block/vpc.c b/block/vpc.c index XXXXXXX..XXXXXXX 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "vpc"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple(qdict, errp); + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "block/qdict.h" +#include "qapi/qmp/qbool.h" #include "qapi/qmp/qlist.h" +#include "qapi/qmp/qnum.h" +#include "qapi/qmp/qstring.h" #include "qemu/cutils.h" #include "qapi/error.h" @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) } /** + * qdict_crumple_for_keyval_qiv: + * @src: the flat dictionary (only scalar values) to crumple + * @errp: location to store error + * + * Like qdict_crumple(), but additionally transforms scalar values so + * the result can be passed to qobject_input_visitor_new_keyval(). + * + * The block subsystem uses this function to prepare its flat QDict + * with possibly confused scalar types for a visit. It should not be + * used for anything else, and it should go away once the block + * subsystem has been cleaned up. + */ +QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp) +{ + QDict *tmp = NULL; + char *buf; + const char *s; + const QDictEntry *ent; + QObject *dst; + + for (ent = qdict_first(src); ent; ent = qdict_next(src, ent)) { + buf = NULL; + switch (qobject_type(ent->value)) { + case QTYPE_QNULL: + case QTYPE_QSTRING: + continue; + case QTYPE_QNUM: + s = buf = qnum_to_string(qobject_to(QNum, ent->value)); + break; + case QTYPE_QDICT: + case QTYPE_QLIST: + /* @src isn't flat; qdict_crumple() will fail */ + continue; + case QTYPE_QBOOL: + s = qbool_get_bool(qobject_to(QBool, ent->value)) + ? "on" : "off"; + break; + default: + abort(); + } + + if (!tmp) { + tmp = qdict_clone_shallow(src); + } + qdict_put(tmp, ent->key, qstring_from_str(s)); + g_free(buf); + } + + dst = qdict_crumple(tmp ?: src, errp); + qobject_unref(tmp); + return dst; +} + +/** * qdict_array_entries(): Returns the number of direct array entries if the * sub-QDict of src specified by the prefix in subqdict (or src itself for * prefix == "") is valid as an array, i.e. the length of the created list if -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> The previous commit fixed -blockdev breakage due to misuse of the qobject input visitor's keyval flavor in bdrv_file_open(). The commit message explain why using the plain flavor would be just as wrong; it would break -drive. Turns out we break it in three places: nbd_open(), sd_open() and ssh_file_open(). They are even marked FIXME. Example breakage: $ qemu-system-x86 -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off qemu-system-x86: -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off: Invalid parameter type for 'numeric', expected: boolean Fix it the same way: replace qdict_crumple() by qdict_crumple_for_keyval_qiv(), and switch from plain to the keyval flavor. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/nbd.c | 12 ++---------- block/sheepdog.c | 12 ++---------- block/ssh.c | 12 ++---------- 3 files changed, 6 insertions(+), 30 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -XXX,XX +XXX,XX @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, goto done; } - crumpled_addr = qdict_crumple(addr, errp); + crumpled_addr = qdict_crumple_for_keyval_qiv(addr, errp); if (!crumpled_addr) { goto done; } - /* - * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive - * server.type=inet. .to doesn't matter, it's ignored anyway. - * That's because when @options come from -blockdev or - * blockdev_add, members are typed according to the QAPI schema, - * but when they come from -drive, they're all QString. The - * visitor expects the former. - */ - iv = qobject_input_visitor_new(crumpled_addr); + iv = qobject_input_visitor_new_keyval(crumpled_addr); visit_type_SocketAddress(iv, NULL, &saddr, &local_err); if (local_err) { error_propagate(errp, local_err); diff --git a/block/sheepdog.c b/block/sheepdog.c index XXXXXXX..XXXXXXX 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -XXX,XX +XXX,XX @@ static SocketAddress *sd_server_config(QDict *options, Error **errp) qdict_extract_subqdict(options, &server, "server."); - crumpled_server = qdict_crumple(server, errp); + crumpled_server = qdict_crumple_for_keyval_qiv(server, errp); if (!crumpled_server) { goto done; } - /* - * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive - * server.type=inet. .to doesn't matter, it's ignored anyway. - * That's because when @options come from -blockdev or - * blockdev_add, members are typed according to the QAPI schema, - * but when they come from -drive, they're all QString. The - * visitor expects the former. - */ - iv = qobject_input_visitor_new(crumpled_server); + iv = qobject_input_visitor_new_keyval(crumpled_server); visit_type_SocketAddress(iv, NULL, &saddr, &local_err); if (local_err) { error_propagate(errp, local_err); diff --git a/block/ssh.c b/block/ssh.c index XXXXXXX..XXXXXXX 100644 --- a/block/ssh.c +++ b/block/ssh.c @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) } /* Create the QAPI object */ - crumpled = qdict_crumple(options, errp); + crumpled = qdict_crumple_for_keyval_qiv(options, errp); if (crumpled == NULL) { goto fail; } - /* - * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive. - * .to doesn't matter, it's ignored anyway. - * That's because when @options come from -blockdev or - * blockdev_add, members are typed according to the QAPI schema, - * but when they come from -drive, they're all QString. The - * visitor expects the former. - */ - v = qobject_input_visitor_new(crumpled); + v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevOptionsSsh(v, NULL, &result, &local_err); visit_free(v); qobject_unref(crumpled); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> The following pattern occurs in the .bdrv_co_create_opts() methods of parallels, qcow, qcow2, qed, vhdx and vpc: qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { ret = -EINVAL; goto done; } v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); [...] ret = 0; done: qobject_unref(qdict); [...] return ret; If qobject_to() fails, we return failure without setting errp. That's wrong. As far as I can tell, it cannot fail here. Clean it up anyway, by removing the useless conversion. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/parallels.c | 9 ++++----- block/qcow.c | 9 ++++----- block/qcow2.c | 9 ++++----- block/qed.c | 9 ++++----- block/vhdx.c | 9 ++++----- block/vpc.c | 9 ++++----- 6 files changed, 24 insertions(+), 30 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index XXXXXXX..XXXXXXX 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, BlockdevCreateOptions *create_options = NULL; Error *local_err = NULL; BlockDriverState *bs = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; int ret; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto done; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, { BlockdevCreateOptions *create_options = NULL; BlockDriverState *bs = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; const char *val; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt Error **errp) { BlockdevCreateOptions *create_options = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt /* Now get the QAPI type BlockdevCreateOptions */ qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto finish; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qed.c b/block/qed.c index XXXXXXX..XXXXXXX 100644 --- a/block/qed.c +++ b/block/qed.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, Error **errp) { BlockdevCreateOptions *create_options = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/vhdx.c b/block/vhdx.c index XXXXXXX..XXXXXXX 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, Error **errp) { BlockdevCreateOptions *create_options = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/vpc.c b/block/vpc.c index XXXXXXX..XXXXXXX 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, QemuOpts *opts, Error **errp) { BlockdevCreateOptions *create_options = NULL; - QDict *qdict = NULL; + QDict *qdict; QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, qdict_put_str(qdict, "file", bs->node_name); qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - qobject_unref(qdict); - qdict = qobject_to(QDict, qobj); - if (qdict == NULL) { + if (!qobj) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_keyval(qobj); + qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/qdict.h | 3 ++- block/nbd.c | 7 ++----- block/nfs.c | 7 ++----- block/parallels.c | 7 ++----- block/qcow.c | 7 ++----- block/qcow2.c | 7 ++----- block/qed.c | 7 ++----- block/rbd.c | 7 ++----- block/sheepdog.c | 14 ++++---------- block/ssh.c | 7 ++----- block/vhdx.c | 7 ++----- block/vpc.c | 7 ++----- qobject/block-qdict.c | 28 +++++++++++++++++++++++++++- 13 files changed, 53 insertions(+), 62 deletions(-) diff --git a/include/block/qdict.h b/include/block/qdict.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/qdict.h +++ b/include/block/qdict.h @@ -XXX,XX +XXX,XX @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); void qdict_array_split(QDict *src, QList **dst); int qdict_array_entries(QDict *src, const char *subqdict); QObject *qdict_crumple(const QDict *src, Error **errp); -QObject *qdict_crumple_for_keyval_qiv(QDict *qdict, Error **errp); void qdict_flatten(QDict *qdict); typedef struct QDictRenames { @@ -XXX,XX +XXX,XX @@ typedef struct QDictRenames { } QDictRenames; bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp); +Visitor *qobject_input_visitor_new_flat_confused(QDict *qdict, + Error **errp); #endif diff --git a/block/nbd.c b/block/nbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -XXX,XX +XXX,XX @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, { SocketAddress *saddr = NULL; QDict *addr = NULL; - QObject *crumpled_addr = NULL; Visitor *iv = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, goto done; } - crumpled_addr = qdict_crumple_for_keyval_qiv(addr, errp); - if (!crumpled_addr) { + iv = qobject_input_visitor_new_flat_confused(addr, errp); + if (!iv) { goto done; } - iv = qobject_input_visitor_new_keyval(crumpled_addr); visit_type_SocketAddress(iv, NULL, &saddr, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -XXX,XX +XXX,XX @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, done: qobject_unref(addr); - qobject_unref(crumpled_addr); visit_free(iv); return saddr; } diff --git a/block/nfs.c b/block/nfs.c index XXXXXXX..XXXXXXX 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, Error **errp) { BlockdevOptionsNfs *opts = NULL; - QObject *crumpled = NULL; Visitor *v; const QDictEntry *e; Error *local_err = NULL; - crumpled = qdict_crumple_for_keyval_qiv(options, errp); - if (crumpled == NULL) { + v = qobject_input_visitor_new_flat_confused(options, errp); + if (!v) { return NULL; } - v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevOptionsNfs(v, NULL, &opts, &local_err); visit_free(v); - qobject_unref(crumpled); if (local_err) { error_propagate(errp, local_err); diff --git a/block/parallels.c b/block/parallels.c index XXXXXXX..XXXXXXX 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, Error *local_err = NULL; BlockDriverState *bs = NULL; QDict *qdict; - QObject *qobj; Visitor *v; int ret; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn parallels_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "parallels"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto done; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, BlockdevCreateOptions *create_options = NULL; BlockDriverState *bs = NULL; QDict *qdict; - QObject *qobj; Visitor *v; const char *val; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "qcow"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt { BlockdevCreateOptions *create_options = NULL; QDict *qdict; - QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt qdict_put_str(qdict, "file", bs->node_name); /* Now get the QAPI type BlockdevCreateOptions */ - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto finish; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/qed.c b/block/qed.c index XXXXXXX..XXXXXXX 100644 --- a/block/qed.c +++ b/block/qed.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, { BlockdevCreateOptions *create_options = NULL; QDict *qdict; - QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "qed"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, BDRVRBDState *s = bs->opaque; BlockdevOptionsRbd *opts = NULL; Visitor *v; - QObject *crumpled = NULL; const QDictEntry *e; Error *local_err = NULL; char *keypairs, *secretid; @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, } /* Convert the remaining options into a QAPI object */ - crumpled = qdict_crumple_for_keyval_qiv(options, errp); - if (crumpled == NULL) { + v = qobject_input_visitor_new_flat_confused(options, errp); + if (!v) { r = -EINVAL; goto out; } - v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevOptionsRbd(v, NULL, &opts, &local_err); visit_free(v); - qobject_unref(crumpled); if (local_err) { error_propagate(errp, local_err); diff --git a/block/sheepdog.c b/block/sheepdog.c index XXXXXXX..XXXXXXX 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -XXX,XX +XXX,XX @@ static void sd_aio_setup(SheepdogAIOCB *acb, BDRVSheepdogState *s, static SocketAddress *sd_server_config(QDict *options, Error **errp) { QDict *server = NULL; - QObject *crumpled_server = NULL; Visitor *iv = NULL; SocketAddress *saddr = NULL; Error *local_err = NULL; qdict_extract_subqdict(options, &server, "server."); - crumpled_server = qdict_crumple_for_keyval_qiv(server, errp); - if (!crumpled_server) { + iv = qobject_input_visitor_new_flat_confused(server, errp); + if (!iv) { goto done; } - iv = qobject_input_visitor_new_keyval(crumpled_server); visit_type_SocketAddress(iv, NULL, &saddr, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -XXX,XX +XXX,XX @@ static SocketAddress *sd_server_config(QDict *options, Error **errp) done: visit_free(iv); - qobject_unref(crumpled_server); qobject_unref(server); return saddr; } @@ -XXX,XX +XXX,XX @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, { BlockdevCreateOptions *create_options = NULL; QDict *qdict, *location_qdict; - QObject *crumpled; Visitor *v; char *redundancy; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, } /* Get the QAPI object */ - crumpled = qdict_crumple_for_keyval_qiv(qdict, errp); - if (crumpled == NULL) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); - qobject_unref(crumpled); if (local_err) { error_propagate(errp, local_err); diff --git a/block/ssh.c b/block/ssh.c index XXXXXXX..XXXXXXX 100644 --- a/block/ssh.c +++ b/block/ssh.c @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) BlockdevOptionsSsh *result = NULL; QemuOpts *opts = NULL; Error *local_err = NULL; - QObject *crumpled; const QDictEntry *e; Visitor *v; @@ -XXX,XX +XXX,XX @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) } /* Create the QAPI object */ - crumpled = qdict_crumple_for_keyval_qiv(options, errp); - if (crumpled == NULL) { + v = qobject_input_visitor_new_flat_confused(options, errp); + if (!v) { goto fail; } - v = qobject_input_visitor_new_keyval(crumpled); visit_type_BlockdevOptionsSsh(v, NULL, &result, &local_err); visit_free(v); - qobject_unref(crumpled); if (local_err) { error_propagate(errp, local_err); diff --git a/block/vhdx.c b/block/vhdx.c index XXXXXXX..XXXXXXX 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, { BlockdevCreateOptions *create_options = NULL; QDict *qdict; - QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "vhdx"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/block/vpc.c b/block/vpc.c index XXXXXXX..XXXXXXX 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, { BlockdevCreateOptions *create_options = NULL; QDict *qdict; - QObject *qobj; Visitor *v; BlockDriverState *bs = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_create_opts(const char *filename, qdict_put_str(qdict, "driver", "vpc"); qdict_put_str(qdict, "file", bs->node_name); - qobj = qdict_crumple_for_keyval_qiv(qdict, errp); - if (!qobj) { + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { ret = -EINVAL; goto fail; } - v = qobject_input_visitor_new_keyval(qobj); - qobject_unref(qobj); visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ #include "qapi/qmp/qlist.h" #include "qapi/qmp/qnum.h" #include "qapi/qmp/qstring.h" +#include "qapi/qobject-input-visitor.h" #include "qemu/cutils.h" #include "qapi/error.h" @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) * used for anything else, and it should go away once the block * subsystem has been cleaned up. */ -QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp) +static QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp) { QDict *tmp = NULL; char *buf; @@ -XXX,XX +XXX,XX @@ bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) } return true; } + +/* + * Create a QObject input visitor for flat @qdict with possibly + * confused scalar types. + * + * The block subsystem uses this function to visit its flat QDict with + * possibly confused scalar types. It should not be used for anything + * else, and it should go away once the block subsystem has been + * cleaned up. + */ +Visitor *qobject_input_visitor_new_flat_confused(QDict *qdict, + Error **errp) +{ + QObject *crumpled; + Visitor *v; + + crumpled = qdict_crumple_for_keyval_qiv(qdict, errp); + if (!crumpled) { + return NULL; + } + + v = qobject_input_visitor_new_keyval(crumpled); + qobject_unref(crumpled); + return v; +} -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Remaining uses of qobject_input_visitor_new_keyval() in the block subsystem: * block_crypto_open_opts_init() Currently doesn't visit any non-string scalars, thus safe. It's called from - block_crypto_open_luks() Creates the QDict with qemu_opts_to_qdict_filtered(), which creates only string scalars, but has a TODO asking for other types. - qcow_open() - qcow2_open(), qcow2_co_invalidate_cache(), qcow2_reopen_prepare() * block_crypto_create_opts_init(), called from - block_crypto_co_create_opts_luks() Also creates the QDict with qemu_opts_to_qdict_filtered(). * vdi_co_create_opts() Also creates the QDict with qemu_opts_to_qdict_filtered(). Replace these uses by qobject_input_visitor_new_flat_confused() for robustness. This adds crumpling. Right now, that's a no-op, but if we ever extend these things in non-flat ways, crumpling will be needed. Signed-off-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/crypto.c | 12 +++++++++--- block/vdi.c | 8 ++++++-- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "crypto/block.h" #include "qapi/opts-visitor.h" #include "qapi/qapi-visit-crypto.h" -#include "qapi/qmp/qdict.h" #include "qapi/qobject-input-visitor.h" #include "qapi/error.h" #include "qemu/option.h" @@ -XXX,XX +XXX,XX @@ block_crypto_open_opts_init(QCryptoBlockFormat format, ret = g_new0(QCryptoBlockOpenOptions, 1); ret->format = format; - v = qobject_input_visitor_new_keyval(QOBJECT(opts)); + v = qobject_input_visitor_new_flat_confused(opts, &local_err); + if (local_err) { + goto out; + } visit_start_struct(v, NULL, NULL, 0, &local_err); if (local_err) { @@ -XXX,XX +XXX,XX @@ block_crypto_create_opts_init(QCryptoBlockFormat format, ret = g_new0(QCryptoBlockCreateOptions, 1); ret->format = format; - v = qobject_input_visitor_new_keyval(QOBJECT(opts)); + v = qobject_input_visitor_new_flat_confused(opts, &local_err); + if (local_err) { + goto out; + } visit_start_struct(v, NULL, NULL, 0, &local_err); if (local_err) { diff --git a/block/vdi.c b/block/vdi.c index XXXXXXX..XXXXXXX 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qapi/error.h" -#include "qapi/qmp/qdict.h" #include "qapi/qobject-input-visitor.h" #include "qapi/qapi-visit-block-core.h" #include "block/block_int.h" +#include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" #include "qemu/option.h" @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vdi_co_create_opts(const char *filename, QemuOpts *opts, } /* Get the QAPI object */ - v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + v = qobject_input_visitor_new_flat_confused(qdict, errp); + if (!v) { + ret = -EINVAL; + goto done; + } visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); visit_free(v); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> There's no need to restart the loop. We don't elsewhere, e.g. in qdict_extract_subqdict(), qdict_join() and qemu_opts_absorb_qdict(). Simplify accordingly. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) QObject *value; const QDictEntry *entry, *next; char *new_key; - bool delete; entry = qdict_first(qdict); while (entry != NULL) { - next = qdict_next(qdict, entry); value = qdict_entry_value(entry); new_key = NULL; - delete = false; if (prefix) { new_key = g_strdup_printf("%s.%s", prefix, entry->key); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) * itself disappears. */ qdict_flatten_qdict(qobject_to(QDict, value), target, new_key ? new_key : entry->key); - delete = true; + qdict_del(qdict, entry->key); } else if (qobject_type(value) == QTYPE_QLIST) { qdict_flatten_qlist(qobject_to(QList, value), target, new_key ? new_key : entry->key); - delete = true; + qdict_del(qdict, entry->key); } else if (prefix) { /* All other objects are moved to the target unchanged. */ qdict_put_obj(target, new_key, qobject_ref(value)); - delete = true; - } - - g_free(new_key); - - if (delete) { qdict_del(qdict, entry->key); - - /* Restart loop after modifying the iterated QDict */ - entry = qdict_first(qdict); - continue; } + g_free(new_key); entry = next; } } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> qdict_flatten_qdict() skips copying scalars from @qdict to @target when the two are the same. Fair enough, but it uses a non-obvious test for "same". Replace it by the obvious one. While there, improve comments. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) value = qlist_entry_obj(entry); new_key = g_strdup_printf("%s.%i", prefix, i); + /* + * Flatten non-empty QDict and QList recursively into @target, + * copy other objects to @target + */ if (qobject_type(value) == QTYPE_QDICT) { qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); } else if (qobject_type(value) == QTYPE_QLIST) { qdict_flatten_qlist(qobject_to(QList, value), target, new_key); } else { - /* All other types are moved to the target unchanged. */ qdict_put_obj(target, new_key, qobject_ref(value)); } @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) new_key = g_strdup_printf("%s.%s", prefix, entry->key); } + /* + * Flatten non-empty QDict and QList recursively into @target, + * copy other objects to @target + */ if (qobject_type(value) == QTYPE_QDICT) { - /* Entries of QDicts are processed recursively, the QDict object - * itself disappears. */ qdict_flatten_qdict(qobject_to(QDict, value), target, new_key ? new_key : entry->key); qdict_del(qdict, entry->key); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) qdict_flatten_qlist(qobject_to(QList, value), target, new_key ? new_key : entry->key); qdict_del(qdict, entry->key); - } else if (prefix) { - /* All other objects are moved to the target unchanged. */ + } else if (target != qdict) { qdict_put_obj(target, new_key, qobject_ref(value)); qdict_del(qdict, entry->key); } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> When you mix scalar and non-scalar keys, whether you get an "already set as scalar" or an "already set as dict" error depends on qdict iteration order. Neither message makes much sense. Replace by ""Cannot mix scalar and non-scalar keys". This is similar to the message we get for mixing list and non-list keys. I find qdict_crumple()'s first loop hard to understand. Rearrange it and add a comment. Signed-off-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static int qdict_is_list(QDict *maybe_list, Error **errp) QObject *qdict_crumple(const QDict *src, Error **errp) { const QDictEntry *ent; - QDict *two_level, *multi_level = NULL; + QDict *two_level, *multi_level = NULL, *child_dict; QObject *dst = NULL, *child; size_t i; char *prefix = NULL; @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) } qdict_split_flat_key(ent->key, &prefix, &suffix); - child = qdict_get(two_level, prefix); + child_dict = qobject_to(QDict, child); + + if (child) { + /* + * If @child_dict, then all previous keys with this prefix + * had a suffix. If @suffix, this one has one as well, + * and we're good, else there's a clash. + */ + if (!child_dict || !suffix) { + error_setg(errp, "Cannot mix scalar and non-scalar keys"); + goto error; + } + } + if (suffix) { - QDict *child_dict = qobject_to(QDict, child); if (!child_dict) { - if (child) { - error_setg(errp, "Key %s prefix is already set as a scalar", - prefix); - goto error; - } - child_dict = qdict_new(); - qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); + qdict_put(two_level, prefix, child_dict); } - qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); } else { - if (child) { - error_setg(errp, "Key %s prefix is already set as a dict", - prefix); - goto error; - } qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 27 +++++++++++---------------- 1 file changed, 11 insertions(+), 16 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static int qdict_is_list(QDict *maybe_list, Error **errp) for (ent = qdict_first(maybe_list); ent != NULL; ent = qdict_next(maybe_list, ent)) { + int is_index = !qemu_strtoi64(ent->key, NULL, 10, &val); - if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) { - if (is_list == -1) { - is_list = 1; - } else if (!is_list) { - error_setg(errp, - "Cannot mix list and non-list keys"); - return -1; - } + if (is_list == -1) { + is_list = is_index; + } + + if (is_index != is_list) { + error_setg(errp, "Cannot mix list and non-list keys"); + return -1; + } + + if (is_index) { len++; if (val > max) { max = val; } - } else { - if (is_list == -1) { - is_list = 0; - } else if (is_list) { - error_setg(errp, - "Cannot mix list and non-list keys"); - return -1; - } } } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/check-block-qdict.c | 57 ++++++++++++++++++++++++----------------------- 1 file changed, 29 insertions(+), 28 deletions(-) diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-block-qdict.c +++ b/tests/check-block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_defaults_test(void) static void qdict_flatten_test(void) { - QList *list1 = qlist_new(); - QList *list2 = qlist_new(); - QDict *dict1 = qdict_new(); - QDict *dict2 = qdict_new(); - QDict *dict3 = qdict_new(); + QList *e_1 = qlist_new(); + QList *e = qlist_new(); + QDict *e_1_2 = qdict_new(); + QDict *f = qdict_new(); + QDict *root = qdict_new(); /* * Test the flattening of @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) * } */ - qdict_put_int(dict1, "a", 0); - qdict_put_int(dict1, "b", 1); + qdict_put_int(e_1_2, "a", 0); + qdict_put_int(e_1_2, "b", 1); - qlist_append_int(list1, 23); - qlist_append_int(list1, 66); - qlist_append(list1, dict1); - qlist_append_int(list2, 42); - qlist_append(list2, list1); + qlist_append_int(e_1, 23); + qlist_append_int(e_1, 66); + qlist_append(e_1, e_1_2); + qlist_append_int(e, 42); + qlist_append(e, e_1); - qdict_put_int(dict2, "c", 2); - qdict_put_int(dict2, "d", 3); - qdict_put(dict3, "e", list2); - qdict_put(dict3, "f", dict2); - qdict_put_int(dict3, "g", 4); + qdict_put_int(f, "c", 2); + qdict_put_int(f, "d", 3); - qdict_flatten(dict3); + qdict_put(root, "e", e); + qdict_put(root, "f", f); + qdict_put_int(root, "g", 4); - g_assert(qdict_get_int(dict3, "e.0") == 42); - g_assert(qdict_get_int(dict3, "e.1.0") == 23); - g_assert(qdict_get_int(dict3, "e.1.1") == 66); - g_assert(qdict_get_int(dict3, "e.1.2.a") == 0); - g_assert(qdict_get_int(dict3, "e.1.2.b") == 1); - g_assert(qdict_get_int(dict3, "f.c") == 2); - g_assert(qdict_get_int(dict3, "f.d") == 3); - g_assert(qdict_get_int(dict3, "g") == 4); + qdict_flatten(root); - g_assert(qdict_size(dict3) == 8); + g_assert(qdict_get_int(root, "e.0") == 42); + g_assert(qdict_get_int(root, "e.1.0") == 23); + g_assert(qdict_get_int(root, "e.1.1") == 66); + g_assert(qdict_get_int(root, "e.1.2.a") == 0); + g_assert(qdict_get_int(root, "e.1.2.b") == 1); + g_assert(qdict_get_int(root, "f.c") == 2); + g_assert(qdict_get_int(root, "f.d") == 3); + g_assert(qdict_get_int(root, "g") == 4); - qobject_unref(dict3); + g_assert(qdict_size(root) == 8); + + qobject_unref(root); } static void qdict_array_split_test(void) -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/check-block-qdict.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-block-qdict.c +++ b/tests/check-block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) QList *e = qlist_new(); QDict *e_1_2 = qdict_new(); QDict *f = qdict_new(); + QList *y = qlist_new(); + QDict *z = qdict_new(); QDict *root = qdict_new(); /* @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) * "c": 2, * "d": 3, * }, - * "g": 4 + * "g": 4, + * "y": [{}], + * "z": {"a": []} * } * * to @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) * "f.d": 3, * "g": 4 * } + * + * Note that "y" and "z" get eaten. */ qdict_put_int(e_1_2, "a", 0); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) qdict_put_int(f, "c", 2); qdict_put_int(f, "d", 3); + qlist_append(y, qdict_new()); + + qdict_put(z, "a", qlist_new()); + qdict_put(root, "e", e); qdict_put(root, "f", f); qdict_put_int(root, "g", 4); + qdict_put(root, "y", y); + qdict_put(root, "z", z); qdict_flatten(root); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> -blockdev and blockdev-add silently ignore empty objects and arrays in their argument. That's because qmp_blockdev_add() converts the argument to a flat QDict, and qdict_flatten() eats empty QDict and QList members. For instance, we ignore an empty BlockdevOptions member @cache. No real harm, as absent means the same as empty there. Thus, the flaw puts an artificial restriction on the QAPI schema: we can't have potentially empty objects and arrays within BlockdevOptions, except when they're optional and "empty" has the same meaning as "absent". Our QAPI schema satisfies this restriction (I checked), but it's a trap for the unwary, and a temptation to employ awkward workarounds for the wary. Let's get rid of it. Change qdict_flatten() and qdict_crumple() to treat empty dictionaries and lists exactly like scalars. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qobject/block-qdict.c | 54 +++++++++++++++++++++++++++++------------------ tests/check-block-qdict.c | 38 ++++++++++++++++++++++++++------- 2 files changed, 63 insertions(+), 29 deletions(-) diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/qobject/block-qdict.c +++ b/qobject/block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) { QObject *value; const QListEntry *entry; + QDict *dict_val; + QList *list_val; char *new_key; int i; @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) for (i = 0; entry; entry = qlist_next(entry), i++) { value = qlist_entry_obj(entry); + dict_val = qobject_to(QDict, value); + list_val = qobject_to(QList, value); new_key = g_strdup_printf("%s.%i", prefix, i); /* * Flatten non-empty QDict and QList recursively into @target, * copy other objects to @target */ - if (qobject_type(value) == QTYPE_QDICT) { - qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); - } else if (qobject_type(value) == QTYPE_QLIST) { - qdict_flatten_qlist(qobject_to(QList, value), target, new_key); + if (dict_val && qdict_size(dict_val)) { + qdict_flatten_qdict(dict_val, target, new_key); + } else if (list_val && !qlist_empty(list_val)) { + qdict_flatten_qlist(list_val, target, new_key); } else { qdict_put_obj(target, new_key, qobject_ref(value)); } @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) { QObject *value; const QDictEntry *entry, *next; + QDict *dict_val; + QList *list_val; char *new_key; entry = qdict_first(qdict); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) while (entry != NULL) { next = qdict_next(qdict, entry); value = qdict_entry_value(entry); + dict_val = qobject_to(QDict, value); + list_val = qobject_to(QList, value); new_key = NULL; if (prefix) { @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) * Flatten non-empty QDict and QList recursively into @target, * copy other objects to @target */ - if (qobject_type(value) == QTYPE_QDICT) { - qdict_flatten_qdict(qobject_to(QDict, value), target, + if (dict_val && qdict_size(dict_val)) { + qdict_flatten_qdict(dict_val, target, new_key ? new_key : entry->key); qdict_del(qdict, entry->key); - } else if (qobject_type(value) == QTYPE_QLIST) { - qdict_flatten_qlist(qobject_to(QList, value), target, + } else if (list_val && !qlist_empty(list_val)) { + qdict_flatten_qlist(list_val, target, new_key ? new_key : entry->key); qdict_del(qdict, entry->key); } else if (target != qdict) { @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) } /** - * qdict_flatten(): For each nested QDict with key x, all fields with key y - * are moved to this QDict and their key is renamed to "x.y". For each nested - * QList with key x, the field at index y is moved to this QDict with the key - * "x.y" (i.e., the reverse of what qdict_array_split() does). + * qdict_flatten(): For each nested non-empty QDict with key x, all + * fields with key y are moved to this QDict and their key is renamed + * to "x.y". For each nested non-empty QList with key x, the field at + * index y is moved to this QDict with the key "x.y" (i.e., the + * reverse of what qdict_array_split() does). * This operation is applied recursively for nested QDicts and QLists. */ void qdict_flatten(QDict *qdict) @@ -XXX,XX +XXX,XX @@ static int qdict_is_list(QDict *maybe_list, Error **errp) * @src: the original flat dictionary (only scalar values) to crumple * * Takes a flat dictionary whose keys use '.' separator to indicate - * nesting, and values are scalars, and crumples it into a nested - * structure. + * nesting, and values are scalars, empty dictionaries or empty lists, + * and crumples it into a nested structure. * * To include a literal '.' in a key name, it must be escaped as '..' * @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) { const QDictEntry *ent; QDict *two_level, *multi_level = NULL, *child_dict; + QDict *dict_val; + QList *list_val; QObject *dst = NULL, *child; size_t i; char *prefix = NULL; @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) /* Step 1: split our totally flat dict into a two level dict */ for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) { - if (qobject_type(ent->value) == QTYPE_QDICT || - qobject_type(ent->value) == QTYPE_QLIST) { - error_setg(errp, "Value %s is not a scalar", - ent->key); + dict_val = qobject_to(QDict, ent->value); + list_val = qobject_to(QList, ent->value); + if ((dict_val && qdict_size(dict_val)) + || (list_val && !qlist_empty(list_val))) { + error_setg(errp, "Value %s is not flat", ent->key); goto error; } @@ -XXX,XX +XXX,XX @@ QObject *qdict_crumple(const QDict *src, Error **errp) multi_level = qdict_new(); for (ent = qdict_first(two_level); ent != NULL; ent = qdict_next(two_level, ent)) { - QDict *dict = qobject_to(QDict, ent->value); - if (dict) { - child = qdict_crumple(dict, errp); + dict_val = qobject_to(QDict, ent->value); + if (dict_val && qdict_size(dict_val)) { + child = qdict_crumple(dict_val, errp); if (!child) { goto error; } diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c index XXXXXXX..XXXXXXX 100644 --- a/tests/check-block-qdict.c +++ b/tests/check-block-qdict.c @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) * "e.1.2.b": 1, * "f.c": 2, * "f.d": 3, - * "g": 4 + * "g": 4, + * "y.0": {}, + * "z.a": [] * } - * - * Note that "y" and "z" get eaten. */ qdict_put_int(e_1_2, "a", 0); @@ -XXX,XX +XXX,XX @@ static void qdict_flatten_test(void) g_assert(qdict_get_int(root, "f.c") == 2); g_assert(qdict_get_int(root, "f.d") == 3); g_assert(qdict_get_int(root, "g") == 4); + g_assert(!qdict_size(qdict_get_qdict(root, "y.0"))); + g_assert(qlist_empty(qdict_get_qlist(root, "z.a"))); - g_assert(qdict_size(root) == 8); + g_assert(qdict_size(root) == 10); qobject_unref(root); } @@ -XXX,XX +XXX,XX @@ static void qdict_join_test(void) static void qdict_crumple_test_recursive(void) { QDict *src, *dst, *rule, *vnc, *acl, *listen; - QList *rules; + QDict *empty, *empty_dict, *empty_list_0; + QList *rules, *empty_list, *empty_dict_a; src = qdict_new(); qdict_put_str(src, "vnc.listen.addr", "127.0.0.1"); @@ -XXX,XX +XXX,XX @@ static void qdict_crumple_test_recursive(void) qdict_put_str(src, "vnc.acl.default", "deny"); qdict_put_str(src, "vnc.acl..name", "acl0"); qdict_put_str(src, "vnc.acl.rule..name", "acl0"); + qdict_put(src, "empty.dict.a", qlist_new()); + qdict_put(src, "empty.list.0", qdict_new()); dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); g_assert(dst); - g_assert_cmpint(qdict_size(dst), ==, 1); + g_assert_cmpint(qdict_size(dst), ==, 2); vnc = qdict_get_qdict(dst, "vnc"); g_assert(vnc); @@ -XXX,XX +XXX,XX @@ static void qdict_crumple_test_recursive(void) g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); + empty = qdict_get_qdict(dst, "empty"); + g_assert(empty); + g_assert_cmpint(qdict_size(empty), ==, 2); + empty_dict = qdict_get_qdict(empty, "dict"); + g_assert(empty_dict); + g_assert_cmpint(qdict_size(empty_dict), ==, 1); + empty_dict_a = qdict_get_qlist(empty_dict, "a"); + g_assert(empty_dict_a && qlist_empty(empty_dict_a)); + empty_list = qdict_get_qlist(empty, "list"); + g_assert(empty_list); + g_assert_cmpint(qlist_size(empty_list), ==, 1); + empty_list_0 = qobject_to(QDict, qlist_pop(empty_list)); + g_assert(empty_list_0); + g_assert_cmpint(qdict_size(empty_list_0), ==, 0); + qobject_unref(src); qobject_unref(dst); } @@ -XXX,XX +XXX,XX @@ static void qdict_rename_keys_test(void) static void qdict_crumple_test_bad_inputs(void) { - QDict *src; + QDict *src, *nested; Error *error = NULL; src = qdict_new(); @@ -XXX,XX +XXX,XX @@ static void qdict_crumple_test_bad_inputs(void) src = qdict_new(); /* The input should be flat, ie no dicts or lists */ - qdict_put(src, "rule.a", qdict_new()); + nested = qdict_new(); + qdict_put(nested, "x", qdict_new()); + qdict_put(src, "rule.a", nested); qdict_put_str(src, "rule.b", "allow"); g_assert(qdict_crumple(src, &error) == NULL); -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Parameter auth-client-required lets you configure authentication methods. We tried to provide that in v2.9.0, but backed out due to interface design doubts (commit 464444fcc16). This commit is similar to what we backed out, but simpler: we use a list of enumeration values instead of a list of objects with a member of enumeration type. Let's review our reasons for backing out the first try, as stated in the commit message: * The implementation uses deprecated rados_conf_set() key "auth_supported". No biggie. Fixed: we use "auth-client-required". * The implementation makes -drive silently ignore invalid parameters "auth" and "auth-supported.*.X" where X isn't "auth". Fixable (in fact I'm going to fix similar bugs around parameter server), so again no biggie. That fix is commit 2836284db60. This commit doesn't bring the bugs back. * BlockdevOptionsRbd member @password-secret applies only to authentication method cephx. Should it be a variant member of RbdAuthMethod? We've had time to ponder, and we decided to stick to the way Ceph configuration works: the key configured separately, and silently ignored if the authentication method doesn't use it. * BlockdevOptionsRbd member @user could apply to both methods cephx and none, but I'm not sure it's actually used with none. If it isn't, should it be a variant member of RbdAuthMethod? Likewise. * The client offers a *set* of authentication methods, not a list. Should the methods be optional members of BlockdevOptionsRbd instead of members of list @auth-supported? The latter begs the question what multiple entries for the same method mean. Trivial question now that RbdAuthMethod contains nothing but @type, but less so when RbdAuthMethod acquires other members, such the ones discussed above. Again, we decided to stick to the way Ceph configuration works, except we make auth-client-required a list of enumeration values instead of a string containing keywords separated by delimiters. * How BlockdevOptionsRbd member @auth-supported interacts with settings from a configuration file specified with @conf is undocumented. I suspect it's untested, too. Not actually true, the documentation for @conf says "Values in the configuration file will be overridden by options specified via QAPI", and we've tested this. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 13 +++++++++++++ block/rbd.c | 42 ++++++++++++++++++++++++++++++++---------- 2 files changed, 45 insertions(+), 10 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ ## +# @RbdAuthMode: +# +# Since: 3.0 +## +{ 'enum': 'RbdAuthMode', + 'data': [ 'cephx', 'none' ] } + +## # @BlockdevOptionsRbd: # # @pool: Ceph pool name. @@ -XXX,XX +XXX,XX @@ # # @user: Ceph id name. # +# @auth-client-required: Acceptable authentication modes. +# This maps to Ceph configuration option +# "auth_client_required". (Since 3.0) +# # @server: Monitor host address and port. This maps # to the "mon_host" Ceph option. # @@ -XXX,XX +XXX,XX @@ '*conf': 'str', '*snapshot': 'str', '*user': 'str', + '*auth-client-required': ['RbdAuthMode'], '*server': ['InetSocketAddressBase'] } } ## diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp) static int qemu_rbd_set_auth(rados_t cluster, const char *secretid, + BlockdevOptionsRbd *opts, Error **errp) { - if (secretid == 0) { - return 0; - } + char *acr; + int r; + GString *accu; + RbdAuthModeList *auth; + + if (secretid) { + gchar *secret = qcrypto_secret_lookup_as_base64(secretid, + errp); + if (!secret) { + return -1; + } - gchar *secret = qcrypto_secret_lookup_as_base64(secretid, - errp); - if (!secret) { - return -1; + rados_conf_set(cluster, "key", secret); + g_free(secret); } - rados_conf_set(cluster, "key", secret); - g_free(secret); + if (opts->has_auth_client_required) { + accu = g_string_new(""); + for (auth = opts->auth_client_required; auth; auth = auth->next) { + if (accu->str[0]) { + g_string_append_c(accu, ';'); + } + g_string_append(accu, RbdAuthMode_str(auth->value)); + } + acr = g_string_free(accu, FALSE); + r = rados_conf_set(cluster, "auth_client_required", acr); + g_free(acr); + if (r < 0) { + error_setg_errno(errp, -r, + "Could not set 'auth_client_required'"); + return r; + } + } return 0; } @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, } } - if (qemu_rbd_set_auth(*cluster, secretid, errp) < 0) { + if (qemu_rbd_set_auth(*cluster, secretid, opts, errp) < 0) { r = -EIO; goto failed_shutdown; } -- 2.13.6 From: Markus Armbruster <armbru@redhat.com> Legacy -drive supports "password-secret" parameter that isn't available with -blockdev / blockdev-add. That's because we backed out our first try to provide it there due to interface design doubts, in commit 577d8c9a811, v2.9.0. This is the second try. It brings back the parameter, except it's named "key-secret" now. Let's review our reasons for backing out the first try, as stated in the commit message: * BlockdevOptionsRbd member @password-secret isn't actually a password, it's a key generated by Ceph. Addressed by the rename. * We're not sure where member @password-secret belongs (see the previous commit). See previous commit. * How @password-secret interacts with settings from a configuration file specified with @conf is undocumented. Not actually true, the documentation for @conf says "Values in the configuration file will be overridden by options specified via QAPI", and we've tested this. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 6 ++++++ block/rbd.c | 41 +++++++++++++++++++++++++---------------- 2 files changed, 31 insertions(+), 16 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # This maps to Ceph configuration option # "auth_client_required". (Since 3.0) # +# @key-secret: ID of a QCryptoSecret object providing a key +# for cephx authentication. +# This maps to Ceph configuration option +# "key". (Since 3.0) +# # @server: Monitor host address and port. This maps # to the "mon_host" Ceph option. # @@ -XXX,XX +XXX,XX @@ '*snapshot': 'str', '*user': 'str', '*auth-client-required': ['RbdAuthMode'], + '*key-secret': 'str', '*server': ['InetSocketAddressBase'] } } ## diff --git a/block/rbd.c b/block/rbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -XXX,XX +XXX,XX @@ static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp) } -static int qemu_rbd_set_auth(rados_t cluster, const char *secretid, - BlockdevOptionsRbd *opts, +static int qemu_rbd_set_auth(rados_t cluster, BlockdevOptionsRbd *opts, Error **errp) { - char *acr; + char *key, *acr; int r; GString *accu; RbdAuthModeList *auth; - if (secretid) { - gchar *secret = qcrypto_secret_lookup_as_base64(secretid, - errp); - if (!secret) { - return -1; + if (opts->key_secret) { + key = qcrypto_secret_lookup_as_base64(opts->key_secret, errp); + if (!key) { + return -EIO; + } + r = rados_conf_set(cluster, "key", key); + g_free(key); + if (r < 0) { + error_setg_errno(errp, -r, "Could not set 'key'"); + return r; } - - rados_conf_set(cluster, "key", secret); - g_free(secret); } if (opts->has_auth_client_required) { @@ -XXX,XX +XXX,XX @@ static QemuOptsList runtime_opts = { }, }; -/* FIXME Deprecate and remove keypairs or make it available in QMP. - * password_secret should eventually be configurable in opts->location. Support - * for it in .bdrv_open will make it work here as well. */ +/* FIXME Deprecate and remove keypairs or make it available in QMP. */ static int qemu_rbd_do_create(BlockdevCreateOptions *options, const char *keypairs, const char *password_secret, Error **errp) @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, Error *local_err = NULL; int r; + if (secretid) { + if (opts->key_secret) { + error_setg(errp, + "Legacy 'password-secret' clashes with 'key-secret'"); + return -EINVAL; + } + opts->key_secret = g_strdup(secretid); + opts->has_key_secret = true; + } + mon_host = qemu_rbd_mon_host(opts, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -XXX,XX +XXX,XX @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, } } - if (qemu_rbd_set_auth(*cluster, secretid, opts, errp) < 0) { - r = -EIO; + r = qemu_rbd_set_auth(*cluster, opts, errp); + if (r < 0) { goto failed_shutdown; } -- 2.13.6 The -drive options cyls, heads, secs and trans were deprecated in QEMU 2.10. It's time to remove them. hd-geo-test tested both the old version with geometry options in -drive and the new one with -device. Therefore the code using -drive doesn't have to be replaced there, we just need to remove the -drive test cases. This in turn allows some simplification of the code. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> --- include/sysemu/blockdev.h | 1 - blockdev.c | 75 +---------------------------------------------- hw/block/block.c | 14 --------- tests/hd-geo-test.c | 37 +++++------------------ hmp-commands.hx | 1 - qemu-doc.texi | 5 ---- qemu-options.hx | 7 +---- 7 files changed, 9 insertions(+), 131 deletions(-) diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/blockdev.h +++ b/include/sysemu/blockdev.h @@ -XXX,XX +XXX,XX @@ struct DriveInfo { int auto_del; /* see blockdev_mark_auto_del() */ bool is_default; /* Added by default_drive() ? */ int media_cd; - int cyls, heads, secs, trans; QemuOpts *opts; char *serial; QTAILQ_ENTRY(DriveInfo) next; diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ QemuOptsList qemu_legacy_drive_opts = { .type = QEMU_OPT_STRING, .help = "interface (ide, scsi, sd, mtd, floppy, pflash, virtio)", },{ - .name = "cyls", - .type = QEMU_OPT_NUMBER, - .help = "number of cylinders (ide disk geometry)", - },{ - .name = "heads", - .type = QEMU_OPT_NUMBER, - .help = "number of heads (ide disk geometry)", - },{ - .name = "secs", - .type = QEMU_OPT_NUMBER, - .help = "number of sectors (ide disk geometry)", - },{ - .name = "trans", - .type = QEMU_OPT_STRING, - .help = "chs translation (auto, lba, none)", - },{ .name = "addr", .type = QEMU_OPT_STRING, .help = "pci address (virtio only)", @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) QemuOpts *legacy_opts; DriveMediaType media = MEDIA_DISK; BlockInterfaceType type; - int cyls, heads, secs, translation; int max_devs, bus_id, unit_id, index; const char *devaddr; const char *werror, *rerror; @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) Error *local_err = NULL; int i; const char *deprecated[] = { - "serial", "trans", "secs", "heads", "cyls", "addr" + "serial", "addr" }; /* Change legacy command line options into QMP ones */ @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) type = block_default_type; } - /* Geometry */ - cyls = qemu_opt_get_number(legacy_opts, "cyls", 0); - heads = qemu_opt_get_number(legacy_opts, "heads", 0); - secs = qemu_opt_get_number(legacy_opts, "secs", 0); - - if (cyls || heads || secs) { - if (cyls < 1) { - error_report("invalid physical cyls number"); - goto fail; - } - if (heads < 1) { - error_report("invalid physical heads number"); - goto fail; - } - if (secs < 1) { - error_report("invalid physical secs number"); - goto fail; - } - } - - translation = BIOS_ATA_TRANSLATION_AUTO; - value = qemu_opt_get(legacy_opts, "trans"); - if (value != NULL) { - if (!cyls) { - error_report("'%s' trans must be used with cyls, heads and secs", - value); - goto fail; - } - if (!strcmp(value, "none")) { - translation = BIOS_ATA_TRANSLATION_NONE; - } else if (!strcmp(value, "lba")) { - translation = BIOS_ATA_TRANSLATION_LBA; - } else if (!strcmp(value, "large")) { - translation = BIOS_ATA_TRANSLATION_LARGE; - } else if (!strcmp(value, "rechs")) { - translation = BIOS_ATA_TRANSLATION_RECHS; - } else if (!strcmp(value, "auto")) { - translation = BIOS_ATA_TRANSLATION_AUTO; - } else { - error_report("'%s' invalid translation type", value); - goto fail; - } - } - - if (media == MEDIA_CDROM) { - if (cyls || secs || heads) { - error_report("CHS can't be set with media=cdrom"); - goto fail; - } - } - /* Device address specified by bus/unit or index. * If none was specified, try to find the first free one. */ bus_id = qemu_opt_get_number(legacy_opts, "bus", 0); @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) dinfo = g_malloc0(sizeof(*dinfo)); dinfo->opts = all_opts; - dinfo->cyls = cyls; - dinfo->heads = heads; - dinfo->secs = secs; - dinfo->trans = translation; - dinfo->type = type; dinfo->bus = bus_id; dinfo->unit = unit_id; diff --git a/hw/block/block.c b/hw/block/block.c index XXXXXXX..XXXXXXX 100644 --- a/hw/block/block.c +++ b/hw/block/block.c @@ -XXX,XX +XXX,XX @@ bool blkconf_geometry(BlockConf *conf, int *ptrans, unsigned cyls_max, unsigned heads_max, unsigned secs_max, Error **errp) { - DriveInfo *dinfo; - - if (!conf->cyls && !conf->heads && !conf->secs) { - /* try to fall back to value set with legacy -drive cyls=... */ - dinfo = blk_legacy_dinfo(conf->blk); - if (dinfo) { - conf->cyls = dinfo->cyls; - conf->heads = dinfo->heads; - conf->secs = dinfo->secs; - if (ptrans) { - *ptrans = dinfo->trans; - } - } - } if (!conf->cyls && !conf->heads && !conf->secs) { hd_geometry_guess(conf->blk, &conf->cyls, &conf->heads, &conf->secs, diff --git a/tests/hd-geo-test.c b/tests/hd-geo-test.c index XXXXXXX..XXXXXXX 100644 --- a/tests/hd-geo-test.c +++ b/tests/hd-geo-test.c @@ -XXX,XX +XXX,XX @@ static void setup_mbr(int img_idx, MBRcontents mbr) static int setup_ide(int argc, char *argv[], int argv_sz, int ide_idx, const char *dev, int img_idx, - MBRcontents mbr, const char *opts) + MBRcontents mbr) { char *s1, *s2, *s3; @@ -XXX,XX +XXX,XX @@ static int setup_ide(int argc, char *argv[], int argv_sz, s3 = g_strdup(",media=cdrom"); } argc = append_arg(argc, argv, argv_sz, - g_strdup_printf("%s%s%s%s", s1, s2, s3, opts)); + g_strdup_printf("%s%s%s", s1, s2, s3)); g_free(s1); g_free(s2); g_free(s3); @@ -XXX,XX +XXX,XX @@ static void test_ide_mbr(bool use_device, MBRcontents mbr) for (i = 0; i < backend_last; i++) { cur_ide[i] = &hd_chst[i][mbr]; dev = use_device ? (is_hd(cur_ide[i]) ? "ide-hd" : "ide-cd") : NULL; - argc = setup_ide(argc, argv, ARGV_SIZE, i, dev, i, mbr, ""); + argc = setup_ide(argc, argv, ARGV_SIZE, i, dev, i, mbr); } args = g_strjoinv(" ", argv); qtest_start(args); @@ -XXX,XX +XXX,XX @@ static void test_ide_drive_user(const char *dev, bool trans) const CHST expected_chst = { secs / (4 * 32) , 4, 32, trans }; argc = setup_common(argv, ARGV_SIZE); - opts = g_strdup_printf("%s,%s%scyls=%d,heads=%d,secs=%d", - dev ?: "", - trans && dev ? "bios-chs-" : "", - trans ? "trans=lba," : "", + opts = g_strdup_printf("%s,%scyls=%d,heads=%d,secs=%d", + dev, trans ? "bios-chs-trans=lba," : "", expected_chst.cyls, expected_chst.heads, expected_chst.secs); cur_ide[0] = &expected_chst; - argc = setup_ide(argc, argv, ARGV_SIZE, - 0, dev ? opts : NULL, backend_small, mbr_chs, - dev ? "" : opts); + argc = setup_ide(argc, argv, ARGV_SIZE, 0, opts, backend_small, mbr_chs); g_free(opts); args = g_strjoinv(" ", argv); qtest_start(args); @@ -XXX,XX +XXX,XX @@ static void test_ide_drive_user(const char *dev, bool trans) } /* - * Test case: IDE device (if=ide) with explicit CHS - */ -static void test_ide_drive_user_chs(void) -{ - test_ide_drive_user(NULL, false); -} - -/* - * Test case: IDE device (if=ide) with explicit CHS and translation - */ -static void test_ide_drive_user_chst(void) -{ - test_ide_drive_user(NULL, true); -} - -/* * Test case: IDE device (if=none) with explicit CHS */ static void test_ide_device_user_chs(void) @@ -XXX,XX +XXX,XX @@ static void test_ide_drive_cd_0(void) for (i = 0; i <= backend_empty; i++) { ide_idx = backend_empty - i; cur_ide[ide_idx] = &hd_chst[i][mbr_blank]; - argc = setup_ide(argc, argv, ARGV_SIZE, - ide_idx, NULL, i, mbr_blank, ""); + argc = setup_ide(argc, argv, ARGV_SIZE, ide_idx, NULL, i, mbr_blank); } args = g_strjoinv(" ", argv); qtest_start(args); @@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv) qtest_add_func("hd-geo/ide/drive/mbr/blank", test_ide_drive_mbr_blank); qtest_add_func("hd-geo/ide/drive/mbr/lba", test_ide_drive_mbr_lba); qtest_add_func("hd-geo/ide/drive/mbr/chs", test_ide_drive_mbr_chs); - qtest_add_func("hd-geo/ide/drive/user/chs", test_ide_drive_user_chs); - qtest_add_func("hd-geo/ide/drive/user/chst", test_ide_drive_user_chst); qtest_add_func("hd-geo/ide/drive/cd_0", test_ide_drive_cd_0); qtest_add_func("hd-geo/ide/device/mbr/blank", test_ide_device_mbr_blank); qtest_add_func("hd-geo/ide/device/mbr/lba", test_ide_device_mbr_lba); diff --git a/hmp-commands.hx b/hmp-commands.hx index XXXXXXX..XXXXXXX 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -XXX,XX +XXX,XX @@ ETEXI .params = "[-n] [[<domain>:]<bus>:]<slot>\n" "[file=file][,if=type][,bus=n]\n" "[,unit=m][,media=d][,index=i]\n" - "[,cyls=c,heads=h,secs=s[,trans=t]]\n" "[,snapshot=on|off][,cache=on|off]\n" "[,readonly=on|off][,copy-on-read=on|off]", .help = "add drive to PCI storage controller", diff --git a/qemu-doc.texi b/qemu-doc.texi index XXXXXXX..XXXXXXX 100644 --- a/qemu-doc.texi +++ b/qemu-doc.texi @@ -XXX,XX +XXX,XX @@ with ``-device ...,netdev=x''), or ``-nic user,smb=/some/dir'' (for embedded NICs). The new syntax allows different settings to be provided per NIC. -@subsection -drive cyls=...,heads=...,secs=...,trans=... (since 2.10.0) - -The drive geometry arguments are replaced by the the geometry arguments -that can be specified with the ``-device'' parameter. - @subsection -drive serial=... (since 2.10.0) The drive serial argument is replaced by the the serial argument diff --git a/qemu-options.hx b/qemu-options.hx index XXXXXXX..XXXXXXX 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -XXX,XX +XXX,XX @@ ETEXI DEF("drive", HAS_ARG, QEMU_OPTION_drive, "-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n" - " [,cyls=c,heads=h,secs=s[,trans=t]][,snapshot=on|off]\n" " [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n" - " [,serial=s][,addr=A][,rerror=ignore|stop|report]\n" + " [,snapshot=on|off][,serial=s][,addr=A][,rerror=ignore|stop|report]\n" " [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n" " [,readonly=on|off][,copy-on-read=on|off]\n" " [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n" @@ -XXX,XX +XXX,XX @@ This option defines where is connected the drive by using an index in the list of available connectors of a given interface type. @item media=@var{media} This option defines the type of the media: disk or cdrom. -@item cyls=@var{c},heads=@var{h},secs=@var{s}[,trans=@var{t}] -Force disk physical geometry and the optional BIOS translation (trans=none or -lba). These parameters are deprecated, use the corresponding parameters -of @code{-device} instead. @item snapshot=@var{snapshot} @var{snapshot} is "on" or "off" and controls snapshot mode for the given drive (see @option{-snapshot}). -- 2.13.6 The -drive option addr was deprecated in QEMU 2.10. It's time to remove it. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> --- include/sysemu/blockdev.h | 1 - blockdev.c | 17 +---------------- device-hotplug.c | 4 ---- qemu-doc.texi | 5 ----- qemu-options.hx | 5 +---- 5 files changed, 2 insertions(+), 30 deletions(-) diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/blockdev.h +++ b/include/sysemu/blockdev.h @@ -XXX,XX +XXX,XX @@ typedef enum { } BlockInterfaceType; struct DriveInfo { - const char *devaddr; BlockInterfaceType type; int bus; int unit; diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ QemuOptsList qemu_legacy_drive_opts = { .type = QEMU_OPT_STRING, .help = "interface (ide, scsi, sd, mtd, floppy, pflash, virtio)", },{ - .name = "addr", - .type = QEMU_OPT_STRING, - .help = "pci address (virtio only)", - },{ .name = "serial", .type = QEMU_OPT_STRING, .help = "disk serial number", @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) DriveMediaType media = MEDIA_DISK; BlockInterfaceType type; int max_devs, bus_id, unit_id, index; - const char *devaddr; const char *werror, *rerror; bool read_only = false; bool copy_on_read; @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) Error *local_err = NULL; int i; const char *deprecated[] = { - "serial", "addr" + "serial" }; /* Change legacy command line options into QMP ones */ @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) } /* Add virtio block device */ - devaddr = qemu_opt_get(legacy_opts, "addr"); - if (devaddr && type != IF_VIRTIO) { - error_report("addr is not supported by this bus type"); - goto fail; - } - if (type == IF_VIRTIO) { QemuOpts *devopts; devopts = qemu_opts_create(qemu_find_opts("device"), NULL, 0, @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) } qemu_opt_set(devopts, "drive", qdict_get_str(bs_opts, "id"), &error_abort); - if (devaddr) { - qemu_opt_set(devopts, "addr", devaddr, &error_abort); - } } filename = qemu_opt_get(legacy_opts, "file"); @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) dinfo->type = type; dinfo->bus = bus_id; dinfo->unit = unit_id; - dinfo->devaddr = devaddr; dinfo->serial = g_strdup(serial); blk_set_legacy_dinfo(blk, dinfo); diff --git a/device-hotplug.c b/device-hotplug.c index XXXXXXX..XXXXXXX 100644 --- a/device-hotplug.c +++ b/device-hotplug.c @@ -XXX,XX +XXX,XX @@ void hmp_drive_add(Monitor *mon, const QDict *qdict) if (!dinfo) { goto err; } - if (dinfo->devaddr) { - monitor_printf(mon, "Parameter addr not supported\n"); - goto err; - } switch (dinfo->type) { case IF_NONE: diff --git a/qemu-doc.texi b/qemu-doc.texi index XXXXXXX..XXXXXXX 100644 --- a/qemu-doc.texi +++ b/qemu-doc.texi @@ -XXX,XX +XXX,XX @@ provided per NIC. The drive serial argument is replaced by the the serial argument that can be specified with the ``-device'' parameter. -@subsection -drive addr=... (since 2.10.0) - -The drive addr argument is replaced by the the addr argument -that can be specified with the ``-device'' parameter. - @subsection -usbdevice (since 2.10.0) The ``-usbdevice DEV'' argument is now a synonym for setting diff --git a/qemu-options.hx b/qemu-options.hx index XXXXXXX..XXXXXXX 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -XXX,XX +XXX,XX @@ ETEXI DEF("drive", HAS_ARG, QEMU_OPTION_drive, "-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n" " [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n" - " [,snapshot=on|off][,serial=s][,addr=A][,rerror=ignore|stop|report]\n" + " [,snapshot=on|off][,serial=s][,rerror=ignore|stop|report]\n" " [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n" " [,readonly=on|off][,copy-on-read=on|off]\n" " [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n" @@ -XXX,XX +XXX,XX @@ an untrusted format header. This option specifies the serial number to assign to the device. This parameter is deprecated, use the corresponding parameter of @code{-device} instead. -@item addr=@var{addr} -Specify the controller's PCI address (if=virtio only). This parameter is -deprecated, use the corresponding parameter of @code{-device} instead. @item werror=@var{action},rerror=@var{action} Specify which @var{action} to take on write and read errors. Valid actions are: "ignore" (ignore the error and try to continue), "stop" (pause QEMU), -- 2.13.6 The -drive option serial was deprecated in QEMU 2.10. It's time to remove it. Tests need to be updated to set the serial number with -global instead of using the -drive option. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> --- include/hw/block/block.h | 1 - include/sysemu/blockdev.h | 1 - block/block-backend.c | 1 - blockdev.c | 10 ---------- hw/block/block.c | 13 ------------- hw/block/nvme.c | 1 - hw/block/virtio-blk.c | 1 - hw/ide/qdev.c | 1 - hw/scsi/scsi-disk.c | 1 - hw/usb/dev-storage.c | 1 - tests/ahci-test.c | 6 +++--- tests/ide-test.c | 8 ++++---- qemu-doc.texi | 5 ----- qemu-options.hx | 6 +----- 14 files changed, 8 insertions(+), 48 deletions(-) diff --git a/include/hw/block/block.h b/include/hw/block/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/block/block.h +++ b/include/hw/block/block.h @@ -XXX,XX +XXX,XX @@ static inline unsigned int get_physical_block_exp(BlockConf *conf) /* Configuration helpers */ -void blkconf_serial(BlockConf *conf, char **serial); bool blkconf_geometry(BlockConf *conf, int *trans, unsigned cyls_max, unsigned heads_max, unsigned secs_max, Error **errp); diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/blockdev.h +++ b/include/sysemu/blockdev.h @@ -XXX,XX +XXX,XX @@ struct DriveInfo { bool is_default; /* Added by default_drive() ? */ int media_cd; QemuOpts *opts; - char *serial; QTAILQ_ENTRY(DriveInfo) next; }; diff --git a/block/block-backend.c b/block/block-backend.c index XXXXXXX..XXXXXXX 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -XXX,XX +XXX,XX @@ static void drive_info_del(DriveInfo *dinfo) return; } qemu_opts_del(dinfo->opts); - g_free(dinfo->serial); g_free(dinfo); } diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ QemuOptsList qemu_legacy_drive_opts = { .type = QEMU_OPT_STRING, .help = "interface (ide, scsi, sd, mtd, floppy, pflash, virtio)", },{ - .name = "serial", - .type = QEMU_OPT_STRING, - .help = "disk serial number", - },{ .name = "file", .type = QEMU_OPT_STRING, .help = "file name", @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) const char *werror, *rerror; bool read_only = false; bool copy_on_read; - const char *serial; const char *filename; Error *local_err = NULL; int i; const char *deprecated[] = { - "serial" }; /* Change legacy command line options into QMP ones */ @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) goto fail; } - /* Serial number */ - serial = qemu_opt_get(legacy_opts, "serial"); - /* no id supplied -> create one */ if (qemu_opts_id(all_opts) == NULL) { char *new_id; @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) dinfo->type = type; dinfo->bus = bus_id; dinfo->unit = unit_id; - dinfo->serial = g_strdup(serial); blk_set_legacy_dinfo(blk, dinfo); diff --git a/hw/block/block.c b/hw/block/block.c index XXXXXXX..XXXXXXX 100644 --- a/hw/block/block.c +++ b/hw/block/block.c @@ -XXX,XX +XXX,XX @@ #include "qapi/qapi-types-block.h" #include "qemu/error-report.h" -void blkconf_serial(BlockConf *conf, char **serial) -{ - DriveInfo *dinfo; - - if (!*serial) { - /* try to fall back to value set with legacy -drive serial=... */ - dinfo = blk_legacy_dinfo(conf->blk); - if (dinfo) { - *serial = g_strdup(dinfo->serial); - } - } -} - void blkconf_blocksizes(BlockConf *conf) { BlockBackend *blk = conf->blk; diff --git a/hw/block/nvme.c b/hw/block/nvme.c index XXXXXXX..XXXXXXX 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -XXX,XX +XXX,XX @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp) return; } - blkconf_serial(&n->conf, &n->serial); if (!n->serial) { error_setg(errp, "serial property not set"); return; diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index XXXXXXX..XXXXXXX 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -XXX,XX +XXX,XX @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp) return; } - blkconf_serial(&conf->conf, &conf->serial); if (!blkconf_apply_backend_options(&conf->conf, blk_is_read_only(conf->conf.blk), true, errp)) { diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c index XXXXXXX..XXXXXXX 100644 --- a/hw/ide/qdev.c +++ b/hw/ide/qdev.c @@ -XXX,XX +XXX,XX @@ static void ide_dev_initfn(IDEDevice *dev, IDEDriveKind kind, Error **errp) return; } - blkconf_serial(&dev->conf, &dev->serial); if (kind != IDE_CD) { if (!blkconf_geometry(&dev->conf, &dev->chs_trans, 65535, 16, 255, errp)) { diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index XXXXXXX..XXXXXXX 100644 --- a/hw/scsi/scsi-disk.c +++ b/hw/scsi/scsi-disk.c @@ -XXX,XX +XXX,XX @@ static void scsi_realize(SCSIDevice *dev, Error **errp) return; } - blkconf_serial(&s->qdev.conf, &s->serial); blkconf_blocksizes(&s->qdev.conf); if (s->qdev.conf.logical_block_size > diff --git a/hw/usb/dev-storage.c b/hw/usb/dev-storage.c index XXXXXXX..XXXXXXX 100644 --- a/hw/usb/dev-storage.c +++ b/hw/usb/dev-storage.c @@ -XXX,XX +XXX,XX @@ static void usb_msd_storage_realize(USBDevice *dev, Error **errp) return; } - blkconf_serial(&s->conf, &dev->serial); blkconf_blocksizes(&s->conf); if (!blkconf_apply_backend_options(&s->conf, blk_is_read_only(blk), true, errp)) { diff --git a/tests/ahci-test.c b/tests/ahci-test.c index XXXXXXX..XXXXXXX 100644 --- a/tests/ahci-test.c +++ b/tests/ahci-test.c @@ -XXX,XX +XXX,XX @@ static AHCIQState *ahci_boot(const char *cli, ...) s = ahci_vboot(cli, ap); va_end(ap); } else { - cli = "-drive if=none,id=drive0,file=%s,cache=writeback,serial=%s" - ",format=%s" + cli = "-drive if=none,id=drive0,file=%s,cache=writeback,format=%s" " -M q35 " "-device ide-hd,drive=drive0 " + "-global ide-hd.serial=%s " "-global ide-hd.ver=%s"; - s = ahci_boot(cli, tmp_path, "testdisk", imgfmt, "version"); + s = ahci_boot(cli, tmp_path, imgfmt, "testdisk", "version"); } return s; diff --git a/tests/ide-test.c b/tests/ide-test.c index XXXXXXX..XXXXXXX 100644 --- a/tests/ide-test.c +++ b/tests/ide-test.c @@ -XXX,XX +XXX,XX @@ static void test_bmdma_no_busmaster(void) static void test_bmdma_setup(void) { ide_test_start( - "-drive file=%s,if=ide,serial=%s,cache=writeback,format=raw " - "-global ide-hd.ver=%s", + "-drive file=%s,if=ide,cache=writeback,format=raw " + "-global ide-hd.serial=%s -global ide-hd.ver=%s", tmp_path, "testdisk", "version"); qtest_irq_intercept_in(global_qtest, "ioapic"); } @@ -XXX,XX +XXX,XX @@ static void test_identify(void) int ret; ide_test_start( - "-drive file=%s,if=ide,serial=%s,cache=writeback,format=raw " - "-global ide-hd.ver=%s", + "-drive file=%s,if=ide,cache=writeback,format=raw " + "-global ide-hd.serial=%s -global ide-hd.ver=%s", tmp_path, "testdisk", "version"); dev = get_pci_device(&bmdma_bar, &ide_bar); diff --git a/qemu-doc.texi b/qemu-doc.texi index XXXXXXX..XXXXXXX 100644 --- a/qemu-doc.texi +++ b/qemu-doc.texi @@ -XXX,XX +XXX,XX @@ with ``-device ...,netdev=x''), or ``-nic user,smb=/some/dir'' (for embedded NICs). The new syntax allows different settings to be provided per NIC. -@subsection -drive serial=... (since 2.10.0) - -The drive serial argument is replaced by the the serial argument -that can be specified with the ``-device'' parameter. - @subsection -usbdevice (since 2.10.0) The ``-usbdevice DEV'' argument is now a synonym for setting diff --git a/qemu-options.hx b/qemu-options.hx index XXXXXXX..XXXXXXX 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -XXX,XX +XXX,XX @@ ETEXI DEF("drive", HAS_ARG, QEMU_OPTION_drive, "-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n" " [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n" - " [,snapshot=on|off][,serial=s][,rerror=ignore|stop|report]\n" + " [,snapshot=on|off][,rerror=ignore|stop|report]\n" " [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n" " [,readonly=on|off][,copy-on-read=on|off]\n" " [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n" @@ -XXX,XX +XXX,XX @@ The default mode is @option{cache=writeback}. Specify which disk @var{format} will be used rather than detecting the format. Can be used to specify format=raw to avoid interpreting an untrusted format header. -@item serial=@var{serial} -This option specifies the serial number to assign to the device. This -parameter is deprecated, use the corresponding parameter of @code{-device} -instead. @item werror=@var{action},rerror=@var{action} Specify which @var{action} to take on write and read errors. Valid actions are: "ignore" (ignore the error and try to continue), "stop" (pause QEMU), -- 2.13.6 We removed all options from the 'deprecated' array, so the code is dead and can be removed as well. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> --- blockdev.c | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) const char *filename; Error *local_err = NULL; int i; - const char *deprecated[] = { - }; /* Change legacy command line options into QMP ones */ static const struct { @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) goto fail; } - /* Other deprecated options */ - if (!qtest_enabled()) { - for (i = 0; i < ARRAY_SIZE(deprecated); i++) { - if (qemu_opt_get(legacy_opts, deprecated[i]) != NULL) { - error_report("'%s' is deprecated, please use the corresponding " - "option of '-device' instead", deprecated[i]); - } - } - } - /* Media type */ value = qemu_opt_get(legacy_opts, "media"); if (value) { -- 2.13.6 The following changes since commit c4e0780ed1ffd056f205348d387a61b4136a45df: Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-4.0-pull-request' into staging (2019-03-07 18:40:43 +0000) are available in the Git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream for you to fetch changes up to e88153ea9a40009a8ae7648282c0eac1b7f5494f: qcow2 spec: Describe string header extensions (2019-03-08 12:26:46 +0100) ---------------------------------------------------------------- Block layer patches: - qcow2: Support for external data files - qcow2: Default to 4KB for the qcow2 cache entry size - Apply block driver whitelist for -drive format=help - Several qemu-iotests improvements ---------------------------------------------------------------- Alberto Garcia (1): qcow2: Default to 4KB for the qcow2 cache entry size Andrey Shinkevich (4): iotests: open notrun files in text mode block: iterate_format with account of whitelisting iotests: ask QEMU for supported formats iotests: check whitelisted formats Kevin Wolf (21): qemu-iotests: Test qcow2 preallocation modes qcow2: Simplify preallocation code qcow2: Extend spec for external data files qcow2: Basic definitions for external data files qcow2: Pass bs to qcow2_get_cluster_type() qcow2: Prepare qcow2_get_cluster_type() for external data file qcow2: Prepare count_contiguous_clusters() for external data file qcow2: Don't assume 0 is an invalid cluster offset qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset() qcow2: Prepare qcow2_co_block_status() for data file qcow2: External file I/O qcow2: Return error for snapshot operation with data file qcow2: Support external data file in qemu-img check qcow2: Add basic data-file infrastructure qcow2: Creating images with external data file qcow2: Store data file name in the image qcow2: Implement data-file-raw create option qemu-iotests: Preallocation with external data file qemu-iotests: General tests for qcow2 with external data file qemu-iotests: amend with external data file qcow2 spec: Describe string header extensions Philippe Mathieu-Daudé (6): tests/multiboot: Improve portability by searching bash in the $PATH tests/bios-tables: Improve portability by searching bash in the $PATH qemu-iotests: Improve portability by searching bash in the $PATH qemu-iotests: Ensure GNU sed is used ahci-test: Add dependency to qemu-img tool qemu-iotests: Add dependency to qemu-nbd tool Stefan Hajnoczi (1): iotests: use iotests.VM in 238 qapi/block-core.json | 26 ++- docs/interop/qcow2.txt | 54 +++++- docs/qcow2-cache.txt | 17 +- block/qcow2.h | 66 +++++-- include/block/block.h | 2 +- include/block/block_int.h | 2 + block.c | 23 ++- block/qcow2-bitmap.c | 7 +- block/qcow2-cache.c | 6 +- block/qcow2-cluster.c | 182 +++++++++++------- block/qcow2-refcount.c | 88 ++++++--- block/qcow2-snapshot.c | 22 ++- block/qcow2.c | 326 +++++++++++++++++++++++++++----- blockdev.c | 4 +- qemu-img.c | 2 +- tests/Makefile.include | 4 +- tests/data/acpi/rebuild-expected-aml.sh | 2 +- tests/multiboot/run_test.sh | 2 +- tests/qemu-iotests/001 | 2 +- tests/qemu-iotests/002 | 2 +- tests/qemu-iotests/003 | 2 +- tests/qemu-iotests/004 | 2 +- tests/qemu-iotests/005 | 2 +- tests/qemu-iotests/007 | 2 +- tests/qemu-iotests/008 | 2 +- tests/qemu-iotests/009 | 2 +- tests/qemu-iotests/010 | 2 +- tests/qemu-iotests/011 | 2 +- tests/qemu-iotests/012 | 2 +- tests/qemu-iotests/013 | 2 +- tests/qemu-iotests/014 | 2 +- tests/qemu-iotests/015 | 2 +- tests/qemu-iotests/017 | 2 +- tests/qemu-iotests/018 | 2 +- tests/qemu-iotests/019 | 2 +- tests/qemu-iotests/020 | 2 +- tests/qemu-iotests/021 | 2 +- tests/qemu-iotests/022 | 2 +- tests/qemu-iotests/023 | 2 +- tests/qemu-iotests/024 | 2 +- tests/qemu-iotests/025 | 2 +- tests/qemu-iotests/026 | 2 +- tests/qemu-iotests/027 | 2 +- tests/qemu-iotests/028 | 2 +- tests/qemu-iotests/029 | 2 +- tests/qemu-iotests/031 | 2 +- tests/qemu-iotests/031.out | 8 +- tests/qemu-iotests/032 | 2 +- tests/qemu-iotests/033 | 2 +- tests/qemu-iotests/034 | 2 +- tests/qemu-iotests/035 | 2 +- tests/qemu-iotests/036 | 2 +- tests/qemu-iotests/036.out | 4 +- tests/qemu-iotests/037 | 2 +- tests/qemu-iotests/038 | 2 +- tests/qemu-iotests/039 | 2 +- tests/qemu-iotests/042 | 2 +- tests/qemu-iotests/043 | 2 +- tests/qemu-iotests/046 | 2 +- tests/qemu-iotests/047 | 2 +- tests/qemu-iotests/048 | 2 +- tests/qemu-iotests/049 | 2 +- tests/qemu-iotests/050 | 2 +- tests/qemu-iotests/051 | 2 +- tests/qemu-iotests/052 | 2 +- tests/qemu-iotests/053 | 2 +- tests/qemu-iotests/054 | 2 +- tests/qemu-iotests/058 | 2 +- tests/qemu-iotests/059 | 2 +- tests/qemu-iotests/060 | 2 +- tests/qemu-iotests/061 | 47 ++++- tests/qemu-iotests/061.out | 103 +++++++++- tests/qemu-iotests/062 | 2 +- tests/qemu-iotests/063 | 2 +- tests/qemu-iotests/064 | 2 +- tests/qemu-iotests/066 | 2 +- tests/qemu-iotests/067 | 2 +- tests/qemu-iotests/068 | 2 +- tests/qemu-iotests/069 | 2 +- tests/qemu-iotests/070 | 2 +- tests/qemu-iotests/071 | 2 +- tests/qemu-iotests/072 | 2 +- tests/qemu-iotests/073 | 2 +- tests/qemu-iotests/074 | 2 +- tests/qemu-iotests/075 | 2 +- tests/qemu-iotests/076 | 2 +- tests/qemu-iotests/077 | 2 +- tests/qemu-iotests/078 | 2 +- tests/qemu-iotests/079 | 2 +- tests/qemu-iotests/080 | 2 +- tests/qemu-iotests/081 | 2 +- tests/qemu-iotests/082 | 2 +- tests/qemu-iotests/082.out | 54 ++++++ tests/qemu-iotests/083 | 2 +- tests/qemu-iotests/084 | 2 +- tests/qemu-iotests/085 | 2 +- tests/qemu-iotests/086 | 2 +- tests/qemu-iotests/087 | 2 +- tests/qemu-iotests/088 | 2 +- tests/qemu-iotests/089 | 2 +- tests/qemu-iotests/090 | 2 +- tests/qemu-iotests/091 | 2 +- tests/qemu-iotests/092 | 2 +- tests/qemu-iotests/094 | 2 +- tests/qemu-iotests/095 | 2 +- tests/qemu-iotests/097 | 2 +- tests/qemu-iotests/098 | 2 +- tests/qemu-iotests/099 | 2 +- tests/qemu-iotests/101 | 2 +- tests/qemu-iotests/102 | 2 +- tests/qemu-iotests/103 | 2 +- tests/qemu-iotests/104 | 2 +- tests/qemu-iotests/105 | 2 +- tests/qemu-iotests/106 | 2 +- tests/qemu-iotests/107 | 2 +- tests/qemu-iotests/108 | 2 +- tests/qemu-iotests/109 | 2 +- tests/qemu-iotests/110 | 2 +- tests/qemu-iotests/111 | 2 +- tests/qemu-iotests/112 | 2 +- tests/qemu-iotests/113 | 2 +- tests/qemu-iotests/114 | 2 +- tests/qemu-iotests/115 | 2 +- tests/qemu-iotests/116 | 2 +- tests/qemu-iotests/117 | 2 +- tests/qemu-iotests/119 | 2 +- tests/qemu-iotests/120 | 2 +- tests/qemu-iotests/121 | 2 +- tests/qemu-iotests/122 | 2 +- tests/qemu-iotests/123 | 2 +- tests/qemu-iotests/125 | 2 +- tests/qemu-iotests/126 | 2 +- tests/qemu-iotests/127 | 2 +- tests/qemu-iotests/128 | 2 +- tests/qemu-iotests/130 | 2 +- tests/qemu-iotests/131 | 2 +- tests/qemu-iotests/133 | 2 +- tests/qemu-iotests/134 | 2 +- tests/qemu-iotests/135 | 2 +- tests/qemu-iotests/137 | 2 +- tests/qemu-iotests/138 | 2 +- tests/qemu-iotests/139 | 3 + tests/qemu-iotests/140 | 2 +- tests/qemu-iotests/141 | 2 +- tests/qemu-iotests/142 | 2 +- tests/qemu-iotests/143 | 2 +- tests/qemu-iotests/144 | 2 +- tests/qemu-iotests/145 | 2 +- tests/qemu-iotests/146 | 2 +- tests/qemu-iotests/150 | 2 +- tests/qemu-iotests/153 | 2 +- tests/qemu-iotests/154 | 2 +- tests/qemu-iotests/156 | 2 +- tests/qemu-iotests/157 | 2 +- tests/qemu-iotests/158 | 2 +- tests/qemu-iotests/159 | 2 +- tests/qemu-iotests/160 | 2 +- tests/qemu-iotests/161 | 2 +- tests/qemu-iotests/162 | 2 +- tests/qemu-iotests/170 | 2 +- tests/qemu-iotests/171 | 2 +- tests/qemu-iotests/172 | 2 +- tests/qemu-iotests/173 | 2 +- tests/qemu-iotests/174 | 2 +- tests/qemu-iotests/175 | 2 +- tests/qemu-iotests/176 | 2 +- tests/qemu-iotests/177 | 2 +- tests/qemu-iotests/178 | 2 +- tests/qemu-iotests/179 | 2 +- tests/qemu-iotests/181 | 2 +- tests/qemu-iotests/182 | 2 +- tests/qemu-iotests/183 | 2 +- tests/qemu-iotests/184 | 2 +- tests/qemu-iotests/185 | 2 +- tests/qemu-iotests/186 | 2 +- tests/qemu-iotests/187 | 2 +- tests/qemu-iotests/188 | 2 +- tests/qemu-iotests/189 | 2 +- tests/qemu-iotests/190 | 2 +- tests/qemu-iotests/191 | 2 +- tests/qemu-iotests/192 | 2 +- tests/qemu-iotests/195 | 2 +- tests/qemu-iotests/197 | 2 +- tests/qemu-iotests/198 | 2 +- tests/qemu-iotests/200 | 2 +- tests/qemu-iotests/201 | 2 +- tests/qemu-iotests/204 | 2 +- tests/qemu-iotests/214 | 2 +- tests/qemu-iotests/215 | 2 +- tests/qemu-iotests/217 | 2 +- tests/qemu-iotests/220 | 2 +- tests/qemu-iotests/220.out | 2 +- tests/qemu-iotests/221 | 2 +- tests/qemu-iotests/223 | 2 +- tests/qemu-iotests/225 | 2 +- tests/qemu-iotests/226 | 2 +- tests/qemu-iotests/227 | 2 +- tests/qemu-iotests/229 | 2 +- tests/qemu-iotests/231 | 2 +- tests/qemu-iotests/232 | 2 +- tests/qemu-iotests/233 | 2 +- tests/qemu-iotests/238 | 7 +- tests/qemu-iotests/243 | 85 +++++++++ tests/qemu-iotests/243.out | 58 ++++++ tests/qemu-iotests/244 | 200 ++++++++++++++++++++ tests/qemu-iotests/244.out | 125 ++++++++++++ tests/qemu-iotests/check | 15 +- tests/qemu-iotests/common.config | 2 +- tests/qemu-iotests/common.filter | 38 ++-- tests/qemu-iotests/common.nbd | 2 +- tests/qemu-iotests/common.pattern | 2 +- tests/qemu-iotests/common.qemu | 2 +- tests/qemu-iotests/common.rc | 15 +- tests/qemu-iotests/common.tls | 2 +- tests/qemu-iotests/group | 2 + tests/qemu-iotests/iotests.py | 45 ++++- 216 files changed, 1585 insertions(+), 423 deletions(-) create mode 100755 tests/qemu-iotests/243 create mode 100644 tests/qemu-iotests/243.out create mode 100755 tests/qemu-iotests/244 create mode 100644 tests/qemu-iotests/244.out From: Stefan Hajnoczi <stefanha@redhat.com> Test 238 does not require the kvm accelerator. Using the qtest accelerator allows the test to run in both non-kvm and non-tcg environments. iotests.VM implicitly uses the qtest accelerator and is really the class that this test should be using. Switch to that instead of qemu.QEMUMachine. Suggested-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/238 | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/tests/qemu-iotests/238 b/tests/qemu-iotests/238 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/238 +++ b/tests/qemu-iotests/238 @@ -XXX,XX +XXX,XX @@ import os import iotests from iotests import log -sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) - -from qemu import QEMUMachine - if iotests.qemu_default_machine == 's390-ccw-virtio': virtio_scsi_device = 'virtio-scsi-ccw' else: virtio_scsi_device = 'virtio-scsi-pci' -vm = QEMUMachine(iotests.qemu_prog) -vm.add_args('-machine', 'accel=kvm') +vm = iotests.VM() vm.launch() log(vm.qmp('blockdev-add', node_name='hd0', driver='null-co')) -- 2.20.1 From: Alberto Garcia <berto@igalia.com> QEMU 2.12 (commit 1221fe6f636754ab5f2c1c87caa77633e9123622) introduced a new setting called l2-cache-entry-size that allows making entries on the qcow2 L2 cache smaller than the cluster size. I have been performing several tests with different cluster and entry sizes and all of them show that reducing the entry size (aka L2 slice) consistently improves I/O performance, notably during random I/O (all tests done with sequential I/O show similar results). This is to be expected because loading and evicting an L2 slice is more expensive the larger the slice is. Here are some numbers on fully populated 40GB qcow2 images. The rightmost column represents the maximum L2 cache size in both cases. Cluster size = 64 KB |-------------+--------------+--------------+--------------| | | 1MB L2 cache | 3MB L2 cache | 5MB L2 cache | |-------------+--------------+--------------+--------------| | 4KB slices | 6545 IOPS | 12045 IOPS | 55680 IOPS | | 16KB slices | 5177 IOPS | 9798 IOPS | 56278 IOPS | | 64KB slices | 2718 IOPS | 5326 IOPS | 57355 IOPS | |-------------+--------------+--------------+--------------| Cluster size = 256 KB |--------------+----------------+--------------+-----------------| | | 512KB L2 cache | 1MB L2 cache | 1280KB L2 cache | |--------------+----------------+--------------+-----------------| | 4KB slices | 8539 IOPS | 21071 IOPS | 55417 IOPS | | 64KB slices | 3598 IOPS | 9772 IOPS | 57687 IOPS | | 256KB slices | 1415 IOPS | 4120 IOPS | 58001 IOPS | |--------------+----------------+--------------+-----------------| As can be seen in the numbers, the only exception to the rule is when the cache is large enough to hold all L2 tables. This is also to be expected because in this case no cache entry is ever evicted so reducing its size doesn't bring any benefit. This patch sets the default L2 cache entry size to 4KB except when the cache is large enough for the whole disk. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- docs/qcow2-cache.txt | 17 +++++++++++------ block/qcow2.c | 12 ++++++++++++ 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt index XXXXXXX..XXXXXXX 100644 --- a/docs/qcow2-cache.txt +++ b/docs/qcow2-cache.txt @@ -XXX,XX +XXX,XX @@ refcount cache is as small as possible unless overridden by the user. Using smaller cache entries --------------------------- -The qcow2 L2 cache stores complete tables by default. This means that -if QEMU needs an entry from an L2 table then the whole table is read -from disk and is kept in the cache. If the cache is full then a -complete table needs to be evicted first. +The qcow2 L2 cache can store complete tables. This means that if QEMU +needs an entry from an L2 table then the whole table is read from disk +and is kept in the cache. If the cache is full then a complete table +needs to be evicted first. This can be inefficient with large cluster sizes since it results in more disk I/O and wastes more cache memory. @@ -XXX,XX +XXX,XX @@ it smaller than the cluster size. This can be configured using the -drive file=hd.qcow2,l2-cache-size=2097152,l2-cache-entry-size=4096 +Since QEMU 4.0 the value of l2-cache-entry-size defaults to 4KB (or +the cluster size if it's smaller). + Some things to take into account: - The L2 cache entry size has the same restrictions as the cluster @@ -XXX,XX +XXX,XX @@ Some things to take into account: - Try different entry sizes to see which one gives faster performance in your case. The block size of the host filesystem is generally a - good default (usually 4096 bytes in the case of ext4). + good default (usually 4096 bytes in the case of ext4, hence the + default). - Only the L2 cache can be configured this way. The refcount cache always uses the cluster size as the entry size. @@ -XXX,XX +XXX,XX @@ Some things to take into account: (as explained in the "Choosing the right cache sizes" and "How to configure the cache sizes" sections in this document) then none of this is necessary and you can omit the "l2-cache-entry-size" - parameter altogether. + parameter altogether. In this case QEMU makes the entry size + equal to the cluster size by default. Reducing the memory usage diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, BDRVQcow2State *s = bs->opaque; uint64_t combined_cache_size, l2_cache_max_setting; bool l2_cache_size_set, refcount_cache_size_set, combined_cache_size_set; + bool l2_cache_entry_size_set; int min_refcount_cache = MIN_REFCOUNT_CACHE_SIZE * s->cluster_size; uint64_t virtual_disk_size = bs->total_sectors * BDRV_SECTOR_SIZE; uint64_t max_l2_cache = virtual_disk_size / (s->cluster_size / 8); @@ -XXX,XX +XXX,XX @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, combined_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_CACHE_SIZE); l2_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_L2_CACHE_SIZE); refcount_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_REFCOUNT_CACHE_SIZE); + l2_cache_entry_size_set = qemu_opt_get(opts, QCOW2_OPT_L2_CACHE_ENTRY_SIZE); combined_cache_size = qemu_opt_get_size(opts, QCOW2_OPT_CACHE_SIZE, 0); l2_cache_max_setting = qemu_opt_get_size(opts, QCOW2_OPT_L2_CACHE_SIZE, @@ -XXX,XX +XXX,XX @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, } } } + + /* + * If the L2 cache is not enough to cover the whole disk then + * default to 4KB entries. Smaller entries reduce the cost of + * loads and evictions and increase I/O performance. + */ + if (*l2_cache_size < max_l2_cache && !l2_cache_entry_size_set) { + *l2_cache_entry_size = MIN(s->cluster_size, 4096); + } + /* l2_cache_size and refcount_cache_size are ensured to have at least * their minimum values in qcow2_update_options_prepare() */ -- 2.20.1 From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Replace the binary mode with the default text one when *.notrun files are opened for skipped tests. That change is made for the compatibility with Python 3 which returns error otherwise. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/iotests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -XXX,XX +XXX,XX @@ def notrun(reason): # Each test in qemu-iotests has a number ("seq") seq = os.path.basename(sys.argv[0]) - open('%s/%s.notrun' % (output_dir, seq), 'wb').write(reason + '\n') + open('%s/%s.notrun' % (output_dir, seq), 'w').write(reason + '\n') print('%s not run: %s' % (seq, reason)) sys.exit(0) -- 2.20.1 From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> bdrv_iterate_format (which is currently only used for printing out the formats supported by the block layer) doesn't take format whitelisting into account. This creates a problem for tests: they enumerate supported formats to decide which tests to enable, but then discover that QEMU doesn't let them actually use some of those formats. To avoid that, exclude formats that are not whitelisted from enumeration, if whitelisting is in use. Since we have separate whitelists for r/w and r/o, take this a parameter to bdrv_iterate_format, and print two lists of supported formats (r/w and r/o) in main qemu. Signed-off-by: Roman Kagan <rkagan@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/block.h | 2 +- block.c | 23 +++++++++++++++++++---- blockdev.c | 4 +++- qemu-img.c | 2 +- 4 files changed, 24 insertions(+), 7 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -XXX,XX +XXX,XX @@ void bdrv_next_cleanup(BdrvNextIterator *it); BlockDriverState *bdrv_next_monitor_owned(BlockDriverState *bs); bool bdrv_is_encrypted(BlockDriverState *bs); void bdrv_iterate_format(void (*it)(void *opaque, const char *name), - void *opaque); + void *opaque, bool read_only); const char *bdrv_get_node_name(const BlockDriverState *bs); const char *bdrv_get_device_name(const BlockDriverState *bs); const char *bdrv_get_device_or_node_name(const BlockDriverState *bs); diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ BlockDriver *bdrv_find_format(const char *format_name) return bdrv_do_find_format(format_name); } -int bdrv_is_whitelisted(BlockDriver *drv, bool read_only) +static int bdrv_format_is_whitelisted(const char *format_name, bool read_only) { static const char *whitelist_rw[] = { CONFIG_BDRV_RW_WHITELIST @@ -XXX,XX +XXX,XX @@ int bdrv_is_whitelisted(BlockDriver *drv, bool read_only) } for (p = whitelist_rw; *p; p++) { - if (!strcmp(drv->format_name, *p)) { + if (!strcmp(format_name, *p)) { return 1; } } if (read_only) { for (p = whitelist_ro; *p; p++) { - if (!strcmp(drv->format_name, *p)) { + if (!strcmp(format_name, *p)) { return 1; } } @@ -XXX,XX +XXX,XX @@ int bdrv_is_whitelisted(BlockDriver *drv, bool read_only) return 0; } +int bdrv_is_whitelisted(BlockDriver *drv, bool read_only) +{ + return bdrv_format_is_whitelisted(drv->format_name, read_only); +} + bool bdrv_uses_whitelist(void) { return use_bdrv_whitelist; @@ -XXX,XX +XXX,XX @@ static int qsort_strcmp(const void *a, const void *b) } void bdrv_iterate_format(void (*it)(void *opaque, const char *name), - void *opaque) + void *opaque, bool read_only) { BlockDriver *drv; int count = 0; @@ -XXX,XX +XXX,XX @@ void bdrv_iterate_format(void (*it)(void *opaque, const char *name), if (drv->format_name) { bool found = false; int i = count; + + if (use_bdrv_whitelist && !bdrv_is_whitelisted(drv, read_only)) { + continue; + } + while (formats && i && !found) { found = !strcmp(formats[--i], drv->format_name); } @@ -XXX,XX +XXX,XX @@ void bdrv_iterate_format(void (*it)(void *opaque, const char *name), bool found = false; int j = count; + if (use_bdrv_whitelist && + !bdrv_format_is_whitelisted(format_name, read_only)) { + continue; + } + while (formats && j && !found) { found = !strcmp(formats[--j], format_name); } diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ static BlockBackend *blockdev_init(const char *file, QDict *bs_opts, if ((buf = qemu_opt_get(opts, "format")) != NULL) { if (is_help_option(buf)) { error_printf("Supported formats:"); - bdrv_iterate_format(bdrv_format_print, NULL); + bdrv_iterate_format(bdrv_format_print, NULL, false); + error_printf("\nSupported formats (read-only):"); + bdrv_iterate_format(bdrv_format_print, NULL, true); error_printf("\n"); goto early_err; } diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static void QEMU_NORETURN help(void) " 'skip=N' skip N bs-sized blocks at the start of input\n"; printf("%s\nSupported formats:", help_msg); - bdrv_iterate_format(format_print, NULL); + bdrv_iterate_format(format_print, NULL, false); printf("\n\n" QEMU_HELP_BOTTOM "\n"); exit(EXIT_SUCCESS); } -- 2.20.1 From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Supported formats listed by 'qemu' may differ from those listed by 'qemu-img' due to whitelists. Some test cases require specific formats that may be used with qemu. They can be inquired directly by running 'qemu -drive format=help'. The response takes whitelists into account. The method supported_formats() serves for that. The method decorator skip_if_unsupported() checks if all requested formats are whitelisted. If not, the test case will be skipped. That has been implemented in the 'check' file in the way similar to the 'test notrun' mechanism. Suggested-by: Roman Kagan <rkagan@virtuozzo.com> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/check | 13 ++++++++++- tests/qemu-iotests/iotests.py | 43 +++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -XXX,XX +XXX,XX @@ try=0 n_bad=0 bad="" notrun="" +casenotrun="" interrupt=true # by default don't output timestamps @@ -XXX,XX +XXX,XX @@ END { if (NR > 0) { echo "Not run:$notrun" echo "Not run:$notrun" >>check.log fi + if [ ! -z "$casenotrun" ] + then + echo "Some cases not run in:$casenotrun" + echo "Some cases not run in:$casenotrun" >>check.log + fi if [ ! -z "$n_bad" -a $n_bad != 0 ] then echo "Failures:$bad" @@ -XXX,XX +XXX,XX @@ do printf " " # prettier output with timestamps. fi rm -f core $seq.notrun + rm -f $seq.casenotrun start=$(_wallclock) $timestamp && printf %s " [$(date "+%T")]" @@ -XXX,XX +XXX,XX @@ do fi fi fi - + if [ -f $seq.casenotrun ] + then + cat $seq.casenotrun + casenotrun="$casenotrun $seq" + fi fi # come here for each test, except when $showme is true diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -XXX,XX +XXX,XX @@ def notrun(reason): print('%s not run: %s' % (seq, reason)) sys.exit(0) +def case_notrun(reason): + '''Skip this test case''' + # Each test in qemu-iotests has a number ("seq") + seq = os.path.basename(sys.argv[0]) + + open('%s/%s.casenotrun' % (output_dir, seq), 'a').write( + ' [case not run] ' + reason + '\n') + def verify_image_format(supported_fmts=[], unsupported_fmts=[]): assert not (supported_fmts and unsupported_fmts) @@ -XXX,XX +XXX,XX @@ def verify_quorum(): if not supports_quorum(): notrun('quorum support missing') +def qemu_pipe(*args): + '''Run qemu with an option to print something and exit (e.g. a help option), + and return its output''' + args = [qemu_prog] + qemu_opts + list(args) + subp = subprocess.Popen(args, stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + universal_newlines=True) + exitcode = subp.wait() + if exitcode < 0: + sys.stderr.write('qemu received signal %i: %s\n' % (-exitcode, + ' '.join(args))) + return subp.communicate()[0] + +def supported_formats(read_only=False): + '''Set 'read_only' to True to check ro-whitelist + Otherwise, rw-whitelist is checked''' + format_message = qemu_pipe("-drive", "format=help") + line = 1 if read_only else 0 + return format_message.splitlines()[line].split(":")[1].split() + +def skip_if_unsupported(required_formats=[], read_only=False): + '''Skip Test Decorator + Runs the test if all the required formats are whitelisted''' + def skip_test_decorator(func): + def func_wrapper(*args, **kwargs): + usf_list = list(set(required_formats) - + set(supported_formats(read_only))) + if usf_list: + case_notrun('{}: formats {} are not whitelisted'.format( + args[0], usf_list)) + else: + return func(*args, **kwargs) + return func_wrapper + return skip_test_decorator + def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], unsupported_fmts=[]): '''Run tests''' -- 2.20.1 From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Some test cases require specific formats. The method decorator skip_if_unsupported() checks if requested formats are whitelisted. The test #139 was selected for a sample output, after running $ ./check -qcow2 131-140 137 3s ... 138 0s ... 139 2s ... [case not run] testBlkDebug (__main__.TestBlockdevDel): formats ['blkdebug'] are not whitelisted [case not run] testBlkVerify (__main__.TestBlockdevDel): formats ['blkverify'] are not whitelisted [case not run] testQuorum (__main__.TestBlockdevDel): formats ['quorum'] are not whitelisted 140 0s ... Not run: 131 135 136 Some cases not run in: 139 Passed all 7 tests Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/139 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/139 +++ b/tests/qemu-iotests/139 @@ -XXX,XX +XXX,XX @@ class TestBlockdevDel(iotests.QMPTestCase): # FIXME mirror0 disappears, drive-mirror doesn't take a reference #self.delBlockDriverState('mirror0') + @iotests.skip_if_unsupported(['blkdebug']) def testBlkDebug(self): self.addBlkDebug('debug0', 'node0') # 'node0' is used by the blkdebug node @@ -XXX,XX +XXX,XX @@ class TestBlockdevDel(iotests.QMPTestCase): self.delBlockDriverState('debug0') self.checkBlockDriverState('node0', False) + @iotests.skip_if_unsupported(['blkverify']) def testBlkVerify(self): self.addBlkVerify('verify0', 'node0', 'node1') # We cannot remove the children of a blkverify device @@ -XXX,XX +XXX,XX @@ class TestBlockdevDel(iotests.QMPTestCase): self.checkBlockDriverState('node0', False) self.checkBlockDriverState('node1', False) + @iotests.skip_if_unsupported(['quorum']) def testQuorum(self): if not iotests.supports_quorum(): return -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Bash is not always installed as /bin/bash. In particular on OpenBSD, the package installs it in /usr/local/bin. Use the 'env' shebang to search bash in the $PATH. Reviewed-by: Kamil Rytarowski <n54@gmx.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/multiboot/run_test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/multiboot/run_test.sh b/tests/multiboot/run_test.sh index XXXXXXX..XXXXXXX 100755 --- a/tests/multiboot/run_test.sh +++ b/tests/multiboot/run_test.sh @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # Copyright (c) 2013 Kevin Wolf <kwolf@redhat.com> # -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Bash is not always installed as /bin/bash. In particular on OpenBSD, the package installs it in /usr/local/bin. Use the 'env' shebang to search bash in the $PATH. Reviewed-by: Kamil Rytarowski <n54@gmx.com> Reviewed-by: Igor Mammedov <imammedo@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/data/acpi/rebuild-expected-aml.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/data/acpi/rebuild-expected-aml.sh b/tests/data/acpi/rebuild-expected-aml.sh index XXXXXXX..XXXXXXX 100755 --- a/tests/data/acpi/rebuild-expected-aml.sh +++ b/tests/data/acpi/rebuild-expected-aml.sh @@ -XXX,XX +XXX,XX @@ -#! /bin/bash +#!/usr/bin/env bash # # Rebuild expected AML files for acpi unit-test -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Bash is not always installed as /bin/bash. In particular on OpenBSD, the package installs it in /usr/local/bin. Use the 'env' shebang to search bash in the $PATH. Patch created mechanically by running: $ git grep -lE '#! ?/bin/bash' -- tests/qemu-iotests \ | while read f; do \ sed -i 's|^#!.\?/bin/bash$|#!/usr/bin/env bash|' $f; \ done Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/001 | 2 +- tests/qemu-iotests/002 | 2 +- tests/qemu-iotests/003 | 2 +- tests/qemu-iotests/004 | 2 +- tests/qemu-iotests/005 | 2 +- tests/qemu-iotests/007 | 2 +- tests/qemu-iotests/008 | 2 +- tests/qemu-iotests/009 | 2 +- tests/qemu-iotests/010 | 2 +- tests/qemu-iotests/011 | 2 +- tests/qemu-iotests/012 | 2 +- tests/qemu-iotests/013 | 2 +- tests/qemu-iotests/014 | 2 +- tests/qemu-iotests/015 | 2 +- tests/qemu-iotests/017 | 2 +- tests/qemu-iotests/018 | 2 +- tests/qemu-iotests/019 | 2 +- tests/qemu-iotests/020 | 2 +- tests/qemu-iotests/021 | 2 +- tests/qemu-iotests/022 | 2 +- tests/qemu-iotests/023 | 2 +- tests/qemu-iotests/024 | 2 +- tests/qemu-iotests/025 | 2 +- tests/qemu-iotests/026 | 2 +- tests/qemu-iotests/027 | 2 +- tests/qemu-iotests/028 | 2 +- tests/qemu-iotests/029 | 2 +- tests/qemu-iotests/031 | 2 +- tests/qemu-iotests/032 | 2 +- tests/qemu-iotests/033 | 2 +- tests/qemu-iotests/034 | 2 +- tests/qemu-iotests/035 | 2 +- tests/qemu-iotests/036 | 2 +- tests/qemu-iotests/037 | 2 +- tests/qemu-iotests/038 | 2 +- tests/qemu-iotests/039 | 2 +- tests/qemu-iotests/042 | 2 +- tests/qemu-iotests/043 | 2 +- tests/qemu-iotests/046 | 2 +- tests/qemu-iotests/047 | 2 +- tests/qemu-iotests/048 | 2 +- tests/qemu-iotests/049 | 2 +- tests/qemu-iotests/050 | 2 +- tests/qemu-iotests/051 | 2 +- tests/qemu-iotests/052 | 2 +- tests/qemu-iotests/053 | 2 +- tests/qemu-iotests/054 | 2 +- tests/qemu-iotests/058 | 2 +- tests/qemu-iotests/059 | 2 +- tests/qemu-iotests/060 | 2 +- tests/qemu-iotests/061 | 2 +- tests/qemu-iotests/062 | 2 +- tests/qemu-iotests/063 | 2 +- tests/qemu-iotests/064 | 2 +- tests/qemu-iotests/066 | 2 +- tests/qemu-iotests/067 | 2 +- tests/qemu-iotests/068 | 2 +- tests/qemu-iotests/069 | 2 +- tests/qemu-iotests/070 | 2 +- tests/qemu-iotests/071 | 2 +- tests/qemu-iotests/072 | 2 +- tests/qemu-iotests/073 | 2 +- tests/qemu-iotests/074 | 2 +- tests/qemu-iotests/075 | 2 +- tests/qemu-iotests/076 | 2 +- tests/qemu-iotests/077 | 2 +- tests/qemu-iotests/078 | 2 +- tests/qemu-iotests/079 | 2 +- tests/qemu-iotests/080 | 2 +- tests/qemu-iotests/081 | 2 +- tests/qemu-iotests/082 | 2 +- tests/qemu-iotests/083 | 2 +- tests/qemu-iotests/084 | 2 +- tests/qemu-iotests/085 | 2 +- tests/qemu-iotests/086 | 2 +- tests/qemu-iotests/087 | 2 +- tests/qemu-iotests/088 | 2 +- tests/qemu-iotests/089 | 2 +- tests/qemu-iotests/090 | 2 +- tests/qemu-iotests/091 | 2 +- tests/qemu-iotests/092 | 2 +- tests/qemu-iotests/094 | 2 +- tests/qemu-iotests/095 | 2 +- tests/qemu-iotests/097 | 2 +- tests/qemu-iotests/098 | 2 +- tests/qemu-iotests/099 | 2 +- tests/qemu-iotests/101 | 2 +- tests/qemu-iotests/102 | 2 +- tests/qemu-iotests/103 | 2 +- tests/qemu-iotests/104 | 2 +- tests/qemu-iotests/105 | 2 +- tests/qemu-iotests/106 | 2 +- tests/qemu-iotests/107 | 2 +- tests/qemu-iotests/108 | 2 +- tests/qemu-iotests/109 | 2 +- tests/qemu-iotests/110 | 2 +- tests/qemu-iotests/111 | 2 +- tests/qemu-iotests/112 | 2 +- tests/qemu-iotests/113 | 2 +- tests/qemu-iotests/114 | 2 +- tests/qemu-iotests/115 | 2 +- tests/qemu-iotests/116 | 2 +- tests/qemu-iotests/117 | 2 +- tests/qemu-iotests/119 | 2 +- tests/qemu-iotests/120 | 2 +- tests/qemu-iotests/121 | 2 +- tests/qemu-iotests/122 | 2 +- tests/qemu-iotests/123 | 2 +- tests/qemu-iotests/125 | 2 +- tests/qemu-iotests/126 | 2 +- tests/qemu-iotests/127 | 2 +- tests/qemu-iotests/128 | 2 +- tests/qemu-iotests/130 | 2 +- tests/qemu-iotests/131 | 2 +- tests/qemu-iotests/133 | 2 +- tests/qemu-iotests/134 | 2 +- tests/qemu-iotests/135 | 2 +- tests/qemu-iotests/137 | 2 +- tests/qemu-iotests/138 | 2 +- tests/qemu-iotests/140 | 2 +- tests/qemu-iotests/141 | 2 +- tests/qemu-iotests/142 | 2 +- tests/qemu-iotests/143 | 2 +- tests/qemu-iotests/144 | 2 +- tests/qemu-iotests/145 | 2 +- tests/qemu-iotests/146 | 2 +- tests/qemu-iotests/150 | 2 +- tests/qemu-iotests/153 | 2 +- tests/qemu-iotests/154 | 2 +- tests/qemu-iotests/156 | 2 +- tests/qemu-iotests/157 | 2 +- tests/qemu-iotests/158 | 2 +- tests/qemu-iotests/159 | 2 +- tests/qemu-iotests/160 | 2 +- tests/qemu-iotests/161 | 2 +- tests/qemu-iotests/162 | 2 +- tests/qemu-iotests/170 | 2 +- tests/qemu-iotests/171 | 2 +- tests/qemu-iotests/172 | 2 +- tests/qemu-iotests/173 | 2 +- tests/qemu-iotests/174 | 2 +- tests/qemu-iotests/175 | 2 +- tests/qemu-iotests/176 | 2 +- tests/qemu-iotests/177 | 2 +- tests/qemu-iotests/178 | 2 +- tests/qemu-iotests/179 | 2 +- tests/qemu-iotests/181 | 2 +- tests/qemu-iotests/182 | 2 +- tests/qemu-iotests/183 | 2 +- tests/qemu-iotests/184 | 2 +- tests/qemu-iotests/185 | 2 +- tests/qemu-iotests/186 | 2 +- tests/qemu-iotests/187 | 2 +- tests/qemu-iotests/188 | 2 +- tests/qemu-iotests/189 | 2 +- tests/qemu-iotests/190 | 2 +- tests/qemu-iotests/191 | 2 +- tests/qemu-iotests/192 | 2 +- tests/qemu-iotests/195 | 2 +- tests/qemu-iotests/197 | 2 +- tests/qemu-iotests/198 | 2 +- tests/qemu-iotests/200 | 2 +- tests/qemu-iotests/201 | 2 +- tests/qemu-iotests/204 | 2 +- tests/qemu-iotests/214 | 2 +- tests/qemu-iotests/215 | 2 +- tests/qemu-iotests/217 | 2 +- tests/qemu-iotests/220 | 2 +- tests/qemu-iotests/221 | 2 +- tests/qemu-iotests/223 | 2 +- tests/qemu-iotests/225 | 2 +- tests/qemu-iotests/226 | 2 +- tests/qemu-iotests/227 | 2 +- tests/qemu-iotests/229 | 2 +- tests/qemu-iotests/231 | 2 +- tests/qemu-iotests/232 | 2 +- tests/qemu-iotests/233 | 2 +- tests/qemu-iotests/check | 2 +- tests/qemu-iotests/common.config | 2 +- tests/qemu-iotests/common.filter | 2 +- tests/qemu-iotests/common.nbd | 2 +- tests/qemu-iotests/common.pattern | 2 +- tests/qemu-iotests/common.qemu | 2 +- tests/qemu-iotests/common.rc | 2 +- tests/qemu-iotests/common.tls | 2 +- 185 files changed, 185 insertions(+), 185 deletions(-) diff --git a/tests/qemu-iotests/001 b/tests/qemu-iotests/001 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/001 +++ b/tests/qemu-iotests/001 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test simple read/write using plain bdrv_read/bdrv_write # diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/002 +++ b/tests/qemu-iotests/002 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test simple read/write using plain bdrv_pread/bdrv_pwrite # diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/003 +++ b/tests/qemu-iotests/003 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test simple read/write using bdrv_aio_readv/bdrv_aio_writev # diff --git a/tests/qemu-iotests/004 b/tests/qemu-iotests/004 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/004 +++ b/tests/qemu-iotests/004 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Make sure we can't read and write outside of the image size. # diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/005 +++ b/tests/qemu-iotests/005 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Make sure qemu-img can create 5TB images # diff --git a/tests/qemu-iotests/007 b/tests/qemu-iotests/007 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/007 +++ b/tests/qemu-iotests/007 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Check for one possible case of qcow2 refcount corruption. # diff --git a/tests/qemu-iotests/008 b/tests/qemu-iotests/008 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/008 +++ b/tests/qemu-iotests/008 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test simple asynchronous read/write operations. # diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/009 +++ b/tests/qemu-iotests/009 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Nolan I qcow2 corruption - incorrectly reports free clusters # diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/010 +++ b/tests/qemu-iotests/010 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Nolan II qcow2 corruption - wrong used cluster # diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/011 +++ b/tests/qemu-iotests/011 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test for AIO allocation on the same cluster # diff --git a/tests/qemu-iotests/012 b/tests/qemu-iotests/012 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/012 +++ b/tests/qemu-iotests/012 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Make sure we can open read-only images # diff --git a/tests/qemu-iotests/013 b/tests/qemu-iotests/013 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/013 +++ b/tests/qemu-iotests/013 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qcow2 pattern test, empty and compressed image - 4k cluster patterns # diff --git a/tests/qemu-iotests/014 b/tests/qemu-iotests/014 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/014 +++ b/tests/qemu-iotests/014 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qcow2 pattern test, complex patterns including compression and snapshots # Using patterns for 4k cluster size. diff --git a/tests/qemu-iotests/015 b/tests/qemu-iotests/015 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/015 +++ b/tests/qemu-iotests/015 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Combined test to grow the refcount table and test snapshots. # diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/017 +++ b/tests/qemu-iotests/017 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Simple backing file reads # diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/018 +++ b/tests/qemu-iotests/018 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Merge backing file into test image when converting the image # diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/019 +++ b/tests/qemu-iotests/019 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # When using a backing file for the output image in qemu-img convert, # the backing file clusters must not copied. The data must still be diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/020 +++ b/tests/qemu-iotests/020 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Commit changes to backing file # diff --git a/tests/qemu-iotests/021 b/tests/qemu-iotests/021 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/021 +++ b/tests/qemu-iotests/021 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test handling of invalid patterns arguments to qemu-io # diff --git a/tests/qemu-iotests/022 b/tests/qemu-iotests/022 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/022 +++ b/tests/qemu-iotests/022 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test bdrv_load/save_vmstate using the usual patterns # diff --git a/tests/qemu-iotests/023 b/tests/qemu-iotests/023 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/023 +++ b/tests/qemu-iotests/023 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qcow2 pattern test with various cluster sizes # diff --git a/tests/qemu-iotests/024 b/tests/qemu-iotests/024 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/024 +++ b/tests/qemu-iotests/024 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Rebasing COW images # diff --git a/tests/qemu-iotests/025 b/tests/qemu-iotests/025 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/025 +++ b/tests/qemu-iotests/025 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Resizing images # diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/026 +++ b/tests/qemu-iotests/026 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qcow2 error path testing # diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/027 +++ b/tests/qemu-iotests/027 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test that sub-cluster allocating writes zero the rest of the cluster # diff --git a/tests/qemu-iotests/028 b/tests/qemu-iotests/028 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/028 +++ b/tests/qemu-iotests/028 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test that backing files can be smaller than the image # diff --git a/tests/qemu-iotests/029 b/tests/qemu-iotests/029 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/029 +++ b/tests/qemu-iotests/029 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qcow2 internal snapshots/VM state tests # diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/031 +++ b/tests/qemu-iotests/031 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test that all qcow2 header extensions survive a header rewrite # diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/032 +++ b/tests/qemu-iotests/032 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test that AIO requests are drained before an image is closed. This used # to segfault because the request coroutine kept running even after the diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/033 +++ b/tests/qemu-iotests/033 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test aligned and misaligned write zeroes operations. # diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/034 +++ b/tests/qemu-iotests/034 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test bdrv_pwrite_zeroes with backing files (see also 154) # diff --git a/tests/qemu-iotests/035 b/tests/qemu-iotests/035 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/035 +++ b/tests/qemu-iotests/035 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Let a few AIO requests run in parallel and have them access different L2 # tables so that the cache has a chance to get used up. diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/036 +++ b/tests/qemu-iotests/036 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qcow2 feature bits # diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/037 +++ b/tests/qemu-iotests/037 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test COW from backing files # diff --git a/tests/qemu-iotests/038 b/tests/qemu-iotests/038 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/038 +++ b/tests/qemu-iotests/038 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test COW from backing files with AIO # diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/039 +++ b/tests/qemu-iotests/039 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qcow2 lazy refcounts # diff --git a/tests/qemu-iotests/042 b/tests/qemu-iotests/042 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/042 +++ b/tests/qemu-iotests/042 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qemu-img operation on zero size images # diff --git a/tests/qemu-iotests/043 b/tests/qemu-iotests/043 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/043 +++ b/tests/qemu-iotests/043 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test that qemu-img info --backing-chain detects infinite loops # diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/046 +++ b/tests/qemu-iotests/046 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test concurrent cluster allocations # diff --git a/tests/qemu-iotests/047 b/tests/qemu-iotests/047 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/047 +++ b/tests/qemu-iotests/047 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Regression test for commit b7ab0fea (which was a corruption fix, # despite the commit message claiming otherwise) diff --git a/tests/qemu-iotests/048 b/tests/qemu-iotests/048 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/048 +++ b/tests/qemu-iotests/048 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash ## ## qemu-img compare test ## diff --git a/tests/qemu-iotests/049 b/tests/qemu-iotests/049 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/049 +++ b/tests/qemu-iotests/049 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Check qemu-img option parsing # diff --git a/tests/qemu-iotests/050 b/tests/qemu-iotests/050 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/050 +++ b/tests/qemu-iotests/050 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qemu-img rebase with zero clusters # diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/051 +++ b/tests/qemu-iotests/051 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test command line configuration of block devices and driver-specific options # diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/052 +++ b/tests/qemu-iotests/052 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test bdrv_read/bdrv_write using BDRV_O_SNAPSHOT # diff --git a/tests/qemu-iotests/053 b/tests/qemu-iotests/053 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/053 +++ b/tests/qemu-iotests/053 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qemu-img convert when image length is not a multiple of cluster size # diff --git a/tests/qemu-iotests/054 b/tests/qemu-iotests/054 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/054 +++ b/tests/qemu-iotests/054 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test huge qcow2 images # diff --git a/tests/qemu-iotests/058 b/tests/qemu-iotests/058 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/058 +++ b/tests/qemu-iotests/058 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test export internal snapshot by qemu-nbd, convert it by qemu-img. # diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/059 +++ b/tests/qemu-iotests/059 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for vmdk # diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for image corruption (overlapping data structures) in qcow2 # diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/061 +++ b/tests/qemu-iotests/061 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for image option amendment in qcow2. # diff --git a/tests/qemu-iotests/062 b/tests/qemu-iotests/062 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/062 +++ b/tests/qemu-iotests/062 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for snapshotting images with unallocated zero clusters in # qcow2 diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/063 +++ b/tests/qemu-iotests/063 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # test of qemu-img convert -n - convert without creation # diff --git a/tests/qemu-iotests/064 b/tests/qemu-iotests/064 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/064 +++ b/tests/qemu-iotests/064 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test VHDX read/write from a sample image created with Hyper-V # diff --git a/tests/qemu-iotests/066 b/tests/qemu-iotests/066 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/066 +++ b/tests/qemu-iotests/066 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for preallocated zero clusters in qcow2 # diff --git a/tests/qemu-iotests/067 b/tests/qemu-iotests/067 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/067 +++ b/tests/qemu-iotests/067 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test automatic deletion of BDSes created by -drive/drive_add # diff --git a/tests/qemu-iotests/068 b/tests/qemu-iotests/068 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/068 +++ b/tests/qemu-iotests/068 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for loading a saved VM state from a qcow2 image # diff --git a/tests/qemu-iotests/069 b/tests/qemu-iotests/069 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/069 +++ b/tests/qemu-iotests/069 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for deleting a backing file # diff --git a/tests/qemu-iotests/070 b/tests/qemu-iotests/070 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/070 +++ b/tests/qemu-iotests/070 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test VHDX log replay from an image with a journal that needs to be # replayed diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/071 +++ b/tests/qemu-iotests/071 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for the QMP blkdebug and blkverify interfaces # diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/072 +++ b/tests/qemu-iotests/072 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for nested image formats # diff --git a/tests/qemu-iotests/073 b/tests/qemu-iotests/073 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/073 +++ b/tests/qemu-iotests/073 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test count_contiguous_clusters in qcow2 # diff --git a/tests/qemu-iotests/074 b/tests/qemu-iotests/074 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/074 +++ b/tests/qemu-iotests/074 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash ## ## qemu-img compare test (qcow2 only ones) ## diff --git a/tests/qemu-iotests/075 b/tests/qemu-iotests/075 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/075 +++ b/tests/qemu-iotests/075 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # cloop format input validation tests # diff --git a/tests/qemu-iotests/076 b/tests/qemu-iotests/076 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/076 +++ b/tests/qemu-iotests/076 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # parallels format input validation tests # diff --git a/tests/qemu-iotests/077 b/tests/qemu-iotests/077 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/077 +++ b/tests/qemu-iotests/077 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test concurrent pread/pwrite # diff --git a/tests/qemu-iotests/078 b/tests/qemu-iotests/078 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/078 +++ b/tests/qemu-iotests/078 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # bochs format input validation tests # diff --git a/tests/qemu-iotests/079 b/tests/qemu-iotests/079 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/079 +++ b/tests/qemu-iotests/079 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qcow2 preallocation with different cluster_sizes # diff --git a/tests/qemu-iotests/080 b/tests/qemu-iotests/080 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/080 +++ b/tests/qemu-iotests/080 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qcow2 format input validation tests # diff --git a/tests/qemu-iotests/081 b/tests/qemu-iotests/081 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/081 +++ b/tests/qemu-iotests/081 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test Quorum block driver # diff --git a/tests/qemu-iotests/082 b/tests/qemu-iotests/082 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/082 +++ b/tests/qemu-iotests/082 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qemu-img command line parsing # diff --git a/tests/qemu-iotests/083 b/tests/qemu-iotests/083 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/083 +++ b/tests/qemu-iotests/083 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test NBD client unexpected disconnect # diff --git a/tests/qemu-iotests/084 b/tests/qemu-iotests/084 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/084 +++ b/tests/qemu-iotests/084 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for VDI header corruption; image too large, and too many blocks. # Also simple test for creating dynamic and static VDI images. diff --git a/tests/qemu-iotests/085 b/tests/qemu-iotests/085 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/085 +++ b/tests/qemu-iotests/085 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Live snapshot tests # diff --git a/tests/qemu-iotests/086 b/tests/qemu-iotests/086 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/086 +++ b/tests/qemu-iotests/086 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qemu-img progress output # diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/087 +++ b/tests/qemu-iotests/087 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test unsupported blockdev-add cases # diff --git a/tests/qemu-iotests/088 b/tests/qemu-iotests/088 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/088 +++ b/tests/qemu-iotests/088 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # vpc (VHD) format input validation tests # diff --git a/tests/qemu-iotests/089 b/tests/qemu-iotests/089 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/089 +++ b/tests/qemu-iotests/089 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for support of JSON filenames # diff --git a/tests/qemu-iotests/090 b/tests/qemu-iotests/090 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/090 +++ b/tests/qemu-iotests/090 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test for discarding compressed clusters on qcow2 images # diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/091 +++ b/tests/qemu-iotests/091 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Live migration test # diff --git a/tests/qemu-iotests/092 b/tests/qemu-iotests/092 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/092 +++ b/tests/qemu-iotests/092 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qcow1 format input validation tests # diff --git a/tests/qemu-iotests/094 b/tests/qemu-iotests/094 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/094 +++ b/tests/qemu-iotests/094 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for drive-mirror to NBD # diff --git a/tests/qemu-iotests/095 b/tests/qemu-iotests/095 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/095 +++ b/tests/qemu-iotests/095 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test for commit of larger active layer # diff --git a/tests/qemu-iotests/097 b/tests/qemu-iotests/097 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/097 +++ b/tests/qemu-iotests/097 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Commit changes into backing chains and empty the top image if the # backing image is not explicitly specified diff --git a/tests/qemu-iotests/098 b/tests/qemu-iotests/098 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/098 +++ b/tests/qemu-iotests/098 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qcow2's bdrv_make_empty for images without internal snapshots # diff --git a/tests/qemu-iotests/099 b/tests/qemu-iotests/099 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/099 +++ b/tests/qemu-iotests/099 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test valid filenames for blkdebug and blkverify representatively for # other protocols (such as NBD) when queried diff --git a/tests/qemu-iotests/101 b/tests/qemu-iotests/101 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/101 +++ b/tests/qemu-iotests/101 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test short file I/O # diff --git a/tests/qemu-iotests/102 b/tests/qemu-iotests/102 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/102 +++ b/tests/qemu-iotests/102 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for qemu-io -c map and qemu-img map # diff --git a/tests/qemu-iotests/103 b/tests/qemu-iotests/103 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/103 +++ b/tests/qemu-iotests/103 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for qcow2 metadata cache size specification # diff --git a/tests/qemu-iotests/104 b/tests/qemu-iotests/104 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/104 +++ b/tests/qemu-iotests/104 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test image creation with aligned and unaligned sizes # diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/105 +++ b/tests/qemu-iotests/105 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Create, read, write big image # diff --git a/tests/qemu-iotests/106 b/tests/qemu-iotests/106 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/106 +++ b/tests/qemu-iotests/106 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test preallocated resize of raw images # diff --git a/tests/qemu-iotests/107 b/tests/qemu-iotests/107 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/107 +++ b/tests/qemu-iotests/107 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Tests updates of the qcow2 L1 table # diff --git a/tests/qemu-iotests/108 b/tests/qemu-iotests/108 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/108 +++ b/tests/qemu-iotests/108 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for repairing qcow2 images which cannot be repaired using # the on-disk refcount structures diff --git a/tests/qemu-iotests/109 b/tests/qemu-iotests/109 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/109 +++ b/tests/qemu-iotests/109 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test writing image headers of other formats into raw images # diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/110 +++ b/tests/qemu-iotests/110 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for relative backing file names in complex BDS trees # diff --git a/tests/qemu-iotests/111 b/tests/qemu-iotests/111 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/111 +++ b/tests/qemu-iotests/111 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for non-existing backing file when creating a qcow2 image # and not specifying the size diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/112 +++ b/tests/qemu-iotests/112 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test cases for different refcount_bits values # diff --git a/tests/qemu-iotests/113 b/tests/qemu-iotests/113 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/113 +++ b/tests/qemu-iotests/113 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for accessing creation options on image formats and # protocols not supporting image creation diff --git a/tests/qemu-iotests/114 b/tests/qemu-iotests/114 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/114 +++ b/tests/qemu-iotests/114 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test invalid backing file format in qcow2 images # diff --git a/tests/qemu-iotests/115 b/tests/qemu-iotests/115 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/115 +++ b/tests/qemu-iotests/115 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for non-self-referential qcow2 refcount blocks # diff --git a/tests/qemu-iotests/116 b/tests/qemu-iotests/116 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/116 +++ b/tests/qemu-iotests/116 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test error code paths for invalid QED images # diff --git a/tests/qemu-iotests/117 b/tests/qemu-iotests/117 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/117 +++ b/tests/qemu-iotests/117 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for shared BDS between backend trees # diff --git a/tests/qemu-iotests/119 b/tests/qemu-iotests/119 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/119 +++ b/tests/qemu-iotests/119 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # NBD test case for overriding BDRV_O_PROTOCOL by explicitly specifying # a driver diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/120 +++ b/tests/qemu-iotests/120 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Non-NBD test cases for overriding BDRV_O_PROTOCOL by explicitly # specifying a driver diff --git a/tests/qemu-iotests/121 b/tests/qemu-iotests/121 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/121 +++ b/tests/qemu-iotests/121 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test cases for qcow2 refcount table growth # diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/122 +++ b/tests/qemu-iotests/122 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test some qemu-img convert cases # diff --git a/tests/qemu-iotests/123 b/tests/qemu-iotests/123 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/123 +++ b/tests/qemu-iotests/123 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for qemu-img convert to NBD # diff --git a/tests/qemu-iotests/125 b/tests/qemu-iotests/125 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/125 +++ b/tests/qemu-iotests/125 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test preallocated growth of qcow2 images # diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/126 +++ b/tests/qemu-iotests/126 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Tests handling of colons in filenames (which may be confused with protocol # prefixes) diff --git a/tests/qemu-iotests/127 b/tests/qemu-iotests/127 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/127 +++ b/tests/qemu-iotests/127 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for mirroring with dataplane # diff --git a/tests/qemu-iotests/128 b/tests/qemu-iotests/128 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/128 +++ b/tests/qemu-iotests/128 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test that opening O_DIRECT succeeds when image file I/O produces EIO # diff --git a/tests/qemu-iotests/130 b/tests/qemu-iotests/130 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/130 +++ b/tests/qemu-iotests/130 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test that temporary backing file overrides (on the command line or in # blockdev-add) don't replace the original path stored in the image during diff --git a/tests/qemu-iotests/131 b/tests/qemu-iotests/131 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/131 +++ b/tests/qemu-iotests/131 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # parallels format validation tests (created by QEMU) # diff --git a/tests/qemu-iotests/133 b/tests/qemu-iotests/133 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/133 +++ b/tests/qemu-iotests/133 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test for reopen # diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/134 +++ b/tests/qemu-iotests/134 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test encrypted read/write using plain bdrv_read/bdrv_write # diff --git a/tests/qemu-iotests/135 b/tests/qemu-iotests/135 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/135 +++ b/tests/qemu-iotests/135 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test VPC open of image with large Max Table Entries value. # diff --git a/tests/qemu-iotests/137 b/tests/qemu-iotests/137 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/137 +++ b/tests/qemu-iotests/137 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qcow2 reopen # diff --git a/tests/qemu-iotests/138 b/tests/qemu-iotests/138 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/138 +++ b/tests/qemu-iotests/138 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # General test case for qcow2's image check # diff --git a/tests/qemu-iotests/140 b/tests/qemu-iotests/140 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/140 +++ b/tests/qemu-iotests/140 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for ejecting a BlockBackend with an NBD server attached to it # diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/141 +++ b/tests/qemu-iotests/141 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for ejecting BDSs with block jobs still running on them # diff --git a/tests/qemu-iotests/142 b/tests/qemu-iotests/142 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/142 +++ b/tests/qemu-iotests/142 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test for configuring cache modes of arbitrary nodes (requires O_DIRECT) # diff --git a/tests/qemu-iotests/143 b/tests/qemu-iotests/143 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/143 +++ b/tests/qemu-iotests/143 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for connecting to a non-existing NBD export name # diff --git a/tests/qemu-iotests/144 b/tests/qemu-iotests/144 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/144 +++ b/tests/qemu-iotests/144 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # Check live snapshot, followed by active commit, and another snapshot. # # This test is to catch the error case of BZ #1300209: diff --git a/tests/qemu-iotests/145 b/tests/qemu-iotests/145 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/145 +++ b/tests/qemu-iotests/145 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test the combination of -incoming and snapshot=on # diff --git a/tests/qemu-iotests/146 b/tests/qemu-iotests/146 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/146 +++ b/tests/qemu-iotests/146 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test VHD image format creator detection and override # diff --git a/tests/qemu-iotests/150 b/tests/qemu-iotests/150 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/150 +++ b/tests/qemu-iotests/150 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test that qemu-img convert -S 0 fully allocates the target image # diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/153 +++ b/tests/qemu-iotests/153 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test image locking # diff --git a/tests/qemu-iotests/154 b/tests/qemu-iotests/154 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/154 +++ b/tests/qemu-iotests/154 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qcow2 specific bdrv_pwrite_zeroes tests with backing files (complements 034) # diff --git a/tests/qemu-iotests/156 b/tests/qemu-iotests/156 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/156 +++ b/tests/qemu-iotests/156 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Tests oVirt-like storage migration: # - Create snapshot diff --git a/tests/qemu-iotests/157 b/tests/qemu-iotests/157 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/157 +++ b/tests/qemu-iotests/157 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test command line configuration of block devices with qdev # diff --git a/tests/qemu-iotests/158 b/tests/qemu-iotests/158 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/158 +++ b/tests/qemu-iotests/158 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test encrypted read/write using backing files # diff --git a/tests/qemu-iotests/159 b/tests/qemu-iotests/159 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/159 +++ b/tests/qemu-iotests/159 @@ -XXX,XX +XXX,XX @@ -#! /bin/bash +#!/usr/bin/env bash # # qemu-img dd test with different block sizes # diff --git a/tests/qemu-iotests/160 b/tests/qemu-iotests/160 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/160 +++ b/tests/qemu-iotests/160 @@ -XXX,XX +XXX,XX @@ -#! /bin/bash +#!/usr/bin/env bash # # qemu-img dd test for the skip option # diff --git a/tests/qemu-iotests/161 b/tests/qemu-iotests/161 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/161 +++ b/tests/qemu-iotests/161 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test reopening a backing image after block-stream and block-commit # diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/162 +++ b/tests/qemu-iotests/162 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for specifying runtime options of the wrong type to some # block drivers diff --git a/tests/qemu-iotests/170 b/tests/qemu-iotests/170 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/170 +++ b/tests/qemu-iotests/170 @@ -XXX,XX +XXX,XX @@ -#! /bin/bash +#!/usr/bin/env bash # # qemu-img dd test # diff --git a/tests/qemu-iotests/171 b/tests/qemu-iotests/171 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/171 +++ b/tests/qemu-iotests/171 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test 'offset' and 'size' options of the raw driver. Make sure we can't # (or can) read and write outside of the image size. diff --git a/tests/qemu-iotests/172 b/tests/qemu-iotests/172 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/172 +++ b/tests/qemu-iotests/172 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test floppy configuration # diff --git a/tests/qemu-iotests/173 b/tests/qemu-iotests/173 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/173 +++ b/tests/qemu-iotests/173 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test QAPI commands looking up protocol based images with relative # filename backing strings diff --git a/tests/qemu-iotests/174 b/tests/qemu-iotests/174 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/174 +++ b/tests/qemu-iotests/174 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test that qemu-io fail with non-zero exit code # diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/175 +++ b/tests/qemu-iotests/175 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test creating raw image preallocation mode # diff --git a/tests/qemu-iotests/176 b/tests/qemu-iotests/176 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/176 +++ b/tests/qemu-iotests/176 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Commit changes into backing chains and empty the top image if the # backing image is not explicitly specified. diff --git a/tests/qemu-iotests/177 b/tests/qemu-iotests/177 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/177 +++ b/tests/qemu-iotests/177 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test corner cases with unusual block geometries # diff --git a/tests/qemu-iotests/178 b/tests/qemu-iotests/178 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/178 +++ b/tests/qemu-iotests/178 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qemu-img measure sub-command tests # diff --git a/tests/qemu-iotests/179 b/tests/qemu-iotests/179 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/179 +++ b/tests/qemu-iotests/179 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for write zeroes with unmap # diff --git a/tests/qemu-iotests/181 b/tests/qemu-iotests/181 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/181 +++ b/tests/qemu-iotests/181 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test postcopy live migration with shared storage # diff --git a/tests/qemu-iotests/182 b/tests/qemu-iotests/182 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/182 +++ b/tests/qemu-iotests/182 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test image locking for POSIX locks # diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/183 +++ b/tests/qemu-iotests/183 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test old-style block migration (migrate -b) # diff --git a/tests/qemu-iotests/184 b/tests/qemu-iotests/184 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/184 +++ b/tests/qemu-iotests/184 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test I/O throttle block filter driver interface # diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/185 +++ b/tests/qemu-iotests/185 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test exiting qemu while jobs are still running # diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/186 +++ b/tests/qemu-iotests/186 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test 'info block' with all kinds of configurations # diff --git a/tests/qemu-iotests/187 b/tests/qemu-iotests/187 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/187 +++ b/tests/qemu-iotests/187 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test switching between read-only and read-write # diff --git a/tests/qemu-iotests/188 b/tests/qemu-iotests/188 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/188 +++ b/tests/qemu-iotests/188 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test encrypted read/write using plain bdrv_read/bdrv_write # diff --git a/tests/qemu-iotests/189 b/tests/qemu-iotests/189 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/189 +++ b/tests/qemu-iotests/189 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test encrypted read/write using backing files # diff --git a/tests/qemu-iotests/190 b/tests/qemu-iotests/190 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/190 +++ b/tests/qemu-iotests/190 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # qemu-img measure sub-command tests on huge qcow2 files # diff --git a/tests/qemu-iotests/191 b/tests/qemu-iotests/191 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/191 +++ b/tests/qemu-iotests/191 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test commit block job where top has two parents # diff --git a/tests/qemu-iotests/192 b/tests/qemu-iotests/192 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/192 +++ b/tests/qemu-iotests/192 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test NBD export with -incoming (non-shared storage migration use case from # libvirt) diff --git a/tests/qemu-iotests/195 b/tests/qemu-iotests/195 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/195 +++ b/tests/qemu-iotests/195 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test change-backing-file command # diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/197 +++ b/tests/qemu-iotests/197 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for copy-on-read into qcow2 # diff --git a/tests/qemu-iotests/198 b/tests/qemu-iotests/198 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/198 +++ b/tests/qemu-iotests/198 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test commit of encrypted qcow2 files # diff --git a/tests/qemu-iotests/200 b/tests/qemu-iotests/200 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/200 +++ b/tests/qemu-iotests/200 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Block job co-routine race condition test. # diff --git a/tests/qemu-iotests/201 b/tests/qemu-iotests/201 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/201 +++ b/tests/qemu-iotests/201 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test savevm and loadvm after live migration with postcopy flag # diff --git a/tests/qemu-iotests/204 b/tests/qemu-iotests/204 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/204 +++ b/tests/qemu-iotests/204 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test corner cases with unusual block geometries # diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/214 +++ b/tests/qemu-iotests/214 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qcow2 image compression # diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/215 +++ b/tests/qemu-iotests/215 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test case for copy-on-read into qcow2, using the COR filter driver # diff --git a/tests/qemu-iotests/217 b/tests/qemu-iotests/217 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/217 +++ b/tests/qemu-iotests/217 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # I/O errors when working with internal qcow2 snapshots, and repairing # the result diff --git a/tests/qemu-iotests/220 b/tests/qemu-iotests/220 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/220 +++ b/tests/qemu-iotests/220 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # max limits on compression in huge qcow2 files # diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/221 +++ b/tests/qemu-iotests/221 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test qemu-img vs. unaligned images # diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/223 +++ b/tests/qemu-iotests/223 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test reading dirty bitmap over NBD # diff --git a/tests/qemu-iotests/225 b/tests/qemu-iotests/225 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/225 +++ b/tests/qemu-iotests/225 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test vmdk backing file correlation # diff --git a/tests/qemu-iotests/226 b/tests/qemu-iotests/226 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/226 +++ b/tests/qemu-iotests/226 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # This test covers expected filetypes for the file, host_cdrom and # host_device drivers. diff --git a/tests/qemu-iotests/227 b/tests/qemu-iotests/227 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/227 +++ b/tests/qemu-iotests/227 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test query-blockstats with different ways to create a BB # diff --git a/tests/qemu-iotests/229 b/tests/qemu-iotests/229 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/229 +++ b/tests/qemu-iotests/229 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test for force canceling a running blockjob that is paused in # an error state. diff --git a/tests/qemu-iotests/231 b/tests/qemu-iotests/231 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/231 +++ b/tests/qemu-iotests/231 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test legacy and modern option parsing for rbd/ceph. This will not # actually connect to a ceph server, but rather looks for the appropriate diff --git a/tests/qemu-iotests/232 b/tests/qemu-iotests/232 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/232 +++ b/tests/qemu-iotests/232 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test for auto-read-only # diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/233 +++ b/tests/qemu-iotests/233 @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Test NBD TLS certificate / authorization integration # diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Copyright (C) 2009 Red Hat, Inc. # Copyright (c) 2000-2002,2006 Silicon Graphics, Inc. All Rights Reserved. diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.config +++ b/tests/qemu-iotests/common.config @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Copyright (C) 2009 Red Hat, Inc. # Copyright (c) 2000-2003,2006 Silicon Graphics, Inc. All Rights Reserved. diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Copyright (C) 2009 Red Hat, Inc. # Copyright (c) 2000-2001 Silicon Graphics, Inc. All Rights Reserved. diff --git a/tests/qemu-iotests/common.nbd b/tests/qemu-iotests/common.nbd index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.nbd +++ b/tests/qemu-iotests/common.nbd @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # -*- shell-script-mode -*- # # Helpers for NBD server related config diff --git a/tests/qemu-iotests/common.pattern b/tests/qemu-iotests/common.pattern index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.pattern +++ b/tests/qemu-iotests/common.pattern @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Copyright (C) 2009 Red Hat, Inc. # diff --git a/tests/qemu-iotests/common.qemu b/tests/qemu-iotests/common.qemu index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.qemu +++ b/tests/qemu-iotests/common.qemu @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # This allows for launching of multiple QEMU instances, with independent # communication possible to each instance. diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Copyright (C) 2009 Red Hat, Inc. # Copyright (c) 2000-2006 Silicon Graphics, Inc. All Rights Reserved. diff --git a/tests/qemu-iotests/common.tls b/tests/qemu-iotests/common.tls index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.tls +++ b/tests/qemu-iotests/common.tls @@ -XXX,XX +XXX,XX @@ -#!/bin/bash +#!/usr/bin/env bash # # Helpers for TLS related config # -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Various sed regexp from common.filter use sed GNU extensions. Instead of spending time to write these regex to be POSIX compliant, verify the GNU sed is available and use it. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/common.filter | 36 ++++++++++++++++---------------- tests/qemu-iotests/common.rc | 13 ++++++++++++ 2 files changed, 31 insertions(+), 18 deletions(-) diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -XXX,XX +XXX,XX @@ # _filter_date() { - sed \ + $SED \ -e 's/[A-Z][a-z][a-z] [A-z][a-z][a-z] *[0-9][0-9]* [0-9][0-9]:[0-9][0-9]:[0-9][0-9] [0-9][0-9][0-9][0-9]$/DATE/' } _filter_generated_node_ids() { - sed -re 's/\#block[0-9]{3,}/NODE_NAME/' + $SED -re 's/\#block[0-9]{3,}/NODE_NAME/' } _filter_qom_path() { - sed -e 's#\(Attached to: *\) /.*#\1 PATH#' + $SED -e 's#\(Attached to: *\) /.*#\1 PATH#' } # replace occurrences of the actual TEST_DIR value with TEST_DIR _filter_testdir() { - sed -e "s#$TEST_DIR/#TEST_DIR/#g" + $SED -e "s#$TEST_DIR/#TEST_DIR/#g" } # replace occurrences of the actual IMGFMT value with IMGFMT _filter_imgfmt() { - sed -e "s#$IMGFMT#IMGFMT#g" + $SED -e "s#$IMGFMT#IMGFMT#g" } # Replace error message when the format is not supported and delete # the output lines after the first one _filter_qemu_img_check() { - sed -e '/allocated.*fragmented.*compressed clusters/d' \ + $SED -e '/allocated.*fragmented.*compressed clusters/d' \ -e 's/qemu-img: This image format does not support checks/No errors were found on the image./' \ -e '/Image end offset: [0-9]\+/d' } @@ -XXX,XX +XXX,XX @@ _filter_qemu_img_check() # Removes \r from messages _filter_win32() { - sed -e 's/\r//g' + $SED -e 's/\r//g' } # sanitize qemu-io output _filter_qemu_io() { - _filter_win32 | sed -e "s/[0-9]* ops\; [0-9/:. sec]* ([0-9/.inf]* [EPTGMKiBbytes]*\/sec and [0-9/.inf]* ops\/sec)/X ops\; XX:XX:XX.X (XXX YYY\/sec and XXX ops\/sec)/" \ + _filter_win32 | $SED -e "s/[0-9]* ops\; [0-9/:. sec]* ([0-9/.inf]* [EPTGMKiBbytes]*\/sec and [0-9/.inf]* ops\/sec)/X ops\; XX:XX:XX.X (XXX YYY\/sec and XXX ops\/sec)/" \ -e "s/: line [0-9][0-9]*: *[0-9][0-9]*\( Aborted\| Killed\)/:\1/" \ -e "s/qemu-io> //g" } @@ -XXX,XX +XXX,XX @@ _filter_qemu_io() # replace occurrences of QEMU_PROG with "qemu" _filter_qemu() { - sed -e "s#\\(^\\|(qemu) \\)$(basename $QEMU_PROG):#\1QEMU_PROG:#" \ + $SED -e "s#\\(^\\|(qemu) \\)$(basename $QEMU_PROG):#\1QEMU_PROG:#" \ -e 's#^QEMU [0-9]\+\.[0-9]\+\.[0-9]\+ monitor#QEMU X.Y.Z monitor#' \ -e $'s#\r##' # QEMU monitor uses \r\n line endings } @@ -XXX,XX +XXX,XX @@ _filter_qemu() _filter_qmp() { _filter_win32 | \ - sed -e 's#\("\(micro\)\?seconds": \)[0-9]\+#\1 TIMESTAMP#g' \ + $SED -e 's#\("\(micro\)\?seconds": \)[0-9]\+#\1 TIMESTAMP#g' \ -e 's#^{"QMP":.*}$#QMP_VERSION#' \ -e '/^ "QMP": {\s*$/, /^ }\s*$/ c\' \ -e ' QMP_VERSION' @@ -XXX,XX +XXX,XX @@ _filter_qmp() # readline makes HMP command strings so long that git complains _filter_hmp() { - sed -e $'s/^\\((qemu) \\)\\?.*\e\\[D/\\1/g' \ + $SED -e $'s/^\\((qemu) \\)\\?.*\e\\[D/\\1/g' \ -e $'s/\e\\[K//g' } # replace block job offset _filter_block_job_offset() { - sed -e 's/, "offset": [0-9]\+,/, "offset": OFFSET,/' + $SED -e 's/, "offset": [0-9]\+,/, "offset": OFFSET,/' } # replace block job len _filter_block_job_len() { - sed -e 's/, "len": [0-9]\+,/, "len": LEN,/g' + $SED -e 's/, "len": [0-9]\+,/, "len": LEN,/g' } # replace actual image size (depends on the host filesystem) _filter_actual_image_size() { - sed -s 's/\("actual-size":\s*\)[0-9]\+/\1SIZE/g' + $SED -s 's/\("actual-size":\s*\)[0-9]\+/\1SIZE/g' } # replace driver-specific options in the "Formatting..." line _filter_img_create() { - sed -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ + $SED -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ -e "s#$IMGFMT#IMGFMT#g" \ @@ -XXX,XX +XXX,XX @@ _filter_img_info() discard=0 regex_json_spec_start='^ *"format-specific": \{' - sed -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ + $SED -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ -e "s#$IMGFMT#IMGFMT#g" \ @@ -XXX,XX +XXX,XX @@ _filter_img_info() # human and json output _filter_qemu_img_map() { - sed -e 's/\([0-9a-fx]* *[0-9a-fx]* *\)[0-9a-fx]* */\1/g' \ + $SED -e 's/\([0-9a-fx]* *[0-9a-fx]* *\)[0-9a-fx]* */\1/g' \ -e 's/"offset": [0-9]\+/"offset": OFFSET/g' \ -e 's/Mapped to *//' | _filter_testdir | _filter_imgfmt } @@ -XXX,XX +XXX,XX @@ _filter_nbd() # receive callbacks sometimes, making them unreliable. # # Filter out the TCP port number since this changes between runs. - sed -e '/nbd\/.*\.c:/d' \ + $SED -e '/nbd\/.*\.c:/d' \ -e 's#127\.0\.0\.1:[0-9]*#127.0.0.1:PORT#g' \ -e "s#?socket=$TEST_DIR#?socket=TEST_DIR#g" \ -e 's#\(foo\|PORT/\?\|.sock\): Failed to .*$#\1#' diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -XXX,XX +XXX,XX @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. # +SED= +for sed in sed gsed; do + ($sed --version | grep 'GNU sed') > /dev/null 2>&1 + if [ "$?" -eq 0 ]; then + SED=$sed + break + fi +done +if [ -z "$SED" ]; then + echo "$0: GNU sed not found" + exit 1 +fi + dd() { if [ "$HOSTOS" == "Linux" ] -- 2.20.1 Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> --- tests/qemu-iotests/243 | 63 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/243.out | 26 ++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 90 insertions(+) create mode 100755 tests/qemu-iotests/243 create mode 100644 tests/qemu-iotests/243.out diff --git a/tests/qemu-iotests/243 b/tests/qemu-iotests/243 new file mode 100755 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/243 @@ -XXX,XX +XXX,XX @@ +#!/bin/bash +# +# Test qcow2 preallocation +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator +owner=kwolf@redhat.com + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +for mode in off metadata falloc full; do + + echo + echo "=== preallocation=$mode ===" + echo + + IMGOPTS="preallocation=$mode" _make_test_img 64M + + printf "File size: " + du -b $TEST_IMG | cut -f1 + + # Can't use precise numbers here because they differ between filesystems + printf "Disk usage: " + [ $(du -B1 $TEST_IMG | cut -f1) -lt 1048576 ] && echo "low" || echo "high" + +done + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/243.out b/tests/qemu-iotests/243.out new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/243.out @@ -XXX,XX +XXX,XX @@ +QA output created by 243 + +=== preallocation=off === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=off +File size: 196616 +Disk usage: low + +=== preallocation=metadata === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=metadata +File size: 67436544 +Disk usage: low + +=== preallocation=falloc === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=falloc +File size: 67436544 +Disk usage: high + +=== preallocation=full === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=full +File size: 67436544 +Disk usage: high +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -XXX,XX +XXX,XX @@ 239 rw auto quick 240 auto quick 242 rw auto quick +243 rw auto quick -- 2.20.1 Image creation already involves a bdrv_co_truncate() call, which allows to specify a preallocation mode. Just pass the right mode there and remove the code that is made redundant by this. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.c | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) goto out; } - if (qcow2_opts->preallocation == PREALLOC_MODE_FULL || - qcow2_opts->preallocation == PREALLOC_MODE_FALLOC) - { - int64_t prealloc_size = - qcow2_calc_prealloc_size(qcow2_opts->size, cluster_size, - refcount_order); - - ret = blk_truncate(blk, prealloc_size, qcow2_opts->preallocation, errp); - if (ret < 0) { - goto out; - } - } - /* Write the header */ QEMU_BUILD_BUG_ON((1 << MIN_CLUSTER_BITS) < sizeof(*header)); header = g_malloc0(cluster_size); @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } /* Okay, now that we have a valid image, let's give it the right size */ - ret = blk_truncate(blk, qcow2_opts->size, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, qcow2_opts->size, qcow2_opts->preallocation, errp); if (ret < 0) { error_prepend(errp, "Could not resize image: "); goto out; @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } } - /* And if we're supposed to preallocate metadata, do that now */ - if (qcow2_opts->preallocation != PREALLOC_MODE_OFF) { - BDRVQcow2State *s = blk_bs(blk)->opaque; - qemu_co_mutex_lock(&s->lock); - ret = preallocate_co(blk_bs(blk), 0, qcow2_opts->size); - qemu_co_mutex_unlock(&s->lock); - - if (ret < 0) { - error_setg_errno(errp, -ret, "Could not preallocate metadata"); - goto out; - } - } - blk_unref(blk); blk = NULL; -- 2.20.1 This adds external data file to the qcow2 spec as a new incompatible feature. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- docs/interop/qcow2.txt | 42 ++++++++++++++++++++++++++++++++++++++---- 1 file changed, 38 insertions(+), 4 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index XXXXXXX..XXXXXXX 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -XXX,XX +XXX,XX @@ in the description of a field. be written to (unless for regaining consistency). - Bits 2-63: Reserved (set to 0) + Bit 2: External data file bit. If this bit is set, an + external data file is used. Guest clusters are + then stored in the external data file. For such + images, clusters in the external data file are + not refcounted. The offset field in the + Standard Cluster Descriptor must match the + guest offset and neither compressed clusters + nor internal snapshots are supported. + + An External Data File Name header extension may + be present if this bit is set. + + Bits 3-63: Reserved (set to 0) 80 - 87: compatible_features Bitmask of compatible features. An implementation can @@ -XXX,XX +XXX,XX @@ in the description of a field. bit is unset, the bitmaps extension data must be considered inconsistent. - Bits 1-63: Reserved (set to 0) + Bit 1: If this bit is set, the external data file can + be read as a consistent standalone raw image + without looking at the qcow2 metadata. + + Setting this bit has a performance impact for + some operations on the image (e.g. writing + zeros requires writing to the data file instead + of only setting the zero flag in the L2 table + entry) and conflicts with backing files. + + This bit may only be set if the External Data + File bit (incompatible feature bit 1) is also + set. + + Bits 2-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count block entry (width @@ -XXX,XX +XXX,XX @@ be stored. Each extension has a structure like the following: 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer + 0x44415441 - External data file name other - Unknown header extension, can be safely ignored @@ -XXX,XX +XXX,XX @@ L2 table entry: This information is only accurate in L2 tables that are reachable from the active L1 table. + With external data files, all guest clusters have an + implicit refcount of 1 (because of the fixed host = guest + mapping for guest cluster offsets), so this bit should be 1 + for all allocated clusters. + Standard Cluster Descriptor: Bit 0: If set to 1, the cluster reads as all zeros. The host @@ -XXX,XX +XXX,XX @@ Standard Cluster Descriptor: 1 - 8: Reserved (set to 0) 9 - 55: Bits 9-55 of host cluster offset. Must be aligned to a - cluster boundary. If the offset is 0, the cluster is - unallocated. + cluster boundary. If the offset is 0 and bit 63 is clear, + the cluster is unallocated. The offset may only be 0 with + bit 63 set (indicating a host cluster offset of 0) when an + external data file is used. 56 - 61: Reserved (set to 0) -- 2.20.1 This adds basic constants, struct fields and helper function for external data file support to the implementation. QCOW2_INCOMPAT_MASK and QCOW2_AUTOCLEAR_MASK are not updated yet so that opening images with an external data file still fails (we don't handle them correctly yet). Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.h | 32 ++++++++++++++++++++++---------- block/qcow2.c | 9 +++++++++ tests/qemu-iotests/031.out | 8 ++++---- tests/qemu-iotests/036.out | 4 ++-- tests/qemu-iotests/061.out | 14 +++++++------- 5 files changed, 44 insertions(+), 23 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ enum { /* Incompatible feature bits */ enum { - QCOW2_INCOMPAT_DIRTY_BITNR = 0, - QCOW2_INCOMPAT_CORRUPT_BITNR = 1, - QCOW2_INCOMPAT_DIRTY = 1 << QCOW2_INCOMPAT_DIRTY_BITNR, - QCOW2_INCOMPAT_CORRUPT = 1 << QCOW2_INCOMPAT_CORRUPT_BITNR, - - QCOW2_INCOMPAT_MASK = QCOW2_INCOMPAT_DIRTY - | QCOW2_INCOMPAT_CORRUPT, + QCOW2_INCOMPAT_DIRTY_BITNR = 0, + QCOW2_INCOMPAT_CORRUPT_BITNR = 1, + QCOW2_INCOMPAT_DATA_FILE_BITNR = 2, + QCOW2_INCOMPAT_DIRTY = 1 << QCOW2_INCOMPAT_DIRTY_BITNR, + QCOW2_INCOMPAT_CORRUPT = 1 << QCOW2_INCOMPAT_CORRUPT_BITNR, + QCOW2_INCOMPAT_DATA_FILE = 1 << QCOW2_INCOMPAT_DATA_FILE_BITNR, + + QCOW2_INCOMPAT_MASK = QCOW2_INCOMPAT_DIRTY + | QCOW2_INCOMPAT_CORRUPT, }; /* Compatible feature bits */ @@ -XXX,XX +XXX,XX @@ enum { /* Autoclear feature bits */ enum { - QCOW2_AUTOCLEAR_BITMAPS_BITNR = 0, - QCOW2_AUTOCLEAR_BITMAPS = 1 << QCOW2_AUTOCLEAR_BITMAPS_BITNR, + QCOW2_AUTOCLEAR_BITMAPS_BITNR = 0, + QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR = 1, + QCOW2_AUTOCLEAR_BITMAPS = 1 << QCOW2_AUTOCLEAR_BITMAPS_BITNR, + QCOW2_AUTOCLEAR_DATA_FILE_RAW = 1 << QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR, - QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS, + QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS, }; enum qcow2_discard_type { @@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcow2State { CoQueue compress_wait_queue; int nb_compress_threads; + + BdrvChild *data_file; } BDRVQcow2State; typedef struct Qcow2COWRegion { @@ -XXX,XX +XXX,XX @@ typedef enum QCow2MetadataOverlap { #define REFT_OFFSET_MASK 0xfffffffffffffe00ULL +static inline bool has_data_file(BlockDriverState *bs) +{ + BDRVQcow2State *s = bs->opaque; + return (s->data_file != bs->file); +} + static inline int64_t start_of_cluster(BDRVQcow2State *s, int64_t offset) { return offset & ~(s->cluster_size - 1); diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ typedef struct { #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 #define QCOW2_EXT_MAGIC_CRYPTO_HEADER 0x0537be77 #define QCOW2_EXT_MAGIC_BITMAPS 0x23852875 +#define QCOW2_EXT_MAGIC_DATA_FILE 0x44415441 static int coroutine_fn qcow2_co_preadv_compressed(BlockDriverState *bs, @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, goto fail; } + /* TODO Open external data file */ + s->data_file = bs->file; + /* qcow2_read_extension may have set up the crypto context * if the crypt method needs a header region, some methods * don't need header extensions, so must check here @@ -XXX,XX +XXX,XX @@ int qcow2_update_header(BlockDriverState *bs) .bit = QCOW2_INCOMPAT_CORRUPT_BITNR, .name = "corrupt bit", }, + { + .type = QCOW2_FEAT_TYPE_INCOMPATIBLE, + .bit = QCOW2_INCOMPAT_DATA_FILE_BITNR, + .name = "external data file", + }, { .type = QCOW2_FEAT_TYPE_COMPATIBLE, .bit = QCOW2_COMPAT_LAZY_REFCOUNTS_BITNR, diff --git a/tests/qemu-iotests/031.out b/tests/qemu-iotests/031.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/031.out +++ b/tests/qemu-iotests/031.out @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> Header extension: @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> Header extension: @@ -XXX,XX +XXX,XX @@ No errors were found on the image. magic 0x514649fb version 3 -backing_file_offset 0x148 +backing_file_offset 0x178 backing_file_size 0x17 cluster_bits 16 size 67108864 @@ -XXX,XX +XXX,XX @@ data 'host_device' Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> Header extension: diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/036.out +++ b/tests/qemu-iotests/036.out @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> *** done diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> magic 0x514649fb @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> magic 0x514649fb @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> ERROR cluster 5 refcount=0 reference=1 @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> magic 0x514649fb @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> read 65536/65536 bytes at offset 44040192 @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> ERROR cluster 5 refcount=0 reference=1 @@ -XXX,XX +XXX,XX @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> read 131072/131072 bytes at offset 0 -- 2.20.1 Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.h | 3 ++- block/qcow2-cluster.c | 37 +++++++++++++++++++------------------ block/qcow2-refcount.c | 10 +++++----- 3 files changed, 26 insertions(+), 24 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ static inline int64_t qcow2_vm_state_offset(BDRVQcow2State *s) return (int64_t)s->l1_vm_state_index << (s->cluster_bits + s->l2_bits); } -static inline QCow2ClusterType qcow2_get_cluster_type(uint64_t l2_entry) +static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs, + uint64_t l2_entry) { if (l2_entry & QCOW_OFLAG_COMPRESSED) { return QCOW2_CLUSTER_COMPRESSED; diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ fail: * as contiguous. (This allows it, for example, to stop at the first compressed * cluster which may require a different handling) */ -static int count_contiguous_clusters(int nb_clusters, int cluster_size, - uint64_t *l2_slice, uint64_t stop_flags) +static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters, + int cluster_size, uint64_t *l2_slice, uint64_t stop_flags) { int i; QCow2ClusterType first_cluster_type; @@ -XXX,XX +XXX,XX @@ static int count_contiguous_clusters(int nb_clusters, int cluster_size, } /* must be allocated */ - first_cluster_type = qcow2_get_cluster_type(first_entry); + first_cluster_type = qcow2_get_cluster_type(bs, first_entry); assert(first_cluster_type == QCOW2_CLUSTER_NORMAL || first_cluster_type == QCOW2_CLUSTER_ZERO_ALLOC); @@ -XXX,XX +XXX,XX @@ static int count_contiguous_clusters(int nb_clusters, int cluster_size, * Checks how many consecutive unallocated clusters in a given L2 * slice have the same cluster type. */ -static int count_contiguous_clusters_unallocated(int nb_clusters, +static int count_contiguous_clusters_unallocated(BlockDriverState *bs, + int nb_clusters, uint64_t *l2_slice, QCow2ClusterType wanted_type) { @@ -XXX,XX +XXX,XX @@ static int count_contiguous_clusters_unallocated(int nb_clusters, wanted_type == QCOW2_CLUSTER_UNALLOCATED); for (i = 0; i < nb_clusters; i++) { uint64_t entry = be64_to_cpu(l2_slice[i]); - QCow2ClusterType type = qcow2_get_cluster_type(entry); + QCow2ClusterType type = qcow2_get_cluster_type(bs, entry); if (type != wanted_type) { break; @@ -XXX,XX +XXX,XX @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, * true */ assert(nb_clusters <= INT_MAX); - type = qcow2_get_cluster_type(*cluster_offset); + type = qcow2_get_cluster_type(bs, *cluster_offset); if (s->qcow_version < 3 && (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC)) { qcow2_signal_corruption(bs, true, -1, -1, "Zero cluster entry found" @@ -XXX,XX +XXX,XX @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, case QCOW2_CLUSTER_ZERO_PLAIN: case QCOW2_CLUSTER_UNALLOCATED: /* how many empty clusters ? */ - c = count_contiguous_clusters_unallocated(nb_clusters, + c = count_contiguous_clusters_unallocated(bs, nb_clusters, &l2_slice[l2_index], type); *cluster_offset = 0; break; case QCOW2_CLUSTER_ZERO_ALLOC: case QCOW2_CLUSTER_NORMAL: /* how many allocated clusters ? */ - c = count_contiguous_clusters(nb_clusters, s->cluster_size, + c = count_contiguous_clusters(bs, nb_clusters, s->cluster_size, &l2_slice[l2_index], QCOW_OFLAG_ZERO); *cluster_offset &= L2E_OFFSET_MASK; if (offset_into_cluster(s, *cluster_offset)) { @@ -XXX,XX +XXX,XX @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m) * write, but require COW to be performed (this includes yet unallocated space, * which must copy from the backing file) */ -static int count_cow_clusters(BDRVQcow2State *s, int nb_clusters, +static int count_cow_clusters(BlockDriverState *bs, int nb_clusters, uint64_t *l2_slice, int l2_index) { int i; for (i = 0; i < nb_clusters; i++) { uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index + i]); - QCow2ClusterType cluster_type = qcow2_get_cluster_type(l2_entry); + QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry); switch(cluster_type) { case QCOW2_CLUSTER_NORMAL: @@ -XXX,XX +XXX,XX @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, cluster_offset = be64_to_cpu(l2_slice[l2_index]); /* Check how many clusters are already allocated and don't need COW */ - if (qcow2_get_cluster_type(cluster_offset) == QCOW2_CLUSTER_NORMAL + if (qcow2_get_cluster_type(bs, cluster_offset) == QCOW2_CLUSTER_NORMAL && (cluster_offset & QCOW_OFLAG_COPIED)) { /* If a specific host_offset is required, check it */ @@ -XXX,XX +XXX,XX @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, /* We keep all QCOW_OFLAG_COPIED clusters */ keep_clusters = - count_contiguous_clusters(nb_clusters, s->cluster_size, + count_contiguous_clusters(bs, nb_clusters, s->cluster_size, &l2_slice[l2_index], QCOW_OFLAG_COPIED | QCOW_OFLAG_ZERO); assert(keep_clusters <= nb_clusters); @@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, if (entry & QCOW_OFLAG_COMPRESSED) { nb_clusters = 1; } else { - nb_clusters = count_cow_clusters(s, nb_clusters, l2_slice, l2_index); + nb_clusters = count_cow_clusters(bs, nb_clusters, l2_slice, l2_index); } /* This function is only called when there were no non-COW clusters, so if @@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, * wrong with our code. */ assert(nb_clusters > 0); - if (qcow2_get_cluster_type(entry) == QCOW2_CLUSTER_ZERO_ALLOC && + if (qcow2_get_cluster_type(bs, entry) == QCOW2_CLUSTER_ZERO_ALLOC && (entry & QCOW_OFLAG_COPIED) && (!*host_offset || start_of_cluster(s, *host_offset) == (entry & L2E_OFFSET_MASK))) @@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, * would be fine, too, but count_cow_clusters() above has limited * nb_clusters already to a range of COW clusters */ preallocated_nb_clusters = - count_contiguous_clusters(nb_clusters, s->cluster_size, + count_contiguous_clusters(bs, nb_clusters, s->cluster_size, &l2_slice[l2_index], QCOW_OFLAG_COPIED); assert(preallocated_nb_clusters > 0); @@ -XXX,XX +XXX,XX @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset, * If full_discard is true, the sector should not read back as zeroes, * but rather fall through to the backing file. */ - switch (qcow2_get_cluster_type(old_l2_entry)) { + switch (qcow2_get_cluster_type(bs, old_l2_entry)) { case QCOW2_CLUSTER_UNALLOCATED: if (full_discard || !bs->backing) { continue; @@ -XXX,XX +XXX,XX @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset, * Minimize L2 changes if the cluster already reads back as * zeroes with correct allocation. */ - cluster_type = qcow2_get_cluster_type(old_offset); + cluster_type = qcow2_get_cluster_type(bs, old_offset); if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN || (cluster_type == QCOW2_CLUSTER_ZERO_ALLOC && !unmap)) { continue; @@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, uint64_t l2_entry = be64_to_cpu(l2_slice[j]); int64_t offset = l2_entry & L2E_OFFSET_MASK; QCow2ClusterType cluster_type = - qcow2_get_cluster_type(l2_entry); + qcow2_get_cluster_type(bs, l2_entry); if (cluster_type != QCOW2_CLUSTER_ZERO_PLAIN && cluster_type != QCOW2_CLUSTER_ZERO_ALLOC) { diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -XXX,XX +XXX,XX @@ void qcow2_free_any_clusters(BlockDriverState *bs, uint64_t l2_entry, { BDRVQcow2State *s = bs->opaque; - switch (qcow2_get_cluster_type(l2_entry)) { + switch (qcow2_get_cluster_type(bs, l2_entry)) { case QCOW2_CLUSTER_COMPRESSED: { int nb_csectors; @@ -XXX,XX +XXX,XX @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs, entry &= ~QCOW_OFLAG_COPIED; offset = entry & L2E_OFFSET_MASK; - switch (qcow2_get_cluster_type(entry)) { + switch (qcow2_get_cluster_type(bs, entry)) { case QCOW2_CLUSTER_COMPRESSED: nb_csectors = ((entry >> s->csize_shift) & s->csize_mask) + 1; @@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, for(i = 0; i < s->l2_size; i++) { l2_entry = be64_to_cpu(l2_table[i]); - switch (qcow2_get_cluster_type(l2_entry)) { + switch (qcow2_get_cluster_type(bs, l2_entry)) { case QCOW2_CLUSTER_COMPRESSED: /* Compressed clusters don't have QCOW_OFLAG_COPIED */ if (l2_entry & QCOW_OFLAG_COPIED) { @@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, /* Correct offsets are cluster aligned */ if (offset_into_cluster(s, offset)) { - if (qcow2_get_cluster_type(l2_entry) == + if (qcow2_get_cluster_type(bs, l2_entry) == QCOW2_CLUSTER_ZERO_ALLOC) { fprintf(stderr, "%s offset=%" PRIx64 ": Preallocated zero " @@ -XXX,XX +XXX,XX @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, for (j = 0; j < s->l2_size; j++) { uint64_t l2_entry = be64_to_cpu(l2_table[j]); uint64_t data_offset = l2_entry & L2E_OFFSET_MASK; - QCow2ClusterType cluster_type = qcow2_get_cluster_type(l2_entry); + QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry); if (cluster_type == QCOW2_CLUSTER_NORMAL || cluster_type == QCOW2_CLUSTER_ZERO_ALLOC) { -- 2.20.1 Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.h | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs, } return QCOW2_CLUSTER_ZERO_PLAIN; } else if (!(l2_entry & L2E_OFFSET_MASK)) { - return QCOW2_CLUSTER_UNALLOCATED; + /* Offset 0 generally means unallocated, but it is ambiguous with + * external data files because 0 is a valid offset there. However, all + * clusters in external data files always have refcount 1, so we can + * rely on QCOW_OFLAG_COPIED to disambiguate. */ + if (has_data_file(bs) && (l2_entry & QCOW_OFLAG_COPIED)) { + return QCOW2_CLUSTER_NORMAL; + } else { + return QCOW2_CLUSTER_UNALLOCATED; + } } else { return QCOW2_CLUSTER_NORMAL; } -- 2.20.1 Offset 0 can be valid for normal (allocated) clusters now, so use qcow2_get_cluster_type() instead. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2-cluster.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters, uint64_t first_entry = be64_to_cpu(l2_slice[0]); uint64_t offset = first_entry & mask; - if (!offset) { + first_cluster_type = qcow2_get_cluster_type(bs, first_entry); + if (first_cluster_type == QCOW2_CLUSTER_UNALLOCATED) { return 0; } /* must be allocated */ - first_cluster_type = qcow2_get_cluster_type(bs, first_entry); assert(first_cluster_type == QCOW2_CLUSTER_NORMAL || first_cluster_type == QCOW2_CLUSTER_ZERO_ALLOC); -- 2.20.1 The cluster allocation code uses 0 as an invalid offset that is used in case of errors or as "offset not yet determined". With external data files, a host cluster offset of 0 becomes valid, though. Define a constant INV_OFFSET (which is not cluster aligned and will therefore never be a valid offset) that can be used for such purposes. This removes the additional host_offset == 0 check that commit ff52aab2df5 introduced; the confusion between an invalid offset and (erroneous) allocation at offset 0 is removed with this change. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.h | 2 ++ block/qcow2-cluster.c | 59 ++++++++++++++++++++----------------------- 2 files changed, 29 insertions(+), 32 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ typedef enum QCow2MetadataOverlap { #define REFT_OFFSET_MASK 0xfffffffffffffe00ULL +#define INV_OFFSET (-1ULL) + static inline bool has_data_file(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset, /* * Checks how many already allocated clusters that don't require a copy on - * write there are at the given guest_offset (up to *bytes). If - * *host_offset is not zero, only physically contiguous clusters beginning at - * this host offset are counted. + * write there are at the given guest_offset (up to *bytes). If *host_offset is + * not INV_OFFSET, only physically contiguous clusters beginning at this host + * offset are counted. * * Note that guest_offset may not be cluster aligned. In this case, the * returned *host_offset points to exact byte referenced by guest_offset and @@ -XXX,XX +XXX,XX @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, trace_qcow2_handle_copied(qemu_coroutine_self(), guest_offset, *host_offset, *bytes); - assert(*host_offset == 0 || offset_into_cluster(s, guest_offset) - == offset_into_cluster(s, *host_offset)); + assert(*host_offset == INV_OFFSET || offset_into_cluster(s, guest_offset) + == offset_into_cluster(s, *host_offset)); /* * Calculate the number of clusters to look for. We stop at L2 slice @@ -XXX,XX +XXX,XX @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, goto out; } - if (*host_offset != 0 && !offset_matches) { + if (*host_offset != INV_OFFSET && !offset_matches) { *bytes = 0; ret = 0; goto out; @@ -XXX,XX +XXX,XX @@ out: * contain the number of clusters that have been allocated and are contiguous * in the image file. * - * If *host_offset is non-zero, it specifies the offset in the image file at - * which the new clusters must start. *nb_clusters can be 0 on return in this - * case if the cluster at host_offset is already in use. If *host_offset is - * zero, the clusters can be allocated anywhere in the image file. + * If *host_offset is not INV_OFFSET, it specifies the offset in the image file + * at which the new clusters must start. *nb_clusters can be 0 on return in + * this case if the cluster at host_offset is already in use. If *host_offset + * is INV_OFFSET, the clusters can be allocated anywhere in the image file. * * *host_offset is updated to contain the offset into the image file at which * the first allocated cluster starts. @@ -XXX,XX +XXX,XX @@ static int do_alloc_cluster_offset(BlockDriverState *bs, uint64_t guest_offset, /* Allocate new clusters */ trace_qcow2_cluster_alloc_phys(qemu_coroutine_self()); - if (*host_offset == 0) { + if (*host_offset == INV_OFFSET) { int64_t cluster_offset = qcow2_alloc_clusters(bs, *nb_clusters * s->cluster_size); if (cluster_offset < 0) { @@ -XXX,XX +XXX,XX @@ static int do_alloc_cluster_offset(BlockDriverState *bs, uint64_t guest_offset, /* * Allocates new clusters for an area that either is yet unallocated or needs a - * copy on write. If *host_offset is non-zero, clusters are only allocated if - * the new allocation can match the specified host offset. + * copy on write. If *host_offset is not INV_OFFSET, clusters are only + * allocated if the new allocation can match the specified host offset. * * Note that guest_offset may not be cluster aligned. In this case, the * returned *host_offset points to exact byte referenced by guest_offset and @@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, int ret; bool keep_old_clusters = false; - uint64_t alloc_cluster_offset = 0; + uint64_t alloc_cluster_offset = INV_OFFSET; trace_qcow2_handle_alloc(qemu_coroutine_self(), guest_offset, *host_offset, *bytes); @@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, if (qcow2_get_cluster_type(bs, entry) == QCOW2_CLUSTER_ZERO_ALLOC && (entry & QCOW_OFLAG_COPIED) && - (!*host_offset || + (*host_offset == INV_OFFSET || start_of_cluster(s, *host_offset) == (entry & L2E_OFFSET_MASK))) { int preallocated_nb_clusters; @@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - if (!alloc_cluster_offset) { + if (alloc_cluster_offset == INV_OFFSET) { /* Allocate, if necessary at a given offset in the image file */ - alloc_cluster_offset = start_of_cluster(s, *host_offset); + alloc_cluster_offset = *host_offset == INV_OFFSET ? INV_OFFSET : + start_of_cluster(s, *host_offset); ret = do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_offset, &nb_clusters); if (ret < 0) { @@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, return 0; } - /* !*host_offset would overwrite the image header and is reserved for - * "no host offset preferred". If 0 was a valid host offset, it'd - * trigger the following overlap check; do that now to avoid having an - * invalid value in *host_offset. */ - if (!alloc_cluster_offset) { - ret = qcow2_pre_write_overlap_check(bs, 0, alloc_cluster_offset, - nb_clusters * s->cluster_size); - assert(ret < 0); - goto fail; - } + assert(alloc_cluster_offset != INV_OFFSET); } /* @@ -XXX,XX +XXX,XX @@ int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, again: start = offset; remaining = *bytes; - cluster_offset = 0; - *host_offset = 0; + cluster_offset = INV_OFFSET; + *host_offset = INV_OFFSET; cur_bytes = 0; *m = NULL; while (true) { - if (!*host_offset) { + if (*host_offset == INV_OFFSET && cluster_offset != INV_OFFSET) { *host_offset = start_of_cluster(s, cluster_offset); } @@ -XXX,XX +XXX,XX @@ again: start += cur_bytes; remaining -= cur_bytes; - cluster_offset += cur_bytes; + + if (cluster_offset != INV_OFFSET) { + cluster_offset += cur_bytes; + } if (remaining == 0) { break; @@ -XXX,XX +XXX,XX @@ again: *bytes -= remaining; assert(*bytes > 0); - assert(*host_offset != 0); + assert(*host_offset != INV_OFFSET); return 0; } -- 2.20.1 qcow2_alloc_compressed_cluster_offset() used to return the cluster offset for success and 0 for error. This doesn't only conflict with 0 as a valid host offset, but also loses the error code. Similar to the change made to qcow2_alloc_cluster_offset() for uncompressed clusters in commit 148da7ea9d6, make the function return 0/-errno and return the allocated cluster offset in a by-reference parameter. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.h | 7 ++++--- block/qcow2-cluster.c | 28 +++++++++++++--------------- block/qcow2.c | 19 ++++++++----------- tests/qemu-iotests/220.out | 2 +- 4 files changed, 26 insertions(+), 30 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, QCowL2Meta **m); -uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, - uint64_t offset, - int compressed_size); +int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, + uint64_t offset, + int compressed_size, + uint64_t *host_offset); int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m); void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m); diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ static int get_cluster_table(BlockDriverState *bs, uint64_t offset, /* * alloc_compressed_cluster_offset * - * For a given offset of the disk image, return cluster offset in - * qcow2 file. - * - * If the offset is not found, allocate a new compressed cluster. - * - * Return the cluster offset if successful, - * Return 0, otherwise. + * For a given offset on the virtual disk, allocate a new compressed cluster + * and put the host offset of the cluster into *host_offset. If a cluster is + * already allocated at the offset, return an error. * + * Return 0 on success and -errno in error cases */ - -uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, - uint64_t offset, - int compressed_size) +int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, + uint64_t offset, + int compressed_size, + uint64_t *host_offset) { BDRVQcow2State *s = bs->opaque; int l2_index, ret; @@ -XXX,XX +XXX,XX @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, ret = get_cluster_table(bs, offset, &l2_slice, &l2_index); if (ret < 0) { - return 0; + return ret; } /* Compression can't overwrite anything. Fail if the cluster was already @@ -XXX,XX +XXX,XX @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, cluster_offset = be64_to_cpu(l2_slice[l2_index]); if (cluster_offset & L2E_OFFSET_MASK) { qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - return 0; + return -EIO; } cluster_offset = qcow2_alloc_bytes(bs, compressed_size); if (cluster_offset < 0) { qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - return 0; + return cluster_offset; } nb_csectors = ((cluster_offset + compressed_size - 1) >> 9) - @@ -XXX,XX +XXX,XX @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, l2_slice[l2_index] = cpu_to_be64(cluster_offset); qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - return cluster_offset; + *host_offset = cluster_offset & s->cluster_offset_mask; + return 0; } static int perform_cow(BlockDriverState *bs, QCowL2Meta *m) diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, int ret; size_t out_len; uint8_t *buf, *out_buf; - int64_t cluster_offset; + uint64_t cluster_offset; if (bytes == 0) { /* align end of file to a sector boundary to ease reading with sector based I/Os */ - cluster_offset = bdrv_getlength(bs->file->bs); - if (cluster_offset < 0) { - return cluster_offset; + int64_t len = bdrv_getlength(bs->file->bs); + if (len < 0) { + return len; } - return bdrv_co_truncate(bs->file, cluster_offset, PREALLOC_MODE_OFF, - NULL); + return bdrv_co_truncate(bs->file, len, PREALLOC_MODE_OFF, NULL); } if (offset_into_cluster(s, offset)) { @@ -XXX,XX +XXX,XX @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, } qemu_co_mutex_lock(&s->lock); - cluster_offset = - qcow2_alloc_compressed_cluster_offset(bs, offset, out_len); - if (!cluster_offset) { + ret = qcow2_alloc_compressed_cluster_offset(bs, offset, out_len, + &cluster_offset); + if (ret < 0) { qemu_co_mutex_unlock(&s->lock); - ret = -EIO; goto fail; } - cluster_offset &= s->cluster_offset_mask; ret = qcow2_pre_write_overlap_check(bs, 0, cluster_offset, out_len); qemu_co_mutex_unlock(&s->lock); diff --git a/tests/qemu-iotests/220.out b/tests/qemu-iotests/220.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/220.out +++ b/tests/qemu-iotests/220.out @@ -XXX,XX +XXX,XX @@ wrote 2097152/2097152 bytes at offset 37748736 No errors were found on the image. image size 39845888 == Trying to write compressed cluster == -write failed: Input/output error +write failed: File too large image size 562949957615616 == Writing normal cluster == wrote 2097152/2097152 bytes at offset 0 -- 2.20.1 Offset 0 cannot be assumed to mean an unallocated cluster any more. Instead, the cluster type needs to be checked. *file must refer to the data file instead of the image file if a valid offset is returned from qcow2_co_block_status(). Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, *pnum = bytes; - if (cluster_offset != 0 && ret != QCOW2_CLUSTER_COMPRESSED && + if ((ret == QCOW2_CLUSTER_NORMAL || ret == QCOW2_CLUSTER_ZERO_ALLOC) && !s->crypto) { index_in_cluster = offset & (s->cluster_size - 1); *map = cluster_offset | index_in_cluster; - *file = bs->file->bs; + *file = s->data_file->bs; status |= BDRV_BLOCK_OFFSET_VALID; } if (ret == QCOW2_CLUSTER_ZERO_PLAIN || ret == QCOW2_CLUSTER_ZERO_ALLOC) { -- 2.20.1 This changes the qcow2 implementation to direct all guest data I/O to s->data_file rather than bs->file, while metadata I/O still uses bs->file. At the moment, this is still always the same, but soon we'll add options to set s->data_file to an external data file. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.h | 2 +- block/qcow2-bitmap.c | 7 +++--- block/qcow2-cache.c | 6 ++--- block/qcow2-cluster.c | 46 +++++++++++++++++++++++++++++++------ block/qcow2-refcount.c | 39 +++++++++++++++++++++++-------- block/qcow2-snapshot.c | 7 +++--- block/qcow2.c | 52 +++++++++++++++++++++++++++++++++--------- 7 files changed, 122 insertions(+), 37 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ void qcow2_process_discards(BlockDriverState *bs, int ret); int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset, int64_t size); int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset, - int64_t size); + int64_t size, bool data_file); int qcow2_inc_refcounts_imrt(BlockDriverState *bs, BdrvCheckResult *res, void **refcount_table, int64_t *refcount_table_size, diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ static int bitmap_list_store(BlockDriverState *bs, Qcow2BitmapList *bm_list, * directory in-place (actually, turn-off the extension), which is checked * in qcow2_check_metadata_overlap() */ ret = qcow2_pre_write_overlap_check( - bs, in_place ? QCOW2_OL_BITMAP_DIRECTORY : 0, dir_offset, dir_size); + bs, in_place ? QCOW2_OL_BITMAP_DIRECTORY : 0, dir_offset, dir_size, + false); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ static uint64_t *store_bitmap_data(BlockDriverState *bs, memset(buf + write_size, 0, s->cluster_size - write_size); } - ret = qcow2_pre_write_overlap_check(bs, 0, off, s->cluster_size); + ret = qcow2_pre_write_overlap_check(bs, 0, off, s->cluster_size, false); if (ret < 0) { error_setg_errno(errp, -ret, "Qcow2 overlap check failed"); goto fail; @@ -XXX,XX +XXX,XX @@ static int store_bitmap(BlockDriverState *bs, Qcow2Bitmap *bm, Error **errp) } ret = qcow2_pre_write_overlap_check(bs, 0, tb_offset, - tb_size * sizeof(tb[0])); + tb_size * sizeof(tb[0]), false); if (ret < 0) { error_setg_errno(errp, -ret, "Qcow2 overlap check failed"); goto fail; diff --git a/block/qcow2-cache.c b/block/qcow2-cache.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cache.c +++ b/block/qcow2-cache.c @@ -XXX,XX +XXX,XX @@ static int qcow2_cache_entry_flush(BlockDriverState *bs, Qcow2Cache *c, int i) if (c == s->refcount_block_cache) { ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_REFCOUNT_BLOCK, - c->entries[i].offset, c->table_size); + c->entries[i].offset, c->table_size, false); } else if (c == s->l2_table_cache) { ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L2, - c->entries[i].offset, c->table_size); + c->entries[i].offset, c->table_size, false); } else { ret = qcow2_pre_write_overlap_check(bs, 0, - c->entries[i].offset, c->table_size); + c->entries[i].offset, c->table_size, false); } if (ret < 0) { diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size, /* the L1 position has not yet been updated, so these clusters must * indeed be completely free */ ret = qcow2_pre_write_overlap_check(bs, 0, new_l1_table_offset, - new_l1_size2); + new_l1_size2, false); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ int qcow2_write_l1_entry(BlockDriverState *bs, int l1_index) } ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L1, - s->l1_table_offset + 8 * l1_start_index, sizeof(buf)); + s->l1_table_offset + 8 * l1_start_index, sizeof(buf), false); if (ret < 0) { return ret; } @@ -XXX,XX +XXX,XX @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, unsigned offset_in_cluster, QEMUIOVector *qiov) { + BDRVQcow2State *s = bs->opaque; int ret; if (qiov->size == 0) { @@ -XXX,XX +XXX,XX @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, } ret = qcow2_pre_write_overlap_check(bs, 0, - cluster_offset + offset_in_cluster, qiov->size); + cluster_offset + offset_in_cluster, qiov->size, true); if (ret < 0) { return ret; } BLKDBG_EVENT(bs->file, BLKDBG_COW_WRITE); - ret = bdrv_co_pwritev(bs->file, cluster_offset + offset_in_cluster, + ret = bdrv_co_pwritev(s->data_file, cluster_offset + offset_in_cluster, qiov->size, qiov, 0); if (ret < 0) { return ret; @@ -XXX,XX +XXX,XX @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, } switch (type) { case QCOW2_CLUSTER_COMPRESSED: + if (has_data_file(bs)) { + qcow2_signal_corruption(bs, true, -1, -1, "Compressed cluster " + "entry found in image with external data " + "file (L2 offset: %#" PRIx64 ", L2 index: " + "%#x)", l2_offset, l2_index); + ret = -EIO; + goto fail; + } /* Compressed clusters can only be processed one by one */ c = 1; *cluster_offset &= L2E_COMPRESSED_OFFSET_SIZE_MASK; @@ -XXX,XX +XXX,XX @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, ret = -EIO; goto fail; } + if (has_data_file(bs) && *cluster_offset != offset - offset_in_cluster) + { + qcow2_signal_corruption(bs, true, -1, -1, + "External data file host cluster offset %#" + PRIx64 " does not match guest cluster " + "offset: %#" PRIx64 + ", L2 index: %#x)", *cluster_offset, + offset - offset_in_cluster, l2_index); + ret = -EIO; + goto fail; + } break; default: abort(); @@ -XXX,XX +XXX,XX @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, int64_t cluster_offset; int nb_csectors; + if (has_data_file(bs)) { + return 0; + } + ret = get_cluster_table(bs, offset, &l2_slice, &l2_index); if (ret < 0) { return ret; @@ -XXX,XX +XXX,XX @@ static int do_alloc_cluster_offset(BlockDriverState *bs, uint64_t guest_offset, trace_qcow2_do_alloc_clusters_offset(qemu_coroutine_self(), guest_offset, *host_offset, *nb_clusters); + if (has_data_file(bs)) { + assert(*host_offset == INV_OFFSET || + *host_offset == start_of_cluster(s, guest_offset)); + *host_offset = start_of_cluster(s, guest_offset); + return 0; + } + /* Allocate new clusters */ trace_qcow2_cluster_alloc_phys(qemu_coroutine_self()); if (*host_offset == INV_OFFSET) { @@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, } ret = qcow2_pre_write_overlap_check(bs, 0, offset, - s->cluster_size); + s->cluster_size, true); if (ret < 0) { if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN) { qcow2_free_clusters(bs, offset, s->cluster_size, @@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, goto fail; } - ret = bdrv_pwrite_zeroes(bs->file, offset, s->cluster_size, 0); + ret = bdrv_pwrite_zeroes(s->data_file, offset, + s->cluster_size, 0); if (ret < 0) { if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN) { qcow2_free_clusters(bs, offset, s->cluster_size, @@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, if (l2_dirty) { ret = qcow2_pre_write_overlap_check( bs, QCOW2_OL_INACTIVE_L2 | QCOW2_OL_ACTIVE_L2, - slice_offset, slice_size2); + slice_offset, slice_size2, false); if (ret < 0) { goto fail; } diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -XXX,XX +XXX,XX @@ void qcow2_free_any_clusters(BlockDriverState *bs, uint64_t l2_entry, int nb_clusters, enum qcow2_discard_type type) { BDRVQcow2State *s = bs->opaque; + QCow2ClusterType ctype = qcow2_get_cluster_type(bs, l2_entry); - switch (qcow2_get_cluster_type(bs, l2_entry)) { + if (has_data_file(bs)) { + if (s->discard_passthrough[type] && + (ctype == QCOW2_CLUSTER_NORMAL || + ctype == QCOW2_CLUSTER_ZERO_ALLOC)) + { + bdrv_pdiscard(s->data_file, l2_entry & L2E_OFFSET_MASK, + nb_clusters << s->cluster_bits); + } + return; + } + + switch (ctype) { case QCOW2_CLUSTER_COMPRESSED: { int nb_csectors; @@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, l2_table[i] = cpu_to_be64(l2_entry); ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L2 | QCOW2_OL_INACTIVE_L2, - l2e_offset, sizeof(uint64_t)); + l2e_offset, sizeof(uint64_t), false); if (ret < 0) { fprintf(stderr, "ERROR: Overlap check failed\n"); res->check_errors++; @@ -XXX,XX +XXX,XX @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, if (l2_dirty) { ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L2, - l2_offset, s->cluster_size); + l2_offset, s->cluster_size, + false); if (ret < 0) { fprintf(stderr, "ERROR: Could not write L2 table; metadata " "overlap check failed: %s\n", strerror(-ret)); @@ -XXX,XX +XXX,XX @@ write_refblocks: } ret = qcow2_pre_write_overlap_check(bs, 0, refblock_offset, - s->cluster_size); + s->cluster_size, false); if (ret < 0) { fprintf(stderr, "ERROR writing refblock: %s\n", strerror(-ret)); goto fail; @@ -XXX,XX +XXX,XX @@ write_refblocks: } ret = qcow2_pre_write_overlap_check(bs, 0, reftable_offset, - reftable_size * sizeof(uint64_t)); + reftable_size * sizeof(uint64_t), + false); if (ret < 0) { fprintf(stderr, "ERROR writing reftable: %s\n", strerror(-ret)); goto fail; @@ -XXX,XX +XXX,XX @@ QEMU_BUILD_BUG_ON(QCOW2_OL_MAX_BITNR != ARRAY_SIZE(metadata_ol_names)); * overlaps; or a negative value (-errno) on error. */ int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset, - int64_t size) + int64_t size, bool data_file) { - int ret = qcow2_check_metadata_overlap(bs, ign, offset, size); + int ret; + + if (data_file && has_data_file(bs)) { + return 0; + } + ret = qcow2_check_metadata_overlap(bs, ign, offset, size); if (ret < 0) { return ret; } else if (ret > 0) { @@ -XXX,XX +XXX,XX @@ static int flush_refblock(BlockDriverState *bs, uint64_t **reftable, if (reftable_index < *reftable_size && (*reftable)[reftable_index]) { offset = (*reftable)[reftable_index]; - ret = qcow2_pre_write_overlap_check(bs, 0, offset, s->cluster_size); + ret = qcow2_pre_write_overlap_check(bs, 0, offset, s->cluster_size, + false); if (ret < 0) { error_setg_errno(errp, -ret, "Overlap check failed"); return ret; @@ -XXX,XX +XXX,XX @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order, /* Write the new reftable */ ret = qcow2_pre_write_overlap_check(bs, 0, new_reftable_offset, - new_reftable_size * sizeof(uint64_t)); + new_reftable_size * sizeof(uint64_t), + false); if (ret < 0) { error_setg_errno(errp, -ret, "Overlap check failed"); goto done; diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -XXX,XX +XXX,XX @@ static int qcow2_write_snapshots(BlockDriverState *bs) /* The snapshot list position has not yet been updated, so these clusters * must indeed be completely free */ - ret = qcow2_pre_write_overlap_check(bs, 0, offset, snapshots_size); + ret = qcow2_pre_write_overlap_check(bs, 0, offset, snapshots_size, false); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info) } ret = qcow2_pre_write_overlap_check(bs, 0, sn->l1_table_offset, - s->l1_size * sizeof(uint64_t)); + s->l1_size * sizeof(uint64_t), false); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ int qcow2_snapshot_goto(BlockDriverState *bs, const char *snapshot_id) } ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L1, - s->l1_table_offset, cur_l1_bytes); + s->l1_table_offset, cur_l1_bytes, + false); if (ret < 0) { goto fail; } diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static ssize_t qcow2_crypto_hdr_init_func(QCryptoBlock *block, size_t headerlen, /* Zero fill remaining space in cluster so it has predictable * content in case of future spec changes */ clusterlen = size_to_clusters(s, headerlen) * s->cluster_size; - assert(qcow2_pre_write_overlap_check(bs, 0, ret, clusterlen) == 0); + assert(qcow2_pre_write_overlap_check(bs, 0, ret, clusterlen, false) == 0); ret = bdrv_pwrite_zeroes(bs->file, ret + headerlen, clusterlen - headerlen, 0); @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, */ if (!cluster_data) { cluster_data = - qemu_try_blockalign(bs->file->bs, + qemu_try_blockalign(s->data_file->bs, QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); if (cluster_data == NULL) { @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); qemu_co_mutex_unlock(&s->lock); - ret = bdrv_co_preadv(bs->file, + ret = bdrv_co_preadv(s->data_file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); qemu_co_mutex_lock(&s->lock); @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, } ret = qcow2_pre_write_overlap_check(bs, 0, - cluster_offset + offset_in_cluster, cur_bytes); + cluster_offset + offset_in_cluster, cur_bytes, true); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); trace_qcow2_writev_data(qemu_coroutine_self(), cluster_offset + offset_in_cluster); - ret = bdrv_co_pwritev(bs->file, + ret = bdrv_co_pwritev(s->data_file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); qemu_co_mutex_lock(&s->lock); @@ -XXX,XX +XXX,XX @@ qcow2_co_copy_range_from(BlockDriverState *bs, goto out; case QCOW2_CLUSTER_NORMAL: - child = bs->file; + child = s->data_file; copy_offset += offset_into_cluster(s, src_offset); if ((copy_offset & 511) != 0) { ret = -EIO; @@ -XXX,XX +XXX,XX @@ qcow2_co_copy_range_to(BlockDriverState *bs, assert((cluster_offset & 511) == 0); ret = qcow2_pre_write_overlap_check(bs, 0, - cluster_offset + offset_in_cluster, cur_bytes); + cluster_offset + offset_in_cluster, cur_bytes, true); if (ret < 0) { goto fail; } qemu_co_mutex_unlock(&s->lock); ret = bdrv_co_copy_range_to(src, src_offset, - bs->file, + s->data_file, cluster_offset + offset_in_cluster, cur_bytes, read_flags, write_flags); qemu_co_mutex_lock(&s->lock); @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, int64_t old_file_size, new_file_size; uint64_t nb_new_data_clusters, nb_new_l2_tables; + /* With a data file, preallocation means just allocating the metadata + * and forwarding the truncate request to the data file */ + if (has_data_file(bs)) { + ret = preallocate_co(bs, old_length, offset); + if (ret < 0) { + error_setg_errno(errp, -ret, "Preallocation failed"); + goto fail; + } + break; + } + old_file_size = bdrv_getlength(bs->file->bs); if (old_file_size < 0) { error_setg_errno(errp, -old_file_size, @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, bs->total_sectors = offset / BDRV_SECTOR_SIZE; + if (has_data_file(bs)) { + if (prealloc == PREALLOC_MODE_METADATA) { + prealloc = PREALLOC_MODE_OFF; + } + ret = bdrv_co_truncate(s->data_file, offset, prealloc, errp); + if (ret < 0) { + goto fail; + } + } + /* write updated header.size */ offset = cpu_to_be64(offset); ret = bdrv_pwrite_sync(bs->file, offsetof(QCowHeader, size), @@ -XXX,XX +XXX,XX @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, uint8_t *buf, *out_buf; uint64_t cluster_offset; + if (has_data_file(bs)) { + return -ENOTSUP; + } + if (bytes == 0) { /* align end of file to a sector boundary to ease reading with sector based I/Os */ @@ -XXX,XX +XXX,XX @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, goto fail; } - ret = qcow2_pre_write_overlap_check(bs, 0, cluster_offset, out_len); + ret = qcow2_pre_write_overlap_check(bs, 0, cluster_offset, out_len, true); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { goto fail; @@ -XXX,XX +XXX,XX @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, qemu_iovec_init_buf(&hd_qiov, out_buf, out_len); - BLKDBG_EVENT(bs->file, BLKDBG_WRITE_COMPRESSED); - ret = bdrv_co_pwritev(bs->file, cluster_offset, out_len, &hd_qiov, 0); + BLKDBG_EVENT(s->data_file, BLKDBG_WRITE_COMPRESSED); + ret = bdrv_co_pwritev(s->data_file, cluster_offset, out_len, &hd_qiov, 0); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version, return -ENOTSUP; } + if (has_data_file(bs)) { + error_setg(errp, "Cannot downgrade an image with a data file"); + return -ENOTSUP; + } + /* clear incompatible features */ if (s->incompatible_features & QCOW2_INCOMPAT_DIRTY) { ret = qcow2_mark_clean(bs); -- 2.20.1 Internal snapshots and an external data file are incompatible because snapshots require refcounting and non-linear mapping. Return an error for all of the snapshot operations if an external data file is in use. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2-snapshot.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -XXX,XX +XXX,XX @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info) return -EFBIG; } + if (has_data_file(bs)) { + return -ENOTSUP; + } + memset(sn, 0, sizeof(*sn)); /* Generate an ID */ @@ -XXX,XX +XXX,XX @@ int qcow2_snapshot_goto(BlockDriverState *bs, const char *snapshot_id) int ret; uint64_t *sn_l1_table = NULL; + if (has_data_file(bs)) { + return -ENOTSUP; + } + /* Search the snapshot */ snapshot_index = find_snapshot_by_id_or_name(bs, snapshot_id); if (snapshot_index < 0) { @@ -XXX,XX +XXX,XX @@ int qcow2_snapshot_delete(BlockDriverState *bs, QCowSnapshot sn; int snapshot_index, ret; + if (has_data_file(bs)) { + return -ENOTSUP; + } + /* Search the snapshot */ snapshot_index = find_snapshot_by_id_and_name(bs, snapshot_id, name); if (snapshot_index < 0) { @@ -XXX,XX +XXX,XX @@ int qcow2_snapshot_list(BlockDriverState *bs, QEMUSnapshotInfo **psn_tab) QCowSnapshot *sn; int i; + if (has_data_file(bs)) { + return -ENOTSUP; + } if (!s->nb_snapshots) { *psn_tab = NULL; return s->nb_snapshots; -- 2.20.1 For external data files, data clusters must be excluded from the refcount calculations. Instead, an implicit refcount of 1 is assumed for the COPIED flag. Compressed clusters and internal snapshots are incompatible with external data files, so print an error if they are in use for images with an external data file. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2-refcount.c | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, res->corruptions++; } + if (has_data_file(bs)) { + fprintf(stderr, "ERROR compressed cluster %d with data file, " + "entry=0x%" PRIx64 "\n", i, l2_entry); + res->corruptions++; + break; + } + /* Mark cluster as used */ nb_csectors = ((l2_entry >> s->csize_shift) & s->csize_mask) + 1; @@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, } /* Mark cluster as used */ - ret = qcow2_inc_refcounts_imrt(bs, res, - refcount_table, refcount_table_size, - offset, s->cluster_size); - if (ret < 0) { - goto fail; + if (!has_data_file(bs)) { + ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, + refcount_table_size, + offset, s->cluster_size); + if (ret < 0) { + goto fail; + } } break; } @@ -XXX,XX +XXX,XX @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, if (cluster_type == QCOW2_CLUSTER_NORMAL || cluster_type == QCOW2_CLUSTER_ZERO_ALLOC) { - ret = qcow2_get_refcount(bs, - data_offset >> s->cluster_bits, - &refcount); - if (ret < 0) { - /* don't print message nor increment check_errors */ - continue; + if (has_data_file(bs)) { + refcount = 1; + } else { + ret = qcow2_get_refcount(bs, + data_offset >> s->cluster_bits, + &refcount); + if (ret < 0) { + /* don't print message nor increment check_errors */ + continue; + } } if ((refcount == 1) != ((l2_entry & QCOW_OFLAG_COPIED) != 0)) { fprintf(stderr, "%s OFLAG_COPIED data cluster: " @@ -XXX,XX +XXX,XX @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res, } /* snapshots */ + if (has_data_file(bs) && s->nb_snapshots) { + fprintf(stderr, "ERROR %d snapshots in image with data file\n", + s->nb_snapshots); + res->corruptions++; + } + for (i = 0; i < s->nb_snapshots; i++) { sn = s->snapshots + i; if (offset_into_cluster(s, sn->l1_table_offset)) { -- 2.20.1 This adds a .bdrv_open option to specify the external data file node. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 7 ++++++- block/qcow2.h | 4 +++- block/qcow2.c | 34 ++++++++++++++++++++++++++++++++-- 3 files changed, 41 insertions(+), 4 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # encrypted images, except when doing a metadata-only # probe of the image. (since 2.10) # +# @data-file: reference to or definition of the external data file. +# This may only be specified for images that require an +# external data file. (since 4.0) +# # Since: 2.9 ## { 'struct': 'BlockdevOptionsQcow2', @@ -XXX,XX +XXX,XX @@ '*l2-cache-entry-size': 'int', '*refcount-cache-size': 'int', '*cache-clean-interval': 'int', - '*encrypt': 'BlockdevQcow2Encryption' } } + '*encrypt': 'BlockdevQcow2Encryption', + '*data-file': 'BlockdevRef' } } ## # @SshHostKeyCheckMode: diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ #define DEFAULT_CLUSTER_SIZE 65536 +#define QCOW2_OPT_DATA_FILE "data-file" #define QCOW2_OPT_LAZY_REFCOUNTS "lazy-refcounts" #define QCOW2_OPT_DISCARD_REQUEST "pass-discard-request" #define QCOW2_OPT_DISCARD_SNAPSHOT "pass-discard-snapshot" @@ -XXX,XX +XXX,XX @@ enum { QCOW2_INCOMPAT_DATA_FILE = 1 << QCOW2_INCOMPAT_DATA_FILE_BITNR, QCOW2_INCOMPAT_MASK = QCOW2_INCOMPAT_DIRTY - | QCOW2_INCOMPAT_CORRUPT, + | QCOW2_INCOMPAT_CORRUPT + | QCOW2_INCOMPAT_DATA_FILE, }; /* Compatible feature bits */ diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, goto fail; } - /* TODO Open external data file */ - s->data_file = bs->file; + /* Open external data file */ + s->data_file = bdrv_open_child(NULL, options, "data-file", bs, &child_file, + true, &local_err); + if (local_err) { + error_propagate(errp, local_err); + ret = -EINVAL; + goto fail; + } + + if (s->incompatible_features & QCOW2_INCOMPAT_DATA_FILE) { + if (!s->data_file) { + error_setg(errp, "'data-file' is required for this image"); + ret = -EINVAL; + goto fail; + } + } else { + if (s->data_file) { + error_setg(errp, "'data-file' can only be set for images with an " + "external data file"); + ret = -EINVAL; + goto fail; + } else { + s->data_file = bs->file; + } + } /* qcow2_read_extension may have set up the crypto context * if the crypt method needs a header region, some methods @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, return ret; fail: + if (has_data_file(bs)) { + bdrv_unref_child(bs, s->data_file); + } g_free(s->unknown_header_fields); cleanup_unknown_header_ext(bs); qcow2_free_snapshots(bs); @@ -XXX,XX +XXX,XX @@ static void qcow2_close(BlockDriverState *bs) g_free(s->image_backing_file); g_free(s->image_backing_format); + if (has_data_file(bs)) { + bdrv_unref_child(bs, s->data_file); + } + qcow2_refcount_close(bs); qcow2_free_snapshots(bs); } -- 2.20.1 This adds a .bdrv_create option to use an external data file. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 4 ++++ include/block/block_int.h | 1 + block/qcow2.c | 26 ++++++++++++++++++++++++++ 3 files changed, 31 insertions(+) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # Driver specific image creation options for qcow2. # # @file Node to create the image format on +# @data-file Node to use as an external data file in which all guest +# data is stored so that only metadata remains in the qcow2 +# file (since: 4.0) # @size Size of the virtual disk in bytes # @version Compatibility level (default: v3) # @backing-file File name of the backing file if a backing file @@ -XXX,XX +XXX,XX @@ ## { 'struct': 'BlockdevCreateOptionsQcow2', 'data': { 'file': 'BlockdevRef', + '*data-file': 'BlockdevRef', 'size': 'size', '*version': 'BlockdevQcow2Version', '*backing-file': 'str', diff --git a/include/block/block_int.h b/include/block/block_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -XXX,XX +XXX,XX @@ #define BLOCK_OPT_NOCOW "nocow" #define BLOCK_OPT_OBJECT_SIZE "object_size" #define BLOCK_OPT_REFCOUNT_BITS "refcount_bits" +#define BLOCK_OPT_DATA_FILE "data_file" #define BLOCK_PROBE_BUF_SIZE 512 diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) */ BlockBackend *blk = NULL; BlockDriverState *bs = NULL; + BlockDriverState *data_bs = NULL; QCowHeader *header; size_t cluster_size; int version; @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } refcount_order = ctz32(qcow2_opts->refcount_bits); + if (qcow2_opts->data_file) { + if (version < 3) { + error_setg(errp, "External data files are only supported with " + "compatibility level 1.1 and above (use version=v3 or " + "greater)"); + ret = -EINVAL; + goto out; + } + data_bs = bdrv_open_blockdev_ref(qcow2_opts->data_file, errp); + if (bs == NULL) { + ret = -EIO; + goto out; + } + } /* Create BlockBackend to write to the image */ blk = blk_new(BLK_PERM_WRITE | BLK_PERM_RESIZE, BLK_PERM_ALL); @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) header->compatible_features |= cpu_to_be64(QCOW2_COMPAT_LAZY_REFCOUNTS); } + if (data_bs) { + header->incompatible_features |= + cpu_to_be64(QCOW2_INCOMPAT_DATA_FILE); + } ret = blk_pwrite(blk, 0, header, cluster_size, 0); g_free(header); @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) options = qdict_new(); qdict_put_str(options, "driver", "qcow2"); qdict_put_str(options, "file", bs->node_name); + if (data_bs) { + qdict_put_str(options, "data-file", data_bs->node_name); + } blk = blk_new_open(NULL, NULL, options, BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_NO_FLUSH, &local_err); @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) options = qdict_new(); qdict_put_str(options, "driver", "qcow2"); qdict_put_str(options, "file", bs->node_name); + if (data_bs) { + qdict_put_str(options, "data-file", data_bs->node_name); + } blk = blk_new_open(NULL, NULL, options, BDRV_O_RDWR | BDRV_O_NO_BACKING | BDRV_O_NO_IO, &local_err); @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) out: blk_unref(blk); bdrv_unref(bs); + bdrv_unref(data_bs); return ret; } -- 2.20.1 Rather than requiring that the external data file node is passed explicitly when creating the qcow2 node, store the filename in the designated header extension during .bdrv_create and read it from there as a default during .bdrv_open. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 8 +++- block/qcow2.h | 1 + block/qcow2.c | 94 +++++++++++++++++++++++++++++++++++++- tests/qemu-iotests/082.out | 27 +++++++++++ 4 files changed, 128 insertions(+), 2 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # # @compat: compatibility level # +# @data-file: the filename of the external data file that is stored in the +# image and used as a default for opening the image (since: 4.0) +# # @lazy-refcounts: on or off; only valid for compat >= 1.1 # # @corrupt: true if the image has been marked corrupt; only valid for @@ -XXX,XX +XXX,XX @@ { 'struct': 'ImageInfoSpecificQCow2', 'data': { 'compat': 'str', + '*data-file': 'str', '*lazy-refcounts': 'bool', '*corrupt': 'bool', 'refcount-bits': 'int', @@ -XXX,XX +XXX,XX @@ # # @data-file: reference to or definition of the external data file. # This may only be specified for images that require an -# external data file. (since 4.0) +# external data file. If it is not specified for such +# an image, the data file name is loaded from the image +# file. (since 4.0) # # Since: 2.9 ## diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcow2State { * override) */ char *image_backing_file; char *image_backing_format; + char *image_data_file; CoQueue compress_wait_queue; int nb_compress_threads; diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset, #endif break; + case QCOW2_EXT_MAGIC_DATA_FILE: + { + s->image_data_file = g_malloc0(ext.len + 1); + ret = bdrv_pread(bs->file, offset, s->image_data_file, ext.len); + if (ret < 0) { + error_setg_errno(errp, -ret, + "ERROR: Could not read data file name"); + return ret; + } +#ifdef DEBUG_EXT + printf("Qcow2: Got external data file %s\n", s->image_data_file); +#endif + break; + } + default: /* unknown magic - save it in case we need to rewrite the header */ /* If you add a new feature, make sure to also update the fast @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, } if (s->incompatible_features & QCOW2_INCOMPAT_DATA_FILE) { + if (!s->data_file && s->image_data_file) { + s->data_file = bdrv_open_child(s->image_data_file, options, + "data-file", bs, &child_file, + false, errp); + if (!s->data_file) { + ret = -EINVAL; + goto fail; + } + } if (!s->data_file) { error_setg(errp, "'data-file' is required for this image"); ret = -EINVAL; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, return ret; fail: + g_free(s->image_data_file); if (has_data_file(bs)) { bdrv_unref_child(bs, s->data_file); } @@ -XXX,XX +XXX,XX @@ static void qcow2_close(BlockDriverState *bs) g_free(s->unknown_header_fields); cleanup_unknown_header_ext(bs); + g_free(s->image_data_file); g_free(s->image_backing_file); g_free(s->image_backing_format); @@ -XXX,XX +XXX,XX @@ int qcow2_update_header(BlockDriverState *bs) buflen -= ret; } + /* External data file header extension */ + if (has_data_file(bs) && s->image_data_file) { + ret = header_ext_add(buf, QCOW2_EXT_MAGIC_DATA_FILE, + s->image_data_file, strlen(s->image_data_file), + buflen); + if (ret < 0) { + goto fail; + } + + buf += ret; + buflen -= ret; + } + /* Full disk encryption header pointer extension */ if (s->crypto_header.offset != 0) { s->crypto_header.offset = cpu_to_be64(s->crypto_header.offset); @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) abort(); } + /* Set the external data file if necessary */ + if (data_bs) { + BDRVQcow2State *s = blk_bs(blk)->opaque; + s->image_data_file = g_strdup(data_bs->filename); + } + /* Create a full header (including things like feature table) */ ret = qcow2_update_header(blk_bs(blk)); if (ret < 0) { @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt QDict *qdict; Visitor *v; BlockDriverState *bs = NULL; + BlockDriverState *data_bs = NULL; Error *local_err = NULL; const char *val; int ret; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt goto finish; } + /* Create and open an external data file (protocol layer) */ + val = qdict_get_try_str(qdict, BLOCK_OPT_DATA_FILE); + if (val) { + ret = bdrv_create_file(val, opts, errp); + if (ret < 0) { + goto finish; + } + + data_bs = bdrv_open(val, NULL, NULL, + BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_PROTOCOL, + errp); + if (data_bs == NULL) { + ret = -EIO; + goto finish; + } + + qdict_del(qdict, BLOCK_OPT_DATA_FILE); + qdict_put_str(qdict, "data-file", data_bs->node_name); + } + /* Set 'driver' and 'node' options */ qdict_put_str(qdict, "driver", "qcow2"); qdict_put_str(qdict, "file", bs->node_name); @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt finish: qobject_unref(qdict); bdrv_unref(bs); + bdrv_unref(data_bs); qapi_free_BlockdevCreateOptions(create_options); return ret; } @@ -XXX,XX +XXX,XX @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs, .refcount_bits = s->refcount_bits, .has_bitmaps = !!bitmaps, .bitmaps = bitmaps, + .has_data_file = !!s->image_data_file, + .data_file = g_strdup(s->image_data_file), }; } else { /* if this assertion fails, this probably means a new version was @@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, BDRVQcow2State *s = bs->opaque; int old_version = s->qcow_version, new_version = old_version; uint64_t new_size = 0; - const char *backing_file = NULL, *backing_format = NULL; + const char *backing_file = NULL, *backing_format = NULL, *data_file = NULL; bool lazy_refcounts = s->use_lazy_refcounts; const char *compat = NULL; uint64_t cluster_size = s->cluster_size; @@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, "may not exceed 64 bits"); return -EINVAL; } + } else if (!strcmp(desc->name, BLOCK_OPT_DATA_FILE)) { + data_file = qemu_opt_get(opts, BLOCK_OPT_DATA_FILE); + if (data_file && !has_data_file(bs)) { + error_setg(errp, "data-file can only be set for images that " + "use an external data file"); + return -EINVAL; + } } else { /* if this point is reached, this probably means a new option was * added without having it covered here */ @@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, } } + if (data_file) { + g_free(s->image_data_file); + s->image_data_file = *data_file ? g_strdup(data_file) : NULL; + } + + ret = qcow2_update_header(bs); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to update the image header"); + return ret; + } + if (backing_file || backing_format) { ret = qcow2_change_backing_file(bs, backing_file ?: s->image_backing_file, @@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_create_opts = { .type = QEMU_OPT_STRING, .help = "Image format of the base image" }, + { + .name = BLOCK_OPT_DATA_FILE, + .type = QEMU_OPT_STRING, + .help = "File name of an external data file" + }, { .name = BLOCK_OPT_ENCRYPT, .type = QEMU_OPT_BOOL, diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' -- 2.20.1 Provide an option to force QEMU to always keep the external data file consistent as a standalone read-only raw image. At the moment, this means making sure that write_zeroes requests are forwarded to the data file instead of just updating the metadata, and checking that no backing file is used. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 9 ++++++ block/qcow2.h | 9 +++++- include/block/block_int.h | 1 + block/qcow2-cluster.c | 10 +++++++ block/qcow2.c | 56 ++++++++++++++++++++++++++++++++++++-- tests/qemu-iotests/082.out | 27 ++++++++++++++++++ 6 files changed, 109 insertions(+), 3 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # @data-file: the filename of the external data file that is stored in the # image and used as a default for opening the image (since: 4.0) # +# @data-file-raw: True if the external data file must stay valid as a +# standalone (read-only) raw image without looking at qcow2 +# metadata (since: 4.0) +# # @lazy-refcounts: on or off; only valid for compat >= 1.1 # # @corrupt: true if the image has been marked corrupt; only valid for @@ -XXX,XX +XXX,XX @@ 'data': { 'compat': 'str', '*data-file': 'str', + '*data-file-raw': 'bool', '*lazy-refcounts': 'bool', '*corrupt': 'bool', 'refcount-bits': 'int', @@ -XXX,XX +XXX,XX @@ # @data-file Node to use as an external data file in which all guest # data is stored so that only metadata remains in the qcow2 # file (since: 4.0) +# @data-file-raw True if the external data file must stay valid as a +# standalone (read-only) raw image without looking at qcow2 +# metadata (default: false; since: 4.0) # @size Size of the virtual disk in bytes # @version Compatibility level (default: v3) # @backing-file File name of the backing file if a backing file @@ -XXX,XX +XXX,XX @@ { 'struct': 'BlockdevCreateOptionsQcow2', 'data': { 'file': 'BlockdevRef', '*data-file': 'BlockdevRef', + '*data-file-raw': 'bool', 'size': 'size', '*version': 'BlockdevQcow2Version', '*backing-file': 'str', diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ enum { QCOW2_AUTOCLEAR_BITMAPS = 1 << QCOW2_AUTOCLEAR_BITMAPS_BITNR, QCOW2_AUTOCLEAR_DATA_FILE_RAW = 1 << QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR, - QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS, + QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS + | QCOW2_AUTOCLEAR_DATA_FILE_RAW, }; enum qcow2_discard_type { @@ -XXX,XX +XXX,XX @@ static inline bool has_data_file(BlockDriverState *bs) return (s->data_file != bs->file); } +static inline bool data_file_is_raw(BlockDriverState *bs) +{ + BDRVQcow2State *s = bs->opaque; + return !!(s->autoclear_features & QCOW2_AUTOCLEAR_DATA_FILE_RAW); +} + static inline int64_t start_of_cluster(BDRVQcow2State *s, int64_t offset) { return offset & ~(s->cluster_size - 1); diff --git a/include/block/block_int.h b/include/block/block_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -XXX,XX +XXX,XX @@ #define BLOCK_OPT_OBJECT_SIZE "object_size" #define BLOCK_OPT_REFCOUNT_BITS "refcount_bits" #define BLOCK_OPT_DATA_FILE "data_file" +#define BLOCK_OPT_DATA_FILE_RAW "data_file_raw" #define BLOCK_PROBE_BUF_SIZE 512 diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset, int64_t cleared; int ret; + /* If we have to stay in sync with an external data file, zero out + * s->data_file first. */ + if (data_file_is_raw(bs)) { + assert(has_data_file(bs)); + ret = bdrv_co_pwrite_zeroes(s->data_file, offset, bytes, flags); + if (ret < 0) { + return ret; + } + } + /* Caller must pass aligned values, except at image end */ assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) || diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, "external data file"); ret = -EINVAL; goto fail; - } else { - s->data_file = bs->file; + } + + s->data_file = bs->file; + + if (data_file_is_raw(bs)) { + error_setg(errp, "data-file-raw requires a data file"); + ret = -EINVAL; + goto fail; } } @@ -XXX,XX +XXX,XX @@ static int qcow2_change_backing_file(BlockDriverState *bs, { BDRVQcow2State *s = bs->opaque; + /* Adding a backing file means that the external data file alone won't be + * enough to make sense of the content */ + if (backing_file && data_file_is_raw(bs)) { + return -EINVAL; + } + if (backing_file && strlen(backing_file) > 1023) { return -EINVAL; } @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } refcount_order = ctz32(qcow2_opts->refcount_bits); + if (qcow2_opts->data_file_raw && !qcow2_opts->data_file) { + error_setg(errp, "data-file-raw requires data-file"); + ret = -EINVAL; + goto out; + } + if (qcow2_opts->data_file_raw && qcow2_opts->has_backing_file) { + error_setg(errp, "Backing file and data-file-raw cannot be used at " + "the same time"); + ret = -EINVAL; + goto out; + } + if (qcow2_opts->data_file) { if (version < 3) { error_setg(errp, "External data files are only supported with " @@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) header->incompatible_features |= cpu_to_be64(QCOW2_INCOMPAT_DATA_FILE); } + if (qcow2_opts->data_file_raw) { + header->autoclear_features |= + cpu_to_be64(QCOW2_AUTOCLEAR_DATA_FILE_RAW); + } ret = blk_pwrite(blk, 0, header, cluster_size, 0); g_free(header); @@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt { BLOCK_OPT_REFCOUNT_BITS, "refcount-bits" }, { BLOCK_OPT_ENCRYPT, BLOCK_OPT_ENCRYPT_FORMAT }, { BLOCK_OPT_COMPAT_LEVEL, "version" }, + { BLOCK_OPT_DATA_FILE_RAW, "data-file-raw" }, { NULL, NULL }, }; @@ -XXX,XX +XXX,XX @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs, .bitmaps = bitmaps, .has_data_file = !!s->image_data_file, .data_file = g_strdup(s->image_data_file), + .has_data_file_raw = has_data_file(bs), + .data_file_raw = data_file_is_raw(bs), }; } else { /* if this assertion fails, this probably means a new version was @@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, uint64_t new_size = 0; const char *backing_file = NULL, *backing_format = NULL, *data_file = NULL; bool lazy_refcounts = s->use_lazy_refcounts; + bool data_file_raw = data_file_is_raw(bs); const char *compat = NULL; uint64_t cluster_size = s->cluster_size; bool encrypt; @@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, "use an external data file"); return -EINVAL; } + } else if (!strcmp(desc->name, BLOCK_OPT_DATA_FILE_RAW)) { + data_file_raw = qemu_opt_get_bool(opts, BLOCK_OPT_DATA_FILE_RAW, + data_file_raw); + if (data_file_raw && !data_file_is_raw(bs)) { + error_setg(errp, "data-file-raw cannot be set on existing " + "images"); + return -EINVAL; + } } else { /* if this point is reached, this probably means a new option was * added without having it covered here */ @@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, } } + /* data-file-raw blocks backing files, so clear it first if requested */ + if (data_file_raw) { + s->autoclear_features |= QCOW2_AUTOCLEAR_DATA_FILE_RAW; + } else { + s->autoclear_features &= ~QCOW2_AUTOCLEAR_DATA_FILE_RAW; + } + if (data_file) { g_free(s->image_data_file); s->image_data_file = *data_file ? g_strdup(data_file) : NULL; @@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_create_opts = { .type = QEMU_OPT_STRING, .help = "File name of an external data file" }, + { + .name = BLOCK_OPT_DATA_FILE_RAW, + .type = QEMU_OPT_BOOL, + .help = "The external data file must stay valid as a raw image" + }, { .name = BLOCK_OPT_ENCRYPT, .type = QEMU_OPT_BOOL, diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' @@ -XXX,XX +XXX,XX @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks' -- 2.20.1 Test that preallocating metadata results in a somewhat larger qcow2 file, but preallocating data only affects the disk usage of the data file and the qcow2 file stays small. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/243 | 22 ++++++++++++++++++++++ tests/qemu-iotests/243.out | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/tests/qemu-iotests/243 b/tests/qemu-iotests/243 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/243 +++ b/tests/qemu-iotests/243 @@ -XXX,XX +XXX,XX @@ status=1 # failure is the default! _cleanup() { _cleanup_test_img + rm -f $TEST_IMG.data } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -XXX,XX +XXX,XX @@ for mode in off metadata falloc full; do done +for mode in off metadata falloc full; do + + echo + echo "=== External data file: preallocation=$mode ===" + echo + + IMGOPTS="data_file=$TEST_IMG.data,preallocation=$mode" _make_test_img 64M + + echo -n "qcow2 file size: " + du -b $TEST_IMG | cut -f1 + echo -n "data file size: " + du -b $TEST_IMG.data | cut -f1 + + # Can't use precise numbers here because they differ between filesystems + echo -n "qcow2 disk usage: " + [ $(du -B1 $TEST_IMG | cut -f1) -lt 1048576 ] && echo "low" || echo "high" + echo -n "data disk usage: " + [ $(du -B1 $TEST_IMG.data | cut -f1) -lt 1048576 ] && echo "low" || echo "high" + +done + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/243.out b/tests/qemu-iotests/243.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/243.out +++ b/tests/qemu-iotests/243.out @@ -XXX,XX +XXX,XX @@ Disk usage: high Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=full File size: 67436544 Disk usage: high + +=== External data file: preallocation=off === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=off +qcow2 file size: 196616 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: low + +=== External data file: preallocation=metadata === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=metadata +qcow2 file size: 327680 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: low + +=== External data file: preallocation=falloc === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=falloc +qcow2 file size: 327680 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: high + +=== External data file: preallocation=full === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=full +qcow2 file size: 327680 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: high *** done -- 2.20.1 Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/244 | 200 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/244.out | 125 +++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 326 insertions(+) create mode 100755 tests/qemu-iotests/244 create mode 100644 tests/qemu-iotests/244.out diff --git a/tests/qemu-iotests/244 b/tests/qemu-iotests/244 new file mode 100755 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/244 @@ -XXX,XX +XXX,XX @@ +#!/bin/bash +# +# Test qcow2 with external data files +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator +owner=kwolf@redhat.com + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + rm -f $TEST_IMG.data + rm -f $TEST_IMG.src +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +echo +echo "=== Create and open image with external data file ===" +echo + +echo "With data file name in the image:" +IMGOPTS="data_file=$TEST_IMG.data" _make_test_img 64M +_check_test_img + +$QEMU_IO -c "open $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=$TEST_IMG.data $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=inexistent $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir + +echo +echo "Data file required, but without data file name in the image:" +$QEMU_IMG amend -odata_file= $TEST_IMG + +$QEMU_IO -c "open $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=$TEST_IMG.data $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=inexistent $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir + +echo +echo "Setting data-file for an image with internal data:" +_make_test_img 64M + +$QEMU_IO -c "open -odata-file.filename=$TEST_IMG.data $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=inexistent $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir + +echo +echo "=== Conflicting features ===" +echo + +echo "Convert to compressed target with data file:" +TEST_IMG="$TEST_IMG.src" _make_test_img 64M + +$QEMU_IO -c 'write -P 0x11 0 1M' \ + -f $IMGFMT "$TEST_IMG.src" | + _filter_qemu_io + +$QEMU_IMG convert -f $IMGFMT -O $IMGFMT -c -odata_file="$TEST_IMG.data" \ + "$TEST_IMG.src" "$TEST_IMG" + +echo +echo "Convert uncompressed, then write compressed data manually:" +$QEMU_IMG convert -f $IMGFMT -O $IMGFMT -odata_file="$TEST_IMG.data" \ + "$TEST_IMG.src" "$TEST_IMG" +$QEMU_IMG compare "$TEST_IMG.src" "$TEST_IMG" + +$QEMU_IO -c 'write -c -P 0x22 0 1M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io +_check_test_img + +echo +echo "Take an internal snapshot:" + +$QEMU_IMG snapshot -c test "$TEST_IMG" +_check_test_img + +echo +echo "=== Standalone image with external data file (efficient) ===" +echo + +IMGOPTS="data_file=$TEST_IMG.data" _make_test_img 64M + +echo -n "qcow2 file size before I/O: " +du -b $TEST_IMG | cut -f1 + +# Create image with the following layout +# 0-1 MB: Unallocated +# 1-2 MB: Written (pattern 0x11) +# 2-3 MB: Discarded +# 3-4 MB: Zero write over discarded space +# 4-5 MB: Zero write over written space +# 5-6 MB: Zero write over unallocated space + +echo +$QEMU_IO -c 'write -P 0x11 1M 4M' \ + -c 'discard 2M 2M' \ + -c 'write -z 3M 3M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io +_check_test_img + +echo +$QEMU_IMG map --output=json "$TEST_IMG" + +echo +$QEMU_IO -c 'read -P 0 0 1M' \ + -c 'read -P 0x11 1M 1M' \ + -c 'read -P 0 2M 4M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io + +# Zero clusters are only marked as such in the qcow2 metadata, but contain +# stale data in the external data file +echo +$QEMU_IO -c 'read -P 0 0 1M' \ + -c 'read -P 0x11 1M 1M' \ + -c 'read -P 0 2M 2M' \ + -c 'read -P 0x11 4M 1M' \ + -c 'read -P 0 5M 1M' \ + -f raw "$TEST_IMG.data" | + _filter_qemu_io + + +echo -n "qcow2 file size after I/O: " +du -b $TEST_IMG | cut -f1 + +echo +echo "=== Standalone image with external data file (valid raw) ===" +echo + +IMGOPTS="data_file=$TEST_IMG.data,data_file_raw=on" _make_test_img 64M + +echo -n "qcow2 file size before I/O: " +du -b $TEST_IMG | cut -f1 + +echo +$QEMU_IO -c 'write -P 0x11 1M 4M' \ + -c 'discard 2M 2M' \ + -c 'write -z 3M 3M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io +_check_test_img + +echo +$QEMU_IMG map --output=json "$TEST_IMG" + +echo +$QEMU_IO -c 'read -P 0 0 1M' \ + -c 'read -P 0x11 1M 1M' \ + -c 'read -P 0 2M 4M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io + +echo +$QEMU_IMG compare "$TEST_IMG" "$TEST_IMG.data" + +echo -n "qcow2 file size after I/O: " +du -b $TEST_IMG | cut -f1 + +echo +echo "=== bdrv_co_block_status test for file and offset=0 ===" +echo + +IMGOPTS="data_file=$TEST_IMG.data" _make_test_img 64M + +$QEMU_IO -c 'write -P 0x11 0 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x11 0 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=human "$TEST_IMG" | _filter_testdir +$QEMU_IMG map --output=json "$TEST_IMG" + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/244.out b/tests/qemu-iotests/244.out new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/244.out @@ -XXX,XX +XXX,XX @@ +QA output created by 244 + +=== Create and open image with external data file === + +With data file name in the image: +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +No errors were found on the image. +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +can't open device TEST_DIR/t.qcow2: Could not open 'inexistent': No such file or directory +no file open, try 'help open' + +Data file required, but without data file name in the image: +can't open device TEST_DIR/t.qcow2: 'data-file' is required for this image +no file open, try 'help open' +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +can't open device TEST_DIR/t.qcow2: Could not open 'inexistent': No such file or directory +no file open, try 'help open' + +Setting data-file for an image with internal data: +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +can't open device TEST_DIR/t.qcow2: 'data-file' can only be set for images with an external data file +no file open, try 'help open' +can't open device TEST_DIR/t.qcow2: Could not open 'inexistent': No such file or directory +no file open, try 'help open' + +=== Conflicting features === + +Convert to compressed target with data file: +Formatting 'TEST_DIR/t.IMGFMT.src', fmt=IMGFMT size=67108864 +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qemu-img: error while writing sector 0: Operation not supported + +Convert uncompressed, then write compressed data manually: +Images are identical. +write failed: Operation not supported +No errors were found on the image. + +Take an internal snapshot: +qemu-img: Could not create snapshot 'test': -95 (Operation not supported) +No errors were found on the image. + +=== Standalone image with external data file (efficient) === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +qcow2 file size before I/O: 196616 + +wrote 4194304/4194304 bytes at offset 1048576 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +discard 2097152/2097152 bytes at offset 2097152 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 3145728/3145728 bytes at offset 3145728 +3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +No errors were found on the image. + +[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false}, +{ "start": 1048576, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": 1048576}, +{ "start": 2097152, "length": 2097152, "depth": 0, "zero": true, "data": false}, +{ "start": 4194304, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": 4194304}, +{ "start": 5242880, "length": 61865984, "depth": 0, "zero": true, "data": false}] + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4194304/4194304 bytes at offset 2097152 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2097152/2097152 bytes at offset 2097152 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 4194304 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 5242880 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qcow2 file size after I/O: 327680 + +=== Standalone image with external data file (valid raw) === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data data_file_raw=on +qcow2 file size before I/O: 196616 + +wrote 4194304/4194304 bytes at offset 1048576 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +discard 2097152/2097152 bytes at offset 2097152 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 3145728/3145728 bytes at offset 3145728 +3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +No errors were found on the image. + +[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false}, +{ "start": 1048576, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": 1048576}, +{ "start": 2097152, "length": 2097152, "depth": 0, "zero": true, "data": false}, +{ "start": 4194304, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": 4194304}, +{ "start": 5242880, "length": 61865984, "depth": 0, "zero": true, "data": false}] + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4194304/4194304 bytes at offset 2097152 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +Images are identical. +qcow2 file size after I/O: 327680 + +=== bdrv_co_block_status test for file and offset=0 === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length Mapped to File +0 0x100000 0 TEST_DIR/t.qcow2.data +[{ "start": 0, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": 0}, +{ "start": 1048576, "length": 66060288, "depth": 0, "zero": true, "data": false}] +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -XXX,XX +XXX,XX @@ 240 auto quick 242 rw auto quick 243 rw auto quick +244 rw auto quick -- 2.20.1 Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/061 | 45 ++++++++++++++++++- tests/qemu-iotests/061.out | 89 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 133 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/061 +++ b/tests/qemu-iotests/061 @@ -XXX,XX +XXX,XX @@ status=1 # failure is the default! _cleanup() { - _cleanup_test_img + _cleanup_test_img + rm -f $TEST_IMG.data } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -XXX,XX +XXX,XX @@ $QEMU_IMG snapshot -c foo "$TEST_IMG" $QEMU_IMG amend -p -o "compat=0.10" "$TEST_IMG" _check_test_img +echo +echo "=== Testing version downgrade with external data file ===" +echo +IMGOPTS="compat=1.1,data_file=$TEST_IMG.data" _make_test_img 64M +$QEMU_IMG amend -o "compat=0.10" "$TEST_IMG" +_img_info --format-specific +_check_test_img + +echo +echo "=== Try changing the external data file ===" +echo +IMGOPTS="compat=1.1" _make_test_img 64M +$QEMU_IMG amend -o "data_file=foo" "$TEST_IMG" + +echo +IMGOPTS="compat=1.1,data_file=$TEST_IMG.data" _make_test_img 64M +$QEMU_IMG amend -o "data_file=foo" "$TEST_IMG" +_img_info --format-specific +TEST_IMG="data-file.filename=$TEST_IMG.data,file.filename=$TEST_IMG" _img_info --format-specific --image-opts + +echo +$QEMU_IMG amend -o "data_file=" --image-opts "data-file.filename=$TEST_IMG.data,file.filename=$TEST_IMG" +_img_info --format-specific +TEST_IMG="data-file.filename=$TEST_IMG.data,file.filename=$TEST_IMG" _img_info --format-specific --image-opts + +echo +echo "=== Clearing and setting data-file-raw ===" +echo +IMGOPTS="compat=1.1,data_file=$TEST_IMG.data,data_file_raw=on" _make_test_img 64M +$QEMU_IMG amend -o "data_file_raw=on" "$TEST_IMG" +_img_info --format-specific +_check_test_img + +$QEMU_IMG amend -o "data_file_raw=off" "$TEST_IMG" +_img_info --format-specific +_check_test_img + +$QEMU_IMG amend -o "data_file_raw=on" "$TEST_IMG" +_img_info --format-specific +_check_test_img + + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -XXX,XX +XXX,XX @@ wrote 65536/65536 bytes at offset 3221225472 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) (0.00/100%) (6.25/100%) (12.50/100%) (18.75/100%) (25.00/100%) (31.25/100%) (37.50/100%) (43.75/100%) (50.00/100%) (56.25/100%) (62.50/100%) (68.75/100%) (75.00/100%) (81.25/100%) (87.50/100%) (93.75/100%) (100.00/100%) (100.00/100%) No errors were found on the image. + +=== Testing version downgrade with external data file === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +qemu-img: Cannot downgrade an image with a data file +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: TEST_DIR/t.IMGFMT.data + data file raw: false + corrupt: false +No errors were found on the image. + +=== Try changing the external data file === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +qemu-img: data-file can only be set for images that use an external data file + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Could not open 'foo': No such file or directory +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: foo + data file raw: false + corrupt: false + +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': 'data-file' is required for this image +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file raw: false + corrupt: false + +=== Clearing and setting data-file-raw === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data data_file_raw=on +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: TEST_DIR/t.IMGFMT.data + data file raw: true + corrupt: false +No errors were found on the image. +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: TEST_DIR/t.IMGFMT.data + data file raw: false + corrupt: false +No errors were found on the image. +qemu-img: data-file-raw cannot be set on existing images +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: TEST_DIR/t.IMGFMT.data + data file raw: false + corrupt: false +No errors were found on the image. *** done -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Since the ahci-test uses qemu-img, add a dependency to build it before using it. This fixes: $ gmake check-qtest V=1 QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img tests/ahci-test Failed to execute child process "/tmp/qemu-test.19tMRF/qemu-img" (No such file or directory) ERROR:tests/libqos/libqos.c:192:mkimg: assertion failed: (ret && !err) Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/Makefile.include | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/Makefile.include b/tests/Makefile.include index XXXXXXX..XXXXXXX 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -XXX,XX +XXX,XX @@ tests/prom-env-test$(EXESUF): tests/prom-env-test.o $(libqos-obj-y) tests/rtas-test$(EXESUF): tests/rtas-test.o $(libqos-spapr-obj-y) tests/fdc-test$(EXESUF): tests/fdc-test.o tests/ide-test$(EXESUF): tests/ide-test.o $(libqos-pc-obj-y) -tests/ahci-test$(EXESUF): tests/ahci-test.o $(libqos-pc-obj-y) +tests/ahci-test$(EXESUF): tests/ahci-test.o $(libqos-pc-obj-y) qemu-img$(EXESUF) tests/ipmi-kcs-test$(EXESUF): tests/ipmi-kcs-test.o tests/ipmi-bt-test$(EXESUF): tests/ipmi-bt-test.o tests/hd-geo-test$(EXESUF): tests/hd-geo-test.o -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Since a9660664fde, some iotests use qemu-nbd. Add a dependency to build it before using it. This fixes: $ make check-block GEN qemu-img-cmds.h CC qemu-img.o LINK qemu-img CC qemu-io.o LINK qemu-io CC tests/qemu-iotests/socket_scm_helper.o LINK tests/qemu-iotests/socket_scm_helper tests/qemu-iotests-quick.sh check: qemu-nbd not found make: *** [tests/Makefile.include:1059: check-tests/qemu-iotests-quick.sh] Error 1 Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/Makefile.include | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/Makefile.include b/tests/Makefile.include index XXXXXXX..XXXXXXX 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -XXX,XX +XXX,XX @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES) QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF) .PHONY: check-tests/qemu-iotests-quick.sh -check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) +check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) $< .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y)) -- 2.20.1 Be more specific about the string representation in header extensions. Suggested-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> --- docs/interop/qcow2.txt | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index XXXXXXX..XXXXXXX 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -XXX,XX +XXX,XX @@ be stored. Each extension has a structure like the following: Byte 0 - 3: Header extension type: 0x00000000 - End of the header extension area - 0xE2792ACA - Backing file format name + 0xE2792ACA - Backing file format name string 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer - 0x44415441 - External data file name + 0x44415441 - External data file name string other - Unknown header extension, can be safely ignored @@ -XXX,XX +XXX,XX @@ data of compatible features that it doesn't support. Compatible features that need space for additional data can use a header extension. +== String header extensions == + +Some header extensions (such as the backing file format name and the external +data file name) are just a single string. In this case, the header extension +length is the string length and the string is not '\0' terminated. (The header +extension padding can make it look like a string is '\0' terminated, but +neither is padding always necessary nor is there a guarantee that zero bytes +are used for padding.) + + == Feature name table == The feature name table is an optional header extension that contains the name -- 2.20.1