The following changes since commit b11365867568ba954de667a0bfe0945b8f78d6bd: Merge remote-tracking branch 'remotes/borntraeger/tags/s390x-20170706' into staging (2017-07-06 11:42:59 +0100) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream for you to fetch changes up to 7d982302db173616d011e07ee43be3b0aae872b1: Merge remote-tracking branch 'mreitz/tags/pull-block-2017-07-07' into queue-block (2017-07-07 18:11:41 +0200) ---------------------------------------------------------------- Block layer patches ---------------------------------------------------------------- Daniel P. Berrange (26): qemu-img: drop -e and -6 options from the 'create' & 'convert' commands block: expose crypto option names / defs to other drivers block: add ability to set a prefix for opt names qcow: document another weakness of qcow AES encryption qcow: require image size to be > 1 for new images iotests: skip 042 with qcow which dosn't support zero sized images iotests: skip 048 with qcow which doesn't support resize block: deprecate "encryption=on" in favor of "encrypt.format=aes" qcow: make encrypt_sectors encrypt in place qcow: convert QCow to use QCryptoBlock for encryption qcow2: make qcow2_encrypt_sectors encrypt in place qcow2: convert QCow2 to use QCryptoBlock for encryption qcow2: extend specification to cover LUKS encryption qcow2: add support for LUKS encryption format qcow2: add iotests to cover LUKS encryption support iotests: enable tests 134 and 158 to work with qcow (v1) block: rip out all traces of password prompting block: remove all encryption handling APIs block: pass option prefix down to crypto layer qcow2: report encryption specific image information docs: document encryption options for qcow, qcow2 and luks iotests: skip 159 & 170 with luks format iotests: fix remainining tests to work with LUKS iotests: reduce PBKDF iterations when testing LUKS iotests: add more LUKS hash combination tests iotests: chown LUKS device before qemu-io launches Eric Blake (26): qemu-io: Don't die on second open block: Guarantee that *file is set on bdrv_get_block_status() block: Simplify use of BDRV_BLOCK_RAW blkdebug: Support .bdrv_co_get_block_status blockjob: Track job ratelimits via bytes, not sectors trace: Show blockjob actions via bytes, not sectors stream: Switch stream_populate() to byte-based stream: Drop reached_end for stream_complete() stream: Switch stream_run() to byte-based commit: Switch commit_populate() to byte-based commit: Switch commit_run() to byte-based mirror: Switch MirrorBlockJob to byte-based mirror: Switch mirror_do_zero_or_discard() to byte-based mirror: Update signature of mirror_clip_sectors() mirror: Switch mirror_cow_align() to byte-based mirror: Switch mirror_do_read() to byte-based mirror: Switch mirror_iteration() to byte-based block: Drop unused bdrv_round_sectors_to_clusters() backup: Switch BackupBlockJob to byte-based backup: Switch block_backup.h to byte-based backup: Switch backup_do_cow() to byte-based backup: Switch backup_run() to byte-based block: Make bdrv_is_allocated() byte-based block: Minimize raw use of bds->total_sectors block: Make bdrv_is_allocated_above() byte-based tests: Avoid non-portable 'echo -ARG' Hervé Poussineau (13): vvfat: fix qemu-img map and qemu-img convert vvfat: replace tabs by 8 spaces vvfat: fix typos vvfat: rename useless enumeration values vvfat: introduce offset_to_bootsector, offset_to_fat and offset_to_root_dir vvfat: fix field names in FAT12/FAT16 and FAT32 boot sectors vvfat: always create . and .. entries at first and in that order vvfat: correctly create long names for non-ASCII filenames vvfat: correctly create base short names for non-ASCII filenames vvfat: correctly generate numeric-tail of short file names vvfat: limit number of entries in root directory in FAT12/FAT16 vvfat: handle KANJI lead byte 0xe5 vvfat: change OEM name to 'MSWIN4.1' Kevin Wolf (1): Merge remote-tracking branch 'mreitz/tags/pull-block-2017-07-07' into queue-block Max Reitz (3): iotests: 181 does not work for all formats iotests: Use absolute paths for executables iotests: Add test for colon handling Thomas Huth (1): blockdev: Print a warning for legacy drive options that belong to -device Vladimir Sementsov-Ogievskiy (30): specs/qcow2: fix bitmap granularity qemu-specific note specs/qcow2: do not use wording 'bitmap header' hbitmap: improve dirty iter tests: add hbitmap iter test block: fix bdrv_dirty_bitmap_granularity signature block/dirty-bitmap: add deserialize_ones func qcow2-refcount: rename inc_refcounts() and make it public qcow2: add bitmaps extension block/dirty-bitmap: fix comment for BlockDirtyBitmap.disabled field block/dirty-bitmap: add readonly field to BdrvDirtyBitmap qcow2: autoloading dirty bitmaps block: refactor bdrv_reopen_commit block: new bdrv_reopen_bitmaps_rw interface qcow2: support .bdrv_reopen_bitmaps_rw block/dirty-bitmap: add autoload field to BdrvDirtyBitmap block: bdrv_close: release bitmaps after drv->bdrv_close block: introduce persistent dirty bitmaps block/dirty-bitmap: add bdrv_dirty_bitmap_next() qcow2: add persistent dirty bitmaps support qcow2: store bitmaps on reopening image as read-only block: add bdrv_can_store_new_dirty_bitmap qcow2: add .bdrv_can_store_new_dirty_bitmap qmp: add persistent flag to block-dirty-bitmap-add qmp: add autoload parameter to block-dirty-bitmap-add qmp: add x-debug-block-dirty-bitmap-sha256 iotests: test qcow2 persistent dirty bitmap block/dirty-bitmap: add bdrv_remove_persistent_dirty_bitmap qcow2: add .bdrv_remove_persistent_dirty_bitmap qmp: block-dirty-bitmap-remove: remove persistent block: release persistent bitmaps on inactivate sochin.jiang (1): mirror: Fix inconsistent backing AioContext for after mirroring block.c | 143 +-- block/Makefile.objs | 2 +- block/backup.c | 128 +-- block/blkdebug.c | 11 + block/commit.c | 56 +- block/crypto.c | 97 +- block/crypto.h | 101 ++ block/dirty-bitmap.c | 154 ++- block/io.c | 110 +- block/mirror.c | 310 ++--- block/qapi.c | 2 +- block/qcow.c | 269 ++--- block/qcow2-bitmap.c | 1481 ++++++++++++++++++++++++ block/qcow2-cluster.c | 66 +- block/qcow2-refcount.c | 69 +- block/qcow2.c | 653 +++++++++-- block/qcow2.h | 60 +- block/raw-format.c | 2 +- block/replication.c | 29 +- block/stream.c | 37 +- block/trace-events | 14 +- block/vpc.c | 2 +- block/vvfat.c | 2336 ++++++++++++++++++++------------------ blockdev.c | 124 +- crypto/block-luks.c | 8 +- crypto/block-qcow.c | 8 +- crypto/block.c | 6 +- crypto/blockpriv.h | 2 + docs/interop/qcow2.txt | 111 +- hmp-commands.hx | 2 + hmp.c | 31 - include/block/block.h | 22 +- include/block/block_backup.h | 11 +- include/block/block_int.h | 17 +- include/block/dirty-bitmap.h | 22 +- include/crypto/block.h | 6 +- include/monitor/monitor.h | 7 - include/qapi/error.h | 1 - include/qemu/hbitmap.h | 49 +- include/qemu/osdep.h | 2 - include/qemu/ratelimit.h | 3 +- migration/block.c | 16 +- monitor.c | 68 -- qapi-schema.json | 10 +- qapi/block-core.json | 172 ++- qapi/common.json | 5 +- qemu-doc.texi | 123 +- qemu-img.c | 76 +- qemu-img.texi | 19 +- qemu-io-cmds.c | 70 +- qemu-io.c | 27 +- qemu-options.hx | 13 +- qmp.c | 12 +- tests/Makefile.include | 2 +- tests/multiboot/run_test.sh | 10 +- tests/qemu-iotests/033 | 12 +- tests/qemu-iotests/042 | 2 +- tests/qemu-iotests/048 | 2 +- tests/qemu-iotests/049 | 2 +- tests/qemu-iotests/049.out | 102 +- tests/qemu-iotests/051 | 7 +- tests/qemu-iotests/060.out | 1 + tests/qemu-iotests/068 | 2 +- tests/qemu-iotests/082.out | 284 ++++- tests/qemu-iotests/085.out | 38 +- tests/qemu-iotests/087 | 39 +- tests/qemu-iotests/087.out | 16 +- tests/qemu-iotests/114.out | 5 +- tests/qemu-iotests/120 | 1 + tests/qemu-iotests/126 | 105 ++ tests/qemu-iotests/126.out | 23 + tests/qemu-iotests/134 | 20 +- tests/qemu-iotests/134.out | 10 +- tests/qemu-iotests/140 | 9 +- tests/qemu-iotests/142 | 48 +- tests/qemu-iotests/144.out | 4 +- tests/qemu-iotests/145 | 19 +- tests/qemu-iotests/149 | 25 +- tests/qemu-iotests/149.out | 1002 +++++++++++----- tests/qemu-iotests/153.out | 6 + tests/qemu-iotests/157 | 17 +- tests/qemu-iotests/157.out | 16 +- tests/qemu-iotests/158 | 21 +- tests/qemu-iotests/158.out | 14 +- tests/qemu-iotests/159 | 1 + tests/qemu-iotests/165 | 105 ++ tests/qemu-iotests/165.out | 5 + tests/qemu-iotests/170 | 1 + tests/qemu-iotests/171 | 14 +- tests/qemu-iotests/174 | 2 +- tests/qemu-iotests/177 | 3 + tests/qemu-iotests/177.out | 5 + tests/qemu-iotests/181 | 23 +- tests/qemu-iotests/185.out | 8 +- tests/qemu-iotests/188 | 76 ++ tests/qemu-iotests/188.out | 18 + tests/qemu-iotests/189 | 86 ++ tests/qemu-iotests/189.out | 26 + tests/qemu-iotests/check | 18 +- tests/qemu-iotests/common | 10 +- tests/qemu-iotests/common.config | 11 + tests/qemu-iotests/common.filter | 3 +- tests/qemu-iotests/common.qemu | 9 +- tests/qemu-iotests/common.rc | 3 + tests/qemu-iotests/group | 4 + tests/rocker/all | 10 +- tests/tcg/cris/Makefile | 8 +- tests/test-crypto-block.c | 8 +- tests/test-hbitmap.c | 19 + util/hbitmap.c | 51 +- util/oslib-posix.c | 66 -- util/oslib-win32.c | 24 - 112 files changed, 6714 insertions(+), 2942 deletions(-) create mode 100644 block/crypto.h create mode 100644 block/qcow2-bitmap.c create mode 100755 tests/qemu-iotests/126 create mode 100644 tests/qemu-iotests/126.out create mode 100755 tests/qemu-iotests/165 create mode 100644 tests/qemu-iotests/165.out create mode 100755 tests/qemu-iotests/188 create mode 100644 tests/qemu-iotests/188.out create mode 100755 tests/qemu-iotests/189 create mode 100644 tests/qemu-iotests/189.out From: Eric Blake <eblake@redhat.com> Most callback commands in qemu-io return 0 to keep the interpreter loop running, or 1 to quit immediately. However, open_f() just passed through the return value of openfile(), which has different semantics of returning 0 if a file was opened, or 1 on any failure. As a result of mixing the return semantics, we are forcing the qemu-io interpreter to exit early on any failures, which is rather annoying when some of the failures are obviously trying to give the user a hint of how to proceed (if we didn't then kill qemu-io out from under the user's feet): $ qemu-io qemu-io> open foo qemu-io> open foo file open already, try 'help close' $ echo $? 0 In general, we WANT openfile() to report failures, since it is the function used in the form 'qemu-io -c "$something" no_such_file' for performing one or more -c options on a single file, and it is not worth attempting $something if the file itself cannot be opened. So the solution is to fix open_f() to always return 0 (when we are in interactive mode, even failure to open should not end the session), and save the return value of openfile() for command line use in main(). Note, however, that we do have some qemu-iotests that do 'qemu-io -c "open file" -c "$something"'; such tests will now proceed to attempt $something whether or not the open succeeded, the same way as if the two commands had been attempted in interactive mode. As such, the expected output for those tests has to be modified. But it also means that it is now possible to use -c close and have a single qemu-io command line operate on more than one file even without using interactive mode. Although the '-c open' action is a subtle change in behavior, remember that qemu-io is for debugging purposes, so as long as it serves the needs of qemu-iotests while still being reasonable for interactive use, it should not be a problem that we are changing tests to the new behavior. This has been awkward since at least as far back as commit e3aff4f, in 2009. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qemu-io.c | 7 ++++--- tests/qemu-iotests/060.out | 1 + tests/qemu-iotests/114.out | 5 +++-- tests/qemu-iotests/153.out | 6 ++++++ 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/qemu-io.c b/qemu-io.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-io.c +++ b/qemu-io.c @@ -XXX,XX +XXX,XX @@ static int open_f(BlockBackend *blk, int argc, char **argv) qemu_opts_reset(&empty_opts); if (optind == argc - 1) { - return openfile(argv[optind], flags, writethrough, force_share, opts); + openfile(argv[optind], flags, writethrough, force_share, opts); } else if (optind == argc) { - return openfile(NULL, flags, writethrough, force_share, opts); + openfile(NULL, flags, writethrough, force_share, opts); } else { QDECREF(opts); - return qemuio_command_usage(&open_cmd); + qemuio_command_usage(&open_cmd); } + return 0; } static int quit_f(BlockBackend *blk, int argc, char **argv) diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -XXX,XX +XXX,XX @@ Format specific information: refcount bits: 16 corrupt: true can't open device TEST_DIR/t.IMGFMT: IMGFMT: Image is corrupt; cannot be opened read/write +no file open, try 'help open' read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/114.out b/tests/qemu-iotests/114.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/114.out +++ b/tests/qemu-iotests/114.out @@ -XXX,XX +XXX,XX @@ QA output created by 114 -Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 64M (67108864 bytes) @@ -XXX,XX +XXX,XX @@ cluster_size: 65536 backing file: TEST_DIR/t.IMGFMT.base backing file format: foo can't open device TEST_DIR/t.qcow2: Could not open backing file: Unknown driver 'foo' +no file open, try 'help open' read 4096/4096 bytes at offset 0 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done diff --git a/tests/qemu-iotests/153.out b/tests/qemu-iotests/153.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/153.out +++ b/tests/qemu-iotests/153.out @@ -XXX,XX +XXX,XX @@ Is another process using the image? _qemu_io_wrapper -c open TEST_DIR/t.qcow2 -c read 0 512 can't open device TEST_DIR/t.qcow2: Failed to get "write" lock Is another process using the image? +no file open, try 'help open' _qemu_io_wrapper -c open -r TEST_DIR/t.qcow2 -c read 0 512 can't open device TEST_DIR/t.qcow2: Failed to get shared "write" lock Is another process using the image? +no file open, try 'help open' _qemu_img_wrapper info TEST_DIR/t.qcow2 qemu-img: Could not open 'TEST_DIR/t.qcow2': Failed to get shared "write" lock @@ -XXX,XX +XXX,XX @@ _qemu_io_wrapper -U -r -c read 0 512 TEST_DIR/t.qcow2 _qemu_io_wrapper -c open -U TEST_DIR/t.qcow2 -c read 0 512 can't open device TEST_DIR/t.qcow2: force-share=on can only be used with read-only images +no file open, try 'help open' _qemu_io_wrapper -c open -r -U TEST_DIR/t.qcow2 -c read 0 512 @@ -XXX,XX +XXX,XX @@ _qemu_io_wrapper -r -c read 0 512 TEST_DIR/t.qcow2 _qemu_io_wrapper -c open TEST_DIR/t.qcow2 -c read 0 512 can't open device TEST_DIR/t.qcow2: Failed to get "write" lock Is another process using the image? +no file open, try 'help open' _qemu_io_wrapper -c open -r TEST_DIR/t.qcow2 -c read 0 512 @@ -XXX,XX +XXX,XX @@ _qemu_io_wrapper -U -r -c read 0 512 TEST_DIR/t.qcow2 _qemu_io_wrapper -c open -U TEST_DIR/t.qcow2 -c read 0 512 can't open device TEST_DIR/t.qcow2: force-share=on can only be used with read-only images +no file open, try 'help open' _qemu_io_wrapper -c open -r -U TEST_DIR/t.qcow2 -c read 0 512 @@ -XXX,XX +XXX,XX @@ _qemu_io_wrapper -U -r -c read 0 512 TEST_DIR/t.qcow2 _qemu_io_wrapper -c open -U TEST_DIR/t.qcow2 -c read 0 512 can't open device TEST_DIR/t.qcow2: force-share=on can only be used with read-only images +no file open, try 'help open' _qemu_io_wrapper -c open -r -U TEST_DIR/t.qcow2 -c read 0 512 -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We document that *file is valid if the return is not an error and includes BDRV_BLOCK_OFFSET_VALID, but forgot to obey this contract when a driver (such as blkdebug) lacks a callback. Messed up in commit 67a0fd2 (v2.6), when we added the file parameter. Enhance qemu-iotest 177 to cover this, using a sequence that would print garbage or even SEGV, because it was dererefencing through uninitialized memory. [The resulting test output shows that we have less-than-ideal block status from the blkdebug driver, but that's a separate fix coming up soon.] Setting *file on all paths that return BDRV_BLOCK_OFFSET_VALID is enough to fix the crash, but we can go one step further: always setting *file, even on error, means that a broken caller that blindly dereferences file without checking for error is now more likely to get a reliable SEGV instead of randomly acting on garbage, making it easier to diagnose such buggy callers. Adding an assertion that file is set where expected doesn't hurt either. CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/io.c | 5 +++-- tests/qemu-iotests/177 | 3 +++ tests/qemu-iotests/177.out | 2 ++ 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/block/io.c b/block/io.c index XXXXXXX..XXXXXXX 100644 --- a/block/io.c +++ b/block/io.c @@ -XXX,XX +XXX,XX @@ static int64_t coroutine_fn bdrv_co_get_block_status(BlockDriverState *bs, int64_t n; int64_t ret, ret2; + *file = NULL; total_sectors = bdrv_nb_sectors(bs); if (total_sectors < 0) { return total_sectors; @@ -XXX,XX +XXX,XX @@ static int64_t coroutine_fn bdrv_co_get_block_status(BlockDriverState *bs, } if (bs->drv->protocol_name) { ret |= BDRV_BLOCK_OFFSET_VALID | (sector_num * BDRV_SECTOR_SIZE); + *file = bs; } return ret; } - *file = NULL; bdrv_inc_in_flight(bs); ret = bs->drv->bdrv_co_get_block_status(bs, sector_num, nb_sectors, pnum, file); @@ -XXX,XX +XXX,XX @@ static int64_t coroutine_fn bdrv_co_get_block_status(BlockDriverState *bs, } if (ret & BDRV_BLOCK_RAW) { - assert(ret & BDRV_BLOCK_OFFSET_VALID); + assert(ret & BDRV_BLOCK_OFFSET_VALID && *file); ret = bdrv_co_get_block_status(*file, ret >> BDRV_SECTOR_BITS, *pnum, pnum, file); goto out; diff --git a/tests/qemu-iotests/177 b/tests/qemu-iotests/177 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/177 +++ b/tests/qemu-iotests/177 @@ -XXX,XX +XXX,XX @@ _supported_proto file CLUSTER_SIZE=1M size=128M options=driver=blkdebug,image.driver=qcow2 +nested_opts=image.file.driver=file,image.file.filename=$TEST_IMG echo echo "== setting up files ==" @@ -XXX,XX +XXX,XX @@ function verify_io() } verify_io | $QEMU_IO -r "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --image-opts "$options,$nested_opts,align=4k" \ + | _filter_qemu_img_map _check_test_img diff --git a/tests/qemu-iotests/177.out b/tests/qemu-iotests/177.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/177.out +++ b/tests/qemu-iotests/177.out @@ -XXX,XX +XXX,XX @@ read 30408704/30408704 bytes at offset 80740352 29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 23068672/23068672 bytes at offset 111149056 22 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0 0x8000000 json:{"image": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}}, "driver": "blkdebug", "align": "4k"} No errors were found on the image. *** done -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> The lone caller that cares about a return of BDRV_BLOCK_RAW (namely, io.c:bdrv_co_get_block_status) completely replaces the return value, so there is no point in passing BDRV_BLOCK_DATA. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/commit.c | 2 +- block/mirror.c | 2 +- block/raw-format.c | 2 +- block/vpc.c | 2 +- include/block/block.h | 6 +++--- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/block/commit.c b/block/commit.c index XXXXXXX..XXXXXXX 100644 --- a/block/commit.c +++ b/block/commit.c @@ -XXX,XX +XXX,XX @@ static int64_t coroutine_fn bdrv_commit_top_get_block_status( { *pnum = nb_sectors; *file = bs->backing->bs; - return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID | BDRV_BLOCK_DATA | + return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID | (sector_num << BDRV_SECTOR_BITS); } diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static int64_t coroutine_fn bdrv_mirror_top_get_block_status( { *pnum = nb_sectors; *file = bs->backing->bs; - return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID | BDRV_BLOCK_DATA | + return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID | (sector_num << BDRV_SECTOR_BITS); } diff --git a/block/raw-format.c b/block/raw-format.c index XXXXXXX..XXXXXXX 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -XXX,XX +XXX,XX @@ static int64_t coroutine_fn raw_co_get_block_status(BlockDriverState *bs, *pnum = nb_sectors; *file = bs->file->bs; sector_num += s->offset / BDRV_SECTOR_SIZE; - return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID | BDRV_BLOCK_DATA | + return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID | (sector_num << BDRV_SECTOR_BITS); } diff --git a/block/vpc.c b/block/vpc.c index XXXXXXX..XXXXXXX 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -XXX,XX +XXX,XX @@ static int64_t coroutine_fn vpc_co_get_block_status(BlockDriverState *bs, if (be32_to_cpu(footer->type) == VHD_FIXED) { *pnum = nb_sectors; *file = bs->file->bs; - return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID | BDRV_BLOCK_DATA | + return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID | (sector_num << BDRV_SECTOR_BITS); } diff --git a/include/block/block.h b/include/block/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -XXX,XX +XXX,XX @@ typedef struct HDGeometry { * BDRV_BLOCK_EOF: the returned pnum covers through end of file for this layer * * Internal flag: - * BDRV_BLOCK_RAW: used internally to indicate that the request was - * answered by a passthrough driver such as raw and that the - * block layer should recompute the answer from bs->file. + * BDRV_BLOCK_RAW: for use by passthrough drivers, such as raw, to request + * that the block layer recompute the answer from the returned + * BDS; must be accompanied by just BDRV_BLOCK_OFFSET_VALID. * * If BDRV_BLOCK_OFFSET_VALID is set, bits 9-62 (BDRV_BLOCK_OFFSET_MASK) * represent the offset in the returned BDS that is allocated for the -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> Without a passthrough status of BDRV_BLOCK_RAW, anything wrapped by blkdebug appears 100% allocated as data. Better is treating it the same as the underlying file being wrapped. Update iotest 177 for the new expected output. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/blkdebug.c | 11 +++++++++++ tests/qemu-iotests/177.out | 5 ++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index XXXXXXX..XXXXXXX 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn blkdebug_co_pdiscard(BlockDriverState *bs, return bdrv_co_pdiscard(bs->file->bs, offset, bytes); } +static int64_t coroutine_fn blkdebug_co_get_block_status( + BlockDriverState *bs, int64_t sector_num, int nb_sectors, int *pnum, + BlockDriverState **file) +{ + *pnum = nb_sectors; + *file = bs->file->bs; + return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID | + (sector_num << BDRV_SECTOR_BITS); +} + static void blkdebug_close(BlockDriverState *bs) { BDRVBlkdebugState *s = bs->opaque; @@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_blkdebug = { .bdrv_co_flush_to_disk = blkdebug_co_flush, .bdrv_co_pwrite_zeroes = blkdebug_co_pwrite_zeroes, .bdrv_co_pdiscard = blkdebug_co_pdiscard, + .bdrv_co_get_block_status = blkdebug_co_get_block_status, .bdrv_debug_event = blkdebug_debug_event, .bdrv_debug_breakpoint = blkdebug_debug_breakpoint, diff --git a/tests/qemu-iotests/177.out b/tests/qemu-iotests/177.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/177.out +++ b/tests/qemu-iotests/177.out @@ -XXX,XX +XXX,XX @@ read 30408704/30408704 bytes at offset 80740352 read 23068672/23068672 bytes at offset 111149056 22 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Offset Length File -0 0x8000000 json:{"image": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}}, "driver": "blkdebug", "align": "4k"} +0 0x800000 TEST_DIR/t.IMGFMT +0x900000 0x2400000 TEST_DIR/t.IMGFMT +0x3c00000 0x1100000 TEST_DIR/t.IMGFMT +0x6a00000 0x1600000 TEST_DIR/t.IMGFMT No errors were found on the image. *** done -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> - bs->total_sectors is the number of sectors of the whole disk - s->sector_count is the number of sectors of the FAT partition This fixes the following assert in qemu-img map: qemu-img.c:2641: get_block_status: Assertion `nb_sectors' failed. This also fixes an infinite loop in qemu-img convert. Fixes: 4480e0f924a42e1db8b8cfcac4d0634dd1bb27a0 Fixes: https://bugs.launchpad.net/qemu/+bug/1599539 Cc: qemu-stable@nongnu.org Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ vvfat_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, static int64_t coroutine_fn vvfat_co_get_block_status(BlockDriverState *bs, int64_t sector_num, int nb_sectors, int *n, BlockDriverState **file) { - BDRVVVFATState* s = bs->opaque; - *n = s->sector_count - sector_num; + *n = bs->total_sectors - sector_num; if (*n > nb_sectors) { *n = nb_sectors; } else if (*n < 0) { -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> This was a complete mess. On 2299 indented lines: - 1329 were with spaces only - 617 with tabulations only - 353 with spaces and tabulations Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 2054 ++++++++++++++++++++++++++++----------------------------- 1 file changed, 1027 insertions(+), 1027 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static inline void* array_get(array_t* array,unsigned int index) { static inline int array_ensure_allocated(array_t* array, int index) { if((index + 1) * array->item_size > array->size) { - int new_size = (index + 32) * array->item_size; - array->pointer = g_realloc(array->pointer, new_size); - if (!array->pointer) - return -1; - array->size = new_size; - array->next = index + 1; + int new_size = (index + 32) * array->item_size; + array->pointer = g_realloc(array->pointer, new_size); + if (!array->pointer) + return -1; + array->size = new_size; + array->next = index + 1; } return 0; @@ -XXX,XX +XXX,XX @@ static inline void* array_get_next(array_t* array) { unsigned int next = array->next; if (array_ensure_allocated(array, next) < 0) - return NULL; + return NULL; array->next = next + 1; return array_get(array, next); @@ -XXX,XX +XXX,XX @@ static inline void* array_get_next(array_t* array) { static inline void* array_insert(array_t* array,unsigned int index,unsigned int count) { if((array->next+count)*array->item_size>array->size) { - int increment=count*array->item_size; - array->pointer=g_realloc(array->pointer,array->size+increment); - if(!array->pointer) + int increment=count*array->item_size; + array->pointer=g_realloc(array->pointer,array->size+increment); + if(!array->pointer) return NULL; - array->size+=increment; + array->size+=increment; } memmove(array->pointer+(index+count)*array->item_size, - array->pointer+index*array->item_size, - (array->next-index)*array->item_size); + array->pointer+index*array->item_size, + (array->next-index)*array->item_size); array->next+=count; return array->pointer+index*array->item_size; } @@ -XXX,XX +XXX,XX @@ static inline int array_roll(array_t* array,int index_to,int index_from,int coun int is; if(!array || - index_to<0 || index_to>=array->next || - index_from<0 || index_from>=array->next) - return -1; + index_to<0 || index_to>=array->next || + index_from<0 || index_from>=array->next) + return -1; if(index_to==index_from) - return 0; + return 0; is=array->item_size; from=array->pointer+index_from*is; @@ -XXX,XX +XXX,XX @@ static inline int array_roll(array_t* array,int index_to,int index_from,int coun memcpy(buf,from,is*count); if(index_to<index_from) - memmove(to+is*count,to,from-to); + memmove(to+is*count,to,from-to); else - memmove(from,from+is*count,to-from); + memmove(from,from+is*count,to-from); memcpy(to,buf,is*count); @@ -XXX,XX +XXX,XX @@ static inline int array_remove_slice(array_t* array,int index, int count) assert(count > 0); assert(index + count <= array->next); if(array_roll(array,array->next-1,index,count)) - return -1; + return -1; array->next -= count; return 0; } @@ -XXX,XX +XXX,XX @@ typedef struct bootsector_t { uint32_t total_sectors; union { struct { - uint8_t drive_number; - uint8_t current_head; - uint8_t signature; - uint32_t id; - uint8_t volume_label[11]; - } QEMU_PACKED fat16; - struct { - uint32_t sectors_per_fat; - uint16_t flags; - uint8_t major,minor; - uint32_t first_cluster_of_root_directory; - uint16_t info_sector; - uint16_t backup_boot_sector; - uint16_t ignored; - } QEMU_PACKED fat32; + uint8_t drive_number; + uint8_t current_head; + uint8_t signature; + uint32_t id; + uint8_t volume_label[11]; + } QEMU_PACKED fat16; + struct { + uint32_t sectors_per_fat; + uint16_t flags; + uint8_t major,minor; + uint32_t first_cluster_of_root_directory; + uint16_t info_sector; + uint16_t backup_boot_sector; + uint16_t ignored; + } QEMU_PACKED fat32; } u; uint8_t fat_type[8]; uint8_t ignored[0x1c0]; @@ -XXX,XX +XXX,XX @@ typedef struct mapping_t { /* the clusters of a file may be in any order; this points to the first */ int first_mapping_index; union { - /* offset is - * - the offset in the file (in clusters) for a file, or - * - the next cluster of the directory for a directory, and - * - the address of the buffer for a faked entry - */ - struct { - uint32_t offset; - } file; - struct { - int parent_mapping_index; - int first_dir_index; - } dir; + /* offset is + * - the offset in the file (in clusters) for a file, or + * - the next cluster of the directory for a directory, and + * - the address of the buffer for a faked entry + */ + struct { + uint32_t offset; + } file; + struct { + int parent_mapping_index; + int first_dir_index; + } dir; } info; /* path contains the full path, i.e. it always starts with s->path */ char* path; enum { MODE_UNDEFINED = 0, MODE_NORMAL = 1, MODE_MODIFIED = 2, - MODE_DIRECTORY = 4, MODE_FAKED = 8, - MODE_DELETED = 16, MODE_RENAMED = 32 } mode; + MODE_DIRECTORY = 4, MODE_FAKED = 8, + MODE_DELETED = 16, MODE_RENAMED = 32 } mode; int read_only; } mapping_t; @@ -XXX,XX +XXX,XX @@ static inline int short2long_name(char* dest,const char* src) int len; for(i=0;i<129 && src[i];i++) { dest[2*i]=src[i]; - dest[2*i+1]=0; + dest[2*i+1]=0; } len=2*i; dest[2*i]=dest[2*i+1]=0; for(i=2*i+2;(i%26);i++) - dest[i]=0xff; + dest[i]=0xff; return len; } @@ -XXX,XX +XXX,XX @@ static inline direntry_t* create_long_filename(BDRVVVFATState* s,const char* fil direntry_t* entry; for(i=0;i<number_of_entries;i++) { - entry=array_get_next(&(s->directory)); - entry->attributes=0xf; - entry->reserved[0]=0; - entry->begin=0; - entry->name[0]=(number_of_entries-i)|(i==0?0x40:0); + entry=array_get_next(&(s->directory)); + entry->attributes=0xf; + entry->reserved[0]=0; + entry->begin=0; + entry->name[0]=(number_of_entries-i)|(i==0?0x40:0); } for(i=0;i<26*number_of_entries;i++) { - int offset=(i%26); - if(offset<10) offset=1+offset; - else if(offset<22) offset=14+offset-10; - else offset=28+offset-22; - entry=array_get(&(s->directory),s->directory.next-1-(i/26)); - entry->name[offset]=buffer[i]; + int offset=(i%26); + if(offset<10) offset=1+offset; + else if(offset<22) offset=14+offset-10; + else offset=28+offset-22; + entry=array_get(&(s->directory),s->directory.next-1-(i/26)); + entry->name[offset]=buffer[i]; } return array_get(&(s->directory),s->directory.next-number_of_entries); } @@ -XXX,XX +XXX,XX @@ static char is_long_name(const direntry_t* direntry) static char is_short_name(const direntry_t* direntry) { return !is_volume_label(direntry) && !is_long_name(direntry) - && !is_free(direntry); + && !is_free(direntry); } static char is_directory(const direntry_t* direntry) @@ -XXX,XX +XXX,XX @@ static uint16_t fat_datetime(time_t time,int return_time) { t = &t1; localtime_r(&time,t); if(return_time) - return cpu_to_le16((t->tm_sec/2)|(t->tm_min<<5)|(t->tm_hour<<11)); + return cpu_to_le16((t->tm_sec/2)|(t->tm_min<<5)|(t->tm_hour<<11)); return cpu_to_le16((t->tm_mday)|((t->tm_mon+1)<<5)|((t->tm_year-80)<<9)); } static inline void fat_set(BDRVVVFATState* s,unsigned int cluster,uint32_t value) { if(s->fat_type==32) { - uint32_t* entry=array_get(&(s->fat),cluster); - *entry=cpu_to_le32(value); + uint32_t* entry=array_get(&(s->fat),cluster); + *entry=cpu_to_le32(value); } else if(s->fat_type==16) { - uint16_t* entry=array_get(&(s->fat),cluster); - *entry=cpu_to_le16(value&0xffff); + uint16_t* entry=array_get(&(s->fat),cluster); + *entry=cpu_to_le16(value&0xffff); } else { - int offset = (cluster*3/2); - unsigned char* p = array_get(&(s->fat), offset); + int offset = (cluster*3/2); + unsigned char* p = array_get(&(s->fat), offset); switch (cluster&1) { - case 0: - p[0] = value&0xff; - p[1] = (p[1]&0xf0) | ((value>>8)&0xf); - break; - case 1: - p[0] = (p[0]&0xf) | ((value&0xf)<<4); - p[1] = (value>>4); - break; - } + case 0: + p[0] = value&0xff; + p[1] = (p[1]&0xf0) | ((value>>8)&0xf); + break; + case 1: + p[0] = (p[0]&0xf) | ((value&0xf)<<4); + p[1] = (value>>4); + break; + } } } static inline uint32_t fat_get(BDRVVVFATState* s,unsigned int cluster) { if(s->fat_type==32) { - uint32_t* entry=array_get(&(s->fat),cluster); - return le32_to_cpu(*entry); + uint32_t* entry=array_get(&(s->fat),cluster); + return le32_to_cpu(*entry); } else if(s->fat_type==16) { - uint16_t* entry=array_get(&(s->fat),cluster); - return le16_to_cpu(*entry); + uint16_t* entry=array_get(&(s->fat),cluster); + return le16_to_cpu(*entry); } else { - const uint8_t* x=(uint8_t*)(s->fat.pointer)+cluster*3/2; - return ((x[0]|(x[1]<<8))>>(cluster&1?4:0))&0x0fff; + const uint8_t* x=(uint8_t*)(s->fat.pointer)+cluster*3/2; + return ((x[0]|(x[1]<<8))>>(cluster&1?4:0))&0x0fff; } } static inline int fat_eof(BDRVVVFATState* s,uint32_t fat_entry) { if(fat_entry>s->max_fat_value-8) - return -1; + return -1; return 0; } static inline void init_fat(BDRVVVFATState* s) { if (s->fat_type == 12) { - array_init(&(s->fat),1); - array_ensure_allocated(&(s->fat), - s->sectors_per_fat * 0x200 * 3 / 2 - 1); + array_init(&(s->fat),1); + array_ensure_allocated(&(s->fat), + s->sectors_per_fat * 0x200 * 3 / 2 - 1); } else { - array_init(&(s->fat),(s->fat_type==32?4:2)); - array_ensure_allocated(&(s->fat), - s->sectors_per_fat * 0x200 / s->fat.item_size - 1); + array_init(&(s->fat),(s->fat_type==32?4:2)); + array_ensure_allocated(&(s->fat), + s->sectors_per_fat * 0x200 / s->fat.item_size - 1); } memset(s->fat.pointer,0,s->fat.size); switch(s->fat_type) { - case 12: s->max_fat_value=0xfff; break; - case 16: s->max_fat_value=0xffff; break; - case 32: s->max_fat_value=0x0fffffff; break; - default: s->max_fat_value=0; /* error... */ + case 12: s->max_fat_value=0xfff; break; + case 16: s->max_fat_value=0xffff; break; + case 32: s->max_fat_value=0x0fffffff; break; + default: s->max_fat_value=0; /* error... */ } } @@ -XXX,XX +XXX,XX @@ static inline void init_fat(BDRVVVFATState* s) /* TODO: in create_short_filename, 0xe5->0x05 is not yet handled! */ /* TODO: in parse_short_filename, 0x05->0xe5 is not yet handled! */ static inline direntry_t* create_short_and_long_name(BDRVVVFATState* s, - unsigned int directory_start, const char* filename, int is_dot) + unsigned int directory_start, const char* filename, int is_dot) { int i,j,long_index=s->directory.next; direntry_t* entry = NULL; direntry_t* entry_long = NULL; if(is_dot) { - entry=array_get_next(&(s->directory)); + entry=array_get_next(&(s->directory)); memset(entry->name, 0x20, sizeof(entry->name)); - memcpy(entry->name,filename,strlen(filename)); - return entry; + memcpy(entry->name,filename,strlen(filename)); + return entry; } entry_long=create_long_filename(s,filename); @@ -XXX,XX +XXX,XX @@ static inline direntry_t* create_short_and_long_name(BDRVVVFATState* s, i = strlen(filename); for(j = i - 1; j>0 && filename[j]!='.';j--); if (j > 0) - i = (j > 8 ? 8 : j); + i = (j > 8 ? 8 : j); else if (i > 8) - i = 8; + i = 8; entry=array_get_next(&(s->directory)); memset(entry->name, 0x20, sizeof(entry->name)); @@ -XXX,XX +XXX,XX @@ static inline direntry_t* create_short_and_long_name(BDRVVVFATState* s, /* upcase & remove unwanted characters */ for(i=10;i>=0;i--) { - if(i==10 || i==7) for(;i>0 && entry->name[i]==' ';i--); - if(entry->name[i]<=' ' || entry->name[i]>0x7f - || strchr(".*?<>|\":/\\[];,+='",entry->name[i])) - entry->name[i]='_'; + if(i==10 || i==7) for(;i>0 && entry->name[i]==' ';i--); + if(entry->name[i]<=' ' || entry->name[i]>0x7f + || strchr(".*?<>|\":/\\[];,+='",entry->name[i])) + entry->name[i]='_'; else if(entry->name[i]>='a' && entry->name[i]<='z') entry->name[i]+='A'-'a'; } /* mangle duplicates */ while(1) { - direntry_t* entry1=array_get(&(s->directory),directory_start); - int j; - - for(;entry1<entry;entry1++) - if(!is_long_name(entry1) && !memcmp(entry1->name,entry->name,11)) - break; /* found dupe */ - if(entry1==entry) /* no dupe found */ - break; - - /* use all 8 characters of name */ - if(entry->name[7]==' ') { - int j; - for(j=6;j>0 && entry->name[j]==' ';j--) - entry->name[j]='~'; - } - - /* increment number */ - for(j=7;j>0 && entry->name[j]=='9';j--) - entry->name[j]='0'; - if(j>0) { - if(entry->name[j]<'0' || entry->name[j]>'9') - entry->name[j]='0'; - else - entry->name[j]++; - } + direntry_t* entry1=array_get(&(s->directory),directory_start); + int j; + + for(;entry1<entry;entry1++) + if(!is_long_name(entry1) && !memcmp(entry1->name,entry->name,11)) + break; /* found dupe */ + if(entry1==entry) /* no dupe found */ + break; + + /* use all 8 characters of name */ + if(entry->name[7]==' ') { + int j; + for(j=6;j>0 && entry->name[j]==' ';j--) + entry->name[j]='~'; + } + + /* increment number */ + for(j=7;j>0 && entry->name[j]=='9';j--) + entry->name[j]='0'; + if(j>0) { + if(entry->name[j]<'0' || entry->name[j]>'9') + entry->name[j]='0'; + else + entry->name[j]++; + } } /* calculate checksum; propagate to long name */ if(entry_long) { uint8_t chksum=fat_chksum(entry); - /* calculate anew, because realloc could have taken place */ - entry_long=array_get(&(s->directory),long_index); - while(entry_long<entry && is_long_name(entry_long)) { - entry_long->reserved[1]=chksum; - entry_long++; - } + /* calculate anew, because realloc could have taken place */ + entry_long=array_get(&(s->directory),long_index); + while(entry_long<entry && is_long_name(entry_long)) { + entry_long->reserved[1]=chksum; + entry_long++; + } } return entry; @@ -XXX,XX +XXX,XX @@ static int read_directory(BDRVVVFATState* s, int mapping_index) assert(mapping->mode & MODE_DIRECTORY); if(!dir) { - mapping->end = mapping->begin; - return -1; + mapping->end = mapping->begin; + return -1; } i = mapping->info.dir.first_dir_index = - first_cluster == 0 ? 0 : s->directory.next; + first_cluster == 0 ? 0 : s->directory.next; /* actually read the directory, and allocate the mappings */ while((entry=readdir(dir))) { - unsigned int length=strlen(dirname)+2+strlen(entry->d_name); + unsigned int length=strlen(dirname)+2+strlen(entry->d_name); char* buffer; - direntry_t* direntry; + direntry_t* direntry; struct stat st; - int is_dot=!strcmp(entry->d_name,"."); - int is_dotdot=!strcmp(entry->d_name,".."); + int is_dot=!strcmp(entry->d_name,"."); + int is_dotdot=!strcmp(entry->d_name,".."); - if(first_cluster == 0 && (is_dotdot || is_dot)) - continue; + if(first_cluster == 0 && (is_dotdot || is_dot)) + continue; - buffer = g_malloc(length); - snprintf(buffer,length,"%s/%s",dirname,entry->d_name); + buffer = g_malloc(length); + snprintf(buffer,length,"%s/%s",dirname,entry->d_name); - if(stat(buffer,&st)<0) { + if(stat(buffer,&st)<0) { g_free(buffer); continue; - } - - /* create directory entry for this file */ - direntry=create_short_and_long_name(s, i, entry->d_name, - is_dot || is_dotdot); - direntry->attributes=(S_ISDIR(st.st_mode)?0x10:0x20); - direntry->reserved[0]=direntry->reserved[1]=0; - direntry->ctime=fat_datetime(st.st_ctime,1); - direntry->cdate=fat_datetime(st.st_ctime,0); - direntry->adate=fat_datetime(st.st_atime,0); - direntry->begin_hi=0; - direntry->mtime=fat_datetime(st.st_mtime,1); - direntry->mdate=fat_datetime(st.st_mtime,0); - if(is_dotdot) - set_begin_of_direntry(direntry, first_cluster_of_parent); - else if(is_dot) - set_begin_of_direntry(direntry, first_cluster); - else - direntry->begin=0; /* do that later */ + } + + /* create directory entry for this file */ + direntry=create_short_and_long_name(s, i, entry->d_name, + is_dot || is_dotdot); + direntry->attributes=(S_ISDIR(st.st_mode)?0x10:0x20); + direntry->reserved[0]=direntry->reserved[1]=0; + direntry->ctime=fat_datetime(st.st_ctime,1); + direntry->cdate=fat_datetime(st.st_ctime,0); + direntry->adate=fat_datetime(st.st_atime,0); + direntry->begin_hi=0; + direntry->mtime=fat_datetime(st.st_mtime,1); + direntry->mdate=fat_datetime(st.st_mtime,0); + if(is_dotdot) + set_begin_of_direntry(direntry, first_cluster_of_parent); + else if(is_dot) + set_begin_of_direntry(direntry, first_cluster); + else + direntry->begin=0; /* do that later */ if (st.st_size > 0x7fffffff) { - fprintf(stderr, "File %s is larger than 2GB\n", buffer); + fprintf(stderr, "File %s is larger than 2GB\n", buffer); g_free(buffer); closedir(dir); - return -2; + return -2; } - direntry->size=cpu_to_le32(S_ISDIR(st.st_mode)?0:st.st_size); - - /* create mapping for this file */ - if(!is_dot && !is_dotdot && (S_ISDIR(st.st_mode) || st.st_size)) { - s->current_mapping = array_get_next(&(s->mapping)); - s->current_mapping->begin=0; - s->current_mapping->end=st.st_size; - /* - * we get the direntry of the most recent direntry, which - * contains the short name and all the relevant information. - */ - s->current_mapping->dir_index=s->directory.next-1; - s->current_mapping->first_mapping_index = -1; - if (S_ISDIR(st.st_mode)) { - s->current_mapping->mode = MODE_DIRECTORY; - s->current_mapping->info.dir.parent_mapping_index = - mapping_index; - } else { - s->current_mapping->mode = MODE_UNDEFINED; - s->current_mapping->info.file.offset = 0; - } - s->current_mapping->path=buffer; - s->current_mapping->read_only = - (st.st_mode & (S_IWUSR | S_IWGRP | S_IWOTH)) == 0; + direntry->size=cpu_to_le32(S_ISDIR(st.st_mode)?0:st.st_size); + + /* create mapping for this file */ + if(!is_dot && !is_dotdot && (S_ISDIR(st.st_mode) || st.st_size)) { + s->current_mapping = array_get_next(&(s->mapping)); + s->current_mapping->begin=0; + s->current_mapping->end=st.st_size; + /* + * we get the direntry of the most recent direntry, which + * contains the short name and all the relevant information. + */ + s->current_mapping->dir_index=s->directory.next-1; + s->current_mapping->first_mapping_index = -1; + if (S_ISDIR(st.st_mode)) { + s->current_mapping->mode = MODE_DIRECTORY; + s->current_mapping->info.dir.parent_mapping_index = + mapping_index; + } else { + s->current_mapping->mode = MODE_UNDEFINED; + s->current_mapping->info.file.offset = 0; + } + s->current_mapping->path=buffer; + s->current_mapping->read_only = + (st.st_mode & (S_IWUSR | S_IWGRP | S_IWOTH)) == 0; } else { g_free(buffer); } @@ -XXX,XX +XXX,XX @@ static int read_directory(BDRVVVFATState* s, int mapping_index) /* fill with zeroes up to the end of the cluster */ while(s->directory.next%(0x10*s->sectors_per_cluster)) { - direntry_t* direntry=array_get_next(&(s->directory)); - memset(direntry,0,sizeof(direntry_t)); + direntry_t* direntry=array_get_next(&(s->directory)); + memset(direntry,0,sizeof(direntry_t)); } /* TODO: if there are more entries, bootsector has to be adjusted! */ #define ROOT_ENTRIES (0x02 * 0x10 * s->sectors_per_cluster) if (mapping_index == 0 && s->directory.next < ROOT_ENTRIES) { - /* root directory */ - int cur = s->directory.next; - array_ensure_allocated(&(s->directory), ROOT_ENTRIES - 1); - s->directory.next = ROOT_ENTRIES; - memset(array_get(&(s->directory), cur), 0, - (ROOT_ENTRIES - cur) * sizeof(direntry_t)); + /* root directory */ + int cur = s->directory.next; + array_ensure_allocated(&(s->directory), ROOT_ENTRIES - 1); + s->directory.next = ROOT_ENTRIES; + memset(array_get(&(s->directory), cur), 0, + (ROOT_ENTRIES - cur) * sizeof(direntry_t)); } /* reget the mapping, since s->mapping was possibly realloc()ed */ mapping = array_get(&(s->mapping), mapping_index); first_cluster += (s->directory.next - mapping->info.dir.first_dir_index) - * 0x20 / s->cluster_size; + * 0x20 / s->cluster_size; mapping->end = first_cluster; direntry = array_get(&(s->directory), mapping->dir_index); @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, /* add volume label */ { - direntry_t* entry=array_get_next(&(s->directory)); - entry->attributes=0x28; /* archive | volume label */ + direntry_t* entry=array_get_next(&(s->directory)); + entry->attributes=0x28; /* archive | volume label */ memcpy(entry->name, s->volume_label, sizeof(entry->name)); } @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, mapping->path = g_strdup(dirname); i = strlen(mapping->path); if (i > 0 && mapping->path[i - 1] == '/') - mapping->path[i - 1] = '\0'; + mapping->path[i - 1] = '\0'; mapping->mode = MODE_DIRECTORY; mapping->read_only = 0; s->path = mapping->path; for (i = 0, cluster = 0; i < s->mapping.next; i++) { - /* MS-DOS expects the FAT to be 0 for the root directory - * (except for the media byte). */ - /* LATER TODO: still true for FAT32? */ - int fix_fat = (i != 0); - mapping = array_get(&(s->mapping), i); + /* MS-DOS expects the FAT to be 0 for the root directory + * (except for the media byte). */ + /* LATER TODO: still true for FAT32? */ + int fix_fat = (i != 0); + mapping = array_get(&(s->mapping), i); if (mapping->mode & MODE_DIRECTORY) { - mapping->begin = cluster; - if(read_directory(s, i)) { + mapping->begin = cluster; + if(read_directory(s, i)) { error_setg(errp, "Could not read directory %s", mapping->path); - return -1; - } - mapping = array_get(&(s->mapping), i); - } else { - assert(mapping->mode == MODE_UNDEFINED); - mapping->mode=MODE_NORMAL; - mapping->begin = cluster; - if (mapping->end > 0) { - direntry_t* direntry = array_get(&(s->directory), - mapping->dir_index); - - mapping->end = cluster + 1 + (mapping->end-1)/s->cluster_size; - set_begin_of_direntry(direntry, mapping->begin); - } else { - mapping->end = cluster + 1; - fix_fat = 0; - } - } - - assert(mapping->begin < mapping->end); - - /* next free cluster */ - cluster = mapping->end; - - if(cluster > s->cluster_count) { + return -1; + } + mapping = array_get(&(s->mapping), i); + } else { + assert(mapping->mode == MODE_UNDEFINED); + mapping->mode=MODE_NORMAL; + mapping->begin = cluster; + if (mapping->end > 0) { + direntry_t* direntry = array_get(&(s->directory), + mapping->dir_index); + + mapping->end = cluster + 1 + (mapping->end-1)/s->cluster_size; + set_begin_of_direntry(direntry, mapping->begin); + } else { + mapping->end = cluster + 1; + fix_fat = 0; + } + } + + assert(mapping->begin < mapping->end); + + /* next free cluster */ + cluster = mapping->end; + + if(cluster > s->cluster_count) { error_setg(errp, "Directory does not fit in FAT%d (capacity %.2f MB)", s->fat_type, s->sector_count / 2000.0); return -1; - } + } - /* fix fat for entry */ - if (fix_fat) { - int j; - for(j = mapping->begin; j < mapping->end - 1; j++) - fat_set(s, j, j+1); - fat_set(s, mapping->end - 1, s->max_fat_value); - } + /* fix fat for entry */ + if (fix_fat) { + int j; + for(j = mapping->begin; j < mapping->end - 1; j++) + fat_set(s, j, j+1); + fat_set(s, mapping->end - 1, s->max_fat_value); + } } mapping = array_get(&(s->mapping), 0); @@ -XXX,XX +XXX,XX @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags, switch (s->fat_type) { case 32: - fprintf(stderr, "Big fat greek warning: FAT32 has not been tested. " + fprintf(stderr, "Big fat greek warning: FAT32 has not been tested. " "You are welcome to do so!\n"); break; case 16: @@ -XXX,XX +XXX,XX @@ static void vvfat_refresh_limits(BlockDriverState *bs, Error **errp) static inline void vvfat_close_current_file(BDRVVVFATState *s) { if(s->current_mapping) { - s->current_mapping = NULL; - if (s->current_fd) { - qemu_close(s->current_fd); - s->current_fd = 0; - } + s->current_mapping = NULL; + if (s->current_fd) { + qemu_close(s->current_fd); + s->current_fd = 0; + } } s->current_cluster = -1; } @@ -XXX,XX +XXX,XX @@ static inline int find_mapping_for_cluster_aux(BDRVVVFATState* s,int cluster_num { while(1) { int index3; - mapping_t* mapping; - index3=(index1+index2)/2; - mapping=array_get(&(s->mapping),index3); - assert(mapping->begin < mapping->end); - if(mapping->begin>=cluster_num) { - assert(index2!=index3 || index2==0); - if(index2==index3) - return index1; - index2=index3; - } else { - if(index1==index3) - return mapping->end<=cluster_num ? index2 : index1; - index1=index3; - } - assert(index1<=index2); - DLOG(mapping=array_get(&(s->mapping),index1); - assert(mapping->begin<=cluster_num); - assert(index2 >= s->mapping.next || - ((mapping = array_get(&(s->mapping),index2)) && - mapping->end>cluster_num))); + mapping_t* mapping; + index3=(index1+index2)/2; + mapping=array_get(&(s->mapping),index3); + assert(mapping->begin < mapping->end); + if(mapping->begin>=cluster_num) { + assert(index2!=index3 || index2==0); + if(index2==index3) + return index1; + index2=index3; + } else { + if(index1==index3) + return mapping->end<=cluster_num ? index2 : index1; + index1=index3; + } + assert(index1<=index2); + DLOG(mapping=array_get(&(s->mapping),index1); + assert(mapping->begin<=cluster_num); + assert(index2 >= s->mapping.next || + ((mapping = array_get(&(s->mapping),index2)) && + mapping->end>cluster_num))); } } @@ -XXX,XX +XXX,XX @@ static inline mapping_t* find_mapping_for_cluster(BDRVVVFATState* s,int cluster_ static int open_file(BDRVVVFATState* s,mapping_t* mapping) { if(!mapping) - return -1; + return -1; if(!s->current_mapping || - strcmp(s->current_mapping->path,mapping->path)) { - /* open file */ - int fd = qemu_open(mapping->path, O_RDONLY | O_BINARY | O_LARGEFILE); - if(fd<0) - return -1; - vvfat_close_current_file(s); - s->current_fd = fd; - s->current_mapping = mapping; + strcmp(s->current_mapping->path,mapping->path)) { + /* open file */ + int fd = qemu_open(mapping->path, O_RDONLY | O_BINARY | O_LARGEFILE); + if(fd<0) + return -1; + vvfat_close_current_file(s); + s->current_fd = fd; + s->current_mapping = mapping; } return 0; } @@ -XXX,XX +XXX,XX @@ static int open_file(BDRVVVFATState* s,mapping_t* mapping) static inline int read_cluster(BDRVVVFATState *s,int cluster_num) { if(s->current_cluster != cluster_num) { - int result=0; - off_t offset; - assert(!s->current_mapping || s->current_fd || (s->current_mapping->mode & MODE_DIRECTORY)); - if(!s->current_mapping - || s->current_mapping->begin>cluster_num - || s->current_mapping->end<=cluster_num) { - /* binary search of mappings for file */ - mapping_t* mapping=find_mapping_for_cluster(s,cluster_num); - - assert(!mapping || (cluster_num>=mapping->begin && cluster_num<mapping->end)); - - if (mapping && mapping->mode & MODE_DIRECTORY) { - vvfat_close_current_file(s); - s->current_mapping = mapping; + int result=0; + off_t offset; + assert(!s->current_mapping || s->current_fd || (s->current_mapping->mode & MODE_DIRECTORY)); + if(!s->current_mapping + || s->current_mapping->begin>cluster_num + || s->current_mapping->end<=cluster_num) { + /* binary search of mappings for file */ + mapping_t* mapping=find_mapping_for_cluster(s,cluster_num); + + assert(!mapping || (cluster_num>=mapping->begin && cluster_num<mapping->end)); + + if (mapping && mapping->mode & MODE_DIRECTORY) { + vvfat_close_current_file(s); + s->current_mapping = mapping; read_cluster_directory: - offset = s->cluster_size*(cluster_num-s->current_mapping->begin); - s->cluster = (unsigned char*)s->directory.pointer+offset - + 0x20*s->current_mapping->info.dir.first_dir_index; - assert(((s->cluster-(unsigned char*)s->directory.pointer)%s->cluster_size)==0); - assert((char*)s->cluster+s->cluster_size <= s->directory.pointer+s->directory.next*s->directory.item_size); - s->current_cluster = cluster_num; - return 0; - } - - if(open_file(s,mapping)) - return -2; - } else if (s->current_mapping->mode & MODE_DIRECTORY) - goto read_cluster_directory; - - assert(s->current_fd); - - offset=s->cluster_size*(cluster_num-s->current_mapping->begin)+s->current_mapping->info.file.offset; - if(lseek(s->current_fd, offset, SEEK_SET)!=offset) - return -3; - s->cluster=s->cluster_buffer; - result=read(s->current_fd,s->cluster,s->cluster_size); - if(result<0) { - s->current_cluster = -1; - return -1; - } - s->current_cluster = cluster_num; + offset = s->cluster_size*(cluster_num-s->current_mapping->begin); + s->cluster = (unsigned char*)s->directory.pointer+offset + + 0x20*s->current_mapping->info.dir.first_dir_index; + assert(((s->cluster-(unsigned char*)s->directory.pointer)%s->cluster_size)==0); + assert((char*)s->cluster+s->cluster_size <= s->directory.pointer+s->directory.next*s->directory.item_size); + s->current_cluster = cluster_num; + return 0; + } + + if(open_file(s,mapping)) + return -2; + } else if (s->current_mapping->mode & MODE_DIRECTORY) + goto read_cluster_directory; + + assert(s->current_fd); + + offset=s->cluster_size*(cluster_num-s->current_mapping->begin)+s->current_mapping->info.file.offset; + if(lseek(s->current_fd, offset, SEEK_SET)!=offset) + return -3; + s->cluster=s->cluster_buffer; + result=read(s->current_fd,s->cluster,s->cluster_size); + if(result<0) { + s->current_cluster = -1; + return -1; + } + s->current_cluster = cluster_num; } return 0; } @@ -XXX,XX +XXX,XX @@ static void print_direntry(const direntry_t* direntry) fprintf(stderr, "direntry %p: ", direntry); if(!direntry) - return; + return; if(is_long_name(direntry)) { - unsigned char* c=(unsigned char*)direntry; - int i; - for(i=1;i<11 && c[i] && c[i]!=0xff;i+=2) + unsigned char* c=(unsigned char*)direntry; + int i; + for(i=1;i<11 && c[i] && c[i]!=0xff;i+=2) #define ADD_CHAR(c) {buffer[j] = (c); if (buffer[j] < ' ') buffer[j] = 0xb0; j++;} - ADD_CHAR(c[i]); - for(i=14;i<26 && c[i] && c[i]!=0xff;i+=2) - ADD_CHAR(c[i]); - for(i=28;i<32 && c[i] && c[i]!=0xff;i+=2) - ADD_CHAR(c[i]); - buffer[j] = 0; - fprintf(stderr, "%s\n", buffer); + ADD_CHAR(c[i]); + for(i=14;i<26 && c[i] && c[i]!=0xff;i+=2) + ADD_CHAR(c[i]); + for(i=28;i<32 && c[i] && c[i]!=0xff;i+=2) + ADD_CHAR(c[i]); + buffer[j] = 0; + fprintf(stderr, "%s\n", buffer); } else { - int i; - for(i=0;i<11;i++) - ADD_CHAR(direntry->name[i]); - buffer[j] = 0; - fprintf(stderr,"%s attributes=0x%02x begin=%d size=%d\n", - buffer, - direntry->attributes, - begin_of_direntry(direntry),le32_to_cpu(direntry->size)); + int i; + for(i=0;i<11;i++) + ADD_CHAR(direntry->name[i]); + buffer[j] = 0; + fprintf(stderr,"%s attributes=0x%02x begin=%d size=%d\n", + buffer, + direntry->attributes, + begin_of_direntry(direntry),le32_to_cpu(direntry->size)); } } @@ -XXX,XX +XXX,XX @@ static void print_mapping(const mapping_t* mapping) mapping->first_mapping_index, mapping->path, mapping->mode); if (mapping->mode & MODE_DIRECTORY) - fprintf(stderr, "parent_mapping_index = %d, first_dir_index = %d\n", mapping->info.dir.parent_mapping_index, mapping->info.dir.first_dir_index); + fprintf(stderr, "parent_mapping_index = %d, first_dir_index = %d\n", mapping->info.dir.parent_mapping_index, mapping->info.dir.first_dir_index); else - fprintf(stderr, "offset = %d\n", mapping->info.file.offset); + fprintf(stderr, "offset = %d\n", mapping->info.file.offset); } #endif @@ -XXX,XX +XXX,XX @@ static int vvfat_read(BlockDriverState *bs, int64_t sector_num, int i; for(i=0;i<nb_sectors;i++,sector_num++) { - if (sector_num >= bs->total_sectors) - return -1; - if (s->qcow) { - int n; + if (sector_num >= bs->total_sectors) + return -1; + if (s->qcow) { + int n; int ret; ret = bdrv_is_allocated(s->qcow->bs, sector_num, nb_sectors - i, &n); @@ -XXX,XX +XXX,XX @@ static int vvfat_read(BlockDriverState *bs, int64_t sector_num, continue; } DLOG(fprintf(stderr, "sector %d not allocated\n", (int)sector_num)); - } - if(sector_num<s->faked_sectors) { - if(sector_num<s->first_sectors_number) - memcpy(buf+i*0x200,&(s->first_sectors[sector_num*0x200]),0x200); - else if(sector_num-s->first_sectors_number<s->sectors_per_fat) - memcpy(buf+i*0x200,&(s->fat.pointer[(sector_num-s->first_sectors_number)*0x200]),0x200); - else if(sector_num-s->first_sectors_number-s->sectors_per_fat<s->sectors_per_fat) - memcpy(buf+i*0x200,&(s->fat.pointer[(sector_num-s->first_sectors_number-s->sectors_per_fat)*0x200]),0x200); - } else { - uint32_t sector=sector_num-s->faked_sectors, - sector_offset_in_cluster=(sector%s->sectors_per_cluster), - cluster_num=sector/s->sectors_per_cluster; - if(cluster_num > s->cluster_count || read_cluster(s, cluster_num) != 0) { - /* LATER TODO: strict: return -1; */ - memset(buf+i*0x200,0,0x200); - continue; - } - memcpy(buf+i*0x200,s->cluster+sector_offset_in_cluster*0x200,0x200); - } + } + if(sector_num<s->faked_sectors) { + if(sector_num<s->first_sectors_number) + memcpy(buf+i*0x200,&(s->first_sectors[sector_num*0x200]),0x200); + else if(sector_num-s->first_sectors_number<s->sectors_per_fat) + memcpy(buf+i*0x200,&(s->fat.pointer[(sector_num-s->first_sectors_number)*0x200]),0x200); + else if(sector_num-s->first_sectors_number-s->sectors_per_fat<s->sectors_per_fat) + memcpy(buf+i*0x200,&(s->fat.pointer[(sector_num-s->first_sectors_number-s->sectors_per_fat)*0x200]),0x200); + } else { + uint32_t sector=sector_num-s->faked_sectors, + sector_offset_in_cluster=(sector%s->sectors_per_cluster), + cluster_num=sector/s->sectors_per_cluster; + if(cluster_num > s->cluster_count || read_cluster(s, cluster_num) != 0) { + /* LATER TODO: strict: return -1; */ + memset(buf+i*0x200,0,0x200); + continue; + } + memcpy(buf+i*0x200,s->cluster+sector_offset_in_cluster*0x200,0x200); + } } return 0; } @@ -XXX,XX +XXX,XX @@ vvfat_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes, typedef struct commit_t { char* path; union { - struct { uint32_t cluster; } rename; - struct { int dir_index; uint32_t modified_offset; } writeout; - struct { uint32_t first_cluster; } new_file; - struct { uint32_t cluster; } mkdir; + struct { uint32_t cluster; } rename; + struct { int dir_index; uint32_t modified_offset; } writeout; + struct { uint32_t first_cluster; } new_file; + struct { uint32_t cluster; } mkdir; } param; /* DELETEs and RMDIRs are handled differently: see handle_deletes() */ enum { - ACTION_RENAME, ACTION_WRITEOUT, ACTION_NEW_FILE, ACTION_MKDIR + ACTION_RENAME, ACTION_WRITEOUT, ACTION_NEW_FILE, ACTION_MKDIR } action; } commit_t; @@ -XXX,XX +XXX,XX @@ static void clear_commits(BDRVVVFATState* s) int i; DLOG(fprintf(stderr, "clear_commits (%d commits)\n", s->commits.next)); for (i = 0; i < s->commits.next; i++) { - commit_t* commit = array_get(&(s->commits), i); - assert(commit->path || commit->action == ACTION_WRITEOUT); - if (commit->action != ACTION_WRITEOUT) { - assert(commit->path); + commit_t* commit = array_get(&(s->commits), i); + assert(commit->path || commit->action == ACTION_WRITEOUT); + if (commit->action != ACTION_WRITEOUT) { + assert(commit->path); g_free(commit->path); - } else - assert(commit->path == NULL); + } else + assert(commit->path == NULL); } s->commits.next = 0; } static void schedule_rename(BDRVVVFATState* s, - uint32_t cluster, char* new_path) + uint32_t cluster, char* new_path) { commit_t* commit = array_get_next(&(s->commits)); commit->path = new_path; @@ -XXX,XX +XXX,XX @@ static void schedule_rename(BDRVVVFATState* s, } static void schedule_writeout(BDRVVVFATState* s, - int dir_index, uint32_t modified_offset) + int dir_index, uint32_t modified_offset) { commit_t* commit = array_get_next(&(s->commits)); commit->path = NULL; @@ -XXX,XX +XXX,XX @@ static void schedule_writeout(BDRVVVFATState* s, } static void schedule_new_file(BDRVVVFATState* s, - char* path, uint32_t first_cluster) + char* path, uint32_t first_cluster) { commit_t* commit = array_get_next(&(s->commits)); commit->path = path; @@ -XXX,XX +XXX,XX @@ static void lfn_init(long_file_name* lfn) /* return 0 if parsed successfully, > 0 if no long name, < 0 if error */ static int parse_long_name(long_file_name* lfn, - const direntry_t* direntry) + const direntry_t* direntry) { int i, j, offset; const unsigned char* pointer = (const unsigned char*)direntry; if (!is_long_name(direntry)) - return 1; + return 1; if (pointer[0] & 0x40) { - lfn->sequence_number = pointer[0] & 0x3f; - lfn->checksum = pointer[13]; - lfn->name[0] = 0; - lfn->name[lfn->sequence_number * 13] = 0; + lfn->sequence_number = pointer[0] & 0x3f; + lfn->checksum = pointer[13]; + lfn->name[0] = 0; + lfn->name[lfn->sequence_number * 13] = 0; } else if ((pointer[0] & 0x3f) != --lfn->sequence_number) - return -1; + return -1; else if (pointer[13] != lfn->checksum) - return -2; + return -2; else if (pointer[12] || pointer[26] || pointer[27]) - return -3; + return -3; offset = 13 * (lfn->sequence_number - 1); for (i = 0, j = 1; i < 13; i++, j+=2) { - if (j == 11) - j = 14; - else if (j == 26) - j = 28; + if (j == 11) + j = 14; + else if (j == 26) + j = 28; - if (pointer[j+1] == 0) - lfn->name[offset + i] = pointer[j]; - else if (pointer[j+1] != 0xff || (pointer[0] & 0x40) == 0) - return -4; - else - lfn->name[offset + i] = 0; + if (pointer[j+1] == 0) + lfn->name[offset + i] = pointer[j]; + else if (pointer[j+1] != 0xff || (pointer[0] & 0x40) == 0) + return -4; + else + lfn->name[offset + i] = 0; } if (pointer[0] & 0x40) - lfn->len = offset + strlen((char*)lfn->name + offset); + lfn->len = offset + strlen((char*)lfn->name + offset); return 0; } /* returns 0 if successful, >0 if no short_name, and <0 on error */ static int parse_short_name(BDRVVVFATState* s, - long_file_name* lfn, direntry_t* direntry) + long_file_name* lfn, direntry_t* direntry) { int i, j; if (!is_short_name(direntry)) - return 1; + return 1; for (j = 7; j >= 0 && direntry->name[j] == ' '; j--); for (i = 0; i <= j; i++) { - if (direntry->name[i] <= ' ' || direntry->name[i] > 0x7f) - return -1; - else if (s->downcase_short_names) - lfn->name[i] = qemu_tolower(direntry->name[i]); - else - lfn->name[i] = direntry->name[i]; + if (direntry->name[i] <= ' ' || direntry->name[i] > 0x7f) + return -1; + else if (s->downcase_short_names) + lfn->name[i] = qemu_tolower(direntry->name[i]); + else + lfn->name[i] = direntry->name[i]; } for (j = 2; j >= 0 && direntry->name[8 + j] == ' '; j--) { } if (j >= 0) { - lfn->name[i++] = '.'; - lfn->name[i + j + 1] = '\0'; - for (;j >= 0; j--) { + lfn->name[i++] = '.'; + lfn->name[i + j + 1] = '\0'; + for (;j >= 0; j--) { uint8_t c = direntry->name[8 + j]; if (c <= ' ' || c > 0x7f) { return -2; @@ -XXX,XX +XXX,XX @@ static int parse_short_name(BDRVVVFATState* s, } else { lfn->name[i + j] = c; } - } + } } else - lfn->name[i + j + 1] = '\0'; + lfn->name[i + j + 1] = '\0'; lfn->len = strlen((char*)lfn->name); @@ -XXX,XX +XXX,XX @@ static int parse_short_name(BDRVVVFATState* s, } static inline uint32_t modified_fat_get(BDRVVVFATState* s, - unsigned int cluster) + unsigned int cluster) { if (cluster < s->last_cluster_of_root_directory) { - if (cluster + 1 == s->last_cluster_of_root_directory) - return s->max_fat_value; - else - return cluster + 1; + if (cluster + 1 == s->last_cluster_of_root_directory) + return s->max_fat_value; + else + return cluster + 1; } if (s->fat_type==32) { @@ -XXX,XX +XXX,XX @@ static const char* get_basename(const char* path) { char* basename = strrchr(path, '/'); if (basename == NULL) - return path; + return path; else - return basename + 1; /* strip '/' */ + return basename + 1; /* strip '/' */ } /* @@ -XXX,XX +XXX,XX @@ typedef enum { * assumed to be *not* deleted (and *only* those). */ static uint32_t get_cluster_count_for_direntry(BDRVVVFATState* s, - direntry_t* direntry, const char* path) + direntry_t* direntry, const char* path) { /* * This is a little bit tricky: @@ -XXX,XX +XXX,XX @@ static uint32_t get_cluster_count_for_direntry(BDRVVVFATState* s, /* the root directory */ if (cluster_num == 0) - return 0; + return 0; /* write support */ if (s->qcow) { - basename2 = get_basename(path); + basename2 = get_basename(path); - mapping = find_mapping_for_cluster(s, cluster_num); + mapping = find_mapping_for_cluster(s, cluster_num); - if (mapping) { - const char* basename; + if (mapping) { + const char* basename; - assert(mapping->mode & MODE_DELETED); - mapping->mode &= ~MODE_DELETED; + assert(mapping->mode & MODE_DELETED); + mapping->mode &= ~MODE_DELETED; - basename = get_basename(mapping->path); + basename = get_basename(mapping->path); - assert(mapping->mode & MODE_NORMAL); + assert(mapping->mode & MODE_NORMAL); - /* rename */ - if (strcmp(basename, basename2)) - schedule_rename(s, cluster_num, g_strdup(path)); - } else if (is_file(direntry)) - /* new file */ - schedule_new_file(s, g_strdup(path), cluster_num); - else { + /* rename */ + if (strcmp(basename, basename2)) + schedule_rename(s, cluster_num, g_strdup(path)); + } else if (is_file(direntry)) + /* new file */ + schedule_new_file(s, g_strdup(path), cluster_num); + else { abort(); - return 0; - } + return 0; + } } while(1) { - if (s->qcow) { - if (!copy_it && cluster_was_modified(s, cluster_num)) { - if (mapping == NULL || - mapping->begin > cluster_num || - mapping->end <= cluster_num) - mapping = find_mapping_for_cluster(s, cluster_num); + if (s->qcow) { + if (!copy_it && cluster_was_modified(s, cluster_num)) { + if (mapping == NULL || + mapping->begin > cluster_num || + mapping->end <= cluster_num) + mapping = find_mapping_for_cluster(s, cluster_num); - if (mapping && - (mapping->mode & MODE_DIRECTORY) == 0) { + if (mapping && + (mapping->mode & MODE_DIRECTORY) == 0) { - /* was modified in qcow */ - if (offset != mapping->info.file.offset + s->cluster_size - * (cluster_num - mapping->begin)) { - /* offset of this cluster in file chain has changed */ + /* was modified in qcow */ + if (offset != mapping->info.file.offset + s->cluster_size + * (cluster_num - mapping->begin)) { + /* offset of this cluster in file chain has changed */ abort(); - copy_it = 1; - } else if (offset == 0) { - const char* basename = get_basename(mapping->path); + copy_it = 1; + } else if (offset == 0) { + const char* basename = get_basename(mapping->path); - if (strcmp(basename, basename2)) - copy_it = 1; - first_mapping_index = array_index(&(s->mapping), mapping); - } + if (strcmp(basename, basename2)) + copy_it = 1; + first_mapping_index = array_index(&(s->mapping), mapping); + } - if (mapping->first_mapping_index != first_mapping_index - && mapping->info.file.offset > 0) { + if (mapping->first_mapping_index != first_mapping_index + && mapping->info.file.offset > 0) { abort(); - copy_it = 1; - } - - /* need to write out? */ - if (!was_modified && is_file(direntry)) { - was_modified = 1; - schedule_writeout(s, mapping->dir_index, offset); - } - } - } - - if (copy_it) { - int i, dummy; - /* - * This is horribly inefficient, but that is okay, since - * it is rarely executed, if at all. - */ - int64_t offset = cluster2sector(s, cluster_num); - - vvfat_close_current_file(s); + copy_it = 1; + } + + /* need to write out? */ + if (!was_modified && is_file(direntry)) { + was_modified = 1; + schedule_writeout(s, mapping->dir_index, offset); + } + } + } + + if (copy_it) { + int i, dummy; + /* + * This is horribly inefficient, but that is okay, since + * it is rarely executed, if at all. + */ + int64_t offset = cluster2sector(s, cluster_num); + + vvfat_close_current_file(s); for (i = 0; i < s->sectors_per_cluster; i++) { int res; @@ -XXX,XX +XXX,XX @@ static uint32_t get_cluster_count_for_direntry(BDRVVVFATState* s, } } } - } - } + } + } - ret++; - if (s->used_clusters[cluster_num] & USED_ANY) - return 0; - s->used_clusters[cluster_num] = USED_FILE; + ret++; + if (s->used_clusters[cluster_num] & USED_ANY) + return 0; + s->used_clusters[cluster_num] = USED_FILE; - cluster_num = modified_fat_get(s, cluster_num); + cluster_num = modified_fat_get(s, cluster_num); - if (fat_eof(s, cluster_num)) - return ret; - else if (cluster_num < 2 || cluster_num > s->max_fat_value - 16) - return -1; + if (fat_eof(s, cluster_num)) + return ret; + else if (cluster_num < 2 || cluster_num > s->max_fat_value - 16) + return -1; - offset += s->cluster_size; + offset += s->cluster_size; } } @@ -XXX,XX +XXX,XX @@ static uint32_t get_cluster_count_for_direntry(BDRVVVFATState* s, * used by the directory, its subdirectories and their files. */ static int check_directory_consistency(BDRVVVFATState *s, - int cluster_num, const char* path) + int cluster_num, const char* path) { int ret = 0; unsigned char* cluster = g_malloc(s->cluster_size); @@ -XXX,XX +XXX,XX @@ static int check_directory_consistency(BDRVVVFATState *s, path2[path_len + 1] = '\0'; if (mapping) { - const char* basename = get_basename(mapping->path); - const char* basename2 = get_basename(path); + const char* basename = get_basename(mapping->path); + const char* basename2 = get_basename(path); - assert(mapping->mode & MODE_DIRECTORY); + assert(mapping->mode & MODE_DIRECTORY); - assert(mapping->mode & MODE_DELETED); - mapping->mode &= ~MODE_DELETED; + assert(mapping->mode & MODE_DELETED); + mapping->mode &= ~MODE_DELETED; - if (strcmp(basename, basename2)) - schedule_rename(s, cluster_num, g_strdup(path)); + if (strcmp(basename, basename2)) + schedule_rename(s, cluster_num, g_strdup(path)); } else - /* new directory */ - schedule_mkdir(s, cluster_num, g_strdup(path)); + /* new directory */ + schedule_mkdir(s, cluster_num, g_strdup(path)); lfn_init(&lfn); do { - int i; - int subret = 0; + int i; + int subret = 0; - ret++; + ret++; - if (s->used_clusters[cluster_num] & USED_ANY) { - fprintf(stderr, "cluster %d used more than once\n", (int)cluster_num); + if (s->used_clusters[cluster_num] & USED_ANY) { + fprintf(stderr, "cluster %d used more than once\n", (int)cluster_num); goto fail; - } - s->used_clusters[cluster_num] = USED_DIRECTORY; + } + s->used_clusters[cluster_num] = USED_DIRECTORY; DLOG(fprintf(stderr, "read cluster %d (sector %d)\n", (int)cluster_num, (int)cluster2sector(s, cluster_num))); - subret = vvfat_read(s->bs, cluster2sector(s, cluster_num), cluster, - s->sectors_per_cluster); - if (subret) { - fprintf(stderr, "Error fetching direntries\n"); - fail: + subret = vvfat_read(s->bs, cluster2sector(s, cluster_num), cluster, + s->sectors_per_cluster); + if (subret) { + fprintf(stderr, "Error fetching direntries\n"); + fail: g_free(cluster); - return 0; - } + return 0; + } - for (i = 0; i < 0x10 * s->sectors_per_cluster; i++) { - int cluster_count = 0; + for (i = 0; i < 0x10 * s->sectors_per_cluster; i++) { + int cluster_count = 0; DLOG(fprintf(stderr, "check direntry %d:\n", i); print_direntry(direntries + i)); - if (is_volume_label(direntries + i) || is_dot(direntries + i) || - is_free(direntries + i)) - continue; - - subret = parse_long_name(&lfn, direntries + i); - if (subret < 0) { - fprintf(stderr, "Error in long name\n"); - goto fail; - } - if (subret == 0 || is_free(direntries + i)) - continue; - - if (fat_chksum(direntries+i) != lfn.checksum) { - subret = parse_short_name(s, &lfn, direntries + i); - if (subret < 0) { - fprintf(stderr, "Error in short name (%d)\n", subret); - goto fail; - } - if (subret > 0 || !strcmp((char*)lfn.name, ".") - || !strcmp((char*)lfn.name, "..")) - continue; - } - lfn.checksum = 0x100; /* cannot use long name twice */ - - if (path_len + 1 + lfn.len >= PATH_MAX) { - fprintf(stderr, "Name too long: %s/%s\n", path, lfn.name); - goto fail; - } + if (is_volume_label(direntries + i) || is_dot(direntries + i) || + is_free(direntries + i)) + continue; + + subret = parse_long_name(&lfn, direntries + i); + if (subret < 0) { + fprintf(stderr, "Error in long name\n"); + goto fail; + } + if (subret == 0 || is_free(direntries + i)) + continue; + + if (fat_chksum(direntries+i) != lfn.checksum) { + subret = parse_short_name(s, &lfn, direntries + i); + if (subret < 0) { + fprintf(stderr, "Error in short name (%d)\n", subret); + goto fail; + } + if (subret > 0 || !strcmp((char*)lfn.name, ".") + || !strcmp((char*)lfn.name, "..")) + continue; + } + lfn.checksum = 0x100; /* cannot use long name twice */ + + if (path_len + 1 + lfn.len >= PATH_MAX) { + fprintf(stderr, "Name too long: %s/%s\n", path, lfn.name); + goto fail; + } pstrcpy(path2 + path_len + 1, sizeof(path2) - path_len - 1, (char*)lfn.name); - if (is_directory(direntries + i)) { - if (begin_of_direntry(direntries + i) == 0) { - DLOG(fprintf(stderr, "invalid begin for directory: %s\n", path2); print_direntry(direntries + i)); - goto fail; - } - cluster_count = check_directory_consistency(s, - begin_of_direntry(direntries + i), path2); - if (cluster_count == 0) { - DLOG(fprintf(stderr, "problem in directory %s:\n", path2); print_direntry(direntries + i)); - goto fail; - } - } else if (is_file(direntries + i)) { - /* check file size with FAT */ - cluster_count = get_cluster_count_for_direntry(s, direntries + i, path2); - if (cluster_count != + if (is_directory(direntries + i)) { + if (begin_of_direntry(direntries + i) == 0) { + DLOG(fprintf(stderr, "invalid begin for directory: %s\n", path2); print_direntry(direntries + i)); + goto fail; + } + cluster_count = check_directory_consistency(s, + begin_of_direntry(direntries + i), path2); + if (cluster_count == 0) { + DLOG(fprintf(stderr, "problem in directory %s:\n", path2); print_direntry(direntries + i)); + goto fail; + } + } else if (is_file(direntries + i)) { + /* check file size with FAT */ + cluster_count = get_cluster_count_for_direntry(s, direntries + i, path2); + if (cluster_count != DIV_ROUND_UP(le32_to_cpu(direntries[i].size), s->cluster_size)) { - DLOG(fprintf(stderr, "Cluster count mismatch\n")); - goto fail; - } - } else + DLOG(fprintf(stderr, "Cluster count mismatch\n")); + goto fail; + } + } else abort(); /* cluster_count = 0; */ - ret += cluster_count; - } + ret += cluster_count; + } - cluster_num = modified_fat_get(s, cluster_num); + cluster_num = modified_fat_get(s, cluster_num); } while(!fat_eof(s, cluster_num)); g_free(cluster); @@ -XXX,XX +XXX,XX @@ DLOG(checkpoint()); * - if all is fine, return number of used clusters */ if (s->fat2 == NULL) { - int size = 0x200 * s->sectors_per_fat; - s->fat2 = g_malloc(size); - memcpy(s->fat2, s->fat.pointer, size); + int size = 0x200 * s->sectors_per_fat; + s->fat2 = g_malloc(size); + memcpy(s->fat2, s->fat.pointer, size); } check = vvfat_read(s->bs, - s->first_sectors_number, s->fat2, s->sectors_per_fat); + s->first_sectors_number, s->fat2, s->sectors_per_fat); if (check) { - fprintf(stderr, "Could not copy fat\n"); - return 0; + fprintf(stderr, "Could not copy fat\n"); + return 0; } assert (s->used_clusters); for (i = 0; i < sector2cluster(s, s->sector_count); i++) - s->used_clusters[i] &= ~USED_ANY; + s->used_clusters[i] &= ~USED_ANY; clear_commits(s); /* mark every mapped file/directory as deleted. * (check_directory_consistency() will unmark those still present). */ if (s->qcow) - for (i = 0; i < s->mapping.next; i++) { - mapping_t* mapping = array_get(&(s->mapping), i); - if (mapping->first_mapping_index < 0) - mapping->mode |= MODE_DELETED; - } + for (i = 0; i < s->mapping.next; i++) { + mapping_t* mapping = array_get(&(s->mapping), i); + if (mapping->first_mapping_index < 0) + mapping->mode |= MODE_DELETED; + } used_clusters_count = check_directory_consistency(s, 0, s->path); if (used_clusters_count <= 0) { - DLOG(fprintf(stderr, "problem in directory\n")); - return 0; + DLOG(fprintf(stderr, "problem in directory\n")); + return 0; } check = s->last_cluster_of_root_directory; for (i = check; i < sector2cluster(s, s->sector_count); i++) { - if (modified_fat_get(s, i)) { - if(!s->used_clusters[i]) { - DLOG(fprintf(stderr, "FAT was modified (%d), but cluster is not used?\n", i)); - return 0; - } - check++; - } + if (modified_fat_get(s, i)) { + if(!s->used_clusters[i]) { + DLOG(fprintf(stderr, "FAT was modified (%d), but cluster is not used?\n", i)); + return 0; + } + check++; + } - if (s->used_clusters[i] == USED_ALLOCATED) { - /* allocated, but not used... */ - DLOG(fprintf(stderr, "unused, modified cluster: %d\n", i)); - return 0; - } + if (s->used_clusters[i] == USED_ALLOCATED) { + /* allocated, but not used... */ + DLOG(fprintf(stderr, "unused, modified cluster: %d\n", i)); + return 0; + } } if (check != used_clusters_count) - return 0; + return 0; return used_clusters_count; } static inline void adjust_mapping_indices(BDRVVVFATState* s, - int offset, int adjust) + int offset, int adjust) { int i; for (i = 0; i < s->mapping.next; i++) { - mapping_t* mapping = array_get(&(s->mapping), i); + mapping_t* mapping = array_get(&(s->mapping), i); #define ADJUST_MAPPING_INDEX(name) \ - if (mapping->name >= offset) \ - mapping->name += adjust + if (mapping->name >= offset) \ + mapping->name += adjust - ADJUST_MAPPING_INDEX(first_mapping_index); - if (mapping->mode & MODE_DIRECTORY) - ADJUST_MAPPING_INDEX(info.dir.parent_mapping_index); + ADJUST_MAPPING_INDEX(first_mapping_index); + if (mapping->mode & MODE_DIRECTORY) + ADJUST_MAPPING_INDEX(info.dir.parent_mapping_index); } } /* insert or update mapping */ static mapping_t* insert_mapping(BDRVVVFATState* s, - uint32_t begin, uint32_t end) + uint32_t begin, uint32_t end) { /* * - find mapping where mapping->begin >= begin, @@ -XXX,XX +XXX,XX @@ static mapping_t* insert_mapping(BDRVVVFATState* s, mapping_t* first_mapping = array_get(&(s->mapping), 0); if (index < s->mapping.next && (mapping = array_get(&(s->mapping), index)) - && mapping->begin < begin) { - mapping->end = begin; - index++; - mapping = array_get(&(s->mapping), index); + && mapping->begin < begin) { + mapping->end = begin; + index++; + mapping = array_get(&(s->mapping), index); } if (index >= s->mapping.next || mapping->begin > begin) { - mapping = array_insert(&(s->mapping), index, 1); - mapping->path = NULL; - adjust_mapping_indices(s, index, +1); + mapping = array_insert(&(s->mapping), index, 1); + mapping->path = NULL; + adjust_mapping_indices(s, index, +1); } mapping->begin = begin; @@ -XXX,XX +XXX,XX @@ assert(index + 1 >= s->mapping.next || next_mapping->begin >= end))); if (s->current_mapping && first_mapping != (mapping_t*)s->mapping.pointer) - s->current_mapping = array_get(&(s->mapping), - s->current_mapping - first_mapping); + s->current_mapping = array_get(&(s->mapping), + s->current_mapping - first_mapping); return mapping; } @@ -XXX,XX +XXX,XX @@ static int remove_mapping(BDRVVVFATState* s, int mapping_index) adjust_mapping_indices(s, mapping_index, -1); if (s->current_mapping && first_mapping != (mapping_t*)s->mapping.pointer) - s->current_mapping = array_get(&(s->mapping), - s->current_mapping - first_mapping); + s->current_mapping = array_get(&(s->mapping), + s->current_mapping - first_mapping); return 0; } @@ -XXX,XX +XXX,XX @@ static void adjust_dirindices(BDRVVVFATState* s, int offset, int adjust) { int i; for (i = 0; i < s->mapping.next; i++) { - mapping_t* mapping = array_get(&(s->mapping), i); - if (mapping->dir_index >= offset) - mapping->dir_index += adjust; - if ((mapping->mode & MODE_DIRECTORY) && - mapping->info.dir.first_dir_index >= offset) - mapping->info.dir.first_dir_index += adjust; + mapping_t* mapping = array_get(&(s->mapping), i); + if (mapping->dir_index >= offset) + mapping->dir_index += adjust; + if ((mapping->mode & MODE_DIRECTORY) && + mapping->info.dir.first_dir_index >= offset) + mapping->info.dir.first_dir_index += adjust; } } static direntry_t* insert_direntries(BDRVVVFATState* s, - int dir_index, int count) + int dir_index, int count) { /* * make room in s->directory, @@ -XXX,XX +XXX,XX @@ static direntry_t* insert_direntries(BDRVVVFATState* s, */ direntry_t* result = array_insert(&(s->directory), dir_index, count); if (result == NULL) - return NULL; + return NULL; adjust_dirindices(s, dir_index, count); return result; } @@ -XXX,XX +XXX,XX @@ static int remove_direntries(BDRVVVFATState* s, int dir_index, int count) { int ret = array_remove_slice(&(s->directory), dir_index, count); if (ret) - return ret; + return ret; adjust_dirindices(s, dir_index, -count); return 0; } @@ -XXX,XX +XXX,XX @@ static int remove_direntries(BDRVVVFATState* s, int dir_index, int count) * adjusted) */ static int commit_mappings(BDRVVVFATState* s, - uint32_t first_cluster, int dir_index) + uint32_t first_cluster, int dir_index) { mapping_t* mapping = find_mapping_for_cluster(s, first_cluster); direntry_t* direntry = array_get(&(s->directory), dir_index); @@ -XXX,XX +XXX,XX @@ static int commit_mappings(BDRVVVFATState* s, mapping->first_mapping_index = -1; mapping->dir_index = dir_index; mapping->mode = (dir_index <= 0 || is_directory(direntry)) ? - MODE_DIRECTORY : MODE_NORMAL; + MODE_DIRECTORY : MODE_NORMAL; while (!fat_eof(s, cluster)) { - uint32_t c, c1; - - for (c = cluster, c1 = modified_fat_get(s, c); c + 1 == c1; - c = c1, c1 = modified_fat_get(s, c1)); - - c++; - if (c > mapping->end) { - int index = array_index(&(s->mapping), mapping); - int i, max_i = s->mapping.next - index; - for (i = 1; i < max_i && mapping[i].begin < c; i++); - while (--i > 0) - remove_mapping(s, index + 1); - } - assert(mapping == array_get(&(s->mapping), s->mapping.next - 1) - || mapping[1].begin >= c); - mapping->end = c; - - if (!fat_eof(s, c1)) { - int i = find_mapping_for_cluster_aux(s, c1, 0, s->mapping.next); - mapping_t* next_mapping = i >= s->mapping.next ? NULL : - array_get(&(s->mapping), i); - - if (next_mapping == NULL || next_mapping->begin > c1) { - int i1 = array_index(&(s->mapping), mapping); - - next_mapping = insert_mapping(s, c1, c1+1); - - if (c1 < c) - i1++; - mapping = array_get(&(s->mapping), i1); - } - - next_mapping->dir_index = mapping->dir_index; - next_mapping->first_mapping_index = - mapping->first_mapping_index < 0 ? - array_index(&(s->mapping), mapping) : - mapping->first_mapping_index; - next_mapping->path = mapping->path; - next_mapping->mode = mapping->mode; - next_mapping->read_only = mapping->read_only; - if (mapping->mode & MODE_DIRECTORY) { - next_mapping->info.dir.parent_mapping_index = - mapping->info.dir.parent_mapping_index; - next_mapping->info.dir.first_dir_index = - mapping->info.dir.first_dir_index + - 0x10 * s->sectors_per_cluster * - (mapping->end - mapping->begin); - } else - next_mapping->info.file.offset = mapping->info.file.offset + - mapping->end - mapping->begin; - - mapping = next_mapping; - } - - cluster = c1; + uint32_t c, c1; + + for (c = cluster, c1 = modified_fat_get(s, c); c + 1 == c1; + c = c1, c1 = modified_fat_get(s, c1)); + + c++; + if (c > mapping->end) { + int index = array_index(&(s->mapping), mapping); + int i, max_i = s->mapping.next - index; + for (i = 1; i < max_i && mapping[i].begin < c; i++); + while (--i > 0) + remove_mapping(s, index + 1); + } + assert(mapping == array_get(&(s->mapping), s->mapping.next - 1) + || mapping[1].begin >= c); + mapping->end = c; + + if (!fat_eof(s, c1)) { + int i = find_mapping_for_cluster_aux(s, c1, 0, s->mapping.next); + mapping_t* next_mapping = i >= s->mapping.next ? NULL : + array_get(&(s->mapping), i); + + if (next_mapping == NULL || next_mapping->begin > c1) { + int i1 = array_index(&(s->mapping), mapping); + + next_mapping = insert_mapping(s, c1, c1+1); + + if (c1 < c) + i1++; + mapping = array_get(&(s->mapping), i1); + } + + next_mapping->dir_index = mapping->dir_index; + next_mapping->first_mapping_index = + mapping->first_mapping_index < 0 ? + array_index(&(s->mapping), mapping) : + mapping->first_mapping_index; + next_mapping->path = mapping->path; + next_mapping->mode = mapping->mode; + next_mapping->read_only = mapping->read_only; + if (mapping->mode & MODE_DIRECTORY) { + next_mapping->info.dir.parent_mapping_index = + mapping->info.dir.parent_mapping_index; + next_mapping->info.dir.first_dir_index = + mapping->info.dir.first_dir_index + + 0x10 * s->sectors_per_cluster * + (mapping->end - mapping->begin); + } else + next_mapping->info.file.offset = mapping->info.file.offset + + mapping->end - mapping->begin; + + mapping = next_mapping; + } + + cluster = c1; } return 0; } static int commit_direntries(BDRVVVFATState* s, - int dir_index, int parent_mapping_index) + int dir_index, int parent_mapping_index) { direntry_t* direntry = array_get(&(s->directory), dir_index); uint32_t first_cluster = dir_index == 0 ? 0 : begin_of_direntry(direntry); @@ -XXX,XX +XXX,XX @@ DLOG(fprintf(stderr, "commit_direntries for %s, parent_mapping_index %d\n", mapp mapping->info.dir.parent_mapping_index = parent_mapping_index; if (first_cluster == 0) { - old_cluster_count = new_cluster_count = - s->last_cluster_of_root_directory; + old_cluster_count = new_cluster_count = + s->last_cluster_of_root_directory; } else { - for (old_cluster_count = 0, c = first_cluster; !fat_eof(s, c); - c = fat_get(s, c)) - old_cluster_count++; + for (old_cluster_count = 0, c = first_cluster; !fat_eof(s, c); + c = fat_get(s, c)) + old_cluster_count++; - for (new_cluster_count = 0, c = first_cluster; !fat_eof(s, c); - c = modified_fat_get(s, c)) - new_cluster_count++; + for (new_cluster_count = 0, c = first_cluster; !fat_eof(s, c); + c = modified_fat_get(s, c)) + new_cluster_count++; } if (new_cluster_count > old_cluster_count) { - if (insert_direntries(s, - current_dir_index + factor * old_cluster_count, - factor * (new_cluster_count - old_cluster_count)) == NULL) - return -1; + if (insert_direntries(s, + current_dir_index + factor * old_cluster_count, + factor * (new_cluster_count - old_cluster_count)) == NULL) + return -1; } else if (new_cluster_count < old_cluster_count) - remove_direntries(s, - current_dir_index + factor * new_cluster_count, - factor * (old_cluster_count - new_cluster_count)); + remove_direntries(s, + current_dir_index + factor * new_cluster_count, + factor * (old_cluster_count - new_cluster_count)); for (c = first_cluster; !fat_eof(s, c); c = modified_fat_get(s, c)) { direntry_t *first_direntry; - void* direntry = array_get(&(s->directory), current_dir_index); - int ret = vvfat_read(s->bs, cluster2sector(s, c), direntry, - s->sectors_per_cluster); - if (ret) - return ret; + void* direntry = array_get(&(s->directory), current_dir_index); + int ret = vvfat_read(s->bs, cluster2sector(s, c), direntry, + s->sectors_per_cluster); + if (ret) + return ret; /* The first directory entry on the filesystem is the volume name */ first_direntry = (direntry_t*) s->directory.pointer; assert(!memcmp(first_direntry->name, s->volume_label, 11)); - current_dir_index += factor; + current_dir_index += factor; } ret = commit_mappings(s, first_cluster, dir_index); if (ret) - return ret; + return ret; /* recurse */ for (i = 0; i < factor * new_cluster_count; i++) { - direntry = array_get(&(s->directory), first_dir_index + i); - if (is_directory(direntry) && !is_dot(direntry)) { - mapping = find_mapping_for_cluster(s, first_cluster); - assert(mapping->mode & MODE_DIRECTORY); - ret = commit_direntries(s, first_dir_index + i, - array_index(&(s->mapping), mapping)); - if (ret) - return ret; - } + direntry = array_get(&(s->directory), first_dir_index + i); + if (is_directory(direntry) && !is_dot(direntry)) { + mapping = find_mapping_for_cluster(s, first_cluster); + assert(mapping->mode & MODE_DIRECTORY); + ret = commit_direntries(s, first_dir_index + i, + array_index(&(s->mapping), mapping)); + if (ret) + return ret; + } } return 0; @@ -XXX,XX +XXX,XX @@ DLOG(fprintf(stderr, "commit_direntries for %s, parent_mapping_index %d\n", mapp /* commit one file (adjust contents, adjust mapping), return first_mapping_index */ static int commit_one_file(BDRVVVFATState* s, - int dir_index, uint32_t offset) + int dir_index, uint32_t offset) { direntry_t* direntry = array_get(&(s->directory), dir_index); uint32_t c = begin_of_direntry(direntry); @@ -XXX,XX +XXX,XX @@ static int commit_one_file(BDRVVVFATState* s, assert((offset % s->cluster_size) == 0); for (i = s->cluster_size; i < offset; i += s->cluster_size) - c = modified_fat_get(s, c); + c = modified_fat_get(s, c); fd = qemu_open(mapping->path, O_RDWR | O_CREAT | O_BINARY, 0666); if (fd < 0) { - fprintf(stderr, "Could not open %s... (%s, %d)\n", mapping->path, - strerror(errno), errno); + fprintf(stderr, "Could not open %s... (%s, %d)\n", mapping->path, + strerror(errno), errno); g_free(cluster); - return fd; + return fd; } if (offset > 0) { if (lseek(fd, offset, SEEK_SET) != offset) { @@ -XXX,XX +XXX,XX @@ static int commit_one_file(BDRVVVFATState* s, } while (offset < size) { - uint32_t c1; - int rest_size = (size - offset > s->cluster_size ? - s->cluster_size : size - offset); - int ret; + uint32_t c1; + int rest_size = (size - offset > s->cluster_size ? + s->cluster_size : size - offset); + int ret; - c1 = modified_fat_get(s, c); + c1 = modified_fat_get(s, c); - assert((size - offset == 0 && fat_eof(s, c)) || - (size > offset && c >=2 && !fat_eof(s, c))); + assert((size - offset == 0 && fat_eof(s, c)) || + (size > offset && c >=2 && !fat_eof(s, c))); - ret = vvfat_read(s->bs, cluster2sector(s, c), - (uint8_t*)cluster, (rest_size + 0x1ff) / 0x200); + ret = vvfat_read(s->bs, cluster2sector(s, c), + (uint8_t*)cluster, (rest_size + 0x1ff) / 0x200); if (ret < 0) { qemu_close(fd); @@ -XXX,XX +XXX,XX @@ static int commit_one_file(BDRVVVFATState* s, return -2; } - offset += rest_size; - c = c1; + offset += rest_size; + c = c1; } if (ftruncate(fd, size)) { @@ -XXX,XX +XXX,XX @@ static void check1(BDRVVVFATState* s) { int i; for (i = 0; i < s->mapping.next; i++) { - mapping_t* mapping = array_get(&(s->mapping), i); - if (mapping->mode & MODE_DELETED) { - fprintf(stderr, "deleted\n"); - continue; - } - assert(mapping->dir_index < s->directory.next); - direntry_t* direntry = array_get(&(s->directory), mapping->dir_index); - assert(mapping->begin == begin_of_direntry(direntry) || mapping->first_mapping_index >= 0); - if (mapping->mode & MODE_DIRECTORY) { - assert(mapping->info.dir.first_dir_index + 0x10 * s->sectors_per_cluster * (mapping->end - mapping->begin) <= s->directory.next); - assert((mapping->info.dir.first_dir_index % (0x10 * s->sectors_per_cluster)) == 0); - } + mapping_t* mapping = array_get(&(s->mapping), i); + if (mapping->mode & MODE_DELETED) { + fprintf(stderr, "deleted\n"); + continue; + } + assert(mapping->dir_index < s->directory.next); + direntry_t* direntry = array_get(&(s->directory), mapping->dir_index); + assert(mapping->begin == begin_of_direntry(direntry) || mapping->first_mapping_index >= 0); + if (mapping->mode & MODE_DIRECTORY) { + assert(mapping->info.dir.first_dir_index + 0x10 * s->sectors_per_cluster * (mapping->end - mapping->begin) <= s->directory.next); + assert((mapping->info.dir.first_dir_index % (0x10 * s->sectors_per_cluster)) == 0); + } } } @@ -XXX,XX +XXX,XX @@ static void check2(BDRVVVFATState* s) int first_mapping = -1; for (i = 0; i < s->directory.next; i++) { - direntry_t* direntry = array_get(&(s->directory), i); - - if (is_short_name(direntry) && begin_of_direntry(direntry)) { - mapping_t* mapping = find_mapping_for_cluster(s, begin_of_direntry(direntry)); - assert(mapping); - assert(mapping->dir_index == i || is_dot(direntry)); - assert(mapping->begin == begin_of_direntry(direntry) || is_dot(direntry)); - } - - if ((i % (0x10 * s->sectors_per_cluster)) == 0) { - /* cluster start */ - int j, count = 0; - - for (j = 0; j < s->mapping.next; j++) { - mapping_t* mapping = array_get(&(s->mapping), j); - if (mapping->mode & MODE_DELETED) - continue; - if (mapping->mode & MODE_DIRECTORY) { - if (mapping->info.dir.first_dir_index <= i && mapping->info.dir.first_dir_index + 0x10 * s->sectors_per_cluster > i) { - assert(++count == 1); - if (mapping->first_mapping_index == -1) - first_mapping = array_index(&(s->mapping), mapping); - else - assert(first_mapping == mapping->first_mapping_index); - if (mapping->info.dir.parent_mapping_index < 0) - assert(j == 0); - else { - mapping_t* parent = array_get(&(s->mapping), mapping->info.dir.parent_mapping_index); - assert(parent->mode & MODE_DIRECTORY); - assert(parent->info.dir.first_dir_index < mapping->info.dir.first_dir_index); - } - } - } - } - if (count == 0) - first_mapping = -1; - } + direntry_t* direntry = array_get(&(s->directory), i); + + if (is_short_name(direntry) && begin_of_direntry(direntry)) { + mapping_t* mapping = find_mapping_for_cluster(s, begin_of_direntry(direntry)); + assert(mapping); + assert(mapping->dir_index == i || is_dot(direntry)); + assert(mapping->begin == begin_of_direntry(direntry) || is_dot(direntry)); + } + + if ((i % (0x10 * s->sectors_per_cluster)) == 0) { + /* cluster start */ + int j, count = 0; + + for (j = 0; j < s->mapping.next; j++) { + mapping_t* mapping = array_get(&(s->mapping), j); + if (mapping->mode & MODE_DELETED) + continue; + if (mapping->mode & MODE_DIRECTORY) { + if (mapping->info.dir.first_dir_index <= i && mapping->info.dir.first_dir_index + 0x10 * s->sectors_per_cluster > i) { + assert(++count == 1); + if (mapping->first_mapping_index == -1) + first_mapping = array_index(&(s->mapping), mapping); + else + assert(first_mapping == mapping->first_mapping_index); + if (mapping->info.dir.parent_mapping_index < 0) + assert(j == 0); + else { + mapping_t* parent = array_get(&(s->mapping), mapping->info.dir.parent_mapping_index); + assert(parent->mode & MODE_DIRECTORY); + assert(parent->info.dir.first_dir_index < mapping->info.dir.first_dir_index); + } + } + } + } + if (count == 0) + first_mapping = -1; + } } } #endif @@ -XXX,XX +XXX,XX @@ static int handle_renames_and_mkdirs(BDRVVVFATState* s) #ifdef DEBUG fprintf(stderr, "handle_renames\n"); for (i = 0; i < s->commits.next; i++) { - commit_t* commit = array_get(&(s->commits), i); - fprintf(stderr, "%d, %s (%d, %d)\n", i, commit->path ? commit->path : "(null)", commit->param.rename.cluster, commit->action); + commit_t* commit = array_get(&(s->commits), i); + fprintf(stderr, "%d, %s (%d, %d)\n", i, commit->path ? commit->path : "(null)", commit->param.rename.cluster, commit->action); } #endif for (i = 0; i < s->commits.next;) { - commit_t* commit = array_get(&(s->commits), i); - if (commit->action == ACTION_RENAME) { - mapping_t* mapping = find_mapping_for_cluster(s, - commit->param.rename.cluster); - char* old_path = mapping->path; - - assert(commit->path); - mapping->path = commit->path; - if (rename(old_path, mapping->path)) - return -2; - - if (mapping->mode & MODE_DIRECTORY) { - int l1 = strlen(mapping->path); - int l2 = strlen(old_path); - int diff = l1 - l2; - direntry_t* direntry = array_get(&(s->directory), - mapping->info.dir.first_dir_index); - uint32_t c = mapping->begin; - int i = 0; - - /* recurse */ - while (!fat_eof(s, c)) { - do { - direntry_t* d = direntry + i; - - if (is_file(d) || (is_directory(d) && !is_dot(d))) { - mapping_t* m = find_mapping_for_cluster(s, - begin_of_direntry(d)); - int l = strlen(m->path); - char* new_path = g_malloc(l + diff + 1); - - assert(!strncmp(m->path, mapping->path, l2)); + commit_t* commit = array_get(&(s->commits), i); + if (commit->action == ACTION_RENAME) { + mapping_t* mapping = find_mapping_for_cluster(s, + commit->param.rename.cluster); + char* old_path = mapping->path; + + assert(commit->path); + mapping->path = commit->path; + if (rename(old_path, mapping->path)) + return -2; + + if (mapping->mode & MODE_DIRECTORY) { + int l1 = strlen(mapping->path); + int l2 = strlen(old_path); + int diff = l1 - l2; + direntry_t* direntry = array_get(&(s->directory), + mapping->info.dir.first_dir_index); + uint32_t c = mapping->begin; + int i = 0; + + /* recurse */ + while (!fat_eof(s, c)) { + do { + direntry_t* d = direntry + i; + + if (is_file(d) || (is_directory(d) && !is_dot(d))) { + mapping_t* m = find_mapping_for_cluster(s, + begin_of_direntry(d)); + int l = strlen(m->path); + char* new_path = g_malloc(l + diff + 1); + + assert(!strncmp(m->path, mapping->path, l2)); pstrcpy(new_path, l + diff + 1, mapping->path); pstrcpy(new_path + l1, l + diff + 1 - l1, m->path + l2); - schedule_rename(s, m->begin, new_path); - } - i++; - } while((i % (0x10 * s->sectors_per_cluster)) != 0); - c = fat_get(s, c); - } - } + schedule_rename(s, m->begin, new_path); + } + i++; + } while((i % (0x10 * s->sectors_per_cluster)) != 0); + c = fat_get(s, c); + } + } g_free(old_path); - array_remove(&(s->commits), i); - continue; - } else if (commit->action == ACTION_MKDIR) { - mapping_t* mapping; - int j, parent_path_len; + array_remove(&(s->commits), i); + continue; + } else if (commit->action == ACTION_MKDIR) { + mapping_t* mapping; + int j, parent_path_len; #ifdef __MINGW32__ if (mkdir(commit->path)) @@ -XXX,XX +XXX,XX @@ static int handle_renames_and_mkdirs(BDRVVVFATState* s) return -5; #endif - mapping = insert_mapping(s, commit->param.mkdir.cluster, - commit->param.mkdir.cluster + 1); - if (mapping == NULL) - return -6; - - mapping->mode = MODE_DIRECTORY; - mapping->read_only = 0; - mapping->path = commit->path; - j = s->directory.next; - assert(j); - insert_direntries(s, s->directory.next, - 0x10 * s->sectors_per_cluster); - mapping->info.dir.first_dir_index = j; - - parent_path_len = strlen(commit->path) - - strlen(get_basename(commit->path)) - 1; - for (j = 0; j < s->mapping.next; j++) { - mapping_t* m = array_get(&(s->mapping), j); - if (m->first_mapping_index < 0 && m != mapping && - !strncmp(m->path, mapping->path, parent_path_len) && - strlen(m->path) == parent_path_len) - break; - } - assert(j < s->mapping.next); - mapping->info.dir.parent_mapping_index = j; - - array_remove(&(s->commits), i); - continue; - } - - i++; + mapping = insert_mapping(s, commit->param.mkdir.cluster, + commit->param.mkdir.cluster + 1); + if (mapping == NULL) + return -6; + + mapping->mode = MODE_DIRECTORY; + mapping->read_only = 0; + mapping->path = commit->path; + j = s->directory.next; + assert(j); + insert_direntries(s, s->directory.next, + 0x10 * s->sectors_per_cluster); + mapping->info.dir.first_dir_index = j; + + parent_path_len = strlen(commit->path) + - strlen(get_basename(commit->path)) - 1; + for (j = 0; j < s->mapping.next; j++) { + mapping_t* m = array_get(&(s->mapping), j); + if (m->first_mapping_index < 0 && m != mapping && + !strncmp(m->path, mapping->path, parent_path_len) && + strlen(m->path) == parent_path_len) + break; + } + assert(j < s->mapping.next); + mapping->info.dir.parent_mapping_index = j; + + array_remove(&(s->commits), i); + continue; + } + + i++; } return 0; } @@ -XXX,XX +XXX,XX @@ static int handle_commits(BDRVVVFATState* s) vvfat_close_current_file(s); for (i = 0; !fail && i < s->commits.next; i++) { - commit_t* commit = array_get(&(s->commits), i); - switch(commit->action) { - case ACTION_RENAME: case ACTION_MKDIR: + commit_t* commit = array_get(&(s->commits), i); + switch(commit->action) { + case ACTION_RENAME: case ACTION_MKDIR: abort(); - fail = -2; - break; - case ACTION_WRITEOUT: { + fail = -2; + break; + case ACTION_WRITEOUT: { #ifndef NDEBUG /* these variables are only used by assert() below */ - direntry_t* entry = array_get(&(s->directory), - commit->param.writeout.dir_index); - uint32_t begin = begin_of_direntry(entry); - mapping_t* mapping = find_mapping_for_cluster(s, begin); + direntry_t* entry = array_get(&(s->directory), + commit->param.writeout.dir_index); + uint32_t begin = begin_of_direntry(entry); + mapping_t* mapping = find_mapping_for_cluster(s, begin); #endif - assert(mapping); - assert(mapping->begin == begin); - assert(commit->path == NULL); - - if (commit_one_file(s, commit->param.writeout.dir_index, - commit->param.writeout.modified_offset)) - fail = -3; - - break; - } - case ACTION_NEW_FILE: { - int begin = commit->param.new_file.first_cluster; - mapping_t* mapping = find_mapping_for_cluster(s, begin); - direntry_t* entry; - int i; - - /* find direntry */ - for (i = 0; i < s->directory.next; i++) { - entry = array_get(&(s->directory), i); - if (is_file(entry) && begin_of_direntry(entry) == begin) - break; - } - - if (i >= s->directory.next) { - fail = -6; - continue; - } - - /* make sure there exists an initial mapping */ - if (mapping && mapping->begin != begin) { - mapping->end = begin; - mapping = NULL; - } - if (mapping == NULL) { - mapping = insert_mapping(s, begin, begin+1); - } - /* most members will be fixed in commit_mappings() */ - assert(commit->path); - mapping->path = commit->path; - mapping->read_only = 0; - mapping->mode = MODE_NORMAL; - mapping->info.file.offset = 0; - - if (commit_one_file(s, i, 0)) - fail = -7; - - break; - } - default: + assert(mapping); + assert(mapping->begin == begin); + assert(commit->path == NULL); + + if (commit_one_file(s, commit->param.writeout.dir_index, + commit->param.writeout.modified_offset)) + fail = -3; + + break; + } + case ACTION_NEW_FILE: { + int begin = commit->param.new_file.first_cluster; + mapping_t* mapping = find_mapping_for_cluster(s, begin); + direntry_t* entry; + int i; + + /* find direntry */ + for (i = 0; i < s->directory.next; i++) { + entry = array_get(&(s->directory), i); + if (is_file(entry) && begin_of_direntry(entry) == begin) + break; + } + + if (i >= s->directory.next) { + fail = -6; + continue; + } + + /* make sure there exists an initial mapping */ + if (mapping && mapping->begin != begin) { + mapping->end = begin; + mapping = NULL; + } + if (mapping == NULL) { + mapping = insert_mapping(s, begin, begin+1); + } + /* most members will be fixed in commit_mappings() */ + assert(commit->path); + mapping->path = commit->path; + mapping->read_only = 0; + mapping->mode = MODE_NORMAL; + mapping->info.file.offset = 0; + + if (commit_one_file(s, i, 0)) + fail = -7; + + break; + } + default: abort(); - } + } } if (i > 0 && array_remove_slice(&(s->commits), 0, i)) - return -1; + return -1; return fail; } @@ -XXX,XX +XXX,XX @@ static int handle_deletes(BDRVVVFATState* s) /* delete files corresponding to mappings marked as deleted */ /* handle DELETEs and unused mappings (modified_fat_get(s, mapping->begin) == 0) */ while (deferred && deleted) { - deferred = 0; - deleted = 0; - - for (i = 1; i < s->mapping.next; i++) { - mapping_t* mapping = array_get(&(s->mapping), i); - if (mapping->mode & MODE_DELETED) { - direntry_t* entry = array_get(&(s->directory), - mapping->dir_index); - - if (is_free(entry)) { - /* remove file/directory */ - if (mapping->mode & MODE_DIRECTORY) { - int j, next_dir_index = s->directory.next, - first_dir_index = mapping->info.dir.first_dir_index; - - if (rmdir(mapping->path) < 0) { - if (errno == ENOTEMPTY) { - deferred++; - continue; - } else - return -5; - } - - for (j = 1; j < s->mapping.next; j++) { - mapping_t* m = array_get(&(s->mapping), j); - if (m->mode & MODE_DIRECTORY && - m->info.dir.first_dir_index > - first_dir_index && - m->info.dir.first_dir_index < - next_dir_index) - next_dir_index = - m->info.dir.first_dir_index; - } - remove_direntries(s, first_dir_index, - next_dir_index - first_dir_index); - - deleted++; - } - } else { - if (unlink(mapping->path)) - return -4; - deleted++; - } - DLOG(fprintf(stderr, "DELETE (%d)\n", i); print_mapping(mapping); print_direntry(entry)); - remove_mapping(s, i); - } - } + deferred = 0; + deleted = 0; + + for (i = 1; i < s->mapping.next; i++) { + mapping_t* mapping = array_get(&(s->mapping), i); + if (mapping->mode & MODE_DELETED) { + direntry_t* entry = array_get(&(s->directory), + mapping->dir_index); + + if (is_free(entry)) { + /* remove file/directory */ + if (mapping->mode & MODE_DIRECTORY) { + int j, next_dir_index = s->directory.next, + first_dir_index = mapping->info.dir.first_dir_index; + + if (rmdir(mapping->path) < 0) { + if (errno == ENOTEMPTY) { + deferred++; + continue; + } else + return -5; + } + + for (j = 1; j < s->mapping.next; j++) { + mapping_t* m = array_get(&(s->mapping), j); + if (m->mode & MODE_DIRECTORY && + m->info.dir.first_dir_index > + first_dir_index && + m->info.dir.first_dir_index < + next_dir_index) + next_dir_index = + m->info.dir.first_dir_index; + } + remove_direntries(s, first_dir_index, + next_dir_index - first_dir_index); + + deleted++; + } + } else { + if (unlink(mapping->path)) + return -4; + deleted++; + } + DLOG(fprintf(stderr, "DELETE (%d)\n", i); print_mapping(mapping); print_direntry(entry)); + remove_mapping(s, i); + } + } } return 0; @@ -XXX,XX +XXX,XX @@ static int do_commit(BDRVVVFATState* s) /* the real meat are the commits. Nothing to do? Move along! */ if (s->commits.next == 0) - return 0; + return 0; vvfat_close_current_file(s); ret = handle_renames_and_mkdirs(s); if (ret) { - fprintf(stderr, "Error handling renames (%d)\n", ret); + fprintf(stderr, "Error handling renames (%d)\n", ret); abort(); - return ret; + return ret; } /* copy FAT (with bdrv_read) */ @@ -XXX,XX +XXX,XX @@ static int do_commit(BDRVVVFATState* s) /* recurse direntries from root (using bs->bdrv_read) */ ret = commit_direntries(s, 0, -1); if (ret) { - fprintf(stderr, "Fatal: error while committing (%d)\n", ret); + fprintf(stderr, "Fatal: error while committing (%d)\n", ret); abort(); - return ret; + return ret; } ret = handle_commits(s); if (ret) { - fprintf(stderr, "Error handling commits (%d)\n", ret); + fprintf(stderr, "Error handling commits (%d)\n", ret); abort(); - return ret; + return ret; } ret = handle_deletes(s); if (ret) { - fprintf(stderr, "Error deleting\n"); + fprintf(stderr, "Error deleting\n"); abort(); - return ret; + return ret; } if (s->qcow->bs->drv->bdrv_make_empty) { @@ -XXX,XX +XXX,XX @@ static int try_commit(BDRVVVFATState* s) vvfat_close_current_file(s); DLOG(checkpoint()); if(!is_consistent(s)) - return -1; + return -1; return do_commit(s); } @@ -XXX,XX +XXX,XX @@ DLOG(checkpoint()); */ if (sector_num < s->first_sectors_number) - return -1; + return -1; for (i = sector2cluster(s, sector_num); - i <= sector2cluster(s, sector_num + nb_sectors - 1);) { - mapping_t* mapping = find_mapping_for_cluster(s, i); - if (mapping) { - if (mapping->read_only) { - fprintf(stderr, "Tried to write to write-protected file %s\n", - mapping->path); - return -1; - } - - if (mapping->mode & MODE_DIRECTORY) { - int begin = cluster2sector(s, i); - int end = begin + s->sectors_per_cluster, k; - int dir_index; - const direntry_t* direntries; - long_file_name lfn; - - lfn_init(&lfn); - - if (begin < sector_num) - begin = sector_num; - if (end > sector_num + nb_sectors) - end = sector_num + nb_sectors; - dir_index = mapping->dir_index + - 0x10 * (begin - mapping->begin * s->sectors_per_cluster); - direntries = (direntry_t*)(buf + 0x200 * (begin - sector_num)); - - for (k = 0; k < (end - begin) * 0x10; k++) { - /* do not allow non-ASCII filenames */ - if (parse_long_name(&lfn, direntries + k) < 0) { - fprintf(stderr, "Warning: non-ASCII filename\n"); - return -1; - } - /* no access to the direntry of a read-only file */ - else if (is_short_name(direntries+k) && - (direntries[k].attributes & 1)) { - if (memcmp(direntries + k, - array_get(&(s->directory), dir_index + k), - sizeof(direntry_t))) { - fprintf(stderr, "Warning: tried to write to write-protected file\n"); - return -1; - } - } - } - } - i = mapping->end; - } else - i++; + i <= sector2cluster(s, sector_num + nb_sectors - 1);) { + mapping_t* mapping = find_mapping_for_cluster(s, i); + if (mapping) { + if (mapping->read_only) { + fprintf(stderr, "Tried to write to write-protected file %s\n", + mapping->path); + return -1; + } + + if (mapping->mode & MODE_DIRECTORY) { + int begin = cluster2sector(s, i); + int end = begin + s->sectors_per_cluster, k; + int dir_index; + const direntry_t* direntries; + long_file_name lfn; + + lfn_init(&lfn); + + if (begin < sector_num) + begin = sector_num; + if (end > sector_num + nb_sectors) + end = sector_num + nb_sectors; + dir_index = mapping->dir_index + + 0x10 * (begin - mapping->begin * s->sectors_per_cluster); + direntries = (direntry_t*)(buf + 0x200 * (begin - sector_num)); + + for (k = 0; k < (end - begin) * 0x10; k++) { + /* do not allow non-ASCII filenames */ + if (parse_long_name(&lfn, direntries + k) < 0) { + fprintf(stderr, "Warning: non-ASCII filename\n"); + return -1; + } + /* no access to the direntry of a read-only file */ + else if (is_short_name(direntries+k) && + (direntries[k].attributes & 1)) { + if (memcmp(direntries + k, + array_get(&(s->directory), dir_index + k), + sizeof(direntry_t))) { + fprintf(stderr, "Warning: tried to write to write-protected file\n"); + return -1; + } + } + } + } + i = mapping->end; + } else + i++; } /* @@ -XXX,XX +XXX,XX @@ DLOG(checkpoint()); DLOG(fprintf(stderr, "Write to qcow backend: %d + %d\n", (int)sector_num, nb_sectors)); ret = bdrv_write(s->qcow, sector_num, buf, nb_sectors); if (ret < 0) { - fprintf(stderr, "Error writing to qcow backend\n"); - return ret; + fprintf(stderr, "Error writing to qcow backend\n"); + return ret; } for (i = sector2cluster(s, sector_num); - i <= sector2cluster(s, sector_num + nb_sectors - 1); i++) - if (i >= 0) - s->used_clusters[i] |= USED_ALLOCATED; + i <= sector2cluster(s, sector_num + nb_sectors - 1); i++) + if (i >= 0) + s->used_clusters[i] |= USED_ALLOCATED; DLOG(checkpoint()); /* TODO: add timeout */ @@ -XXX,XX +XXX,XX @@ vvfat_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, } static int64_t coroutine_fn vvfat_co_get_block_status(BlockDriverState *bs, - int64_t sector_num, int nb_sectors, int *n, BlockDriverState **file) + int64_t sector_num, int nb_sectors, int *n, BlockDriverState **file) { *n = bs->total_sectors - sector_num; if (*n > nb_sectors) { @@ -XXX,XX +XXX,XX @@ static void checkpoint(void) { assert(!vvv->current_mapping || vvv->current_fd || (vvv->current_mapping->mode & MODE_DIRECTORY)); #if 0 if (((direntry_t*)vvv->directory.pointer)[1].attributes != 0xf) - fprintf(stderr, "Nonono!\n"); + fprintf(stderr, "Nonono!\n"); mapping_t* mapping; direntry_t* direntry; assert(vvv->mapping.size >= vvv->mapping.item_size * vvv->mapping.next); assert(vvv->directory.size >= vvv->directory.item_size * vvv->directory.next); if (vvv->mapping.next<47) - return; + return; assert((mapping = array_get(&(vvv->mapping), 47))); assert(mapping->dir_index < vvv->directory.next); direntry = array_get(&(vvv->directory), mapping->dir_index); -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static void init_mbr(BDRVVVFATState *s, int cyls, int heads, int secs) /* FAT12/FAT16/FAT32 */ /* DOS uses different types when partition is LBA, probably to prevent older versions from using CHS on them */ - partition->fs_type= s->fat_type==12 ? 0x1: - s->fat_type==16 ? (lba?0xe:0x06): - /*fat_tyoe==32*/ (lba?0xc:0x0b); + partition->fs_type = s->fat_type == 12 ? 0x1 : + s->fat_type == 16 ? (lba ? 0xe : 0x06) : + /*s->fat_type == 32*/ (lba ? 0xc : 0x0b); real_mbr->magic[0]=0x55; real_mbr->magic[1]=0xaa; } @@ -XXX,XX +XXX,XX @@ static int read_directory(BDRVVVFATState* s, int mapping_index) (ROOT_ENTRIES - cur) * sizeof(direntry_t)); } - /* reget the mapping, since s->mapping was possibly realloc()ed */ + /* re-get the mapping, since s->mapping was possibly realloc()ed */ mapping = array_get(&(s->mapping), mapping_index); first_cluster += (s->directory.next - mapping->info.dir.first_dir_index) * 0x20 / s->cluster_size; -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> MODE_FAKED and MODE_RENAMED are not and were never used. Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ typedef struct mapping_t { union { /* offset is * - the offset in the file (in clusters) for a file, or - * - the next cluster of the directory for a directory, and - * - the address of the buffer for a faked entry + * - the next cluster of the directory for a directory */ struct { uint32_t offset; @@ -XXX,XX +XXX,XX @@ typedef struct mapping_t { /* path contains the full path, i.e. it always starts with s->path */ char* path; - enum { MODE_UNDEFINED = 0, MODE_NORMAL = 1, MODE_MODIFIED = 2, - MODE_DIRECTORY = 4, MODE_FAKED = 8, - MODE_DELETED = 16, MODE_RENAMED = 32 } mode; + enum { + MODE_UNDEFINED = 0, + MODE_NORMAL = 1, + MODE_MODIFIED = 2, + MODE_DIRECTORY = 4, + MODE_DELETED = 8, + } mode; int read_only; } mapping_t; -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> - offset_to_bootsector is the number of sectors up to FAT bootsector - offset_to_fat is the number of sectors up to first File Allocation Table - offset_to_root_dir is the number of sectors up to root directory sector Replace first_sectors_number - 1 by offset_to_bootsector. Replace first_sectors_number by offset_to_fat. Replace faked_sectors by offset_to_rootdir. Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 70 ++++++++++++++++++++++++++++++++++++----------------------- 1 file changed, 43 insertions(+), 27 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static void print_mapping(const struct mapping_t* mapping); typedef struct BDRVVVFATState { CoMutex lock; BlockDriverState* bs; /* pointer to parent */ - unsigned int first_sectors_number; /* 1 for a single partition, 0x40 for a disk with partition table */ unsigned char first_sectors[0x40*0x200]; int fat_type; /* 16 or 32 */ array_t fat,directory,mapping; char volume_label[11]; + uint32_t offset_to_bootsector; /* 0 for floppy, 0x3f for disk */ + unsigned int cluster_size; unsigned int sectors_per_cluster; unsigned int sectors_per_fat; unsigned int sectors_of_root_directory; uint32_t last_cluster_of_root_directory; - unsigned int faked_sectors; /* how many sectors are faked before file data */ uint32_t sector_count; /* total number of sectors of the partition */ uint32_t cluster_count; /* total number of clusters of this partition */ uint32_t max_fat_value; + uint32_t offset_to_fat; + uint32_t offset_to_root_dir; int current_fd; mapping_t* current_mapping; @@ -XXX,XX +XXX,XX @@ static void init_mbr(BDRVVVFATState *s, int cyls, int heads, int secs) partition->attributes=0x80; /* bootable */ /* LBA is used when partition is outside the CHS geometry */ - lba = sector2CHS(&partition->start_CHS, s->first_sectors_number - 1, + lba = sector2CHS(&partition->start_CHS, s->offset_to_bootsector, cyls, heads, secs); lba |= sector2CHS(&partition->end_CHS, s->bs->total_sectors - 1, cyls, heads, secs); /*LBA partitions are identified only by start/length_sector_long not by CHS*/ - partition->start_sector_long = cpu_to_le32(s->first_sectors_number - 1); + partition->start_sector_long = cpu_to_le32(s->offset_to_bootsector); partition->length_sector_long = cpu_to_le32(s->bs->total_sectors - - s->first_sectors_number + 1); + - s->offset_to_bootsector); /* FAT12/FAT16/FAT32 */ /* DOS uses different types when partition is LBA, @@ -XXX,XX +XXX,XX @@ static int read_directory(BDRVVVFATState* s, int mapping_index) static inline uint32_t sector2cluster(BDRVVVFATState* s,off_t sector_num) { - return (sector_num-s->faked_sectors)/s->sectors_per_cluster; + return (sector_num - s->offset_to_root_dir) / s->sectors_per_cluster; } static inline off_t cluster2sector(BDRVVVFATState* s, uint32_t cluster_num) { - return s->faked_sectors + s->sectors_per_cluster * cluster_num; + return s->offset_to_root_dir + s->sectors_per_cluster * cluster_num; } static int init_directories(BDRVVVFATState* s, @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, i = 1+s->sectors_per_cluster*0x200*8/s->fat_type; s->sectors_per_fat=(s->sector_count+i)/i; /* round up */ + s->offset_to_fat = s->offset_to_bootsector + 1; + s->offset_to_root_dir = s->offset_to_fat + s->sectors_per_fat * 2; + array_init(&(s->mapping),sizeof(mapping_t)); array_init(&(s->directory),sizeof(direntry_t)); @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, /* Now build FAT, and write back information into directory */ init_fat(s); - s->faked_sectors=s->first_sectors_number+s->sectors_per_fat*2; s->cluster_count=sector2cluster(s, s->sector_count); mapping = array_get_next(&(s->mapping)); @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, s->current_mapping = NULL; - bootsector=(bootsector_t*)(s->first_sectors+(s->first_sectors_number-1)*0x200); + bootsector = (bootsector_t *)(s->first_sectors + + s->offset_to_bootsector * 0x200); bootsector->jump[0]=0xeb; bootsector->jump[1]=0x3e; bootsector->jump[2]=0x90; @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, bootsector->number_of_fats=0x2; /* number of FATs */ bootsector->root_entries=cpu_to_le16(s->sectors_of_root_directory*0x10); bootsector->total_sectors16=s->sector_count>0xffff?0:cpu_to_le16(s->sector_count); - bootsector->media_type=(s->first_sectors_number>1?0xf8:0xf0); /* media descriptor (f8=hd, f0=3.5 fd)*/ + /* media descriptor: hard disk=0xf8, floppy=0xf0 */ + bootsector->media_type = (s->offset_to_bootsector > 0 ? 0xf8 : 0xf0); s->fat.pointer[0] = bootsector->media_type; bootsector->sectors_per_fat=cpu_to_le16(s->sectors_per_fat); bootsector->sectors_per_track = cpu_to_le16(secs); bootsector->number_of_heads = cpu_to_le16(heads); - bootsector->hidden_sectors=cpu_to_le32(s->first_sectors_number==1?0:0x3f); + bootsector->hidden_sectors = cpu_to_le32(s->offset_to_bootsector); bootsector->total_sectors=cpu_to_le32(s->sector_count>0xffff?s->sector_count:0); /* LATER TODO: if FAT32, this is wrong */ - bootsector->u.fat16.drive_number=s->first_sectors_number==1?0:0x80; /* fda=0, hda=0x80 */ + /* drive_number: fda=0, hda=0x80 */ + bootsector->u.fat16.drive_number = s->offset_to_bootsector == 0 ? 0 : 0x80; bootsector->u.fat16.current_head=0; bootsector->u.fat16.signature=0x29; bootsector->u.fat16.id=cpu_to_le32(0xfabe1afd); @@ -XXX,XX +XXX,XX @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags, secs = s->fat_type == 12 ? 18 : 36; s->sectors_per_cluster = 1; } - s->first_sectors_number = 1; cyls = 80; heads = 2; } else { @@ -XXX,XX +XXX,XX @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags, if (!s->fat_type) { s->fat_type = 16; } - s->first_sectors_number = 0x40; + s->offset_to_bootsector = 0x3f; cyls = s->fat_type == 12 ? 64 : 1024; heads = 16; secs = 63; @@ -XXX,XX +XXX,XX @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags, fprintf(stderr, "vvfat %s chs %d,%d,%d\n", dirname, cyls, heads, secs); - s->sector_count = cyls * heads * secs - (s->first_sectors_number - 1); + s->sector_count = cyls * heads * secs - s->offset_to_bootsector; if (qemu_opt_get_bool(opts, "rw", false)) { if (!bdrv_is_read_only(bs)) { @@ -XXX,XX +XXX,XX @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags, goto fail; } - s->sector_count = s->faked_sectors + s->sectors_per_cluster*s->cluster_count; + s->sector_count = s->offset_to_root_dir + + s->sectors_per_cluster * s->cluster_count; /* Disable migration when vvfat is used rw */ if (s->qcow) { @@ -XXX,XX +XXX,XX @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags, } } - if (s->first_sectors_number == 0x40) { + if (s->offset_to_bootsector > 0) { init_mbr(s, cyls, heads, secs); } @@ -XXX,XX +XXX,XX @@ static int vvfat_read(BlockDriverState *bs, int64_t sector_num, } DLOG(fprintf(stderr, "sector %d not allocated\n", (int)sector_num)); } - if(sector_num<s->faked_sectors) { - if(sector_num<s->first_sectors_number) - memcpy(buf+i*0x200,&(s->first_sectors[sector_num*0x200]),0x200); - else if(sector_num-s->first_sectors_number<s->sectors_per_fat) - memcpy(buf+i*0x200,&(s->fat.pointer[(sector_num-s->first_sectors_number)*0x200]),0x200); - else if(sector_num-s->first_sectors_number-s->sectors_per_fat<s->sectors_per_fat) - memcpy(buf+i*0x200,&(s->fat.pointer[(sector_num-s->first_sectors_number-s->sectors_per_fat)*0x200]),0x200); + if (sector_num < s->offset_to_root_dir) { + if (sector_num < s->offset_to_fat) { + memcpy(buf + i * 0x200, + &(s->first_sectors[sector_num * 0x200]), + 0x200); + } else if (sector_num < s->offset_to_fat + s->sectors_per_fat) { + memcpy(buf + i * 0x200, + &(s->fat.pointer[(sector_num + - s->offset_to_fat) * 0x200]), + 0x200); + } else if (sector_num < s->offset_to_root_dir) { + memcpy(buf + i * 0x200, + &(s->fat.pointer[(sector_num - s->offset_to_fat + - s->sectors_per_fat) * 0x200]), + 0x200); + } } else { - uint32_t sector=sector_num-s->faked_sectors, + uint32_t sector = sector_num - s->offset_to_root_dir, sector_offset_in_cluster=(sector%s->sectors_per_cluster), cluster_num=sector/s->sectors_per_cluster; if(cluster_num > s->cluster_count || read_cluster(s, cluster_num) != 0) { @@ -XXX,XX +XXX,XX @@ DLOG(checkpoint()); memcpy(s->fat2, s->fat.pointer, size); } check = vvfat_read(s->bs, - s->first_sectors_number, s->fat2, s->sectors_per_fat); + s->offset_to_fat, s->fat2, s->sectors_per_fat); if (check) { fprintf(stderr, "Could not copy fat\n"); return 0; @@ -XXX,XX +XXX,XX @@ DLOG(checkpoint()); * - do not allow to write non-ASCII filenames */ - if (sector_num < s->first_sectors_number) + if (sector_num < s->offset_to_fat) return -1; for (i = sector2cluster(s, sector_num); -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> Specification: "FAT: General overview of on-disk format" v1.03, pages 11-13 Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ typedef struct bootsector_t { union { struct { uint8_t drive_number; - uint8_t current_head; + uint8_t reserved1; uint8_t signature; uint32_t id; uint8_t volume_label[11]; + uint8_t fat_type[8]; + uint8_t ignored[0x1c0]; } QEMU_PACKED fat16; struct { uint32_t sectors_per_fat; uint16_t flags; uint8_t major,minor; - uint32_t first_cluster_of_root_directory; + uint32_t first_cluster_of_root_dir; uint16_t info_sector; uint16_t backup_boot_sector; - uint16_t ignored; + uint8_t reserved[12]; + uint8_t drive_number; + uint8_t reserved1; + uint8_t signature; + uint32_t id; + uint8_t volume_label[11]; + uint8_t fat_type[8]; + uint8_t ignored[0x1a4]; } QEMU_PACKED fat32; } u; - uint8_t fat_type[8]; - uint8_t ignored[0x1c0]; uint8_t magic[2]; } QEMU_PACKED bootsector_t; @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, /* LATER TODO: if FAT32, this is wrong */ /* drive_number: fda=0, hda=0x80 */ bootsector->u.fat16.drive_number = s->offset_to_bootsector == 0 ? 0 : 0x80; - bootsector->u.fat16.current_head=0; bootsector->u.fat16.signature=0x29; bootsector->u.fat16.id=cpu_to_le32(0xfabe1afd); memcpy(bootsector->u.fat16.volume_label, s->volume_label, sizeof(bootsector->u.fat16.volume_label)); - memcpy(bootsector->fat_type,(s->fat_type==12?"FAT12 ":s->fat_type==16?"FAT16 ":"FAT32 "),8); + memcpy(bootsector->u.fat16.fat_type, + s->fat_type == 12 ? "FAT12 " : "FAT16 ", 8); bootsector->magic[0]=0x55; bootsector->magic[1]=0xaa; return 0; -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> readdir() doesn't always return . and .. entries at first and in that order. This leads to not creating them at first in the directory, which raises some errors on file system checking utilities like MS-DOS Scandisk. Specification: "FAT: General overview of on-disk format" v1.03, page 25 Fixes: https://bugs.launchpad.net/qemu/+bug/1599539 Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static int read_directory(BDRVVVFATState* s, int mapping_index) i = mapping->info.dir.first_dir_index = first_cluster == 0 ? 0 : s->directory.next; + if (first_cluster != 0) { + /* create the top entries of a subdirectory */ + (void)create_short_and_long_name(s, i, ".", 1); + (void)create_short_and_long_name(s, i, "..", 1); + } + /* actually read the directory, and allocate the mappings */ while((entry=readdir(dir))) { unsigned int length=strlen(dirname)+2+strlen(entry->d_name); @@ -XXX,XX +XXX,XX @@ static int read_directory(BDRVVVFATState* s, int mapping_index) } /* create directory entry for this file */ - direntry=create_short_and_long_name(s, i, entry->d_name, - is_dot || is_dotdot); + if (!is_dot && !is_dotdot) { + direntry = create_short_and_long_name(s, i, entry->d_name, 0); + } else { + direntry = array_get(&(s->directory), is_dot ? i : i + 1); + } direntry->attributes=(S_ISDIR(st.st_mode)?0x10:0x20); direntry->reserved[0]=direntry->reserved[1]=0; direntry->ctime=fat_datetime(st.st_ctime,1); -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> Assume that input filename is encoded as UTF-8, so correctly create UTF-16 encoding. Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 38 ++++++++++++++++++-------------------- 1 file changed, 18 insertions(+), 20 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static void init_mbr(BDRVVVFATState *s, int cyls, int heads, int secs) /* direntry functions */ -/* dest is assumed to hold 258 bytes, and pads with 0xffff up to next multiple of 26 */ -static inline int short2long_name(char* dest,const char* src) +static direntry_t *create_long_filename(BDRVVVFATState *s, const char *filename) { - int i; - int len; - for(i=0;i<129 && src[i];i++) { - dest[2*i]=src[i]; - dest[2*i+1]=0; + int number_of_entries, i; + glong length; + direntry_t *entry; + + gunichar2 *longname = g_utf8_to_utf16(filename, -1, NULL, &length, NULL); + if (!longname) { + fprintf(stderr, "vvfat: invalid UTF-8 name: %s\n", filename); + return NULL; } - len=2*i; - dest[2*i]=dest[2*i+1]=0; - for(i=2*i+2;(i%26);i++) - dest[i]=0xff; - return len; -} -static inline direntry_t* create_long_filename(BDRVVVFATState* s,const char* filename) -{ - char buffer[258]; - int length=short2long_name(buffer,filename), - number_of_entries=(length+25)/26,i; - direntry_t* entry; + number_of_entries = (length * 2 + 25) / 26; for(i=0;i<number_of_entries;i++) { entry=array_get_next(&(s->directory)); @@ -XXX,XX +XXX,XX @@ static inline direntry_t* create_long_filename(BDRVVVFATState* s,const char* fil else if(offset<22) offset=14+offset-10; else offset=28+offset-22; entry=array_get(&(s->directory),s->directory.next-1-(i/26)); - entry->name[offset]=buffer[i]; + if (i >= 2 * length + 2) { + entry->name[offset] = 0xff; + } else if (i % 2 == 0) { + entry->name[offset] = longname[i / 2] & 0xff; + } else { + entry->name[offset] = longname[i / 2] >> 8; + } } + g_free(longname); return array_get(&(s->directory),s->directory.next-number_of_entries); } -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> More specifically, create short name from filename and change blacklist of invalid chars to whitelist of valid chars. Windows 9x also now correctly see long file names of filenames containing a space, but Scandisk still complains about mismatch between SFN and LFN. [kwolf: Build fix for this intermediate patch (it included declarations for variables that are only used in the next patch) ] Specification: "FAT: General overview of on-disk format" v1.03, pages 30-31 Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 104 ++++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 76 insertions(+), 28 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static void set_begin_of_direntry(direntry_t* direntry, uint32_t begin) direntry->begin_hi = cpu_to_le16((begin >> 16) & 0xffff); } +static uint8_t to_valid_short_char(gunichar c) +{ + c = g_unichar_toupper(c); + if ((c >= '0' && c <= '9') || + (c >= 'A' && c <= 'Z') || + strchr("$%'-_@~`!(){}^#&", c) != 0) { + return c; + } else { + return 0; + } +} + +static direntry_t *create_short_filename(BDRVVVFATState *s, + const char *filename) +{ + int j = 0; + direntry_t *entry = array_get_next(&(s->directory)); + const gchar *p, *last_dot = NULL; + gunichar c; + bool lossy_conversion = false; + + if (!entry) { + return NULL; + } + memset(entry->name, 0x20, sizeof(entry->name)); + + /* copy filename and search last dot */ + for (p = filename; ; p = g_utf8_next_char(p)) { + c = g_utf8_get_char(p); + if (c == '\0') { + break; + } else if (c == '.') { + if (j == 0) { + /* '.' at start of filename */ + lossy_conversion = true; + } else { + if (last_dot) { + lossy_conversion = true; + } + last_dot = p; + } + } else if (!last_dot) { + /* first part of the name; copy it */ + uint8_t v = to_valid_short_char(c); + if (j < 8 && v) { + entry->name[j++] = v; + } else { + lossy_conversion = true; + } + } + } + + /* copy extension (if any) */ + if (last_dot) { + j = 0; + for (p = g_utf8_next_char(last_dot); ; p = g_utf8_next_char(p)) { + c = g_utf8_get_char(p); + if (c == '\0') { + break; + } else { + /* extension; copy it */ + uint8_t v = to_valid_short_char(c); + if (j < 3 && v) { + entry->name[8 + (j++)] = v; + } else { + lossy_conversion = true; + } + } + } + } + (void)lossy_conversion; + return entry; +} + /* fat functions */ static inline uint8_t fat_chksum(const direntry_t* entry) @@ -XXX,XX +XXX,XX @@ static inline void init_fat(BDRVVVFATState* s) static inline direntry_t* create_short_and_long_name(BDRVVVFATState* s, unsigned int directory_start, const char* filename, int is_dot) { - int i,j,long_index=s->directory.next; + int long_index = s->directory.next; direntry_t* entry = NULL; direntry_t* entry_long = NULL; @@ -XXX,XX +XXX,XX @@ static inline direntry_t* create_short_and_long_name(BDRVVVFATState* s, } entry_long=create_long_filename(s,filename); - - i = strlen(filename); - for(j = i - 1; j>0 && filename[j]!='.';j--); - if (j > 0) - i = (j > 8 ? 8 : j); - else if (i > 8) - i = 8; - - entry=array_get_next(&(s->directory)); - memset(entry->name, 0x20, sizeof(entry->name)); - memcpy(entry->name, filename, i); - - if (j > 0) { - for (i = 0; i < 3 && filename[j + 1 + i]; i++) { - entry->name[8 + i] = filename[j + 1 + i]; - } - } - - /* upcase & remove unwanted characters */ - for(i=10;i>=0;i--) { - if(i==10 || i==7) for(;i>0 && entry->name[i]==' ';i--); - if(entry->name[i]<=' ' || entry->name[i]>0x7f - || strchr(".*?<>|\":/\\[];,+='",entry->name[i])) - entry->name[i]='_'; - else if(entry->name[i]>='a' && entry->name[i]<='z') - entry->name[i]+='A'-'a'; - } + entry = create_short_filename(s, filename); /* mangle duplicates */ while(1) { -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> More specifically: - try without numeric-tail only if LFN didn't have invalid short chars - start at ~1 (instead of ~0) - handle case if numeric tail is more than one char (ie > 10) Windows 9x Scandisk doesn't see anymore mismatches between short file names and long file names for non-ASCII filenames. Specification: "FAT: General overview of on-disk format" v1.03, page 31 Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 65 ++++++++++++++++++++++++++++------------------------------- 1 file changed, 31 insertions(+), 34 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static uint8_t to_valid_short_char(gunichar c) } static direntry_t *create_short_filename(BDRVVVFATState *s, - const char *filename) + const char *filename, + unsigned int directory_start) { - int j = 0; + int i, j = 0; direntry_t *entry = array_get_next(&(s->directory)); const gchar *p, *last_dot = NULL; gunichar c; bool lossy_conversion = false; + char tail[11]; if (!entry) { return NULL; @@ -XXX,XX +XXX,XX @@ static direntry_t *create_short_filename(BDRVVVFATState *s, } } } - (void)lossy_conversion; - return entry; + + /* numeric-tail generation */ + for (j = 0; j < 8; j++) { + if (entry->name[j] == ' ') { + break; + } + } + for (i = lossy_conversion ? 1 : 0; i < 999999; i++) { + direntry_t *entry1; + if (i > 0) { + int len = sprintf(tail, "~%d", i); + memcpy(entry->name + MIN(j, 8 - len), tail, len); + } + for (entry1 = array_get(&(s->directory), directory_start); + entry1 < entry; entry1++) { + if (!is_long_name(entry1) && + !memcmp(entry1->name, entry->name, 11)) { + break; /* found dupe */ + } + } + if (entry1 == entry) { + /* no dupe found */ + return entry; + } + } + return NULL; } /* fat functions */ @@ -XXX,XX +XXX,XX @@ static inline direntry_t* create_short_and_long_name(BDRVVVFATState* s, } entry_long=create_long_filename(s,filename); - entry = create_short_filename(s, filename); - - /* mangle duplicates */ - while(1) { - direntry_t* entry1=array_get(&(s->directory),directory_start); - int j; - - for(;entry1<entry;entry1++) - if(!is_long_name(entry1) && !memcmp(entry1->name,entry->name,11)) - break; /* found dupe */ - if(entry1==entry) /* no dupe found */ - break; - - /* use all 8 characters of name */ - if(entry->name[7]==' ') { - int j; - for(j=6;j>0 && entry->name[j]==' ';j--) - entry->name[j]='~'; - } - - /* increment number */ - for(j=7;j>0 && entry->name[j]=='9';j--) - entry->name[j]='0'; - if(j>0) { - if(entry->name[j]<'0' || entry->name[j]>'9') - entry->name[j]='0'; - else - entry->name[j]++; - } - } + entry = create_short_filename(s, filename, directory_start); /* calculate checksum; propagate to long name */ if(entry_long) { -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> FAT12/FAT16 root directory is two sectors in size, which allows only 512 directory entries. Prevent QEMU startup if too much files exist, instead of overflowing root directory. Also introduce variable root_entries, which will be required for FAT32. Fixes: https://bugs.launchpad.net/qemu/+bug/1599539/comments/4 Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ typedef struct BDRVVVFATState { unsigned int cluster_size; unsigned int sectors_per_cluster; unsigned int sectors_per_fat; - unsigned int sectors_of_root_directory; uint32_t last_cluster_of_root_directory; + /* how many entries are available in root directory (0 for FAT32) */ + uint16_t root_entries; uint32_t sector_count; /* total number of sectors of the partition */ uint32_t cluster_count; /* total number of clusters of this partition */ uint32_t max_fat_value; @@ -XXX,XX +XXX,XX @@ static int read_directory(BDRVVVFATState* s, int mapping_index) int is_dot=!strcmp(entry->d_name,"."); int is_dotdot=!strcmp(entry->d_name,".."); + if (first_cluster == 0 && s->directory.next >= s->root_entries - 1) { + fprintf(stderr, "Too many entries in root directory\n"); + closedir(dir); + return -2; + } + if(first_cluster == 0 && (is_dotdot || is_dot)) continue; @@ -XXX,XX +XXX,XX @@ static int read_directory(BDRVVVFATState* s, int mapping_index) memset(direntry,0,sizeof(direntry_t)); } -/* TODO: if there are more entries, bootsector has to be adjusted! */ -#define ROOT_ENTRIES (0x02 * 0x10 * s->sectors_per_cluster) - if (mapping_index == 0 && s->directory.next < ROOT_ENTRIES) { + if (s->fat_type != 32 && + mapping_index == 0 && + s->directory.next < s->root_entries) { /* root directory */ int cur = s->directory.next; - array_ensure_allocated(&(s->directory), ROOT_ENTRIES - 1); - s->directory.next = ROOT_ENTRIES; + array_ensure_allocated(&(s->directory), s->root_entries - 1); + s->directory.next = s->root_entries; memset(array_get(&(s->directory), cur), 0, - (ROOT_ENTRIES - cur) * sizeof(direntry_t)); + (s->root_entries - cur) * sizeof(direntry_t)); } /* re-get the mapping, since s->mapping was possibly realloc()ed */ @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, /* Now build FAT, and write back information into directory */ init_fat(s); + /* TODO: if there are more entries, bootsector has to be adjusted! */ + s->root_entries = 0x02 * 0x10 * s->sectors_per_cluster; s->cluster_count=sector2cluster(s, s->sector_count); mapping = array_get_next(&(s->mapping)); @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, } mapping = array_get(&(s->mapping), 0); - s->sectors_of_root_directory = mapping->end * s->sectors_per_cluster; s->last_cluster_of_root_directory = mapping->end; /* the FAT signature */ @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, bootsector->sectors_per_cluster=s->sectors_per_cluster; bootsector->reserved_sectors=cpu_to_le16(1); bootsector->number_of_fats=0x2; /* number of FATs */ - bootsector->root_entries=cpu_to_le16(s->sectors_of_root_directory*0x10); + bootsector->root_entries = cpu_to_le16(s->root_entries); bootsector->total_sectors16=s->sector_count>0xffff?0:cpu_to_le16(s->sector_count); /* media descriptor: hard disk=0xf8, floppy=0xf0 */ bootsector->media_type = (s->offset_to_bootsector > 0 ? 0xf8 : 0xf0); -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> Specification: "FAT: General overview of on-disk format" v1.03, page 23 Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static direntry_t *create_short_filename(BDRVVVFATState *s, } } + if (entry->name[0] == 0xe5) { + entry->name[0] = 0x05; + } + /* numeric-tail generation */ for (j = 0; j < 8; j++) { if (entry->name[j] == ' ') { @@ -XXX,XX +XXX,XX @@ static inline void init_fat(BDRVVVFATState* s) } -/* TODO: in create_short_filename, 0xe5->0x05 is not yet handled! */ -/* TODO: in parse_short_filename, 0x05->0xe5 is not yet handled! */ static inline direntry_t* create_short_and_long_name(BDRVVVFATState* s, unsigned int directory_start, const char* filename, int is_dot) { @@ -XXX,XX +XXX,XX @@ static int parse_short_name(BDRVVVFATState* s, } else lfn->name[i + j + 1] = '\0'; + if (lfn->name[0] == 0x05) { + lfn->name[0] = 0xe5; + } lfn->len = strlen((char*)lfn->name); return 0; -- 1.8.3.1 From: Hervé Poussineau <hpoussin@reactos.org> According to specification: "'MSWIN4.1' is the recommanded setting, because it is the setting least likely to cause compatibility problems. If you want to put something else in here, that is your option, but the result may be that some FAT drivers might not recognize the volume." Specification: "FAT: General overview of on-disk format" v1.03, page 9 Signed-off-by: Hervé Poussineau <hpoussin@reactos.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vvfat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static int init_directories(BDRVVVFATState* s, bootsector->jump[0]=0xeb; bootsector->jump[1]=0x3e; bootsector->jump[2]=0x90; - memcpy(bootsector->name,"QEMU ",8); + memcpy(bootsector->name, "MSWIN4.1", 8); bootsector->sector_size=cpu_to_le16(0x200); bootsector->sectors_per_cluster=s->sectors_per_cluster; bootsector->reserved_sectors=cpu_to_le16(1); -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> The '-e' and '-6' options to the 'create' & 'convert' commands were "deprecated" in favour of the more generic '-o' option many years ago: commit eec77d9e712bd4157a4e1c0b5a9249d168add738 Author: Jes Sorensen <Jes.Sorensen@redhat.com> Date: Tue Dec 7 17:44:34 2010 +0100 qemu-img: Deprecate obsolete -6 and -e options Except this was never actually a deprecation, which would imply giving the user a warning while the functionality continues to work for a number of releases before eventual removal. Instead the options were immediately turned into an error + exit. Given that the functionality is already broken, there's no point in keeping these psuedo-deprecation messages around any longer. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qemu-img.c | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static int img_create(int argc, char **argv) {"object", required_argument, 0, OPTION_OBJECT}, {0, 0, 0, 0} }; - c = getopt_long(argc, argv, ":F:b:f:he6o:q", + c = getopt_long(argc, argv, ":F:b:f:ho:q", long_options, NULL); if (c == -1) { break; @@ -XXX,XX +XXX,XX @@ static int img_create(int argc, char **argv) case 'f': fmt = optarg; break; - case 'e': - error_report("option -e is deprecated, please use \'-o " - "encryption\' instead!"); - goto fail; - case '6': - error_report("option -6 is deprecated, please use \'-o " - "compat6\' instead!"); - goto fail; case 'o': if (!is_valid_option_list(optarg)) { error_report("Invalid option list: %s", optarg); @@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv) {"target-image-opts", no_argument, 0, OPTION_TARGET_IMAGE_OPTS}, {0, 0, 0, 0} }; - c = getopt_long(argc, argv, ":hf:O:B:ce6o:s:l:S:pt:T:qnm:WU", + c = getopt_long(argc, argv, ":hf:O:B:co:s:l:S:pt:T:qnm:WU", long_options, NULL); if (c == -1) { break; @@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv) case 'c': s.compressed = true; break; - case 'e': - error_report("option -e is deprecated, please use \'-o " - "encryption\' instead!"); - goto fail_getopt; - case '6': - error_report("option -6 is deprecated, please use \'-o " - "compat6\' instead!"); - goto fail_getopt; case 'o': if (!is_valid_option_list(optarg)) { error_report("Invalid option list: %s", optarg); -- 1.8.3.1 From: Thomas Huth <thuth@redhat.com> We likely do not want to carry these legacy -drive options along forever. Let's emit a deprecation warning for the -drive options that have a replacement with the -device option, so that the (hopefully few) remaining users are aware of this and can adapt their scripts / behaviour accordingly. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- blockdev.c | 14 ++++++++++++++ qemu-options.hx | 9 +++++++-- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ #include "qmp-commands.h" #include "block/trace.h" #include "sysemu/arch_init.h" +#include "sysemu/qtest.h" #include "qemu/cutils.h" #include "qemu/help_option.h" #include "qemu/throttle-options.h" @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) const char *filename; Error *local_err = NULL; int i; + const char *deprecated[] = { + "serial", "trans", "secs", "heads", "cyls", "addr" + }; /* Change legacy command line options into QMP ones */ static const struct { @@ -XXX,XX +XXX,XX @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) "update your scripts.\n"); } + /* Other deprecated options */ + if (!qtest_enabled()) { + for (i = 0; i < ARRAY_SIZE(deprecated); i++) { + if (qemu_opt_get(legacy_opts, deprecated[i]) != NULL) { + error_report("'%s' is deprecated, please use the corresponding " + "option of '-device' instead", deprecated[i]); + } + } + } + /* Media type */ value = qemu_opt_get(legacy_opts, "media"); if (value) { diff --git a/qemu-options.hx b/qemu-options.hx index XXXXXXX..XXXXXXX 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -XXX,XX +XXX,XX @@ of available connectors of a given interface type. This option defines the type of the media: disk or cdrom. @item cyls=@var{c},heads=@var{h},secs=@var{s}[,trans=@var{t}] These options have the same definition as they have in @option{-hdachs}. +These parameters are deprecated, use the corresponding parameters +of @code{-device} instead. @item snapshot=@var{snapshot} @var{snapshot} is "on" or "off" and controls snapshot mode for the given drive (see @option{-snapshot}). @@ -XXX,XX +XXX,XX @@ Specify which disk @var{format} will be used rather than detecting the format. Can be used to specify format=raw to avoid interpreting an untrusted format header. @item serial=@var{serial} -This option specifies the serial number to assign to the device. +This option specifies the serial number to assign to the device. This +parameter is deprecated, use the corresponding parameter of @code{-device} +instead. @item addr=@var{addr} -Specify the controller's PCI address (if=virtio only). +Specify the controller's PCI address (if=virtio only). This parameter is +deprecated, use the corresponding parameter of @code{-device} instead. @item werror=@var{action},rerror=@var{action} Specify which @var{action} to take on write and read errors. Valid actions are: "ignore" (ignore the error and try to continue), "stop" (pause QEMU), -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> The user interface specifies job rate limits in bytes/second. It's pointless to have our internal representation track things in sectors/second, particularly since we want to move away from sector-based interfaces. Fix up a doc typo found while verifying that the ratelimit code handles the scaling difference. Repetition of expressions like 'n * BDRV_SECTOR_SIZE' will be cleaned up later when functions are converted to iterate over images by bytes rather than by sectors. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/backup.c | 5 +++-- block/commit.c | 5 +++-- block/mirror.c | 13 +++++++------ block/stream.c | 5 +++-- include/qemu/ratelimit.h | 3 ++- 5 files changed, 18 insertions(+), 13 deletions(-) diff --git a/block/backup.c b/block/backup.c index XXXXXXX..XXXXXXX 100644 --- a/block/backup.c +++ b/block/backup.c @@ -XXX,XX +XXX,XX @@ static void backup_set_speed(BlockJob *job, int64_t speed, Error **errp) error_setg(errp, QERR_INVALID_PARAMETER, "speed"); return; } - ratelimit_set_speed(&s->limit, speed / BDRV_SECTOR_SIZE, SLICE_TIME); + ratelimit_set_speed(&s->limit, speed, SLICE_TIME); } static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) @@ -XXX,XX +XXX,XX @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job) */ if (job->common.speed) { uint64_t delay_ns = ratelimit_calculate_delay(&job->limit, - job->sectors_read); + job->sectors_read * + BDRV_SECTOR_SIZE); job->sectors_read = 0; block_job_sleep_ns(&job->common, QEMU_CLOCK_REALTIME, delay_ns); } else { diff --git a/block/commit.c b/block/commit.c index XXXXXXX..XXXXXXX 100644 --- a/block/commit.c +++ b/block/commit.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) s->common.offset += n * BDRV_SECTOR_SIZE; if (copy && s->common.speed) { - delay_ns = ratelimit_calculate_delay(&s->limit, n); + delay_ns = ratelimit_calculate_delay(&s->limit, + n * BDRV_SECTOR_SIZE); } } @@ -XXX,XX +XXX,XX @@ static void commit_set_speed(BlockJob *job, int64_t speed, Error **errp) error_setg(errp, QERR_INVALID_PARAMETER, "speed"); return; } - ratelimit_set_speed(&s->limit, speed / BDRV_SECTOR_SIZE, SLICE_TIME); + ratelimit_set_speed(&s->limit, speed, SLICE_TIME); } static const BlockJobDriver commit_job_driver = { diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) bitmap_set(s->in_flight_bitmap, sector_num / sectors_per_chunk, nb_chunks); while (nb_chunks > 0 && sector_num < end) { int64_t ret; - int io_sectors, io_sectors_acct; + int io_sectors; + int64_t io_bytes_acct; BlockDriverState *file; enum MirrorMethod { MIRROR_METHOD_COPY, @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) switch (mirror_method) { case MIRROR_METHOD_COPY: io_sectors = mirror_do_read(s, sector_num, io_sectors); - io_sectors_acct = io_sectors; + io_bytes_acct = io_sectors * BDRV_SECTOR_SIZE; break; case MIRROR_METHOD_ZERO: case MIRROR_METHOD_DISCARD: mirror_do_zero_or_discard(s, sector_num, io_sectors, mirror_method == MIRROR_METHOD_DISCARD); if (write_zeroes_ok) { - io_sectors_acct = 0; + io_bytes_acct = 0; } else { - io_sectors_acct = io_sectors; + io_bytes_acct = io_sectors * BDRV_SECTOR_SIZE; } break; default: @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) sector_num += io_sectors; nb_chunks -= DIV_ROUND_UP(io_sectors, sectors_per_chunk); if (s->common.speed) { - delay_ns = ratelimit_calculate_delay(&s->limit, io_sectors_acct); + delay_ns = ratelimit_calculate_delay(&s->limit, io_bytes_acct); } } return delay_ns; @@ -XXX,XX +XXX,XX @@ static void mirror_set_speed(BlockJob *job, int64_t speed, Error **errp) error_setg(errp, QERR_INVALID_PARAMETER, "speed"); return; } - ratelimit_set_speed(&s->limit, speed / BDRV_SECTOR_SIZE, SLICE_TIME); + ratelimit_set_speed(&s->limit, speed, SLICE_TIME); } static void mirror_complete(BlockJob *job, Error **errp) diff --git a/block/stream.c b/block/stream.c index XXXXXXX..XXXXXXX 100644 --- a/block/stream.c +++ b/block/stream.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) /* Publish progress */ s->common.offset += n * BDRV_SECTOR_SIZE; if (copy && s->common.speed) { - delay_ns = ratelimit_calculate_delay(&s->limit, n); + delay_ns = ratelimit_calculate_delay(&s->limit, + n * BDRV_SECTOR_SIZE); } } @@ -XXX,XX +XXX,XX @@ static void stream_set_speed(BlockJob *job, int64_t speed, Error **errp) error_setg(errp, QERR_INVALID_PARAMETER, "speed"); return; } - ratelimit_set_speed(&s->limit, speed / BDRV_SECTOR_SIZE, SLICE_TIME); + ratelimit_set_speed(&s->limit, speed, SLICE_TIME); } static const BlockJobDriver stream_job_driver = { diff --git a/include/qemu/ratelimit.h b/include/qemu/ratelimit.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/ratelimit.h +++ b/include/qemu/ratelimit.h @@ -XXX,XX +XXX,XX @@ typedef struct { /** Calculate and return delay for next request in ns * - * Record that we sent @p n data units. If we may send more data units + * Record that we sent @n data units (where @n matches the scale chosen + * during ratelimit_set_speed). If we may send more data units * in the current time slice, return 0 (i.e. no delay). Otherwise * return the amount of time (in ns) until the start of the next time * slice that will permit sending the next chunk of data. -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> Upcoming patches are going to switch to byte-based interfaces instead of sector-based. Even worse, trace_backup_do_cow_enter() had a weird mix of cluster and sector indices. The trace interface is low enough that there are no stability guarantees, and therefore nothing wrong with changing our units, even in cases like trace_backup_do_cow_skip() where we are not changing the trace output. So make the tracing uniformly use bytes. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/backup.c | 16 ++++++++++------ block/commit.c | 3 ++- block/mirror.c | 26 +++++++++++++++++--------- block/stream.c | 3 ++- block/trace-events | 14 +++++++------- 5 files changed, 38 insertions(+), 24 deletions(-) diff --git a/block/backup.c b/block/backup.c index XXXXXXX..XXXXXXX 100644 --- a/block/backup.c +++ b/block/backup.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, void *bounce_buffer = NULL; int ret = 0; int64_t sectors_per_cluster = cluster_size_sectors(job); + int64_t bytes_per_cluster = sectors_per_cluster * BDRV_SECTOR_SIZE; int64_t start, end; int n; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, start = sector_num / sectors_per_cluster; end = DIV_ROUND_UP(sector_num + nb_sectors, sectors_per_cluster); - trace_backup_do_cow_enter(job, start, sector_num, nb_sectors); + trace_backup_do_cow_enter(job, start * bytes_per_cluster, + sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE); wait_for_overlapping_requests(job, start, end); cow_request_begin(&cow_request, job, start, end); for (; start < end; start++) { if (test_bit(start, job->done_bitmap)) { - trace_backup_do_cow_skip(job, start); + trace_backup_do_cow_skip(job, start * bytes_per_cluster); continue; /* already copied */ } - trace_backup_do_cow_process(job, start); + trace_backup_do_cow_process(job, start * bytes_per_cluster); n = MIN(sectors_per_cluster, job->common.len / BDRV_SECTOR_SIZE - @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, bounce_qiov.size, &bounce_qiov, is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0); if (ret < 0) { - trace_backup_do_cow_read_fail(job, start, ret); + trace_backup_do_cow_read_fail(job, start * bytes_per_cluster, ret); if (error_is_read) { *error_is_read = true; } @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0); } if (ret < 0) { - trace_backup_do_cow_write_fail(job, start, ret); + trace_backup_do_cow_write_fail(job, start * bytes_per_cluster, ret); if (error_is_read) { *error_is_read = false; } @@ -XXX,XX +XXX,XX @@ out: cow_request_end(&cow_request); - trace_backup_do_cow_return(job, sector_num, nb_sectors, ret); + trace_backup_do_cow_return(job, sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE, ret); qemu_co_rwlock_unlock(&job->flush_rwlock); diff --git a/block/commit.c b/block/commit.c index XXXXXXX..XXXXXXX 100644 --- a/block/commit.c +++ b/block/commit.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) COMMIT_BUFFER_SIZE / BDRV_SECTOR_SIZE, &n); copy = (ret == 1); - trace_commit_one_iteration(s, sector_num, n, ret); + trace_commit_one_iteration(s, sector_num * BDRV_SECTOR_SIZE, + n * BDRV_SECTOR_SIZE, ret); if (copy) { ret = commit_populate(s->top, s->base, sector_num, n, buf); bytes_written += n * BDRV_SECTOR_SIZE; diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static void mirror_iteration_done(MirrorOp *op, int ret) int64_t chunk_num; int i, nb_chunks, sectors_per_chunk; - trace_mirror_iteration_done(s, op->sector_num, op->nb_sectors, ret); + trace_mirror_iteration_done(s, op->sector_num * BDRV_SECTOR_SIZE, + op->nb_sectors * BDRV_SECTOR_SIZE, ret); s->in_flight--; s->sectors_in_flight -= op->nb_sectors; @@ -XXX,XX +XXX,XX @@ static int mirror_do_read(MirrorBlockJob *s, int64_t sector_num, nb_chunks = DIV_ROUND_UP(nb_sectors, sectors_per_chunk); while (s->buf_free_count < nb_chunks) { - trace_mirror_yield_in_flight(s, sector_num, s->in_flight); + trace_mirror_yield_in_flight(s, sector_num * BDRV_SECTOR_SIZE, + s->in_flight); mirror_wait_for_io(s); } @@ -XXX,XX +XXX,XX @@ static int mirror_do_read(MirrorBlockJob *s, int64_t sector_num, /* Copy the dirty cluster. */ s->in_flight++; s->sectors_in_flight += nb_sectors; - trace_mirror_one_iteration(s, sector_num, nb_sectors); + trace_mirror_one_iteration(s, sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE); blk_aio_preadv(source, sector_num * BDRV_SECTOR_SIZE, &op->qiov, 0, mirror_read_complete, op); @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) if (sector_num < 0) { bdrv_set_dirty_iter(s->dbi, 0); sector_num = bdrv_dirty_iter_next(s->dbi); - trace_mirror_restart_iter(s, bdrv_get_dirty_count(s->dirty_bitmap)); + trace_mirror_restart_iter(s, bdrv_get_dirty_count(s->dirty_bitmap) * + BDRV_SECTOR_SIZE); assert(sector_num >= 0); } bdrv_dirty_bitmap_unlock(s->dirty_bitmap); first_chunk = sector_num / sectors_per_chunk; while (test_bit(first_chunk, s->in_flight_bitmap)) { - trace_mirror_yield_in_flight(s, sector_num, s->in_flight); + trace_mirror_yield_in_flight(s, sector_num * BDRV_SECTOR_SIZE, + s->in_flight); mirror_wait_for_io(s); } @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) } while (s->in_flight >= MAX_IN_FLIGHT) { - trace_mirror_yield_in_flight(s, sector_num, s->in_flight); + trace_mirror_yield_in_flight(s, sector_num * BDRV_SECTOR_SIZE, + s->in_flight); mirror_wait_for_io(s); } @@ -XXX,XX +XXX,XX @@ static void coroutine_fn mirror_run(void *opaque) s->common.iostatus == BLOCK_DEVICE_IO_STATUS_OK) { if (s->in_flight >= MAX_IN_FLIGHT || s->buf_free_count == 0 || (cnt == 0 && s->in_flight > 0)) { - trace_mirror_yield(s, cnt, s->buf_free_count, s->in_flight); + trace_mirror_yield(s, cnt * BDRV_SECTOR_SIZE, + s->buf_free_count, s->in_flight); mirror_wait_for_io(s); continue; } else if (cnt != 0) { @@ -XXX,XX +XXX,XX @@ static void coroutine_fn mirror_run(void *opaque) * whether to switch to target check one last time if I/O has * come in the meanwhile, and if not flush the data to disk. */ - trace_mirror_before_drain(s, cnt); + trace_mirror_before_drain(s, cnt * BDRV_SECTOR_SIZE); bdrv_drained_begin(bs); cnt = bdrv_get_dirty_count(s->dirty_bitmap); @@ -XXX,XX +XXX,XX @@ static void coroutine_fn mirror_run(void *opaque) } ret = 0; - trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); + trace_mirror_before_sleep(s, cnt * BDRV_SECTOR_SIZE, + s->synced, delay_ns); if (!s->synced) { block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, delay_ns); if (block_job_is_cancelled(&s->common)) { diff --git a/block/stream.c b/block/stream.c index XXXXXXX..XXXXXXX 100644 --- a/block/stream.c +++ b/block/stream.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) copy = (ret == 1); } - trace_stream_one_iteration(s, sector_num, n, ret); + trace_stream_one_iteration(s, sector_num * BDRV_SECTOR_SIZE, + n * BDRV_SECTOR_SIZE, ret); if (copy) { ret = stream_populate(blk, sector_num, n, buf); } diff --git a/block/trace-events b/block/trace-events index XXXXXXX..XXXXXXX 100644 --- a/block/trace-events +++ b/block/trace-events @@ -XXX,XX +XXX,XX @@ bdrv_co_pwrite_zeroes(void *bs, int64_t offset, int count, int flags) "bs %p off bdrv_co_do_copy_on_readv(void *bs, int64_t offset, unsigned int bytes, int64_t cluster_offset, unsigned int cluster_bytes) "bs %p offset %"PRId64" bytes %u cluster_offset %"PRId64" cluster_bytes %u" # block/stream.c -stream_one_iteration(void *s, int64_t sector_num, int nb_sectors, int is_allocated) "s %p sector_num %"PRId64" nb_sectors %d is_allocated %d" +stream_one_iteration(void *s, int64_t offset, uint64_t bytes, int is_allocated) "s %p offset %" PRId64 " bytes %" PRIu64 " is_allocated %d" stream_start(void *bs, void *base, void *s) "bs %p base %p s %p" # block/commit.c -commit_one_iteration(void *s, int64_t sector_num, int nb_sectors, int is_allocated) "s %p sector_num %"PRId64" nb_sectors %d is_allocated %d" +commit_one_iteration(void *s, int64_t offset, uint64_t bytes, int is_allocated) "s %p offset %" PRId64 " bytes %" PRIu64 " is_allocated %d" commit_start(void *bs, void *base, void *top, void *s) "bs %p base %p top %p s %p" # block/mirror.c @@ -XXX,XX +XXX,XX @@ mirror_restart_iter(void *s, int64_t cnt) "s %p dirty count %"PRId64 mirror_before_flush(void *s) "s %p" mirror_before_drain(void *s, int64_t cnt) "s %p dirty count %"PRId64 mirror_before_sleep(void *s, int64_t cnt, int synced, uint64_t delay_ns) "s %p dirty count %"PRId64" synced %d delay %"PRIu64"ns" -mirror_one_iteration(void *s, int64_t sector_num, int nb_sectors) "s %p sector_num %"PRId64" nb_sectors %d" -mirror_iteration_done(void *s, int64_t sector_num, int nb_sectors, int ret) "s %p sector_num %"PRId64" nb_sectors %d ret %d" +mirror_one_iteration(void *s, int64_t offset, uint64_t bytes) "s %p offset %" PRId64 " bytes %" PRIu64 +mirror_iteration_done(void *s, int64_t offset, uint64_t bytes, int ret) "s %p offset %" PRId64 " bytes %" PRIu64 " ret %d" mirror_yield(void *s, int64_t cnt, int buf_free_count, int in_flight) "s %p dirty count %"PRId64" free buffers %d in_flight %d" -mirror_yield_in_flight(void *s, int64_t sector_num, int in_flight) "s %p sector_num %"PRId64" in_flight %d" +mirror_yield_in_flight(void *s, int64_t offset, int in_flight) "s %p offset %" PRId64 " in_flight %d" # block/backup.c -backup_do_cow_enter(void *job, int64_t start, int64_t sector_num, int nb_sectors) "job %p start %"PRId64" sector_num %"PRId64" nb_sectors %d" -backup_do_cow_return(void *job, int64_t sector_num, int nb_sectors, int ret) "job %p sector_num %"PRId64" nb_sectors %d ret %d" +backup_do_cow_enter(void *job, int64_t start, int64_t offset, uint64_t bytes) "job %p start %" PRId64 " offset %" PRId64 " bytes %" PRIu64 +backup_do_cow_return(void *job, int64_t offset, uint64_t bytes, int ret) "job %p offset %" PRId64 " bytes %" PRIu64 " ret %d" backup_do_cow_skip(void *job, int64_t start) "job %p start %"PRId64 backup_do_cow_process(void *job, int64_t start) "job %p start %"PRId64 backup_do_cow_read_fail(void *job, int64_t start, int ret) "job %p start %"PRId64" ret %d" -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Start by converting an internal function (no semantic change). Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/stream.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/block/stream.c b/block/stream.c index XXXXXXX..XXXXXXX 100644 --- a/block/stream.c +++ b/block/stream.c @@ -XXX,XX +XXX,XX @@ typedef struct StreamBlockJob { } StreamBlockJob; static int coroutine_fn stream_populate(BlockBackend *blk, - int64_t sector_num, int nb_sectors, + int64_t offset, uint64_t bytes, void *buf) { struct iovec iov = { .iov_base = buf, - .iov_len = nb_sectors * BDRV_SECTOR_SIZE, + .iov_len = bytes, }; QEMUIOVector qiov; + assert(bytes < SIZE_MAX); qemu_iovec_init_external(&qiov, &iov, 1); /* Copy-on-read the unallocated clusters */ - return blk_co_preadv(blk, sector_num * BDRV_SECTOR_SIZE, qiov.size, &qiov, - BDRV_REQ_COPY_ON_READ); + return blk_co_preadv(blk, offset, qiov.size, &qiov, BDRV_REQ_COPY_ON_READ); } typedef struct { @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) trace_stream_one_iteration(s, sector_num * BDRV_SECTOR_SIZE, n * BDRV_SECTOR_SIZE, ret); if (copy) { - ret = stream_populate(blk, sector_num, n, buf); + ret = stream_populate(blk, sector_num * BDRV_SECTOR_SIZE, + n * BDRV_SECTOR_SIZE, buf); } if (ret < 0) { BlockErrorAction action = -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> stream_complete() skips the work of rewriting the backing file if the job was cancelled, if data->reached_end is false, or if there was an error detected (non-zero data->ret) during the streaming. But note that in stream_run(), data->reached_end is only set if the loop ran to completion, and data->ret is only 0 in two cases: either the loop ran to completion (possibly by cancellation, but stream_complete checks for that), or we took an early goto out because there is no bs->backing. Thus, we can preserve the same semantics without the use of reached_end, by merely checking for bs->backing (and logically, if there was no backing file, streaming is a no-op, so there is no backing file to rewrite). Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/stream.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/block/stream.c b/block/stream.c index XXXXXXX..XXXXXXX 100644 --- a/block/stream.c +++ b/block/stream.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn stream_populate(BlockBackend *blk, typedef struct { int ret; - bool reached_end; } StreamCompleteData; static void stream_complete(BlockJob *job, void *opaque) @@ -XXX,XX +XXX,XX @@ static void stream_complete(BlockJob *job, void *opaque) BlockDriverState *base = s->base; Error *local_err = NULL; - if (!block_job_is_cancelled(&s->common) && data->reached_end && + if (!block_job_is_cancelled(&s->common) && bs->backing && data->ret == 0) { const char *base_id = NULL, *base_fmt = NULL; if (base) { @@ -XXX,XX +XXX,XX @@ out: /* Modify backing chain and close BDSes in main loop */ data = g_malloc(sizeof(*data)); data->ret = ret; - data->reached_end = sector_num == end; block_job_defer_to_main_loop(&s->common, stream_complete, data); } -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Change the internal loop iteration of streaming to track by bytes instead of sectors (although we are still guaranteed that we iterate by steps that are sector-aligned). Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/stream.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/block/stream.c b/block/stream.c index XXXXXXX..XXXXXXX 100644 --- a/block/stream.c +++ b/block/stream.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) BlockBackend *blk = s->common.blk; BlockDriverState *bs = blk_bs(blk); BlockDriverState *base = s->base; - int64_t sector_num = 0; - int64_t end = -1; + int64_t offset = 0; uint64_t delay_ns = 0; int error = 0; int ret = 0; - int n = 0; + int n = 0; /* sectors */ void *buf; if (!bs->backing) { @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) goto out; } - end = s->common.len >> BDRV_SECTOR_BITS; buf = qemu_blockalign(bs, STREAM_BUFFER_SIZE); /* Turn on copy-on-read for the whole block device so that guest read @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) bdrv_enable_copy_on_read(bs); } - for (sector_num = 0; sector_num < end; sector_num += n) { + for ( ; offset < s->common.len; offset += n * BDRV_SECTOR_SIZE) { bool copy; /* Note that even when no rate limit is applied we need to yield @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) copy = false; - ret = bdrv_is_allocated(bs, sector_num, + ret = bdrv_is_allocated(bs, offset / BDRV_SECTOR_SIZE, STREAM_BUFFER_SIZE / BDRV_SECTOR_SIZE, &n); if (ret == 1) { /* Allocated in the top, no need to copy. */ } else if (ret >= 0) { /* Copy if allocated in the intermediate images. Limit to the - * known-unallocated area [sector_num, sector_num+n). */ + * known-unallocated area [offset, offset+n*BDRV_SECTOR_SIZE). */ ret = bdrv_is_allocated_above(backing_bs(bs), base, - sector_num, n, &n); + offset / BDRV_SECTOR_SIZE, n, &n); /* Finish early if end of backing file has been reached */ if (ret == 0 && n == 0) { - n = end - sector_num; + n = (s->common.len - offset) / BDRV_SECTOR_SIZE; } copy = (ret == 1); } - trace_stream_one_iteration(s, sector_num * BDRV_SECTOR_SIZE, - n * BDRV_SECTOR_SIZE, ret); + trace_stream_one_iteration(s, offset, n * BDRV_SECTOR_SIZE, ret); if (copy) { - ret = stream_populate(blk, sector_num * BDRV_SECTOR_SIZE, - n * BDRV_SECTOR_SIZE, buf); + ret = stream_populate(blk, offset, n * BDRV_SECTOR_SIZE, buf); } if (ret < 0) { BlockErrorAction action = -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Start by converting an internal function (no semantic change). Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/commit.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/block/commit.c b/block/commit.c index XXXXXXX..XXXXXXX 100644 --- a/block/commit.c +++ b/block/commit.c @@ -XXX,XX +XXX,XX @@ typedef struct CommitBlockJob { } CommitBlockJob; static int coroutine_fn commit_populate(BlockBackend *bs, BlockBackend *base, - int64_t sector_num, int nb_sectors, + int64_t offset, uint64_t bytes, void *buf) { int ret = 0; QEMUIOVector qiov; struct iovec iov = { .iov_base = buf, - .iov_len = nb_sectors * BDRV_SECTOR_SIZE, + .iov_len = bytes, }; + assert(bytes < SIZE_MAX); qemu_iovec_init_external(&qiov, &iov, 1); - ret = blk_co_preadv(bs, sector_num * BDRV_SECTOR_SIZE, - qiov.size, &qiov, 0); + ret = blk_co_preadv(bs, offset, qiov.size, &qiov, 0); if (ret < 0) { return ret; } - ret = blk_co_pwritev(base, sector_num * BDRV_SECTOR_SIZE, - qiov.size, &qiov, 0); + ret = blk_co_pwritev(base, offset, qiov.size, &qiov, 0); if (ret < 0) { return ret; } @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) trace_commit_one_iteration(s, sector_num * BDRV_SECTOR_SIZE, n * BDRV_SECTOR_SIZE, ret); if (copy) { - ret = commit_populate(s->top, s->base, sector_num, n, buf); + ret = commit_populate(s->top, s->base, + sector_num * BDRV_SECTOR_SIZE, + n * BDRV_SECTOR_SIZE, buf); bytes_written += n * BDRV_SECTOR_SIZE; } if (ret < 0) { -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Change the internal loop iteration of committing to track by bytes instead of sectors (although we are still guaranteed that we iterate by steps that are sector-aligned). Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/commit.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/block/commit.c b/block/commit.c index XXXXXXX..XXXXXXX 100644 --- a/block/commit.c +++ b/block/commit.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) { CommitBlockJob *s = opaque; CommitCompleteData *data; - int64_t sector_num, end; + int64_t offset; uint64_t delay_ns = 0; int ret = 0; - int n = 0; + int n = 0; /* sectors */ void *buf = NULL; int bytes_written = 0; int64_t base_len; ret = s->common.len = blk_getlength(s->top); - if (s->common.len < 0) { goto out; } @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) } } - end = s->common.len >> BDRV_SECTOR_BITS; buf = blk_blockalign(s->top, COMMIT_BUFFER_SIZE); - for (sector_num = 0; sector_num < end; sector_num += n) { + for (offset = 0; offset < s->common.len; offset += n * BDRV_SECTOR_SIZE) { bool copy; /* Note that even when no rate limit is applied we need to yield @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) } /* Copy if allocated above the base */ ret = bdrv_is_allocated_above(blk_bs(s->top), blk_bs(s->base), - sector_num, + offset / BDRV_SECTOR_SIZE, COMMIT_BUFFER_SIZE / BDRV_SECTOR_SIZE, &n); copy = (ret == 1); - trace_commit_one_iteration(s, sector_num * BDRV_SECTOR_SIZE, - n * BDRV_SECTOR_SIZE, ret); + trace_commit_one_iteration(s, offset, n * BDRV_SECTOR_SIZE, ret); if (copy) { - ret = commit_populate(s->top, s->base, - sector_num * BDRV_SECTOR_SIZE, + ret = commit_populate(s->top, s->base, offset, n * BDRV_SECTOR_SIZE, buf); bytes_written += n * BDRV_SECTOR_SIZE; } -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Continue by converting an internal structure (no semantic change), and all references to the buffer size. Add an assertion that our use of s->granularity >> BDRV_SECTOR_BITS (necessary for interaction with sector-based dirty bitmaps, until a later patch converts those to be byte-based) does not suffer from truncation problems. [checkpatch has a false positive on use of MIN() in this patch] Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/mirror.c | 84 +++++++++++++++++++++++++++++----------------------------- 1 file changed, 42 insertions(+), 42 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ #define SLICE_TIME 100000000ULL /* ns */ #define MAX_IN_FLIGHT 16 -#define MAX_IO_SECTORS ((1 << 20) >> BDRV_SECTOR_BITS) /* 1 Mb */ -#define DEFAULT_MIRROR_BUF_SIZE \ - (MAX_IN_FLIGHT * MAX_IO_SECTORS * BDRV_SECTOR_SIZE) +#define MAX_IO_BYTES (1 << 20) /* 1 Mb */ +#define DEFAULT_MIRROR_BUF_SIZE (MAX_IN_FLIGHT * MAX_IO_BYTES) /* The mirroring buffer is a list of granularity-sized chunks. * Free chunks are organized in a list. @@ -XXX,XX +XXX,XX @@ typedef struct MirrorBlockJob { uint64_t last_pause_ns; unsigned long *in_flight_bitmap; int in_flight; - int64_t sectors_in_flight; + int64_t bytes_in_flight; int ret; bool unmap; bool waiting_for_io; - int target_cluster_sectors; + int target_cluster_size; int max_iov; bool initial_zeroing_ongoing; } MirrorBlockJob; @@ -XXX,XX +XXX,XX @@ typedef struct MirrorBlockJob { typedef struct MirrorOp { MirrorBlockJob *s; QEMUIOVector qiov; - int64_t sector_num; - int nb_sectors; + int64_t offset; + uint64_t bytes; } MirrorOp; static BlockErrorAction mirror_error_action(MirrorBlockJob *s, bool read, @@ -XXX,XX +XXX,XX @@ static void mirror_iteration_done(MirrorOp *op, int ret) MirrorBlockJob *s = op->s; struct iovec *iov; int64_t chunk_num; - int i, nb_chunks, sectors_per_chunk; + int i, nb_chunks; - trace_mirror_iteration_done(s, op->sector_num * BDRV_SECTOR_SIZE, - op->nb_sectors * BDRV_SECTOR_SIZE, ret); + trace_mirror_iteration_done(s, op->offset, op->bytes, ret); s->in_flight--; - s->sectors_in_flight -= op->nb_sectors; + s->bytes_in_flight -= op->bytes; iov = op->qiov.iov; for (i = 0; i < op->qiov.niov; i++) { MirrorBuffer *buf = (MirrorBuffer *) iov[i].iov_base; @@ -XXX,XX +XXX,XX @@ static void mirror_iteration_done(MirrorOp *op, int ret) s->buf_free_count++; } - sectors_per_chunk = s->granularity >> BDRV_SECTOR_BITS; - chunk_num = op->sector_num / sectors_per_chunk; - nb_chunks = DIV_ROUND_UP(op->nb_sectors, sectors_per_chunk); + chunk_num = op->offset / s->granularity; + nb_chunks = DIV_ROUND_UP(op->bytes, s->granularity); bitmap_clear(s->in_flight_bitmap, chunk_num, nb_chunks); if (ret >= 0) { if (s->cow_bitmap) { bitmap_set(s->cow_bitmap, chunk_num, nb_chunks); } if (!s->initial_zeroing_ongoing) { - s->common.offset += (uint64_t)op->nb_sectors * BDRV_SECTOR_SIZE; + s->common.offset += op->bytes; } } qemu_iovec_destroy(&op->qiov); @@ -XXX,XX +XXX,XX @@ static void mirror_write_complete(void *opaque, int ret) if (ret < 0) { BlockErrorAction action; - bdrv_set_dirty_bitmap(s->dirty_bitmap, op->sector_num, op->nb_sectors); + bdrv_set_dirty_bitmap(s->dirty_bitmap, op->offset >> BDRV_SECTOR_BITS, + op->bytes >> BDRV_SECTOR_BITS); action = mirror_error_action(s, false, -ret); if (action == BLOCK_ERROR_ACTION_REPORT && s->ret >= 0) { s->ret = ret; @@ -XXX,XX +XXX,XX @@ static void mirror_read_complete(void *opaque, int ret) if (ret < 0) { BlockErrorAction action; - bdrv_set_dirty_bitmap(s->dirty_bitmap, op->sector_num, op->nb_sectors); + bdrv_set_dirty_bitmap(s->dirty_bitmap, op->offset >> BDRV_SECTOR_BITS, + op->bytes >> BDRV_SECTOR_BITS); action = mirror_error_action(s, true, -ret); if (action == BLOCK_ERROR_ACTION_REPORT && s->ret >= 0) { s->ret = ret; @@ -XXX,XX +XXX,XX @@ static void mirror_read_complete(void *opaque, int ret) mirror_iteration_done(op, ret); } else { - blk_aio_pwritev(s->target, op->sector_num * BDRV_SECTOR_SIZE, &op->qiov, + blk_aio_pwritev(s->target, op->offset, &op->qiov, 0, mirror_write_complete, op); } aio_context_release(blk_get_aio_context(s->common.blk)); @@ -XXX,XX +XXX,XX @@ static int mirror_cow_align(MirrorBlockJob *s, align_nb_sectors = max_sectors; if (need_cow) { align_nb_sectors = QEMU_ALIGN_DOWN(align_nb_sectors, - s->target_cluster_sectors); + s->target_cluster_size >> + BDRV_SECTOR_BITS); } } /* Clipping may result in align_nb_sectors unaligned to chunk boundary, but @@ -XXX,XX +XXX,XX @@ static int mirror_do_read(MirrorBlockJob *s, int64_t sector_num, /* Allocate a MirrorOp that is used as an AIO callback. */ op = g_new(MirrorOp, 1); op->s = s; - op->sector_num = sector_num; - op->nb_sectors = nb_sectors; + op->offset = sector_num * BDRV_SECTOR_SIZE; + op->bytes = nb_sectors * BDRV_SECTOR_SIZE; /* Now make a QEMUIOVector taking enough granularity-sized chunks * from s->buf_free. @@ -XXX,XX +XXX,XX @@ static int mirror_do_read(MirrorBlockJob *s, int64_t sector_num, /* Copy the dirty cluster. */ s->in_flight++; - s->sectors_in_flight += nb_sectors; + s->bytes_in_flight += nb_sectors * BDRV_SECTOR_SIZE; trace_mirror_one_iteration(s, sector_num * BDRV_SECTOR_SIZE, nb_sectors * BDRV_SECTOR_SIZE); @@ -XXX,XX +XXX,XX @@ static void mirror_do_zero_or_discard(MirrorBlockJob *s, * so the freeing in mirror_iteration_done is nop. */ op = g_new0(MirrorOp, 1); op->s = s; - op->sector_num = sector_num; - op->nb_sectors = nb_sectors; + op->offset = sector_num * BDRV_SECTOR_SIZE; + op->bytes = nb_sectors * BDRV_SECTOR_SIZE; s->in_flight++; - s->sectors_in_flight += nb_sectors; + s->bytes_in_flight += nb_sectors * BDRV_SECTOR_SIZE; if (is_discard) { blk_aio_pdiscard(s->target, sector_num << BDRV_SECTOR_BITS, - op->nb_sectors << BDRV_SECTOR_BITS, - mirror_write_complete, op); + op->bytes, mirror_write_complete, op); } else { blk_aio_pwrite_zeroes(s->target, sector_num * BDRV_SECTOR_SIZE, - op->nb_sectors * BDRV_SECTOR_SIZE, - s->unmap ? BDRV_REQ_MAY_UNMAP : 0, + op->bytes, s->unmap ? BDRV_REQ_MAY_UNMAP : 0, mirror_write_complete, op); } } @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) int64_t end = s->bdev_length / BDRV_SECTOR_SIZE; int sectors_per_chunk = s->granularity >> BDRV_SECTOR_BITS; bool write_zeroes_ok = bdrv_can_write_zeroes_with_unmap(blk_bs(s->target)); - int max_io_sectors = MAX((s->buf_size >> BDRV_SECTOR_BITS) / MAX_IN_FLIGHT, - MAX_IO_SECTORS); + int max_io_bytes = MAX(s->buf_size / MAX_IN_FLIGHT, MAX_IO_BYTES); bdrv_dirty_bitmap_lock(s->dirty_bitmap); sector_num = bdrv_dirty_iter_next(s->dbi); @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) nb_chunks * sectors_per_chunk, &io_sectors, &file); if (ret < 0) { - io_sectors = MIN(nb_chunks * sectors_per_chunk, max_io_sectors); + io_sectors = MIN(nb_chunks * sectors_per_chunk, + max_io_bytes >> BDRV_SECTOR_BITS); } else if (ret & BDRV_BLOCK_DATA) { - io_sectors = MIN(io_sectors, max_io_sectors); + io_sectors = MIN(io_sectors, max_io_bytes >> BDRV_SECTOR_BITS); } io_sectors -= io_sectors % sectors_per_chunk; @@ -XXX,XX +XXX,XX @@ static void coroutine_fn mirror_run(void *opaque) char backing_filename[2]; /* we only need 2 characters because we are only checking for a NULL string */ int ret = 0; - int target_cluster_size = BDRV_SECTOR_SIZE; if (block_job_is_cancelled(&s->common)) { goto immediate_exit; @@ -XXX,XX +XXX,XX @@ static void coroutine_fn mirror_run(void *opaque) bdrv_get_backing_filename(target_bs, backing_filename, sizeof(backing_filename)); if (!bdrv_get_info(target_bs, &bdi) && bdi.cluster_size) { - target_cluster_size = bdi.cluster_size; + s->target_cluster_size = bdi.cluster_size; + } else { + s->target_cluster_size = BDRV_SECTOR_SIZE; } - if (backing_filename[0] && !target_bs->backing - && s->granularity < target_cluster_size) { - s->buf_size = MAX(s->buf_size, target_cluster_size); + if (backing_filename[0] && !target_bs->backing && + s->granularity < s->target_cluster_size) { + s->buf_size = MAX(s->buf_size, s->target_cluster_size); s->cow_bitmap = bitmap_new(length); } - s->target_cluster_sectors = target_cluster_size >> BDRV_SECTOR_BITS; s->max_iov = MIN(bs->bl.max_iov, target_bs->bl.max_iov); s->buf = qemu_try_blockalign(bs, s->buf_size); @@ -XXX,XX +XXX,XX @@ static void coroutine_fn mirror_run(void *opaque) cnt = bdrv_get_dirty_count(s->dirty_bitmap); /* s->common.offset contains the number of bytes already processed so * far, cnt is the number of dirty sectors remaining and - * s->sectors_in_flight is the number of sectors currently being + * s->bytes_in_flight is the number of bytes currently being * processed; together those are the current total operation length */ - s->common.len = s->common.offset + - (cnt + s->sectors_in_flight) * BDRV_SECTOR_SIZE; + s->common.len = s->common.offset + s->bytes_in_flight + + cnt * BDRV_SECTOR_SIZE; /* Note that even when no rate limit is applied we need to yield * periodically with no pending I/O so that bdrv_drain_all() returns. @@ -XXX,XX +XXX,XX @@ static void mirror_start_job(const char *job_id, BlockDriverState *bs, } assert ((granularity & (granularity - 1)) == 0); + /* Granularity must be large enough for sector-based dirty bitmap */ + assert(granularity >= BDRV_SECTOR_SIZE); if (buf_size < 0) { error_setg(errp, "Invalid parameter 'buf-size'"); -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert another internal function (no semantic change). Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/mirror.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static int mirror_do_read(MirrorBlockJob *s, int64_t sector_num, } static void mirror_do_zero_or_discard(MirrorBlockJob *s, - int64_t sector_num, - int nb_sectors, + int64_t offset, + uint64_t bytes, bool is_discard) { MirrorOp *op; @@ -XXX,XX +XXX,XX @@ static void mirror_do_zero_or_discard(MirrorBlockJob *s, * so the freeing in mirror_iteration_done is nop. */ op = g_new0(MirrorOp, 1); op->s = s; - op->offset = sector_num * BDRV_SECTOR_SIZE; - op->bytes = nb_sectors * BDRV_SECTOR_SIZE; + op->offset = offset; + op->bytes = bytes; s->in_flight++; - s->bytes_in_flight += nb_sectors * BDRV_SECTOR_SIZE; + s->bytes_in_flight += bytes; if (is_discard) { - blk_aio_pdiscard(s->target, sector_num << BDRV_SECTOR_BITS, + blk_aio_pdiscard(s->target, offset, op->bytes, mirror_write_complete, op); } else { - blk_aio_pwrite_zeroes(s->target, sector_num * BDRV_SECTOR_SIZE, + blk_aio_pwrite_zeroes(s->target, offset, op->bytes, s->unmap ? BDRV_REQ_MAY_UNMAP : 0, mirror_write_complete, op); } @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) break; case MIRROR_METHOD_ZERO: case MIRROR_METHOD_DISCARD: - mirror_do_zero_or_discard(s, sector_num, io_sectors, + mirror_do_zero_or_discard(s, sector_num * BDRV_SECTOR_SIZE, + io_sectors * BDRV_SECTOR_SIZE, mirror_method == MIRROR_METHOD_DISCARD); if (write_zeroes_ok) { io_bytes_acct = 0; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s) continue; } - mirror_do_zero_or_discard(s, sector_num, nb_sectors, false); + mirror_do_zero_or_discard(s, sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE, false); sector_num += nb_sectors; } -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> Rather than having a void function that modifies its input in-place as the output, change the signature to reduce a layer of indirection and return the result. Suggested-by: John Snow <jsnow@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/mirror.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static void mirror_read_complete(void *opaque, int ret) aio_context_release(blk_get_aio_context(s->common.blk)); } -static inline void mirror_clip_sectors(MirrorBlockJob *s, - int64_t sector_num, - int *nb_sectors) +static inline int mirror_clip_sectors(MirrorBlockJob *s, + int64_t sector_num, + int nb_sectors) { - *nb_sectors = MIN(*nb_sectors, - s->bdev_length / BDRV_SECTOR_SIZE - sector_num); + return MIN(nb_sectors, + s->bdev_length / BDRV_SECTOR_SIZE - sector_num); } /* Round sector_num and/or nb_sectors to target cluster if COW is needed, and @@ -XXX,XX +XXX,XX @@ static int mirror_cow_align(MirrorBlockJob *s, } /* Clipping may result in align_nb_sectors unaligned to chunk boundary, but * that doesn't matter because it's already the end of source image. */ - mirror_clip_sectors(s, align_sector_num, &align_nb_sectors); + align_nb_sectors = mirror_clip_sectors(s, align_sector_num, + align_nb_sectors); ret = align_sector_num + align_nb_sectors - (*sector_num + *nb_sectors); *sector_num = align_sector_num; @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) return 0; } - mirror_clip_sectors(s, sector_num, &io_sectors); + io_sectors = mirror_clip_sectors(s, sector_num, io_sectors); switch (mirror_method) { case MIRROR_METHOD_COPY: io_sectors = mirror_do_read(s, sector_num, io_sectors); -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert another internal function (no semantic change), and add mirror_clip_bytes() as a counterpart to mirror_clip_sectors(). Some of the conversion is a bit tricky, requiring temporaries to convert between units; it will be cleared up in a following patch. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/mirror.c | 63 ++++++++++++++++++++++++++++++++++------------------------ 1 file changed, 37 insertions(+), 26 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static void mirror_read_complete(void *opaque, int ret) aio_context_release(blk_get_aio_context(s->common.blk)); } +/* Clip bytes relative to offset to not exceed end-of-file */ +static inline int64_t mirror_clip_bytes(MirrorBlockJob *s, + int64_t offset, + int64_t bytes) +{ + return MIN(bytes, s->bdev_length - offset); +} + +/* Clip nb_sectors relative to sector_num to not exceed end-of-file */ static inline int mirror_clip_sectors(MirrorBlockJob *s, int64_t sector_num, int nb_sectors) @@ -XXX,XX +XXX,XX @@ static inline int mirror_clip_sectors(MirrorBlockJob *s, s->bdev_length / BDRV_SECTOR_SIZE - sector_num); } -/* Round sector_num and/or nb_sectors to target cluster if COW is needed, and - * return the offset of the adjusted tail sector against original. */ -static int mirror_cow_align(MirrorBlockJob *s, - int64_t *sector_num, - int *nb_sectors) +/* Round offset and/or bytes to target cluster if COW is needed, and + * return the offset of the adjusted tail against original. */ +static int mirror_cow_align(MirrorBlockJob *s, int64_t *offset, + unsigned int *bytes) { bool need_cow; int ret = 0; - int chunk_sectors = s->granularity >> BDRV_SECTOR_BITS; - int64_t align_sector_num = *sector_num; - int align_nb_sectors = *nb_sectors; - int max_sectors = chunk_sectors * s->max_iov; + int64_t align_offset = *offset; + unsigned int align_bytes = *bytes; + int max_bytes = s->granularity * s->max_iov; - need_cow = !test_bit(*sector_num / chunk_sectors, s->cow_bitmap); - need_cow |= !test_bit((*sector_num + *nb_sectors - 1) / chunk_sectors, + need_cow = !test_bit(*offset / s->granularity, s->cow_bitmap); + need_cow |= !test_bit((*offset + *bytes - 1) / s->granularity, s->cow_bitmap); if (need_cow) { - bdrv_round_sectors_to_clusters(blk_bs(s->target), *sector_num, - *nb_sectors, &align_sector_num, - &align_nb_sectors); + bdrv_round_to_clusters(blk_bs(s->target), *offset, *bytes, + &align_offset, &align_bytes); } - if (align_nb_sectors > max_sectors) { - align_nb_sectors = max_sectors; + if (align_bytes > max_bytes) { + align_bytes = max_bytes; if (need_cow) { - align_nb_sectors = QEMU_ALIGN_DOWN(align_nb_sectors, - s->target_cluster_size >> - BDRV_SECTOR_BITS); + align_bytes = QEMU_ALIGN_DOWN(align_bytes, s->target_cluster_size); } } - /* Clipping may result in align_nb_sectors unaligned to chunk boundary, but + /* Clipping may result in align_bytes unaligned to chunk boundary, but * that doesn't matter because it's already the end of source image. */ - align_nb_sectors = mirror_clip_sectors(s, align_sector_num, - align_nb_sectors); + align_bytes = mirror_clip_bytes(s, align_offset, align_bytes); - ret = align_sector_num + align_nb_sectors - (*sector_num + *nb_sectors); - *sector_num = align_sector_num; - *nb_sectors = align_nb_sectors; + ret = align_offset + align_bytes - (*offset + *bytes); + *offset = align_offset; + *bytes = align_bytes; assert(ret >= 0); return ret; } @@ -XXX,XX +XXX,XX @@ static int mirror_do_read(MirrorBlockJob *s, int64_t sector_num, nb_sectors = MIN(s->buf_size >> BDRV_SECTOR_BITS, nb_sectors); nb_sectors = MIN(max_sectors, nb_sectors); assert(nb_sectors); + assert(nb_sectors < BDRV_REQUEST_MAX_SECTORS); ret = nb_sectors; if (s->cow_bitmap) { - ret += mirror_cow_align(s, &sector_num, &nb_sectors); + int64_t offset = sector_num * BDRV_SECTOR_SIZE; + unsigned int bytes = nb_sectors * BDRV_SECTOR_SIZE; + int gap; + + gap = mirror_cow_align(s, &offset, &bytes); + sector_num = offset / BDRV_SECTOR_SIZE; + nb_sectors = bytes / BDRV_SECTOR_SIZE; + ret += gap / BDRV_SECTOR_SIZE; } assert(nb_sectors << BDRV_SECTOR_BITS <= s->buf_size); /* The sector range must meet granularity because: -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert another internal function, preserving all existing semantics, and adding one more assertion that things are still sector-aligned (so that conversions to sectors in mirror_read_complete don't need to round). Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/mirror.c | 74 ++++++++++++++++++++++++++-------------------------------- 1 file changed, 33 insertions(+), 41 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static inline int mirror_clip_sectors(MirrorBlockJob *s, /* Round offset and/or bytes to target cluster if COW is needed, and * return the offset of the adjusted tail against original. */ static int mirror_cow_align(MirrorBlockJob *s, int64_t *offset, - unsigned int *bytes) + uint64_t *bytes) { bool need_cow; int ret = 0; @@ -XXX,XX +XXX,XX @@ static int mirror_cow_align(MirrorBlockJob *s, int64_t *offset, unsigned int align_bytes = *bytes; int max_bytes = s->granularity * s->max_iov; + assert(*bytes < INT_MAX); need_cow = !test_bit(*offset / s->granularity, s->cow_bitmap); need_cow |= !test_bit((*offset + *bytes - 1) / s->granularity, s->cow_bitmap); @@ -XXX,XX +XXX,XX @@ static inline void mirror_wait_for_io(MirrorBlockJob *s) } /* Submit async read while handling COW. - * Returns: The number of sectors copied after and including sector_num, - * excluding any sectors copied prior to sector_num due to alignment. - * This will be nb_sectors if no alignment is necessary, or - * (new_end - sector_num) if tail is rounded up or down due to + * Returns: The number of bytes copied after and including offset, + * excluding any bytes copied prior to offset due to alignment. + * This will be @bytes if no alignment is necessary, or + * (new_end - offset) if tail is rounded up or down due to * alignment or buffer limit. */ -static int mirror_do_read(MirrorBlockJob *s, int64_t sector_num, - int nb_sectors) +static uint64_t mirror_do_read(MirrorBlockJob *s, int64_t offset, + uint64_t bytes) { BlockBackend *source = s->common.blk; - int sectors_per_chunk, nb_chunks; - int ret; + int nb_chunks; + uint64_t ret; MirrorOp *op; - int max_sectors; + uint64_t max_bytes; - sectors_per_chunk = s->granularity >> BDRV_SECTOR_BITS; - max_sectors = sectors_per_chunk * s->max_iov; + max_bytes = s->granularity * s->max_iov; /* We can only handle as much as buf_size at a time. */ - nb_sectors = MIN(s->buf_size >> BDRV_SECTOR_BITS, nb_sectors); - nb_sectors = MIN(max_sectors, nb_sectors); - assert(nb_sectors); - assert(nb_sectors < BDRV_REQUEST_MAX_SECTORS); - ret = nb_sectors; + bytes = MIN(s->buf_size, MIN(max_bytes, bytes)); + assert(bytes); + assert(bytes < BDRV_REQUEST_MAX_BYTES); + ret = bytes; if (s->cow_bitmap) { - int64_t offset = sector_num * BDRV_SECTOR_SIZE; - unsigned int bytes = nb_sectors * BDRV_SECTOR_SIZE; - int gap; - - gap = mirror_cow_align(s, &offset, &bytes); - sector_num = offset / BDRV_SECTOR_SIZE; - nb_sectors = bytes / BDRV_SECTOR_SIZE; - ret += gap / BDRV_SECTOR_SIZE; + ret += mirror_cow_align(s, &offset, &bytes); } - assert(nb_sectors << BDRV_SECTOR_BITS <= s->buf_size); - /* The sector range must meet granularity because: + assert(bytes <= s->buf_size); + /* The offset is granularity-aligned because: * 1) Caller passes in aligned values; * 2) mirror_cow_align is used only when target cluster is larger. */ - assert(!(sector_num % sectors_per_chunk)); - nb_chunks = DIV_ROUND_UP(nb_sectors, sectors_per_chunk); + assert(QEMU_IS_ALIGNED(offset, s->granularity)); + /* The range is sector-aligned, since bdrv_getlength() rounds up. */ + assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE)); + nb_chunks = DIV_ROUND_UP(bytes, s->granularity); while (s->buf_free_count < nb_chunks) { - trace_mirror_yield_in_flight(s, sector_num * BDRV_SECTOR_SIZE, - s->in_flight); + trace_mirror_yield_in_flight(s, offset, s->in_flight); mirror_wait_for_io(s); } /* Allocate a MirrorOp that is used as an AIO callback. */ op = g_new(MirrorOp, 1); op->s = s; - op->offset = sector_num * BDRV_SECTOR_SIZE; - op->bytes = nb_sectors * BDRV_SECTOR_SIZE; + op->offset = offset; + op->bytes = bytes; /* Now make a QEMUIOVector taking enough granularity-sized chunks * from s->buf_free. @@ -XXX,XX +XXX,XX @@ static int mirror_do_read(MirrorBlockJob *s, int64_t sector_num, qemu_iovec_init(&op->qiov, nb_chunks); while (nb_chunks-- > 0) { MirrorBuffer *buf = QSIMPLEQ_FIRST(&s->buf_free); - size_t remaining = nb_sectors * BDRV_SECTOR_SIZE - op->qiov.size; + size_t remaining = bytes - op->qiov.size; QSIMPLEQ_REMOVE_HEAD(&s->buf_free, next); s->buf_free_count--; @@ -XXX,XX +XXX,XX @@ static int mirror_do_read(MirrorBlockJob *s, int64_t sector_num, /* Copy the dirty cluster. */ s->in_flight++; - s->bytes_in_flight += nb_sectors * BDRV_SECTOR_SIZE; - trace_mirror_one_iteration(s, sector_num * BDRV_SECTOR_SIZE, - nb_sectors * BDRV_SECTOR_SIZE); + s->bytes_in_flight += bytes; + trace_mirror_one_iteration(s, offset, bytes); - blk_aio_preadv(source, sector_num * BDRV_SECTOR_SIZE, &op->qiov, 0, - mirror_read_complete, op); + blk_aio_preadv(source, offset, &op->qiov, 0, mirror_read_complete, op); return ret; } @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) io_sectors = mirror_clip_sectors(s, sector_num, io_sectors); switch (mirror_method) { case MIRROR_METHOD_COPY: - io_sectors = mirror_do_read(s, sector_num, io_sectors); - io_bytes_acct = io_sectors * BDRV_SECTOR_SIZE; + io_bytes_acct = mirror_do_read(s, sector_num * BDRV_SECTOR_SIZE, + io_sectors * BDRV_SECTOR_SIZE); + io_sectors = io_bytes_acct / BDRV_SECTOR_SIZE; break; case MIRROR_METHOD_ZERO: case MIRROR_METHOD_DISCARD: -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Change the internal loop iteration of mirroring to track by bytes instead of sectors (although we are still guaranteed that we iterate by steps that are both sector-aligned and multiples of the granularity). Drop the now-unused mirror_clip_sectors(). Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/mirror.c | 105 +++++++++++++++++++++++++-------------------------------- 1 file changed, 46 insertions(+), 59 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static inline int64_t mirror_clip_bytes(MirrorBlockJob *s, return MIN(bytes, s->bdev_length - offset); } -/* Clip nb_sectors relative to sector_num to not exceed end-of-file */ -static inline int mirror_clip_sectors(MirrorBlockJob *s, - int64_t sector_num, - int nb_sectors) -{ - return MIN(nb_sectors, - s->bdev_length / BDRV_SECTOR_SIZE - sector_num); -} - /* Round offset and/or bytes to target cluster if COW is needed, and * return the offset of the adjusted tail against original. */ static int mirror_cow_align(MirrorBlockJob *s, int64_t *offset, @@ -XXX,XX +XXX,XX @@ static void mirror_do_zero_or_discard(MirrorBlockJob *s, static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) { BlockDriverState *source = s->source; - int64_t sector_num, first_chunk; + int64_t offset, first_chunk; uint64_t delay_ns = 0; /* At least the first dirty chunk is mirrored in one iteration. */ int nb_chunks = 1; - int64_t end = s->bdev_length / BDRV_SECTOR_SIZE; int sectors_per_chunk = s->granularity >> BDRV_SECTOR_BITS; bool write_zeroes_ok = bdrv_can_write_zeroes_with_unmap(blk_bs(s->target)); int max_io_bytes = MAX(s->buf_size / MAX_IN_FLIGHT, MAX_IO_BYTES); bdrv_dirty_bitmap_lock(s->dirty_bitmap); - sector_num = bdrv_dirty_iter_next(s->dbi); - if (sector_num < 0) { + offset = bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; + if (offset < 0) { bdrv_set_dirty_iter(s->dbi, 0); - sector_num = bdrv_dirty_iter_next(s->dbi); + offset = bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; trace_mirror_restart_iter(s, bdrv_get_dirty_count(s->dirty_bitmap) * BDRV_SECTOR_SIZE); - assert(sector_num >= 0); + assert(offset >= 0); } bdrv_dirty_bitmap_unlock(s->dirty_bitmap); - first_chunk = sector_num / sectors_per_chunk; + first_chunk = offset / s->granularity; while (test_bit(first_chunk, s->in_flight_bitmap)) { - trace_mirror_yield_in_flight(s, sector_num * BDRV_SECTOR_SIZE, - s->in_flight); + trace_mirror_yield_in_flight(s, offset, s->in_flight); mirror_wait_for_io(s); } @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) /* Find the number of consective dirty chunks following the first dirty * one, and wait for in flight requests in them. */ bdrv_dirty_bitmap_lock(s->dirty_bitmap); - while (nb_chunks * sectors_per_chunk < (s->buf_size >> BDRV_SECTOR_BITS)) { + while (nb_chunks * s->granularity < s->buf_size) { int64_t next_dirty; - int64_t next_sector = sector_num + nb_chunks * sectors_per_chunk; - int64_t next_chunk = next_sector / sectors_per_chunk; - if (next_sector >= end || - !bdrv_get_dirty_locked(source, s->dirty_bitmap, next_sector)) { + int64_t next_offset = offset + nb_chunks * s->granularity; + int64_t next_chunk = next_offset / s->granularity; + if (next_offset >= s->bdev_length || + !bdrv_get_dirty_locked(source, s->dirty_bitmap, + next_offset >> BDRV_SECTOR_BITS)) { break; } if (test_bit(next_chunk, s->in_flight_bitmap)) { break; } - next_dirty = bdrv_dirty_iter_next(s->dbi); - if (next_dirty > next_sector || next_dirty < 0) { + next_dirty = bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; + if (next_dirty > next_offset || next_dirty < 0) { /* The bitmap iterator's cache is stale, refresh it */ - bdrv_set_dirty_iter(s->dbi, next_sector); - next_dirty = bdrv_dirty_iter_next(s->dbi); + bdrv_set_dirty_iter(s->dbi, next_offset >> BDRV_SECTOR_BITS); + next_dirty = bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; } - assert(next_dirty == next_sector); + assert(next_dirty == next_offset); nb_chunks++; } @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) * calling bdrv_get_block_status_above could yield - if some blocks are * marked dirty in this window, we need to know. */ - bdrv_reset_dirty_bitmap_locked(s->dirty_bitmap, sector_num, - nb_chunks * sectors_per_chunk); + bdrv_reset_dirty_bitmap_locked(s->dirty_bitmap, offset >> BDRV_SECTOR_BITS, + nb_chunks * sectors_per_chunk); bdrv_dirty_bitmap_unlock(s->dirty_bitmap); - bitmap_set(s->in_flight_bitmap, sector_num / sectors_per_chunk, nb_chunks); - while (nb_chunks > 0 && sector_num < end) { + bitmap_set(s->in_flight_bitmap, offset / s->granularity, nb_chunks); + while (nb_chunks > 0 && offset < s->bdev_length) { int64_t ret; int io_sectors; + unsigned int io_bytes; int64_t io_bytes_acct; BlockDriverState *file; enum MirrorMethod { @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) MIRROR_METHOD_DISCARD } mirror_method = MIRROR_METHOD_COPY; - assert(!(sector_num % sectors_per_chunk)); - ret = bdrv_get_block_status_above(source, NULL, sector_num, + assert(!(offset % s->granularity)); + ret = bdrv_get_block_status_above(source, NULL, + offset >> BDRV_SECTOR_BITS, nb_chunks * sectors_per_chunk, &io_sectors, &file); + io_bytes = io_sectors * BDRV_SECTOR_SIZE; if (ret < 0) { - io_sectors = MIN(nb_chunks * sectors_per_chunk, - max_io_bytes >> BDRV_SECTOR_BITS); + io_bytes = MIN(nb_chunks * s->granularity, max_io_bytes); } else if (ret & BDRV_BLOCK_DATA) { - io_sectors = MIN(io_sectors, max_io_bytes >> BDRV_SECTOR_BITS); + io_bytes = MIN(io_bytes, max_io_bytes); } - io_sectors -= io_sectors % sectors_per_chunk; - if (io_sectors < sectors_per_chunk) { - io_sectors = sectors_per_chunk; + io_bytes -= io_bytes % s->granularity; + if (io_bytes < s->granularity) { + io_bytes = s->granularity; } else if (ret >= 0 && !(ret & BDRV_BLOCK_DATA)) { - int64_t target_sector_num; - int target_nb_sectors; - bdrv_round_sectors_to_clusters(blk_bs(s->target), sector_num, - io_sectors, &target_sector_num, - &target_nb_sectors); - if (target_sector_num == sector_num && - target_nb_sectors == io_sectors) { + int64_t target_offset; + unsigned int target_bytes; + bdrv_round_to_clusters(blk_bs(s->target), offset, io_bytes, + &target_offset, &target_bytes); + if (target_offset == offset && + target_bytes == io_bytes) { mirror_method = ret & BDRV_BLOCK_ZERO ? MIRROR_METHOD_ZERO : MIRROR_METHOD_DISCARD; @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) } while (s->in_flight >= MAX_IN_FLIGHT) { - trace_mirror_yield_in_flight(s, sector_num * BDRV_SECTOR_SIZE, - s->in_flight); + trace_mirror_yield_in_flight(s, offset, s->in_flight); mirror_wait_for_io(s); } @@ -XXX,XX +XXX,XX @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) return 0; } - io_sectors = mirror_clip_sectors(s, sector_num, io_sectors); + io_bytes = mirror_clip_bytes(s, offset, io_bytes); switch (mirror_method) { case MIRROR_METHOD_COPY: - io_bytes_acct = mirror_do_read(s, sector_num * BDRV_SECTOR_SIZE, - io_sectors * BDRV_SECTOR_SIZE); - io_sectors = io_bytes_acct / BDRV_SECTOR_SIZE; + io_bytes = io_bytes_acct = mirror_do_read(s, offset, io_bytes); break; case MIRROR_METHOD_ZERO: case MIRROR_METHOD_DISCARD: - mirror_do_zero_or_discard(s, sector_num * BDRV_SECTOR_SIZE, - io_sectors * BDRV_SECTOR_SIZE, + mirror_do_zero_or_discard(s, offset, io_bytes, mirror_method == MIRROR_METHOD_DISCARD); if (write_zeroes_ok) { io_bytes_acct = 0; } else { - io_bytes_acct = io_sectors * BDRV_SECTOR_SIZE; + io_bytes_acct = io_bytes; } break; default: abort(); } - assert(io_sectors); - sector_num += io_sectors; - nb_chunks -= DIV_ROUND_UP(io_sectors, sectors_per_chunk); + assert(io_bytes); + offset += io_bytes; + nb_chunks -= DIV_ROUND_UP(io_bytes, s->granularity); if (s->common.speed) { delay_ns = ratelimit_calculate_delay(&s->limit, io_bytes_acct); } -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> Now that the last user [mirror_iteration()] has converted to using bytes, we no longer need a function to round sectors to clusters. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/io.c | 21 --------------------- include/block/block.h | 4 ---- 2 files changed, 25 deletions(-) diff --git a/block/io.c b/block/io.c index XXXXXXX..XXXXXXX 100644 --- a/block/io.c +++ b/block/io.c @@ -XXX,XX +XXX,XX @@ static void mark_request_serialising(BdrvTrackedRequest *req, uint64_t align) } /** - * Round a region to cluster boundaries (sector-based) - */ -void bdrv_round_sectors_to_clusters(BlockDriverState *bs, - int64_t sector_num, int nb_sectors, - int64_t *cluster_sector_num, - int *cluster_nb_sectors) -{ - BlockDriverInfo bdi; - - if (bdrv_get_info(bs, &bdi) < 0 || bdi.cluster_size == 0) { - *cluster_sector_num = sector_num; - *cluster_nb_sectors = nb_sectors; - } else { - int64_t c = bdi.cluster_size / BDRV_SECTOR_SIZE; - *cluster_sector_num = QEMU_ALIGN_DOWN(sector_num, c); - *cluster_nb_sectors = QEMU_ALIGN_UP(sector_num - *cluster_sector_num + - nb_sectors, c); - } -} - -/** * Round a region to cluster boundaries */ void bdrv_round_to_clusters(BlockDriverState *bs, diff --git a/include/block/block.h b/include/block/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -XXX,XX +XXX,XX @@ const char *bdrv_get_device_or_node_name(const BlockDriverState *bs); int bdrv_get_flags(BlockDriverState *bs); int bdrv_get_info(BlockDriverState *bs, BlockDriverInfo *bdi); ImageInfoSpecific *bdrv_get_specific_info(BlockDriverState *bs); -void bdrv_round_sectors_to_clusters(BlockDriverState *bs, - int64_t sector_num, int nb_sectors, - int64_t *cluster_sector_num, - int *cluster_nb_sectors); void bdrv_round_to_clusters(BlockDriverState *bs, int64_t offset, unsigned int bytes, int64_t *cluster_offset, -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Continue by converting an internal structure (no semantic change), and all references to tracking progress. Drop a redundant local variable bytes_per_cluster. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/backup.c | 33 +++++++++++++++------------------ 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/block/backup.c b/block/backup.c index XXXXXXX..XXXXXXX 100644 --- a/block/backup.c +++ b/block/backup.c @@ -XXX,XX +XXX,XX @@ typedef struct BackupBlockJob { BlockdevOnError on_source_error; BlockdevOnError on_target_error; CoRwlock flush_rwlock; - uint64_t sectors_read; + uint64_t bytes_read; unsigned long *done_bitmap; int64_t cluster_size; bool compress; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, void *bounce_buffer = NULL; int ret = 0; int64_t sectors_per_cluster = cluster_size_sectors(job); - int64_t bytes_per_cluster = sectors_per_cluster * BDRV_SECTOR_SIZE; - int64_t start, end; - int n; + int64_t start, end; /* clusters */ + int n; /* bytes */ qemu_co_rwlock_rdlock(&job->flush_rwlock); start = sector_num / sectors_per_cluster; end = DIV_ROUND_UP(sector_num + nb_sectors, sectors_per_cluster); - trace_backup_do_cow_enter(job, start * bytes_per_cluster, + trace_backup_do_cow_enter(job, start * job->cluster_size, sector_num * BDRV_SECTOR_SIZE, nb_sectors * BDRV_SECTOR_SIZE); @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, for (; start < end; start++) { if (test_bit(start, job->done_bitmap)) { - trace_backup_do_cow_skip(job, start * bytes_per_cluster); + trace_backup_do_cow_skip(job, start * job->cluster_size); continue; /* already copied */ } - trace_backup_do_cow_process(job, start * bytes_per_cluster); + trace_backup_do_cow_process(job, start * job->cluster_size); - n = MIN(sectors_per_cluster, - job->common.len / BDRV_SECTOR_SIZE - - start * sectors_per_cluster); + n = MIN(job->cluster_size, + job->common.len - start * job->cluster_size); if (!bounce_buffer) { bounce_buffer = blk_blockalign(blk, job->cluster_size); } iov.iov_base = bounce_buffer; - iov.iov_len = n * BDRV_SECTOR_SIZE; + iov.iov_len = n; qemu_iovec_init_external(&bounce_qiov, &iov, 1); ret = blk_co_preadv(blk, start * job->cluster_size, bounce_qiov.size, &bounce_qiov, is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0); if (ret < 0) { - trace_backup_do_cow_read_fail(job, start * bytes_per_cluster, ret); + trace_backup_do_cow_read_fail(job, start * job->cluster_size, ret); if (error_is_read) { *error_is_read = true; } @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0); } if (ret < 0) { - trace_backup_do_cow_write_fail(job, start * bytes_per_cluster, ret); + trace_backup_do_cow_write_fail(job, start * job->cluster_size, ret); if (error_is_read) { *error_is_read = false; } @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, /* Publish progress, guest I/O counts as progress too. Note that the * offset field is an opaque progress value, it is not a disk offset. */ - job->sectors_read += n; - job->common.offset += n * BDRV_SECTOR_SIZE; + job->bytes_read += n; + job->common.offset += n; } out: @@ -XXX,XX +XXX,XX @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job) */ if (job->common.speed) { uint64_t delay_ns = ratelimit_calculate_delay(&job->limit, - job->sectors_read * - BDRV_SECTOR_SIZE); - job->sectors_read = 0; + job->bytes_read); + job->bytes_read = 0; block_job_sleep_ns(&job->common, QEMU_CLOCK_REALTIME, delay_ns); } else { block_job_sleep_ns(&job->common, QEMU_CLOCK_REALTIME, 0); -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Continue by converting the public interface to backup jobs (no semantic change), including a change to CowRequest to track by bytes instead of cluster indices. Note that this does not change the difference between the public interface (starting point, and size of the subsequent range) and the internal interface (starting and end points). Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Xie Changlong <xiechanglong@cmss.chinamobile.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/backup.c | 31 +++++++++++++++---------------- block/replication.c | 12 ++++++++---- include/block/block_backup.h | 11 +++++------ 3 files changed, 28 insertions(+), 26 deletions(-) diff --git a/block/backup.c b/block/backup.c index XXXXXXX..XXXXXXX 100644 --- a/block/backup.c +++ b/block/backup.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job, do { retry = false; QLIST_FOREACH(req, &job->inflight_reqs, list) { - if (end > req->start && start < req->end) { + if (end > req->start_byte && start < req->end_byte) { qemu_co_queue_wait(&req->wait_queue, NULL); retry = true; break; @@ -XXX,XX +XXX,XX @@ static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job, /* Keep track of an in-flight request */ static void cow_request_begin(CowRequest *req, BackupBlockJob *job, - int64_t start, int64_t end) + int64_t start, int64_t end) { - req->start = start; - req->end = end; + req->start_byte = start; + req->end_byte = end; qemu_co_queue_init(&req->wait_queue); QLIST_INSERT_HEAD(&job->inflight_reqs, req, list); } @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, sector_num * BDRV_SECTOR_SIZE, nb_sectors * BDRV_SECTOR_SIZE); - wait_for_overlapping_requests(job, start, end); - cow_request_begin(&cow_request, job, start, end); + wait_for_overlapping_requests(job, start * job->cluster_size, + end * job->cluster_size); + cow_request_begin(&cow_request, job, start * job->cluster_size, + end * job->cluster_size); for (; start < end; start++) { if (test_bit(start, job->done_bitmap)) { @@ -XXX,XX +XXX,XX @@ void backup_do_checkpoint(BlockJob *job, Error **errp) bitmap_zero(backup_job->done_bitmap, len); } -void backup_wait_for_overlapping_requests(BlockJob *job, int64_t sector_num, - int nb_sectors) +void backup_wait_for_overlapping_requests(BlockJob *job, int64_t offset, + uint64_t bytes) { BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common); - int64_t sectors_per_cluster = cluster_size_sectors(backup_job); int64_t start, end; assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP); - start = sector_num / sectors_per_cluster; - end = DIV_ROUND_UP(sector_num + nb_sectors, sectors_per_cluster); + start = QEMU_ALIGN_DOWN(offset, backup_job->cluster_size); + end = QEMU_ALIGN_UP(offset + bytes, backup_job->cluster_size); wait_for_overlapping_requests(backup_job, start, end); } void backup_cow_request_begin(CowRequest *req, BlockJob *job, - int64_t sector_num, - int nb_sectors) + int64_t offset, uint64_t bytes) { BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common); - int64_t sectors_per_cluster = cluster_size_sectors(backup_job); int64_t start, end; assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP); - start = sector_num / sectors_per_cluster; - end = DIV_ROUND_UP(sector_num + nb_sectors, sectors_per_cluster); + start = QEMU_ALIGN_DOWN(offset, backup_job->cluster_size); + end = QEMU_ALIGN_UP(offset + bytes, backup_job->cluster_size); cow_request_begin(req, backup_job, start, end); } diff --git a/block/replication.c b/block/replication.c index XXXXXXX..XXXXXXX 100644 --- a/block/replication.c +++ b/block/replication.c @@ -XXX,XX +XXX,XX @@ static coroutine_fn int replication_co_readv(BlockDriverState *bs, } if (job) { - backup_wait_for_overlapping_requests(child->bs->job, sector_num, - remaining_sectors); - backup_cow_request_begin(&req, child->bs->job, sector_num, - remaining_sectors); + uint64_t remaining_bytes = remaining_sectors * BDRV_SECTOR_SIZE; + + backup_wait_for_overlapping_requests(child->bs->job, + sector_num * BDRV_SECTOR_SIZE, + remaining_bytes); + backup_cow_request_begin(&req, child->bs->job, + sector_num * BDRV_SECTOR_SIZE, + remaining_bytes); ret = bdrv_co_readv(bs->file, sector_num, remaining_sectors, qiov); backup_cow_request_end(&req); diff --git a/include/block/block_backup.h b/include/block/block_backup.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_backup.h +++ b/include/block/block_backup.h @@ -XXX,XX +XXX,XX @@ #include "block/block_int.h" typedef struct CowRequest { - int64_t start; - int64_t end; + int64_t start_byte; + int64_t end_byte; QLIST_ENTRY(CowRequest) list; CoQueue wait_queue; /* coroutines blocked on this request */ } CowRequest; -void backup_wait_for_overlapping_requests(BlockJob *job, int64_t sector_num, - int nb_sectors); +void backup_wait_for_overlapping_requests(BlockJob *job, int64_t offset, + uint64_t bytes); void backup_cow_request_begin(CowRequest *req, BlockJob *job, - int64_t sector_num, - int nb_sectors); + int64_t offset, uint64_t bytes); void backup_cow_request_end(CowRequest *req); void backup_do_checkpoint(BlockJob *job, Error **errp); -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert another internal function (no semantic change). Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/backup.c | 62 ++++++++++++++++++++++++---------------------------------- 1 file changed, 26 insertions(+), 36 deletions(-) diff --git a/block/backup.c b/block/backup.c index XXXXXXX..XXXXXXX 100644 --- a/block/backup.c +++ b/block/backup.c @@ -XXX,XX +XXX,XX @@ static void cow_request_end(CowRequest *req) } static int coroutine_fn backup_do_cow(BackupBlockJob *job, - int64_t sector_num, int nb_sectors, + int64_t offset, uint64_t bytes, bool *error_is_read, bool is_write_notifier) { @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, QEMUIOVector bounce_qiov; void *bounce_buffer = NULL; int ret = 0; - int64_t sectors_per_cluster = cluster_size_sectors(job); - int64_t start, end; /* clusters */ + int64_t start, end; /* bytes */ int n; /* bytes */ qemu_co_rwlock_rdlock(&job->flush_rwlock); - start = sector_num / sectors_per_cluster; - end = DIV_ROUND_UP(sector_num + nb_sectors, sectors_per_cluster); + start = QEMU_ALIGN_DOWN(offset, job->cluster_size); + end = QEMU_ALIGN_UP(bytes + offset, job->cluster_size); - trace_backup_do_cow_enter(job, start * job->cluster_size, - sector_num * BDRV_SECTOR_SIZE, - nb_sectors * BDRV_SECTOR_SIZE); + trace_backup_do_cow_enter(job, start, offset, bytes); - wait_for_overlapping_requests(job, start * job->cluster_size, - end * job->cluster_size); - cow_request_begin(&cow_request, job, start * job->cluster_size, - end * job->cluster_size); + wait_for_overlapping_requests(job, start, end); + cow_request_begin(&cow_request, job, start, end); - for (; start < end; start++) { - if (test_bit(start, job->done_bitmap)) { - trace_backup_do_cow_skip(job, start * job->cluster_size); + for (; start < end; start += job->cluster_size) { + if (test_bit(start / job->cluster_size, job->done_bitmap)) { + trace_backup_do_cow_skip(job, start); continue; /* already copied */ } - trace_backup_do_cow_process(job, start * job->cluster_size); + trace_backup_do_cow_process(job, start); - n = MIN(job->cluster_size, - job->common.len - start * job->cluster_size); + n = MIN(job->cluster_size, job->common.len - start); if (!bounce_buffer) { bounce_buffer = blk_blockalign(blk, job->cluster_size); @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, iov.iov_len = n; qemu_iovec_init_external(&bounce_qiov, &iov, 1); - ret = blk_co_preadv(blk, start * job->cluster_size, - bounce_qiov.size, &bounce_qiov, + ret = blk_co_preadv(blk, start, bounce_qiov.size, &bounce_qiov, is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0); if (ret < 0) { - trace_backup_do_cow_read_fail(job, start * job->cluster_size, ret); + trace_backup_do_cow_read_fail(job, start, ret); if (error_is_read) { *error_is_read = true; } @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, } if (buffer_is_zero(iov.iov_base, iov.iov_len)) { - ret = blk_co_pwrite_zeroes(job->target, start * job->cluster_size, + ret = blk_co_pwrite_zeroes(job->target, start, bounce_qiov.size, BDRV_REQ_MAY_UNMAP); } else { - ret = blk_co_pwritev(job->target, start * job->cluster_size, + ret = blk_co_pwritev(job->target, start, bounce_qiov.size, &bounce_qiov, job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0); } if (ret < 0) { - trace_backup_do_cow_write_fail(job, start * job->cluster_size, ret); + trace_backup_do_cow_write_fail(job, start, ret); if (error_is_read) { *error_is_read = false; } goto out; } - set_bit(start, job->done_bitmap); + set_bit(start / job->cluster_size, job->done_bitmap); /* Publish progress, guest I/O counts as progress too. Note that the * offset field is an opaque progress value, it is not a disk offset. @@ -XXX,XX +XXX,XX @@ out: cow_request_end(&cow_request); - trace_backup_do_cow_return(job, sector_num * BDRV_SECTOR_SIZE, - nb_sectors * BDRV_SECTOR_SIZE, ret); + trace_backup_do_cow_return(job, offset, bytes, ret); qemu_co_rwlock_unlock(&job->flush_rwlock); @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_before_write_notify( { BackupBlockJob *job = container_of(notifier, BackupBlockJob, before_write); BdrvTrackedRequest *req = opaque; - int64_t sector_num = req->offset >> BDRV_SECTOR_BITS; - int nb_sectors = req->bytes >> BDRV_SECTOR_BITS; assert(req->bs == blk_bs(job->common.blk)); - assert((req->offset & (BDRV_SECTOR_SIZE - 1)) == 0); - assert((req->bytes & (BDRV_SECTOR_SIZE - 1)) == 0); + assert(QEMU_IS_ALIGNED(req->offset, BDRV_SECTOR_SIZE)); + assert(QEMU_IS_ALIGNED(req->bytes, BDRV_SECTOR_SIZE)); - return backup_do_cow(job, sector_num, nb_sectors, NULL, true); + return backup_do_cow(job, req->offset, req->bytes, NULL, true); } static void backup_set_speed(BlockJob *job, int64_t speed, Error **errp) @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job) if (yield_and_check(job)) { goto out; } - ret = backup_do_cow(job, cluster * sectors_per_cluster, - sectors_per_cluster, &error_is_read, + ret = backup_do_cow(job, cluster * job->cluster_size, + job->cluster_size, &error_is_read, false); if ((ret < 0) && backup_error_action(job, error_is_read, -ret) == @@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque) if (alloced < 0) { ret = alloced; } else { - ret = backup_do_cow(job, start * sectors_per_cluster, - sectors_per_cluster, &error_is_read, + ret = backup_do_cow(job, start * job->cluster_size, + job->cluster_size, &error_is_read, false); } if (ret < 0) { -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Change the internal loop iteration of backups to track by bytes instead of sectors (although we are still guaranteed that we iterate by steps that are cluster-aligned). Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/backup.c | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/block/backup.c b/block/backup.c index XXXXXXX..XXXXXXX 100644 --- a/block/backup.c +++ b/block/backup.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job) int ret = 0; int clusters_per_iter; uint32_t granularity; - int64_t sector; + int64_t offset; int64_t cluster; int64_t end; int64_t last_cluster = -1; - int64_t sectors_per_cluster = cluster_size_sectors(job); BdrvDirtyBitmapIter *dbi; granularity = bdrv_dirty_bitmap_granularity(job->sync_bitmap); @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job) dbi = bdrv_dirty_iter_new(job->sync_bitmap, 0); /* Find the next dirty sector(s) */ - while ((sector = bdrv_dirty_iter_next(dbi)) != -1) { - cluster = sector / sectors_per_cluster; + while ((offset = bdrv_dirty_iter_next(dbi) * BDRV_SECTOR_SIZE) >= 0) { + cluster = offset / job->cluster_size; /* Fake progress updates for any clusters we skipped */ if (cluster != last_cluster + 1) { @@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job) /* If the bitmap granularity is smaller than the backup granularity, * we need to advance the iterator pointer to the next cluster. */ if (granularity < job->cluster_size) { - bdrv_set_dirty_iter(dbi, cluster * sectors_per_cluster); + bdrv_set_dirty_iter(dbi, + cluster * job->cluster_size / BDRV_SECTOR_SIZE); } last_cluster = cluster - 1; @@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque) BackupBlockJob *job = opaque; BackupCompleteData *data; BlockDriverState *bs = blk_bs(job->common.blk); - int64_t start, end; + int64_t offset; int64_t sectors_per_cluster = cluster_size_sectors(job); int ret = 0; QLIST_INIT(&job->inflight_reqs); qemu_co_rwlock_init(&job->flush_rwlock); - start = 0; - end = DIV_ROUND_UP(job->common.len, job->cluster_size); - - job->done_bitmap = bitmap_new(end); + job->done_bitmap = bitmap_new(DIV_ROUND_UP(job->common.len, + job->cluster_size)); job->before_write.notify = backup_before_write_notify; bdrv_add_before_write_notifier(bs, &job->before_write); @@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque) ret = backup_run_incremental(job); } else { /* Both FULL and TOP SYNC_MODE's require copying.. */ - for (; start < end; start++) { + for (offset = 0; offset < job->common.len; + offset += job->cluster_size) { bool error_is_read; int alloced = 0; @@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque) * needed but at some point that is always the case. */ alloced = bdrv_is_allocated(bs, - start * sectors_per_cluster + i, - sectors_per_cluster - i, &n); + (offset >> BDRV_SECTOR_BITS) + i, + sectors_per_cluster - i, &n); i += n; if (alloced || n == 0) { @@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque) if (alloced < 0) { ret = alloced; } else { - ret = backup_do_cow(job, start * job->cluster_size, - job->cluster_size, &error_is_read, - false); + ret = backup_do_cow(job, offset, job->cluster_size, + &error_is_read, false); } if (ret < 0) { /* Depending on error action, fail now or retry cluster */ @@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque) if (action == BLOCK_ERROR_ACTION_REPORT) { break; } else { - start--; + offset -= job->cluster_size; continue; } } -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually moving away from sector-based interfaces, towards byte-based. In the common case, allocation is unlikely to ever use values that are not naturally sector-aligned, but it is possible that byte-based values will let us be more precise about allocation at the end of an unaligned file that can do byte-based access. Changing the signature of the function to use int64_t *pnum ensures that the compiler enforces that all callers are updated. For now, the io.c layer still assert()s that all callers are sector-aligned on input and that *pnum is sector-aligned on return to the caller, but that can be relaxed when a later patch implements byte-based block status. Therefore, this code adds usages like DIV_ROUND_UP(,BDRV_SECTOR_SIZE) to callers that still want aligned values, where the call might reasonbly give non-aligned results in the future; on the other hand, no rounding is needed for callers that should just continue to work with byte alignment. For the most part this patch is just the addition of scaling at the callers followed by inverse scaling at bdrv_is_allocated(). But some code, particularly bdrv_commit(), gets a lot simpler because it no longer has to mess with sectors; also, it is now possible to pass NULL if the caller does not care how much of the image is allocated beyond the initial offset. Leave comments where we can further simplify once a later patch eliminates the need for sector-aligned requests through bdrv_is_allocated(). For ease of review, bdrv_is_allocated_above() will be tackled separately. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/backup.c | 17 ++++--------- block/commit.c | 21 +++++++--------- block/io.c | 54 ++++++++++++++++++++++++++------------- block/stream.c | 7 ++++-- block/vvfat.c | 34 ++++++++++++++----------- include/block/block.h | 4 +-- migration/block.c | 16 +++++++----- qemu-img.c | 8 +++++- qemu-io-cmds.c | 70 +++++++++++++++++++++++---------------------------- 9 files changed, 126 insertions(+), 105 deletions(-) diff --git a/block/backup.c b/block/backup.c index XXXXXXX..XXXXXXX 100644 --- a/block/backup.c +++ b/block/backup.c @@ -XXX,XX +XXX,XX @@ typedef struct BackupBlockJob { QLIST_HEAD(, CowRequest) inflight_reqs; } BackupBlockJob; -/* Size of a cluster in sectors, instead of bytes. */ -static inline int64_t cluster_size_sectors(BackupBlockJob *job) -{ - return job->cluster_size / BDRV_SECTOR_SIZE; -} - /* See if in-flight requests overlap and wait for them to complete */ static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job, int64_t start, @@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque) BackupCompleteData *data; BlockDriverState *bs = blk_bs(job->common.blk); int64_t offset; - int64_t sectors_per_cluster = cluster_size_sectors(job); int ret = 0; QLIST_INIT(&job->inflight_reqs); @@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque) } if (job->sync_mode == MIRROR_SYNC_MODE_TOP) { - int i, n; + int i; + int64_t n; /* Check to see if these blocks are already in the * backing file. */ - for (i = 0; i < sectors_per_cluster;) { + for (i = 0; i < job->cluster_size;) { /* bdrv_is_allocated() only returns true/false based * on the first set of sectors it comes across that * are are all in the same state. @@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque) * backup cluster length. We end up copying more than * needed but at some point that is always the case. */ alloced = - bdrv_is_allocated(bs, - (offset >> BDRV_SECTOR_BITS) + i, - sectors_per_cluster - i, &n); + bdrv_is_allocated(bs, offset + i, + job->cluster_size - i, &n); i += n; if (alloced || n == 0) { diff --git a/block/commit.c b/block/commit.c index XXXXXXX..XXXXXXX 100644 --- a/block/commit.c +++ b/block/commit.c @@ -XXX,XX +XXX,XX @@ fail: } -#define COMMIT_BUF_SECTORS 2048 +#define COMMIT_BUF_SIZE (2048 * BDRV_SECTOR_SIZE) /* commit COW file into the raw image */ int bdrv_commit(BlockDriverState *bs) @@ -XXX,XX +XXX,XX @@ int bdrv_commit(BlockDriverState *bs) BlockDriverState *backing_file_bs = NULL; BlockDriverState *commit_top_bs = NULL; BlockDriver *drv = bs->drv; - int64_t sector, total_sectors, length, backing_length; - int n, ro, open_flags; + int64_t offset, length, backing_length; + int ro, open_flags; + int64_t n; int ret = 0; uint8_t *buf = NULL; Error *local_err = NULL; @@ -XXX,XX +XXX,XX @@ int bdrv_commit(BlockDriverState *bs) } } - total_sectors = length >> BDRV_SECTOR_BITS; - /* blk_try_blockalign() for src will choose an alignment that works for * backing as well, so no need to compare the alignment manually. */ - buf = blk_try_blockalign(src, COMMIT_BUF_SECTORS * BDRV_SECTOR_SIZE); + buf = blk_try_blockalign(src, COMMIT_BUF_SIZE); if (buf == NULL) { ret = -ENOMEM; goto ro_cleanup; } - for (sector = 0; sector < total_sectors; sector += n) { - ret = bdrv_is_allocated(bs, sector, COMMIT_BUF_SECTORS, &n); + for (offset = 0; offset < length; offset += n) { + ret = bdrv_is_allocated(bs, offset, COMMIT_BUF_SIZE, &n); if (ret < 0) { goto ro_cleanup; } if (ret) { - ret = blk_pread(src, sector * BDRV_SECTOR_SIZE, buf, - n * BDRV_SECTOR_SIZE); + ret = blk_pread(src, offset, buf, n); if (ret < 0) { goto ro_cleanup; } - ret = blk_pwrite(backing, sector * BDRV_SECTOR_SIZE, buf, - n * BDRV_SECTOR_SIZE, 0); + ret = blk_pwrite(backing, offset, buf, n, 0); if (ret < 0) { goto ro_cleanup; } diff --git a/block/io.c b/block/io.c index XXXXXXX..XXXXXXX 100644 --- a/block/io.c +++ b/block/io.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child, } if (flags & BDRV_REQ_COPY_ON_READ) { - int64_t start_sector = offset >> BDRV_SECTOR_BITS; - int64_t end_sector = DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE); - unsigned int nb_sectors = end_sector - start_sector; - int pnum; + /* TODO: Simplify further once bdrv_is_allocated no longer + * requires sector alignment */ + int64_t start = QEMU_ALIGN_DOWN(offset, BDRV_SECTOR_SIZE); + int64_t end = QEMU_ALIGN_UP(offset + bytes, BDRV_SECTOR_SIZE); + int64_t pnum; - ret = bdrv_is_allocated(bs, start_sector, nb_sectors, &pnum); + ret = bdrv_is_allocated(bs, start, end - start, &pnum); if (ret < 0) { goto out; } - if (!ret || pnum != nb_sectors) { + if (!ret || pnum != end - start) { ret = bdrv_co_do_copy_on_readv(child, offset, bytes, qiov); goto out; } @@ -XXX,XX +XXX,XX @@ int64_t bdrv_get_block_status(BlockDriverState *bs, sector_num, nb_sectors, pnum, file); } -int coroutine_fn bdrv_is_allocated(BlockDriverState *bs, int64_t sector_num, - int nb_sectors, int *pnum) +int coroutine_fn bdrv_is_allocated(BlockDriverState *bs, int64_t offset, + int64_t bytes, int64_t *pnum) { BlockDriverState *file; - int64_t ret = bdrv_get_block_status(bs, sector_num, nb_sectors, pnum, - &file); + int64_t sector_num = offset >> BDRV_SECTOR_BITS; + int nb_sectors = bytes >> BDRV_SECTOR_BITS; + int64_t ret; + int psectors; + + assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE)); + assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE) && bytes < INT_MAX); + ret = bdrv_get_block_status(bs, sector_num, nb_sectors, &psectors, + &file); if (ret < 0) { return ret; } + if (pnum) { + *pnum = psectors * BDRV_SECTOR_SIZE; + } return !!(ret & BDRV_BLOCK_ALLOCATED); } @@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_is_allocated(BlockDriverState *bs, int64_t sector_num, * * Return true if the given sector is allocated in any image between * BASE and TOP (inclusive). BASE can be NULL to check if the given - * sector is allocated in any image of the chain. Return false otherwise. + * sector is allocated in any image of the chain. Return false otherwise, + * or negative errno on failure. * * 'pnum' is set to the number of sectors (including and immediately following * the specified sector) that are known to be in the same @@ -XXX,XX +XXX,XX @@ int bdrv_is_allocated_above(BlockDriverState *top, intermediate = top; while (intermediate && intermediate != base) { - int pnum_inter; - ret = bdrv_is_allocated(intermediate, sector_num, nb_sectors, + int64_t pnum_inter; + int psectors_inter; + + ret = bdrv_is_allocated(intermediate, sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE, &pnum_inter); if (ret < 0) { return ret; - } else if (ret) { - *pnum = pnum_inter; + } + assert(pnum_inter < INT_MAX * BDRV_SECTOR_SIZE); + psectors_inter = pnum_inter >> BDRV_SECTOR_BITS; + if (ret) { + *pnum = psectors_inter; return 1; } @@ -XXX,XX +XXX,XX @@ int bdrv_is_allocated_above(BlockDriverState *top, * * [sector_num+x, nr_sectors] allocated. */ - if (n > pnum_inter && + if (n > psectors_inter && (intermediate == top || - sector_num + pnum_inter < intermediate->total_sectors)) { - n = pnum_inter; + sector_num + psectors_inter < intermediate->total_sectors)) { + n = psectors_inter; } intermediate = backing_bs(intermediate); diff --git a/block/stream.c b/block/stream.c index XXXXXXX..XXXXXXX 100644 --- a/block/stream.c +++ b/block/stream.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) for ( ; offset < s->common.len; offset += n * BDRV_SECTOR_SIZE) { bool copy; + int64_t count = 0; /* Note that even when no rate limit is applied we need to yield * with no pending I/O here so that bdrv_drain_all() returns. @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) copy = false; - ret = bdrv_is_allocated(bs, offset / BDRV_SECTOR_SIZE, - STREAM_BUFFER_SIZE / BDRV_SECTOR_SIZE, &n); + ret = bdrv_is_allocated(bs, offset, STREAM_BUFFER_SIZE, &count); + /* TODO relax this once bdrv_is_allocated does not enforce sectors */ + assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); + n = count >> BDRV_SECTOR_BITS; if (ret == 1) { /* Allocated in the top, no need to copy. */ } else if (ret >= 0) { diff --git a/block/vvfat.c b/block/vvfat.c index XXXXXXX..XXXXXXX 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -XXX,XX +XXX,XX @@ static int vvfat_read(BlockDriverState *bs, int64_t sector_num, if (sector_num >= bs->total_sectors) return -1; if (s->qcow) { - int n; + int64_t n; int ret; - ret = bdrv_is_allocated(s->qcow->bs, sector_num, - nb_sectors - i, &n); + ret = bdrv_is_allocated(s->qcow->bs, sector_num * BDRV_SECTOR_SIZE, + (nb_sectors - i) * BDRV_SECTOR_SIZE, &n); if (ret < 0) { return ret; } if (ret) { - DLOG(fprintf(stderr, "sectors %d+%d allocated\n", - (int)sector_num, n)); - if (bdrv_read(s->qcow, sector_num, buf + i * 0x200, n)) { + DLOG(fprintf(stderr, "sectors %" PRId64 "+%" PRId64 + " allocated\n", sector_num, + n >> BDRV_SECTOR_BITS)); + if (bdrv_read(s->qcow, sector_num, buf + i * 0x200, + n >> BDRV_SECTOR_BITS)) { return -1; } - i += n - 1; - sector_num += n - 1; + i += (n >> BDRV_SECTOR_BITS) - 1; + sector_num += (n >> BDRV_SECTOR_BITS) - 1; continue; } -DLOG(fprintf(stderr, "sector %d not allocated\n", (int)sector_num)); + DLOG(fprintf(stderr, "sector %" PRId64 " not allocated\n", + sector_num)); } if (sector_num < s->offset_to_root_dir) { if (sector_num < s->offset_to_fat) { @@ -XXX,XX +XXX,XX @@ static inline bool cluster_was_modified(BDRVVVFATState *s, uint32_t cluster_num) { int was_modified = 0; - int i, dummy; + int i; if (s->qcow == NULL) { return 0; @@ -XXX,XX +XXX,XX @@ static inline bool cluster_was_modified(BDRVVVFATState *s, for (i = 0; !was_modified && i < s->sectors_per_cluster; i++) { was_modified = bdrv_is_allocated(s->qcow->bs, - cluster2sector(s, cluster_num) + i, - 1, &dummy); + (cluster2sector(s, cluster_num) + + i) * BDRV_SECTOR_SIZE, + BDRV_SECTOR_SIZE, NULL); } /* @@ -XXX,XX +XXX,XX @@ static uint32_t get_cluster_count_for_direntry(BDRVVVFATState* s, } if (copy_it) { - int i, dummy; + int i; /* * This is horribly inefficient, but that is okay, since * it is rarely executed, if at all. @@ -XXX,XX +XXX,XX @@ static uint32_t get_cluster_count_for_direntry(BDRVVVFATState* s, for (i = 0; i < s->sectors_per_cluster; i++) { int res; - res = bdrv_is_allocated(s->qcow->bs, offset + i, 1, &dummy); + res = bdrv_is_allocated(s->qcow->bs, + (offset + i) * BDRV_SECTOR_SIZE, + BDRV_SECTOR_SIZE, NULL); if (res < 0) { return -1; } diff --git a/include/block/block.h b/include/block/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -XXX,XX +XXX,XX @@ int64_t bdrv_get_block_status_above(BlockDriverState *bs, int64_t sector_num, int nb_sectors, int *pnum, BlockDriverState **file); -int bdrv_is_allocated(BlockDriverState *bs, int64_t sector_num, int nb_sectors, - int *pnum); +int bdrv_is_allocated(BlockDriverState *bs, int64_t offset, int64_t bytes, + int64_t *pnum); int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, int64_t sector_num, int nb_sectors, int *pnum); diff --git a/migration/block.c b/migration/block.c index XXXXXXX..XXXXXXX 100644 --- a/migration/block.c +++ b/migration/block.c @@ -XXX,XX +XXX,XX @@ #define BLK_MIG_FLAG_PROGRESS 0x04 #define BLK_MIG_FLAG_ZERO_BLOCK 0x08 -#define MAX_IS_ALLOCATED_SEARCH 65536 +#define MAX_IS_ALLOCATED_SEARCH (65536 * BDRV_SECTOR_SIZE) #define MAX_INFLIGHT_IO 512 @@ -XXX,XX +XXX,XX @@ static int mig_save_device_bulk(QEMUFile *f, BlkMigDevState *bmds) BlockBackend *bb = bmds->blk; BlkMigBlock *blk; int nr_sectors; + int64_t count; if (bmds->shared_base) { qemu_mutex_lock_iothread(); aio_context_acquire(blk_get_aio_context(bb)); - /* Skip unallocated sectors; intentionally treats failure as - * an allocated sector */ + /* Skip unallocated sectors; intentionally treats failure or + * partial sector as an allocated sector */ while (cur_sector < total_sectors && - !bdrv_is_allocated(blk_bs(bb), cur_sector, - MAX_IS_ALLOCATED_SEARCH, &nr_sectors)) { - cur_sector += nr_sectors; + !bdrv_is_allocated(blk_bs(bb), cur_sector * BDRV_SECTOR_SIZE, + MAX_IS_ALLOCATED_SEARCH, &count)) { + if (count < BDRV_SECTOR_SIZE) { + break; + } + cur_sector += count >> BDRV_SECTOR_BITS; } aio_context_release(blk_get_aio_context(bb)); qemu_mutex_unlock_iothread(); diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static int img_rebase(int argc, char **argv) int64_t new_backing_num_sectors = 0; uint64_t sector; int n; + int64_t count; float local_progress = 0; buf_old = blk_blockalign(blk, IO_BUF_SIZE); @@ -XXX,XX +XXX,XX @@ static int img_rebase(int argc, char **argv) } /* If the cluster is allocated, we don't need to take action */ - ret = bdrv_is_allocated(bs, sector, n, &n); + ret = bdrv_is_allocated(bs, sector << BDRV_SECTOR_BITS, + n << BDRV_SECTOR_BITS, &count); if (ret < 0) { error_report("error while reading image metadata: %s", strerror(-ret)); goto out; } + /* TODO relax this once bdrv_is_allocated does not enforce + * sector alignment */ + assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); + n = count >> BDRV_SECTOR_BITS; if (ret) { continue; } diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-io-cmds.c +++ b/qemu-io-cmds.c @@ -XXX,XX +XXX,XX @@ out: static int alloc_f(BlockBackend *blk, int argc, char **argv) { BlockDriverState *bs = blk_bs(blk); - int64_t offset, sector_num, nb_sectors, remaining, count; + int64_t offset, start, remaining, count; char s1[64]; - int num, ret; - int64_t sum_alloc; + int ret; + int64_t num, sum_alloc; - offset = cvtnum(argv[1]); + start = offset = cvtnum(argv[1]); if (offset < 0) { print_cvtnum_err(offset, argv[1]); return 0; @@ -XXX,XX +XXX,XX @@ static int alloc_f(BlockBackend *blk, int argc, char **argv) count); return 0; } - nb_sectors = count >> BDRV_SECTOR_BITS; - remaining = nb_sectors; + remaining = count; sum_alloc = 0; - sector_num = offset >> 9; while (remaining) { - ret = bdrv_is_allocated(bs, sector_num, remaining, &num); + ret = bdrv_is_allocated(bs, offset, remaining, &num); if (ret < 0) { printf("is_allocated failed: %s\n", strerror(-ret)); return 0; } - sector_num += num; + offset += num; remaining -= num; if (ret) { sum_alloc += num; } if (num == 0) { - nb_sectors -= remaining; + count -= remaining; remaining = 0; } } - cvtstr(offset, s1, sizeof(s1)); + cvtstr(start, s1, sizeof(s1)); printf("%"PRId64"/%"PRId64" bytes allocated at offset %s\n", - sum_alloc << BDRV_SECTOR_BITS, nb_sectors << BDRV_SECTOR_BITS, s1); + sum_alloc, count, s1); return 0; } @@ -XXX,XX +XXX,XX @@ static const cmdinfo_t alloc_cmd = { }; -static int map_is_allocated(BlockDriverState *bs, int64_t sector_num, - int64_t nb_sectors, int64_t *pnum) +static int map_is_allocated(BlockDriverState *bs, int64_t offset, + int64_t bytes, int64_t *pnum) { - int num, num_checked; + int64_t num; + int num_checked; int ret, firstret; - num_checked = MIN(nb_sectors, INT_MAX); - ret = bdrv_is_allocated(bs, sector_num, num_checked, &num); + num_checked = MIN(bytes, BDRV_REQUEST_MAX_BYTES); + ret = bdrv_is_allocated(bs, offset, num_checked, &num); if (ret < 0) { return ret; } @@ -XXX,XX +XXX,XX @@ static int map_is_allocated(BlockDriverState *bs, int64_t sector_num, firstret = ret; *pnum = num; - while (nb_sectors > 0 && ret == firstret) { - sector_num += num; - nb_sectors -= num; + while (bytes > 0 && ret == firstret) { + offset += num; + bytes -= num; - num_checked = MIN(nb_sectors, INT_MAX); - ret = bdrv_is_allocated(bs, sector_num, num_checked, &num); + num_checked = MIN(bytes, BDRV_REQUEST_MAX_BYTES); + ret = bdrv_is_allocated(bs, offset, num_checked, &num); if (ret == firstret && num) { *pnum += num; } else { @@ -XXX,XX +XXX,XX @@ static int map_is_allocated(BlockDriverState *bs, int64_t sector_num, static int map_f(BlockBackend *blk, int argc, char **argv) { - int64_t offset; - int64_t nb_sectors, total_sectors; + int64_t offset, bytes; char s1[64], s2[64]; int64_t num; int ret; const char *retstr; offset = 0; - total_sectors = blk_nb_sectors(blk); - if (total_sectors < 0) { - error_report("Failed to query image length: %s", - strerror(-total_sectors)); + bytes = blk_getlength(blk); + if (bytes < 0) { + error_report("Failed to query image length: %s", strerror(-bytes)); return 0; } - nb_sectors = total_sectors; - - do { - ret = map_is_allocated(blk_bs(blk), offset, nb_sectors, &num); + while (bytes) { + ret = map_is_allocated(blk_bs(blk), offset, bytes, &num); if (ret < 0) { error_report("Failed to get allocation status: %s", strerror(-ret)); return 0; @@ -XXX,XX +XXX,XX @@ static int map_f(BlockBackend *blk, int argc, char **argv) } retstr = ret ? " allocated" : "not allocated"; - cvtstr(num << BDRV_SECTOR_BITS, s1, sizeof(s1)); - cvtstr(offset << BDRV_SECTOR_BITS, s2, sizeof(s2)); + cvtstr(num, s1, sizeof(s1)); + cvtstr(offset, s2, sizeof(s2)); printf("%s (0x%" PRIx64 ") bytes %s at offset %s (0x%" PRIx64 ")\n", - s1, num << BDRV_SECTOR_BITS, retstr, - s2, offset << BDRV_SECTOR_BITS); + s1, num, retstr, s2, offset); offset += num; - nb_sectors -= num; - } while (offset < total_sectors); + bytes -= num; + } return 0; } -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> bdrv_is_allocated_above() was relying on intermediate->total_sectors, which is a field that can have stale contents depending on the value of intermediate->has_variable_length. An audit shows that we are safe (we were first calling through bdrv_co_get_block_status() which in turn calls bdrv_nb_sectors() and therefore just refreshed the current length), but it's nicer to favor our accessor functions to avoid having to repeat such an audit, even if it means refresh_total_sectors() is called more frequently. Suggested-by: John Snow <jsnow@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Manos Pitsidianakis <el13635@mail.ntua.gr> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/io.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/block/io.c b/block/io.c index XXXXXXX..XXXXXXX 100644 --- a/block/io.c +++ b/block/io.c @@ -XXX,XX +XXX,XX @@ int bdrv_is_allocated_above(BlockDriverState *top, intermediate = top; while (intermediate && intermediate != base) { int64_t pnum_inter; + int64_t size_inter; int psectors_inter; ret = bdrv_is_allocated(intermediate, sector_num * BDRV_SECTOR_SIZE, @@ -XXX,XX +XXX,XX @@ int bdrv_is_allocated_above(BlockDriverState *top, return 1; } - /* - * [sector_num, nb_sectors] is unallocated on top but intermediate - * might have - * - * [sector_num+x, nr_sectors] allocated. - */ + size_inter = bdrv_nb_sectors(intermediate); + if (size_inter < 0) { + return size_inter; + } if (n > psectors_inter && - (intermediate == top || - sector_num + psectors_inter < intermediate->total_sectors)) { + (intermediate == top || sector_num + psectors_inter < size_inter)) { n = psectors_inter; } -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> We are gradually moving away from sector-based interfaces, towards byte-based. In the common case, allocation is unlikely to ever use values that are not naturally sector-aligned, but it is possible that byte-based values will let us be more precise about allocation at the end of an unaligned file that can do byte-based access. Changing the signature of the function to use int64_t *pnum ensures that the compiler enforces that all callers are updated. For now, the io.c layer still assert()s that all callers are sector-aligned, but that can be relaxed when a later patch implements byte-based block status. Therefore, for the most part this patch is just the addition of scaling at the callers followed by inverse scaling at bdrv_is_allocated(). But some code, particularly stream_run(), gets a lot simpler because it no longer has to mess with sectors. Leave comments where we can further simplify by switching to byte-based iterations, once later patches eliminate the need for sector-aligned operations. For ease of review, bdrv_is_allocated() was tackled separately. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/commit.c | 20 ++++++++------------ block/io.c | 38 ++++++++++++++++++-------------------- block/mirror.c | 8 +++++++- block/replication.c | 17 ++++++++++++----- block/stream.c | 23 +++++++++-------------- include/block/block.h | 2 +- qemu-img.c | 13 ++++++++++--- 7 files changed, 65 insertions(+), 56 deletions(-) diff --git a/block/commit.c b/block/commit.c index XXXXXXX..XXXXXXX 100644 --- a/block/commit.c +++ b/block/commit.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) int64_t offset; uint64_t delay_ns = 0; int ret = 0; - int n = 0; /* sectors */ + int64_t n = 0; /* bytes */ void *buf = NULL; int bytes_written = 0; int64_t base_len; @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) buf = blk_blockalign(s->top, COMMIT_BUFFER_SIZE); - for (offset = 0; offset < s->common.len; offset += n * BDRV_SECTOR_SIZE) { + for (offset = 0; offset < s->common.len; offset += n) { bool copy; /* Note that even when no rate limit is applied we need to yield @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) } /* Copy if allocated above the base */ ret = bdrv_is_allocated_above(blk_bs(s->top), blk_bs(s->base), - offset / BDRV_SECTOR_SIZE, - COMMIT_BUFFER_SIZE / BDRV_SECTOR_SIZE, - &n); + offset, COMMIT_BUFFER_SIZE, &n); copy = (ret == 1); - trace_commit_one_iteration(s, offset, n * BDRV_SECTOR_SIZE, ret); + trace_commit_one_iteration(s, offset, n, ret); if (copy) { - ret = commit_populate(s->top, s->base, offset, - n * BDRV_SECTOR_SIZE, buf); - bytes_written += n * BDRV_SECTOR_SIZE; + ret = commit_populate(s->top, s->base, offset, n, buf); + bytes_written += n; } if (ret < 0) { BlockErrorAction action = @@ -XXX,XX +XXX,XX @@ static void coroutine_fn commit_run(void *opaque) } } /* Publish progress */ - s->common.offset += n * BDRV_SECTOR_SIZE; + s->common.offset += n; if (copy && s->common.speed) { - delay_ns = ratelimit_calculate_delay(&s->limit, - n * BDRV_SECTOR_SIZE); + delay_ns = ratelimit_calculate_delay(&s->limit, n); } } diff --git a/block/io.c b/block/io.c index XXXXXXX..XXXXXXX 100644 --- a/block/io.c +++ b/block/io.c @@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_is_allocated(BlockDriverState *bs, int64_t offset, /* * Given an image chain: ... -> [BASE] -> [INTER1] -> [INTER2] -> [TOP] * - * Return true if the given sector is allocated in any image between - * BASE and TOP (inclusive). BASE can be NULL to check if the given - * sector is allocated in any image of the chain. Return false otherwise, + * Return true if (a prefix of) the given range is allocated in any image + * between BASE and TOP (inclusive). BASE can be NULL to check if the given + * offset is allocated in any image of the chain. Return false otherwise, * or negative errno on failure. * - * 'pnum' is set to the number of sectors (including and immediately following - * the specified sector) that are known to be in the same - * allocated/unallocated state. + * 'pnum' is set to the number of bytes (including and immediately + * following the specified offset) that are known to be in the same + * allocated/unallocated state. Note that a subsequent call starting + * at 'offset + *pnum' may return the same allocation status (in other + * words, the result is not necessarily the maximum possible range); + * but 'pnum' will only be 0 when end of file is reached. * */ int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, - int64_t sector_num, - int nb_sectors, int *pnum) + int64_t offset, int64_t bytes, int64_t *pnum) { BlockDriverState *intermediate; - int ret, n = nb_sectors; + int ret; + int64_t n = bytes; intermediate = top; while (intermediate && intermediate != base) { int64_t pnum_inter; int64_t size_inter; - int psectors_inter; - ret = bdrv_is_allocated(intermediate, sector_num * BDRV_SECTOR_SIZE, - nb_sectors * BDRV_SECTOR_SIZE, - &pnum_inter); + ret = bdrv_is_allocated(intermediate, offset, bytes, &pnum_inter); if (ret < 0) { return ret; } - assert(pnum_inter < INT_MAX * BDRV_SECTOR_SIZE); - psectors_inter = pnum_inter >> BDRV_SECTOR_BITS; if (ret) { - *pnum = psectors_inter; + *pnum = pnum_inter; return 1; } - size_inter = bdrv_nb_sectors(intermediate); + size_inter = bdrv_getlength(intermediate); if (size_inter < 0) { return size_inter; } - if (n > psectors_inter && - (intermediate == top || sector_num + psectors_inter < size_inter)) { - n = psectors_inter; + if (n > pnum_inter && + (intermediate == top || offset + pnum_inter < size_inter)) { + n = pnum_inter; } intermediate = backing_bs(intermediate); diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s) BlockDriverState *bs = s->source; BlockDriverState *target_bs = blk_bs(s->target); int ret, n; + int64_t count; end = s->bdev_length / BDRV_SECTOR_SIZE; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s) return 0; } - ret = bdrv_is_allocated_above(bs, base, sector_num, nb_sectors, &n); + ret = bdrv_is_allocated_above(bs, base, sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE, &count); if (ret < 0) { return ret; } + /* TODO: Relax this once bdrv_is_allocated_above and dirty + * bitmaps no longer require sector alignment. */ + assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); + n = count >> BDRV_SECTOR_BITS; assert(n > 0); if (ret == 1) { bdrv_set_dirty_bitmap(s->dirty_bitmap, sector_num, n); diff --git a/block/replication.c b/block/replication.c index XXXXXXX..XXXXXXX 100644 --- a/block/replication.c +++ b/block/replication.c @@ -XXX,XX +XXX,XX @@ static coroutine_fn int replication_co_writev(BlockDriverState *bs, BdrvChild *top = bs->file; BdrvChild *base = s->secondary_disk; BdrvChild *target; - int ret, n; + int ret; + int64_t n; ret = replication_get_io_status(s); if (ret < 0) { @@ -XXX,XX +XXX,XX @@ static coroutine_fn int replication_co_writev(BlockDriverState *bs, */ qemu_iovec_init(&hd_qiov, qiov->niov); while (remaining_sectors > 0) { - ret = bdrv_is_allocated_above(top->bs, base->bs, sector_num, - remaining_sectors, &n); + int64_t count; + + ret = bdrv_is_allocated_above(top->bs, base->bs, + sector_num * BDRV_SECTOR_SIZE, + remaining_sectors * BDRV_SECTOR_SIZE, + &count); if (ret < 0) { goto out1; } + assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); + n = count >> BDRV_SECTOR_BITS; qemu_iovec_reset(&hd_qiov); - qemu_iovec_concat(&hd_qiov, qiov, bytes_done, n * BDRV_SECTOR_SIZE); + qemu_iovec_concat(&hd_qiov, qiov, bytes_done, count); target = ret ? top : base; ret = bdrv_co_writev(target, sector_num, n, &hd_qiov); @@ -XXX,XX +XXX,XX @@ static coroutine_fn int replication_co_writev(BlockDriverState *bs, remaining_sectors -= n; sector_num += n; - bytes_done += n * BDRV_SECTOR_SIZE; + bytes_done += count; } out1: diff --git a/block/stream.c b/block/stream.c index XXXXXXX..XXXXXXX 100644 --- a/block/stream.c +++ b/block/stream.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) uint64_t delay_ns = 0; int error = 0; int ret = 0; - int n = 0; /* sectors */ + int64_t n = 0; /* bytes */ void *buf; if (!bs->backing) { @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) bdrv_enable_copy_on_read(bs); } - for ( ; offset < s->common.len; offset += n * BDRV_SECTOR_SIZE) { + for ( ; offset < s->common.len; offset += n) { bool copy; - int64_t count = 0; /* Note that even when no rate limit is applied we need to yield * with no pending I/O here so that bdrv_drain_all() returns. @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) copy = false; - ret = bdrv_is_allocated(bs, offset, STREAM_BUFFER_SIZE, &count); - /* TODO relax this once bdrv_is_allocated does not enforce sectors */ - assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); - n = count >> BDRV_SECTOR_BITS; + ret = bdrv_is_allocated(bs, offset, STREAM_BUFFER_SIZE, &n); if (ret == 1) { /* Allocated in the top, no need to copy. */ } else if (ret >= 0) { /* Copy if allocated in the intermediate images. Limit to the * known-unallocated area [offset, offset+n*BDRV_SECTOR_SIZE). */ ret = bdrv_is_allocated_above(backing_bs(bs), base, - offset / BDRV_SECTOR_SIZE, n, &n); + offset, n, &n); /* Finish early if end of backing file has been reached */ if (ret == 0 && n == 0) { - n = (s->common.len - offset) / BDRV_SECTOR_SIZE; + n = s->common.len - offset; } copy = (ret == 1); } - trace_stream_one_iteration(s, offset, n * BDRV_SECTOR_SIZE, ret); + trace_stream_one_iteration(s, offset, n, ret); if (copy) { - ret = stream_populate(blk, offset, n * BDRV_SECTOR_SIZE, buf); + ret = stream_populate(blk, offset, n, buf); } if (ret < 0) { BlockErrorAction action = @@ -XXX,XX +XXX,XX @@ static void coroutine_fn stream_run(void *opaque) ret = 0; /* Publish progress */ - s->common.offset += n * BDRV_SECTOR_SIZE; + s->common.offset += n; if (copy && s->common.speed) { - delay_ns = ratelimit_calculate_delay(&s->limit, - n * BDRV_SECTOR_SIZE); + delay_ns = ratelimit_calculate_delay(&s->limit, n); } } diff --git a/include/block/block.h b/include/block/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -XXX,XX +XXX,XX @@ int64_t bdrv_get_block_status_above(BlockDriverState *bs, int bdrv_is_allocated(BlockDriverState *bs, int64_t offset, int64_t bytes, int64_t *pnum); int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, - int64_t sector_num, int nb_sectors, int *pnum); + int64_t offset, int64_t bytes, int64_t *pnum); bool bdrv_is_read_only(BlockDriverState *bs); bool bdrv_is_writable(BlockDriverState *bs); diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static int img_compare(int argc, char **argv) } for (;;) { + int64_t count; + nb_sectors = sectors_to_process(total_sectors_over, sector_num); if (nb_sectors <= 0) { break; } - ret = bdrv_is_allocated_above(blk_bs(blk_over), NULL, sector_num, - nb_sectors, &pnum); + ret = bdrv_is_allocated_above(blk_bs(blk_over), NULL, + sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE, + &count); if (ret < 0) { ret = 3; error_report("Sector allocation test failed for %s", @@ -XXX,XX +XXX,XX @@ static int img_compare(int argc, char **argv) goto out; } - nb_sectors = pnum; + /* TODO relax this once bdrv_is_allocated_above does not enforce + * sector alignment */ + assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); + nb_sectors = count >> BDRV_SECTOR_BITS; if (ret) { ret = check_empty_sectors(blk_over, sector_num, nb_sectors, filename_over, buf1, quiet); -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> The block/crypto.c defines a set of QemuOpts that provide parameters for encryption. This will also be needed by the qcow/qcow2 integration, so expose the relevant pieces in a new block/crypto.h header. Some helper methods taking QemuOpts are changed to take QDict to simplify usage in other places. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-2-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/crypto.c | 82 +++++++++++++++++----------------------------------- block/crypto.h | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+), 56 deletions(-) create mode 100644 block/crypto.h diff --git a/block/crypto.c b/block/crypto.c index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -XXX,XX +XXX,XX @@ #include "sysemu/block-backend.h" #include "crypto/block.h" #include "qapi/opts-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "qapi-visit.h" #include "qapi/error.h" - -#define BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET "key-secret" -#define BLOCK_CRYPTO_OPT_LUKS_CIPHER_ALG "cipher-alg" -#define BLOCK_CRYPTO_OPT_LUKS_CIPHER_MODE "cipher-mode" -#define BLOCK_CRYPTO_OPT_LUKS_IVGEN_ALG "ivgen-alg" -#define BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG "ivgen-hash-alg" -#define BLOCK_CRYPTO_OPT_LUKS_HASH_ALG "hash-alg" -#define BLOCK_CRYPTO_OPT_LUKS_ITER_TIME "iter-time" +#include "block/crypto.h" typedef struct BlockCrypto BlockCrypto; @@ -XXX,XX +XXX,XX @@ static QemuOptsList block_crypto_runtime_opts_luks = { .name = "crypto", .head = QTAILQ_HEAD_INITIALIZER(block_crypto_runtime_opts_luks.head), .desc = { - { - .name = BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET, - .type = QEMU_OPT_STRING, - .help = "ID of the secret that provides the encryption key", - }, + BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET, { /* end of list */ } }, }; @@ -XXX,XX +XXX,XX @@ static QemuOptsList block_crypto_create_opts_luks = { .type = QEMU_OPT_SIZE, .help = "Virtual disk size" }, - { - .name = BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET, - .type = QEMU_OPT_STRING, - .help = "ID of the secret that provides the encryption key", - }, - { - .name = BLOCK_CRYPTO_OPT_LUKS_CIPHER_ALG, - .type = QEMU_OPT_STRING, - .help = "Name of encryption cipher algorithm", - }, - { - .name = BLOCK_CRYPTO_OPT_LUKS_CIPHER_MODE, - .type = QEMU_OPT_STRING, - .help = "Name of encryption cipher mode", - }, - { - .name = BLOCK_CRYPTO_OPT_LUKS_IVGEN_ALG, - .type = QEMU_OPT_STRING, - .help = "Name of IV generator algorithm", - }, - { - .name = BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG, - .type = QEMU_OPT_STRING, - .help = "Name of IV generator hash algorithm", - }, - { - .name = BLOCK_CRYPTO_OPT_LUKS_HASH_ALG, - .type = QEMU_OPT_STRING, - .help = "Name of encryption hash algorithm", - }, - { - .name = BLOCK_CRYPTO_OPT_LUKS_ITER_TIME, - .type = QEMU_OPT_NUMBER, - .help = "Time to spend in PBKDF in milliseconds", - }, + BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET, + BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG, + BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE, + BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG, + BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG, + BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG, + BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME, { /* end of list */ } }, }; -static QCryptoBlockOpenOptions * +QCryptoBlockOpenOptions * block_crypto_open_opts_init(QCryptoBlockFormat format, - QemuOpts *opts, + QDict *opts, Error **errp) { Visitor *v; @@ -XXX,XX +XXX,XX @@ block_crypto_open_opts_init(QCryptoBlockFormat format, ret = g_new0(QCryptoBlockOpenOptions, 1); ret->format = format; - v = opts_visitor_new(opts); + v = qobject_input_visitor_new_keyval(QOBJECT(opts)); visit_start_struct(v, NULL, NULL, 0, &local_err); if (local_err) { @@ -XXX,XX +XXX,XX @@ block_crypto_open_opts_init(QCryptoBlockFormat format, } -static QCryptoBlockCreateOptions * +QCryptoBlockCreateOptions * block_crypto_create_opts_init(QCryptoBlockFormat format, - QemuOpts *opts, + QDict *opts, Error **errp) { Visitor *v; @@ -XXX,XX +XXX,XX @@ block_crypto_create_opts_init(QCryptoBlockFormat format, ret = g_new0(QCryptoBlockCreateOptions, 1); ret->format = format; - v = opts_visitor_new(opts); + v = qobject_input_visitor_new_keyval(QOBJECT(opts)); visit_start_struct(v, NULL, NULL, 0, &local_err); if (local_err) { @@ -XXX,XX +XXX,XX @@ static int block_crypto_open_generic(QCryptoBlockFormat format, int ret = -EINVAL; QCryptoBlockOpenOptions *open_opts = NULL; unsigned int cflags = 0; + QDict *cryptoopts = NULL; bs->file = bdrv_open_child(NULL, options, "file", bs, &child_file, false, errp); @@ -XXX,XX +XXX,XX @@ static int block_crypto_open_generic(QCryptoBlockFormat format, goto cleanup; } - open_opts = block_crypto_open_opts_init(format, opts, errp); + cryptoopts = qemu_opts_to_qdict(opts, NULL); + + open_opts = block_crypto_open_opts_init(format, cryptoopts, errp); if (!open_opts) { goto cleanup; } @@ -XXX,XX +XXX,XX @@ static int block_crypto_open_generic(QCryptoBlockFormat format, ret = 0; cleanup: + QDECREF(cryptoopts); qapi_free_QCryptoBlockOpenOptions(open_opts); return ret; } @@ -XXX,XX +XXX,XX @@ static int block_crypto_create_generic(QCryptoBlockFormat format, .opts = opts, .filename = filename, }; + QDict *cryptoopts; + + cryptoopts = qemu_opts_to_qdict(opts, NULL); - create_opts = block_crypto_create_opts_init(format, opts, errp); + create_opts = block_crypto_create_opts_init(format, cryptoopts, errp); if (!create_opts) { return -1; } @@ -XXX,XX +XXX,XX @@ static int block_crypto_create_generic(QCryptoBlockFormat format, ret = 0; cleanup: + QDECREF(cryptoopts); qcrypto_block_free(crypto); blk_unref(data.blk); qapi_free_QCryptoBlockCreateOptions(create_opts); diff --git a/block/crypto.h b/block/crypto.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/block/crypto.h @@ -XXX,XX +XXX,XX @@ +/* + * QEMU block full disk encryption + * + * Copyright (c) 2015-2017 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see <http://www.gnu.org/licenses/>. + * + */ + +#ifndef BLOCK_CRYPTO_H__ +#define BLOCK_CRYPTO_H__ + +#define BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET "key-secret" +#define BLOCK_CRYPTO_OPT_LUKS_CIPHER_ALG "cipher-alg" +#define BLOCK_CRYPTO_OPT_LUKS_CIPHER_MODE "cipher-mode" +#define BLOCK_CRYPTO_OPT_LUKS_IVGEN_ALG "ivgen-alg" +#define BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG "ivgen-hash-alg" +#define BLOCK_CRYPTO_OPT_LUKS_HASH_ALG "hash-alg" +#define BLOCK_CRYPTO_OPT_LUKS_ITER_TIME "iter-time" + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET \ + { \ + .name = BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET, \ + .type = QEMU_OPT_STRING, \ + .help = "ID of the secret that provides the keyslot passphrase", \ + } + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG \ + { \ + .name = BLOCK_CRYPTO_OPT_LUKS_CIPHER_ALG, \ + .type = QEMU_OPT_STRING, \ + .help = "Name of encryption cipher algorithm", \ + } + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE \ + { \ + .name = BLOCK_CRYPTO_OPT_LUKS_CIPHER_MODE, \ + .type = QEMU_OPT_STRING, \ + .help = "Name of encryption cipher mode", \ + } + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG \ + { \ + .name = BLOCK_CRYPTO_OPT_LUKS_IVGEN_ALG, \ + .type = QEMU_OPT_STRING, \ + .help = "Name of IV generator algorithm", \ + } + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG \ + { \ + .name = BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG, \ + .type = QEMU_OPT_STRING, \ + .help = "Name of IV generator hash algorithm", \ + } + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG \ + { \ + .name = BLOCK_CRYPTO_OPT_LUKS_HASH_ALG, \ + .type = QEMU_OPT_STRING, \ + .help = "Name of encryption hash algorithm", \ + } + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME \ + { \ + .name = BLOCK_CRYPTO_OPT_LUKS_ITER_TIME, \ + .type = QEMU_OPT_NUMBER, \ + .help = "Time to spend in PBKDF in milliseconds", \ + } + +QCryptoBlockCreateOptions * +block_crypto_create_opts_init(QCryptoBlockFormat format, + QDict *opts, + Error **errp); + +QCryptoBlockOpenOptions * +block_crypto_open_opts_init(QCryptoBlockFormat format, + QDict *opts, + Error **errp); + +#endif /* BLOCK_CRYPTO_H__ */ -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> When integrating the crypto support with qcow/qcow2, we don't want to use the bare LUKS option names "hash-alg", "key-secret", etc. We need to namespace them to match the nested QAPI schema. e.g. "encrypt.hash-alg", "encrypt.key-secret" so that they don't clash with any general qcow options at a later date. Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-3-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/crypto.c | 16 ++++++++-------- block/crypto.h | 40 ++++++++++++++++++++-------------------- 2 files changed, 28 insertions(+), 28 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -XXX,XX +XXX,XX @@ static QemuOptsList block_crypto_runtime_opts_luks = { .name = "crypto", .head = QTAILQ_HEAD_INITIALIZER(block_crypto_runtime_opts_luks.head), .desc = { - BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET, + BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET(""), { /* end of list */ } }, }; @@ -XXX,XX +XXX,XX @@ static QemuOptsList block_crypto_create_opts_luks = { .type = QEMU_OPT_SIZE, .help = "Virtual disk size" }, - BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET, - BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG, - BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE, - BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG, - BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG, - BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG, - BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME, + BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET(""), + BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG(""), + BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE(""), + BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG(""), + BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG(""), + BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG(""), + BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME(""), { /* end of list */ } }, }; diff --git a/block/crypto.h b/block/crypto.h index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.h +++ b/block/crypto.h @@ -XXX,XX +XXX,XX @@ #define BLOCK_CRYPTO_OPT_LUKS_HASH_ALG "hash-alg" #define BLOCK_CRYPTO_OPT_LUKS_ITER_TIME "iter-time" -#define BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET \ +#define BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET(prefix) \ { \ - .name = BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET, \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET, \ .type = QEMU_OPT_STRING, \ .help = "ID of the secret that provides the keyslot passphrase", \ } -#define BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG \ +#define BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG(prefix) \ { \ - .name = BLOCK_CRYPTO_OPT_LUKS_CIPHER_ALG, \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_CIPHER_ALG, \ .type = QEMU_OPT_STRING, \ .help = "Name of encryption cipher algorithm", \ } -#define BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE \ - { \ - .name = BLOCK_CRYPTO_OPT_LUKS_CIPHER_MODE, \ - .type = QEMU_OPT_STRING, \ - .help = "Name of encryption cipher mode", \ +#define BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE(prefix) \ + { \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_CIPHER_MODE, \ + .type = QEMU_OPT_STRING, \ + .help = "Name of encryption cipher mode", \ } -#define BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG \ - { \ - .name = BLOCK_CRYPTO_OPT_LUKS_IVGEN_ALG, \ - .type = QEMU_OPT_STRING, \ - .help = "Name of IV generator algorithm", \ +#define BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG(prefix) \ + { \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_IVGEN_ALG, \ + .type = QEMU_OPT_STRING, \ + .help = "Name of IV generator algorithm", \ } -#define BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG \ +#define BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG(prefix) \ { \ - .name = BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG, \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG, \ .type = QEMU_OPT_STRING, \ .help = "Name of IV generator hash algorithm", \ } -#define BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG \ +#define BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG(prefix) \ { \ - .name = BLOCK_CRYPTO_OPT_LUKS_HASH_ALG, \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_HASH_ALG, \ .type = QEMU_OPT_STRING, \ .help = "Name of encryption hash algorithm", \ } -#define BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME \ +#define BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME(prefix) \ { \ - .name = BLOCK_CRYPTO_OPT_LUKS_ITER_TIME, \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_ITER_TIME, \ .type = QEMU_OPT_NUMBER, \ .help = "Time to spend in PBKDF in milliseconds", \ } -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Document that use of guest virtual sector numbers as the basis for the initialization vectors is a potential weakness, when combined with internal snapshots or multiple images using the same passphrase. This fixes the formatting of the itemized list too. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-4-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- qemu-img.texi | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/qemu-img.texi b/qemu-img.texi index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.texi +++ b/qemu-img.texi @@ -XXX,XX +XXX,XX @@ The use of encryption in qcow and qcow2 images is considered to be flawed by modern cryptography standards, suffering from a number of design problems: @itemize @minus -@item The AES-CBC cipher is used with predictable initialization vectors based +@item +The AES-CBC cipher is used with predictable initialization vectors based on the sector number. This makes it vulnerable to chosen plaintext attacks which can reveal the existence of encrypted data. -@item The user passphrase is directly used as the encryption key. A poorly +@item +The user passphrase is directly used as the encryption key. A poorly chosen or short passphrase will compromise the security of the encryption. -@item In the event of the passphrase being compromised there is no way to +@item +In the event of the passphrase being compromised there is no way to change the passphrase to protect data in any qcow images. The files must be cloned, using a different encryption passphrase in the new file. The original file must then be securely erased using a program like shred, though even this is ineffective with many modern storage technologies. +@item +Initialization vectors used to encrypt sectors are based on the +guest virtual sector number, instead of the host physical sector. When +a disk image has multiple internal snapshots this means that data in +multiple physical sectors is encrypted with the same initialization +vector. With the CBC mode, this opens the possibility of watermarking +attacks if the attack can collect multiple sectors encrypted with the +same IV and some predictable data. Having multiple qcow2 images with +the same passphrase also exposes this weakness since the passphrase +is directly used as the key. @end itemize Use of qcow / qcow2 encryption is thus strongly discouraged. Users are -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> The qcow driver refuses to open images which are less than 2 bytes in size, but will happily create such images. Add a check in the create path to avoid this discrepancy. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-5-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp) /* Read out options */ total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0), BDRV_SECTOR_SIZE); + if (total_size == 0) { + error_setg(errp, "Image size is too small, cannot be zero length"); + ret = -EINVAL; + goto cleanup; + } + backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE); if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ENCRYPT, false)) { flags |= BLOCK_FLAG_ENCRYPT; -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Test 042 is designed to verify operation with zero sized images. Such images are not supported with qcow (v1), so this test has always failed. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-6-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/042 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/042 b/tests/qemu-iotests/042 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/042 +++ b/tests/qemu-iotests/042 @@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -_supported_fmt qcow2 qcow qed +_supported_fmt qcow2 qed _supported_proto file _supported_os Linux -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Test 048 is designed to verify data preservation during an image resize. The qcow (v1) format impl has never supported resize so always fails. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-7-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/048 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/048 b/tests/qemu-iotests/048 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/048 +++ b/tests/qemu-iotests/048 @@ -XXX,XX +XXX,XX @@ _compare() . ./common.filter . ./common.pattern -_supported_fmt raw qcow qcow2 qed luks +_supported_fmt raw qcow2 qed luks _supported_proto file _supported_os Linux -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Historically the qcow & qcow2 image formats supported a property "encryption=on" to enable their built-in AES encryption. We'll soon be supporting LUKS for qcow2, so need a more general purpose way to enable encryption, with a choice of formats. This introduces an "encrypt.format" option, which will later be joined by a number of other "encrypt.XXX" options. The use of a "encrypt." prefix instead of "encrypt-" is done to facilitate mapping to a nested QAPI schema at later date. e.g. the preferred syntax is now qemu-img create -f qcow2 -o encrypt.format=aes demo.qcow2 Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-8-berrange@redhat.com Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow.c | 31 ++++++++++++--- block/qcow2.c | 34 ++++++++++++---- include/block/block_int.h | 2 +- qemu-img.c | 4 +- tests/qemu-iotests/049.out | 98 +++++++++++++++++++++++----------------------- tests/qemu-iotests/082.out | 95 ++++++++++++++++++++++++++++---------------- tests/qemu-iotests/085.out | 38 +++++++++--------- tests/qemu-iotests/144.out | 4 +- tests/qemu-iotests/185.out | 8 ++-- 9 files changed, 191 insertions(+), 123 deletions(-) diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp) uint8_t *tmp; int64_t total_size = 0; char *backing_file = NULL; - int flags = 0; Error *local_err = NULL; int ret; BlockBackend *qcow_blk; + const char *encryptfmt = NULL; /* Read out options */ total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0), @@ -XXX,XX +XXX,XX @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp) } backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE); - if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ENCRYPT, false)) { - flags |= BLOCK_FLAG_ENCRYPT; + encryptfmt = qemu_opt_get_del(opts, BLOCK_OPT_ENCRYPT_FORMAT); + if (encryptfmt) { + if (qemu_opt_get(opts, BLOCK_OPT_ENCRYPT)) { + error_setg(errp, "Options " BLOCK_OPT_ENCRYPT " and " + BLOCK_OPT_ENCRYPT_FORMAT " are mutually exclusive"); + ret = -EINVAL; + goto cleanup; + } + } else if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ENCRYPT, false)) { + encryptfmt = "aes"; } ret = bdrv_create_file(filename, opts, &local_err); @@ -XXX,XX +XXX,XX @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp) l1_size = (total_size + (1LL << shift) - 1) >> shift; header.l1_table_offset = cpu_to_be64(header_size); - if (flags & BLOCK_FLAG_ENCRYPT) { + if (encryptfmt) { + if (!g_str_equal(encryptfmt, "aes")) { + error_setg(errp, "Unknown encryption format '%s', expected 'aes'", + encryptfmt); + ret = -EINVAL; + goto exit; + } header.crypt_method = cpu_to_be32(QCOW_CRYPT_AES); } else { header.crypt_method = cpu_to_be32(QCOW_CRYPT_NONE); @@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow_create_opts = { { .name = BLOCK_OPT_ENCRYPT, .type = QEMU_OPT_BOOL, - .help = "Encrypt the image", - .def_value_str = "off" + .help = "Encrypt the image with format 'aes'. (Deprecated " + "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)", + }, + { + .name = BLOCK_OPT_ENCRYPT_FORMAT, + .type = QEMU_OPT_STRING, + .help = "Encrypt the image, format choices: 'aes'", }, { /* end of list */ } } diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int qcow2_create2(const char *filename, int64_t total_size, const char *backing_file, const char *backing_format, int flags, size_t cluster_size, PreallocMode prealloc, QemuOpts *opts, int version, int refcount_order, - Error **errp) + const char *encryptfmt, Error **errp) { int cluster_bits; QDict *options; @@ -XXX,XX +XXX,XX @@ static int qcow2_create2(const char *filename, int64_t total_size, .header_length = cpu_to_be32(sizeof(*header)), }; - if (flags & BLOCK_FLAG_ENCRYPT) { + if (encryptfmt) { + if (!g_str_equal(encryptfmt, "aes")) { + error_setg(errp, "Unknown encryption format '%s', expected 'aes'", + encryptfmt); + ret = -EINVAL; + goto out; + } header->crypt_method = cpu_to_be32(QCOW_CRYPT_AES); } else { header->crypt_method = cpu_to_be32(QCOW_CRYPT_NONE); @@ -XXX,XX +XXX,XX @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp) int version = 3; uint64_t refcount_bits = 16; int refcount_order; + const char *encryptfmt = NULL; Error *local_err = NULL; int ret; @@ -XXX,XX +XXX,XX @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp) BDRV_SECTOR_SIZE); backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE); backing_fmt = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FMT); - if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ENCRYPT, false)) { - flags |= BLOCK_FLAG_ENCRYPT; + encryptfmt = qemu_opt_get_del(opts, BLOCK_OPT_ENCRYPT_FORMAT); + if (encryptfmt) { + if (qemu_opt_get_del(opts, BLOCK_OPT_ENCRYPT)) { + error_setg(errp, "Options " BLOCK_OPT_ENCRYPT " and " + BLOCK_OPT_ENCRYPT_FORMAT " are mutually exclusive"); + ret = -EINVAL; + goto finish; + } + } else if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ENCRYPT, false)) { + encryptfmt = "aes"; } cluster_size = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE, DEFAULT_CLUSTER_SIZE); @@ -XXX,XX +XXX,XX @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp) ret = qcow2_create2(filename, size, backing_file, backing_fmt, flags, cluster_size, prealloc, opts, version, refcount_order, - &local_err); + encryptfmt, &local_err); error_propagate(errp, local_err); finish: @@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_create_opts = { { .name = BLOCK_OPT_ENCRYPT, .type = QEMU_OPT_BOOL, - .help = "Encrypt the image", - .def_value_str = "off" + .help = "Encrypt the image with format 'aes'. (Deprecated " + "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)", + }, + { + .name = BLOCK_OPT_ENCRYPT_FORMAT, + .type = QEMU_OPT_STRING, + .help = "Encrypt the image, format choices: 'aes'", }, { .name = BLOCK_OPT_CLUSTER_SIZE, diff --git a/include/block/block_int.h b/include/block/block_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -XXX,XX +XXX,XX @@ #include "qemu/main-loop.h" #include "qemu/throttle.h" -#define BLOCK_FLAG_ENCRYPT 1 #define BLOCK_FLAG_LAZY_REFCOUNTS 8 #define BLOCK_OPT_SIZE "size" #define BLOCK_OPT_ENCRYPT "encryption" +#define BLOCK_OPT_ENCRYPT_FORMAT "encrypt.format" #define BLOCK_OPT_COMPAT6 "compat6" #define BLOCK_OPT_HWVERSION "hwversion" #define BLOCK_OPT_BACKING_FILE "backing_file" diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv) if (s.compressed) { bool encryption = qemu_opt_get_bool(opts, BLOCK_OPT_ENCRYPT, false); + const char *encryptfmt = + qemu_opt_get(opts, BLOCK_OPT_ENCRYPT_FORMAT); const char *preallocation = qemu_opt_get(opts, BLOCK_OPT_PREALLOC); @@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv) goto out; } - if (encryption) { + if (encryption || encryptfmt) { error_report("Compression and encryption not supported at " "the same time"); ret = -1; diff --git a/tests/qemu-iotests/049.out b/tests/qemu-iotests/049.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/049.out +++ b/tests/qemu-iotests/049.out @@ -XXX,XX +XXX,XX @@ QA output created by 049 == 1. Traditional size parameter == qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024b -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1k -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1K -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1G -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1T -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0b -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5k -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5K -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5G -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5T -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 == 2. Specifying size via -o == qemu-img create -f qcow2 -o size=1024 TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1024b TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1k TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1K TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1M TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1G TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1T TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1024.0 TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1024.0b TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5k TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5K TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5M TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5G TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5T TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 == 3. Invalid sizes == @@ -XXX,XX +XXX,XX @@ qemu-img: TEST_DIR/t.qcow2: Invalid options for file format 'qcow2' == Check correct interpretation of suffixes for cluster size == qemu-img create -f qcow2 -o cluster_size=1024 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1024b TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1k TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1K TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1M TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=1048576 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1048576 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1024.0 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1024.0b TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=0.5k TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=512 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=0.5K TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=512 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=0.5M TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=524288 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=524288 lazy_refcounts=off refcount_bits=16 == Check compat level option == qemu-img create -f qcow2 -o compat=0.10 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=1.1 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=0.42 TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Invalid compatibility level: '0.42' -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.42 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.42 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=foobar TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Invalid compatibility level: 'foobar' -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=foobar encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=foobar cluster_size=65536 lazy_refcounts=off refcount_bits=16 == Check preallocation option == qemu-img create -f qcow2 -o preallocation=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o preallocation=metadata TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o preallocation=1234 TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: invalid parameter value: 1234 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 preallocation=1234 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=1234 lazy_refcounts=off refcount_bits=16 == Check encryption option == @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=on cluster_siz == Check lazy_refcounts option (only with v3) == qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 encryption=off cluster_size=65536 lazy_refcounts=on refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=on refcount_bits=16 qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=on TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater) -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 encryption=off cluster_size=65536 lazy_refcounts=on refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=on refcount_bits=16 *** done diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -XXX,XX +XXX,XX @@ QA output created by 082 === create: Options specified more than once === Testing: create -f foo -f qcow2 TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128M (134217728 bytes) cluster_size: 65536 Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 encryption=off cluster_size=4096 lazy_refcounts=on refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=4096 lazy_refcounts=on refcount_bits=16 image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128M (134217728 bytes) @@ -XXX,XX +XXX,XX @@ Format specific information: corrupt: false Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 encryption=off cluster_size=8192 lazy_refcounts=on refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=on refcount_bits=16 image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128M (134217728 bytes) @@ -XXX,XX +XXX,XX @@ Format specific information: corrupt: false Testing: create -f qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 encryption=off cluster_size=8192 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=off refcount_bits=16 image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128M (134217728 bytes) @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ refcount_bits Width of a reference count entry in bits nocow Turn off copy-on-write (valid only on btrfs) Testing: create -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,help encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,help cluster_size=65536 lazy_refcounts=off refcount_bits=16 Testing: create -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,? TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,? encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,? cluster_size=65536 lazy_refcounts=off refcount_bits=16 Testing: create -f qcow2 -o backing_file=TEST_DIR/t.qcow2, -o help TEST_DIR/t.qcow2 128M qemu-img: Invalid option list: backing_file=TEST_DIR/t.qcow2, @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size === convert: Options specified more than once === Testing: create -f qcow2 TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 Testing: convert -f foo -f qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base image: TEST_DIR/t.IMGFMT.base @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ size Virtual disk size compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image -encryption Encrypt the image +encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) +encrypt.format Encrypt the image, format choices: 'aes' cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates diff --git a/tests/qemu-iotests/085.out b/tests/qemu-iotests/085.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/085.out +++ b/tests/qemu-iotests/085.out @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 === Create a single snapshot on virtio0 === -Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} === Invalid command - missing device and nodename === @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file === Create several transactional group snapshots === -Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} -Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} -Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} -Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} -Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} -Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} -Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} -Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} -Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} === Create a couple of snapshots using blockdev-snapshot === diff --git a/tests/qemu-iotests/144.out b/tests/qemu-iotests/144.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/144.out +++ b/tests/qemu-iotests/144.out @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=536870912 === Performing Live Snapshot 1 === {"return": {}} -Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} === Performing block-commit on active layer === @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/ === Performing Live Snapshot 2 === -Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} *** done diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/185.out +++ b/tests/qemu-iotests/185.out @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 === Creating backing chain === -Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} wrote 4194304/4194304 bytes at offset 0 4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} === Start commit job and exit qemu === @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q === Start mirror job and exit qemu === {"return": {}} -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 encryption=off clust === Start backup job and exit qemu === {"return": {}} -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Instead of requiring separate input/output buffers for encrypting data, change encrypt_sectors() to assume use of a single buffer, encrypting in place. One current caller uses the same buffer for input/output already and the other two callers are easily converted to do so. Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-9-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow.c | 45 +++++++++++++++------------------------------ 1 file changed, 15 insertions(+), 30 deletions(-) diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int qcow_set_key(BlockDriverState *bs, const char *key) } /* The crypt function is compatible with the linux cryptoloop - algorithm for < 4 GB images. NOTE: out_buf == in_buf is - supported */ + algorithm for < 4 GB images. */ static int encrypt_sectors(BDRVQcowState *s, int64_t sector_num, - uint8_t *out_buf, const uint8_t *in_buf, - int nb_sectors, bool enc, Error **errp) + uint8_t *buf, int nb_sectors, bool enc, + Error **errp) { union { uint64_t ll[2]; @@ -XXX,XX +XXX,XX @@ static int encrypt_sectors(BDRVQcowState *s, int64_t sector_num, } if (enc) { ret = qcrypto_cipher_encrypt(s->cipher, - in_buf, - out_buf, + buf, buf, 512, errp); } else { ret = qcrypto_cipher_decrypt(s->cipher, - in_buf, - out_buf, + buf, buf, 512, errp); } @@ -XXX,XX +XXX,XX @@ static int encrypt_sectors(BDRVQcowState *s, int64_t sector_num, return -1; } sector_num++; - in_buf += 512; - out_buf += 512; + buf += 512; } return 0; } @@ -XXX,XX +XXX,XX @@ static uint64_t get_cluster_offset(BlockDriverState *bs, uint64_t start_sect; assert(s->cipher); start_sect = (offset & ~(s->cluster_size - 1)) >> 9; - memset(s->cluster_data + 512, 0x00, 512); for(i = 0; i < s->cluster_sectors; i++) { if (i < n_start || i >= n_end) { Error *err = NULL; + memset(s->cluster_data, 0x00, 512); if (encrypt_sectors(s, start_sect + i, - s->cluster_data, - s->cluster_data + 512, 1, + s->cluster_data, 1, true, &err) < 0) { error_free(err); errno = EIO; @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_readv(BlockDriverState *bs, int64_t sector_num, } if (bs->encrypted) { assert(s->cipher); - if (encrypt_sectors(s, sector_num, buf, buf, + if (encrypt_sectors(s, sector_num, buf, n, false, &err) < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_writev(BlockDriverState *bs, int64_t sector_num, BDRVQcowState *s = bs->opaque; int index_in_cluster; uint64_t cluster_offset; - const uint8_t *src_buf; int ret = 0, n; - uint8_t *cluster_data = NULL; struct iovec hd_iov; QEMUIOVector hd_qiov; uint8_t *buf; @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_writev(BlockDriverState *bs, int64_t sector_num, s->cluster_cache_offset = -1; /* disable compressed cache */ - if (qiov->niov > 1) { + /* We must always copy the iov when encrypting, so we + * don't modify the original data buffer during encryption */ + if (bs->encrypted || qiov->niov > 1) { buf = orig_buf = qemu_try_blockalign(bs, qiov->size); if (buf == NULL) { return -ENOMEM; @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_writev(BlockDriverState *bs, int64_t sector_num, if (bs->encrypted) { Error *err = NULL; assert(s->cipher); - if (!cluster_data) { - cluster_data = g_malloc0(s->cluster_size); - } - if (encrypt_sectors(s, sector_num, cluster_data, buf, - n, true, &err) < 0) { + if (encrypt_sectors(s, sector_num, buf, n, true, &err) < 0) { error_free(err); ret = -EIO; break; } - src_buf = cluster_data; - } else { - src_buf = buf; } - hd_iov.iov_base = (void *)src_buf; + hd_iov.iov_base = (void *)buf; hd_iov.iov_len = n * 512; qemu_iovec_init_external(&hd_qiov, &hd_iov, 1); qemu_co_mutex_unlock(&s->lock); @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_writev(BlockDriverState *bs, int64_t sector_num, } qemu_co_mutex_unlock(&s->lock); - if (qiov->niov > 1) { - qemu_vfree(orig_buf); - } - g_free(cluster_data); + qemu_vfree(orig_buf); return ret; } -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> This converts the qcow driver to make use of the QCryptoBlock APIs for encrypting image content. This is only wired up to permit use of the legacy QCow encryption format. Users who wish to have the strong LUKS format should switch to qcow2 instead. With this change it is now required to use the QCryptoSecret object for providing passwords, instead of the current block password APIs / interactive prompting. $QEMU \ -object secret,id=sec0,file=/home/berrange/encrypted.pw \ -drive file=/home/berrange/encrypted.qcow,encrypt.format=aes,\ encrypt.key-secret=sec0 Though note that running QEMU system emulators with the AES encryption is no longer supported, so while the above syntax is valid, QEMU will refuse to actually run the VM in this particular example. Likewise when creating images with the legacy AES-CBC format qemu-img create -f qcow \ --object secret,id=sec0,file=/home/berrange/encrypted.pw \ -o encrypt.format=aes,encrypt.key-secret=sec0 \ /home/berrange/encrypted.qcow 64M Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-10-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/crypto.c | 10 +++ block/crypto.h | 20 ++++-- block/qcow.c | 198 +++++++++++++++++++++++++-------------------------- qapi/block-core.json | 38 +++++++++- 4 files changed, 158 insertions(+), 108 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -XXX,XX +XXX,XX @@ block_crypto_open_opts_init(QCryptoBlockFormat format, v, &ret->u.luks, &local_err); break; + case Q_CRYPTO_BLOCK_FORMAT_QCOW: + visit_type_QCryptoBlockOptionsQCow_members( + v, &ret->u.qcow, &local_err); + break; + default: error_setg(&local_err, "Unsupported block format %d", format); break; @@ -XXX,XX +XXX,XX @@ block_crypto_create_opts_init(QCryptoBlockFormat format, v, &ret->u.luks, &local_err); break; + case Q_CRYPTO_BLOCK_FORMAT_QCOW: + visit_type_QCryptoBlockOptionsQCow_members( + v, &ret->u.qcow, &local_err); + break; + default: error_setg(&local_err, "Unsupported block format %d", format); break; diff --git a/block/crypto.h b/block/crypto.h index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.h +++ b/block/crypto.h @@ -XXX,XX +XXX,XX @@ #ifndef BLOCK_CRYPTO_H__ #define BLOCK_CRYPTO_H__ +#define BLOCK_CRYPTO_OPT_DEF_KEY_SECRET(prefix, helpstr) \ + { \ + .name = prefix BLOCK_CRYPTO_OPT_QCOW_KEY_SECRET, \ + .type = QEMU_OPT_STRING, \ + .help = helpstr, \ + } + +#define BLOCK_CRYPTO_OPT_QCOW_KEY_SECRET "key-secret" + +#define BLOCK_CRYPTO_OPT_DEF_QCOW_KEY_SECRET(prefix) \ + BLOCK_CRYPTO_OPT_DEF_KEY_SECRET(prefix, \ + "ID of the secret that provides the AES encryption key") + #define BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET "key-secret" #define BLOCK_CRYPTO_OPT_LUKS_CIPHER_ALG "cipher-alg" #define BLOCK_CRYPTO_OPT_LUKS_CIPHER_MODE "cipher-mode" @@ -XXX,XX +XXX,XX @@ #define BLOCK_CRYPTO_OPT_LUKS_ITER_TIME "iter-time" #define BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET(prefix) \ - { \ - .name = prefix BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET, \ - .type = QEMU_OPT_STRING, \ - .help = "ID of the secret that provides the keyslot passphrase", \ - } + BLOCK_CRYPTO_OPT_DEF_KEY_SECRET(prefix, \ + "ID of the secret that provides the keyslot passphrase") #define BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG(prefix) \ { \ diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ #include "qemu/bswap.h" #include <zlib.h> #include "qapi/qmp/qerror.h" -#include "crypto/cipher.h" +#include "qapi/qmp/qstring.h" +#include "crypto/block.h" #include "migration/blocker.h" +#include "block/crypto.h" /**************************************************************/ /* QEMU COW block driver with compression and encryption support */ @@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcowState { uint8_t *cluster_cache; uint8_t *cluster_data; uint64_t cluster_cache_offset; - QCryptoCipher *cipher; /* NULL if no key yet */ + QCryptoBlock *crypto; /* Disk encryption format driver */ uint32_t crypt_method_header; CoMutex lock; Error *migration_blocker; @@ -XXX,XX +XXX,XX @@ static int qcow_probe(const uint8_t *buf, int buf_size, const char *filename) return 0; } +static QemuOptsList qcow_runtime_opts = { + .name = "qcow", + .head = QTAILQ_HEAD_INITIALIZER(qcow_runtime_opts.head), + .desc = { + BLOCK_CRYPTO_OPT_DEF_QCOW_KEY_SECRET("encrypt."), + { /* end of list */ } + }, +}; + static int qcow_open(BlockDriverState *bs, QDict *options, int flags, Error **errp) { @@ -XXX,XX +XXX,XX @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags, int ret; QCowHeader header; Error *local_err = NULL; + QCryptoBlockOpenOptions *crypto_opts = NULL; + unsigned int cflags = 0; + QDict *encryptopts = NULL; + const char *encryptfmt; + + qdict_extract_subqdict(options, &encryptopts, "encrypt."); + encryptfmt = qdict_get_try_str(encryptopts, "format"); bs->file = bdrv_open_child(NULL, options, "file", bs, &child_file, false, errp); if (!bs->file) { - return -EINVAL; + ret = -EINVAL; + goto fail; } ret = bdrv_pread(bs->file, 0, &header, sizeof(header)); @@ -XXX,XX +XXX,XX @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags, goto fail; } - if (header.crypt_method > QCOW_CRYPT_AES) { - error_setg(errp, "invalid encryption method in qcow header"); - ret = -EINVAL; - goto fail; - } - if (!qcrypto_cipher_supports(QCRYPTO_CIPHER_ALG_AES_128, - QCRYPTO_CIPHER_MODE_CBC)) { - error_setg(errp, "AES cipher not available"); - ret = -EINVAL; - goto fail; - } s->crypt_method_header = header.crypt_method; if (s->crypt_method_header) { if (bdrv_uses_whitelist() && @@ -XXX,XX +XXX,XX @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags, ret = -ENOSYS; goto fail; } + if (s->crypt_method_header == QCOW_CRYPT_AES) { + if (encryptfmt && !g_str_equal(encryptfmt, "aes")) { + error_setg(errp, + "Header reported 'aes' encryption format but " + "options specify '%s'", encryptfmt); + ret = -EINVAL; + goto fail; + } + qdict_del(encryptopts, "format"); + crypto_opts = block_crypto_open_opts_init( + Q_CRYPTO_BLOCK_FORMAT_QCOW, encryptopts, errp); + if (!crypto_opts) { + ret = -EINVAL; + goto fail; + } + if (flags & BDRV_O_NO_IO) { + cflags |= QCRYPTO_BLOCK_OPEN_NO_IO; + } + s->crypto = qcrypto_block_open(crypto_opts, NULL, NULL, + cflags, errp); + if (!s->crypto) { + ret = -EINVAL; + goto fail; + } + } else { + error_setg(errp, "invalid encryption method in qcow header"); + ret = -EINVAL; + goto fail; + } bs->encrypted = true; + bs->valid_key = true; } s->cluster_bits = header.cluster_bits; s->cluster_size = 1 << s->cluster_bits; @@ -XXX,XX +XXX,XX @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags, goto fail; } + QDECREF(encryptopts); + qapi_free_QCryptoBlockOpenOptions(crypto_opts); qemu_co_mutex_init(&s->lock); return 0; @@ -XXX,XX +XXX,XX @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags, qemu_vfree(s->l2_cache); g_free(s->cluster_cache); g_free(s->cluster_data); + qcrypto_block_free(s->crypto); + QDECREF(encryptopts); + qapi_free_QCryptoBlockOpenOptions(crypto_opts); return ret; } @@ -XXX,XX +XXX,XX @@ static int qcow_reopen_prepare(BDRVReopenState *state, return 0; } -static int qcow_set_key(BlockDriverState *bs, const char *key) -{ - BDRVQcowState *s = bs->opaque; - uint8_t keybuf[16]; - int len, i; - Error *err; - - memset(keybuf, 0, 16); - len = strlen(key); - if (len > 16) - len = 16; - /* XXX: we could compress the chars to 7 bits to increase - entropy */ - for(i = 0;i < len;i++) { - keybuf[i] = key[i]; - } - assert(bs->encrypted); - - qcrypto_cipher_free(s->cipher); - s->cipher = qcrypto_cipher_new( - QCRYPTO_CIPHER_ALG_AES_128, - QCRYPTO_CIPHER_MODE_CBC, - keybuf, G_N_ELEMENTS(keybuf), - &err); - - if (!s->cipher) { - /* XXX would be nice if errors in this method could - * be properly propagate to the caller. Would need - * the bdrv_set_key() API signature to be fixed. */ - error_free(err); - return -1; - } - return 0; -} - -/* The crypt function is compatible with the linux cryptoloop - algorithm for < 4 GB images. */ -static int encrypt_sectors(BDRVQcowState *s, int64_t sector_num, - uint8_t *buf, int nb_sectors, bool enc, - Error **errp) -{ - union { - uint64_t ll[2]; - uint8_t b[16]; - } ivec; - int i; - int ret; - - for(i = 0; i < nb_sectors; i++) { - ivec.ll[0] = cpu_to_le64(sector_num); - ivec.ll[1] = 0; - if (qcrypto_cipher_setiv(s->cipher, - ivec.b, G_N_ELEMENTS(ivec.b), - errp) < 0) { - return -1; - } - if (enc) { - ret = qcrypto_cipher_encrypt(s->cipher, - buf, buf, - 512, - errp); - } else { - ret = qcrypto_cipher_decrypt(s->cipher, - buf, buf, - 512, - errp); - } - if (ret < 0) { - return -1; - } - sector_num++; - buf += 512; - } - return 0; -} /* 'allocate' is: * @@ -XXX,XX +XXX,XX @@ static uint64_t get_cluster_offset(BlockDriverState *bs, if (bs->encrypted && (n_end - n_start) < s->cluster_sectors) { uint64_t start_sect; - assert(s->cipher); + assert(s->crypto); start_sect = (offset & ~(s->cluster_size - 1)) >> 9; for(i = 0; i < s->cluster_sectors; i++) { if (i < n_start || i >= n_end) { Error *err = NULL; memset(s->cluster_data, 0x00, 512); - if (encrypt_sectors(s, start_sect + i, - s->cluster_data, 1, - true, &err) < 0) { + if (qcrypto_block_encrypt(s->crypto, start_sect + i, + s->cluster_data, + BDRV_SECTOR_SIZE, + &err) < 0) { error_free(err); errno = EIO; return -1; @@ -XXX,XX +XXX,XX @@ static int64_t coroutine_fn qcow_co_get_block_status(BlockDriverState *bs, if (!cluster_offset) { return 0; } - if ((cluster_offset & QCOW_OFLAG_COMPRESSED) || s->cipher) { + if ((cluster_offset & QCOW_OFLAG_COMPRESSED) || s->crypto) { return BDRV_BLOCK_DATA; } cluster_offset |= (index_in_cluster << BDRV_SECTOR_BITS); @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_readv(BlockDriverState *bs, int64_t sector_num, break; } if (bs->encrypted) { - assert(s->cipher); - if (encrypt_sectors(s, sector_num, buf, - n, false, &err) < 0) { + assert(s->crypto); + if (qcrypto_block_decrypt(s->crypto, sector_num, buf, + n * BDRV_SECTOR_SIZE, &err) < 0) { goto fail; } } @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_writev(BlockDriverState *bs, int64_t sector_num, } if (bs->encrypted) { Error *err = NULL; - assert(s->cipher); - if (encrypt_sectors(s, sector_num, buf, n, true, &err) < 0) { + assert(s->crypto); + if (qcrypto_block_encrypt(s->crypto, sector_num, buf, + n * BDRV_SECTOR_SIZE, &err) < 0) { error_free(err); ret = -EIO; break; @@ -XXX,XX +XXX,XX @@ static void qcow_close(BlockDriverState *bs) { BDRVQcowState *s = bs->opaque; - qcrypto_cipher_free(s->cipher); - s->cipher = NULL; + qcrypto_block_free(s->crypto); + s->crypto = NULL; g_free(s->l1_table); qemu_vfree(s->l2_cache); g_free(s->cluster_cache); @@ -XXX,XX +XXX,XX @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp) int ret; BlockBackend *qcow_blk; const char *encryptfmt = NULL; + QDict *options; + QDict *encryptopts = NULL; + QCryptoBlockCreateOptions *crypto_opts = NULL; + QCryptoBlock *crypto = NULL; /* Read out options */ total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0), @@ -XXX,XX +XXX,XX @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp) l1_size = (total_size + (1LL << shift) - 1) >> shift; header.l1_table_offset = cpu_to_be64(header_size); + + options = qemu_opts_to_qdict(opts, NULL); + qdict_extract_subqdict(options, &encryptopts, "encrypt."); + QDECREF(options); if (encryptfmt) { if (!g_str_equal(encryptfmt, "aes")) { error_setg(errp, "Unknown encryption format '%s', expected 'aes'", @@ -XXX,XX +XXX,XX @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp) goto exit; } header.crypt_method = cpu_to_be32(QCOW_CRYPT_AES); + + crypto_opts = block_crypto_create_opts_init( + Q_CRYPTO_BLOCK_FORMAT_QCOW, encryptopts, errp); + if (!crypto_opts) { + ret = -EINVAL; + goto exit; + } + + crypto = qcrypto_block_create(crypto_opts, NULL, NULL, NULL, errp); + if (!crypto) { + ret = -EINVAL; + goto exit; + } } else { header.crypt_method = cpu_to_be32(QCOW_CRYPT_NONE); } @@ -XXX,XX +XXX,XX @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp) exit: blk_unref(qcow_blk); cleanup: + QDECREF(encryptopts); + qcrypto_block_free(crypto); + qapi_free_QCryptoBlockCreateOptions(crypto_opts); g_free(backing_file); return ret; } @@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow_create_opts = { .type = QEMU_OPT_STRING, .help = "Encrypt the image, format choices: 'aes'", }, + BLOCK_CRYPTO_OPT_DEF_QCOW_KEY_SECRET("encrypt."), { /* end of list */ } } }; @@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_qcow = { .bdrv_co_writev = qcow_co_writev, .bdrv_co_get_block_status = qcow_co_get_block_status, - .bdrv_set_key = qcow_set_key, .bdrv_make_empty = qcow_make_empty, .bdrv_co_pwritev_compressed = qcow_co_pwritev_compressed, .bdrv_get_info = qcow_get_info, diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ 'mode': 'Qcow2OverlapCheckMode' } } ## +# @BlockdevQcowEncryptionFormat: +# +# @aes: AES-CBC with plain64 initialization vectors +# +# Since: 2.10 +## +{ 'enum': 'BlockdevQcowEncryptionFormat', + 'data': [ 'aes' ] } + +## +# @BlockdevQcowEncryption: +# +# Since: 2.10 +## +{ 'union': 'BlockdevQcowEncryption', + 'base': { 'format': 'BlockdevQcowEncryptionFormat' }, + 'discriminator': 'format', + 'data': { 'aes': 'QCryptoBlockOptionsQCow' } } + +## +# @BlockdevOptionsQcow: +# +# Driver specific block device options for qcow. +# +# @encrypt: Image decryption options. Mandatory for +# encrypted images, except when doing a metadata-only +# probe of the image. +# +# Since: 2.10 +## +{ 'struct': 'BlockdevOptionsQcow', + 'base': 'BlockdevOptionsGenericCOWFormat', + 'data': { '*encrypt': 'BlockdevQcowEncryption' } } + + +## # @BlockdevOptionsQcow2: # # Driver specific block device options for qcow2. @@ -XXX,XX +XXX,XX @@ 'null-co': 'BlockdevOptionsNull', 'parallels': 'BlockdevOptionsGenericFormat', 'qcow2': 'BlockdevOptionsQcow2', - 'qcow': 'BlockdevOptionsGenericCOWFormat', + 'qcow': 'BlockdevOptionsQcow', 'qed': 'BlockdevOptionsGenericCOWFormat', 'quorum': 'BlockdevOptionsQuorum', 'raw': 'BlockdevOptionsRaw', -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Instead of requiring separate input/output buffers for encrypting data, change qcow2_encrypt_sectors() to assume use of a single buffer, encrypting in place. The current callers all used the same buffer for input/output already. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-11-berrange@redhat.com Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-cluster.c | 17 ++++++----------- block/qcow2.c | 4 ++-- block/qcow2.h | 3 +-- 3 files changed, 9 insertions(+), 15 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ static int count_contiguous_clusters_unallocated(int nb_clusters, } /* The crypt function is compatible with the linux cryptoloop - algorithm for < 4 GB images. NOTE: out_buf == in_buf is - supported */ + algorithm for < 4 GB images. */ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, - uint8_t *out_buf, const uint8_t *in_buf, - int nb_sectors, bool enc, + uint8_t *buf, int nb_sectors, bool enc, Error **errp) { union { @@ -XXX,XX +XXX,XX @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, } if (enc) { ret = qcrypto_cipher_encrypt(s->cipher, - in_buf, - out_buf, + buf, buf, 512, errp); } else { ret = qcrypto_cipher_decrypt(s->cipher, - in_buf, - out_buf, + buf, buf, 512, errp); } @@ -XXX,XX +XXX,XX @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, return -1; } sector_num++; - in_buf += 512; - out_buf += 512; + buf += 512; } return 0; } @@ -XXX,XX +XXX,XX @@ static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs, assert(s->cipher); assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0); assert((bytes & ~BDRV_SECTOR_MASK) == 0); - if (qcow2_encrypt_sectors(s, sector, buffer, buffer, + if (qcow2_encrypt_sectors(s, sector, buffer, bytes >> BDRV_SECTOR_BITS, true, NULL) < 0) { return false; } diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0); Error *err = NULL; if (qcow2_encrypt_sectors(s, offset >> BDRV_SECTOR_BITS, - cluster_data, cluster_data, + cluster_data, cur_bytes >> BDRV_SECTOR_BITS, false, &err) < 0) { error_free(err); @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, qemu_iovec_to_buf(&hd_qiov, 0, cluster_data, hd_qiov.size); if (qcow2_encrypt_sectors(s, offset >> BDRV_SECTOR_BITS, - cluster_data, cluster_data, + cluster_data, cur_bytes >>BDRV_SECTOR_BITS, true, &err) < 0) { error_free(err); diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size, int qcow2_write_l1_entry(BlockDriverState *bs, int l1_index); int qcow2_decompress_cluster(BlockDriverState *bs, uint64_t cluster_offset); int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, - uint8_t *out_buf, const uint8_t *in_buf, - int nb_sectors, bool enc, Error **errp); + uint8_t *buf, int nb_sectors, bool enc, Error **errp); int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *cluster_offset); -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> This converts the qcow2 driver to make use of the QCryptoBlock APIs for encrypting image content, using the legacy QCow2 AES scheme. With this change it is now required to use the QCryptoSecret object for providing passwords, instead of the current block password APIs / interactive prompting. $QEMU \ -object secret,id=sec0,file=/home/berrange/encrypted.pw \ -drive file=/home/berrange/encrypted.qcow2,encrypt.key-secret=sec0 The test 087 could be simplified since there is no longer a difference in behaviour when using blockdev_add with encrypted images for the running vs stopped CPU state. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-12-berrange@redhat.com Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-cluster.c | 47 +--------- block/qcow2.c | 226 ++++++++++++++++++++++++++++++--------------- block/qcow2.h | 5 +- qapi/block-core.json | 27 +++++- tests/qemu-iotests/049 | 2 +- tests/qemu-iotests/049.out | 4 +- tests/qemu-iotests/082.out | 27 ++++++ tests/qemu-iotests/087 | 28 +++--- tests/qemu-iotests/087.out | 12 +-- tests/qemu-iotests/134 | 18 +++- tests/qemu-iotests/134.out | 10 +- tests/qemu-iotests/158 | 19 ++-- tests/qemu-iotests/158.out | 14 +-- tests/qemu-iotests/common | 10 +- 14 files changed, 263 insertions(+), 186 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ static int count_contiguous_clusters_unallocated(int nb_clusters, return i; } -/* The crypt function is compatible with the linux cryptoloop - algorithm for < 4 GB images. */ -int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, - uint8_t *buf, int nb_sectors, bool enc, - Error **errp) -{ - union { - uint64_t ll[2]; - uint8_t b[16]; - } ivec; - int i; - int ret; - - for(i = 0; i < nb_sectors; i++) { - ivec.ll[0] = cpu_to_le64(sector_num); - ivec.ll[1] = 0; - if (qcrypto_cipher_setiv(s->cipher, - ivec.b, G_N_ELEMENTS(ivec.b), - errp) < 0) { - return -1; - } - if (enc) { - ret = qcrypto_cipher_encrypt(s->cipher, - buf, buf, - 512, - errp); - } else { - ret = qcrypto_cipher_decrypt(s->cipher, - buf, buf, - 512, - errp); - } - if (ret < 0) { - return -1; - } - sector_num++; - buf += 512; - } - return 0; -} - static int coroutine_fn do_perform_cow_read(BlockDriverState *bs, uint64_t src_cluster_offset, unsigned offset_in_cluster, @@ -XXX,XX +XXX,XX @@ static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs, BDRVQcow2State *s = bs->opaque; int64_t sector = (src_cluster_offset + offset_in_cluster) >> BDRV_SECTOR_BITS; - assert(s->cipher); assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0); assert((bytes & ~BDRV_SECTOR_MASK) == 0); - if (qcow2_encrypt_sectors(s, sector, buffer, - bytes >> BDRV_SECTOR_BITS, true, NULL) < 0) { + assert(s->crypto); + if (qcrypto_block_encrypt(s->crypto, sector, buffer, + bytes, NULL) < 0) { return false; } } diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ #include "qemu/option_int.h" #include "qemu/cutils.h" #include "qemu/bswap.h" +#include "qapi/opts-visitor.h" +#include "qapi-visit.h" +#include "block/crypto.h" /* Differences with QCOW: @@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_runtime_opts = { .type = QEMU_OPT_NUMBER, .help = "Clean unused cache entries after this time (in seconds)", }, + BLOCK_CRYPTO_OPT_DEF_QCOW_KEY_SECRET("encrypt."), { /* end of list */ } }, }; @@ -XXX,XX +XXX,XX @@ typedef struct Qcow2ReopenState { int overlap_check; bool discard_passthrough[QCOW2_DISCARD_MAX]; uint64_t cache_clean_interval; + QCryptoBlockOpenOptions *crypto_opts; /* Disk encryption runtime options */ } Qcow2ReopenState; static int qcow2_update_options_prepare(BlockDriverState *bs, @@ -XXX,XX +XXX,XX @@ static int qcow2_update_options_prepare(BlockDriverState *bs, int overlap_check_template = 0; uint64_t l2_cache_size, refcount_cache_size; int i; + const char *encryptfmt; + QDict *encryptopts = NULL; Error *local_err = NULL; int ret; + qdict_extract_subqdict(options, &encryptopts, "encrypt."); + encryptfmt = qdict_get_try_str(encryptopts, "format"); + opts = qemu_opts_create(&qcow2_runtime_opts, NULL, 0, &error_abort); qemu_opts_absorb_qdict(opts, options, &local_err); if (local_err) { @@ -XXX,XX +XXX,XX @@ static int qcow2_update_options_prepare(BlockDriverState *bs, r->discard_passthrough[QCOW2_DISCARD_OTHER] = qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_OTHER, false); + switch (s->crypt_method_header) { + case QCOW_CRYPT_NONE: + if (encryptfmt) { + error_setg(errp, "No encryption in image header, but options " + "specified format '%s'", encryptfmt); + ret = -EINVAL; + goto fail; + } + break; + + case QCOW_CRYPT_AES: + if (encryptfmt && !g_str_equal(encryptfmt, "aes")) { + error_setg(errp, + "Header reported 'aes' encryption format but " + "options specify '%s'", encryptfmt); + ret = -EINVAL; + goto fail; + } + qdict_del(encryptopts, "format"); + r->crypto_opts = block_crypto_open_opts_init( + Q_CRYPTO_BLOCK_FORMAT_QCOW, encryptopts, errp); + break; + + default: + error_setg(errp, "Unsupported encryption method %d", + s->crypt_method_header); + break; + } + if (s->crypt_method_header != QCOW_CRYPT_NONE && !r->crypto_opts) { + ret = -EINVAL; + goto fail; + } + ret = 0; fail: + QDECREF(encryptopts); qemu_opts_del(opts); opts = NULL; return ret; @@ -XXX,XX +XXX,XX @@ static void qcow2_update_options_commit(BlockDriverState *bs, s->cache_clean_interval = r->cache_clean_interval; cache_clean_timer_init(bs, bdrv_get_aio_context(bs)); } + + qapi_free_QCryptoBlockOpenOptions(s->crypto_opts); + s->crypto_opts = r->crypto_opts; } static void qcow2_update_options_abort(BlockDriverState *bs, @@ -XXX,XX +XXX,XX @@ static void qcow2_update_options_abort(BlockDriverState *bs, if (r->refcount_block_cache) { qcow2_cache_destroy(bs, r->refcount_block_cache); } + qapi_free_QCryptoBlockOpenOptions(r->crypto_opts); } static int qcow2_update_options(BlockDriverState *bs, QDict *options, @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, ret = -EINVAL; goto fail; } - if (!qcrypto_cipher_supports(QCRYPTO_CIPHER_ALG_AES_128, - QCRYPTO_CIPHER_MODE_CBC)) { - error_setg(errp, "AES cipher not available"); - ret = -EINVAL; - goto fail; - } s->crypt_method_header = header.crypt_method; if (s->crypt_method_header) { if (bdrv_uses_whitelist() && @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, } bs->encrypted = true; + bs->valid_key = true; } s->l2_bits = s->cluster_bits - 3; /* L2 is always one cluster */ @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, goto fail; } + if (s->crypt_method_header == QCOW_CRYPT_AES) { + unsigned int cflags = 0; + if (flags & BDRV_O_NO_IO) { + cflags |= QCRYPTO_BLOCK_OPEN_NO_IO; + } + s->crypto = qcrypto_block_open(s->crypto_opts, NULL, NULL, + cflags, errp); + if (!s->crypto) { + ret = -EINVAL; + goto fail; + } + } + /* read the backing file name */ if (header.backing_file_offset != 0) { len = header.backing_file_size; @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, } g_free(s->cluster_cache); qemu_vfree(s->cluster_data); + qcrypto_block_free(s->crypto); + qapi_free_QCryptoBlockOpenOptions(s->crypto_opts); return ret; } @@ -XXX,XX +XXX,XX @@ static void qcow2_refresh_limits(BlockDriverState *bs, Error **errp) bs->bl.pdiscard_alignment = s->cluster_size; } -static int qcow2_set_key(BlockDriverState *bs, const char *key) -{ - BDRVQcow2State *s = bs->opaque; - uint8_t keybuf[16]; - int len, i; - Error *err = NULL; - - memset(keybuf, 0, 16); - len = strlen(key); - if (len > 16) - len = 16; - /* XXX: we could compress the chars to 7 bits to increase - entropy */ - for(i = 0;i < len;i++) { - keybuf[i] = key[i]; - } - assert(bs->encrypted); - - qcrypto_cipher_free(s->cipher); - s->cipher = qcrypto_cipher_new( - QCRYPTO_CIPHER_ALG_AES_128, - QCRYPTO_CIPHER_MODE_CBC, - keybuf, G_N_ELEMENTS(keybuf), - &err); - - if (!s->cipher) { - /* XXX would be nice if errors in this method could - * be properly propagate to the caller. Would need - * the bdrv_set_key() API signature to be fixed. */ - error_free(err); - return -1; - } - return 0; -} - static int qcow2_reopen_prepare(BDRVReopenState *state, BlockReopenQueue *queue, Error **errp) { @@ -XXX,XX +XXX,XX @@ static int64_t coroutine_fn qcow2_co_get_block_status(BlockDriverState *bs, *pnum = bytes >> BDRV_SECTOR_BITS; if (cluster_offset != 0 && ret != QCOW2_CLUSTER_COMPRESSED && - !s->cipher) { + !s->crypto) { index_in_cluster = sector_num & (s->cluster_sectors - 1); cluster_offset |= (index_in_cluster << BDRV_SECTOR_BITS); *file = bs->file->bs; @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, /* prepare next request */ cur_bytes = MIN(bytes, INT_MAX); - if (s->cipher) { + if (s->crypto) { cur_bytes = MIN(cur_bytes, QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); } @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, } if (bs->encrypted) { - assert(s->cipher); + assert(s->crypto); /* * For encrypted images, read everything into a temporary @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, goto fail; } if (bs->encrypted) { - assert(s->cipher); + assert(s->crypto); assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0); assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0); Error *err = NULL; - if (qcow2_encrypt_sectors(s, offset >> BDRV_SECTOR_BITS, + if (qcrypto_block_decrypt(s->crypto, + offset >> BDRV_SECTOR_BITS, cluster_data, - cur_bytes >> BDRV_SECTOR_BITS, - false, &err) < 0) { + cur_bytes, + &err) < 0) { error_free(err); ret = -EIO; goto fail; @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, if (bs->encrypted) { Error *err = NULL; - assert(s->cipher); + assert(s->crypto); if (!cluster_data) { cluster_data = qemu_try_blockalign(bs->file->bs, QCOW_MAX_CRYPT_CLUSTERS @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); qemu_iovec_to_buf(&hd_qiov, 0, cluster_data, hd_qiov.size); - if (qcow2_encrypt_sectors(s, offset >> BDRV_SECTOR_BITS, + if (qcrypto_block_encrypt(s->crypto, offset >> BDRV_SECTOR_BITS, cluster_data, - cur_bytes >>BDRV_SECTOR_BITS, - true, &err) < 0) { + cur_bytes, &err) < 0) { error_free(err); ret = -EIO; goto fail; @@ -XXX,XX +XXX,XX @@ static void qcow2_close(BlockDriverState *bs) qcow2_cache_destroy(bs, s->l2_table_cache); qcow2_cache_destroy(bs, s->refcount_block_cache); - qcrypto_cipher_free(s->cipher); - s->cipher = NULL; + qcrypto_block_free(s->crypto); + s->crypto = NULL; g_free(s->unknown_header_fields); cleanup_unknown_header_ext(bs); @@ -XXX,XX +XXX,XX @@ static void qcow2_invalidate_cache(BlockDriverState *bs, Error **errp) { BDRVQcow2State *s = bs->opaque; int flags = s->flags; - QCryptoCipher *cipher = NULL; + QCryptoBlock *crypto = NULL; QDict *options; Error *local_err = NULL; int ret; @@ -XXX,XX +XXX,XX @@ static void qcow2_invalidate_cache(BlockDriverState *bs, Error **errp) * that means we don't have to worry about reopening them here. */ - cipher = s->cipher; - s->cipher = NULL; + crypto = s->crypto; + s->crypto = NULL; qcow2_close(bs); @@ -XXX,XX +XXX,XX @@ static void qcow2_invalidate_cache(BlockDriverState *bs, Error **errp) return; } - s->cipher = cipher; + s->crypto = crypto; } static size_t header_ext_add(char *buf, uint32_t magic, const void *s, @@ -XXX,XX +XXX,XX @@ static int qcow2_change_backing_file(BlockDriverState *bs, return qcow2_update_header(bs); } + +static int qcow2_set_up_encryption(BlockDriverState *bs, const char *encryptfmt, + QemuOpts *opts, Error **errp) +{ + BDRVQcow2State *s = bs->opaque; + QCryptoBlockCreateOptions *cryptoopts = NULL; + QCryptoBlock *crypto = NULL; + int ret = -EINVAL; + QDict *options, *encryptopts; + + options = qemu_opts_to_qdict(opts, NULL); + qdict_extract_subqdict(options, &encryptopts, "encrypt."); + QDECREF(options); + + if (!g_str_equal(encryptfmt, "aes")) { + error_setg(errp, "Unknown encryption format '%s', expected 'aes'", + encryptfmt); + ret = -EINVAL; + goto out; + } + cryptoopts = block_crypto_create_opts_init( + Q_CRYPTO_BLOCK_FORMAT_QCOW, encryptopts, errp); + if (!cryptoopts) { + ret = -EINVAL; + goto out; + } + s->crypt_method_header = QCOW_CRYPT_AES; + + crypto = qcrypto_block_create(cryptoopts, + NULL, NULL, + bs, errp); + if (!crypto) { + ret = -EINVAL; + goto out; + } + + ret = qcow2_update_header(bs); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not write encryption header"); + goto out; + } + + out: + QDECREF(encryptopts); + qcrypto_block_free(crypto); + qapi_free_QCryptoBlockCreateOptions(cryptoopts); + return ret; +} + + static int preallocate(BlockDriverState *bs) { uint64_t bytes; @@ -XXX,XX +XXX,XX @@ static int qcow2_create2(const char *filename, int64_t total_size, .header_length = cpu_to_be32(sizeof(*header)), }; - if (encryptfmt) { - if (!g_str_equal(encryptfmt, "aes")) { - error_setg(errp, "Unknown encryption format '%s', expected 'aes'", - encryptfmt); - ret = -EINVAL; - goto out; - } - header->crypt_method = cpu_to_be32(QCOW_CRYPT_AES); - } else { - header->crypt_method = cpu_to_be32(QCOW_CRYPT_NONE); - } + /* We'll update this to correct value later */ + header->crypt_method = cpu_to_be32(QCOW_CRYPT_NONE); if (flags & BLOCK_FLAG_LAZY_REFCOUNTS) { header->compatible_features |= @@ -XXX,XX +XXX,XX @@ static int qcow2_create2(const char *filename, int64_t total_size, } } + /* Want encryption? There you go. */ + if (encryptfmt) { + ret = qcow2_set_up_encryption(blk_bs(blk), encryptfmt, opts, errp); + if (ret < 0) { + goto out; + } + } + /* And if we're supposed to preallocate metadata, do that now */ if (prealloc != PREALLOC_MODE_OFF) { BDRVQcow2State *s = blk_bs(blk)->opaque; @@ -XXX,XX +XXX,XX @@ static int qcow2_create2(const char *filename, int64_t total_size, blk_unref(blk); blk = NULL; - /* Reopen the image without BDRV_O_NO_FLUSH to flush it before returning */ + /* Reopen the image without BDRV_O_NO_FLUSH to flush it before returning. + * Using BDRV_O_NO_IO, since encryption is now setup we don't want to + * have to setup decryption context. We're not doing any I/O on the top + * level BlockDriverState, only lower layers, where BDRV_O_NO_IO does + * not have effect. + */ options = qdict_new(); qdict_put_str(options, "driver", "qcow2"); blk = blk_new_open(filename, NULL, options, - BDRV_O_RDWR | BDRV_O_NO_BACKING, &local_err); + BDRV_O_RDWR | BDRV_O_NO_BACKING | BDRV_O_NO_IO, + &local_err); if (blk == NULL) { error_propagate(errp, local_err); ret = -EIO; @@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, backing_format = qemu_opt_get(opts, BLOCK_OPT_BACKING_FMT); } else if (!strcmp(desc->name, BLOCK_OPT_ENCRYPT)) { encrypt = qemu_opt_get_bool(opts, BLOCK_OPT_ENCRYPT, - !!s->cipher); + !!s->crypto); - if (encrypt != !!s->cipher) { + if (encrypt != !!s->crypto) { error_report("Changing the encryption flag is not supported"); return -ENOTSUP; } @@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_create_opts = { .type = QEMU_OPT_STRING, .help = "Encrypt the image, format choices: 'aes'", }, + BLOCK_CRYPTO_OPT_DEF_QCOW_KEY_SECRET("encrypt."), { .name = BLOCK_OPT_CLUSTER_SIZE, .type = QEMU_OPT_SIZE, @@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = { .bdrv_create = qcow2_create, .bdrv_has_zero_init = bdrv_has_zero_init_1, .bdrv_co_get_block_status = qcow2_co_get_block_status, - .bdrv_set_key = qcow2_set_key, .bdrv_co_preadv = qcow2_co_preadv, .bdrv_co_pwritev = qcow2_co_pwritev, diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ #ifndef BLOCK_QCOW2_H #define BLOCK_QCOW2_H -#include "crypto/cipher.h" +#include "crypto/block.h" #include "qemu/coroutine.h" //#define DEBUG_ALLOC @@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcow2State { CoMutex lock; - QCryptoCipher *cipher; /* current cipher, NULL if no key yet */ + QCryptoBlockOpenOptions *crypto_opts; /* Disk encryption runtime options */ + QCryptoBlock *crypto; /* Disk encryption format driver */ uint32_t crypt_method_header; uint64_t snapshots_offset; int snapshots_size; diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ 'data': { '*encrypt': 'BlockdevQcowEncryption' } } + +## +# @BlockdevQcow2EncryptionFormat: +# @aes: AES-CBC with plain64 initialization venctors +# +# Since: 2.10 +## +{ 'enum': 'BlockdevQcow2EncryptionFormat', + 'data': [ 'aes' ] } + +## +# @BlockdevQcow2Encryption: +# +# Since: 2.10 +## +{ 'union': 'BlockdevQcow2Encryption', + 'base': { 'format': 'BlockdevQcow2EncryptionFormat' }, + 'discriminator': 'format', + 'data': { 'aes': 'QCryptoBlockOptionsQCow' } } + ## # @BlockdevOptionsQcow2: # @@ -XXX,XX +XXX,XX @@ # @cache-clean-interval: clean unused entries in the L2 and refcount # caches. The interval is in seconds. The default value # is 0 and it disables this feature (since 2.5) +# @encrypt: Image decryption options. Mandatory for +# encrypted images, except when doing a metadata-only +# probe of the image. (since 2.10) # # Since: 2.9 ## @@ -XXX,XX +XXX,XX @@ '*cache-size': 'int', '*l2-cache-size': 'int', '*refcount-cache-size': 'int', - '*cache-clean-interval': 'int' } } - + '*cache-clean-interval': 'int', + '*encrypt': 'BlockdevQcow2Encryption' } } ## # @BlockdevOptionsSsh: diff --git a/tests/qemu-iotests/049 b/tests/qemu-iotests/049 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/049 +++ b/tests/qemu-iotests/049 @@ -XXX,XX +XXX,XX @@ test_qemu_img create -f $IMGFMT -o preallocation=1234 "$TEST_IMG" 64M echo "== Check encryption option ==" echo test_qemu_img create -f $IMGFMT -o encryption=off "$TEST_IMG" 64M -test_qemu_img create -f $IMGFMT -o encryption=on "$TEST_IMG" 64M +test_qemu_img create -f $IMGFMT --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 "$TEST_IMG" 64M echo "== Check lazy_refcounts option (only with v3) ==" echo diff --git a/tests/qemu-iotests/049.out b/tests/qemu-iotests/049.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/049.out +++ b/tests/qemu-iotests/049.out @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preall qemu-img create -f qcow2 -o encryption=off TEST_DIR/t.qcow2 64M Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 -qemu-img create -f qcow2 -o encryption=on TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=on cluster_size=65536 lazy_refcounts=off refcount_bits=16 +qemu-img create -f qcow2 --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 TEST_DIR/t.qcow2 64M +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=on encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off refcount_bits=16 == Check lazy_refcounts option (only with v3) == diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) encrypt.format Encrypt the image, format choices: 'aes' +encrypt.key-secret ID of the secret that provides the AES encryption key cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/087 +++ b/tests/qemu-iotests/087 @@ -XXX,XX +XXX,XX @@ echo echo === Encrypted image === echo -_make_test_img -o encryption=on $size -run_qemu -S <<EOF +_make_test_img --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 $size +run_qemu <<EOF { "execute": "qmp_capabilities" } -{ "execute": "blockdev-add", +{ "execute": "object-add", "arguments": { - "driver": "$IMGFMT", - "node-name": "disk", - "file": { - "driver": "file", - "filename": "$TEST_IMG" + "qom-type": "secret", + "id": "sec0", + "props": { + "data": "123456" } - } } -{ "execute": "quit" } -EOF - -run_qemu <<EOF -{ "execute": "qmp_capabilities" } +} { "execute": "blockdev-add", "arguments": { "driver": "$IMGFMT", @@ -XXX,XX +XXX,XX @@ run_qemu <<EOF "file": { "driver": "file", "filename": "$TEST_IMG" + }, + "encrypt": { + "format": "aes", + "key-secret": "sec0" } } } @@ -XXX,XX +XXX,XX @@ echo echo === Missing driver === echo -_make_test_img -o encryption=on $size +_make_test_img --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 $size run_qemu -S <<EOF { "execute": "qmp_capabilities" } { "execute": "blockdev-add", diff --git a/tests/qemu-iotests/087.out b/tests/qemu-iotests/087.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/087.out +++ b/tests/qemu-iotests/087.out @@ -XXX,XX +XXX,XX @@ QMP_VERSION === Encrypted image === -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on -Testing: -S -QMP_VERSION -{"return": {}} -{"error": {"class": "GenericError", "desc": "Use of AES-CBC encrypted IMGFMT images is no longer supported in system emulators"}} -{"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} - +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 Testing: QMP_VERSION {"return": {}} +{"return": {}} {"error": {"class": "GenericError", "desc": "Use of AES-CBC encrypted IMGFMT images is no longer supported in system emulators"}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} @@ -XXX,XX +XXX,XX @@ QMP_VERSION === Missing driver === -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 Testing: -S QMP_VERSION {"return": {}} diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/134 +++ b/tests/qemu-iotests/134 @@ -XXX,XX +XXX,XX @@ _supported_os Linux size=128M -IMGOPTS="encryption=on" _make_test_img $size + +SECRET="secret,id=sec0,data=astrochicken" +SECRETALT="secret,id=sec0,data=platypus" + +_make_test_img --object $SECRET -o "encryption=on,encrypt.key-secret=sec0" $size + +IMGSPEC="driver=$IMGFMT,file.filename=$TEST_IMG,encrypt.key-secret=sec0" + +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT echo echo "== reading whole image ==" -echo "astrochicken" | $QEMU_IO -c "read 0 $size" "$TEST_IMG" | _filter_qemu_io | _filter_testdir +$QEMU_IO --object $SECRET -c "read 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir echo echo "== rewriting whole image ==" -echo "astrochicken" | $QEMU_IO -c "write -P 0xa 0 $size" "$TEST_IMG" | _filter_qemu_io | _filter_testdir +$QEMU_IO --object $SECRET -c "write -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir echo echo "== verify pattern ==" -echo "astrochicken" | $QEMU_IO -c "read -P 0xa 0 $size" "$TEST_IMG" | _filter_qemu_io | _filter_testdir +$QEMU_IO --object $SECRET -c "read -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir echo echo "== verify pattern failure with wrong password ==" -echo "platypus" | $QEMU_IO -c "read -P 0xa 0 $size" "$TEST_IMG" | _filter_qemu_io | _filter_testdir +$QEMU_IO --object $SECRETALT -c "read -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir # success, all done diff --git a/tests/qemu-iotests/134.out b/tests/qemu-iotests/134.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/134.out +++ b/tests/qemu-iotests/134.out @@ -XXX,XX +XXX,XX @@ QA output created by 134 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 == reading whole image == -Disk image 'TEST_DIR/t.qcow2' is encrypted. -password: read 134217728/134217728 bytes at offset 0 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == rewriting whole image == -Disk image 'TEST_DIR/t.qcow2' is encrypted. -password: wrote 134217728/134217728 bytes at offset 0 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == verify pattern == -Disk image 'TEST_DIR/t.qcow2' is encrypted. -password: read 134217728/134217728 bytes at offset 0 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == verify pattern failure with wrong password == -Disk image 'TEST_DIR/t.qcow2' is encrypted. -password: Pattern verification failed at offset 0, 134217728 bytes read 134217728/134217728 bytes at offset 0 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/158 b/tests/qemu-iotests/158 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/158 +++ b/tests/qemu-iotests/158 @@ -XXX,XX +XXX,XX @@ _supported_os Linux size=128M TEST_IMG_BASE=$TEST_IMG.base +SECRET="secret,id=sec0,data=astrochicken" TEST_IMG_SAVE=$TEST_IMG TEST_IMG=$TEST_IMG_BASE echo "== create base ==" -IMGOPTS="encryption=on" _make_test_img $size +_make_test_img --object $SECRET -o "encryption=on,encrypt.key-secret=sec0" $size TEST_IMG=$TEST_IMG_SAVE +IMGSPECBASE="driver=$IMGFMT,file.filename=$TEST_IMG_BASE,encrypt.key-secret=sec0" +IMGSPEC="driver=$IMGFMT,file.filename=$TEST_IMG,backing.driver=$IMGFMT,backing.file.filename=$TEST_IMG_BASE,backing.encrypt.key-secret=sec0,encrypt.key-secret=sec0" +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT + echo echo "== writing whole image ==" -echo "astrochicken" | $QEMU_IO -c "write -P 0xa 0 $size" "$TEST_IMG_BASE" | _filter_qemu_io | _filter_testdir +$QEMU_IO --object $SECRET -c "write -P 0xa 0 $size" --image-opts $IMGSPECBASE | _filter_qemu_io | _filter_testdir echo echo "== verify pattern ==" -echo "astrochicken" | $QEMU_IO -c "read -P 0xa 0 $size" "$TEST_IMG_BASE" | _filter_qemu_io | _filter_testdir +$QEMU_IO --object $SECRET -c "read -P 0xa 0 $size" --image-opts $IMGSPECBASE | _filter_qemu_io | _filter_testdir echo "== create overlay ==" -IMGOPTS="encryption=on" _make_test_img -b "$TEST_IMG_BASE" $size +_make_test_img --object $SECRET -o "encryption=on,encrypt.key-secret=sec0" -b "$TEST_IMG_BASE" $size echo echo "== writing part of a cluster ==" -echo "astrochicken" | $QEMU_IO -c "write -P 0xe 0 1024" "$TEST_IMG" | _filter_qemu_io | _filter_testdir +$QEMU_IO --object $SECRET -c "write -P 0xe 0 1024" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir echo echo "== verify pattern ==" -echo "astrochicken" | $QEMU_IO -c "read -P 0xe 0 1024" "$TEST_IMG" | _filter_qemu_io | _filter_testdir +$QEMU_IO --object $SECRET -c "read -P 0xe 0 1024" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir echo echo "== verify pattern ==" -echo "astrochicken" | $QEMU_IO -c "read -P 0xa 1024 64512" "$TEST_IMG" | _filter_qemu_io | _filter_testdir +$QEMU_IO --object $SECRET -c "read -P 0xa 1024 64512" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir # success, all done diff --git a/tests/qemu-iotests/158.out b/tests/qemu-iotests/158.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/158.out +++ b/tests/qemu-iotests/158.out @@ -XXX,XX +XXX,XX @@ QA output created by 158 == create base == -Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 == writing whole image == -Disk image 'TEST_DIR/t.qcow2.base' is encrypted. -password: wrote 134217728/134217728 bytes at offset 0 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == verify pattern == -Disk image 'TEST_DIR/t.qcow2.base' is encrypted. -password: read 134217728/134217728 bytes at offset 0 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == create overlay == -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on encrypt.key-secret=sec0 == writing part of a cluster == -Disk image 'TEST_DIR/t.qcow2' is encrypted. -password: wrote 1024/1024 bytes at offset 0 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == verify pattern == -Disk image 'TEST_DIR/t.qcow2' is encrypted. -password: read 1024/1024 bytes at offset 0 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == verify pattern == -Disk image 'TEST_DIR/t.qcow2' is encrypted. -password: read 64512/64512 bytes at offset 1024 63 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common +++ b/tests/qemu-iotests/common @@ -XXX,XX +XXX,XX @@ export IMGPROTO=file export IMGOPTS="" export CACHEMODE="writeback" export QEMU_IO_OPTIONS="" +export QEMU_IO_OPTIONS_NO_FMT="" export CACHEMODE_IS_DEFAULT=true export QEMU_OPTIONS="-nodefaults -machine accel=qtest" export VALGRIND_QEMU= @@ -XXX,XX +XXX,XX @@ BEGIN { for (t='$start'; t<='$end'; t++) printf "%03d\n",t }' \ done # Set qemu-io cache mode with $CACHEMODE we have -if [ "$IMGOPTSSYNTAX" = "true" ]; then - QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS --cache $CACHEMODE" -else - QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS -f $IMGFMT --cache $CACHEMODE" +QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS --cache $CACHEMODE" + +QEMU_IO_OPTIONS_NO_FMT="$QEMU_IO_OPTIONS" +if [ "$IMGOPTSSYNTAX" != "true" ]; then + QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS -f $IMGFMT" fi # Set default options for qemu-img create -o if they were not specified -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Update the qcow2 specification to describe how the LUKS header is placed inside a qcow2 file, when using LUKS encryption for the qcow2 payload instead of the legacy AES-CBC encryption Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-13-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- docs/interop/qcow2.txt | 103 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index XXXXXXX..XXXXXXX 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -XXX,XX +XXX,XX @@ The first cluster of a qcow2 image contains the file header: 32 - 35: crypt_method 0 for no encryption 1 for AES encryption + 2 for LUKS encryption 36 - 39: l1_size Number of entries in the active L1 table @@ -XXX,XX +XXX,XX @@ be stored. Each extension has a structure like the following: 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension + 0x0537be77 - Full disk encryption header pointer other - Unknown header extension, can be safely ignored @@ -XXX,XX +XXX,XX @@ The fields of the bitmaps extension are: Offset into the image file at which the bitmap directory starts. Must be aligned to a cluster boundary. +== Full disk encryption header pointer == + +The full disk encryption header must be present if, and only if, the +'crypt_method' header requires metadata. Currently this is only true +of the 'LUKS' crypt method. The header extension must be absent for +other methods. + +This header provides the offset at which the crypt method can store +its additional data, as well as the length of such data. + + Byte 0 - 7: Offset into the image file at which the encryption + header starts in bytes. Must be aligned to a cluster + boundary. + Byte 8 - 15: Length of the written encryption header in bytes. + Note actual space allocated in the qcow2 file may + be larger than this value, since it will be rounded + to the nearest multiple of the cluster size. Any + unused bytes in the allocated space will be initialized + to 0. + +For the LUKS crypt method, the encryption header works as follows. + +The first 592 bytes of the header clusters will contain the LUKS +partition header. This is then followed by the key material data areas. +The size of the key material data areas is determined by the number of +stripes in the key slot and key size. Refer to the LUKS format +specification ('docs/on-disk-format.pdf' in the cryptsetup source +package) for details of the LUKS partition header format. + +In the LUKS partition header, the "payload-offset" field will be +calculated as normal for the LUKS spec. ie the size of the LUKS +header, plus key material regions, plus padding, relative to the +start of the LUKS header. This offset value is not required to be +qcow2 cluster aligned. Its value is currently never used in the +context of qcow2, since the qcow2 file format itself defines where +the real payload offset is, but none the less a valid payload offset +should always be present. + +In the LUKS key slots header, the "key-material-offset" is relative +to the start of the LUKS header clusters in the qcow2 container, +not the start of the qcow2 file. + +Logically the layout looks like + + +-----------------------------+ + | QCow2 header | + | QCow2 header extension X | + | QCow2 header extension FDE | + | QCow2 header extension ... | + | QCow2 header extension Z | + +-----------------------------+ + | ....other QCow2 tables.... | + . . + . . + +-----------------------------+ + | +-------------------------+ | + | | LUKS partition header | | + | +-------------------------+ | + | | LUKS key material 1 | | + | +-------------------------+ | + | | LUKS key material 2 | | + | +-------------------------+ | + | | LUKS key material ... | | + | +-------------------------+ | + | | LUKS key material 8 | | + | +-------------------------+ | + +-----------------------------+ + | QCow2 cluster payload | + . . + . . + . . + | | + +-----------------------------+ + +== Data encryption == + +When an encryption method is requested in the header, the image payload +data must be encrypted/decrypted on every write/read. The image headers +and metadata are never encrypted. + +The algorithms used for encryption vary depending on the method + + - AES: + + The AES cipher, in CBC mode, with 256 bit keys. + + Initialization vectors generated using plain64 method, with + the virtual disk sector as the input tweak. + + This format is no longer supported in QEMU system emulators, due + to a number of design flaws affecting its security. It is only + supported in the command line tools for the sake of back compatibility + and data liberation. + + - LUKS: + + The algorithms are specified in the LUKS header. + + Initialization vectors generated using the method specified + in the LUKS header, with the physical disk sector as the + input tweak. == Host cluster management == -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> This adds support for using LUKS as an encryption format with the qcow2 file, using the new encrypt.format parameter to request "luks" format. e.g. # qemu-img create --object secret,data=123456,id=sec0 \ -f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0 \ test.qcow2 10G The legacy "encryption=on" parameter still results in creation of the old qcow2 AES format (and is equivalent to the new 'encryption-format=aes'). e.g. the following are equivalent: # qemu-img create --object secret,data=123456,id=sec0 \ -f qcow2 -o encryption=on,encrypt.key-secret=sec0 \ test.qcow2 10G # qemu-img create --object secret,data=123456,id=sec0 \ -f qcow2 -o encryption-format=aes,encrypt.key-secret=sec0 \ test.qcow2 10G With the LUKS format it is necessary to store the LUKS partition header and key material in the QCow2 file. This data can be many MB in size, so cannot go into the QCow2 header region directly. Thus the spec defines a FDE (Full Disk Encryption) header extension that specifies the offset of a set of clusters to hold the FDE headers, as well as the length of that region. The LUKS header is thus stored in these extra allocated clusters before the main image payload. Aside from all the cryptographic differences implied by use of the LUKS format, there is one further key difference between the use of legacy AES and LUKS encryption in qcow2. For LUKS, the initialiazation vectors are generated using the host physical sector as the input, rather than the guest virtual sector. This guarantees unique initialization vectors for all sectors when qcow2 internal snapshots are used, thus giving stronger protection against watermarking attacks. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-14-berrange@redhat.com Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-cluster.c | 14 ++- block/qcow2-refcount.c | 10 ++ block/qcow2.c | 268 ++++++++++++++++++++++++++++++++++++++------ block/qcow2.h | 9 ++ qapi/block-core.json | 5 +- tests/qemu-iotests/082.out | 270 ++++++++++++++++++++++++++++++++++++--------- 6 files changed, 484 insertions(+), 92 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn do_perform_cow_read(BlockDriverState *bs, static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs, uint64_t src_cluster_offset, + uint64_t cluster_offset, unsigned offset_in_cluster, uint8_t *buffer, unsigned bytes) { if (bytes && bs->encrypted) { BDRVQcow2State *s = bs->opaque; - int64_t sector = (src_cluster_offset + offset_in_cluster) + int64_t sector = (s->crypt_physical_offset ? + (cluster_offset + offset_in_cluster) : + (src_cluster_offset + offset_in_cluster)) >> BDRV_SECTOR_BITS; assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0); assert((bytes & ~BDRV_SECTOR_MASK) == 0); @@ -XXX,XX +XXX,XX @@ static int perform_cow(BlockDriverState *bs, QCowL2Meta *m) /* Encrypt the data if necessary before writing it */ if (bs->encrypted) { - if (!do_perform_cow_encrypt(bs, m->offset, start->offset, - start_buffer, start->nb_bytes) || - !do_perform_cow_encrypt(bs, m->offset, end->offset, - end_buffer, end->nb_bytes)) { + if (!do_perform_cow_encrypt(bs, m->offset, m->alloc_offset, + start->offset, start_buffer, + start->nb_bytes) || + !do_perform_cow_encrypt(bs, m->offset, m->alloc_offset, + end->offset, end_buffer, end->nb_bytes)) { ret = -EIO; goto fail; } diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -XXX,XX +XXX,XX @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res, return ret; } + /* encryption */ + if (s->crypto_header.length) { + ret = inc_refcounts(bs, res, refcount_table, nb_clusters, + s->crypto_header.offset, + s->crypto_header.length); + if (ret < 0) { + return ret; + } + } + return check_refblocks(bs, res, fix, rebuild, refcount_table, nb_clusters); } diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ typedef struct { #define QCOW2_EXT_MAGIC_END 0 #define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xE2792ACA #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 +#define QCOW2_EXT_MAGIC_CRYPTO_HEADER 0x0537be77 static int qcow2_probe(const uint8_t *buf, int buf_size, const char *filename) { @@ -XXX,XX +XXX,XX @@ static int qcow2_probe(const uint8_t *buf, int buf_size, const char *filename) } +static ssize_t qcow2_crypto_hdr_read_func(QCryptoBlock *block, size_t offset, + uint8_t *buf, size_t buflen, + void *opaque, Error **errp) +{ + BlockDriverState *bs = opaque; + BDRVQcow2State *s = bs->opaque; + ssize_t ret; + + if ((offset + buflen) > s->crypto_header.length) { + error_setg(errp, "Request for data outside of extension header"); + return -1; + } + + ret = bdrv_pread(bs->file, + s->crypto_header.offset + offset, buf, buflen); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not read encryption header"); + return -1; + } + return ret; +} + + +static ssize_t qcow2_crypto_hdr_init_func(QCryptoBlock *block, size_t headerlen, + void *opaque, Error **errp) +{ + BlockDriverState *bs = opaque; + BDRVQcow2State *s = bs->opaque; + int64_t ret; + int64_t clusterlen; + + ret = qcow2_alloc_clusters(bs, headerlen); + if (ret < 0) { + error_setg_errno(errp, -ret, + "Cannot allocate cluster for LUKS header size %zu", + headerlen); + return -1; + } + + s->crypto_header.length = headerlen; + s->crypto_header.offset = ret; + + /* Zero fill remaining space in cluster so it has predictable + * content in case of future spec changes */ + clusterlen = size_to_clusters(s, headerlen) * s->cluster_size; + ret = bdrv_pwrite_zeroes(bs->file, + ret + headerlen, + clusterlen - headerlen, 0); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not zero fill encryption header"); + return -1; + } + + return ret; +} + + +static ssize_t qcow2_crypto_hdr_write_func(QCryptoBlock *block, size_t offset, + const uint8_t *buf, size_t buflen, + void *opaque, Error **errp) +{ + BlockDriverState *bs = opaque; + BDRVQcow2State *s = bs->opaque; + ssize_t ret; + + if ((offset + buflen) > s->crypto_header.length) { + error_setg(errp, "Request for data outside of extension header"); + return -1; + } + + ret = bdrv_pwrite(bs->file, + s->crypto_header.offset + offset, buf, buflen); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not read encryption header"); + return -1; + } + return ret; +} + + /* * read qcow2 extension and fill bs * start reading from start_offset @@ -XXX,XX +XXX,XX @@ static int qcow2_probe(const uint8_t *buf, int buf_size, const char *filename) */ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset, uint64_t end_offset, void **p_feature_table, - Error **errp) + int flags, Error **errp) { BDRVQcow2State *s = bs->opaque; QCowExtension ext; @@ -XXX,XX +XXX,XX @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset, } break; + case QCOW2_EXT_MAGIC_CRYPTO_HEADER: { + unsigned int cflags = 0; + if (s->crypt_method_header != QCOW_CRYPT_LUKS) { + error_setg(errp, "CRYPTO header extension only " + "expected with LUKS encryption method"); + return -EINVAL; + } + if (ext.len != sizeof(Qcow2CryptoHeaderExtension)) { + error_setg(errp, "CRYPTO header extension size %u, " + "but expected size %zu", ext.len, + sizeof(Qcow2CryptoHeaderExtension)); + return -EINVAL; + } + + ret = bdrv_pread(bs->file, offset, &s->crypto_header, ext.len); + if (ret < 0) { + error_setg_errno(errp, -ret, + "Unable to read CRYPTO header extension"); + return ret; + } + be64_to_cpus(&s->crypto_header.offset); + be64_to_cpus(&s->crypto_header.length); + + if ((s->crypto_header.offset % s->cluster_size) != 0) { + error_setg(errp, "Encryption header offset '%" PRIu64 "' is " + "not a multiple of cluster size '%u'", + s->crypto_header.offset, s->cluster_size); + return -EINVAL; + } + + if (flags & BDRV_O_NO_IO) { + cflags |= QCRYPTO_BLOCK_OPEN_NO_IO; + } + s->crypto = qcrypto_block_open(s->crypto_opts, + qcow2_crypto_hdr_read_func, + bs, cflags, errp); + if (!s->crypto) { + return -EINVAL; + } + } break; + default: /* unknown magic - save it in case we need to rewrite the header */ { @@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_runtime_opts = { .type = QEMU_OPT_NUMBER, .help = "Clean unused cache entries after this time (in seconds)", }, - BLOCK_CRYPTO_OPT_DEF_QCOW_KEY_SECRET("encrypt."), + BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.", + "ID of secret providing qcow2 AES key or LUKS passphrase"), { /* end of list */ } }, }; @@ -XXX,XX +XXX,XX @@ static int qcow2_update_options_prepare(BlockDriverState *bs, Q_CRYPTO_BLOCK_FORMAT_QCOW, encryptopts, errp); break; + case QCOW_CRYPT_LUKS: + if (encryptfmt && !g_str_equal(encryptfmt, "luks")) { + error_setg(errp, + "Header reported 'luks' encryption format but " + "options specify '%s'", encryptfmt); + ret = -EINVAL; + goto fail; + } + qdict_del(encryptopts, "format"); + r->crypto_opts = block_crypto_open_opts_init( + Q_CRYPTO_BLOCK_FORMAT_LUKS, encryptopts, errp); + break; + default: error_setg(errp, "Unsupported encryption method %d", s->crypt_method_header); @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, if (s->incompatible_features & ~QCOW2_INCOMPAT_MASK) { void *feature_table = NULL; qcow2_read_extensions(bs, header.header_length, ext_end, - &feature_table, NULL); + &feature_table, flags, NULL); report_unsupported_feature(errp, feature_table, s->incompatible_features & ~QCOW2_INCOMPAT_MASK); @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, s->refcount_max = UINT64_C(1) << (s->refcount_bits - 1); s->refcount_max += s->refcount_max - 1; - if (header.crypt_method > QCOW_CRYPT_AES) { - error_setg(errp, "Unsupported encryption method: %" PRIu32, - header.crypt_method); - ret = -EINVAL; - goto fail; - } s->crypt_method_header = header.crypt_method; if (s->crypt_method_header) { if (bdrv_uses_whitelist() && @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, goto fail; } + if (s->crypt_method_header == QCOW_CRYPT_AES) { + s->crypt_physical_offset = false; + } else { + /* Assuming LUKS and any future crypt methods we + * add will all use physical offsets, due to the + * fact that the alternative is insecure... */ + s->crypt_physical_offset = true; + } + bs->encrypted = true; bs->valid_key = true; } @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, /* read qcow2 extensions */ if (qcow2_read_extensions(bs, header.header_length, ext_end, NULL, - &local_err)) { + flags, &local_err)) { error_propagate(errp, local_err); ret = -EINVAL; goto fail; } - if (s->crypt_method_header == QCOW_CRYPT_AES) { - unsigned int cflags = 0; - if (flags & BDRV_O_NO_IO) { - cflags |= QCRYPTO_BLOCK_OPEN_NO_IO; - } - s->crypto = qcrypto_block_open(s->crypto_opts, NULL, NULL, - cflags, errp); - if (!s->crypto) { + /* qcow2_read_extension may have set up the crypto context + * if the crypt method needs a header region, some methods + * don't need header extensions, so must check here + */ + if (s->crypt_method_header && !s->crypto) { + if (s->crypt_method_header == QCOW_CRYPT_AES) { + unsigned int cflags = 0; + if (flags & BDRV_O_NO_IO) { + cflags |= QCRYPTO_BLOCK_OPEN_NO_IO; + } + s->crypto = qcrypto_block_open(s->crypto_opts, NULL, NULL, + cflags, errp); + if (!s->crypto) { + ret = -EINVAL; + goto fail; + } + } else if (!(flags & BDRV_O_NO_IO)) { + error_setg(errp, "Missing CRYPTO header for crypt method %d", + s->crypt_method_header); ret = -EINVAL; goto fail; } @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0); Error *err = NULL; if (qcrypto_block_decrypt(s->crypto, - offset >> BDRV_SECTOR_BITS, + (s->crypt_physical_offset ? + cluster_offset + offset_in_cluster : + offset) >> BDRV_SECTOR_BITS, cluster_data, cur_bytes, &err) < 0) { @@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); qemu_iovec_to_buf(&hd_qiov, 0, cluster_data, hd_qiov.size); - if (qcrypto_block_encrypt(s->crypto, offset >> BDRV_SECTOR_BITS, + if (qcrypto_block_encrypt(s->crypto, + (s->crypt_physical_offset ? + cluster_offset + offset_in_cluster : + offset) >> BDRV_SECTOR_BITS, cluster_data, cur_bytes, &err) < 0) { error_free(err); @@ -XXX,XX +XXX,XX @@ int qcow2_update_header(BlockDriverState *bs) buflen -= ret; } + /* Full disk encryption header pointer extension */ + if (s->crypto_header.offset != 0) { + cpu_to_be64s(&s->crypto_header.offset); + cpu_to_be64s(&s->crypto_header.length); + ret = header_ext_add(buf, QCOW2_EXT_MAGIC_CRYPTO_HEADER, + &s->crypto_header, sizeof(s->crypto_header), + buflen); + be64_to_cpus(&s->crypto_header.offset); + be64_to_cpus(&s->crypto_header.length); + if (ret < 0) { + goto fail; + } + buf += ret; + buflen -= ret; + } + /* Feature table */ if (s->qcow_version >= 3) { Qcow2Feature features[] = { @@ -XXX,XX +XXX,XX @@ static int qcow2_change_backing_file(BlockDriverState *bs, return qcow2_update_header(bs); } +static int qcow2_crypt_method_from_format(const char *encryptfmt) +{ + if (g_str_equal(encryptfmt, "luks")) { + return QCOW_CRYPT_LUKS; + } else if (g_str_equal(encryptfmt, "aes")) { + return QCOW_CRYPT_AES; + } else { + return -EINVAL; + } +} static int qcow2_set_up_encryption(BlockDriverState *bs, const char *encryptfmt, QemuOpts *opts, Error **errp) @@ -XXX,XX +XXX,XX @@ static int qcow2_set_up_encryption(BlockDriverState *bs, const char *encryptfmt, QCryptoBlock *crypto = NULL; int ret = -EINVAL; QDict *options, *encryptopts; + int fmt; options = qemu_opts_to_qdict(opts, NULL); qdict_extract_subqdict(options, &encryptopts, "encrypt."); QDECREF(options); - if (!g_str_equal(encryptfmt, "aes")) { - error_setg(errp, "Unknown encryption format '%s', expected 'aes'", - encryptfmt); - ret = -EINVAL; - goto out; + fmt = qcow2_crypt_method_from_format(encryptfmt); + + switch (fmt) { + case QCOW_CRYPT_LUKS: + cryptoopts = block_crypto_create_opts_init( + Q_CRYPTO_BLOCK_FORMAT_LUKS, encryptopts, errp); + break; + case QCOW_CRYPT_AES: + cryptoopts = block_crypto_create_opts_init( + Q_CRYPTO_BLOCK_FORMAT_QCOW, encryptopts, errp); + break; + default: + error_setg(errp, "Unknown encryption format '%s'", encryptfmt); + break; } - cryptoopts = block_crypto_create_opts_init( - Q_CRYPTO_BLOCK_FORMAT_QCOW, encryptopts, errp); if (!cryptoopts) { ret = -EINVAL; goto out; } - s->crypt_method_header = QCOW_CRYPT_AES; + s->crypt_method_header = fmt; crypto = qcrypto_block_create(cryptoopts, - NULL, NULL, + qcow2_crypto_hdr_init_func, + qcow2_crypto_hdr_write_func, bs, errp); if (!crypto) { ret = -EINVAL; @@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, const char *compat = NULL; uint64_t cluster_size = s->cluster_size; bool encrypt; + int encformat; int refcount_bits = s->refcount_bits; Error *local_err = NULL; int ret; @@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, error_report("Changing the encryption flag is not supported"); return -ENOTSUP; } + } else if (!strcmp(desc->name, BLOCK_OPT_ENCRYPT_FORMAT)) { + encformat = qcow2_crypt_method_from_format( + qemu_opt_get(opts, BLOCK_OPT_ENCRYPT_FORMAT)); + + if (encformat != s->crypt_method_header) { + error_report("Changing the encryption format is not supported"); + return -ENOTSUP; + } } else if (!strcmp(desc->name, BLOCK_OPT_CLUSTER_SIZE)) { cluster_size = qemu_opt_get_size(opts, BLOCK_OPT_CLUSTER_SIZE, cluster_size); @@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_create_opts = { { .name = BLOCK_OPT_ENCRYPT_FORMAT, .type = QEMU_OPT_STRING, - .help = "Encrypt the image, format choices: 'aes'", + .help = "Encrypt the image, format choices: 'aes', 'luks'", }, - BLOCK_CRYPTO_OPT_DEF_QCOW_KEY_SECRET("encrypt."), + BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.", + "ID of secret providing qcow AES key or LUKS passphrase"), + BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."), + BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."), + BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."), + BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."), + BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."), + BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."), { .name = BLOCK_OPT_CLUSTER_SIZE, .type = QEMU_OPT_SIZE, diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ #define QCOW_CRYPT_NONE 0 #define QCOW_CRYPT_AES 1 +#define QCOW_CRYPT_LUKS 2 #define QCOW_MAX_CRYPT_CLUSTERS 32 #define QCOW_MAX_SNAPSHOTS 65536 @@ -XXX,XX +XXX,XX @@ typedef struct QCowSnapshot { struct Qcow2Cache; typedef struct Qcow2Cache Qcow2Cache; +typedef struct Qcow2CryptoHeaderExtension { + uint64_t offset; + uint64_t length; +} QEMU_PACKED Qcow2CryptoHeaderExtension; + typedef struct Qcow2UnknownHeaderExtension { uint32_t magic; uint32_t len; @@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcow2State { CoMutex lock; + Qcow2CryptoHeaderExtension crypto_header; /* QCow2 header extension */ QCryptoBlockOpenOptions *crypto_opts; /* Disk encryption runtime options */ QCryptoBlock *crypto; /* Disk encryption format driver */ + bool crypt_physical_offset; /* Whether to use virtual or physical offset + for encryption initialization vector tweak */ uint32_t crypt_method_header; uint64_t snapshots_offset; int snapshots_size; diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # Since: 2.10 ## { 'enum': 'BlockdevQcow2EncryptionFormat', - 'data': [ 'aes' ] } + 'data': [ 'aes', 'luks' ] } ## # @BlockdevQcow2Encryption: @@ -XXX,XX +XXX,XX @@ { 'union': 'BlockdevQcow2Encryption', 'base': { 'format': 'BlockdevQcow2EncryptionFormat' }, 'discriminator': 'format', - 'data': { 'aes': 'QCryptoBlockOptionsQCow' } } + 'data': { 'aes': 'QCryptoBlockOptionsQCow', + 'luks': 'QCryptoBlockOptionsLUKS'} } ## # @BlockdevOptionsQcow2: diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates @@ -XXX,XX +XXX,XX @@ compat Compatibility level (0.10 or 1.1) backing_file File name of a base image backing_fmt Image format of the base image encryption Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) -encrypt.format Encrypt the image, format choices: 'aes' -encrypt.key-secret ID of the secret that provides the AES encryption key +encrypt.format Encrypt the image, format choices: 'aes', 'luks' +encrypt.key-secret ID of secret providing qcow AES key or LUKS passphrase +encrypt.cipher-alg Name of encryption cipher algorithm +encrypt.cipher-mode Name of encryption cipher mode +encrypt.ivgen-alg Name of IV generator algorithm +encrypt.ivgen-hash-alg Name of IV generator hash algorithm +encrypt.hash-alg Name of encryption hash algorithm +encrypt.iter-time Time to spend in PBKDF in milliseconds cluster_size qcow2 cluster size preallocation Preallocation mode (allowed values: off, metadata, falloc, full) lazy_refcounts Postpone refcount updates -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> This extends the 087 iotest to cover LUKS encryption when doing blockdev-add. Two further tests are added to validate read/write of LUKS encrypted images with a single file and with a backing file. Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-15-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/087 | 35 ++++++++++++++++++- tests/qemu-iotests/087.out | 14 +++++++- tests/qemu-iotests/188 | 76 ++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/188.out | 18 ++++++++++ tests/qemu-iotests/189 | 86 ++++++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/189.out | 26 ++++++++++++++ tests/qemu-iotests/group | 2 ++ 7 files changed, 255 insertions(+), 2 deletions(-) create mode 100755 tests/qemu-iotests/188 create mode 100644 tests/qemu-iotests/188.out create mode 100755 tests/qemu-iotests/189 create mode 100644 tests/qemu-iotests/189.out diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/087 +++ b/tests/qemu-iotests/087 @@ -XXX,XX +XXX,XX @@ run_qemu <<EOF EOF echo -echo === Encrypted image === +echo === Encrypted image QCow === echo _make_test_img --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 $size @@ -XXX,XX +XXX,XX @@ run_qemu <<EOF EOF echo +echo === Encrypted image LUKS === +echo + +_make_test_img --object secret,id=sec0,data=123456 -o encrypt.format=luks,encrypt.key-secret=sec0 $size +run_qemu <<EOF +{ "execute": "qmp_capabilities" } +{ "execute": "object-add", + "arguments": { + "qom-type": "secret", + "id": "sec0", + "props": { + "data": "123456" + } + } +} +{ "execute": "blockdev-add", + "arguments": { + "driver": "$IMGFMT", + "node-name": "disk", + "file": { + "driver": "file", + "filename": "$TEST_IMG" + }, + "encrypt": { + "format": "luks", + "key-secret": "sec0" + } + } + } +{ "execute": "quit" } +EOF + +echo echo === Missing driver === echo diff --git a/tests/qemu-iotests/087.out b/tests/qemu-iotests/087.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/087.out +++ b/tests/qemu-iotests/087.out @@ -XXX,XX +XXX,XX @@ QMP_VERSION {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} -=== Encrypted image === +=== Encrypted image QCow === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 Testing: @@ -XXX,XX +XXX,XX @@ QMP_VERSION {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +=== Encrypted image LUKS === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encrypt.format=luks encrypt.key-secret=sec0 +Testing: +QMP_VERSION +{"return": {}} +{"return": {}} +{"return": {}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + + === Missing driver === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 diff --git a/tests/qemu-iotests/188 b/tests/qemu-iotests/188 new file mode 100755 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/188 @@ -XXX,XX +XXX,XX @@ +#!/bin/bash +# +# Test encrypted read/write using plain bdrv_read/bdrv_write +# +# Copyright (C) 2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator +owner=berrange@redhat.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto generic +_supported_os Linux + + +size=16M + +SECRET="secret,id=sec0,data=astrochicken" +SECRETALT="secret,id=sec0,data=platypus" + +_make_test_img --object $SECRET -o "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10" $size + +IMGSPEC="driver=$IMGFMT,file.filename=$TEST_IMG,encrypt.key-secret=sec0" + +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT + +echo +echo "== reading whole image ==" +$QEMU_IO --object $SECRET -c "read -P 0 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir + +echo +echo "== rewriting whole image ==" +$QEMU_IO --object $SECRET -c "write -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir + +echo +echo "== verify pattern ==" +$QEMU_IO --object $SECRET -c "read -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir + +echo +echo "== verify open failure with wrong password ==" +$QEMU_IO --object $SECRETALT -c "read -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir + + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/188.out b/tests/qemu-iotests/188.out new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/188.out @@ -XXX,XX +XXX,XX @@ +QA output created by 188 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 + +== reading whole image == +read 16777216/16777216 bytes at offset 0 +16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== rewriting whole image == +wrote 16777216/16777216 bytes at offset 0 +16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== verify pattern == +read 16777216/16777216 bytes at offset 0 +16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== verify open failure with wrong password == +can't open: Invalid password, cannot unlock any keyslot +*** done diff --git a/tests/qemu-iotests/189 b/tests/qemu-iotests/189 new file mode 100755 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/189 @@ -XXX,XX +XXX,XX @@ +#!/bin/bash +# +# Test encrypted read/write using backing files +# +# Copyright (C) 2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator +owner=berrange@redhat.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto generic +_supported_os Linux + + +size=16M +TEST_IMG_BASE=$TEST_IMG.base +SECRET0="secret,id=sec0,data=astrochicken" +SECRET1="secret,id=sec1,data=furby" + +TEST_IMG_SAVE=$TEST_IMG +TEST_IMG=$TEST_IMG_BASE +echo "== create base ==" +_make_test_img --object $SECRET0 -o "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10" $size +TEST_IMG=$TEST_IMG_SAVE + +IMGSPECBASE="driver=$IMGFMT,file.filename=$TEST_IMG_BASE,encrypt.key-secret=sec0" +IMGSPEC="driver=$IMGFMT,file.filename=$TEST_IMG,backing.driver=$IMGFMT,backing.file.filename=$TEST_IMG_BASE,backing.encrypt.key-secret=sec0,encrypt.key-secret=sec1" +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT + +echo +echo "== writing whole image ==" +$QEMU_IO --object $SECRET0 -c "write -P 0xa 0 $size" --image-opts $IMGSPECBASE | _filter_qemu_io | _filter_testdir + +echo +echo "== verify pattern ==" +$QEMU_IO --object $SECRET0 -c "read -P 0xa 0 $size" --image-opts $IMGSPECBASE | _filter_qemu_io | _filter_testdir + +echo "== create overlay ==" +_make_test_img --object $SECRET1 -o "encrypt.format=luks,encrypt.key-secret=sec1,encrypt.iter-time=10" -b "$TEST_IMG_BASE" $size + +echo +echo "== writing part of a cluster ==" +$QEMU_IO --object $SECRET0 --object $SECRET1 -c "write -P 0xe 0 1024" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir + +echo +echo "== verify pattern ==" +$QEMU_IO --object $SECRET0 --object $SECRET1 -c "read -P 0xe 0 1024" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir +echo +echo "== verify pattern ==" +$QEMU_IO --object $SECRET0 --object $SECRET1 -c "read -P 0xa 1024 64512" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir + + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/189.out b/tests/qemu-iotests/189.out new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/189.out @@ -XXX,XX +XXX,XX @@ +QA output created by 189 +== create base == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 + +== writing whole image == +wrote 16777216/16777216 bytes at offset 0 +16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== verify pattern == +read 16777216/16777216 bytes at offset 0 +16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== create overlay == +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10 + +== writing part of a cluster == +wrote 1024/1024 bytes at offset 0 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== verify pattern == +read 1024/1024 bytes at offset 0 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== verify pattern == +read 64512/64512 bytes at offset 1024 +63 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -XXX,XX +XXX,XX @@ 182 rw auto quick 183 rw auto migration 185 rw auto +188 rw auto quick +189 rw auto quick -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> The 138 and 158 iotests exercise the legacy qcow2 aes encryption code path and they work fine with qcow v1 too. Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-16-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/134 | 2 +- tests/qemu-iotests/158 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/134 +++ b/tests/qemu-iotests/134 @@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -_supported_fmt qcow2 +_supported_fmt qcow qcow2 _supported_proto generic _unsupported_proto vxhs _supported_os Linux diff --git a/tests/qemu-iotests/158 b/tests/qemu-iotests/158 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/158 +++ b/tests/qemu-iotests/158 @@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -_supported_fmt qcow2 +_supported_fmt qcow qcow2 _supported_proto generic _unsupported_proto vxhs _supported_os Linux -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Now that qcow & qcow2 are wired up to get encryption keys via the QCryptoSecret object, nothing is relying on the interactive prompting for passwords. All the code related to password prompting can thus be ripped out. Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-17-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- hmp.c | 31 --------------------- include/monitor/monitor.h | 7 ----- include/qemu/osdep.h | 2 -- monitor.c | 68 ----------------------------------------------- qapi-schema.json | 10 +------ qemu-img.c | 31 --------------------- qemu-io.c | 20 -------------- qmp.c | 12 +-------- util/oslib-posix.c | 66 --------------------------------------------- util/oslib-win32.c | 24 ----------------- 10 files changed, 2 insertions(+), 269 deletions(-) diff --git a/hmp.c b/hmp.c index XXXXXXX..XXXXXXX 100644 --- a/hmp.c +++ b/hmp.c @@ -XXX,XX +XXX,XX @@ void hmp_ringbuf_read(Monitor *mon, const QDict *qdict) g_free(data); } -static void hmp_cont_cb(void *opaque, int err) -{ - if (!err) { - qmp_cont(NULL); - } -} - -static bool key_is_missing(const BlockInfo *bdev) -{ - return (bdev->inserted && bdev->inserted->encryption_key_missing); -} - void hmp_cont(Monitor *mon, const QDict *qdict) { - BlockInfoList *bdev_list, *bdev; Error *err = NULL; - bdev_list = qmp_query_block(NULL); - for (bdev = bdev_list; bdev; bdev = bdev->next) { - if (key_is_missing(bdev->value)) { - monitor_read_block_device_key(mon, bdev->value->device, - hmp_cont_cb, NULL); - goto out; - } - } - qmp_cont(&err); hmp_handle_error(mon, &err); - -out: - qapi_free_BlockInfoList(bdev_list); } void hmp_system_wakeup(Monitor *mon, const QDict *qdict) @@ -XXX,XX +XXX,XX @@ void hmp_change(Monitor *mon, const QDict *qdict) qmp_blockdev_change_medium(true, device, false, NULL, target, !!arg, arg, !!read_only, read_only_mode, &err); - if (err && - error_get_class(err) == ERROR_CLASS_DEVICE_ENCRYPTED) { - error_free(err); - monitor_read_block_device_key(mon, device, NULL, NULL); - return; - } } hmp_handle_error(mon, &err); diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h index XXXXXXX..XXXXXXX 100644 --- a/include/monitor/monitor.h +++ b/include/monitor/monitor.h @@ -XXX,XX +XXX,XX @@ void monitor_cleanup(void); int monitor_suspend(Monitor *mon); void monitor_resume(Monitor *mon); -int monitor_read_bdrv_key_start(Monitor *mon, BlockDriverState *bs, - BlockCompletionFunc *completion_cb, - void *opaque); -int monitor_read_block_device_key(Monitor *mon, const char *device, - BlockCompletionFunc *completion_cb, - void *opaque); - int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp); int monitor_fd_param(Monitor *mon, const char *fdname, Error **errp); diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -XXX,XX +XXX,XX @@ void qemu_set_tty_echo(int fd, bool echo); void os_mem_prealloc(int fd, char *area, size_t sz, int smp_cpus, Error **errp); -int qemu_read_password(char *buf, int buf_size); - /** * qemu_get_pid_name: * @pid: pid of a process diff --git a/monitor.c b/monitor.c index XXXXXXX..XXXXXXX 100644 --- a/monitor.c +++ b/monitor.c @@ -XXX,XX +XXX,XX @@ void monitor_cleanup(void) qemu_mutex_unlock(&monitor_lock); } -static void bdrv_password_cb(void *opaque, const char *password, - void *readline_opaque) -{ - Monitor *mon = opaque; - BlockDriverState *bs = readline_opaque; - int ret = 0; - Error *local_err = NULL; - - bdrv_add_key(bs, password, &local_err); - if (local_err) { - error_report_err(local_err); - ret = -EPERM; - } - if (mon->password_completion_cb) - mon->password_completion_cb(mon->password_opaque, ret); - - monitor_read_command(mon, 1); -} - -int monitor_read_bdrv_key_start(Monitor *mon, BlockDriverState *bs, - BlockCompletionFunc *completion_cb, - void *opaque) -{ - int err; - - monitor_printf(mon, "%s (%s) is encrypted.\n", bdrv_get_device_name(bs), - bdrv_get_encrypted_filename(bs)); - - mon->password_completion_cb = completion_cb; - mon->password_opaque = opaque; - - err = monitor_read_password(mon, bdrv_password_cb, bs); - - if (err && completion_cb) - completion_cb(opaque, err); - - return err; -} - -int monitor_read_block_device_key(Monitor *mon, const char *device, - BlockCompletionFunc *completion_cb, - void *opaque) -{ - Error *err = NULL; - BlockBackend *blk; - - blk = blk_by_name(device); - if (!blk) { - monitor_printf(mon, "Device not found %s\n", device); - return -1; - } - if (!blk_bs(blk)) { - monitor_printf(mon, "Device '%s' has no medium\n", device); - return -1; - } - - bdrv_add_key(blk_bs(blk), NULL, &err); - if (err) { - error_free(err); - return monitor_read_bdrv_key_start(mon, blk_bs(blk), completion_cb, opaque); - } - - if (completion_cb) { - completion_cb(opaque, 0); - } - return 0; -} - QemuOptsList qemu_mon_opts = { .name = "mon", .implied_opt_name = "chardev", diff --git a/qapi-schema.json b/qapi-schema.json index XXXXXXX..XXXXXXX 100644 --- a/qapi-schema.json +++ b/qapi-schema.json @@ -XXX,XX +XXX,XX @@ # Since: 0.14.0 # # Returns: If successful, nothing -# If QEMU was started with an encrypted block device and a key has -# not yet been set, DeviceEncrypted. # # Notes: This command will succeed if the guest is currently running. It # will also succeed if the guest is in the "inmigrate" state; in @@ -XXX,XX +XXX,XX @@ # * This command is stateless, this means that commands that depend # on state information (such as getfd) might not work # -# * Commands that prompt the user for data (eg. 'cont' when the block -# device is encrypted) don't currently work +# * Commands that prompt the user for data don't currently work # # Example: # @@ -XXX,XX +XXX,XX @@ # # Returns: Nothing on success. # If @device is not a valid block device, DeviceNotFound -# If the new block device is encrypted, DeviceEncrypted. Note that -# if this error is returned, the device has been opened successfully -# and an additional call to @block_passwd is required to set the -# device's password. The behavior of reads and writes to the block -# device between when these calls are executed is undefined. # # Notes: This interface is deprecated, and it is strongly recommended that you # avoid using it. For changing block devices, use diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static int print_block_option_help(const char *filename, const char *fmt) } -static int img_open_password(BlockBackend *blk, const char *filename, - int flags, bool quiet) -{ - BlockDriverState *bs; - char password[256]; - - bs = blk_bs(blk); - if (bdrv_is_encrypted(bs) && bdrv_key_required(bs) && - !(flags & BDRV_O_NO_IO)) { - qprintf(quiet, "Disk image '%s' is encrypted.\n", filename); - if (qemu_read_password(password, sizeof(password)) < 0) { - error_report("No password given"); - return -1; - } - if (bdrv_set_key(bs, password) < 0) { - error_report("invalid password"); - return -1; - } - } - return 0; -} - - static BlockBackend *img_open_opts(const char *optstr, QemuOpts *opts, int flags, bool writethrough, bool quiet, bool force_share) @@ -XXX,XX +XXX,XX @@ static BlockBackend *img_open_opts(const char *optstr, } blk_set_enable_write_cache(blk, !writethrough); - if (img_open_password(blk, optstr, flags, quiet) < 0) { - blk_unref(blk); - return NULL; - } return blk; } @@ -XXX,XX +XXX,XX @@ static BlockBackend *img_open_file(const char *filename, } blk_set_enable_write_cache(blk, !writethrough); - if (img_open_password(blk, filename, flags, quiet) < 0) { - blk_unref(blk); - return NULL; - } return blk; } diff --git a/qemu-io.c b/qemu-io.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-io.c +++ b/qemu-io.c @@ -XXX,XX +XXX,XX @@ static int openfile(char *name, int flags, bool writethrough, bool force_share, QDict *opts) { Error *local_err = NULL; - BlockDriverState *bs; if (qemuio_blk) { error_report("file open already, try 'help close'"); @@ -XXX,XX +XXX,XX @@ static int openfile(char *name, int flags, bool writethrough, bool force_share, return 1; } - bs = blk_bs(qemuio_blk); - if (bdrv_is_encrypted(bs) && bdrv_key_required(bs)) { - char password[256]; - printf("Disk image '%s' is encrypted.\n", name); - if (qemu_read_password(password, sizeof(password)) < 0) { - error_report("No password given"); - goto error; - } - if (bdrv_set_key(bs, password) < 0) { - error_report("invalid password"); - goto error; - } - } - blk_set_enable_write_cache(qemuio_blk, !writethrough); return 0; - - error: - blk_unref(qemuio_blk); - qemuio_blk = NULL; - return 1; } static void open_help(void) diff --git a/qmp.c b/qmp.c index XXXXXXX..XXXXXXX 100644 --- a/qmp.c +++ b/qmp.c @@ -XXX,XX +XXX,XX @@ SpiceInfo *qmp_query_spice(Error **errp) void qmp_cont(Error **errp) { - Error *local_err = NULL; BlockBackend *blk; - BlockDriverState *bs; - BdrvNextIterator it; + Error *local_err = NULL; /* if there is a dump in background, we should wait until the dump * finished */ @@ -XXX,XX +XXX,XX @@ void qmp_cont(Error **errp) blk_iostatus_reset(blk); } - for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { - bdrv_add_key(bs, NULL, &local_err); - if (local_err) { - error_propagate(errp, local_err); - return; - } - } - /* Continuing after completed migration. Images have been inactivated to * allow the destination to take control. Need to get control back now. * diff --git a/util/oslib-posix.c b/util/oslib-posix.c index XXXXXXX..XXXXXXX 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -XXX,XX +XXX,XX @@ void os_mem_prealloc(int fd, char *area, size_t memory, int smp_cpus, } -static struct termios oldtty; - -static void term_exit(void) -{ - tcsetattr(0, TCSANOW, &oldtty); -} - -static void term_init(void) -{ - struct termios tty; - - tcgetattr(0, &tty); - oldtty = tty; - - tty.c_iflag &= ~(IGNBRK|BRKINT|PARMRK|ISTRIP - |INLCR|IGNCR|ICRNL|IXON); - tty.c_oflag |= OPOST; - tty.c_lflag &= ~(ECHO|ECHONL|ICANON|IEXTEN); - tty.c_cflag &= ~(CSIZE|PARENB); - tty.c_cflag |= CS8; - tty.c_cc[VMIN] = 1; - tty.c_cc[VTIME] = 0; - - tcsetattr(0, TCSANOW, &tty); - - atexit(term_exit); -} - -int qemu_read_password(char *buf, int buf_size) -{ - uint8_t ch; - int i, ret; - - printf("password: "); - fflush(stdout); - term_init(); - i = 0; - for (;;) { - ret = read(0, &ch, 1); - if (ret == -1) { - if (errno == EAGAIN || errno == EINTR) { - continue; - } else { - break; - } - } else if (ret == 0) { - ret = -1; - break; - } else { - if (ch == '\r' || - ch == '\n') { - ret = 0; - break; - } - if (i < (buf_size - 1)) { - buf[i++] = ch; - } - } - } - term_exit(); - buf[i] = '\0'; - printf("\n"); - return ret; -} - - char *qemu_get_pid_name(pid_t pid) { char *name = NULL; diff --git a/util/oslib-win32.c b/util/oslib-win32.c index XXXXXXX..XXXXXXX 100644 --- a/util/oslib-win32.c +++ b/util/oslib-win32.c @@ -XXX,XX +XXX,XX @@ void os_mem_prealloc(int fd, char *area, size_t memory, int smp_cpus, } -/* XXX: put correct support for win32 */ -int qemu_read_password(char *buf, int buf_size) -{ - int c, i; - - printf("Password: "); - fflush(stdout); - i = 0; - for (;;) { - c = getchar(); - if (c < 0) { - buf[i] = '\0'; - return -1; - } else if (c == '\n') { - break; - } else if (i < (buf_size - 1)) { - buf[i++] = c; - } - } - buf[i] = '\0'; - return 0; -} - - char *qemu_get_pid_name(pid_t pid) { /* XXX Implement me */ -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Now that all encryption keys must be provided upfront via the QCryptoSecret API and associated block driver properties there is no need for any explicit encryption handling APIs in the block layer. Encryption can be handled transparently within the block driver. We only retain an API for querying whether an image is encrypted or not, since that is a potentially useful piece of metadata to report to the user. Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-18-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block.c | 77 +---------------------------------------------- block/crypto.c | 1 - block/qapi.c | 2 +- block/qcow.c | 8 ++++- block/qcow2.c | 1 - blockdev.c | 37 ++--------------------- hmp-commands.hx | 2 ++ include/block/block.h | 3 -- include/block/block_int.h | 1 - include/qapi/error.h | 1 - qapi/block-core.json | 37 ++--------------------- qapi/common.json | 5 +-- 12 files changed, 16 insertions(+), 159 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static BlockDriverState *bdrv_open_inherit(const char *filename, goto close_and_fail; } - if (!bdrv_key_required(bs)) { - bdrv_parent_cb_change_media(bs, true); - } else if (!runstate_check(RUN_STATE_PRELAUNCH) - && !runstate_check(RUN_STATE_INMIGRATE) - && !runstate_check(RUN_STATE_PAUSED)) { /* HACK */ - error_setg(errp, - "Guest must be stopped for opening of encrypted image"); - goto close_and_fail; - } + bdrv_parent_cb_change_media(bs, true); QDECREF(options); @@ -XXX,XX +XXX,XX @@ static void bdrv_close(BlockDriverState *bs) bs->backing_format[0] = '\0'; bs->total_sectors = 0; bs->encrypted = false; - bs->valid_key = false; bs->sg = false; QDECREF(bs->options); QDECREF(bs->explicit_options); @@ -XXX,XX +XXX,XX @@ bool bdrv_is_encrypted(BlockDriverState *bs) return bs->encrypted; } -bool bdrv_key_required(BlockDriverState *bs) -{ - BdrvChild *backing = bs->backing; - - if (backing && backing->bs->encrypted && !backing->bs->valid_key) { - return true; - } - return (bs->encrypted && !bs->valid_key); -} - -int bdrv_set_key(BlockDriverState *bs, const char *key) -{ - int ret; - if (bs->backing && bs->backing->bs->encrypted) { - ret = bdrv_set_key(bs->backing->bs, key); - if (ret < 0) - return ret; - if (!bs->encrypted) - return 0; - } - if (!bs->encrypted) { - return -EINVAL; - } else if (!bs->drv || !bs->drv->bdrv_set_key) { - return -ENOMEDIUM; - } - ret = bs->drv->bdrv_set_key(bs, key); - if (ret < 0) { - bs->valid_key = false; - } else if (!bs->valid_key) { - /* call the change callback now, we skipped it on open */ - bs->valid_key = true; - bdrv_parent_cb_change_media(bs, true); - } - return ret; -} - -/* - * Provide an encryption key for @bs. - * If @key is non-null: - * If @bs is not encrypted, fail. - * Else if the key is invalid, fail. - * Else set @bs's key to @key, replacing the existing key, if any. - * If @key is null: - * If @bs is encrypted and still lacks a key, fail. - * Else do nothing. - * On failure, store an error object through @errp if non-null. - */ -void bdrv_add_key(BlockDriverState *bs, const char *key, Error **errp) -{ - if (key) { - if (!bdrv_is_encrypted(bs)) { - error_setg(errp, "Node '%s' is not encrypted", - bdrv_get_device_or_node_name(bs)); - } else if (bdrv_set_key(bs, key) < 0) { - error_setg(errp, QERR_INVALID_PASSWORD); - } - } else { - if (bdrv_key_required(bs)) { - error_set(errp, ERROR_CLASS_DEVICE_ENCRYPTED, - "'%s' (%s) is encrypted", - bdrv_get_device_or_node_name(bs), - bdrv_get_encrypted_filename(bs)); - } - } -} - const char *bdrv_get_format_name(BlockDriverState *bs) { return bs->drv ? bs->drv->format_name : NULL; diff --git a/block/crypto.c b/block/crypto.c index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -XXX,XX +XXX,XX @@ static int block_crypto_open_generic(QCryptoBlockFormat format, } bs->encrypted = true; - bs->valid_key = true; ret = 0; cleanup: diff --git a/block/qapi.c b/block/qapi.c index XXXXXXX..XXXXXXX 100644 --- a/block/qapi.c +++ b/block/qapi.c @@ -XXX,XX +XXX,XX @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *blk, info->ro = bs->read_only; info->drv = g_strdup(bs->drv->format_name); info->encrypted = bs->encrypted; - info->encryption_key_missing = bdrv_key_required(bs); + info->encryption_key_missing = false; info->cache = g_new(BlockdevCacheInfo, 1); *info->cache = (BlockdevCacheInfo) { diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags, goto fail; } bs->encrypted = true; - bs->valid_key = true; + } else { + if (encryptfmt) { + error_setg(errp, "No encryption in image header, but options " + "specified format '%s'", encryptfmt); + ret = -EINVAL; + goto fail; + } } s->cluster_bits = header.cluster_bits; s->cluster_size = 1 << s->cluster_bits; diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, } bs->encrypted = true; - bs->valid_key = true; } s->l2_bits = s->cluster_bits - 3; /* L2 is always one cluster */ diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ static BlockBackend *blockdev_init(const char *file, QDict *bs_opts, bs->detect_zeroes = detect_zeroes; - if (bdrv_key_required(bs)) { - autostart = 0; - } - block_acct_setup(blk_get_stats(blk), account_invalid, account_failed); if (!parse_stats_intervals(blk_get_stats(blk), interval_list, errp)) { @@ -XXX,XX +XXX,XX @@ void qmp_block_passwd(bool has_device, const char *device, bool has_node_name, const char *node_name, const char *password, Error **errp) { - Error *local_err = NULL; - BlockDriverState *bs; - AioContext *aio_context; - - bs = bdrv_lookup_bs(has_device ? device : NULL, - has_node_name ? node_name : NULL, - &local_err); - if (local_err) { - error_propagate(errp, local_err); - return; - } - - aio_context = bdrv_get_aio_context(bs); - aio_context_acquire(aio_context); - - bdrv_add_key(bs, password, errp); - - aio_context_release(aio_context); + error_setg(errp, + "Setting block passwords directly is no longer supported"); } /* @@ -XXX,XX +XXX,XX @@ void qmp_blockdev_change_medium(bool has_device, const char *device, goto fail; } - bdrv_add_key(medium_bs, NULL, &err); - if (err) { - error_propagate(errp, err); - goto fail; - } - rc = do_open_tray(has_device ? device : NULL, has_id ? id : NULL, false, &err); @@ -XXX,XX +XXX,XX @@ void qmp_blockdev_add(BlockdevOptions *options, Error **errp) QTAILQ_INSERT_TAIL(&monitor_bdrv_states, bs, monitor_list); - if (bs && bdrv_key_required(bs)) { - QTAILQ_REMOVE(&monitor_bdrv_states, bs, monitor_list); - bdrv_unref(bs); - error_setg(errp, "blockdev-add doesn't support encrypted devices"); - goto fail; - } - fail: visit_free(v); } diff --git a/hmp-commands.hx b/hmp-commands.hx index XXXXXXX..XXXXXXX 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -XXX,XX +XXX,XX @@ STEXI @item block_passwd @var{device} @var{password} @findex block_passwd Set the encrypted device @var{device} password to @var{password} + +This command is now obsolete and will always return an error since 2.10 ETEXI { diff --git a/include/block/block.h b/include/block/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -XXX,XX +XXX,XX @@ BlockDriverState *bdrv_next(BdrvNextIterator *it); BlockDriverState *bdrv_next_monitor_owned(BlockDriverState *bs); bool bdrv_is_encrypted(BlockDriverState *bs); -bool bdrv_key_required(BlockDriverState *bs); -int bdrv_set_key(BlockDriverState *bs, const char *key); -void bdrv_add_key(BlockDriverState *bs, const char *key, Error **errp); void bdrv_iterate_format(void (*it)(void *opaque, const char *name), void *opaque); const char *bdrv_get_node_name(const BlockDriverState *bs); diff --git a/include/block/block_int.h b/include/block/block_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -XXX,XX +XXX,XX @@ struct BlockDriverState { int open_flags; /* flags used to open the file, re-used for re-open */ bool read_only; /* if true, the media is read only */ bool encrypted; /* if true, the media is encrypted */ - bool valid_key; /* if true, a valid encryption key has been set */ bool sg; /* if true, the device is a /dev/sg* */ bool probed; /* if true, format was probed rather than specified */ bool force_share; /* if true, always allow all shared permissions */ diff --git a/include/qapi/error.h b/include/qapi/error.h index XXXXXXX..XXXXXXX 100644 --- a/include/qapi/error.h +++ b/include/qapi/error.h @@ -XXX,XX +XXX,XX @@ typedef enum ErrorClass { ERROR_CLASS_GENERIC_ERROR = QAPI_ERROR_CLASS_GENERICERROR, ERROR_CLASS_COMMAND_NOT_FOUND = QAPI_ERROR_CLASS_COMMANDNOTFOUND, - ERROR_CLASS_DEVICE_ENCRYPTED = QAPI_ERROR_CLASS_DEVICEENCRYPTED, ERROR_CLASS_DEVICE_NOT_ACTIVE = QAPI_ERROR_CLASS_DEVICENOTACTIVE, ERROR_CLASS_DEVICE_NOT_FOUND = QAPI_ERROR_CLASS_DEVICENOTFOUND, ERROR_CLASS_KVM_MISSING_CAP = QAPI_ERROR_CLASS_KVMMISSINGCAP, diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # # @encrypted: true if the backing device is encrypted # -# @encryption_key_missing: true if the backing device is encrypted but an -# valid encryption key is missing +# @encryption_key_missing: Deprecated; always false # # @detect_zeroes: detect and optimize zero writes (Since 2.1) # @@ -XXX,XX +XXX,XX @@ # This command sets the password of a block device that has not been open # with a password and requires one. # -# The two cases where this can happen are a block device is created through -# QEMU's initial command line or a block device is changed through the legacy -# @change interface. -# -# In the event that the block device is created through the initial command -# line, the VM will start in the stopped state regardless of whether '-S' is -# used. The intention is for a management tool to query the block devices to -# determine which ones are encrypted, set the passwords with this command, and -# then start the guest with the @cont command. -# -# Either @device or @node-name must be set but not both. -# -# @device: the name of the block backend device to set the password on -# -# @node-name: graph node name to set the password on (Since 2.0) -# -# @password: the password to use for the device -# -# Returns: nothing on success -# If @device is not a valid block device, DeviceNotFound -# If @device is not encrypted, DeviceNotEncrypted -# -# Notes: Not all block formats support encryption and some that do are not -# able to validate that a password is correct. Disk corruption may -# occur if an invalid password is specified. -# -# Since: 0.14.0 -# -# Example: -# -# -> { "execute": "block_passwd", "arguments": { "device": "ide0-hd0", -# "password": "12345" } } -# <- { "return": {} } +# This command is now obsolete and will always return an error since 2.10 # ## { 'command': 'block_passwd', 'data': {'*device': 'str', diff --git a/qapi/common.json b/qapi/common.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/common.json +++ b/qapi/common.json @@ -XXX,XX +XXX,XX @@ # # @CommandNotFound: the requested command has not been found # -# @DeviceEncrypted: the requested operation can't be fulfilled because the -# selected device is encrypted -# # @DeviceNotActive: a device has failed to be become active # # @DeviceNotFound: the requested device has not been found @@ -XXX,XX +XXX,XX @@ ## { 'enum': 'QapiErrorClass', # Keep this in sync with ErrorClass in error.h - 'data': [ 'GenericError', 'CommandNotFound', 'DeviceEncrypted', + 'data': [ 'GenericError', 'CommandNotFound', 'DeviceNotActive', 'DeviceNotFound', 'KVMMissingCap' ] } ## -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> While the crypto layer uses a fixed option name "key-secret", the upper block layer may have a prefix on the options. e.g. "encrypt.key-secret", in order to avoid clashes between crypto option names & other block option names. To ensure the crypto layer can report accurate error messages, we must tell it what option name prefix was used. Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-19-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/crypto.c | 4 ++-- block/qcow.c | 7 ++++--- block/qcow2.c | 8 ++++---- crypto/block-luks.c | 8 ++++++-- crypto/block-qcow.c | 8 ++++++-- crypto/block.c | 6 ++++-- crypto/blockpriv.h | 2 ++ include/crypto/block.h | 6 +++++- tests/test-crypto-block.c | 8 ++++---- 9 files changed, 37 insertions(+), 20 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index XXXXXXX..XXXXXXX 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -XXX,XX +XXX,XX @@ static int block_crypto_open_generic(QCryptoBlockFormat format, if (flags & BDRV_O_NO_IO) { cflags |= QCRYPTO_BLOCK_OPEN_NO_IO; } - crypto->block = qcrypto_block_open(open_opts, + crypto->block = qcrypto_block_open(open_opts, NULL, block_crypto_read_func, bs, cflags, @@ -XXX,XX +XXX,XX @@ static int block_crypto_create_generic(QCryptoBlockFormat format, return -1; } - crypto = qcrypto_block_create(create_opts, + crypto = qcrypto_block_create(create_opts, NULL, block_crypto_init_func, block_crypto_write_func, &data, diff --git a/block/qcow.c b/block/qcow.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -XXX,XX +XXX,XX @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags, if (flags & BDRV_O_NO_IO) { cflags |= QCRYPTO_BLOCK_OPEN_NO_IO; } - s->crypto = qcrypto_block_open(crypto_opts, NULL, NULL, - cflags, errp); + s->crypto = qcrypto_block_open(crypto_opts, "encrypt.", + NULL, NULL, cflags, errp); if (!s->crypto) { ret = -EINVAL; goto fail; @@ -XXX,XX +XXX,XX @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp) goto exit; } - crypto = qcrypto_block_create(crypto_opts, NULL, NULL, NULL, errp); + crypto = qcrypto_block_create(crypto_opts, "encrypt.", + NULL, NULL, NULL, errp); if (!crypto) { ret = -EINVAL; goto exit; diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset, if (flags & BDRV_O_NO_IO) { cflags |= QCRYPTO_BLOCK_OPEN_NO_IO; } - s->crypto = qcrypto_block_open(s->crypto_opts, + s->crypto = qcrypto_block_open(s->crypto_opts, "encrypt.", qcow2_crypto_hdr_read_func, bs, cflags, errp); if (!s->crypto) { @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, if (flags & BDRV_O_NO_IO) { cflags |= QCRYPTO_BLOCK_OPEN_NO_IO; } - s->crypto = qcrypto_block_open(s->crypto_opts, NULL, NULL, - cflags, errp); + s->crypto = qcrypto_block_open(s->crypto_opts, "encrypt.", + NULL, NULL, cflags, errp); if (!s->crypto) { ret = -EINVAL; goto fail; @@ -XXX,XX +XXX,XX @@ static int qcow2_set_up_encryption(BlockDriverState *bs, const char *encryptfmt, } s->crypt_method_header = fmt; - crypto = qcrypto_block_create(cryptoopts, + crypto = qcrypto_block_create(cryptoopts, "encrypt.", qcow2_crypto_hdr_init_func, qcow2_crypto_hdr_write_func, bs, errp); diff --git a/crypto/block-luks.c b/crypto/block-luks.c index XXXXXXX..XXXXXXX 100644 --- a/crypto/block-luks.c +++ b/crypto/block-luks.c @@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_find_key(QCryptoBlock *block, static int qcrypto_block_luks_open(QCryptoBlock *block, QCryptoBlockOpenOptions *options, + const char *optprefix, QCryptoBlockReadFunc readfunc, void *opaque, unsigned int flags, @@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_open(QCryptoBlock *block, if (!(flags & QCRYPTO_BLOCK_OPEN_NO_IO)) { if (!options->u.luks.key_secret) { - error_setg(errp, "Parameter 'key-secret' is required for cipher"); + error_setg(errp, "Parameter '%skey-secret' is required for cipher", + optprefix ? optprefix : ""); return -1; } password = qcrypto_secret_lookup_as_utf8( @@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_uuid_gen(uint8_t *uuidstr) static int qcrypto_block_luks_create(QCryptoBlock *block, QCryptoBlockCreateOptions *options, + const char *optprefix, QCryptoBlockInitFunc initfunc, QCryptoBlockWriteFunc writefunc, void *opaque, @@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_create(QCryptoBlock *block, * be silently ignored, for compatibility with dm-crypt */ if (!options->u.luks.key_secret) { - error_setg(errp, "Parameter 'key-secret' is required for cipher"); + error_setg(errp, "Parameter '%skey-secret' is required for cipher", + optprefix ? optprefix : ""); return -1; } password = qcrypto_secret_lookup_as_utf8(luks_opts.key_secret, errp); diff --git a/crypto/block-qcow.c b/crypto/block-qcow.c index XXXXXXX..XXXXXXX 100644 --- a/crypto/block-qcow.c +++ b/crypto/block-qcow.c @@ -XXX,XX +XXX,XX @@ qcrypto_block_qcow_init(QCryptoBlock *block, static int qcrypto_block_qcow_open(QCryptoBlock *block, QCryptoBlockOpenOptions *options, + const char *optprefix, QCryptoBlockReadFunc readfunc G_GNUC_UNUSED, void *opaque G_GNUC_UNUSED, unsigned int flags, @@ -XXX,XX +XXX,XX @@ qcrypto_block_qcow_open(QCryptoBlock *block, } else { if (!options->u.qcow.key_secret) { error_setg(errp, - "Parameter 'key-secret' is required for cipher"); + "Parameter '%skey-secret' is required for cipher", + optprefix ? optprefix : ""); return -1; } return qcrypto_block_qcow_init(block, @@ -XXX,XX +XXX,XX @@ qcrypto_block_qcow_open(QCryptoBlock *block, static int qcrypto_block_qcow_create(QCryptoBlock *block, QCryptoBlockCreateOptions *options, + const char *optprefix, QCryptoBlockInitFunc initfunc G_GNUC_UNUSED, QCryptoBlockWriteFunc writefunc G_GNUC_UNUSED, void *opaque G_GNUC_UNUSED, Error **errp) { if (!options->u.qcow.key_secret) { - error_setg(errp, "Parameter 'key-secret' is required for cipher"); + error_setg(errp, "Parameter '%skey-secret' is required for cipher", + optprefix ? optprefix : ""); return -1; } /* QCow2 has no special header, since everything is hardwired */ diff --git a/crypto/block.c b/crypto/block.c index XXXXXXX..XXXXXXX 100644 --- a/crypto/block.c +++ b/crypto/block.c @@ -XXX,XX +XXX,XX @@ bool qcrypto_block_has_format(QCryptoBlockFormat format, QCryptoBlock *qcrypto_block_open(QCryptoBlockOpenOptions *options, + const char *optprefix, QCryptoBlockReadFunc readfunc, void *opaque, unsigned int flags, @@ -XXX,XX +XXX,XX @@ QCryptoBlock *qcrypto_block_open(QCryptoBlockOpenOptions *options, block->driver = qcrypto_block_drivers[options->format]; - if (block->driver->open(block, options, + if (block->driver->open(block, options, optprefix, readfunc, opaque, flags, errp) < 0) { g_free(block); return NULL; @@ -XXX,XX +XXX,XX @@ QCryptoBlock *qcrypto_block_open(QCryptoBlockOpenOptions *options, QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options, + const char *optprefix, QCryptoBlockInitFunc initfunc, QCryptoBlockWriteFunc writefunc, void *opaque, @@ -XXX,XX +XXX,XX @@ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options, block->driver = qcrypto_block_drivers[options->format]; - if (block->driver->create(block, options, initfunc, + if (block->driver->create(block, options, optprefix, initfunc, writefunc, opaque, errp) < 0) { g_free(block); return NULL; diff --git a/crypto/blockpriv.h b/crypto/blockpriv.h index XXXXXXX..XXXXXXX 100644 --- a/crypto/blockpriv.h +++ b/crypto/blockpriv.h @@ -XXX,XX +XXX,XX @@ struct QCryptoBlock { struct QCryptoBlockDriver { int (*open)(QCryptoBlock *block, QCryptoBlockOpenOptions *options, + const char *optprefix, QCryptoBlockReadFunc readfunc, void *opaque, unsigned int flags, @@ -XXX,XX +XXX,XX @@ struct QCryptoBlockDriver { int (*create)(QCryptoBlock *block, QCryptoBlockCreateOptions *options, + const char *optprefix, QCryptoBlockInitFunc initfunc, QCryptoBlockWriteFunc writefunc, void *opaque, diff --git a/include/crypto/block.h b/include/crypto/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/crypto/block.h +++ b/include/crypto/block.h @@ -XXX,XX +XXX,XX @@ typedef enum { /** * qcrypto_block_open: * @options: the encryption options + * @optprefix: name prefix for options * @readfunc: callback for reading data from the volume * @opaque: data to pass to @readfunc * @flags: bitmask of QCryptoBlockOpenFlags values @@ -XXX,XX +XXX,XX @@ typedef enum { * Returns: a block encryption format, or NULL on error */ QCryptoBlock *qcrypto_block_open(QCryptoBlockOpenOptions *options, + const char *optprefix, QCryptoBlockReadFunc readfunc, void *opaque, unsigned int flags, @@ -XXX,XX +XXX,XX @@ QCryptoBlock *qcrypto_block_open(QCryptoBlockOpenOptions *options, /** * qcrypto_block_create: - * @format: the encryption format + * @options: the encryption options + * @optprefix: name prefix for options * @initfunc: callback for initializing volume header * @writefunc: callback for writing data to the volume header * @opaque: data to pass to @initfunc and @writefunc @@ -XXX,XX +XXX,XX @@ QCryptoBlock *qcrypto_block_open(QCryptoBlockOpenOptions *options, * Returns: a block encryption format, or NULL on error */ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options, + const char *optprefix, QCryptoBlockInitFunc initfunc, QCryptoBlockWriteFunc writefunc, void *opaque, diff --git a/tests/test-crypto-block.c b/tests/test-crypto-block.c index XXXXXXX..XXXXXXX 100644 --- a/tests/test-crypto-block.c +++ b/tests/test-crypto-block.c @@ -XXX,XX +XXX,XX @@ static void test_block(gconstpointer opaque) memset(&header, 0, sizeof(header)); buffer_init(&header, "header"); - blk = qcrypto_block_create(data->create_opts, + blk = qcrypto_block_create(data->create_opts, NULL, test_block_init_func, test_block_write_func, &header, @@ -XXX,XX +XXX,XX @@ static void test_block(gconstpointer opaque) object_unparent(sec); /* Ensure we can't open without the secret */ - blk = qcrypto_block_open(data->open_opts, + blk = qcrypto_block_open(data->open_opts, NULL, test_block_read_func, &header, 0, @@ -XXX,XX +XXX,XX @@ static void test_block(gconstpointer opaque) g_assert(blk == NULL); /* Ensure we can't open without the secret, unless NO_IO */ - blk = qcrypto_block_open(data->open_opts, + blk = qcrypto_block_open(data->open_opts, NULL, test_block_read_func, &header, QCRYPTO_BLOCK_OPEN_NO_IO, @@ -XXX,XX +XXX,XX @@ static void test_block(gconstpointer opaque) /* Now open for real with secret */ sec = test_block_secret(); - blk = qcrypto_block_open(data->open_opts, + blk = qcrypto_block_open(data->open_opts, NULL, test_block_read_func, &header, 0, -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Currently 'qemu-img info' reports a simple "encrypted: yes" field. This is not very useful now that qcow2 can support multiple encryption formats. Users want to know which format is in use and some data related to it. Wire up usage of the qcrypto_block_get_info() method so that 'qemu-img info' can report about the encryption format and parameters in use $ qemu-img create \ --object secret,id=sec0,data=123456 \ -o encrypt.format=luks,encrypt.key-secret=sec0 \ -f qcow2 demo.qcow2 1G Formatting 'demo.qcow2', fmt=qcow2 size=1073741824 \ encryption=off encrypt.format=luks encrypt.key-secret=sec0 \ cluster_size=65536 lazy_refcounts=off refcount_bits=16 $ qemu-img info demo.qcow2 image: demo.qcow2 file format: qcow2 virtual size: 1.0G (1073741824 bytes) disk size: 480K encrypted: yes cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 encrypt: ivgen alg: plain64 hash alg: sha256 cipher alg: aes-256 uuid: 3fa930c4-58c8-4ef7-b3c5-314bb5af21f3 format: luks cipher mode: xts slots: [0]: active: true iters: 1839058 key offset: 4096 stripes: 4000 [1]: active: false key offset: 262144 [2]: active: false key offset: 520192 [3]: active: false key offset: 778240 [4]: active: false key offset: 1036288 [5]: active: false key offset: 1294336 [6]: active: false key offset: 1552384 [7]: active: false key offset: 1810432 payload offset: 2068480 master key iters: 438487 corrupt: false With the legacy "AES" encryption we just report the format name $ qemu-img create \ --object secret,id=sec0,data=123456 \ -o encrypt.format=aes,encrypt.key-secret=sec0 \ -f qcow2 demo.qcow2 1G Formatting 'demo.qcow2', fmt=qcow2 size=1073741824 \ encryption=off encrypt.format=aes encrypt.key-secret=sec0 \ cluster_size=65536 lazy_refcounts=off refcount_bits=16 $ ./qemu-img info demo.qcow2 image: demo.qcow2 file format: qcow2 virtual size: 1.0G (1073741824 bytes) disk size: 196K encrypted: yes cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 encrypt: format: aes corrupt: false Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-20-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2.c | 32 +++++++++++++++++++++++++++++++- qapi/block-core.json | 27 ++++++++++++++++++++++++++- 2 files changed, 57 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int qcow2_get_info(BlockDriverState *bs, BlockDriverInfo *bdi) static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; - ImageInfoSpecific *spec_info = g_new(ImageInfoSpecific, 1); + ImageInfoSpecific *spec_info; + QCryptoBlockInfo *encrypt_info = NULL; + if (s->crypto != NULL) { + encrypt_info = qcrypto_block_get_info(s->crypto, &error_abort); + } + + spec_info = g_new(ImageInfoSpecific, 1); *spec_info = (ImageInfoSpecific){ .type = IMAGE_INFO_SPECIFIC_KIND_QCOW2, .u.qcow2.data = g_new(ImageInfoSpecificQCow2, 1), @@ -XXX,XX +XXX,XX @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs) assert(false); } + if (encrypt_info) { + ImageInfoSpecificQCow2Encryption *qencrypt = + g_new(ImageInfoSpecificQCow2Encryption, 1); + switch (encrypt_info->format) { + case Q_CRYPTO_BLOCK_FORMAT_QCOW: + qencrypt->format = BLOCKDEV_QCOW2_ENCRYPTION_FORMAT_AES; + qencrypt->u.aes = encrypt_info->u.qcow; + break; + case Q_CRYPTO_BLOCK_FORMAT_LUKS: + qencrypt->format = BLOCKDEV_QCOW2_ENCRYPTION_FORMAT_LUKS; + qencrypt->u.luks = encrypt_info->u.luks; + break; + default: + abort(); + } + /* Since we did shallow copy above, erase any pointers + * in the original info */ + memset(&encrypt_info->u, 0, sizeof(encrypt_info->u)); + qapi_free_QCryptoBlockInfo(encrypt_info); + + spec_info->u.qcow2.data->has_encrypt = true; + spec_info->u.qcow2.data->encrypt = qencrypt; + } + return spec_info; } diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ 'vm-clock-sec': 'int', 'vm-clock-nsec': 'int' } } ## +# @ImageInfoSpecificQCow2EncryptionBase: +# +# @format: The encryption format +# +# Since: 2.10 +## +{ 'struct': 'ImageInfoSpecificQCow2EncryptionBase', + 'data': { 'format': 'BlockdevQcow2EncryptionFormat'}} + +## +# @ImageInfoSpecificQCow2Encryption: +# +# Since: 2.10 +## +{ 'union': 'ImageInfoSpecificQCow2Encryption', + 'base': 'ImageInfoSpecificQCow2EncryptionBase', + 'discriminator': 'format', + 'data': { 'aes': 'QCryptoBlockInfoQCow', + 'luks': 'QCryptoBlockInfoLUKS' } } + +## # @ImageInfoSpecificQCow2: # # @compat: compatibility level @@ -XXX,XX +XXX,XX @@ # # @refcount-bits: width of a refcount entry in bits (since 2.3) # +# @encrypt: details about encryption parameters; only set if image +# is encrypted (since 2.10) +# # Since: 1.7 ## { 'struct': 'ImageInfoSpecificQCow2', @@ -XXX,XX +XXX,XX @@ 'compat': 'str', '*lazy-refcounts': 'bool', '*corrupt': 'bool', - 'refcount-bits': 'int' + 'refcount-bits': 'int', + '*encrypt': 'ImageInfoSpecificQCow2Encryption' } } ## -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Expand the image format docs to cover the new options for the qcow, qcow2 and luks disk image formats Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170623162419.26068-21-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- qemu-doc.texi | 123 ++++++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 115 insertions(+), 8 deletions(-) diff --git a/qemu-doc.texi b/qemu-doc.texi index XXXXXXX..XXXXXXX 100644 --- a/qemu-doc.texi +++ b/qemu-doc.texi @@ -XXX,XX +XXX,XX @@ File name of a base image (see @option{create} subcommand) @item backing_fmt Image format of the base image @item encryption -If this option is set to @code{on}, the image is encrypted with 128-bit AES-CBC. +This option is deprecated and equivalent to @code{encrypt.format=aes} -The use of encryption in qcow and qcow2 images is considered to be flawed by -modern cryptography standards, suffering from a number of design problems: +@item encrypt.format + +If this is set to @code{luks}, it requests that the qcow2 payload (not +qcow2 header) be encrypted using the LUKS format. The passphrase to +use to unlock the LUKS key slot is given by the @code{encrypt.key-secret} +parameter. LUKS encryption parameters can be tuned with the other +@code{encrypt.*} parameters. + +If this is set to @code{aes}, the image is encrypted with 128-bit AES-CBC. +The encryption key is given by the @code{encrypt.key-secret} parameter. +This encryption format is considered to be flawed by modern cryptography +standards, suffering from a number of design problems: @itemize @minus @item The AES-CBC cipher is used with predictable initialization vectors based @@ -XXX,XX +XXX,XX @@ original file must then be securely erased using a program like shred, though even this is ineffective with many modern storage technologies. @end itemize -Use of qcow / qcow2 encryption with QEMU is deprecated, and support for -it will go away in a future release. Users are recommended to use an -alternative encryption technology such as the Linux dm-crypt / LUKS -system. +The use of this is no longer supported in system emulators. Support only +remains in the command line utilities, for the purposes of data liberation +and interoperability with old versions of QEMU. The @code{luks} format +should be used instead. + +@item encrypt.key-secret + +Provides the ID of a @code{secret} object that contains the passphrase +(@code{encrypt.format=luks}) or encryption key (@code{encrypt.format=aes}). + +@item encrypt.cipher-alg + +Name of the cipher algorithm and key length. Currently defaults +to @code{aes-256}. Only used when @code{encrypt.format=luks}. + +@item encrypt.cipher-mode + +Name of the encryption mode to use. Currently defaults to @code{xts}. +Only used when @code{encrypt.format=luks}. + +@item encrypt.ivgen-alg + +Name of the initialization vector generator algorithm. Currently defaults +to @code{plain64}. Only used when @code{encrypt.format=luks}. + +@item encrypt.ivgen-hash-alg + +Name of the hash algorithm to use with the initialization vector generator +(if required). Defaults to @code{sha256}. Only used when @code{encrypt.format=luks}. + +@item encrypt.hash-alg + +Name of the hash algorithm to use for PBKDF algorithm +Defaults to @code{sha256}. Only used when @code{encrypt.format=luks}. + +@item encrypt.iter-time + +Amount of time, in milliseconds, to use for PBKDF algorithm per key slot. +Defaults to @code{2000}. Only used when @code{encrypt.format=luks}. @item cluster_size Changes the qcow2 cluster size (must be between 512 and 2M). Smaller cluster @@ -XXX,XX +XXX,XX @@ Supported options: @item backing_file File name of a base image (see @option{create} subcommand) @item encryption -If this option is set to @code{on}, the image is encrypted. +This option is deprecated and equivalent to @code{encrypt.format=aes} + +@item encrypt.format +If this is set to @code{aes}, the image is encrypted with 128-bit AES-CBC. +The encryption key is given by the @code{encrypt.key-secret} parameter. +This encryption format is considered to be flawed by modern cryptography +standards, suffering from a number of design problems enumerated previously +against the @code{qcow2} image format. + +The use of this is no longer supported in system emulators. Support only +remains in the command line utilities, for the purposes of data liberation +and interoperability with old versions of QEMU. + +Users requiring native encryption should use the @code{qcow2} format +instead with @code{encrypt.format=luks}. + +@item encrypt.key-secret + +Provides the ID of a @code{secret} object that contains the encryption +key (@code{encrypt.format=aes}). + +@end table + +@item luks + +LUKS v1 encryption format, compatible with Linux dm-crypt/cryptsetup + +Supported options: +@table @code + +@item key-secret + +Provides the ID of a @code{secret} object that contains the passphrase. + +@item cipher-alg + +Name of the cipher algorithm and key length. Currently defaults +to @code{aes-256}. + +@item cipher-mode + +Name of the encryption mode to use. Currently defaults to @code{xts}. + +@item ivgen-alg + +Name of the initialization vector generator algorithm. Currently defaults +to @code{plain64}. + +@item ivgen-hash-alg + +Name of the hash algorithm to use with the initialization vector generator +(if required). Defaults to @code{sha256}. + +@item hash-alg + +Name of the hash algorithm to use for PBKDF algorithm +Defaults to @code{sha256}. + +@item iter-time + +Amount of time, in milliseconds, to use for PBKDF algorithm per key slot. +Defaults to @code{2000}. + @end table @item vdi -- 1.8.3.1 From: Max Reitz <mreitz@redhat.com> Test 181 only works for formats which support live migration (naturally, as it is a live migration test). Disable it for all formats which do not. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20170621131157.16584-1-mreitz@redhat.com Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/181 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/181 b/tests/qemu-iotests/181 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/181 +++ b/tests/qemu-iotests/181 @@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt generic +# Formats that do not support live migration +_unsupported_fmt qcow vdi vhdx vmdk vpc vvfat _supported_proto generic _supported_os Linux -- 1.8.3.1 From: "sochin.jiang" <sochin.jiang@huawei.com> mirror_complete opens the backing chain, which should have the same AioContext as the top when using iothreads. Make the code guarantee this, which fixes a failed assertion in bdrv_attach_child. Signed-off-by: sochin.jiang <sochin.jiang@huawei.com> Message-id: 1498475064-39816-1-git-send-email-sochin.jiang@huawei.com [mreitz: Reworded commit message] Signed-off-by: Max Reitz <mreitz@redhat.com> --- block.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options, ret = -EINVAL; goto free_exit; } + bdrv_set_aio_context(backing_hd, bdrv_get_aio_context(bs)); /* Hook up the backing file link; drop our reference, bs owns the * backing_hd reference now */ -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-2-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- docs/interop/qcow2.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index XXXXXXX..XXXXXXX 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -XXX,XX +XXX,XX @@ Structure of a bitmap directory entry: 17: granularity_bits Granularity bits. Valid values: 0 - 63. - Note: Qemu currently doesn't support granularity_bits - greater than 31. + Note: Qemu currently supports only values 9 - 31. Granularity is calculated as granularity = 1 << granularity_bits -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> A bitmap directory entry is sometimes called a 'bitmap header'. This patch leaves only one name - 'bitmap directory entry'. The name 'bitmap header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap header extension' (which is extension of qcow2 header) Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-3-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- docs/interop/qcow2.txt | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index XXXXXXX..XXXXXXX 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -XXX,XX +XXX,XX @@ The fields of the bitmaps extension are: 8 - 15: bitmap_directory_size Size of the bitmap directory in bytes. It is the cumulative - size of all (nb_bitmaps) bitmap headers. + size of all (nb_bitmaps) bitmap directory entries. 16 - 23: bitmap_directory_offset Offset into the image file at which the bitmap directory @@ -XXX,XX +XXX,XX @@ Each bitmap saved in the image is described in a bitmap directory entry. The bitmap directory is a contiguous area in the image file, whose starting offset and length are given by the header extension fields bitmap_directory_offset and bitmap_directory_size. The entries of the bitmap directory have variable -length, depending on the lengths of the bitmap name and extra data. These -entries are also called bitmap headers. +length, depending on the lengths of the bitmap name and extra data. Structure of a bitmap directory entry: -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Make dirty iter resistant to resetting bits in corresponding HBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-4-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- include/qemu/hbitmap.h | 26 ++++---------------------- util/hbitmap.c | 23 ++++++++++++++++++++++- 2 files changed, 26 insertions(+), 23 deletions(-) diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -XXX,XX +XXX,XX @@ void hbitmap_free(HBitmap *hb); * the lowest-numbered bit that is set in @hb, starting at @first. * * Concurrent setting of bits is acceptable, and will at worst cause the - * iteration to miss some of those bits. Resetting bits before the current - * position of the iterator is also okay. However, concurrent resetting of - * bits can lead to unexpected behavior if the iterator has not yet reached - * those bits. + * iteration to miss some of those bits. + * + * The concurrent resetting of bits is OK. */ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first); @@ -XXX,XX +XXX,XX @@ void hbitmap_free_meta(HBitmap *hb); * Return the next bit that is set in @hbi's associated HBitmap, * or -1 if all remaining bits are zero. */ -static inline int64_t hbitmap_iter_next(HBitmapIter *hbi) -{ - unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1]; - int64_t item; - - if (cur == 0) { - cur = hbitmap_iter_skip_words(hbi); - if (cur == 0) { - return -1; - } - } - - /* The next call will resume work from the next bit. */ - hbi->cur[HBITMAP_LEVELS - 1] = cur & (cur - 1); - item = ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur); - - return item << hbi->granularity; -} +int64_t hbitmap_iter_next(HBitmapIter *hbi); /** * hbitmap_iter_next_word: diff --git a/util/hbitmap.c b/util/hbitmap.c index XXXXXXX..XXXXXXX 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -XXX,XX +XXX,XX @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi) unsigned long cur; do { - cur = hbi->cur[--i]; + i--; pos >>= BITS_PER_LEVEL; + cur = hbi->cur[i] & hb->levels[i][pos]; } while (cur == 0); /* Check for end of iteration. We always use fewer than BITS_PER_LONG @@ -XXX,XX +XXX,XX @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi) return cur; } +int64_t hbitmap_iter_next(HBitmapIter *hbi) +{ + unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1] & + hbi->hb->levels[HBITMAP_LEVELS - 1][hbi->pos]; + int64_t item; + + if (cur == 0) { + cur = hbitmap_iter_skip_words(hbi); + if (cur == 0) { + return -1; + } + } + + /* The next call will resume work from the next bit. */ + hbi->cur[HBITMAP_LEVELS - 1] = cur & (cur - 1); + item = ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur); + + return item << hbi->granularity; +} + void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first) { unsigned i, bit; -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-5-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/test-hbitmap.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c index XXXXXXX..XXXXXXX 100644 --- a/tests/test-hbitmap.c +++ b/tests/test-hbitmap.c @@ -XXX,XX +XXX,XX @@ static void hbitmap_test_add(const char *testpath, hbitmap_test_teardown); } +static void test_hbitmap_iter_and_reset(TestHBitmapData *data, + const void *unused) +{ + HBitmapIter hbi; + + hbitmap_test_init(data, L1 * 2, 0); + hbitmap_set(data->hb, 0, data->size); + + hbitmap_iter_init(&hbi, data->hb, BITS_PER_LONG - 1); + + hbitmap_iter_next(&hbi); + + hbitmap_reset_all(data->hb); + hbitmap_iter_next(&hbi); +} + int main(int argc, char **argv) { g_test_init(&argc, &argv, NULL); @@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv) test_hbitmap_serialize_part); hbitmap_test_add("/hbitmap/serialize/zeroes", test_hbitmap_serialize_zeroes); + + hbitmap_test_add("/hbitmap/iter/iter_and_reset", + test_hbitmap_iter_and_reset); g_test_run(); return 0; -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Make getter signature const-correct. This allows other functions with const dirty bitmap parameter use bdrv_dirty_bitmap_granularity(). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20170628120530.31251-6-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/dirty-bitmap.c | 2 +- include/block/dirty-bitmap.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs) return granularity; } -uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap) +uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirtyBitmap *bitmap) { return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap); } diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -XXX,XX +XXX,XX @@ void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap); void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap); BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs); uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs); -uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap); +uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirtyBitmap *bitmap); uint32_t bdrv_dirty_bitmap_meta_granularity(BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap); -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-7-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/dirty-bitmap.c | 7 +++++++ include/block/dirty-bitmap.h | 3 +++ include/qemu/hbitmap.h | 15 +++++++++++++++ util/hbitmap.c | 17 +++++++++++++++++ 4 files changed, 42 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap, hbitmap_deserialize_zeroes(bitmap->bitmap, start, count, finish); } +void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap, + uint64_t start, uint64_t count, + bool finish) +{ + hbitmap_deserialize_ones(bitmap->bitmap, start, count, finish); +} + void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap) { hbitmap_deserialize_finish(bitmap->bitmap); diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap, void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap, uint64_t start, uint64_t count, bool finish); +void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap, + uint64_t start, uint64_t count, + bool finish); void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap); /* Functions that require manual locking. */ diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -XXX,XX +XXX,XX @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count, bool finish); /** + * hbitmap_deserialize_ones + * @hb: HBitmap to operate on. + * @start: First bit to restore. + * @count: Number of bits to restore. + * @finish: Whether to call hbitmap_deserialize_finish automatically. + * + * Fills the bitmap with ones. + * + * If @finish is false, caller must call hbitmap_serialize_finish before using + * the bitmap. + */ +void hbitmap_deserialize_ones(HBitmap *hb, uint64_t start, uint64_t count, + bool finish); + +/** * hbitmap_deserialize_finish * @hb: HBitmap to operate on. * diff --git a/util/hbitmap.c b/util/hbitmap.c index XXXXXXX..XXXXXXX 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -XXX,XX +XXX,XX @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count, } } +void hbitmap_deserialize_ones(HBitmap *hb, uint64_t start, uint64_t count, + bool finish) +{ + uint64_t el_count; + unsigned long *first; + + if (!count) { + return; + } + serialization_chunk(hb, start, count, &first, &el_count); + + memset(first, 0xff, el_count * sizeof(unsigned long)); + if (finish) { + hbitmap_deserialize_finish(hb); + } +} + void hbitmap_deserialize_finish(HBitmap *bitmap) { int64_t i, size, prev_size; -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> This is needed for the following patch, which will introduce refcounts checking for qcow2 bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-8-vsementsov@virtuozzo.com [mreitz: s/inc_refcounts/qcow2_inc_refcounts_imrt/ in one more (new) place] Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-refcount.c | 59 ++++++++++++++++++++++++++------------------------ block/qcow2.h | 4 ++++ 2 files changed, 35 insertions(+), 28 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -XXX,XX +XXX,XX @@ static int realloc_refcount_array(BDRVQcow2State *s, void **array, * * Modifies the number of errors in res. */ -static int inc_refcounts(BlockDriverState *bs, - BdrvCheckResult *res, - void **refcount_table, - int64_t *refcount_table_size, - int64_t offset, int64_t size) +int qcow2_inc_refcounts_imrt(BlockDriverState *bs, BdrvCheckResult *res, + void **refcount_table, + int64_t *refcount_table_size, + int64_t offset, int64_t size) { BDRVQcow2State *s = bs->opaque; uint64_t start, last, cluster_offset, k, refcount; @@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, nb_csectors = ((l2_entry >> s->csize_shift) & s->csize_mask) + 1; l2_entry &= s->cluster_offset_mask; - ret = inc_refcounts(bs, res, refcount_table, refcount_table_size, - l2_entry & ~511, nb_csectors * 512); + ret = qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_size, + l2_entry & ~511, nb_csectors * 512); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, } /* Mark cluster as used */ - ret = inc_refcounts(bs, res, refcount_table, refcount_table_size, - offset, s->cluster_size); + ret = qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_size, + offset, s->cluster_size); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ static int check_refcounts_l1(BlockDriverState *bs, l1_size2 = l1_size * sizeof(uint64_t); /* Mark L1 table as used */ - ret = inc_refcounts(bs, res, refcount_table, refcount_table_size, - l1_table_offset, l1_size2); + ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, refcount_table_size, + l1_table_offset, l1_size2); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ static int check_refcounts_l1(BlockDriverState *bs, if (l2_offset) { /* Mark L2 table as used */ l2_offset &= L1E_OFFSET_MASK; - ret = inc_refcounts(bs, res, refcount_table, refcount_table_size, - l2_offset, s->cluster_size); + ret = qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_size, + l2_offset, s->cluster_size); if (ret < 0) { goto fail; } @@ -XXX,XX +XXX,XX @@ static int check_refblocks(BlockDriverState *bs, BdrvCheckResult *res, } res->corruptions_fixed++; - ret = inc_refcounts(bs, res, refcount_table, nb_clusters, - offset, s->cluster_size); + ret = qcow2_inc_refcounts_imrt(bs, res, + refcount_table, nb_clusters, + offset, s->cluster_size); if (ret < 0) { return ret; } /* No need to check whether the refcount is now greater than 1: * This area was just allocated and zeroed, so it can only be - * exactly 1 after inc_refcounts() */ + * exactly 1 after qcow2_inc_refcounts_imrt() */ continue; resize_fail: @@ -XXX,XX +XXX,XX @@ resize_fail: } if (offset != 0) { - ret = inc_refcounts(bs, res, refcount_table, nb_clusters, - offset, s->cluster_size); + ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_clusters, + offset, s->cluster_size); if (ret < 0) { return ret; } @@ -XXX,XX +XXX,XX @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res, } /* header */ - ret = inc_refcounts(bs, res, refcount_table, nb_clusters, - 0, s->cluster_size); + ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_clusters, + 0, s->cluster_size); if (ret < 0) { return ret; } @@ -XXX,XX +XXX,XX @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res, return ret; } } - ret = inc_refcounts(bs, res, refcount_table, nb_clusters, - s->snapshots_offset, s->snapshots_size); + ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_clusters, + s->snapshots_offset, s->snapshots_size); if (ret < 0) { return ret; } /* refcount data */ - ret = inc_refcounts(bs, res, refcount_table, nb_clusters, - s->refcount_table_offset, - s->refcount_table_size * sizeof(uint64_t)); + ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_clusters, + s->refcount_table_offset, + s->refcount_table_size * sizeof(uint64_t)); if (ret < 0) { return ret; } /* encryption */ if (s->crypto_header.length) { - ret = inc_refcounts(bs, res, refcount_table, nb_clusters, - s->crypto_header.offset, - s->crypto_header.length); + ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_clusters, + s->crypto_header.offset, + s->crypto_header.length); if (ret < 0) { return ret; } diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset, int64_t size); int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset, int64_t size); +int qcow2_inc_refcounts_imrt(BlockDriverState *bs, BdrvCheckResult *res, + void **refcount_table, + int64_t *refcount_table_size, + int64_t offset, int64_t size); int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order, BlockDriverAmendStatusCB *status_cb, -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Add bitmap extension as specified in docs/specs/qcow2.txt. For now, just mirror extension header into Qcow2 state and check constraints. Also, calculate refcounts for qcow2 bitmaps, to not break qemu-img check. For now, disable image resize if it has bitmaps. It will be fixed later. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-9-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/Makefile.objs | 2 +- block/qcow2-bitmap.c | 439 +++++++++++++++++++++++++++++++++++++++++++++++++ block/qcow2-refcount.c | 6 + block/qcow2.c | 125 +++++++++++++- block/qcow2.h | 27 +++ 5 files changed, 593 insertions(+), 6 deletions(-) create mode 100644 block/qcow2-bitmap.c diff --git a/block/Makefile.objs b/block/Makefile.objs index XXXXXXX..XXXXXXX 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -XXX,XX +XXX,XX @@ block-obj-y += raw-format.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vvfat.o dmg.o -block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o +block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o qcow2-bitmap.o block-obj-y += qed.o qed-l2-cache.o qed-table.o qed-cluster.o block-obj-y += qed-check.o block-obj-y += vhdx.o vhdx-endian.o vhdx-log.o diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ +/* + * Bitmaps for the QCOW version 2 format + * + * Copyright (c) 2014-2017 Vladimir Sementsov-Ogievskiy + * + * This file is derived from qcow2-snapshot.c, original copyright: + * Copyright (c) 2004-2006 Fabrice Bellard + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" + +#include "block/block_int.h" +#include "block/qcow2.h" + +/* NOTICE: BME here means Bitmaps Extension and used as a namespace for + * _internal_ constants. Please do not use this _internal_ abbreviation for + * other needs and/or outside of this file. */ + +/* Bitmap directory entry constraints */ +#define BME_MAX_TABLE_SIZE 0x8000000 +#define BME_MAX_PHYS_SIZE 0x20000000 /* restrict BdrvDirtyBitmap size in RAM */ +#define BME_MAX_GRANULARITY_BITS 31 +#define BME_MIN_GRANULARITY_BITS 9 +#define BME_MAX_NAME_SIZE 1023 + +/* Bitmap directory entry flags */ +#define BME_RESERVED_FLAGS 0xfffffffcU + +/* bits [1, 8] U [56, 63] are reserved */ +#define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001feULL +#define BME_TABLE_ENTRY_OFFSET_MASK 0x00fffffffffffe00ULL +#define BME_TABLE_ENTRY_FLAG_ALL_ONES (1ULL << 0) + +typedef struct QEMU_PACKED Qcow2BitmapDirEntry { + /* header is 8 byte aligned */ + uint64_t bitmap_table_offset; + + uint32_t bitmap_table_size; + uint32_t flags; + + uint8_t type; + uint8_t granularity_bits; + uint16_t name_size; + uint32_t extra_data_size; + /* extra data follows */ + /* name follows */ +} Qcow2BitmapDirEntry; + +typedef struct Qcow2BitmapTable { + uint64_t offset; + uint32_t size; /* number of 64bit entries */ + QSIMPLEQ_ENTRY(Qcow2BitmapTable) entry; +} Qcow2BitmapTable; + +typedef struct Qcow2Bitmap { + Qcow2BitmapTable table; + uint32_t flags; + uint8_t granularity_bits; + char *name; + + QSIMPLEQ_ENTRY(Qcow2Bitmap) entry; +} Qcow2Bitmap; +typedef QSIMPLEQ_HEAD(Qcow2BitmapList, Qcow2Bitmap) Qcow2BitmapList; + +typedef enum BitmapType { + BT_DIRTY_TRACKING_BITMAP = 1 +} BitmapType; + +static int check_table_entry(uint64_t entry, int cluster_size) +{ + uint64_t offset; + + if (entry & BME_TABLE_ENTRY_RESERVED_MASK) { + return -EINVAL; + } + + offset = entry & BME_TABLE_ENTRY_OFFSET_MASK; + if (offset != 0) { + /* if offset specified, bit 0 is reserved */ + if (entry & BME_TABLE_ENTRY_FLAG_ALL_ONES) { + return -EINVAL; + } + + if (offset % cluster_size != 0) { + return -EINVAL; + } + } + + return 0; +} + +static int bitmap_table_load(BlockDriverState *bs, Qcow2BitmapTable *tb, + uint64_t **bitmap_table) +{ + int ret; + BDRVQcow2State *s = bs->opaque; + uint32_t i; + uint64_t *table; + + assert(tb->size != 0); + table = g_try_new(uint64_t, tb->size); + if (table == NULL) { + return -ENOMEM; + } + + assert(tb->size <= BME_MAX_TABLE_SIZE); + ret = bdrv_pread(bs->file, tb->offset, + table, tb->size * sizeof(uint64_t)); + if (ret < 0) { + goto fail; + } + + for (i = 0; i < tb->size; ++i) { + be64_to_cpus(&table[i]); + ret = check_table_entry(table[i], s->cluster_size); + if (ret < 0) { + goto fail; + } + } + + *bitmap_table = table; + return 0; + +fail: + g_free(table); + + return ret; +} + +/* + * Bitmap List + */ + +/* + * Bitmap List private functions + * Only Bitmap List knows about bitmap directory structure in Qcow2. + */ + +static inline void bitmap_dir_entry_to_cpu(Qcow2BitmapDirEntry *entry) +{ + be64_to_cpus(&entry->bitmap_table_offset); + be32_to_cpus(&entry->bitmap_table_size); + be32_to_cpus(&entry->flags); + be16_to_cpus(&entry->name_size); + be32_to_cpus(&entry->extra_data_size); +} + +static inline int calc_dir_entry_size(size_t name_size, size_t extra_data_size) +{ + return align_offset(sizeof(Qcow2BitmapDirEntry) + + name_size + extra_data_size, 8); +} + +static inline int dir_entry_size(Qcow2BitmapDirEntry *entry) +{ + return calc_dir_entry_size(entry->name_size, entry->extra_data_size); +} + +static inline const char *dir_entry_name_field(Qcow2BitmapDirEntry *entry) +{ + return (const char *)(entry + 1) + entry->extra_data_size; +} + +static inline char *dir_entry_copy_name(Qcow2BitmapDirEntry *entry) +{ + const char *name_field = dir_entry_name_field(entry); + return g_strndup(name_field, entry->name_size); +} + +static inline Qcow2BitmapDirEntry *next_dir_entry(Qcow2BitmapDirEntry *entry) +{ + return (Qcow2BitmapDirEntry *)((uint8_t *)entry + dir_entry_size(entry)); +} + +static int check_dir_entry(BlockDriverState *bs, Qcow2BitmapDirEntry *entry) +{ + BDRVQcow2State *s = bs->opaque; + uint64_t phys_bitmap_bytes; + int64_t len; + + bool fail = (entry->bitmap_table_size == 0) || + (entry->bitmap_table_offset == 0) || + (entry->bitmap_table_offset % s->cluster_size) || + (entry->bitmap_table_size > BME_MAX_TABLE_SIZE) || + (entry->granularity_bits > BME_MAX_GRANULARITY_BITS) || + (entry->granularity_bits < BME_MIN_GRANULARITY_BITS) || + (entry->flags & BME_RESERVED_FLAGS) || + (entry->name_size > BME_MAX_NAME_SIZE) || + (entry->type != BT_DIRTY_TRACKING_BITMAP); + + if (fail) { + return -EINVAL; + } + + phys_bitmap_bytes = (uint64_t)entry->bitmap_table_size * s->cluster_size; + len = bdrv_getlength(bs); + + if (len < 0) { + return len; + } + + fail = (phys_bitmap_bytes > BME_MAX_PHYS_SIZE) || + (len > ((phys_bitmap_bytes * 8) << entry->granularity_bits)); + + return fail ? -EINVAL : 0; +} + +/* + * Bitmap List public functions + */ + +static void bitmap_free(Qcow2Bitmap *bm) +{ + g_free(bm->name); + g_free(bm); +} + +static void bitmap_list_free(Qcow2BitmapList *bm_list) +{ + Qcow2Bitmap *bm; + + if (bm_list == NULL) { + return; + } + + while ((bm = QSIMPLEQ_FIRST(bm_list)) != NULL) { + QSIMPLEQ_REMOVE_HEAD(bm_list, entry); + bitmap_free(bm); + } + + g_free(bm_list); +} + +static Qcow2BitmapList *bitmap_list_new(void) +{ + Qcow2BitmapList *bm_list = g_new(Qcow2BitmapList, 1); + QSIMPLEQ_INIT(bm_list); + + return bm_list; +} + +/* bitmap_list_load + * Get bitmap list from qcow2 image. Actually reads bitmap directory, + * checks it and convert to bitmap list. + */ +static Qcow2BitmapList *bitmap_list_load(BlockDriverState *bs, uint64_t offset, + uint64_t size, Error **errp) +{ + int ret; + BDRVQcow2State *s = bs->opaque; + uint8_t *dir, *dir_end; + Qcow2BitmapDirEntry *e; + uint32_t nb_dir_entries = 0; + Qcow2BitmapList *bm_list = NULL; + + if (size == 0) { + error_setg(errp, "Requested bitmap directory size is zero"); + return NULL; + } + + if (size > QCOW2_MAX_BITMAP_DIRECTORY_SIZE) { + error_setg(errp, "Requested bitmap directory size is too big"); + return NULL; + } + + dir = g_try_malloc(size); + if (dir == NULL) { + error_setg(errp, "Failed to allocate space for bitmap directory"); + return NULL; + } + dir_end = dir + size; + + ret = bdrv_pread(bs->file, offset, dir, size); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read bitmap directory"); + goto fail; + } + + bm_list = bitmap_list_new(); + for (e = (Qcow2BitmapDirEntry *)dir; + e < (Qcow2BitmapDirEntry *)dir_end; + e = next_dir_entry(e)) + { + Qcow2Bitmap *bm; + + if ((uint8_t *)(e + 1) > dir_end) { + goto broken_dir; + } + + if (++nb_dir_entries > s->nb_bitmaps) { + error_setg(errp, "More bitmaps found than specified in header" + " extension"); + goto fail; + } + bitmap_dir_entry_to_cpu(e); + + if ((uint8_t *)next_dir_entry(e) > dir_end) { + goto broken_dir; + } + + if (e->extra_data_size != 0) { + error_setg(errp, "Bitmap extra data is not supported"); + goto fail; + } + + ret = check_dir_entry(bs, e); + if (ret < 0) { + error_setg(errp, "Bitmap '%.*s' doesn't satisfy the constraints", + e->name_size, dir_entry_name_field(e)); + goto fail; + } + + bm = g_new(Qcow2Bitmap, 1); + bm->table.offset = e->bitmap_table_offset; + bm->table.size = e->bitmap_table_size; + bm->flags = e->flags; + bm->granularity_bits = e->granularity_bits; + bm->name = dir_entry_copy_name(e); + QSIMPLEQ_INSERT_TAIL(bm_list, bm, entry); + } + + if (nb_dir_entries != s->nb_bitmaps) { + error_setg(errp, "Less bitmaps found than specified in header" + " extension"); + goto fail; + } + + if ((uint8_t *)e != dir_end) { + goto broken_dir; + } + + g_free(dir); + return bm_list; + +broken_dir: + ret = -EINVAL; + error_setg(errp, "Broken bitmap directory"); + +fail: + g_free(dir); + bitmap_list_free(bm_list); + + return NULL; +} + +int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res, + void **refcount_table, + int64_t *refcount_table_size) +{ + int ret; + BDRVQcow2State *s = bs->opaque; + Qcow2BitmapList *bm_list; + Qcow2Bitmap *bm; + + if (s->nb_bitmaps == 0) { + return 0; + } + + ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, refcount_table_size, + s->bitmap_directory_offset, + s->bitmap_directory_size); + if (ret < 0) { + return ret; + } + + bm_list = bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, NULL); + if (bm_list == NULL) { + res->corruptions++; + return -EINVAL; + } + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + uint64_t *bitmap_table = NULL; + int i; + + ret = qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_size, + bm->table.offset, + bm->table.size * sizeof(uint64_t)); + if (ret < 0) { + goto out; + } + + ret = bitmap_table_load(bs, &bm->table, &bitmap_table); + if (ret < 0) { + res->corruptions++; + goto out; + } + + for (i = 0; i < bm->table.size; ++i) { + uint64_t entry = bitmap_table[i]; + uint64_t offset = entry & BME_TABLE_ENTRY_OFFSET_MASK; + + if (check_table_entry(entry, s->cluster_size) < 0) { + res->corruptions++; + continue; + } + + if (offset == 0) { + continue; + } + + ret = qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_size, + offset, s->cluster_size); + if (ret < 0) { + g_free(bitmap_table); + goto out; + } + } + + g_free(bitmap_table); + } + +out: + bitmap_list_free(bm_list); + + return ret; +} diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -XXX,XX +XXX,XX @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res, } } + /* bitmaps */ + ret = qcow2_check_bitmaps_refcounts(bs, res, refcount_table, nb_clusters); + if (ret < 0) { + return ret; + } + return check_refblocks(bs, res, fix, rebuild, refcount_table, nb_clusters); } diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ typedef struct { #define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xE2792ACA #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 #define QCOW2_EXT_MAGIC_CRYPTO_HEADER 0x0537be77 +#define QCOW2_EXT_MAGIC_BITMAPS 0x23852875 static int qcow2_probe(const uint8_t *buf, int buf_size, const char *filename) { @@ -XXX,XX +XXX,XX @@ static ssize_t qcow2_crypto_hdr_write_func(QCryptoBlock *block, size_t offset, */ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset, uint64_t end_offset, void **p_feature_table, - int flags, Error **errp) + int flags, bool *need_update_header, + Error **errp) { BDRVQcow2State *s = bs->opaque; QCowExtension ext; uint64_t offset; int ret; + Qcow2BitmapHeaderExt bitmaps_ext; + + if (need_update_header != NULL) { + *need_update_header = false; + } #ifdef DEBUG_EXT printf("qcow2_read_extensions: start=%ld end=%ld\n", start_offset, end_offset); @@ -XXX,XX +XXX,XX @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset, } } break; + case QCOW2_EXT_MAGIC_BITMAPS: + if (ext.len != sizeof(bitmaps_ext)) { + error_setg_errno(errp, -ret, "bitmaps_ext: " + "Invalid extension length"); + return -EINVAL; + } + + if (!(s->autoclear_features & QCOW2_AUTOCLEAR_BITMAPS)) { + error_report("WARNING: a program lacking bitmap support " + "modified this file, so all bitmaps are now " + "considered inconsistent. Some clusters may be " + "leaked, run 'qemu-img check -r' on the image " + "file to fix."); + if (need_update_header != NULL) { + /* Updating is needed to drop invalid bitmap extension. */ + *need_update_header = true; + } + break; + } + + ret = bdrv_pread(bs->file, offset, &bitmaps_ext, ext.len); + if (ret < 0) { + error_setg_errno(errp, -ret, "bitmaps_ext: " + "Could not read ext header"); + return ret; + } + + if (bitmaps_ext.reserved32 != 0) { + error_setg_errno(errp, -ret, "bitmaps_ext: " + "Reserved field is not zero"); + return -EINVAL; + } + + be32_to_cpus(&bitmaps_ext.nb_bitmaps); + be64_to_cpus(&bitmaps_ext.bitmap_directory_size); + be64_to_cpus(&bitmaps_ext.bitmap_directory_offset); + + if (bitmaps_ext.nb_bitmaps > QCOW2_MAX_BITMAPS) { + error_setg(errp, + "bitmaps_ext: Image has %" PRIu32 " bitmaps, " + "exceeding the QEMU supported maximum of %d", + bitmaps_ext.nb_bitmaps, QCOW2_MAX_BITMAPS); + return -EINVAL; + } + + if (bitmaps_ext.nb_bitmaps == 0) { + error_setg(errp, "found bitmaps extension with zero bitmaps"); + return -EINVAL; + } + + if (bitmaps_ext.bitmap_directory_offset & (s->cluster_size - 1)) { + error_setg(errp, "bitmaps_ext: " + "invalid bitmap directory offset"); + return -EINVAL; + } + + if (bitmaps_ext.bitmap_directory_size > + QCOW2_MAX_BITMAP_DIRECTORY_SIZE) { + error_setg(errp, "bitmaps_ext: " + "bitmap directory size (%" PRIu64 ") exceeds " + "the maximum supported size (%d)", + bitmaps_ext.bitmap_directory_size, + QCOW2_MAX_BITMAP_DIRECTORY_SIZE); + return -EINVAL; + } + + s->nb_bitmaps = bitmaps_ext.nb_bitmaps; + s->bitmap_directory_offset = + bitmaps_ext.bitmap_directory_offset; + s->bitmap_directory_size = + bitmaps_ext.bitmap_directory_size; + +#ifdef DEBUG_EXT + printf("Qcow2: Got bitmaps extension: " + "offset=%" PRIu64 " nb_bitmaps=%" PRIu32 "\n", + s->bitmap_directory_offset, s->nb_bitmaps); +#endif + break; + default: /* unknown magic - save it in case we need to rewrite the header */ { @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, Error *local_err = NULL; uint64_t ext_end; uint64_t l1_vm_state_index; + bool update_header = false; ret = bdrv_pread(bs->file, 0, &header, sizeof(header)); if (ret < 0) { @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, if (s->incompatible_features & ~QCOW2_INCOMPAT_MASK) { void *feature_table = NULL; qcow2_read_extensions(bs, header.header_length, ext_end, - &feature_table, flags, NULL); + &feature_table, flags, NULL, NULL); report_unsupported_feature(errp, feature_table, s->incompatible_features & ~QCOW2_INCOMPAT_MASK); @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, /* read qcow2 extensions */ if (qcow2_read_extensions(bs, header.header_length, ext_end, NULL, - flags, &local_err)) { + flags, &update_header, &local_err)) { error_propagate(errp, local_err); ret = -EINVAL; goto fail; @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, } /* Clear unknown autoclear feature bits */ - if (!bs->read_only && !(flags & BDRV_O_INACTIVE) && s->autoclear_features) { - s->autoclear_features = 0; + update_header |= s->autoclear_features & ~QCOW2_AUTOCLEAR_MASK; + + if (update_header && !bs->read_only && !(flags & BDRV_O_INACTIVE)) { + s->autoclear_features &= QCOW2_AUTOCLEAR_MASK; ret = qcow2_update_header(bs); if (ret < 0) { error_setg_errno(errp, -ret, "Could not update qcow2 header"); @@ -XXX,XX +XXX,XX @@ int qcow2_update_header(BlockDriverState *bs) buflen -= ret; } + /* Bitmap extension */ + if (s->nb_bitmaps > 0) { + Qcow2BitmapHeaderExt bitmaps_header = { + .nb_bitmaps = cpu_to_be32(s->nb_bitmaps), + .bitmap_directory_size = + cpu_to_be64(s->bitmap_directory_size), + .bitmap_directory_offset = + cpu_to_be64(s->bitmap_directory_offset) + }; + ret = header_ext_add(buf, QCOW2_EXT_MAGIC_BITMAPS, + &bitmaps_header, sizeof(bitmaps_header), + buflen); + if (ret < 0) { + goto fail; + } + buf += ret; + buflen -= ret; + } + /* Keep unknown header extensions */ QLIST_FOREACH(uext, &s->unknown_header_ext, next) { ret = header_ext_add(buf, uext->magic, uext->data, uext->len, buflen); @@ -XXX,XX +XXX,XX @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, Error **errp) return -ENOTSUP; } + /* cannot proceed if image has bitmaps */ + if (s->nb_bitmaps) { + /* TODO: resize bitmaps in the image */ + error_setg(errp, "Can't resize an image which has bitmaps"); + return -ENOTSUP; + } + /* shrinking is currently not supported */ if (offset < bs->total_sectors * 512) { error_setg(errp, "qcow2 doesn't support shrinking images yet"); diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ * space for snapshot names and IDs */ #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS) +/* Bitmap header extension constraints */ +#define QCOW2_MAX_BITMAPS 65535 +#define QCOW2_MAX_BITMAP_DIRECTORY_SIZE (1024 * QCOW2_MAX_BITMAPS) + /* indicate that the refcount of the referenced cluster is exactly one. */ #define QCOW_OFLAG_COPIED (1ULL << 63) /* indicate that the cluster is compressed (they never have the copied flag) */ @@ -XXX,XX +XXX,XX @@ enum { QCOW2_COMPAT_FEAT_MASK = QCOW2_COMPAT_LAZY_REFCOUNTS, }; +/* Autoclear feature bits */ +enum { + QCOW2_AUTOCLEAR_BITMAPS_BITNR = 0, + QCOW2_AUTOCLEAR_BITMAPS = 1 << QCOW2_AUTOCLEAR_BITMAPS_BITNR, + + QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS, +}; + enum qcow2_discard_type { QCOW2_DISCARD_NEVER = 0, QCOW2_DISCARD_ALWAYS, @@ -XXX,XX +XXX,XX @@ typedef uint64_t Qcow2GetRefcountFunc(const void *refcount_array, typedef void Qcow2SetRefcountFunc(void *refcount_array, uint64_t index, uint64_t value); +typedef struct Qcow2BitmapHeaderExt { + uint32_t nb_bitmaps; + uint32_t reserved32; + uint64_t bitmap_directory_size; + uint64_t bitmap_directory_offset; +} QEMU_PACKED Qcow2BitmapHeaderExt; + typedef struct BDRVQcow2State { int cluster_bits; int cluster_size; @@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcow2State { unsigned int nb_snapshots; QCowSnapshot *snapshots; + uint32_t nb_bitmaps; + uint64_t bitmap_directory_size; + uint64_t bitmap_directory_offset; + int flags; int qcow_version; bool use_lazy_refcounts; @@ -XXX,XX +XXX,XX @@ int qcow2_cache_get_empty(BlockDriverState *bs, Qcow2Cache *c, uint64_t offset, void **table); void qcow2_cache_put(BlockDriverState *bs, Qcow2Cache *c, void **table); +/* qcow2-bitmap.c functions */ +int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res, + void **refcount_table, + int64_t *refcount_table_size); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170628120530.31251-10-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/dirty-bitmap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ struct BdrvDirtyBitmap { BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status */ char *name; /* Optional non-empty unique ID */ int64_t size; /* Size of the bitmap (Number of sectors) */ - bool disabled; /* Bitmap is read-only */ + bool disabled; /* Bitmap is disabled. It ignores all writes to + the device */ int active_iterators; /* How many iterators are active */ QLIST_ENTRY(BdrvDirtyBitmap) list; }; -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> It will be needed in following commits for persistent bitmaps. If bitmap is loaded from read-only storage (and we can't mark it "in use" in this storage) corresponding BdrvDirtyBitmap should be read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20170628120530.31251-11-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/dirty-bitmap.c | 36 ++++++++++++++++++++++++++++++++++++ block/io.c | 8 ++++++++ blockdev.c | 6 ++++++ include/block/dirty-bitmap.h | 4 ++++ 4 files changed, 54 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ struct BdrvDirtyBitmap { bool disabled; /* Bitmap is disabled. It ignores all writes to the device */ int active_iterators; /* How many iterators are active */ + bool readonly; /* Bitmap is read-only. This field also + prevents the respective image from being + modified (i.e. blocks writes and discards). + Such operations must fail and both the image + and this bitmap must remain unchanged while + this flag is set. */ QLIST_ENTRY(BdrvDirtyBitmap) list; }; @@ -XXX,XX +XXX,XX @@ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t cur_sector, int64_t nr_sectors) { assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors); } @@ -XXX,XX +XXX,XX @@ void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t cur_sector, int64_t nr_sectors) { assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors); } @@ -XXX,XX +XXX,XX @@ void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap, void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out) { assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); bdrv_dirty_bitmap_lock(bitmap); if (!out) { hbitmap_reset_all(bitmap->bitmap); @@ -XXX,XX +XXX,XX @@ void bdrv_undo_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap *in) { HBitmap *tmp = bitmap->bitmap; assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); bitmap->bitmap = in; hbitmap_free(tmp); } @@ -XXX,XX +XXX,XX @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector, if (!bdrv_dirty_bitmap_enabled(bitmap)) { continue; } + assert(!bdrv_dirty_bitmap_readonly(bitmap)); hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors); } bdrv_dirty_bitmaps_unlock(bs); @@ -XXX,XX +XXX,XX @@ int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap) { return hbitmap_count(bitmap->meta); } + +bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap) +{ + return bitmap->readonly; +} + +/* Called with BQL taken. */ +void bdrv_dirty_bitmap_set_readonly(BdrvDirtyBitmap *bitmap, bool value) +{ + qemu_mutex_lock(bitmap->mutex); + bitmap->readonly = value; + qemu_mutex_unlock(bitmap->mutex); +} + +bool bdrv_has_readonly_bitmaps(BlockDriverState *bs) +{ + BdrvDirtyBitmap *bm; + QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) { + if (bm->readonly) { + return true; + } + } + + return false; +} diff --git a/block/io.c b/block/io.c index XXXXXXX..XXXXXXX 100644 --- a/block/io.c +++ b/block/io.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child, uint64_t bytes_remaining = bytes; int max_transfer; + if (bdrv_has_readonly_bitmaps(bs)) { + return -EPERM; + } + assert(is_power_of_2(align)); assert((offset & (align - 1)) == 0); assert((bytes & (align - 1)) == 0); @@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_pdiscard(BlockDriverState *bs, int64_t offset, return -ENOMEDIUM; } + if (bdrv_has_readonly_bitmaps(bs)) { + return -EPERM; + } + ret = bdrv_check_byte_request(bs, offset, bytes); if (ret < 0) { return ret; diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ static void block_dirty_bitmap_clear_prepare(BlkActionState *common, } else if (!bdrv_dirty_bitmap_enabled(state->bitmap)) { error_setg(errp, "Cannot clear a disabled bitmap"); return; + } else if (bdrv_dirty_bitmap_readonly(state->bitmap)) { + error_setg(errp, "Cannot clear a readonly bitmap"); + return; } bdrv_clear_dirty_bitmap(state->bitmap, &state->backup); @@ -XXX,XX +XXX,XX @@ void qmp_block_dirty_bitmap_clear(const char *node, const char *name, "Bitmap '%s' is currently disabled and cannot be cleared", name); return; + } else if (bdrv_dirty_bitmap_readonly(bitmap)) { + error_setg(errp, "Bitmap '%s' is readonly and cannot be cleared", name); + return; } bdrv_clear_dirty_bitmap(bitmap, NULL); diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap, bool finish); void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap); +void bdrv_dirty_bitmap_set_readonly(BdrvDirtyBitmap *bitmap, bool value); + /* Functions that require manual locking. */ void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_unlock(BdrvDirtyBitmap *bitmap); @@ -XXX,XX +XXX,XX @@ void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t sector_num); int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap); int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_truncate(BlockDriverState *bs); +bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap); +bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They are loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Extra data in bitmaps is not supported for now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-12-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-bitmap.c | 389 +++++++++++++++++++++++++++++++++++++++++++++++++++ block/qcow2.c | 17 ++- block/qcow2.h | 2 + 3 files changed, 406 insertions(+), 2 deletions(-) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ /* Bitmap directory entry flags */ #define BME_RESERVED_FLAGS 0xfffffffcU +#define BME_FLAG_IN_USE (1U << 0) +#define BME_FLAG_AUTO (1U << 1) /* bits [1, 8] U [56, 63] are reserved */ #define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001feULL @@ -XXX,XX +XXX,XX @@ typedef enum BitmapType { BT_DIRTY_TRACKING_BITMAP = 1 } BitmapType; +static inline bool can_write(BlockDriverState *bs) +{ + return !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE); +} + +static int update_header_sync(BlockDriverState *bs) +{ + int ret; + + ret = qcow2_update_header(bs); + if (ret < 0) { + return ret; + } + + return bdrv_flush(bs); +} + static int check_table_entry(uint64_t entry, int cluster_size) { uint64_t offset; @@ -XXX,XX +XXX,XX @@ fail: return ret; } +/* This function returns the number of disk sectors covered by a single qcow2 + * cluster of bitmap data. */ +static uint64_t sectors_covered_by_bitmap_cluster(const BDRVQcow2State *s, + const BdrvDirtyBitmap *bitmap) +{ + uint32_t sector_granularity = + bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS; + + return (uint64_t)sector_granularity * (s->cluster_size << 3); +} + +/* load_bitmap_data + * @bitmap_table entries must satisfy specification constraints. + * @bitmap must be cleared */ +static int load_bitmap_data(BlockDriverState *bs, + const uint64_t *bitmap_table, + uint32_t bitmap_table_size, + BdrvDirtyBitmap *bitmap) +{ + int ret = 0; + BDRVQcow2State *s = bs->opaque; + uint64_t sector, sbc; + uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap); + uint8_t *buf = NULL; + uint64_t i, tab_size = + size_to_clusters(s, + bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size)); + + if (tab_size != bitmap_table_size || tab_size > BME_MAX_TABLE_SIZE) { + return -EINVAL; + } + + buf = g_malloc(s->cluster_size); + sbc = sectors_covered_by_bitmap_cluster(s, bitmap); + for (i = 0, sector = 0; i < tab_size; ++i, sector += sbc) { + uint64_t count = MIN(bm_size - sector, sbc); + uint64_t entry = bitmap_table[i]; + uint64_t offset = entry & BME_TABLE_ENTRY_OFFSET_MASK; + + assert(check_table_entry(entry, s->cluster_size) == 0); + + if (offset == 0) { + if (entry & BME_TABLE_ENTRY_FLAG_ALL_ONES) { + bdrv_dirty_bitmap_deserialize_ones(bitmap, sector, count, + false); + } else { + /* No need to deserialize zeros because the dirty bitmap is + * already cleared */ + } + } else { + ret = bdrv_pread(bs->file, offset, buf, s->cluster_size); + if (ret < 0) { + goto finish; + } + bdrv_dirty_bitmap_deserialize_part(bitmap, buf, sector, count, + false); + } + } + ret = 0; + + bdrv_dirty_bitmap_deserialize_finish(bitmap); + +finish: + g_free(buf); + + return ret; +} + +static BdrvDirtyBitmap *load_bitmap(BlockDriverState *bs, + Qcow2Bitmap *bm, Error **errp) +{ + int ret; + uint64_t *bitmap_table = NULL; + uint32_t granularity; + BdrvDirtyBitmap *bitmap = NULL; + + if (bm->flags & BME_FLAG_IN_USE) { + error_setg(errp, "Bitmap '%s' is in use", bm->name); + goto fail; + } + + ret = bitmap_table_load(bs, &bm->table, &bitmap_table); + if (ret < 0) { + error_setg_errno(errp, -ret, + "Could not read bitmap_table table from image for " + "bitmap '%s'", bm->name); + goto fail; + } + + granularity = 1U << bm->granularity_bits; + bitmap = bdrv_create_dirty_bitmap(bs, granularity, bm->name, errp); + if (bitmap == NULL) { + goto fail; + } + + ret = load_bitmap_data(bs, bitmap_table, bm->table.size, bitmap); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not read bitmap '%s' from image", + bm->name); + goto fail; + } + + g_free(bitmap_table); + return bitmap; + +fail: + g_free(bitmap_table); + if (bitmap != NULL) { + bdrv_release_dirty_bitmap(bs, bitmap); + } + + return NULL; +} + /* * Bitmap List */ @@ -XXX,XX +XXX,XX @@ static inline void bitmap_dir_entry_to_cpu(Qcow2BitmapDirEntry *entry) be32_to_cpus(&entry->extra_data_size); } +static inline void bitmap_dir_entry_to_be(Qcow2BitmapDirEntry *entry) +{ + cpu_to_be64s(&entry->bitmap_table_offset); + cpu_to_be32s(&entry->bitmap_table_size); + cpu_to_be32s(&entry->flags); + cpu_to_be16s(&entry->name_size); + cpu_to_be32s(&entry->extra_data_size); +} + static inline int calc_dir_entry_size(size_t name_size, size_t extra_data_size) { return align_offset(sizeof(Qcow2BitmapDirEntry) + @@ -XXX,XX +XXX,XX @@ static int check_dir_entry(BlockDriverState *bs, Qcow2BitmapDirEntry *entry) return fail ? -EINVAL : 0; } +static inline void bitmap_directory_to_be(uint8_t *dir, size_t size) +{ + uint8_t *end = dir + size; + while (dir < end) { + Qcow2BitmapDirEntry *e = (Qcow2BitmapDirEntry *)dir; + dir += dir_entry_size(e); + + bitmap_dir_entry_to_be(e); + } +} + /* * Bitmap List public functions */ @@ -XXX,XX +XXX,XX @@ static Qcow2BitmapList *bitmap_list_new(void) return bm_list; } +static uint32_t bitmap_list_count(Qcow2BitmapList *bm_list) +{ + Qcow2Bitmap *bm; + uint32_t nb_bitmaps = 0; + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + nb_bitmaps++; + } + + return nb_bitmaps; +} + /* bitmap_list_load * Get bitmap list from qcow2 image. Actually reads bitmap directory, * checks it and convert to bitmap list. @@ -XXX,XX +XXX,XX @@ out: return ret; } + +/* bitmap_list_store + * Store bitmap list to qcow2 image as a bitmap directory. + * Everything is checked. + */ +static int bitmap_list_store(BlockDriverState *bs, Qcow2BitmapList *bm_list, + uint64_t *offset, uint64_t *size, bool in_place) +{ + int ret; + uint8_t *dir; + int64_t dir_offset = 0; + uint64_t dir_size = 0; + Qcow2Bitmap *bm; + Qcow2BitmapDirEntry *e; + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + dir_size += calc_dir_entry_size(strlen(bm->name), 0); + } + + if (dir_size == 0 || dir_size > QCOW2_MAX_BITMAP_DIRECTORY_SIZE) { + return -EINVAL; + } + + if (in_place) { + if (*size != dir_size || *offset == 0) { + return -EINVAL; + } + + dir_offset = *offset; + } + + dir = g_try_malloc(dir_size); + if (dir == NULL) { + return -ENOMEM; + } + + e = (Qcow2BitmapDirEntry *)dir; + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + e->bitmap_table_offset = bm->table.offset; + e->bitmap_table_size = bm->table.size; + e->flags = bm->flags; + e->type = BT_DIRTY_TRACKING_BITMAP; + e->granularity_bits = bm->granularity_bits; + e->name_size = strlen(bm->name); + e->extra_data_size = 0; + memcpy(e + 1, bm->name, e->name_size); + + if (check_dir_entry(bs, e) < 0) { + ret = -EINVAL; + goto fail; + } + + e = next_dir_entry(e); + } + + bitmap_directory_to_be(dir, dir_size); + + if (!in_place) { + dir_offset = qcow2_alloc_clusters(bs, dir_size); + if (dir_offset < 0) { + ret = dir_offset; + goto fail; + } + } + + ret = qcow2_pre_write_overlap_check(bs, 0, dir_offset, dir_size); + if (ret < 0) { + goto fail; + } + + ret = bdrv_pwrite(bs->file, dir_offset, dir, dir_size); + if (ret < 0) { + goto fail; + } + + g_free(dir); + + if (!in_place) { + *size = dir_size; + *offset = dir_offset; + } + + return 0; + +fail: + g_free(dir); + + if (!in_place && dir_offset > 0) { + qcow2_free_clusters(bs, dir_offset, dir_size, QCOW2_DISCARD_OTHER); + } + + return ret; +} + +/* + * Bitmap List end + */ + +static int update_ext_header_and_dir_in_place(BlockDriverState *bs, + Qcow2BitmapList *bm_list) +{ + BDRVQcow2State *s = bs->opaque; + int ret; + + if (!(s->autoclear_features & QCOW2_AUTOCLEAR_BITMAPS) || + bm_list == NULL || QSIMPLEQ_EMPTY(bm_list) || + bitmap_list_count(bm_list) != s->nb_bitmaps) + { + return -EINVAL; + } + + s->autoclear_features &= ~(uint64_t)QCOW2_AUTOCLEAR_BITMAPS; + ret = update_header_sync(bs); + if (ret < 0) { + /* Two variants are possible here: + * 1. Autoclear flag is dropped, all bitmaps will be lost. + * 2. Autoclear flag is not dropped, old state is left. + */ + return ret; + } + + /* autoclear bit is not set, so we can safely update bitmap directory */ + + ret = bitmap_list_store(bs, bm_list, &s->bitmap_directory_offset, + &s->bitmap_directory_size, true); + if (ret < 0) { + /* autoclear bit is cleared, so all leaked clusters would be removed on + * qemu-img check */ + return ret; + } + + ret = update_header_sync(bs); + if (ret < 0) { + /* autoclear bit is cleared, so all leaked clusters would be removed on + * qemu-img check */ + return ret; + } + + s->autoclear_features |= QCOW2_AUTOCLEAR_BITMAPS; + return update_header_sync(bs); + /* If final update_header_sync() fails, two variants are possible: + * 1. Autoclear flag is not set, all bitmaps will be lost. + * 2. Autoclear flag is set, header and directory are successfully updated. + */ +} + +/* for g_slist_foreach for GSList of BdrvDirtyBitmap* elements */ +static void release_dirty_bitmap_helper(gpointer bitmap, + gpointer bs) +{ + bdrv_release_dirty_bitmap(bs, bitmap); +} + +/* for g_slist_foreach for GSList of BdrvDirtyBitmap* elements */ +static void set_readonly_helper(gpointer bitmap, gpointer value) +{ + bdrv_dirty_bitmap_set_readonly(bitmap, (bool)value); +} + +/* qcow2_load_autoloading_dirty_bitmaps() + * Return value is a hint for caller: true means that the Qcow2 header was + * updated. (false doesn't mean that the header should be updated by the + * caller, it just means that updating was not needed or the image cannot be + * written to). + * On failure the function returns false. + */ +bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **errp) +{ + BDRVQcow2State *s = bs->opaque; + Qcow2BitmapList *bm_list; + Qcow2Bitmap *bm; + GSList *created_dirty_bitmaps = NULL; + bool header_updated = false; + + if (s->nb_bitmaps == 0) { + /* No bitmaps - nothing to do */ + return false; + } + + bm_list = bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list == NULL) { + return false; + } + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if ((bm->flags & BME_FLAG_AUTO) && !(bm->flags & BME_FLAG_IN_USE)) { + BdrvDirtyBitmap *bitmap = load_bitmap(bs, bm, errp); + if (bitmap == NULL) { + goto fail; + } + bm->flags |= BME_FLAG_IN_USE; + created_dirty_bitmaps = + g_slist_append(created_dirty_bitmaps, bitmap); + } + } + + if (created_dirty_bitmaps != NULL) { + if (can_write(bs)) { + /* in_use flags must be updated */ + int ret = update_ext_header_and_dir_in_place(bs, bm_list); + if (ret < 0) { + error_setg_errno(errp, -ret, "Can't update bitmap directory"); + goto fail; + } + header_updated = true; + } else { + g_slist_foreach(created_dirty_bitmaps, set_readonly_helper, + (gpointer)true); + } + } + + g_slist_free(created_dirty_bitmaps); + bitmap_list_free(bm_list); + + return header_updated; + +fail: + g_slist_foreach(created_dirty_bitmaps, release_dirty_bitmap_helper, bs); + g_slist_free(created_dirty_bitmaps); + bitmap_list_free(bm_list); + + return false; +} diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags, /* Clear unknown autoclear feature bits */ update_header |= s->autoclear_features & ~QCOW2_AUTOCLEAR_MASK; - - if (update_header && !bs->read_only && !(flags & BDRV_O_INACTIVE)) { + update_header = + update_header && !bs->read_only && !(flags & BDRV_O_INACTIVE); + if (update_header) { s->autoclear_features &= QCOW2_AUTOCLEAR_MASK; + } + + if (qcow2_load_autoloading_dirty_bitmaps(bs, &local_err)) { + update_header = false; + } + if (local_err != NULL) { + error_propagate(errp, local_err); + ret = -EINVAL; + goto fail; + } + + if (update_header) { ret = qcow2_update_header(bs); if (ret < 0) { error_setg_errno(errp, -ret, "Could not update qcow2 header"); diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ void qcow2_cache_put(BlockDriverState *bs, Qcow2Cache *c, void **table); int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res, void **refcount_table, int64_t *refcount_table_size); +bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **errp); + #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Add bs local variable to simplify code. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-13-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ error: void bdrv_reopen_commit(BDRVReopenState *reopen_state) { BlockDriver *drv; + BlockDriverState *bs; assert(reopen_state != NULL); - drv = reopen_state->bs->drv; + bs = reopen_state->bs; + drv = bs->drv; assert(drv != NULL); /* If there are any driver level actions to take */ @@ -XXX,XX +XXX,XX @@ void bdrv_reopen_commit(BDRVReopenState *reopen_state) } /* set BDS specific flags now */ - QDECREF(reopen_state->bs->explicit_options); + QDECREF(bs->explicit_options); - reopen_state->bs->explicit_options = reopen_state->explicit_options; - reopen_state->bs->open_flags = reopen_state->flags; - reopen_state->bs->read_only = !(reopen_state->flags & BDRV_O_RDWR); + bs->explicit_options = reopen_state->explicit_options; + bs->open_flags = reopen_state->flags; + bs->read_only = !(reopen_state->flags & BDRV_O_RDWR); - bdrv_refresh_limits(reopen_state->bs, NULL); + bdrv_refresh_limits(bs, NULL); } /* -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Add format driver handler, which should mark loaded read-only bitmaps as 'IN_USE' in the image and unset read_only field in corresponding BdrvDirtyBitmap's. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170628120530.31251-14-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block.c | 19 +++++++++++++++++++ include/block/block_int.h | 7 +++++++ 2 files changed, 26 insertions(+) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ void bdrv_reopen_commit(BDRVReopenState *reopen_state) { BlockDriver *drv; BlockDriverState *bs; + bool old_can_write, new_can_write; assert(reopen_state != NULL); bs = reopen_state->bs; drv = bs->drv; assert(drv != NULL); + old_can_write = + !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE); + /* If there are any driver level actions to take */ if (drv->bdrv_reopen_commit) { drv->bdrv_reopen_commit(reopen_state); @@ -XXX,XX +XXX,XX @@ void bdrv_reopen_commit(BDRVReopenState *reopen_state) bs->read_only = !(reopen_state->flags & BDRV_O_RDWR); bdrv_refresh_limits(bs, NULL); + + new_can_write = + !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE); + if (!old_can_write && new_can_write && drv->bdrv_reopen_bitmaps_rw) { + Error *local_err = NULL; + if (drv->bdrv_reopen_bitmaps_rw(bs, &local_err) < 0) { + /* This is not fatal, bitmaps just left read-only, so all following + * writes will fail. User can remove read-only bitmaps to unblock + * writes. + */ + error_reportf_err(local_err, + "%s: Failed to make dirty bitmaps writable: ", + bdrv_get_node_name(bs)); + } + } } /* diff --git a/include/block/block_int.h b/include/block/block_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -XXX,XX +XXX,XX @@ struct BlockDriver { uint64_t parent_perm, uint64_t parent_shared, uint64_t *nperm, uint64_t *nshared); + /** + * Bitmaps should be marked as 'IN_USE' in the image on reopening image + * as rw. This handler should realize it. It also should unset readonly + * field of BlockDirtyBitmap's in case of success. + */ + int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp); + QLIST_ENTRY(BlockDriver) list; }; -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Realize bdrv_reopen_bitmaps_rw interface. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170628120530.31251-15-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-bitmap.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++++++++ block/qcow2.c | 2 ++ block/qcow2.h | 1 + 3 files changed, 64 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ fail: return false; } + +int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp) +{ + BDRVQcow2State *s = bs->opaque; + Qcow2BitmapList *bm_list; + Qcow2Bitmap *bm; + GSList *ro_dirty_bitmaps = NULL; + int ret = 0; + + if (s->nb_bitmaps == 0) { + /* No bitmaps - nothing to do */ + return 0; + } + + if (!can_write(bs)) { + error_setg(errp, "Can't write to the image on reopening bitmaps rw"); + return -EINVAL; + } + + bm_list = bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list == NULL) { + return -EINVAL; + } + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if (!(bm->flags & BME_FLAG_IN_USE)) { + BdrvDirtyBitmap *bitmap = bdrv_find_dirty_bitmap(bs, bm->name); + if (bitmap == NULL) { + continue; + } + + if (!bdrv_dirty_bitmap_readonly(bitmap)) { + error_setg(errp, "Bitmap %s is not readonly but not marked" + "'IN_USE' in the image. Something went wrong," + "all the bitmaps may be corrupted", bm->name); + ret = -EINVAL; + goto out; + } + + bm->flags |= BME_FLAG_IN_USE; + ro_dirty_bitmaps = g_slist_append(ro_dirty_bitmaps, bitmap); + } + } + + if (ro_dirty_bitmaps != NULL) { + /* in_use flags must be updated */ + ret = update_ext_header_and_dir_in_place(bs, bm_list); + if (ret < 0) { + error_setg_errno(errp, -ret, "Can't update bitmap directory"); + goto out; + } + g_slist_foreach(ro_dirty_bitmaps, set_readonly_helper, false); + } + +out: + g_slist_free(ro_dirty_bitmaps); + bitmap_list_free(bm_list); + + return ret; +} diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = { .bdrv_detach_aio_context = qcow2_detach_aio_context, .bdrv_attach_aio_context = qcow2_attach_aio_context, + + .bdrv_reopen_bitmaps_rw = qcow2_reopen_bitmaps_rw, }; static void bdrv_qcow2_init(void) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res, void **refcount_table, int64_t *refcount_table_size); bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **errp); +int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be needed in future, to save this flag back to Qcow2 for persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20170628120530.31251-16-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/dirty-bitmap.c | 18 ++++++++++++++++++ block/qcow2-bitmap.c | 2 ++ include/block/dirty-bitmap.h | 2 ++ 3 files changed, 22 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ struct BdrvDirtyBitmap { Such operations must fail and both the image and this bitmap must remain unchanged while this flag is set. */ + bool autoload; /* For persistent bitmaps: bitmap must be + autoloaded on image opening */ QLIST_ENTRY(BdrvDirtyBitmap) list; }; @@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap) assert(!bdrv_dirty_bitmap_frozen(bitmap)); g_free(bitmap->name); bitmap->name = NULL; + bitmap->autoload = false; } /* Called with BQL taken. */ @@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverState *bs, bitmap->name = NULL; successor->name = name; bitmap->successor = NULL; + successor->autoload = bitmap->autoload; + bitmap->autoload = false; bdrv_release_dirty_bitmap(bs, bitmap); return successor; @@ -XXX,XX +XXX,XX @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs) return false; } + +/* Called with BQL taken. */ +void bdrv_dirty_bitmap_set_autoload(BdrvDirtyBitmap *bitmap, bool autoload) +{ + qemu_mutex_lock(bitmap->mutex); + bitmap->autoload = autoload; + qemu_mutex_unlock(bitmap->mutex); +} + +bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap) +{ + return bitmap->autoload; +} diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **errp) if (bitmap == NULL) { goto fail; } + + bdrv_dirty_bitmap_set_autoload(bitmap, true); bm->flags |= BME_FLAG_IN_USE; created_dirty_bitmaps = g_slist_append(created_dirty_bitmaps, bitmap); diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap, void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_set_readonly(BdrvDirtyBitmap *bitmap, bool value); +void bdrv_dirty_bitmap_set_autoload(BdrvDirtyBitmap *bitmap, bool autoload); /* Functions that require manual locking. */ void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); @@ -XXX,XX +XXX,XX @@ int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_truncate(BlockDriverState *bs); bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap); bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); +bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Release bitmaps after 'if (bs->drv) { ... }' block. This will allow format driver to save persistent bitmaps, which will appear in following commits. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170628120530.31251-17-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static void bdrv_close(BlockDriverState *bs) bdrv_flush(bs); bdrv_drain(bs); /* in case flush left pending I/O */ - bdrv_release_named_dirty_bitmaps(bs); - assert(QLIST_EMPTY(&bs->dirty_bitmaps)); - if (bs->drv) { BdrvChild *child, *next; @@ -XXX,XX +XXX,XX @@ static void bdrv_close(BlockDriverState *bs) bs->full_open_options = NULL; } + bdrv_release_named_dirty_bitmaps(bs); + assert(QLIST_EMPTY(&bs->dirty_bitmaps)); + QLIST_FOREACH_SAFE(ban, &bs->aio_notifiers, list, ban_next) { g_free(ban); } -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> New field BdrvDirtyBitmap.persistent means, that bitmap should be saved by format driver in .bdrv_close and .bdrv_inactivate. No format driver supports it for now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20170628120530.31251-18-vsementsov@virtuozzo.com [mreitz: Fixed indentation] Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/dirty-bitmap.c | 29 +++++++++++++++++++++++++++++ block/qcow2-bitmap.c | 1 + include/block/dirty-bitmap.h | 4 ++++ 3 files changed, 34 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ struct BdrvDirtyBitmap { this flag is set. */ bool autoload; /* For persistent bitmaps: bitmap must be autoloaded on image opening */ + bool persistent; /* bitmap must be saved to owner disk image */ QLIST_ENTRY(BdrvDirtyBitmap) list; }; @@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap) assert(!bdrv_dirty_bitmap_frozen(bitmap)); g_free(bitmap->name); bitmap->name = NULL; + bitmap->persistent = false; bitmap->autoload = false; } @@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverState *bs, bitmap->name = NULL; successor->name = name; bitmap->successor = NULL; + successor->persistent = bitmap->persistent; + bitmap->persistent = false; successor->autoload = bitmap->autoload; bitmap->autoload = false; bdrv_release_dirty_bitmap(bs, bitmap); @@ -XXX,XX +XXX,XX @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap) { return bitmap->autoload; } + +/* Called with BQL taken. */ +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persistent) +{ + qemu_mutex_lock(bitmap->mutex); + bitmap->persistent = persistent; + qemu_mutex_unlock(bitmap->mutex); +} + +bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap) +{ + return bitmap->persistent; +} + +bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs) +{ + BdrvDirtyBitmap *bm; + QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) { + if (bm->persistent && !bm->readonly) { + return true; + } + } + + return false; +} diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **errp) goto fail; } + bdrv_dirty_bitmap_set_persistance(bitmap, true); bdrv_dirty_bitmap_set_autoload(bitmap, true); bm->flags |= BME_FLAG_IN_USE; created_dirty_bitmaps = diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_set_readonly(BdrvDirtyBitmap *bitmap, bool value); void bdrv_dirty_bitmap_set_autoload(BdrvDirtyBitmap *bitmap, bool autoload); +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, + bool persistent); /* Functions that require manual locking. */ void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); @@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_truncate(BlockDriverState *bs); bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap); bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap); +bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap); +bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-19-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/dirty-bitmap.c | 7 +++++++ include/block/dirty-bitmap.h | 2 ++ 2 files changed, 9 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs) return false; } + +BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap) +{ + return bitmap == NULL ? QLIST_FIRST(&bs->dirty_bitmaps) : + QLIST_NEXT(bitmap, list); +} diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -XXX,XX +XXX,XX @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap); bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs); +BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Store persistent dirty bitmaps in qcow2 image. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170628120530.31251-20-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-bitmap.c | 475 +++++++++++++++++++++++++++++++++++++++++++++++++++ block/qcow2.c | 9 + block/qcow2.h | 1 + 3 files changed, 485 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qapi/error.h" +#include "qemu/cutils.h" #include "block/block_int.h" #include "block/qcow2.h" @@ -XXX,XX +XXX,XX @@ #define BME_MIN_GRANULARITY_BITS 9 #define BME_MAX_NAME_SIZE 1023 +#if BME_MAX_TABLE_SIZE * 8ULL > INT_MAX +#error In the code bitmap table physical size assumed to fit into int +#endif + /* Bitmap directory entry flags */ #define BME_RESERVED_FLAGS 0xfffffffcU #define BME_FLAG_IN_USE (1U << 0) @@ -XXX,XX +XXX,XX @@ typedef struct Qcow2BitmapTable { uint32_t size; /* number of 64bit entries */ QSIMPLEQ_ENTRY(Qcow2BitmapTable) entry; } Qcow2BitmapTable; +typedef QSIMPLEQ_HEAD(Qcow2BitmapTableList, Qcow2BitmapTable) + Qcow2BitmapTableList; typedef struct Qcow2Bitmap { Qcow2BitmapTable table; @@ -XXX,XX +XXX,XX @@ typedef struct Qcow2Bitmap { uint8_t granularity_bits; char *name; + BdrvDirtyBitmap *dirty_bitmap; + QSIMPLEQ_ENTRY(Qcow2Bitmap) entry; } Qcow2Bitmap; typedef QSIMPLEQ_HEAD(Qcow2BitmapList, Qcow2Bitmap) Qcow2BitmapList; @@ -XXX,XX +XXX,XX @@ static int update_header_sync(BlockDriverState *bs) return bdrv_flush(bs); } +static inline void bitmap_table_to_be(uint64_t *bitmap_table, size_t size) +{ + size_t i; + + for (i = 0; i < size; ++i) { + cpu_to_be64s(&bitmap_table[i]); + } +} + static int check_table_entry(uint64_t entry, int cluster_size) { uint64_t offset; @@ -XXX,XX +XXX,XX @@ static int check_table_entry(uint64_t entry, int cluster_size) return 0; } +static int check_constraints_on_bitmap(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp) +{ + BDRVQcow2State *s = bs->opaque; + int granularity_bits = ctz32(granularity); + int64_t len = bdrv_getlength(bs); + + assert(granularity > 0); + assert((granularity & (granularity - 1)) == 0); + + if (len < 0) { + error_setg_errno(errp, -len, "Failed to get size of '%s'", + bdrv_get_device_or_node_name(bs)); + return len; + } + + if (granularity_bits > BME_MAX_GRANULARITY_BITS) { + error_setg(errp, "Granularity exceeds maximum (%llu bytes)", + 1ULL << BME_MAX_GRANULARITY_BITS); + return -EINVAL; + } + if (granularity_bits < BME_MIN_GRANULARITY_BITS) { + error_setg(errp, "Granularity is under minimum (%llu bytes)", + 1ULL << BME_MIN_GRANULARITY_BITS); + return -EINVAL; + } + + if ((len > (uint64_t)BME_MAX_PHYS_SIZE << granularity_bits) || + (len > (uint64_t)BME_MAX_TABLE_SIZE * s->cluster_size << + granularity_bits)) + { + error_setg(errp, "Too much space will be occupied by the bitmap. " + "Use larger granularity"); + return -EINVAL; + } + + if (strlen(name) > BME_MAX_NAME_SIZE) { + error_setg(errp, "Name length exceeds maximum (%u characters)", + BME_MAX_NAME_SIZE); + return -EINVAL; + } + + return 0; +} + +static void clear_bitmap_table(BlockDriverState *bs, uint64_t *bitmap_table, + uint32_t bitmap_table_size) +{ + BDRVQcow2State *s = bs->opaque; + int i; + + for (i = 0; i < bitmap_table_size; ++i) { + uint64_t addr = bitmap_table[i] & BME_TABLE_ENTRY_OFFSET_MASK; + if (!addr) { + continue; + } + + qcow2_free_clusters(bs, addr, s->cluster_size, QCOW2_DISCARD_OTHER); + bitmap_table[i] = 0; + } +} + static int bitmap_table_load(BlockDriverState *bs, Qcow2BitmapTable *tb, uint64_t **bitmap_table) { @@ -XXX,XX +XXX,XX @@ fail: return ret; } +static int free_bitmap_clusters(BlockDriverState *bs, Qcow2BitmapTable *tb) +{ + int ret; + uint64_t *bitmap_table; + + ret = bitmap_table_load(bs, tb, &bitmap_table); + if (ret < 0) { + assert(bitmap_table == NULL); + return ret; + } + + clear_bitmap_table(bs, bitmap_table, tb->size); + qcow2_free_clusters(bs, tb->offset, tb->size * sizeof(uint64_t), + QCOW2_DISCARD_OTHER); + g_free(bitmap_table); + + tb->offset = 0; + tb->size = 0; + + return 0; +} + /* This function returns the number of disk sectors covered by a single qcow2 * cluster of bitmap data. */ static uint64_t sectors_covered_by_bitmap_cluster(const BDRVQcow2State *s, @@ -XXX,XX +XXX,XX @@ static int update_ext_header_and_dir_in_place(BlockDriverState *bs, */ } +static int update_ext_header_and_dir(BlockDriverState *bs, + Qcow2BitmapList *bm_list) +{ + BDRVQcow2State *s = bs->opaque; + int ret; + uint64_t new_offset = 0; + uint64_t new_size = 0; + uint32_t new_nb_bitmaps = 0; + uint64_t old_offset = s->bitmap_directory_offset; + uint64_t old_size = s->bitmap_directory_size; + uint32_t old_nb_bitmaps = s->nb_bitmaps; + uint64_t old_autocl = s->autoclear_features; + + if (bm_list != NULL && !QSIMPLEQ_EMPTY(bm_list)) { + new_nb_bitmaps = bitmap_list_count(bm_list); + + if (new_nb_bitmaps > QCOW2_MAX_BITMAPS) { + return -EINVAL; + } + + ret = bitmap_list_store(bs, bm_list, &new_offset, &new_size, false); + if (ret < 0) { + return ret; + } + + ret = bdrv_flush(bs->file->bs); + if (ret < 0) { + goto fail; + } + + s->autoclear_features |= QCOW2_AUTOCLEAR_BITMAPS; + } else { + s->autoclear_features &= ~(uint64_t)QCOW2_AUTOCLEAR_BITMAPS; + } + + s->bitmap_directory_offset = new_offset; + s->bitmap_directory_size = new_size; + s->nb_bitmaps = new_nb_bitmaps; + + ret = update_header_sync(bs); + if (ret < 0) { + goto fail; + } + + if (old_size > 0) { + qcow2_free_clusters(bs, old_offset, old_size, QCOW2_DISCARD_OTHER); + } + + return 0; + +fail: + if (new_offset > 0) { + qcow2_free_clusters(bs, new_offset, new_size, QCOW2_DISCARD_OTHER); + } + + s->bitmap_directory_offset = old_offset; + s->bitmap_directory_size = old_size; + s->nb_bitmaps = old_nb_bitmaps; + s->autoclear_features = old_autocl; + + return ret; +} + /* for g_slist_foreach for GSList of BdrvDirtyBitmap* elements */ static void release_dirty_bitmap_helper(gpointer bitmap, gpointer bs) @@ -XXX,XX +XXX,XX @@ out: return ret; } + +/* store_bitmap_data() + * Store bitmap to image, filling bitmap table accordingly. + */ +static uint64_t *store_bitmap_data(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap, + uint32_t *bitmap_table_size, Error **errp) +{ + int ret; + BDRVQcow2State *s = bs->opaque; + int64_t sector; + uint64_t sbc; + uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap); + const char *bm_name = bdrv_dirty_bitmap_name(bitmap); + uint8_t *buf = NULL; + BdrvDirtyBitmapIter *dbi; + uint64_t *tb; + uint64_t tb_size = + size_to_clusters(s, + bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size)); + + if (tb_size > BME_MAX_TABLE_SIZE || + tb_size * s->cluster_size > BME_MAX_PHYS_SIZE) + { + error_setg(errp, "Bitmap '%s' is too big", bm_name); + return NULL; + } + + tb = g_try_new0(uint64_t, tb_size); + if (tb == NULL) { + error_setg(errp, "No memory"); + return NULL; + } + + dbi = bdrv_dirty_iter_new(bitmap, 0); + buf = g_malloc(s->cluster_size); + sbc = sectors_covered_by_bitmap_cluster(s, bitmap); + assert(DIV_ROUND_UP(bm_size, sbc) == tb_size); + + while ((sector = bdrv_dirty_iter_next(dbi)) != -1) { + uint64_t cluster = sector / sbc; + uint64_t end, write_size; + int64_t off; + + sector = cluster * sbc; + end = MIN(bm_size, sector + sbc); + write_size = + bdrv_dirty_bitmap_serialization_size(bitmap, sector, end - sector); + assert(write_size <= s->cluster_size); + + off = qcow2_alloc_clusters(bs, s->cluster_size); + if (off < 0) { + error_setg_errno(errp, -off, + "Failed to allocate clusters for bitmap '%s'", + bm_name); + goto fail; + } + tb[cluster] = off; + + bdrv_dirty_bitmap_serialize_part(bitmap, buf, sector, end - sector); + if (write_size < s->cluster_size) { + memset(buf + write_size, 0, s->cluster_size - write_size); + } + + ret = qcow2_pre_write_overlap_check(bs, 0, off, s->cluster_size); + if (ret < 0) { + error_setg_errno(errp, -ret, "Qcow2 overlap check failed"); + goto fail; + } + + ret = bdrv_pwrite(bs->file, off, buf, s->cluster_size); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to write bitmap '%s' to file", + bm_name); + goto fail; + } + + if (end >= bm_size) { + break; + } + + bdrv_set_dirty_iter(dbi, end); + } + + *bitmap_table_size = tb_size; + g_free(buf); + bdrv_dirty_iter_free(dbi); + + return tb; + +fail: + clear_bitmap_table(bs, tb, tb_size); + g_free(buf); + bdrv_dirty_iter_free(dbi); + g_free(tb); + + return NULL; +} + +/* store_bitmap() + * Store bm->dirty_bitmap to qcow2. + * Set bm->table_offset and bm->table_size accordingly. + */ +static int store_bitmap(BlockDriverState *bs, Qcow2Bitmap *bm, Error **errp) +{ + int ret; + uint64_t *tb; + int64_t tb_offset; + uint32_t tb_size; + BdrvDirtyBitmap *bitmap = bm->dirty_bitmap; + const char *bm_name; + + assert(bitmap != NULL); + + bm_name = bdrv_dirty_bitmap_name(bitmap); + + tb = store_bitmap_data(bs, bitmap, &tb_size, errp); + if (tb == NULL) { + return -EINVAL; + } + + assert(tb_size <= BME_MAX_TABLE_SIZE); + tb_offset = qcow2_alloc_clusters(bs, tb_size * sizeof(tb[0])); + if (tb_offset < 0) { + error_setg_errno(errp, -tb_offset, + "Failed to allocate clusters for bitmap '%s'", + bm_name); + goto fail; + } + + ret = qcow2_pre_write_overlap_check(bs, 0, tb_offset, + tb_size * sizeof(tb[0])); + if (ret < 0) { + error_setg_errno(errp, -ret, "Qcow2 overlap check failed"); + goto fail; + } + + bitmap_table_to_be(tb, tb_size); + ret = bdrv_pwrite(bs->file, tb_offset, tb, tb_size * sizeof(tb[0])); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to write bitmap '%s' to file", + bm_name); + goto fail; + } + + g_free(tb); + + bm->table.offset = tb_offset; + bm->table.size = tb_size; + + return 0; + +fail: + clear_bitmap_table(bs, tb, tb_size); + + if (tb_offset > 0) { + qcow2_free_clusters(bs, tb_offset, tb_size * sizeof(tb[0]), + QCOW2_DISCARD_OTHER); + } + + g_free(tb); + + return ret; +} + +static Qcow2Bitmap *find_bitmap_by_name(Qcow2BitmapList *bm_list, + const char *name) +{ + Qcow2Bitmap *bm; + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if (strcmp(name, bm->name) == 0) { + return bm; + } + } + + return NULL; +} + +void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp) +{ + BdrvDirtyBitmap *bitmap; + BDRVQcow2State *s = bs->opaque; + uint32_t new_nb_bitmaps = s->nb_bitmaps; + uint64_t new_dir_size = s->bitmap_directory_size; + int ret; + Qcow2BitmapList *bm_list; + Qcow2Bitmap *bm; + Qcow2BitmapTableList drop_tables; + Qcow2BitmapTable *tb, *tb_next; + + if (!bdrv_has_changed_persistent_bitmaps(bs)) { + /* nothing to do */ + return; + } + + if (!can_write(bs)) { + error_setg(errp, "No write access"); + return; + } + + QSIMPLEQ_INIT(&drop_tables); + + if (s->nb_bitmaps == 0) { + bm_list = bitmap_list_new(); + } else { + bm_list = bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list == NULL) { + return; + } + } + + /* check constraints and names */ + for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap != NULL; + bitmap = bdrv_dirty_bitmap_next(bs, bitmap)) + { + const char *name = bdrv_dirty_bitmap_name(bitmap); + uint32_t granularity = bdrv_dirty_bitmap_granularity(bitmap); + Qcow2Bitmap *bm; + + if (!bdrv_dirty_bitmap_get_persistance(bitmap) || + bdrv_dirty_bitmap_readonly(bitmap)) + { + continue; + } + + if (check_constraints_on_bitmap(bs, name, granularity, errp) < 0) { + error_prepend(errp, "Bitmap '%s' doesn't satisfy the constraints: ", + name); + goto fail; + } + + bm = find_bitmap_by_name(bm_list, name); + if (bm == NULL) { + if (++new_nb_bitmaps > QCOW2_MAX_BITMAPS) { + error_setg(errp, "Too many persistent bitmaps"); + goto fail; + } + + new_dir_size += calc_dir_entry_size(strlen(name), 0); + if (new_dir_size > QCOW2_MAX_BITMAP_DIRECTORY_SIZE) { + error_setg(errp, "Bitmap directory is too large"); + goto fail; + } + + bm = g_new0(Qcow2Bitmap, 1); + bm->name = g_strdup(name); + QSIMPLEQ_INSERT_TAIL(bm_list, bm, entry); + } else { + if (!(bm->flags & BME_FLAG_IN_USE)) { + error_setg(errp, "Bitmap '%s' already exists in the image", + name); + goto fail; + } + tb = g_memdup(&bm->table, sizeof(bm->table)); + bm->table.offset = 0; + bm->table.size = 0; + QSIMPLEQ_INSERT_TAIL(&drop_tables, tb, entry); + } + bm->flags = bdrv_dirty_bitmap_get_autoload(bitmap) ? BME_FLAG_AUTO : 0; + bm->granularity_bits = ctz32(bdrv_dirty_bitmap_granularity(bitmap)); + bm->dirty_bitmap = bitmap; + } + + /* allocate clusters and store bitmaps */ + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if (bm->dirty_bitmap == NULL) { + continue; + } + + ret = store_bitmap(bs, bm, errp); + if (ret < 0) { + goto fail; + } + } + + ret = update_ext_header_and_dir(bs, bm_list); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to update bitmap extension"); + goto fail; + } + + /* Bitmap directory was successfully updated, so, old data can be dropped. + * TODO it is better to reuse these clusters */ + QSIMPLEQ_FOREACH_SAFE(tb, &drop_tables, entry, tb_next) { + free_bitmap_clusters(bs, tb); + g_free(tb); + } + + bitmap_list_free(bm_list); + return; + +fail: + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if (bm->dirty_bitmap == NULL || bm->table.offset == 0) { + continue; + } + + free_bitmap_clusters(bs, &bm->table); + } + + QSIMPLEQ_FOREACH_SAFE(tb, &drop_tables, entry, tb_next) { + g_free(tb); + } + + bitmap_list_free(bm_list); +} diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int qcow2_inactivate(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; int ret, result = 0; + Error *local_err = NULL; ret = qcow2_cache_flush(bs, s->l2_table_cache); if (ret) { @@ -XXX,XX +XXX,XX @@ static int qcow2_inactivate(BlockDriverState *bs) strerror(-ret)); } + qcow2_store_persistent_dirty_bitmaps(bs, &local_err); + if (local_err != NULL) { + result = -EINVAL; + error_report_err(local_err); + error_report("Persistent bitmaps are lost for node '%s'", + bdrv_get_device_or_node_name(bs)); + } + if (result == 0) { qcow2_mark_clean(bs); } diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res, int64_t *refcount_table_size); bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **errp); int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); +void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Store bitmaps and mark them read-only on reopening image as read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170628120530.31251-21-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-bitmap.c | 22 ++++++++++++++++++++++ block/qcow2.c | 5 +++++ block/qcow2.h | 1 + 3 files changed, 28 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ fail: bitmap_list_free(bm_list); } + +int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp) +{ + BdrvDirtyBitmap *bitmap; + Error *local_err = NULL; + + qcow2_store_persistent_dirty_bitmaps(bs, &local_err); + if (local_err != NULL) { + error_propagate(errp, local_err); + return -EINVAL; + } + + for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap != NULL; + bitmap = bdrv_dirty_bitmap_next(bs, bitmap)) + { + if (bdrv_dirty_bitmap_get_persistance(bitmap)) { + bdrv_dirty_bitmap_set_readonly(bitmap, true); + } + } + + return 0; +} diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static int qcow2_reopen_prepare(BDRVReopenState *state, /* We need to write out any unwritten data if we reopen read-only. */ if ((state->flags & BDRV_O_RDWR) == 0) { + ret = qcow2_reopen_bitmaps_ro(state->bs, errp); + if (ret < 0) { + goto fail; + } + ret = bdrv_flush(state->bs); if (ret < 0) { goto fail; diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res, bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **errp); int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp); +int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> This will be needed to check some restrictions before making bitmap persistent in qmp-block-dirty-bitmap-add (this functionality will be added by future patch) Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-22-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block.c | 22 ++++++++++++++++++++++ include/block/block.h | 3 +++ include/block/block_int.h | 4 ++++ 3 files changed, 29 insertions(+) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ void bdrv_del_child(BlockDriverState *parent_bs, BdrvChild *child, Error **errp) parent_bs->drv->bdrv_del_child(parent_bs, child, errp); } + +bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name, + uint32_t granularity, Error **errp) +{ + BlockDriver *drv = bs->drv; + + if (!drv) { + error_setg_errno(errp, ENOMEDIUM, + "Can't store persistent bitmaps to %s", + bdrv_get_device_or_node_name(bs)); + return false; + } + + if (!drv->bdrv_can_store_new_dirty_bitmap) { + error_setg_errno(errp, ENOTSUP, + "Can't store persistent bitmaps to %s", + bdrv_get_device_or_node_name(bs)); + return false; + } + + return drv->bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp); +} diff --git a/include/block/block.h b/include/block/block.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -XXX,XX +XXX,XX @@ void bdrv_add_child(BlockDriverState *parent, BlockDriverState *child, Error **errp); void bdrv_del_child(BlockDriverState *parent, BdrvChild *child, Error **errp); +bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name, + uint32_t granularity, Error **errp); + #endif diff --git a/include/block/block_int.h b/include/block/block_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -XXX,XX +XXX,XX @@ struct BlockDriver { * field of BlockDirtyBitmap's in case of success. */ int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp); + bool (*bdrv_can_store_new_dirty_bitmap)(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp); QLIST_ENTRY(BlockDriver) list; }; -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Realize .bdrv_can_store_new_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170628120530.31251-23-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-bitmap.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ block/qcow2.c | 1 + block/qcow2.h | 4 ++++ 3 files changed, 56 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp) return 0; } + +bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp) +{ + BDRVQcow2State *s = bs->opaque; + bool found; + Qcow2BitmapList *bm_list; + + if (check_constraints_on_bitmap(bs, name, granularity, errp) != 0) { + goto fail; + } + + if (s->nb_bitmaps == 0) { + return true; + } + + if (s->nb_bitmaps >= QCOW2_MAX_BITMAPS) { + error_setg(errp, + "Maximum number of persistent bitmaps is already reached"); + goto fail; + } + + if (s->bitmap_directory_size + calc_dir_entry_size(strlen(name), 0) > + QCOW2_MAX_BITMAP_DIRECTORY_SIZE) + { + error_setg(errp, "Not enough space in the bitmap directory"); + goto fail; + } + + bm_list = bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list == NULL) { + goto fail; + } + + found = find_bitmap_by_name(bm_list, name); + bitmap_list_free(bm_list); + if (found) { + error_setg(errp, "Bitmap with the same name is already stored"); + goto fail; + } + + return true; + +fail: + error_prepend(errp, "Can't make bitmap '%s' persistent in '%s': ", + name, bdrv_get_device_or_node_name(bs)); + return false; +} diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = { .bdrv_attach_aio_context = qcow2_attach_aio_context, .bdrv_reopen_bitmaps_rw = qcow2_reopen_bitmaps_rw, + .bdrv_can_store_new_dirty_bitmap = qcow2_can_store_new_dirty_bitmap, }; static void bdrv_qcow2_init(void) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **errp); int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp); int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp); +bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-24-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- blockdev.c | 18 +++++++++++++++++- qapi/block-core.json | 8 +++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ static void block_dirty_bitmap_add_prepare(BlkActionState *common, /* AIO context taken and released within qmp_block_dirty_bitmap_add */ qmp_block_dirty_bitmap_add(action->node, action->name, action->has_granularity, action->granularity, + action->has_persistent, action->persistent, &local_err); if (!local_err) { @@ -XXX,XX +XXX,XX @@ out: void qmp_block_dirty_bitmap_add(const char *node, const char *name, bool has_granularity, uint32_t granularity, + bool has_persistent, bool persistent, Error **errp) { BlockDriverState *bs; + BdrvDirtyBitmap *bitmap; if (!name || name[0] == '\0') { error_setg(errp, "Bitmap name cannot be empty"); @@ -XXX,XX +XXX,XX @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name, granularity = bdrv_get_default_bitmap_granularity(bs); } - bdrv_create_dirty_bitmap(bs, granularity, name, errp); + if (!has_persistent) { + persistent = false; + } + + if (persistent && + !bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp)) + { + return; + } + + bitmap = bdrv_create_dirty_bitmap(bs, granularity, name, errp); + if (bitmap != NULL) { + bdrv_dirty_bitmap_set_persistance(bitmap, persistent); + } } void qmp_block_dirty_bitmap_remove(const char *node, const char *name, diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # @granularity: the bitmap granularity, default is 64k for # block-dirty-bitmap-add # +# @persistent: the bitmap is persistent, i.e. it will be saved to the +# corresponding block device image file on its close. For now only +# Qcow2 disks support persistent bitmaps. Default is false for +# block-dirty-bitmap-add. (Since: 2.10) +# # Since: 2.4 ## { 'struct': 'BlockDirtyBitmapAdd', - 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32' } } + 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32', + '*persistent': 'bool' } } ## # @block-dirty-bitmap-add: -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-25-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- blockdev.c | 18 ++++++++++++++++-- qapi/block-core.json | 6 +++++- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ static void block_dirty_bitmap_add_prepare(BlkActionState *common, qmp_block_dirty_bitmap_add(action->node, action->name, action->has_granularity, action->granularity, action->has_persistent, action->persistent, + action->has_autoload, action->autoload, &local_err); if (!local_err) { @@ -XXX,XX +XXX,XX @@ out: void qmp_block_dirty_bitmap_add(const char *node, const char *name, bool has_granularity, uint32_t granularity, bool has_persistent, bool persistent, + bool has_autoload, bool autoload, Error **errp) { BlockDriverState *bs; @@ -XXX,XX +XXX,XX @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name, if (!has_persistent) { persistent = false; } + if (!has_autoload) { + autoload = false; + } + + if (has_autoload && !persistent) { + error_setg(errp, "Autoload flag must be used only for persistent " + "bitmaps"); + return; + } if (persistent && !bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp)) @@ -XXX,XX +XXX,XX @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name, } bitmap = bdrv_create_dirty_bitmap(bs, granularity, name, errp); - if (bitmap != NULL) { - bdrv_dirty_bitmap_set_persistance(bitmap, persistent); + if (bitmap == NULL) { + return; } + + bdrv_dirty_bitmap_set_persistance(bitmap, persistent); + bdrv_dirty_bitmap_set_autoload(bitmap, autoload); } void qmp_block_dirty_bitmap_remove(const char *node, const char *name, diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # Qcow2 disks support persistent bitmaps. Default is false for # block-dirty-bitmap-add. (Since: 2.10) # +# @autoload: the bitmap will be automatically loaded when the image it is stored +# in is opened. This flag may only be specified for persistent +# bitmaps. Default is false for block-dirty-bitmap-add. (Since: 2.10) +# # Since: 2.4 ## { 'struct': 'BlockDirtyBitmapAdd', 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32', - '*persistent': 'bool' } } + '*persistent': 'bool', '*autoload': 'bool' } } ## # @block-dirty-bitmap-add: -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20170628120530.31251-26-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/dirty-bitmap.c | 5 +++++ blockdev.c | 25 +++++++++++++++++++++++++ include/block/dirty-bitmap.h | 1 + include/qemu/hbitmap.h | 8 ++++++++ qapi/block-core.json | 27 +++++++++++++++++++++++++++ tests/Makefile.include | 2 +- util/hbitmap.c | 11 +++++++++++ 7 files changed, 78 insertions(+), 1 deletion(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs, return bitmap == NULL ? QLIST_FIRST(&bs->dirty_bitmaps) : QLIST_NEXT(bitmap, list); } + +char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp) +{ + return hbitmap_sha256(bitmap->bitmap, errp); +} diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ void qmp_block_dirty_bitmap_clear(const char *node, const char *name, bdrv_clear_dirty_bitmap(bitmap, NULL); } +BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node, + const char *name, + Error **errp) +{ + BdrvDirtyBitmap *bitmap; + BlockDriverState *bs; + BlockDirtyBitmapSha256 *ret = NULL; + char *sha256; + + bitmap = block_dirty_bitmap_lookup(node, name, &bs, errp); + if (!bitmap || !bs) { + return NULL; + } + + sha256 = bdrv_dirty_bitmap_sha256(bitmap, errp); + if (sha256 == NULL) { + return NULL; + } + + ret = g_new(BlockDirtyBitmapSha256, 1); + ret->sha256 = sha256; + + return ret; +} + void hmp_drive_del(Monitor *mon, const QDict *qdict) { const char *id = qdict_get_str(qdict, "id"); diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -XXX,XX +XXX,XX @@ bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap); bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs); BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs, BdrvDirtyBitmap *bitmap); +char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp); #endif diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -XXX,XX +XXX,XX @@ void hbitmap_deserialize_ones(HBitmap *hb, uint64_t start, uint64_t count, void hbitmap_deserialize_finish(HBitmap *hb); /** + * hbitmap_sha256: + * @bitmap: HBitmap to operate on. + * + * Returns SHA256 hash of the last level. + */ +char *hbitmap_sha256(const HBitmap *bitmap, Error **errp); + +/** * hbitmap_free: * @hb: HBitmap to operate on. * diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ 'data': 'BlockDirtyBitmap' } ## +# @BlockDirtyBitmapSha256: +# +# SHA256 hash of dirty bitmap data +# +# @sha256: ASCII representation of SHA256 bitmap hash +# +# Since: 2.10 +## + { 'struct': 'BlockDirtyBitmapSha256', + 'data': {'sha256': 'str'} } + +## +# @x-debug-block-dirty-bitmap-sha256: +# +# Get bitmap SHA256 +# +# Returns: BlockDirtyBitmapSha256 on success +# If @node is not a valid block device, DeviceNotFound +# If @name is not found or if hashing has failed, GenericError with an +# explanation +# +# Since: 2.10 +## + { 'command': 'x-debug-block-dirty-bitmap-sha256', + 'data': 'BlockDirtyBitmap', 'returns': 'BlockDirtyBitmapSha256' } + +## # @blockdev-mirror: # # Start mirroring a block device's writes to a new destination. diff --git a/tests/Makefile.include b/tests/Makefile.include index XXXXXXX..XXXXXXX 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -XXX,XX +XXX,XX @@ tests/test-blockjob$(EXESUF): tests/test-blockjob.o $(test-block-obj-y) $(test-u tests/test-blockjob-txn$(EXESUF): tests/test-blockjob-txn.o $(test-block-obj-y) $(test-util-obj-y) tests/test-thread-pool$(EXESUF): tests/test-thread-pool.o $(test-block-obj-y) tests/test-iov$(EXESUF): tests/test-iov.o $(test-util-obj-y) -tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o $(test-util-obj-y) +tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o $(test-util-obj-y) $(test-crypto-obj-y) tests/test-x86-cpuid$(EXESUF): tests/test-x86-cpuid.o tests/test-xbzrle$(EXESUF): tests/test-xbzrle.o migration/xbzrle.o migration/page_cache.o $(test-util-obj-y) tests/test-cutils$(EXESUF): tests/test-cutils.o util/cutils.o diff --git a/util/hbitmap.c b/util/hbitmap.c index XXXXXXX..XXXXXXX 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -XXX,XX +XXX,XX @@ #include "qemu/hbitmap.h" #include "qemu/host-utils.h" #include "trace.h" +#include "crypto/hash.h" /* HBitmaps provides an array of bits. The bits are stored as usual in an * array of unsigned longs, but HBitmap is also optimized to provide fast @@ -XXX,XX +XXX,XX @@ void hbitmap_free_meta(HBitmap *hb) hbitmap_free(hb->meta); hb->meta = NULL; } + +char *hbitmap_sha256(const HBitmap *bitmap, Error **errp) +{ + size_t size = bitmap->sizes[HBITMAP_LEVELS - 1] * sizeof(unsigned long); + char *data = (char *)bitmap->levels[HBITMAP_LEVELS - 1]; + char *hash = NULL; + qcrypto_hash_digest(QCRYPTO_HASH_ALG_SHA256, data, size, &hash, errp); + + return hash; +} -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170628120530.31251-27-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/165 | 105 +++++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/165.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 111 insertions(+) create mode 100755 tests/qemu-iotests/165 create mode 100644 tests/qemu-iotests/165.out diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165 new file mode 100755 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/165 @@ -XXX,XX +XXX,XX @@ +#!/usr/bin/env python +# +# Tests for persistent dirty bitmaps. +# +# Copyright: Vladimir Sementsov-Ogievskiy 2015-2017 +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +import os +import re +import iotests +from iotests import qemu_img + +disk = os.path.join(iotests.test_dir, 'disk') +disk_size = 0x40000000 # 1G + +# regions for qemu_io: (start, count) in bytes +regions1 = ((0, 0x100000), + (0x200000, 0x100000)) + +regions2 = ((0x10000000, 0x20000), + (0x3fff0000, 0x10000)) + +class TestPersistentDirtyBitmap(iotests.QMPTestCase): + + def setUp(self): + qemu_img('create', '-f', iotests.imgfmt, disk, str(disk_size)) + + def tearDown(self): + os.remove(disk) + + def mkVm(self): + return iotests.VM().add_drive(disk) + + def mkVmRo(self): + return iotests.VM().add_drive(disk, opts='readonly=on') + + def getSha256(self): + result = self.vm.qmp('x-debug-block-dirty-bitmap-sha256', + node='drive0', name='bitmap0') + return result['return']['sha256'] + + def checkBitmap(self, sha256): + result = self.vm.qmp('x-debug-block-dirty-bitmap-sha256', + node='drive0', name='bitmap0') + self.assert_qmp(result, 'return/sha256', sha256); + + def writeRegions(self, regions): + for r in regions: + self.vm.hmp_qemu_io('drive0', + 'write %d %d' % r) + + def qmpAddBitmap(self): + self.vm.qmp('block-dirty-bitmap-add', node='drive0', + name='bitmap0', persistent=True, autoload=True) + + def test_persistent(self): + self.vm = self.mkVm() + self.vm.launch() + self.qmpAddBitmap() + + self.writeRegions(regions1) + sha256 = self.getSha256() + + self.vm.shutdown() + + self.vm = self.mkVmRo() + self.vm.launch() + self.vm.shutdown() + + #catch 'Persistent bitmaps are lost' possible error + log = self.vm.get_log() + log = re.sub(r'^\[I \d+\.\d+\] OPENED\n', '', log) + log = re.sub(r'\[I \+\d+\.\d+\] CLOSED\n?$', '', log) + if log: + print log + + self.vm = self.mkVm() + self.vm.launch() + + self.checkBitmap(sha256) + self.writeRegions(regions2) + sha256 = self.getSha256() + + self.vm.shutdown() + self.vm.launch() + + self.checkBitmap(sha256) + + self.vm.shutdown() + +if __name__ == '__main__': + iotests.main(supported_fmts=['qcow2']) diff --git a/tests/qemu-iotests/165.out b/tests/qemu-iotests/165.out new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/165.out @@ -XXX,XX +XXX,XX @@ +. +---------------------------------------------------------------------- +Ran 1 tests + +OK diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -XXX,XX +XXX,XX @@ 159 rw auto quick 160 rw auto quick 162 auto quick +165 rw auto quick 170 rw auto quick 171 rw auto quick 172 auto -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Interface for removing persistent bitmap from its storage. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-28-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/dirty-bitmap.c | 18 ++++++++++++++++++ include/block/block_int.h | 3 +++ include/block/dirty-bitmap.h | 3 +++ 3 files changed, 24 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap) /** * Release all named dirty bitmaps attached to a BDS (for use in bdrv_close()). * There must not be any frozen bitmaps attached. + * This function does not remove persistent bitmaps from the storage. * Called with BQL taken. */ void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs) @@ -XXX,XX +XXX,XX @@ void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs) bdrv_do_release_matching_dirty_bitmap(bs, NULL, true); } +/** + * Remove persistent dirty bitmap from the storage if it exists. + * Absence of bitmap is not an error, because we have the following scenario: + * BdrvDirtyBitmap can have .persistent = true but not yet saved and have no + * stored version. For such bitmap bdrv_remove_persistent_dirty_bitmap() should + * not fail. + * This function doesn't release corresponding BdrvDirtyBitmap. + */ +void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp) +{ + if (bs->drv && bs->drv->bdrv_remove_persistent_dirty_bitmap) { + bs->drv->bdrv_remove_persistent_dirty_bitmap(bs, name, errp); + } +} + /* Called with BQL taken. */ void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap) { diff --git a/include/block/block_int.h b/include/block/block_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -XXX,XX +XXX,XX @@ struct BlockDriver { const char *name, uint32_t granularity, Error **errp); + void (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs, + const char *name, + Error **errp); QLIST_ENTRY(BlockDriver) list; }; diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap); void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap); void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs); +void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp); void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap); void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap); BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs); -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Realize .bdrv_remove_persistent_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-29-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block/qcow2-bitmap.c | 41 +++++++++++++++++++++++++++++++++++++++++ block/qcow2.c | 1 + block/qcow2.h | 3 +++ 3 files changed, 45 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -XXX,XX +XXX,XX @@ static Qcow2Bitmap *find_bitmap_by_name(Qcow2BitmapList *bm_list, return NULL; } +void qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp) +{ + int ret; + BDRVQcow2State *s = bs->opaque; + Qcow2Bitmap *bm; + Qcow2BitmapList *bm_list; + + if (s->nb_bitmaps == 0) { + /* Absence of the bitmap is not an error: see explanation above + * bdrv_remove_persistent_dirty_bitmap() definition. */ + return; + } + + bm_list = bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list == NULL) { + return; + } + + bm = find_bitmap_by_name(bm_list, name); + if (bm == NULL) { + goto fail; + } + + QSIMPLEQ_REMOVE(bm_list, bm, Qcow2Bitmap, entry); + + ret = update_ext_header_and_dir(bs, bm_list); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to update bitmap extension"); + goto fail; + } + + free_bitmap_clusters(bs, &bm->table); + +fail: + bitmap_free(bm); + bitmap_list_free(bm_list); +} + void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp) { BdrvDirtyBitmap *bitmap; diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = { .bdrv_reopen_bitmaps_rw = qcow2_reopen_bitmaps_rw, .bdrv_can_store_new_dirty_bitmap = qcow2_can_store_new_dirty_bitmap, + .bdrv_remove_persistent_dirty_bitmap = qcow2_remove_persistent_dirty_bitmap, }; static void bdrv_qcow2_init(void) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name, uint32_t granularity, Error **errp); +void qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp); #endif -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Remove persistent bitmap from the storage on block-dirty-bitmap-remove. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20170628120530.31251-30-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- blockdev.c | 10 ++++++++++ qapi/block-core.json | 3 ++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ void qmp_block_dirty_bitmap_remove(const char *node, const char *name, { BlockDriverState *bs; BdrvDirtyBitmap *bitmap; + Error *local_err = NULL; bitmap = block_dirty_bitmap_lookup(node, name, &bs, errp); if (!bitmap || !bs) { @@ -XXX,XX +XXX,XX @@ void qmp_block_dirty_bitmap_remove(const char *node, const char *name, name); return; } + + if (bdrv_dirty_bitmap_get_persistance(bitmap)) { + bdrv_remove_persistent_dirty_bitmap(bs, name, &local_err); + if (local_err != NULL) { + error_propagate(errp, local_err); + return; + } + } + bdrv_dirty_bitmap_make_anon(bitmap); bdrv_release_dirty_bitmap(bs, bitmap); } diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # @block-dirty-bitmap-remove: # # Stop write tracking and remove the dirty bitmap that was created -# with block-dirty-bitmap-add. +# with block-dirty-bitmap-add. If the bitmap is persistent, remove it from its +# storage too. # # Returns: nothing on success # If @node is not a valid block device or node, DeviceNotFound -- 1.8.3.1 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> We should release them here to reload on invalidate cache. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170628120530.31251-31-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- block.c | 4 ++++ block/dirty-bitmap.c | 29 +++++++++++++++++++++++------ include/block/dirty-bitmap.h | 1 + 3 files changed, 28 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static int bdrv_inactivate_recurse(BlockDriverState *bs, } } + /* At this point persistent bitmaps should be already stored by the format + * driver */ + bdrv_release_persistent_dirty_bitmaps(bs); + return 0; } diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index XXXXXXX..XXXXXXX 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_truncate(BlockDriverState *bs) bdrv_dirty_bitmaps_unlock(bs); } +static bool bdrv_dirty_bitmap_has_name(BdrvDirtyBitmap *bitmap) +{ + return !!bdrv_dirty_bitmap_name(bitmap); +} + /* Called with BQL taken. */ -static void bdrv_do_release_matching_dirty_bitmap(BlockDriverState *bs, - BdrvDirtyBitmap *bitmap, - bool only_named) +static void bdrv_do_release_matching_dirty_bitmap( + BlockDriverState *bs, BdrvDirtyBitmap *bitmap, + bool (*cond)(BdrvDirtyBitmap *bitmap)) { BdrvDirtyBitmap *bm, *next; bdrv_dirty_bitmaps_lock(bs); QLIST_FOREACH_SAFE(bm, &bs->dirty_bitmaps, list, next) { - if ((!bitmap || bm == bitmap) && (!only_named || bm->name)) { + if ((!bitmap || bm == bitmap) && (!cond || cond(bm))) { assert(!bm->active_iterators); assert(!bdrv_dirty_bitmap_frozen(bm)); assert(!bm->meta); @@ -XXX,XX +XXX,XX @@ out: /* Called with BQL taken. */ void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap) { - bdrv_do_release_matching_dirty_bitmap(bs, bitmap, false); + bdrv_do_release_matching_dirty_bitmap(bs, bitmap, NULL); } /** @@ -XXX,XX +XXX,XX @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap) */ void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs) { - bdrv_do_release_matching_dirty_bitmap(bs, NULL, true); + bdrv_do_release_matching_dirty_bitmap(bs, NULL, bdrv_dirty_bitmap_has_name); +} + +/** + * Release all persistent dirty bitmaps attached to a BDS (for use in + * bdrv_inactivate_recurse()). + * There must not be any frozen bitmaps attached. + * This function does not remove persistent bitmaps from the storage. + */ +void bdrv_release_persistent_dirty_bitmaps(BlockDriverState *bs) +{ + bdrv_do_release_matching_dirty_bitmap(bs, NULL, + bdrv_dirty_bitmap_get_persistance); } /** diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap); void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap); void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs); +void bdrv_release_persistent_dirty_bitmaps(BlockDriverState *bs); void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name, Error **errp); -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> While the qemu-img dd command does accept --image-opts this is not sufficient to make it work with the LUKS image yet. This is because bdrv_create() still always requires the non-image-opts syntax. Thus we must skip 159/170 with luks for now Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170626123510.20134-2-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/159 | 1 + tests/qemu-iotests/170 | 1 + 2 files changed, 2 insertions(+) diff --git a/tests/qemu-iotests/159 b/tests/qemu-iotests/159 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/159 +++ b/tests/qemu-iotests/159 @@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt generic _supported_proto file _supported_os Linux +_unsupported_fmt luks TEST_SIZES="5 512 1024 1999 1K 64K 1M" diff --git a/tests/qemu-iotests/170 b/tests/qemu-iotests/170 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/170 +++ b/tests/qemu-iotests/170 @@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt generic _supported_proto file _supported_os Linux +_unsupported_fmt luks echo echo "== Creating image ==" -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> The tests 033, 140, 145 and 157 were all broken when run with LUKS, since they did not correctly use the required image opts args syntax to specify the decryption secret. Further, the 120 test simply does not make sense to run with luks, as the scenario exercised is not relevant. The test 181 was broken when run with LUKS because it didn't take account of fact that $TEST_IMG was already in image opts syntax. The launch_qemu helper also didn't register the secret object providing the LUKS password. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170626123510.20134-3-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/033 | 12 ++++++++++-- tests/qemu-iotests/120 | 1 + tests/qemu-iotests/140 | 9 ++++++++- tests/qemu-iotests/145 | 19 +++++++++++++++++-- tests/qemu-iotests/157 | 17 ++++++++++++++--- tests/qemu-iotests/157.out | 16 ++++++++-------- tests/qemu-iotests/174 | 2 +- tests/qemu-iotests/181 | 21 ++++++++++++++++----- tests/qemu-iotests/common.qemu | 9 +++++++-- 9 files changed, 82 insertions(+), 24 deletions(-) diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/033 +++ b/tests/qemu-iotests/033 @@ -XXX,XX +XXX,XX @@ do_test() local align=$1 local iocmd=$2 local img=$3 + if [ "$IMGOPTSSYNTAX" = "true" ] + then + IO_OPEN_ARG="$img" + IO_EXTRA_ARGS="--image-opts" + else + IO_OPEN_ARG="-o driver=$IMGFMT,file.align=$align blkdebug::$img" + IO_EXTRA_ARGS="" + fi { - echo "open -o driver=$IMGFMT,file.align=$align blkdebug::$img" + echo "open $IO_OPEN_ARG" echo $iocmd - } | $QEMU_IO + } | $QEMU_IO $IO_EXTRA_ARGS } for write_zero_cmd in "write -z" "aio_write -z"; do diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/120 +++ b/tests/qemu-iotests/120 @@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt generic _supported_proto file _supported_os Linux +_unsupported_fmt luks _make_test_img 64M diff --git a/tests/qemu-iotests/140 b/tests/qemu-iotests/140 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/140 +++ b/tests/qemu-iotests/140 @@ -XXX,XX +XXX,XX @@ _make_test_img 64k $QEMU_IO -c 'write -P 42 0 64k' "$TEST_IMG" | _filter_qemu_io +if test "$IMGOPTSSYNTAX" = "true" +then + SYSEMU_DRIVE_ARG=if=none,media=cdrom,id=drv,"$TEST_IMG" +else + SYSEMU_DRIVE_ARG=if=none,media=cdrom,id=drv,file="$TEST_IMG",driver=$IMGFMT +fi + keep_stderr=y \ -_launch_qemu -drive if=none,media=cdrom,id=drv,file="$TEST_IMG",format=$IMGFMT \ +_launch_qemu -drive $SYSEMU_DRIVE_ARG \ 2> >(_filter_nbd) _send_qemu_cmd $QEMU_HANDLE \ diff --git a/tests/qemu-iotests/145 b/tests/qemu-iotests/145 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/145 +++ b/tests/qemu-iotests/145 @@ -XXX,XX +XXX,XX @@ _supported_proto generic _supported_os Linux _make_test_img 1M -echo quit | $QEMU -nographic -hda "$TEST_IMG" -incoming 'exec:true' -snapshot -serial none -monitor stdio | - _filter_qemu | _filter_hmp + +if test "$IMGOPTSSYNTAX" = "true" +then + SYSEMU_DRIVE_ARG=if=none,$TEST_IMG + SYSEMU_EXTRA_ARGS="" + if [ -n "$IMGKEYSECRET" ]; then + SECRET_ARG="secret,id=keysec0,data=$IMGKEYSECRET" + SYSEMU_EXTRA_ARGS="-object $SECRET_ARG" + fi +else + SYSEMU_DRIVE_ARG=if=none,file="$TEST_IMG",driver=$IMGFMT + SYSEMU_EXTRA_ARGS="" +fi + +echo quit | $QEMU -nographic $SYSEMU_EXTRA_ARGS -drive $SYSEMU_DRIVE_ARG \ + -incoming 'exec:true' -snapshot -serial none -monitor stdio \ + | _filter_qemu | _filter_hmp # success, all done echo "*** done" diff --git a/tests/qemu-iotests/157 b/tests/qemu-iotests/157 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/157 +++ b/tests/qemu-iotests/157 @@ -XXX,XX +XXX,XX @@ _supported_os Linux function do_run_qemu() { - echo Testing: "$@" ( if ! test -t 0; then while read cmd; do @@ -XXX,XX +XXX,XX @@ function run_qemu() size=128M -drive="if=none,file=$TEST_IMG,driver=$IMGFMT" +if test "$IMGOPTSSYNTAX" = "true" +then + SYSEMU_DRIVE_ARG=if=none,$TEST_IMG + SYSEMU_EXTRA_ARGS="" + if [ -n "$IMGKEYSECRET" ]; then + SECRET_ARG="secret,id=keysec0,data=$IMGKEYSECRET" + SYSEMU_EXTRA_ARGS="-object $SECRET_ARG" + fi +else + SYSEMU_DRIVE_ARG=if=none,file="$TEST_IMG",driver=$IMGFMT + SYSEMU_EXTRA_ARGS="" +fi _make_test_img $size @@ -XXX,XX +XXX,XX @@ echo for cache in "writeback" "writethrough"; do for wce in "" ",write-cache=auto" ",write-cache=on" ",write-cache=off"; do + echo "Testing: cache='$cache' wce='$wce'" echo "info block" \ - | run_qemu -drive "$drive,cache=$cache" \ + | run_qemu $SYSEMU_EXTRA_ARGS -drive "$SYSEMU_DRIVE_ARG,cache=$cache" \ -device "virtio-blk,drive=none0$wce" \ | grep -e "Testing" -e "Cache mode" done diff --git a/tests/qemu-iotests/157.out b/tests/qemu-iotests/157.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/157.out +++ b/tests/qemu-iotests/157.out @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 === Setting WCE with qdev and with manually created BB === -Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,driver=IMGFMT,cache=writeback -device virtio-blk,drive=none0 +Testing: cache='writeback' wce='' Cache mode: writeback -Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,driver=IMGFMT,cache=writeback -device virtio-blk,drive=none0,write-cache=auto +Testing: cache='writeback' wce=',write-cache=auto' Cache mode: writeback -Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,driver=IMGFMT,cache=writeback -device virtio-blk,drive=none0,write-cache=on +Testing: cache='writeback' wce=',write-cache=on' Cache mode: writeback -Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,driver=IMGFMT,cache=writeback -device virtio-blk,drive=none0,write-cache=off +Testing: cache='writeback' wce=',write-cache=off' Cache mode: writethrough -Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,driver=IMGFMT,cache=writethrough -device virtio-blk,drive=none0 +Testing: cache='writethrough' wce='' Cache mode: writethrough -Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,driver=IMGFMT,cache=writethrough -device virtio-blk,drive=none0,write-cache=auto +Testing: cache='writethrough' wce=',write-cache=auto' Cache mode: writethrough -Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,driver=IMGFMT,cache=writethrough -device virtio-blk,drive=none0,write-cache=on +Testing: cache='writethrough' wce=',write-cache=on' Cache mode: writeback -Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,driver=IMGFMT,cache=writethrough -device virtio-blk,drive=none0,write-cache=off +Testing: cache='writethrough' wce=',write-cache=off' Cache mode: writethrough *** done diff --git a/tests/qemu-iotests/174 b/tests/qemu-iotests/174 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/174 +++ b/tests/qemu-iotests/174 @@ -XXX,XX +XXX,XX @@ _unsupported_fmt raw size=256K -IMGFMT=raw IMGOPTS= _make_test_img $size | _filter_imgfmt +IMGFMT=raw IMGKEYSECRET= IMGOPTS= _make_test_img $size | _filter_imgfmt echo echo "== reading wrong format should fail ==" diff --git a/tests/qemu-iotests/181 b/tests/qemu-iotests/181 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/181 +++ b/tests/qemu-iotests/181 @@ -XXX,XX +XXX,XX @@ echo qemu_comm_method="monitor" -_launch_qemu \ - -drive file="${TEST_IMG}",cache=${CACHEMODE},driver=$IMGFMT,id=disk +if [ "$IMGOPTSSYNTAX" = "true" ]; then + _launch_qemu \ + -drive "${TEST_IMG}",cache=${CACHEMODE},id=disk +else + _launch_qemu \ + -drive file="${TEST_IMG}",cache=${CACHEMODE},driver=$IMGFMT,id=disk +fi src=$QEMU_HANDLE -_launch_qemu \ - -drive file="${TEST_IMG}",cache=${CACHEMODE},driver=$IMGFMT,id=disk \ - -incoming "unix:${MIG_SOCKET}" +if [ "$IMGOPTSSYNTAX" = "true" ]; then + _launch_qemu \ + -drive "${TEST_IMG}",cache=${CACHEMODE},id=disk \ + -incoming "unix:${MIG_SOCKET}" +else + _launch_qemu \ + -drive file="${TEST_IMG}",cache=${CACHEMODE},driver=$IMGFMT,id=disk \ + -incoming "unix:${MIG_SOCKET}" +fi dest=$QEMU_HANDLE echo diff --git a/tests/qemu-iotests/common.qemu b/tests/qemu-iotests/common.qemu index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.qemu +++ b/tests/qemu-iotests/common.qemu @@ -XXX,XX +XXX,XX @@ function _launch_qemu() mkfifo "${fifo_out}" mkfifo "${fifo_in}" + object_options= + if [ -n "$IMGKEYSECRET" ]; then + object_options="--object secret,id=keysec0,data=$IMGKEYSECRET" + fi + if [ -z "$keep_stderr" ]; then QEMU_NEED_PID='y'\ - ${QEMU} -nographic -serial none ${comm} "${@}" >"${fifo_out}" \ + ${QEMU} ${object_options} -nographic -serial none ${comm} "${@}" >"${fifo_out}" \ 2>&1 \ <"${fifo_in}" & elif [ "$keep_stderr" = "y" ]; then QEMU_NEED_PID='y'\ - ${QEMU} -nographic -serial none ${comm} "${@}" >"${fifo_out}" \ + ${QEMU} ${object_options} -nographic -serial none ${comm} "${@}" >"${fifo_out}" \ <"${fifo_in}" & else exit 1 -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> By default the PBKDF algorithm used with LUKS is tuned based on the number of iterations to produce 1 second of running time. This makes running the I/O test with the LUKS format orders of magnitude slower than with qcow2/raw formats. When creating LUKS images, set the iteration time to a 10ms to reduce the time overhead for LUKS, since security does not matter in I/O tests. Previously a full 'check -luks' would take $ time ./check -luks Passed all 22 tests real 23m9.988s user 21m46.223s sys 0m22.841s Now it takes $ time ./check -luks Passed all 22 tests real 4m39.235s user 3m29.590s sys 0m24.234s Still slow compared to qcow2/raw, but much improved none the less. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170626123510.20134-4-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/149 | 3 + tests/qemu-iotests/149.out | 118 +++++++++++++++++++-------------------- tests/qemu-iotests/common.filter | 3 +- tests/qemu-iotests/common.rc | 3 + 4 files changed, 67 insertions(+), 60 deletions(-) diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/149 +++ b/tests/qemu-iotests/149 @@ -XXX,XX +XXX,XX @@ def cryptsetup_add_password(config, slot): args = ["luksAddKey", config.image_path(), "--key-slot", slot, "--key-file", "-", + "--iter-time", "10", pwfile] cryptsetup(args, password) @@ -XXX,XX +XXX,XX @@ def cryptsetup_format(config): args.extend(["--hash", config.hash]) args.extend(["--key-slot", slot]) args.extend(["--key-file", "-"]) + args.extend(["--iter-time", "10"]) args.append(config.image_path()) cryptsetup(args, password) @@ -XXX,XX +XXX,XX @@ def qemu_img_create(config, size_mb): opts = [ "key-secret=sec0", + "iter-time=10", "cipher-alg=%s-%d" % (config.cipher, config.keylen), "cipher-mode=%s" % config.mode, "ivgen-alg=%s" % config.ivgen, diff --git a/tests/qemu-iotests/149.out b/tests/qemu-iotests/149.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/149.out +++ b/tests/qemu-iotests/149.out @@ -XXX,XX +XXX,XX @@ # Create image truncate TEST_DIR/luks-aes-256-xts-plain64-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-256-xts-plain64-sha1.img +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha1.img qiotest-145-aes-256-xts-plain64-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha1.img # ================= qemu-img aes-256-xts-plain64-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain64-sha1.img 4194304M -Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain64-sha1.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha1.img qiotest-145-aes-256-xts-plain64-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha1.img # Create image truncate TEST_DIR/luks-twofish-256-xts-plain64-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher twofish-xts-plain64 --key-size 512 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-twofish-256-xts-plain64-sha1.img +sudo cryptsetup -q -v luksFormat --cipher twofish-xts-plain64 --key-size 512 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-twofish-256-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-256-xts-plain64-sha1.img qiotest-145-twofish-256-xts-plain64-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-twofish-256-xts-plain64-sha1.img # ================= qemu-img twofish-256-xts-plain64-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=twofish-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-256-xts-plain64-sha1.img 4194304M -Formatting 'TEST_DIR/luks-twofish-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=twofish-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-256-xts-plain64-sha1.img 4194304M +Formatting 'TEST_DIR/luks-twofish-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-256-xts-plain64-sha1.img qiotest-145-twofish-256-xts-plain64-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-twofish-256-xts-plain64-sha1.img # Create image truncate TEST_DIR/luks-serpent-256-xts-plain64-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher serpent-xts-plain64 --key-size 512 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-serpent-256-xts-plain64-sha1.img +sudo cryptsetup -q -v luksFormat --cipher serpent-xts-plain64 --key-size 512 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-serpent-256-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-256-xts-plain64-sha1.img qiotest-145-serpent-256-xts-plain64-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-256-xts-plain64-sha1.img # ================= qemu-img serpent-256-xts-plain64-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=serpent-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-256-xts-plain64-sha1.img 4194304M -Formatting 'TEST_DIR/luks-serpent-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-256-xts-plain64-sha1.img 4194304M +Formatting 'TEST_DIR/luks-serpent-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-256-xts-plain64-sha1.img qiotest-145-serpent-256-xts-plain64-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-256-xts-plain64-sha1.img # Create image truncate TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher cast5-cbc-plain64 --key-size 128 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img +sudo cryptsetup -q -v luksFormat --cipher cast5-cbc-plain64 --key-size 128 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img qiotest-145-cast5-128-cbc-plain64-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img # ================= qemu-img cast5-128-cbc-plain64-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=cast5-128,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img 4194304M -Formatting 'TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=cast5-128 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=cast5-128,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img 4194304M +Formatting 'TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=cast5-128 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img qiotest-145-cast5-128-cbc-plain64-sha1 @@ -XXX,XX +XXX,XX @@ Skipping cast6-256-xts-plain64-sha1 in blacklist # Create image truncate TEST_DIR/luks-aes-256-cbc-plain-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-cbc-plain --key-size 256 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-256-cbc-plain-sha1.img +sudo cryptsetup -q -v luksFormat --cipher aes-cbc-plain --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-plain-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain-sha1.img qiotest-145-aes-256-cbc-plain-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain-sha1.img # ================= qemu-img aes-256-cbc-plain-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain-sha1.img 4194304M -Formatting 'TEST_DIR/luks-aes-256-cbc-plain-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain-sha1.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-cbc-plain-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain-sha1.img qiotest-145-aes-256-cbc-plain-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain-sha1.img # Create image truncate TEST_DIR/luks-aes-256-cbc-plain64-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-cbc-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-256-cbc-plain64-sha1.img +sudo cryptsetup -q -v luksFormat --cipher aes-cbc-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha1.img qiotest-145-aes-256-cbc-plain64-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain64-sha1.img # ================= qemu-img aes-256-cbc-plain64-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain64-sha1.img 4194304M -Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain64-sha1.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha1.img qiotest-145-aes-256-cbc-plain64-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain64-sha1.img # Create image truncate TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-cbc-essiv:sha256 --key-size 256 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img +sudo cryptsetup -q -v luksFormat --cipher aes-cbc-essiv:sha256 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img qiotest-145-aes-256-cbc-essiv-sha256-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img # ================= qemu-img aes-256-cbc-essiv-sha256-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img 4194304M -Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img qiotest-145-aes-256-cbc-essiv-sha256-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img # Create image truncate TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-xts-essiv:sha256 --key-size 512 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img +sudo cryptsetup -q -v luksFormat --cipher aes-xts-essiv:sha256 --key-size 512 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img qiotest-145-aes-256-xts-essiv-sha256-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img # ================= qemu-img aes-256-xts-essiv-sha256-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img 4194304M -Formatting 'TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img qiotest-145-aes-256-xts-essiv-sha256-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img # Create image truncate TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img qiotest-145-aes-128-xts-plain64-sha256-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img # ================= qemu-img aes-128-xts-plain64-sha256-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img 4194304M -Formatting 'TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img 4194304M +Formatting 'TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img qiotest-145-aes-128-xts-plain64-sha256-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img # Create image truncate TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 384 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 384 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img qiotest-145-aes-192-xts-plain64-sha256-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img # ================= qemu-img aes-192-xts-plain64-sha256-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img 4194304M -Formatting 'TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img 4194304M +Formatting 'TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img qiotest-145-aes-192-xts-plain64-sha256-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img # Create image truncate TEST_DIR/luks-twofish-128-xts-plain64-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher twofish-xts-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-twofish-128-xts-plain64-sha1.img +sudo cryptsetup -q -v luksFormat --cipher twofish-xts-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-twofish-128-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-128-xts-plain64-sha1.img qiotest-145-twofish-128-xts-plain64-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-twofish-128-xts-plain64-sha1.img # ================= qemu-img twofish-128-xts-plain64-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=twofish-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-128-xts-plain64-sha1.img 4194304M -Formatting 'TEST_DIR/luks-twofish-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=twofish-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-128-xts-plain64-sha1.img 4194304M +Formatting 'TEST_DIR/luks-twofish-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-128-xts-plain64-sha1.img qiotest-145-twofish-128-xts-plain64-sha1 @@ -XXX,XX +XXX,XX @@ Skipping twofish-192-xts-plain64-sha1 in blacklist # Create image truncate TEST_DIR/luks-serpent-128-xts-plain64-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher serpent-xts-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-serpent-128-xts-plain64-sha1.img +sudo cryptsetup -q -v luksFormat --cipher serpent-xts-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-serpent-128-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-128-xts-plain64-sha1.img qiotest-145-serpent-128-xts-plain64-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-128-xts-plain64-sha1.img # ================= qemu-img serpent-128-xts-plain64-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=serpent-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-128-xts-plain64-sha1.img 4194304M -Formatting 'TEST_DIR/luks-serpent-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-128-xts-plain64-sha1.img 4194304M +Formatting 'TEST_DIR/luks-serpent-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-128-xts-plain64-sha1.img qiotest-145-serpent-128-xts-plain64-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-128-xts-plain64-sha1.img # Create image truncate TEST_DIR/luks-serpent-192-xts-plain64-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher serpent-xts-plain64 --key-size 384 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-serpent-192-xts-plain64-sha1.img +sudo cryptsetup -q -v luksFormat --cipher serpent-xts-plain64 --key-size 384 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-serpent-192-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-192-xts-plain64-sha1.img qiotest-145-serpent-192-xts-plain64-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-192-xts-plain64-sha1.img # ================= qemu-img serpent-192-xts-plain64-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=serpent-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-192-xts-plain64-sha1.img 4194304M -Formatting 'TEST_DIR/luks-serpent-192-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-192-xts-plain64-sha1.img 4194304M +Formatting 'TEST_DIR/luks-serpent-192-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-192-xts-plain64-sha1.img qiotest-145-serpent-192-xts-plain64-sha1 @@ -XXX,XX +XXX,XX @@ Skipping cast6-192-xts-plain64-sha1 in blacklist # Create image truncate TEST_DIR/luks-aes-256-xts-plain64-sha256.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha256 --key-slot 0 --key-file - TEST_DIR/luks-aes-256-xts-plain64-sha256.img +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha256 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha256.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha256.img qiotest-145-aes-256-xts-plain64-sha256 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha256.img # ================= qemu-img aes-256-xts-plain64-sha256 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha256 TEST_DIR/luks-aes-256-xts-plain64-sha256.img 4194304M -Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha256.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha256 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha256 TEST_DIR/luks-aes-256-xts-plain64-sha256.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha256.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha256 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha256.img qiotest-145-aes-256-xts-plain64-sha256 @@ -XXX,XX +XXX,XX @@ Skipping aes-256-xts-plain64-ripemd160 in blacklist # Create image truncate TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain --key-size 512 --hash sha1 --key-slot 3 --key-file - TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain --key-size 512 --hash sha1 --key-slot 3 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img qiotest-145-aes-256-xts-plain-sha1-pwslot3 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img # Create image truncate TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain --key-size 512 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain --key-size 512 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img # Add password slot 1 -sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 1 --key-file - TEST_DIR/passwd.txt +sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 1 --key-file - --iter-time 10 TEST_DIR/passwd.txt # Add password slot 2 -sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 2 --key-file - TEST_DIR/passwd.txt +sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 2 --key-file - --iter-time 10 TEST_DIR/passwd.txt # Add password slot 3 -sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 3 --key-file - TEST_DIR/passwd.txt +sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 3 --key-file - --iter-time 10 TEST_DIR/passwd.txt # Add password slot 4 -sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 4 --key-file - TEST_DIR/passwd.txt +sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 4 --key-file - --iter-time 10 TEST_DIR/passwd.txt # Add password slot 5 -sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 5 --key-file - TEST_DIR/passwd.txt +sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 5 --key-file - --iter-time 10 TEST_DIR/passwd.txt # Add password slot 6 -sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 6 --key-file - TEST_DIR/passwd.txt +sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 6 --key-file - --iter-time 10 TEST_DIR/passwd.txt # Add password slot 7 -sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 7 --key-file - TEST_DIR/passwd.txt +sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 7 --key-file - --iter-time 10 TEST_DIR/passwd.txt # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img qiotest-145-aes-256-xts-plain-sha1-pwallslots # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img # ================= qemu-img aes-256-xts-plain-sha1-pwallslots ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=c2xvdDE=,format=base64 -o key-secret=sec0,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img 4194304M -Formatting 'TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=c2xvdDE=,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img qiotest-145-aes-256-xts-plain-sha1-pwallslots @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img # Create image truncate TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-cbc-essiv:sha256 --key-size 256 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img +sudo cryptsetup -q -v luksFormat --cipher aes-cbc-essiv:sha256 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img qiotest-145-aes-256-cbc-essiv-auto-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img # ================= qemu-img aes-256-cbc-essiv-auto-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img 4194304M -Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img qiotest-145-aes-256-cbc-essiv-auto-sha1 @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img # Create image truncate TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img --size 4194304MB # Format image -sudo cryptsetup -q -v luksFormat --cipher aes-cbc-plain64:sha256 --key-size 256 --hash sha1 --key-slot 0 --key-file - TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img +sudo cryptsetup -q -v luksFormat --cipher aes-cbc-plain64:sha256 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img qiotest-145-aes-256-cbc-plain64-sha256-sha1 # Set dev owner @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img # ================= qemu-img aes-256-cbc-plain64-sha256-sha1 ================= # Create image -qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img 4194304M -Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 ivgen-hash-alg=sha256 hash-alg=sha1 +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img qiotest-145-aes-256-cbc-plain64-sha256-sha1 diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -XXX,XX +XXX,XX @@ _filter_img_create() -e "s# block_state_zero=\\(on\\|off\\)##g" \ -e "s# log_size=[0-9]\\+##g" \ -e "s# refcount_bits=[0-9]\\+##g" \ - -e "s# key-secret=[a-zA-Z0-9]\\+##g" + -e "s# key-secret=[a-zA-Z0-9]\\+##g" \ + -e "s# iter-time=[0-9]\\+##g" } _filter_img_info() diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -XXX,XX +XXX,XX @@ _set_default_imgopts() if [ "$IMGFMT" == "qcow2" ] && ! (echo "$IMGOPTS" | grep "compat=" > /dev/null); then IMGOPTS=$(_optstr_add "$IMGOPTS" "compat=1.1") fi + if [ "$IMGFMT" == "luks" ] && ! (echo "$IMGOPTS" | grep "iter-time=" > /dev/null); then + IMGOPTS=$(_optstr_add "$IMGOPTS" "iter-time=10") + fi } _use_sample_img() -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> Add tests for sha224, sha512, sha384 and ripemd160 hash algorithms. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170626123510.20134-5-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/149 | 10 +- tests/qemu-iotests/149.out | 482 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 484 insertions(+), 8 deletions(-) diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/149 +++ b/tests/qemu-iotests/149 @@ -XXX,XX +XXX,XX @@ configs = [ # LUKS default but diff hash + LUKSConfig("aes-256-xts-plain64-sha224", + "aes", 256, "xts", "plain64", None, "sha224"), LUKSConfig("aes-256-xts-plain64-sha256", "aes", 256, "xts", "plain64", None, "sha256"), + LUKSConfig("aes-256-xts-plain64-sha384", + "aes", 256, "xts", "plain64", None, "sha384"), LUKSConfig("aes-256-xts-plain64-sha512", "aes", 256, "xts", "plain64", None, "sha512"), LUKSConfig("aes-256-xts-plain64-ripemd160", @@ -XXX,XX +XXX,XX @@ blacklist = [ # GCrypt doesn't support Twofish with 192 bit key "twofish-192-xts-plain64-sha1", - - # We don't have sha512 hash wired up yet - "aes-256-xts-plain64-sha512", - - # We don't have ripemd160 hash wired up yet - "aes-256-xts-plain64-ripemd160", ] whitelist = [] diff --git a/tests/qemu-iotests/149.out b/tests/qemu-iotests/149.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/149.out +++ b/tests/qemu-iotests/149.out @@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-192-xts-plain64-sha1.img Skipping cast6-128-xts-plain64-sha1 in blacklist Skipping cast6-192-xts-plain64-sha1 in blacklist +# ================= dm-crypt aes-256-xts-plain64-sha224 ================= +# Create image +truncate TEST_DIR/luks-aes-256-xts-plain64-sha224.img --size 4194304MB +# Format image +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha224 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha224.img +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +# Write test pattern 0xa7 +qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x13 +qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha224 +# Read test pattern 0xa7 +qemu-io -c read -P 0xa7 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha224.img +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x13 +qemu-io -c read -P 0x13 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha224.img +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x91 +qemu-io -c write -P 0x91 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha224.img +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x5e +qemu-io -c write -P 0x5e 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha224.img +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +# Read test pattern 0x91 +qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x5e +qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha224 +# Delete image +unlink TEST_DIR/luks-aes-256-xts-plain64-sha224.img + +# ================= qemu-img aes-256-xts-plain64-sha224 ================= +# Create image +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha224 TEST_DIR/luks-aes-256-xts-plain64-sha224.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha224.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha224 iter-time=10 + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +# Write test pattern 0xa7 +qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x13 +qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha224 +# Read test pattern 0xa7 +qemu-io -c read -P 0xa7 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha224.img +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x13 +qemu-io -c read -P 0x13 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha224.img +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x91 +qemu-io -c write -P 0x91 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha224.img +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x5e +qemu-io -c write -P 0x5e 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha224.img +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +# Read test pattern 0x91 +qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x5e +qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha224 +# Delete image +unlink TEST_DIR/luks-aes-256-xts-plain64-sha224.img + # ================= dm-crypt aes-256-xts-plain64-sha256 ================= # Create image truncate TEST_DIR/luks-aes-256-xts-plain64-sha256.img --size 4194304MB @@ -XXX,XX +XXX,XX @@ sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha256 # Delete image unlink TEST_DIR/luks-aes-256-xts-plain64-sha256.img -Skipping aes-256-xts-plain64-sha512 in blacklist -Skipping aes-256-xts-plain64-ripemd160 in blacklist +# ================= dm-crypt aes-256-xts-plain64-sha384 ================= +# Create image +truncate TEST_DIR/luks-aes-256-xts-plain64-sha384.img --size 4194304MB +# Format image +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha384 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha384.img +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +# Write test pattern 0xa7 +qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x13 +qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha384 +# Read test pattern 0xa7 +qemu-io -c read -P 0xa7 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha384.img +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x13 +qemu-io -c read -P 0x13 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha384.img +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x91 +qemu-io -c write -P 0x91 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha384.img +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x5e +qemu-io -c write -P 0x5e 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha384.img +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +# Read test pattern 0x91 +qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x5e +qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha384 +# Delete image +unlink TEST_DIR/luks-aes-256-xts-plain64-sha384.img + +# ================= qemu-img aes-256-xts-plain64-sha384 ================= +# Create image +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha384 TEST_DIR/luks-aes-256-xts-plain64-sha384.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha384.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha384 iter-time=10 + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +# Write test pattern 0xa7 +qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x13 +qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha384 +# Read test pattern 0xa7 +qemu-io -c read -P 0xa7 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha384.img +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x13 +qemu-io -c read -P 0x13 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha384.img +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x91 +qemu-io -c write -P 0x91 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha384.img +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x5e +qemu-io -c write -P 0x5e 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha384.img +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +# Read test pattern 0x91 +qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x5e +qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha384 +# Delete image +unlink TEST_DIR/luks-aes-256-xts-plain64-sha384.img + +# ================= dm-crypt aes-256-xts-plain64-sha512 ================= +# Create image +truncate TEST_DIR/luks-aes-256-xts-plain64-sha512.img --size 4194304MB +# Format image +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha512 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha512.img +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +# Write test pattern 0xa7 +qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x13 +qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha512 +# Read test pattern 0xa7 +qemu-io -c read -P 0xa7 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha512.img +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x13 +qemu-io -c read -P 0x13 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha512.img +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x91 +qemu-io -c write -P 0x91 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha512.img +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x5e +qemu-io -c write -P 0x5e 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha512.img +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +# Read test pattern 0x91 +qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x5e +qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha512 +# Delete image +unlink TEST_DIR/luks-aes-256-xts-plain64-sha512.img + +# ================= qemu-img aes-256-xts-plain64-sha512 ================= +# Create image +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha512 TEST_DIR/luks-aes-256-xts-plain64-sha512.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha512.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha512 iter-time=10 + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +# Write test pattern 0xa7 +qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x13 +qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha512 +# Read test pattern 0xa7 +qemu-io -c read -P 0xa7 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha512.img +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x13 +qemu-io -c read -P 0x13 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha512.img +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x91 +qemu-io -c write -P 0x91 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha512.img +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x5e +qemu-io -c write -P 0x5e 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-sha512.img +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +# Read test pattern 0x91 +qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x5e +qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-sha512 +# Delete image +unlink TEST_DIR/luks-aes-256-xts-plain64-sha512.img + +# ================= dm-crypt aes-256-xts-plain64-ripemd160 ================= +# Create image +truncate TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img --size 4194304MB +# Format image +sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash ripemd160 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +# Write test pattern 0xa7 +qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x13 +qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-ripemd160 +# Read test pattern 0xa7 +qemu-io -c read -P 0xa7 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x13 +qemu-io -c read -P 0x13 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x91 +qemu-io -c write -P 0x91 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x5e +qemu-io -c write -P 0x5e 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +# Read test pattern 0x91 +qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x5e +qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-ripemd160 +# Delete image +unlink TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img + +# ================= qemu-img aes-256-xts-plain64-ripemd160 ================= +# Create image +qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=ripemd160 TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img 4194304M +Formatting 'TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=ripemd160 iter-time=10 + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +# Write test pattern 0xa7 +qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x13 +qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-ripemd160 +# Read test pattern 0xa7 +qemu-io -c read -P 0xa7 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x13 +qemu-io -c read -P 0x13 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x91 +qemu-io -c write -P 0x91 100M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img +wrote 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Write test pattern 0x5e +qemu-io -c write -P 0x5e 3145728M 10M --object secret,id=sec0,data=MTIzNDU2,format=base64 --image-opts driver=luks,key-secret=sec0,file.filename=TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img +wrote 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Open dev +sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160 +# Set dev owner +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +# Read test pattern 0x91 +qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +read 10485760/10485760 bytes at offset 104857600 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Read test pattern 0x5e +qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 +read 10485760/10485760 bytes at offset 3298534883328 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +# Close dev +sudo cryptsetup -q -v luksClose qiotest-145-aes-256-xts-plain64-ripemd160 +# Delete image +unlink TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img + # ================= dm-crypt aes-256-xts-plain-sha1-pwslot3 ================= # Create image truncate TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img --size 4194304MB -- 1.8.3.1 From: "Daniel P. Berrange" <berrange@redhat.com> On some distros, whenever you close a block device file descriptor there is a udev rule that resets the file permissions. This can race with the test script when we run qemu-io multiple times against the same block device. Occasionally the second qemu-io invocation will find udev has reset the permissions causing failure. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 20170626123510.20134-6-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/149 | 12 +- tests/qemu-iotests/149.out | 344 ++++++++++++++++++++++----------------------- 2 files changed, 177 insertions(+), 179 deletions(-) diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/149 +++ b/tests/qemu-iotests/149 @@ -XXX,XX +XXX,XX @@ def chown(config): msg = proc.communicate()[0] if proc.returncode != 0: - raise Exception("Cannot change owner on %s" % path) + raise Exception(msg) def cryptsetup_open(config): @@ -XXX,XX +XXX,XX @@ def qemu_io_image_args(config, dev=False): def qemu_io_write_pattern(config, pattern, offset_mb, size_mb, dev=False): """Write a pattern of data to a LUKS image or device""" + if dev: + chown(config) args = ["-c", "write -P 0x%x %dM %dM" % (pattern, offset_mb, size_mb)] args.extend(qemu_io_image_args(config, dev)) iotests.log("qemu-io " + " ".join(args), filters=[iotests.filter_test_dir]) @@ -XXX,XX +XXX,XX @@ def qemu_io_write_pattern(config, pattern, offset_mb, size_mb, dev=False): def qemu_io_read_pattern(config, pattern, offset_mb, size_mb, dev=False): """Read a pattern of data to a LUKS image or device""" + if dev: + chown(config) args = ["-c", "read -P 0x%x %dM %dM" % (pattern, offset_mb, size_mb)] args.extend(qemu_io_image_args(config, dev)) iotests.log("qemu-io " + " ".join(args), filters=[iotests.filter_test_dir]) @@ -XXX,XX +XXX,XX @@ def test_once(config, qemu_img=False): cryptsetup_open(config) try: - iotests.log("# Set dev owner") - chown(config) - iotests.log("# Write test pattern 0xa7") qemu_io_write_pattern(config, 0xa7, lowOffsetMB, 10, dev=True) iotests.log("# Write test pattern 0x13") @@ -XXX,XX +XXX,XX @@ def test_once(config, qemu_img=False): cryptsetup_open(config) try: - iotests.log("# Set dev owner") - chown(config) - iotests.log("# Read test pattern 0x91") qemu_io_read_pattern(config, 0x91, lowOffsetMB, 10, dev=True) iotests.log("# Read test pattern 0x5e") diff --git a/tests/qemu-iotests/149.out b/tests/qemu-iotests/149.out index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/149.out +++ b/tests/qemu-iotests/149.out @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-xts-plain64-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha1.img qiotest-145-aes-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha1.img qiotest-145-aes-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha1.img', fmt=luks size=439804651 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha1.img qiotest-145-aes-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha1.img qiotest-145-aes-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-twofish-256-xts-plain64-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher twofish-xts-plain64 --key-size 512 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-twofish-256-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-256-xts-plain64-sha1.img qiotest-145-twofish-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-256-xts-plain64-sha1.img qiotest-145-twofish-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-twofish-256-xts-plain64-sha1.img', fmt=luks size=43980 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-256-xts-plain64-sha1.img qiotest-145-twofish-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-256-xts-plain64-sha1.img qiotest-145-twofish-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-serpent-256-xts-plain64-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher serpent-xts-plain64 --key-size 512 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-serpent-256-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-256-xts-plain64-sha1.img qiotest-145-serpent-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-256-xts-plain64-sha1.img qiotest-145-serpent-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-serpent-256-xts-plain64-sha1.img', fmt=luks size=43980 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-256-xts-plain64-sha1.img qiotest-145-serpent-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-256-xts-plain64-sha1.img qiotest-145-serpent-256-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-256-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher cast5-cbc-plain64 --key-size 128 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img qiotest-145-cast5-128-cbc-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img qiotest-145-cast5-128-cbc-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img', fmt=luks size=4398046 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img qiotest-145-cast5-128-cbc-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img qiotest-145-cast5-128-cbc-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-cast5-128-cbc-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-cbc-plain-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-cbc-plain --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-plain-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain-sha1.img qiotest-145-aes-256-cbc-plain-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain-sha1.img qiotest-145-aes-256-cbc-plain-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-cbc-plain-sha1.img', fmt=luks size=43980465111 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain-sha1.img qiotest-145-aes-256-cbc-plain-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain-sha1.img qiotest-145-aes-256-cbc-plain-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-cbc-plain64-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-cbc-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha1.img qiotest-145-aes-256-cbc-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha1.img qiotest-145-aes-256-cbc-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha1.img', fmt=luks size=439804651 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha1.img qiotest-145-aes-256-cbc-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha1.img qiotest-145-aes-256-cbc-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-cbc-essiv:sha256 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img qiotest-145-aes-256-cbc-essiv-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img qiotest-145-aes-256-cbc-essiv-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img', fmt=luks size=4398 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img qiotest-145-aes-256-cbc-essiv-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img qiotest-145-aes-256-cbc-essiv-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-essiv:sha256 --key-size 512 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img qiotest-145-aes-256-xts-essiv-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img qiotest-145-aes-256-xts-essiv-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img', fmt=luks size=4398 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img qiotest-145-aes-256-xts-essiv-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img qiotest-145-aes-256-xts-essiv-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-essiv-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img qiotest-145-aes-128-xts-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img qiotest-145-aes-128-xts-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img', fmt=luks size=43 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img qiotest-145-aes-128-xts-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img qiotest-145-aes-128-xts-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-128-xts-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 384 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img qiotest-145-aes-192-xts-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img qiotest-145-aes-192-xts-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img', fmt=luks size=43 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img qiotest-145-aes-192-xts-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img qiotest-145-aes-192-xts-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-192-xts-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-twofish-128-xts-plain64-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher twofish-xts-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-twofish-128-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-128-xts-plain64-sha1.img qiotest-145-twofish-128-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-128-xts-plain64-sha1.img qiotest-145-twofish-128-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-twofish-128-xts-plain64-sha1.img', fmt=luks size=43980 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-128-xts-plain64-sha1.img qiotest-145-twofish-128-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-128-xts-plain64-sha1.img qiotest-145-twofish-128-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-twofish-128-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-serpent-128-xts-plain64-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher serpent-xts-plain64 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-serpent-128-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-128-xts-plain64-sha1.img qiotest-145-serpent-128-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-128-xts-plain64-sha1.img qiotest-145-serpent-128-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-serpent-128-xts-plain64-sha1.img', fmt=luks size=43980 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-128-xts-plain64-sha1.img qiotest-145-serpent-128-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-128-xts-plain64-sha1.img qiotest-145-serpent-128-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-128-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-serpent-192-xts-plain64-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher serpent-xts-plain64 --key-size 384 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-serpent-192-xts-plain64-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-192-xts-plain64-sha1.img qiotest-145-serpent-192-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-192-xts-plain64-sha1.img qiotest-145-serpent-192-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-serpent-192-xts-plain64-sha1.img', fmt=luks size=43980 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-192-xts-plain64-sha1.img qiotest-145-serpent-192-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-192-xts-plain64-sha1.img qiotest-145-serpent-192-xts-plain64-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-serpent-192-xts-plain64-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-xts-plain64-sha224.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha224 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha224.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha224.img', fmt=luks size=4398046 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha224 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-xts-plain64-sha256.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha256 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha256.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha256.img qiotest-145-aes-256-xts-plain64-sha256 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha256.img qiotest-145-aes-256-xts-plain64-sha256 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha256.img', fmt=luks size=4398046 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha256.img qiotest-145-aes-256-xts-plain64-sha256 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha256.img qiotest-145-aes-256-xts-plain64-sha256 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha256 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-xts-plain64-sha384.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha384 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha384.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha384.img', fmt=luks size=4398046 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha384 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-xts-plain64-sha512.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha512 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-sha512.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha512.img', fmt=luks size=4398046 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-sha512 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain64 --key-size 512 --hash ripemd160 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img', fmt=luks size=4398 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain64-ripemd160 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-xts-plain --key-size 512 --hash sha1 --key-slot 3 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img qiotest-145-aes-256-xts-plain-sha1-pwslot3 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwslot3.img qiotest-145-aes-256-xts-plain-sha1-pwslot3 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwslot3 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots sudo cryptsetup -q -v luksAddKey TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img --key-slot 7 --key-file - --iter-time 10 TEST_DIR/passwd.txt # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img qiotest-145-aes-256-xts-plain-sha1-pwallslots -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img qiotest-145-aes-256-xts-plain-sha1-pwallslots -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img', fmt=luks size= # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img qiotest-145-aes-256-xts-plain-sha1-pwallslots -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img qiotest-145-aes-256-xts-plain-sha1-pwallslots -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-xts-plain-sha1-pwallslots read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-cbc-essiv:sha256 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img qiotest-145-aes-256-cbc-essiv-auto-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img qiotest-145-aes-256-cbc-essiv-auto-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img', fmt=luks size=439804 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img qiotest-145-aes-256-cbc-essiv-auto-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img qiotest-145-aes-256-cbc-essiv-auto-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-essiv-auto-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ truncate TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img --size 4194304MB sudo cryptsetup -q -v luksFormat --cipher aes-cbc-plain64:sha256 --key-size 256 --hash sha1 --key-slot 0 --key-file - --iter-time 10 TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img qiotest-145-aes-256-cbc-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img qiotest-145-aes-256-cbc-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img', fmt=luks size=43 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img qiotest-145-aes-256-cbc-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 # Write test pattern 0xa7 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 qemu-io -c write -P 0xa7 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Write test pattern 0x13 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 qemu-io -c write -P 0x13 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 wrote 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -XXX,XX +XXX,XX @@ wrote 10485760/10485760 bytes at offset 3298534883328 # Open dev sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img qiotest-145-aes-256-cbc-plain64-sha256-sha1 -# Set dev owner -sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 # Read test pattern 0x91 +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 qemu-io -c read -P 0x91 100M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 104857600 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) # Read test pattern 0x5e +sudo chown UID:GID /dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 qemu-io -c read -P 0x5e 3145728M 10M --image-opts driver=file,filename=/dev/mapper/qiotest-145-aes-256-cbc-plain64-sha256-sha1 read 10485760/10485760 bytes at offset 3298534883328 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -- 1.8.3.1 From: Max Reitz <mreitz@redhat.com> A user may specify a relative path for accessing qemu, qemu-img, etc. through environment variables ($QEMU_PROG and friends) or a symlink. If a test decides to change its working directory, relative paths will cease to work, however. Work around this by making all of the paths to programs that should undergo testing absolute. Besides "realpath", we also have to use "type -p" to support programs in $PATH. As a side effect, this fixes specifying these programs as symlinks for out-of-tree builds: Before, you would have to create two symlinks, one in the build and one in the source tree (the first one for common.config to find, the second one for the iotest to use). Now it is sufficient to create one in the build tree because common.config will resolve it. Reported-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20170702150510.23276-2-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/common.config | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/common.config +++ b/tests/qemu-iotests/common.config @@ -XXX,XX +XXX,XX @@ if [ -z "$QEMU_VXHS_PROG" ]; then export QEMU_VXHS_PROG="`set_prog_path qnio_server`" fi +export QEMU_PROG=$(realpath -- "$(type -p "$QEMU_PROG")") +export QEMU_IMG_PROG=$(realpath -- "$(type -p "$QEMU_IMG_PROG")") +export QEMU_IO_PROG=$(realpath -- "$(type -p "$QEMU_IO_PROG")") +export QEMU_NBD_PROG=$(realpath -- "$(type -p "$QEMU_NBD_PROG")") + +# This program is not built as part of qemu but (possibly) provided by the +# system, so it may not be present at all +if [ -n "$QEMU_VXHS_PROG" ]; then + export QEMU_VXHS_PROG=$(realpath -- "$(type -p "$QEMU_VXHS_PROG")") +fi + _qemu_wrapper() { ( -- 1.8.3.1 From: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20170702150510.23276-3-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/126 | 105 +++++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/126.out | 23 ++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 129 insertions(+) create mode 100755 tests/qemu-iotests/126 create mode 100644 tests/qemu-iotests/126.out diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126 new file mode 100755 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/126 @@ -XXX,XX +XXX,XX @@ +#!/bin/bash +# +# Tests handling of colons in filenames (which may be confused with protocol +# prefixes) +# +# Copyright (C) 2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator +owner=mreitz@redhat.com + +seq="$(basename $0)" +echo "QA output created by $seq" + +here="$PWD" +status=1 # failure is the default! + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# Needs backing file support +_supported_fmt qcow qcow2 qed vmdk +# This is the default protocol (and we want to test the difference between +# colons which separate a protocol prefix from the rest and colons which are +# just part of the filename, so we cannot test protocols which require a prefix) +_supported_proto file +_supported_os Linux + +echo +echo '=== Testing plain files ===' +echo + +# A colon after a slash is not a protocol prefix separator +TEST_IMG="$TEST_DIR/a:b.$IMGFMT" _make_test_img 64M +_rm_test_img "$TEST_DIR/a:b.$IMGFMT" + +# But if you want to be really sure, you can do this +TEST_IMG="file:$TEST_DIR/a:b.$IMGFMT" _make_test_img 64M +_rm_test_img "$TEST_DIR/a:b.$IMGFMT" + + +echo +echo '=== Testing relative backing filename resolution ===' +echo + +BASE_IMG="$TEST_DIR/image:base.$IMGFMT" +TOP_IMG="$TEST_DIR/image:top.$IMGFMT" + +TEST_IMG=$BASE_IMG _make_test_img 64M +TEST_IMG=$TOP_IMG _make_test_img -b ./image:base.$IMGFMT + +# The default cluster size depends on the image format +TEST_IMG=$TOP_IMG _img_info | grep -v 'cluster_size' + +_rm_test_img "$BASE_IMG" +_rm_test_img "$TOP_IMG" + + +# Do another test where we access both top and base without any slash in them +echo +pushd "$TEST_DIR" >/dev/null + +BASE_IMG="base.$IMGFMT" +TOP_IMG="file:image:top.$IMGFMT" + +TEST_IMG=$BASE_IMG _make_test_img 64M +TEST_IMG=$TOP_IMG _make_test_img -b "$BASE_IMG" + +TEST_IMG=$TOP_IMG _img_info | grep -v 'cluster_size' + +_rm_test_img "$BASE_IMG" +_rm_test_img "image:top.$IMGFMT" + +popd >/dev/null + +# Note that we could also do the same test with BASE_IMG=file:image:base.$IMGFMT +# -- but behavior for that case is a bit strange. Protocol-prefixed paths are +# in a sense always absolute paths, so such paths will never be combined with +# the path of the overlay. But since "image:base.$IMGFMT" is actually a +# relative path, it will always be evaluated relative to qemu's CWD (but not +# relative to the overlay!). While this is more or less intended, it is still +# pretty strange and thus not something that is tested here. +# (The root of the issue is the use of a relative path with a protocol prefix. +# This may always give you weird results because in one sense, qemu considers +# such paths absolute, whereas in another, they are still relative.) + + +# success, all done +echo '*** done' +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/126.out b/tests/qemu-iotests/126.out new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemu-iotests/126.out @@ -XXX,XX +XXX,XX @@ +QA output created by 126 + +=== Testing plain files === + +Formatting 'TEST_DIR/a:b.IMGFMT', fmt=IMGFMT size=67108864 +Formatting 'TEST_DIR/a:b.IMGFMT', fmt=IMGFMT size=67108864 + +=== Testing relative backing filename resolution === + +Formatting 'TEST_DIR/image:base.IMGFMT', fmt=IMGFMT size=67108864 +Formatting 'TEST_DIR/image:top.IMGFMT', fmt=IMGFMT size=67108864 backing_file=./image:base.IMGFMT +image: TEST_DIR/image:top.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +backing file: ./image:base.IMGFMT (actual path: TEST_DIR/./image:base.IMGFMT) + +Formatting 'base.IMGFMT', fmt=IMGFMT size=67108864 +Formatting 'file:image:top.IMGFMT', fmt=IMGFMT size=67108864 backing_file=base.IMGFMT +image: ./image:top.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +backing file: base.IMGFMT (actual path: ./base.IMGFMT) +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index XXXXXXX..XXXXXXX 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -XXX,XX +XXX,XX @@ 122 rw auto 123 rw auto quick 124 rw auto backing +126 rw auto backing 128 rw auto quick 129 rw auto quick 130 rw auto quick -- 1.8.3.1 From: Eric Blake <eblake@redhat.com> POSIX says that backslashes in the arguments to 'echo', as well as any use of 'echo -n' and 'echo -e', are non-portable; it recommends people should favor 'printf' instead. This is definitely true where we do not control which shell is running (such as in makefile snippets or in documentation examples). But even for scripts where we require bash (and therefore, where echo does what we want by default), it is still possible to use 'shopt -s xpg_echo' to change bash's behavior of echo. And setting a good example never hurts when we are not sure if a snippet will be copied from a bash-only script to a general shell script (although I don't change the use of non-portable \e for ESC when we know the running shell is bash). Replace 'echo -n "..."' with 'printf %s "..."', and 'echo -e "..."' with 'printf %b "...\n"', with the optimization that the %s/%b argument can be omitted if the string being printed is a strict literal with no '%', '$', or '`' (we could technically also make this optimization when there are $ or `` substitutions but where we can prove their results will not be problematic, but proving that such substitutions are safe makes the patch less trivial compared to just being consistent). In the qemu-iotests check script, fix unusual shell quoting that would result in word-splitting if 'date' outputs a space. In test 051, take an opportunity to shorten the line. In test 068, get rid of a pointless second invocation of bash. CC: qemu-trivial@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Message-id: 20170703180950.9895-1-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com> --- qemu-options.hx | 4 ++-- tests/multiboot/run_test.sh | 10 +++++----- tests/qemu-iotests/051 | 7 ++++--- tests/qemu-iotests/068 | 2 +- tests/qemu-iotests/142 | 48 ++++++++++++++++++++++----------------------- tests/qemu-iotests/171 | 14 ++++++------- tests/qemu-iotests/check | 18 ++++++++--------- tests/rocker/all | 10 +++++----- tests/tcg/cris/Makefile | 8 ++++---- 9 files changed, 61 insertions(+), 60 deletions(-) diff --git a/qemu-options.hx b/qemu-options.hx index XXXXXXX..XXXXXXX 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -XXX,XX +XXX,XX @@ The simplest (insecure) usage is to provide the secret inline The simplest secure usage is to provide the secret via a file - # echo -n "letmein" > mypasswd.txt + # printf "letmein" > mypasswd.txt # $QEMU -object secret,id=sec0,file=mypasswd.txt,format=raw For greater security, AES-256-CBC should be used. To illustrate usage, @@ -XXX,XX +XXX,XX @@ telling openssl to base64 encode the result, but it could be left as raw bytes if desired. @example - # SECRET=$(echo -n "letmein" | + # SECRET=$(printf "letmein" | openssl enc -aes-256-cbc -a -K $KEY -iv $IV) @end example diff --git a/tests/multiboot/run_test.sh b/tests/multiboot/run_test.sh index XXXXXXX..XXXXXXX 100755 --- a/tests/multiboot/run_test.sh +++ b/tests/multiboot/run_test.sh @@ -XXX,XX +XXX,XX @@ run_qemu() { local kernel=$1 shift - echo -e "\n\n=== Running test case: $kernel $@ ===\n" >> test.log + printf %b "\n\n=== Running test case: $kernel $@ ===\n\n" >> test.log $QEMU \ -kernel $kernel \ @@ -XXX,XX +XXX,XX @@ for t in mmap modules; do pass=1 if [ $debugexit != 1 ]; then - echo -e "\e[31m ?? \e[0m $t (no debugexit used, exit code $ret)" + printf %b "\e[31m ?? \e[0m $t (no debugexit used, exit code $ret)\n" pass=0 elif [ $ret != 0 ]; then - echo -e "\e[31mFAIL\e[0m $t (exit code $ret)" + printf %b "\e[31mFAIL\e[0m $t (exit code $ret)\n" pass=0 fi if ! diff $t.out test.log > /dev/null 2>&1; then - echo -e "\e[31mFAIL\e[0m $t (output difference)" + printf %b "\e[31mFAIL\e[0m $t (output difference)\n" diff -u $t.out test.log pass=0 fi if [ $pass == 1 ]; then - echo -e "\e[32mPASS\e[0m $t" + printf %b "\e[32mPASS\e[0m $t\n" fi done diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/051 +++ b/tests/qemu-iotests/051 @@ -XXX,XX +XXX,XX @@ run_qemu -drive driver=null-co,cache=invalid_value # Test 142 checks the direct=on cases for cache in writeback writethrough unsafe invalid_value; do - echo -e "info block\ninfo block file\ninfo block backing\ninfo block backing-file" | \ + printf "info block %s\n" '' file backing backing-file | \ run_qemu -drive file="$TEST_IMG",cache=$cache,backing.file.filename="$TEST_IMG.base",backing.cache.no-flush=on,backing.node-name=backing,backing.file.node-name=backing-file,file.node-name=file,if=none,id=$device_id -nodefaults done @@ -XXX,XX +XXX,XX @@ echo "qemu-io $device_id \"write -P 0x22 0 4k\"" | run_qemu -drive file="$TEST_I $QEMU_IO -c "read -P 0x22 0 4k" "$TEST_IMG" | _filter_qemu_io -echo -e "qemu-io $device_id \"write -P 0x33 0 4k\"\ncommit $device_id" | run_qemu -drive file="$TEST_IMG",snapshot=on,if=none,id=$device_id\ - | _filter_qemu_io +printf %b "qemu-io $device_id \"write -P 0x33 0 4k\"\ncommit $device_id\n" | + run_qemu -drive file="$TEST_IMG",snapshot=on,if=none,id=$device_id | + _filter_qemu_io $QEMU_IO -c "read -P 0x33 0 4k" "$TEST_IMG" | _filter_qemu_io diff --git a/tests/qemu-iotests/068 b/tests/qemu-iotests/068 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/068 +++ b/tests/qemu-iotests/068 @@ -XXX,XX +XXX,XX @@ for extra_args in \ _make_test_img $IMG_SIZE # Give qemu some time to boot before saving the VM state - bash -c 'sleep 1; echo -e "savevm 0\nquit"' | _qemu $extra_args + { sleep 1; printf "savevm 0\nquit\n"; } | _qemu $extra_args # Now try to continue from that VM state (this should just work) echo quit | _qemu $extra_args -loadvm 0 done diff --git a/tests/qemu-iotests/142 b/tests/qemu-iotests/142 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/142 +++ b/tests/qemu-iotests/142 @@ -XXX,XX +XXX,XX @@ function check_cache_all() # cache.direct is supposed to be inherited by both bs->file and # bs->backing - echo -e "cache.direct=on on none0" + printf "cache.direct=on on none0\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",cache.direct=on | grep -e "Cache" -e "[Cc]annot|[Cc]ould not|[Cc]an't" - echo -e "\ncache.direct=on on file" + printf "\ncache.direct=on on file\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",file.cache.direct=on | grep -e "Cache" -e "[Cc]annot|[Cc]ould not|[Cc]an't" - echo -e "\ncache.direct=on on backing" + printf "\ncache.direct=on on backing\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",backing.cache.direct=on | grep -e "Cache" -e "[Cc]annot|[Cc]ould not|[Cc]an't" - echo -e "\ncache.direct=on on backing-file" + printf "\ncache.direct=on on backing-file\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",backing.file.cache.direct=on | grep -e "Cache" -e "[Cc]annot|[Cc]ould not|[Cc]an't" # cache.writeback is supposed to be inherited by bs->backing; bs->file # always gets cache.writeback=on - echo -e "\n\ncache.writeback=off on none0" + printf "\n\ncache.writeback=off on none0\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",cache.writeback=off | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.writeback=off on file" + printf "\ncache.writeback=off on file\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",file.cache.writeback=off | grep -e "doesn't" -e "does not" - echo -e "\ncache.writeback=off on backing" + printf "\ncache.writeback=off on backing\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",backing.cache.writeback=off | grep -e "doesn't" -e "does not" - echo -e "\ncache.writeback=off on backing-file" + printf "\ncache.writeback=off on backing-file\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",backing.file.cache.writeback=off | grep -e "doesn't" -e "does not" # cache.no-flush is supposed to be inherited by both bs->file and bs->backing - echo -e "\n\ncache.no-flush=on on none0" + printf "\n\ncache.no-flush=on on none0\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",cache.no-flush=on | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.no-flush=on on file" + printf "\ncache.no-flush=on on file\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",file.cache.no-flush=on | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.no-flush=on on backing" + printf "\ncache.no-flush=on on backing\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",backing.cache.no-flush=on | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.no-flush=on on backing-file" + printf "\ncache.no-flush=on on backing-file\n" echo "$hmp_cmds" | run_qemu -drive "$files","$ids",backing.file.cache.no-flush=on | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" } @@ -XXX,XX +XXX,XX @@ function check_cache_all_separate() { # Check cache.direct - echo -e "cache.direct=on on blk" + printf "cache.direct=on on blk\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile" -drive "$drv_bk" -drive "$drv_file" -drive "$drv_img",cache.direct=on | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.direct=on on file" + printf "\ncache.direct=on on file\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile" -drive "$drv_bk" -drive "$drv_file",cache.direct=on -drive "$drv_img" | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.direct=on on backing" + printf "\ncache.direct=on on backing\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile" -drive "$drv_bk",cache.direct=on -drive "$drv_file" -drive "$drv_img" | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.direct=on on backing-file" + printf "\ncache.direct=on on backing-file\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile",cache.direct=on -drive "$drv_bk" -drive "$drv_file" -drive "$drv_img" | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" # Check cache.writeback - echo -e "\n\ncache.writeback=off on blk" + printf "\n\ncache.writeback=off on blk\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile" -drive "$drv_bk" -drive "$drv_file" -drive "$drv_img",cache.writeback=off | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.writeback=off on file" + printf "\ncache.writeback=off on file\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile" -drive "$drv_bk" -drive "$drv_file",cache.writeback=off -drive "$drv_img" | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.writeback=off on backing" + printf "\ncache.writeback=off on backing\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile" -drive "$drv_bk",cache.writeback=off -drive "$drv_file" -drive "$drv_img" | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.writeback=off on backing-file" + printf "\ncache.writeback=off on backing-file\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile",cache.writeback=off -drive "$drv_bk" -drive "$drv_file" -drive "$drv_img" | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" # Check cache.no-flush - echo -e "\n\ncache.no-flush=on on blk" + printf "\n\ncache.no-flush=on on blk\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile" -drive "$drv_bk" -drive "$drv_file" -drive "$drv_img",cache.no-flush=on | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.no-flush=on on file" + printf "\ncache.no-flush=on on file\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile" -drive "$drv_bk" -drive "$drv_file",cache.no-flush=on -drive "$drv_img" | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.no-flush=on on backing" + printf "\ncache.no-flush=on on backing\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile" -drive "$drv_bk",cache.no-flush=on -drive "$drv_file" -drive "$drv_img" | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" - echo -e "\ncache.no-flush=on on backing-file" + printf "\ncache.no-flush=on on backing-file\n" echo "$hmp_cmds" | run_qemu -drive "$drv_bkfile",cache.no-flush=on -drive "$drv_bk" -drive "$drv_file" -drive "$drv_img" | grep -e "Cache" -e "[Cc]annot\|[Cc]ould not\|[Cc]an't" } diff --git a/tests/qemu-iotests/171 b/tests/qemu-iotests/171 index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/171 +++ b/tests/qemu-iotests/171 @@ -XXX,XX +XXX,XX @@ _supported_os Linux # Create JSON with options img_json() { - echo -n 'json:{"driver":"raw", ' - echo -n "\"offset\":\"$img_offset\", " + printf %s 'json:{"driver":"raw", ' + printf %s "\"offset\":\"$img_offset\", " if [ "$img_size" -ne -1 ] ; then - echo -n "\"size\":\"$img_size\", " + printf %s "\"size\":\"$img_size\", " fi - echo -n '"file": {' - echo -n '"driver":"file", ' - echo -n "\"filename\":\"$TEST_IMG\" " - echo -n "} }" + printf %s '"file": {' + printf %s '"driver":"file", ' + printf %s "\"filename\":\"$TEST_IMG\" " + printf %s "} }" } do_general_test() { diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index XXXXXXX..XXXXXXX 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -XXX,XX +XXX,XX @@ _wallclock() _timestamp() { now=`date "+%T"` - echo -n " [$now]" + printf %s " [$now]" } _wrapup() @@ -XXX,XX +XXX,XX @@ seq="check" for seq in $list do err=false - echo -n "$seq" + printf %s "$seq" if [ -n "$TESTS_REMAINING_LOG" ] ; then sed -e "s/$seq//" -e 's/ / /' -e 's/^ *//' $TESTS_REMAINING_LOG > $TESTS_REMAINING_LOG.tmp mv $TESTS_REMAINING_LOG.tmp $TESTS_REMAINING_LOG @@ -XXX,XX +XXX,XX @@ do rm -f $seq.out.bad lasttime=`sed -n -e "/^$seq /s/.* //p" <$TIMESTAMP_FILE` if [ "X$lasttime" != X ]; then - echo -n " ${lasttime}s ..." + printf %s " ${lasttime}s ..." else - echo -n " " # prettier output with timestamps. + printf " " # prettier output with timestamps. fi rm -f core $seq.notrun @@ -XXX,XX +XXX,XX @@ do echo "$seq" > "${TEST_DIR}"/check.sts start=`_wallclock` - $timestamp && echo -n " ["`date "+%T"`"]" + $timestamp && printf %s " [$(date "+%T")]" if [ "$(head -n 1 "$source_iotests/$seq")" == "#!/usr/bin/env python" ]; then run_command="$PYTHON $seq" @@ -XXX,XX +XXX,XX @@ do if [ -f core ] then - echo -n " [dumped core]" + printf " [dumped core]" mv core $seq.core err=true fi if [ -f $seq.notrun ] then - $timestamp || echo -n " [not run] " - $timestamp && echo " [not run]" && echo -n " $seq -- " + $timestamp || printf " [not run] " + $timestamp && echo " [not run]" && printf %s " $seq -- " cat $seq.notrun notrun="$notrun $seq" else if [ $sts -ne 0 ] then - echo -n " [failed, exit status $sts]" + printf %s " [failed, exit status $sts]" err=true fi diff --git a/tests/rocker/all b/tests/rocker/all index XXXXXXX..XXXXXXX 100755 --- a/tests/rocker/all +++ b/tests/rocker/all @@ -XXX,XX +XXX,XX @@ -echo -n "Running port test... " +printf "Running port test... " ./port if [ $? -eq 0 ]; then echo "pass"; else echo "FAILED"; exit 1; fi -echo -n "Running bridge test... " +printf "Running bridge test... " ./bridge if [ $? -eq 0 ]; then echo "pass"; else echo "FAILED"; exit 1; fi -echo -n "Running bridge STP test... " +printf "Running bridge STP test... " ./bridge-stp if [ $? -eq 0 ]; then echo "pass"; else echo "FAILED"; exit 1; fi -echo -n "Running bridge VLAN test... " +printf "Running bridge VLAN test... " ./bridge-vlan if [ $? -eq 0 ]; then echo "pass"; else echo "FAILED"; exit 1; fi -echo -n "Running bridge VLAN STP test... " +printf "Running bridge VLAN STP test... " ./bridge-vlan-stp if [ $? -eq 0 ]; then echo "pass"; else echo "FAILED"; exit 1; fi diff --git a/tests/tcg/cris/Makefile b/tests/tcg/cris/Makefile index XXXXXXX..XXXXXXX 100644 --- a/tests/tcg/cris/Makefile +++ b/tests/tcg/cris/Makefile @@ -XXX,XX +XXX,XX @@ check_addcv17.tst: crtv10.o sysv10.o build: $(CRT) $(SYS) $(TESTCASES) check: $(CRT) $(SYS) $(TESTCASES) - @echo -e "\nQEMU simulator." + @printf "\nQEMU simulator.\n" for case in $(TESTCASES); do \ - echo -n "$$case "; \ + printf %s "$$case "; \ SIMARGS=; \ case $$case in *v17*) SIMARGS="-cpu crisv17";; esac; \ $(SIM) $$SIMARGS ./$$case; \ done check-g: $(CRT) $(SYS) $(TESTCASES) - @echo -e "\nGDB simulator." + @printf "\nGDB simulator.\n" @for case in $(TESTCASES); do \ - echo -n "$$case "; \ + printf %s "$$case "; \ $(SIMG) $$case; \ done -- 1.8.3.1 The following changes since commit f1d33f55c47dfdaf8daacd618588ad3ae4c452d1: Merge tag 'pull-testing-gdbstub-plugins-gitdm-061022-3' of https://github.com/stsquad/qemu into staging (2022-10-06 07:11:56 -0400) are available in the Git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream for you to fetch changes up to a7ca2eb488ff149c898f43abe103f8bd8e3ca3c4: file-posix: Remove unused s->discard_zeroes (2022-10-07 12:11:41 +0200) ---------------------------------------------------------------- Block layer patches - job: replace AioContext lock with job_mutex - Fixes to make coroutine_fn annotations more accurate - QAPI schema: Fix incorrect example - Code cleanup ---------------------------------------------------------------- Alberto Faria (1): coroutine: Drop coroutine_fn annotation from qemu_coroutine_self() Emanuele Giuseppe Esposito (20): job.c: make job_mutex and job_lock/unlock() public job.h: categorize fields in struct Job job.c: API functions not used outside should be static aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED job.c: add job_lock/unlock while keeping job.h intact job: move and update comments from blockjob.c blockjob: introduce block_job _locked() APIs jobs: add job lock in find_* functions jobs: use job locks also in the unit tests block/mirror.c: use of job helpers in drivers jobs: group together API calls under the same job lock jobs: protect job.aio_context with BQL and job_mutex blockjob.h: categorize fields in struct BlockJob blockjob: rename notifier callbacks as _locked blockjob: protect iostatus field in BlockJob struct job.h: categorize JobDriver callbacks that need the AioContext lock job.c: enable job lock/unlock and remove Aiocontext locks block_job_query: remove atomic read blockjob: remove unused functions job: remove unused functions Kevin Wolf (2): quorum: Remove unnecessary forward declaration file-posix: Remove unused s->discard_zeroes Marc-André Lureau (3): 9p: add missing coroutine_fn annotations migration: add missing coroutine_fn annotations test-coroutine: add missing coroutine_fn annotations Markus Armbruster (1): Revert "qapi: fix examples of blockdev-add with qcow2" Paolo Bonzini (23): block/nvme: separate nvme_get_free_req cases for coroutine/non-coroutine context block: add missing coroutine_fn annotations qcow2: remove incorrect coroutine_fn annotations nbd: remove incorrect coroutine_fn annotations coroutine: remove incorrect coroutine_fn annotations blkverify: add missing coroutine_fn annotations file-posix: add missing coroutine_fn annotations iscsi: add missing coroutine_fn annotations nbd: add missing coroutine_fn annotations nfs: add missing coroutine_fn annotations nvme: add missing coroutine_fn annotations parallels: add missing coroutine_fn annotations qcow2: add missing coroutine_fn annotations copy-before-write: add missing coroutine_fn annotations curl: add missing coroutine_fn annotations qed: add missing coroutine_fn annotations quorum: add missing coroutine_fn annotations throttle: add missing coroutine_fn annotations vmdk: add missing coroutine_fn annotations job: add missing coroutine_fn annotations coroutine-lock: add missing coroutine_fn annotations raw-format: add missing coroutine_fn annotations job: detect change of aiocontext within job coroutine qapi/block-core.json | 10 +- block/qcow2.h | 19 +- hw/9pfs/9p.h | 9 +- include/block/aio-wait.h | 17 +- include/block/blockjob.h | 59 +++- include/block/nbd.h | 2 +- include/qemu/coroutine.h | 4 +- include/qemu/job.h | 306 +++++++++++++----- block.c | 24 +- block/blkverify.c | 2 +- block/block-backend.c | 10 +- block/copy-before-write.c | 9 +- block/curl.c | 2 +- block/file-posix.c | 11 +- block/io.c | 22 +- block/iscsi.c | 3 +- block/mirror.c | 19 +- block/nbd.c | 11 +- block/nfs.c | 2 +- block/nvme.c | 54 ++-- block/parallels.c | 5 +- block/qcow2-cluster.c | 21 +- block/qcow2-refcount.c | 6 +- block/qcow2.c | 5 +- block/qed.c | 4 +- block/quorum.c | 38 +-- block/raw-format.c | 3 +- block/replication.c | 3 + block/throttle.c | 2 +- block/vmdk.c | 22 +- blockdev.c | 129 ++++---- blockjob.c | 132 ++++---- job-qmp.c | 92 +++--- job.c | 674 +++++++++++++++++++++++++-------------- migration/migration.c | 3 +- monitor/qmp-cmds.c | 7 +- qemu-img.c | 17 +- tests/unit/test-bdrv-drain.c | 80 +++-- tests/unit/test-block-iothread.c | 8 +- tests/unit/test-blockjob-txn.c | 24 +- tests/unit/test-blockjob.c | 136 ++++---- tests/unit/test-coroutine.c | 2 +- util/qemu-coroutine-lock.c | 14 +- util/qemu-coroutine.c | 2 +- 44 files changed, 1237 insertions(+), 787 deletions(-) From: Markus Armbruster <armbru@redhat.com> This reverts commit b6522938327141235b97ab38e40c6c4512587373. Kevin Wolf NAKed this patch, because: 'file' is a required member (defined in BlockdevOptionsGenericFormat), removing it makes the example invalid. 'data-file' is only an additional optional member to be used for external data files (i.e. when the guest data is kept separate from the metadata in the .qcow2 file). However, it had already been merged then. Revert. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20220930171908.846769-1-armbru@redhat.com> Reviewed-by: Victor Toso <victortoso@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- qapi/block-core.json | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index XXXXXXX..XXXXXXX 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -XXX,XX +XXX,XX @@ # -> { "execute": "blockdev-add", # "arguments": { "driver": "qcow2", # "node-name": "node1534", -# "data-file": { "driver": "file", -# "filename": "hd1.qcow2" }, +# "file": { "driver": "file", +# "filename": "hd1.qcow2" }, # "backing": null } } # # <- { "return": {} } @@ -XXX,XX +XXX,XX @@ # "arguments": { # "driver": "qcow2", # "node-name": "test1", -# "data-file": { +# "file": { # "driver": "file", # "filename": "test.qcow2" # } @@ -XXX,XX +XXX,XX @@ # "cache": { # "direct": true # }, -# "data-file": { +# "file": { # "driver": "file", # "filename": "/tmp/test.qcow2" # }, @@ -XXX,XX +XXX,XX @@ # "arguments": { # "driver": "qcow2", # "node-name": "node0", -# "data-file": { +# "file": { # "driver": "file", # "filename": "test.qcow2" # } -- 2.37.3 From: Alberto Faria <afaria@redhat.com> qemu_coroutine_self() can be called from outside coroutine context, returning the leader coroutine, and several such invocations currently exist (mostly in qcow2 tracing calls). Signed-off-by: Alberto Faria <afaria@redhat.com> Message-Id: <20221005175209.975797-1-afaria@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/coroutine.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/coroutine.h +++ b/include/qemu/coroutine.h @@ -XXX,XX +XXX,XX @@ AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co); /** * Get the currently executing coroutine */ -Coroutine *coroutine_fn qemu_coroutine_self(void); +Coroutine *qemu_coroutine_self(void); /** * Return whether or not currently inside a coroutine -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> nvme_get_free_req has very difference semantics when called in coroutine context (where it waits) and in non-coroutine context (where it doesn't). Split the two cases to make it clear what is being requested. Cc: qemu-block@nongnu.org Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-2-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/nvme.c | 48 ++++++++++++++++++++++++++++-------------------- 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index XXXXXXX..XXXXXXX 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -XXX,XX +XXX,XX @@ static void nvme_kick(NVMeQueuePair *q) q->need_kick = 0; } -/* Find a free request element if any, otherwise: - * a) if in coroutine context, try to wait for one to become available; - * b) if not in coroutine, return NULL; - */ -static NVMeRequest *nvme_get_free_req(NVMeQueuePair *q) +static NVMeRequest *nvme_get_free_req_nofail_locked(NVMeQueuePair *q) { NVMeRequest *req; - qemu_mutex_lock(&q->lock); - - while (q->free_req_head == -1) { - if (qemu_in_coroutine()) { - trace_nvme_free_req_queue_wait(q->s, q->index); - qemu_co_queue_wait(&q->free_req_queue, &q->lock); - } else { - qemu_mutex_unlock(&q->lock); - return NULL; - } - } - req = &q->reqs[q->free_req_head]; q->free_req_head = req->free_req_next; req->free_req_next = -1; - - qemu_mutex_unlock(&q->lock); return req; } +/* Return a free request element if any, otherwise return NULL. */ +static NVMeRequest *nvme_get_free_req_nowait(NVMeQueuePair *q) +{ + QEMU_LOCK_GUARD(&q->lock); + if (q->free_req_head == -1) { + return NULL; + } + return nvme_get_free_req_nofail_locked(q); +} + +/* + * Wait for a free request to become available if necessary, then + * return it. + */ +static coroutine_fn NVMeRequest *nvme_get_free_req(NVMeQueuePair *q) +{ + QEMU_LOCK_GUARD(&q->lock); + + while (q->free_req_head == -1) { + trace_nvme_free_req_queue_wait(q->s, q->index); + qemu_co_queue_wait(&q->free_req_queue, &q->lock); + } + + return nvme_get_free_req_nofail_locked(q); +} + /* With q->lock */ static void nvme_put_free_req_locked(NVMeQueuePair *q, NVMeRequest *req) { @@ -XXX,XX +XXX,XX @@ static int nvme_admin_cmd_sync(BlockDriverState *bs, NvmeCmd *cmd) AioContext *aio_context = bdrv_get_aio_context(bs); NVMeRequest *req; int ret = -EINPROGRESS; - req = nvme_get_free_req(q); + req = nvme_get_free_req_nowait(q); if (!req) { return -EBUSY; } -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-3-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block.c | 7 ++++--- block/block-backend.c | 10 +++++----- block/io.c | 22 +++++++++++----------- 3 files changed, 20 insertions(+), 19 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static int64_t create_file_fallback_truncate(BlockBackend *blk, * Helper function for bdrv_create_file_fallback(): Zero the first * sector to remove any potentially pre-existing image header. */ -static int create_file_fallback_zero_first_sector(BlockBackend *blk, - int64_t current_size, - Error **errp) +static int coroutine_fn +create_file_fallback_zero_first_sector(BlockBackend *blk, + int64_t current_size, + Error **errp) { int64_t bytes_to_clear; int ret; diff --git a/block/block-backend.c b/block/block-backend.c index XXXXXXX..XXXXXXX 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -XXX,XX +XXX,XX @@ static BlockAIOCB *blk_aio_prwv(BlockBackend *blk, int64_t offset, return &acb->common; } -static void blk_aio_read_entry(void *opaque) +static void coroutine_fn blk_aio_read_entry(void *opaque) { BlkAioEmAIOCB *acb = opaque; BlkRwCo *rwco = &acb->rwco; @@ -XXX,XX +XXX,XX @@ static void blk_aio_read_entry(void *opaque) blk_aio_complete(acb); } -static void blk_aio_write_entry(void *opaque) +static void coroutine_fn blk_aio_write_entry(void *opaque) { BlkAioEmAIOCB *acb = opaque; BlkRwCo *rwco = &acb->rwco; @@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_ioctl(BlockBackend *blk, unsigned long int req, return ret; } -static void blk_aio_ioctl_entry(void *opaque) +static void coroutine_fn blk_aio_ioctl_entry(void *opaque) { BlkAioEmAIOCB *acb = opaque; BlkRwCo *rwco = &acb->rwco; @@ -XXX,XX +XXX,XX @@ blk_co_do_pdiscard(BlockBackend *blk, int64_t offset, int64_t bytes) return bdrv_co_pdiscard(blk->root, offset, bytes); } -static void blk_aio_pdiscard_entry(void *opaque) +static void coroutine_fn blk_aio_pdiscard_entry(void *opaque) { BlkAioEmAIOCB *acb = opaque; BlkRwCo *rwco = &acb->rwco; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn blk_co_do_flush(BlockBackend *blk) return bdrv_co_flush(blk_bs(blk)); } -static void blk_aio_flush_entry(void *opaque) +static void coroutine_fn blk_aio_flush_entry(void *opaque) { BlkAioEmAIOCB *acb = opaque; BlkRwCo *rwco = &acb->rwco; diff --git a/block/io.c b/block/io.c index XXXXXXX..XXXXXXX 100644 --- a/block/io.c +++ b/block/io.c @@ -XXX,XX +XXX,XX @@ static void coroutine_fn tracked_request_end(BdrvTrackedRequest *req) /** * Add an active request to the tracked requests list */ -static void tracked_request_begin(BdrvTrackedRequest *req, - BlockDriverState *bs, - int64_t offset, - int64_t bytes, - enum BdrvTrackedRequestType type) +static void coroutine_fn tracked_request_begin(BdrvTrackedRequest *req, + BlockDriverState *bs, + int64_t offset, + int64_t bytes, + enum BdrvTrackedRequestType type) { bdrv_check_request(offset, bytes, &error_abort); @@ -XXX,XX +XXX,XX @@ static bool tracked_request_overlaps(BdrvTrackedRequest *req, } /* Called with self->bs->reqs_lock held */ -static BdrvTrackedRequest * +static coroutine_fn BdrvTrackedRequest * bdrv_find_conflicting_request(BdrvTrackedRequest *self) { BdrvTrackedRequest *req; @@ -XXX,XX +XXX,XX @@ static bool bdrv_init_padding(BlockDriverState *bs, return true; } -static int bdrv_padding_rmw_read(BdrvChild *child, - BdrvTrackedRequest *req, - BdrvRequestPadding *pad, - bool zero_middle) +static coroutine_fn int bdrv_padding_rmw_read(BdrvChild *child, + BdrvTrackedRequest *req, + BdrvRequestPadding *pad, + bool zero_middle) { QEMUIOVector local_qiov; BlockDriverState *bs = child->bs; @@ -XXX,XX +XXX,XX @@ out: return ret; } -int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf) +int coroutine_fn bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf) { BlockDriver *drv = bs->drv; CoroutineIOCompletion co = { -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> This is incorrect because qcow2_mark_clean() calls qcow2_flush_caches(). qcow2_mark_clean() is called from non-coroutine context in qcow2_inactivate() and qcow2_amend_options(). Reviewed-by: Alberto Faria <afaria@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-4-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.h | 4 ++-- block/qcow2-refcount.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ void qcow2_free_any_cluster(BlockDriverState *bs, uint64_t l2_entry, int qcow2_update_snapshot_refcount(BlockDriverState *bs, int64_t l1_table_offset, int l1_size, int addend); -int coroutine_fn qcow2_flush_caches(BlockDriverState *bs); -int coroutine_fn qcow2_write_caches(BlockDriverState *bs); +int qcow2_flush_caches(BlockDriverState *bs); +int qcow2_write_caches(BlockDriverState *bs); int qcow2_check_refcounts(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix); diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -XXX,XX +XXX,XX @@ void qcow2_free_any_cluster(BlockDriverState *bs, uint64_t l2_entry, } } -int coroutine_fn qcow2_write_caches(BlockDriverState *bs) +int qcow2_write_caches(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; int ret; @@ -XXX,XX +XXX,XX @@ int coroutine_fn qcow2_write_caches(BlockDriverState *bs) return 0; } -int coroutine_fn qcow2_flush_caches(BlockDriverState *bs) +int qcow2_flush_caches(BlockDriverState *bs) { int ret = qcow2_write_caches(bs); if (ret < 0) { -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> nbd_co_establish_connection_cancel() cancels a coroutine but is not called from coroutine context itself, for example in nbd_cancel_in_flight() and in timer callbacks reconnect_delay_timer_cb() and open_timer_cb(). Reviewed-by: Alberto Faria <afaria@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-5-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/nbd.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/block/nbd.h b/include/block/nbd.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/nbd.h +++ b/include/block/nbd.h @@ -XXX,XX +XXX,XX @@ QIOChannel *coroutine_fn nbd_co_establish_connection(NBDClientConnection *conn, NBDExportInfo *info, bool blocking, Error **errp); -void coroutine_fn nbd_co_establish_connection_cancel(NBDClientConnection *conn); +void nbd_co_establish_connection_cancel(NBDClientConnection *conn); #endif -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> qemu_coroutine_get_aio_context inspects a coroutine, but it does not have to be called from the coroutine itself (or from any coroutine). Reviewed-by: Alberto Faria <afaria@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-6-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/coroutine.h | 2 +- util/qemu-coroutine.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/coroutine.h +++ b/include/qemu/coroutine.h @@ -XXX,XX +XXX,XX @@ void coroutine_fn qemu_coroutine_yield(void); /** * Get the AioContext of the given coroutine */ -AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co); +AioContext *qemu_coroutine_get_aio_context(Coroutine *co); /** * Get the currently executing coroutine diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c index XXXXXXX..XXXXXXX 100644 --- a/util/qemu-coroutine.c +++ b/util/qemu-coroutine.c @@ -XXX,XX +XXX,XX @@ bool qemu_coroutine_entered(Coroutine *co) return co->caller; } -AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co) +AioContext *qemu_coroutine_get_aio_context(Coroutine *co) { return co->ctx; } -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-8-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/blkverify.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blkverify.c b/block/blkverify.c index XXXXXXX..XXXXXXX 100644 --- a/block/blkverify.c +++ b/block/blkverify.c @@ -XXX,XX +XXX,XX @@ blkverify_co_pwritev(BlockDriverState *bs, int64_t offset, int64_t bytes, return blkverify_co_prwv(bs, &r, offset, bytes, qiov, qiov, flags, true); } -static int blkverify_co_flush(BlockDriverState *bs) +static int coroutine_fn blkverify_co_flush(BlockDriverState *bs) { BDRVBlkverifyState *s = bs->opaque; -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-9-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/file-posix.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/file-posix.c b/block/file-posix.c index XXXXXXX..XXXXXXX 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -XXX,XX +XXX,XX @@ static void raw_aio_unplug(BlockDriverState *bs) #endif } -static int raw_co_flush_to_disk(BlockDriverState *bs) +static int coroutine_fn raw_co_flush_to_disk(BlockDriverState *bs) { BDRVRawState *s = bs->opaque; RawPosixAIOData acb; -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-10-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/iscsi.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/iscsi.c b/block/iscsi.c index XXXXXXX..XXXXXXX 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -XXX,XX +XXX,XX @@ iscsi_co_generic_cb(struct iscsi_context *iscsi, int status, } } -static void iscsi_co_init_iscsitask(IscsiLun *iscsilun, struct IscsiTask *iTask) +static void coroutine_fn +iscsi_co_init_iscsitask(IscsiLun *iscsilun, struct IscsiTask *iTask) { *iTask = (struct IscsiTask) { .co = qemu_coroutine_self(), -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-11-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/nbd.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index XXXXXXX..XXXXXXX 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -XXX,XX +XXX,XX @@ static void nbd_iter_request_error(NBDReplyChunkIter *iter, int ret) * nbd_reply_chunk_iter_receive * The pointer stored in @payload requires g_free() to free it. */ -static bool nbd_reply_chunk_iter_receive(BDRVNBDState *s, - NBDReplyChunkIter *iter, - uint64_t handle, - QEMUIOVector *qiov, NBDReply *reply, - void **payload) +static bool coroutine_fn nbd_reply_chunk_iter_receive(BDRVNBDState *s, + NBDReplyChunkIter *iter, + uint64_t handle, + QEMUIOVector *qiov, + NBDReply *reply, + void **payload) { int ret, request_ret; NBDReply local_reply; -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-12-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/nfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/nfs.c b/block/nfs.c index XXXXXXX..XXXXXXX 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -XXX,XX +XXX,XX @@ static void nfs_process_write(void *arg) qemu_mutex_unlock(&client->mutex); } -static void nfs_co_init_task(BlockDriverState *bs, NFSRPC *task) +static void coroutine_fn nfs_co_init_task(BlockDriverState *bs, NFSRPC *task) { *task = (NFSRPC) { .co = qemu_coroutine_self(), -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-13-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/nvme.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index XXXXXXX..XXXXXXX 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -XXX,XX +XXX,XX @@ static inline bool nvme_qiov_aligned(BlockDriverState *bs, return true; } -static int nvme_co_prw(BlockDriverState *bs, uint64_t offset, uint64_t bytes, - QEMUIOVector *qiov, bool is_write, int flags) +static coroutine_fn int nvme_co_prw(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, bool is_write, + int flags) { BDRVNVMeState *s = bs->opaque; int r; -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-14-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/parallels.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index XXXXXXX..XXXXXXX 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -XXX,XX +XXX,XX @@ static int64_t block_status(BDRVParallelsState *s, int64_t sector_num, return start_off; } -static int64_t allocate_clusters(BlockDriverState *bs, int64_t sector_num, - int nb_sectors, int *pnum) +static coroutine_fn int64_t allocate_clusters(BlockDriverState *bs, + int64_t sector_num, + int nb_sectors, int *pnum) { int ret = 0; BDRVParallelsState *s = bs->opaque; -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-15-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.h | 15 ++++++++------- block/qcow2-cluster.c | 21 ++++++++++++--------- block/qcow2-refcount.c | 2 +- block/qcow2.c | 5 +++-- 4 files changed, 24 insertions(+), 19 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -XXX,XX +XXX,XX @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order, void *cb_opaque, Error **errp); int qcow2_shrink_reftable(BlockDriverState *bs); int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size); -int qcow2_detect_metadata_preallocation(BlockDriverState *bs); +int coroutine_fn qcow2_detect_metadata_preallocation(BlockDriverState *bs); /* qcow2-cluster.c functions */ int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size, @@ -XXX,XX +XXX,XX @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, QCow2SubclusterType *subcluster_type); -int qcow2_alloc_host_offset(BlockDriverState *bs, uint64_t offset, - unsigned int *bytes, uint64_t *host_offset, - QCowL2Meta **m); +int coroutine_fn qcow2_alloc_host_offset(BlockDriverState *bs, uint64_t offset, + unsigned int *bytes, + uint64_t *host_offset, QCowL2Meta **m); int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, uint64_t offset, int compressed_size, @@ -XXX,XX +XXX,XX @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, void qcow2_parse_compressed_l2_entry(BlockDriverState *bs, uint64_t l2_entry, uint64_t *coffset, int *csize); -int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m); +int coroutine_fn qcow2_alloc_cluster_link_l2(BlockDriverState *bs, + QCowL2Meta *m); void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m); int qcow2_cluster_discard(BlockDriverState *bs, uint64_t offset, uint64_t bytes, enum qcow2_discard_type type, bool full_discard); -int qcow2_subcluster_zeroize(BlockDriverState *bs, uint64_t offset, - uint64_t bytes, int flags); +int coroutine_fn qcow2_subcluster_zeroize(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, int flags); int qcow2_expand_zero_clusters(BlockDriverState *bs, BlockDriverAmendStatusCB *status_cb, diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -XXX,XX +XXX,XX @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, return 0; } -static int perform_cow(BlockDriverState *bs, QCowL2Meta *m) +static int coroutine_fn perform_cow(BlockDriverState *bs, QCowL2Meta *m) { BDRVQcow2State *s = bs->opaque; Qcow2COWRegion *start = &m->cow_start; @@ -XXX,XX +XXX,XX @@ fail: return ret; } -int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m) +int coroutine_fn qcow2_alloc_cluster_link_l2(BlockDriverState *bs, + QCowL2Meta *m) { BDRVQcow2State *s = bs->opaque; int i, j = 0, l2_index, ret; @@ -XXX,XX +XXX,XX @@ static int count_single_write_clusters(BlockDriverState *bs, int nb_clusters, * information on cluster allocation may be invalid now. The caller * must start over anyway, so consider *cur_bytes undefined. */ -static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset, - uint64_t *cur_bytes, QCowL2Meta **m) +static int coroutine_fn handle_dependencies(BlockDriverState *bs, + uint64_t guest_offset, + uint64_t *cur_bytes, QCowL2Meta **m) { BDRVQcow2State *s = bs->opaque; QCowL2Meta *old_alloc; @@ -XXX,XX +XXX,XX @@ out: * * Return 0 on success and -errno in error cases */ -int qcow2_alloc_host_offset(BlockDriverState *bs, uint64_t offset, - unsigned int *bytes, uint64_t *host_offset, - QCowL2Meta **m) +int coroutine_fn qcow2_alloc_host_offset(BlockDriverState *bs, uint64_t offset, + unsigned int *bytes, + uint64_t *host_offset, + QCowL2Meta **m) { BDRVQcow2State *s = bs->opaque; uint64_t start, remaining; @@ -XXX,XX +XXX,XX @@ out: return ret; } -int qcow2_subcluster_zeroize(BlockDriverState *bs, uint64_t offset, - uint64_t bytes, int flags) +int coroutine_fn qcow2_subcluster_zeroize(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, int flags) { BDRVQcow2State *s = bs->opaque; uint64_t end_offset = offset + bytes; diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -XXX,XX +XXX,XX @@ int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size) return -EIO; } -int qcow2_detect_metadata_preallocation(BlockDriverState *bs) +int coroutine_fn qcow2_detect_metadata_preallocation(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; int64_t i, end_cluster, cluster_count = 0, threshold; diff --git a/block/qcow2.c b/block/qcow2.c index XXXXXXX..XXXXXXX 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -XXX,XX +XXX,XX @@ static bool merge_cow(uint64_t offset, unsigned bytes, * Return 1 if the COW regions read as zeroes, 0 if not, < 0 on error. * Note that returning 0 does not guarantee non-zero data. */ -static int is_zero_cow(BlockDriverState *bs, QCowL2Meta *m) +static int coroutine_fn is_zero_cow(BlockDriverState *bs, QCowL2Meta *m) { /* * This check is designed for optimization shortcut so it must be @@ -XXX,XX +XXX,XX @@ static int is_zero_cow(BlockDriverState *bs, QCowL2Meta *m) m->cow_end.nb_bytes); } -static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) +static int coroutine_fn handle_alloc_space(BlockDriverState *bs, + QCowL2Meta *l2meta) { BDRVQcow2State *s = bs->opaque; QCowL2Meta *m; -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-16-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/copy-before-write.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index XXXXXXX..XXXXXXX 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn cbw_co_flush(BlockDriverState *bs) * It's guaranteed that guest writes will not interact in the region until * cbw_snapshot_read_unlock() called. */ -static BlockReq *cbw_snapshot_read_lock(BlockDriverState *bs, - int64_t offset, int64_t bytes, - int64_t *pnum, BdrvChild **file) +static coroutine_fn BlockReq * +cbw_snapshot_read_lock(BlockDriverState *bs, int64_t offset, int64_t bytes, + int64_t *pnum, BdrvChild **file) { BDRVCopyBeforeWriteState *s = bs->opaque; BlockReq *req = g_new(BlockReq, 1); @@ -XXX,XX +XXX,XX @@ static BlockReq *cbw_snapshot_read_lock(BlockDriverState *bs, return req; } -static void cbw_snapshot_read_unlock(BlockDriverState *bs, BlockReq *req) +static coroutine_fn void +cbw_snapshot_read_unlock(BlockDriverState *bs, BlockReq *req) { BDRVCopyBeforeWriteState *s = bs->opaque; -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-17-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/curl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/curl.c b/block/curl.c index XXXXXXX..XXXXXXX 100644 --- a/block/curl.c +++ b/block/curl.c @@ -XXX,XX +XXX,XX @@ out_noclean: return -EINVAL; } -static void curl_setup_preadv(BlockDriverState *bs, CURLAIOCB *acb) +static void coroutine_fn curl_setup_preadv(BlockDriverState *bs, CURLAIOCB *acb) { CURLState *state; int running; -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-18-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qed.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qed.c b/block/qed.c index XXXXXXX..XXXXXXX 100644 --- a/block/qed.c +++ b/block/qed.c @@ -XXX,XX +XXX,XX @@ static CachedL2Table *qed_new_l2_table(BDRVQEDState *s) return l2_table; } -static bool qed_plug_allocating_write_reqs(BDRVQEDState *s) +static bool coroutine_fn qed_plug_allocating_write_reqs(BDRVQEDState *s) { qemu_co_mutex_lock(&s->table_lock); @@ -XXX,XX +XXX,XX @@ static bool qed_plug_allocating_write_reqs(BDRVQEDState *s) return true; } -static void qed_unplug_allocating_write_reqs(BDRVQEDState *s) +static void coroutine_fn qed_unplug_allocating_write_reqs(BDRVQEDState *s) { qemu_co_mutex_lock(&s->table_lock); assert(s->allocating_write_reqs_plugged); -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-19-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/quorum.c | 36 +++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/block/quorum.c b/block/quorum.c index XXXXXXX..XXXXXXX 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -XXX,XX +XXX,XX @@ static bool quorum_64bits_compare(QuorumVoteValue *a, QuorumVoteValue *b) return a->l == b->l; } -static QuorumAIOCB *quorum_aio_get(BlockDriverState *bs, - QEMUIOVector *qiov, - uint64_t offset, - uint64_t bytes, - int flags) +static QuorumAIOCB *coroutine_fn quorum_aio_get(BlockDriverState *bs, + QEMUIOVector *qiov, + uint64_t offset, uint64_t bytes, + int flags) { BDRVQuorumState *s = bs->opaque; QuorumAIOCB *acb = g_new(QuorumAIOCB, 1); @@ -XXX,XX +XXX,XX @@ static void quorum_report_bad_versions(BDRVQuorumState *s, } } -static void quorum_rewrite_entry(void *opaque) +static void coroutine_fn quorum_rewrite_entry(void *opaque) { QuorumCo *co = opaque; QuorumAIOCB *acb = co->acb; @@ -XXX,XX +XXX,XX @@ free_exit: quorum_free_vote_list(&acb->votes); } -static void read_quorum_children_entry(void *opaque) +static void coroutine_fn read_quorum_children_entry(void *opaque) { QuorumCo *co = opaque; QuorumAIOCB *acb = co->acb; @@ -XXX,XX +XXX,XX @@ static void read_quorum_children_entry(void *opaque) } } -static int read_quorum_children(QuorumAIOCB *acb) +static int coroutine_fn read_quorum_children(QuorumAIOCB *acb) { BDRVQuorumState *s = acb->bs->opaque; int i; @@ -XXX,XX +XXX,XX @@ static int read_quorum_children(QuorumAIOCB *acb) return acb->vote_ret; } -static int read_fifo_child(QuorumAIOCB *acb) +static int coroutine_fn read_fifo_child(QuorumAIOCB *acb) { BDRVQuorumState *s = acb->bs->opaque; int n, ret; @@ -XXX,XX +XXX,XX @@ static int read_fifo_child(QuorumAIOCB *acb) return ret; } -static int quorum_co_preadv(BlockDriverState *bs, int64_t offset, int64_t bytes, - QEMUIOVector *qiov, BdrvRequestFlags flags) +static int coroutine_fn quorum_co_preadv(BlockDriverState *bs, + int64_t offset, int64_t bytes, + QEMUIOVector *qiov, + BdrvRequestFlags flags) { BDRVQuorumState *s = bs->opaque; QuorumAIOCB *acb = quorum_aio_get(bs, qiov, offset, bytes, flags); @@ -XXX,XX +XXX,XX @@ static int quorum_co_preadv(BlockDriverState *bs, int64_t offset, int64_t bytes, return ret; } -static void write_quorum_entry(void *opaque) +static void coroutine_fn write_quorum_entry(void *opaque) { QuorumCo *co = opaque; QuorumAIOCB *acb = co->acb; @@ -XXX,XX +XXX,XX @@ static void write_quorum_entry(void *opaque) } } -static int quorum_co_pwritev(BlockDriverState *bs, int64_t offset, - int64_t bytes, QEMUIOVector *qiov, - BdrvRequestFlags flags) +static int coroutine_fn quorum_co_pwritev(BlockDriverState *bs, int64_t offset, + int64_t bytes, QEMUIOVector *qiov, + BdrvRequestFlags flags) { BDRVQuorumState *s = bs->opaque; QuorumAIOCB *acb = quorum_aio_get(bs, qiov, offset, bytes, flags); @@ -XXX,XX +XXX,XX @@ static int quorum_co_pwritev(BlockDriverState *bs, int64_t offset, return ret; } -static int quorum_co_pwrite_zeroes(BlockDriverState *bs, int64_t offset, - int64_t bytes, BdrvRequestFlags flags) +static int coroutine_fn quorum_co_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, int64_t bytes, + BdrvRequestFlags flags) { return quorum_co_pwritev(bs, offset, bytes, NULL, -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-20-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/throttle.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/throttle.c b/block/throttle.c index XXXXXXX..XXXXXXX 100644 --- a/block/throttle.c +++ b/block/throttle.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn throttle_co_pwritev_compressed(BlockDriverState *bs, BDRV_REQ_WRITE_COMPRESSED); } -static int throttle_co_flush(BlockDriverState *bs) +static int coroutine_fn throttle_co_flush(BlockDriverState *bs) { return bdrv_co_flush(bs->file->bs); } -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-21-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/vmdk.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index XXXXXXX..XXXXXXX 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn vmdk_co_block_status(BlockDriverState *bs, return ret; } -static int vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset, - int64_t offset_in_cluster, QEMUIOVector *qiov, - uint64_t qiov_offset, uint64_t n_bytes, - uint64_t offset) +static int coroutine_fn +vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset, + int64_t offset_in_cluster, QEMUIOVector *qiov, + uint64_t qiov_offset, uint64_t n_bytes, + uint64_t offset) { int ret; VmdkGrainMarker *data = NULL; @@ -XXX,XX +XXX,XX @@ static int vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset, return ret; } -static int vmdk_read_extent(VmdkExtent *extent, int64_t cluster_offset, - int64_t offset_in_cluster, QEMUIOVector *qiov, - int bytes) +static int coroutine_fn +vmdk_read_extent(VmdkExtent *extent, int64_t cluster_offset, + int64_t offset_in_cluster, QEMUIOVector *qiov, + int bytes) { int ret; int cluster_bytes, buf_bytes; @@ -XXX,XX +XXX,XX @@ fail: * * Returns: error code with 0 for success. */ -static int vmdk_pwritev(BlockDriverState *bs, uint64_t offset, - uint64_t bytes, QEMUIOVector *qiov, - bool zeroed, bool zero_dry_run) +static int coroutine_fn vmdk_pwritev(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, QEMUIOVector *qiov, + bool zeroed, bool zero_dry_run) { BDRVVmdkState *s = bs->opaque; VmdkExtent *extent = NULL; -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-22-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 2 +- job.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -XXX,XX +XXX,XX @@ void coroutine_fn job_pause_point(Job *job); * * Yield the job coroutine. */ -void job_yield(Job *job); +void coroutine_fn job_yield(Job *job); /** * @job: The job that calls the function. diff --git a/job.c b/job.c index XXXXXXX..XXXXXXX 100644 --- a/job.c +++ b/job.c @@ -XXX,XX +XXX,XX @@ void coroutine_fn job_pause_point(Job *job) } } -void job_yield(Job *job) +void coroutine_fn job_yield(Job *job) { assert(job->busy); -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-23-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- util/qemu-coroutine-lock.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c index XXXXXXX..XXXXXXX 100644 --- a/util/qemu-coroutine-lock.c +++ b/util/qemu-coroutine-lock.c @@ -XXX,XX +XXX,XX @@ typedef struct CoWaitRecord { QSLIST_ENTRY(CoWaitRecord) next; } CoWaitRecord; -static void push_waiter(CoMutex *mutex, CoWaitRecord *w) +static void coroutine_fn push_waiter(CoMutex *mutex, CoWaitRecord *w) { w->co = qemu_coroutine_self(); QSLIST_INSERT_HEAD_ATOMIC(&mutex->from_push, w, next); @@ -XXX,XX +XXX,XX @@ void qemu_co_rwlock_init(CoRwlock *lock) } /* Releases the internal CoMutex. */ -static void qemu_co_rwlock_maybe_wake_one(CoRwlock *lock) +static void coroutine_fn qemu_co_rwlock_maybe_wake_one(CoRwlock *lock) { CoRwTicket *tkt = QSIMPLEQ_FIRST(&lock->tickets); Coroutine *co = NULL; @@ -XXX,XX +XXX,XX @@ static void qemu_co_rwlock_maybe_wake_one(CoRwlock *lock) } } -void qemu_co_rwlock_rdlock(CoRwlock *lock) +void coroutine_fn qemu_co_rwlock_rdlock(CoRwlock *lock) { Coroutine *self = qemu_coroutine_self(); @@ -XXX,XX +XXX,XX @@ void qemu_co_rwlock_rdlock(CoRwlock *lock) self->locks_held++; } -void qemu_co_rwlock_unlock(CoRwlock *lock) +void coroutine_fn qemu_co_rwlock_unlock(CoRwlock *lock) { Coroutine *self = qemu_coroutine_self(); @@ -XXX,XX +XXX,XX @@ void qemu_co_rwlock_unlock(CoRwlock *lock) qemu_co_rwlock_maybe_wake_one(lock); } -void qemu_co_rwlock_downgrade(CoRwlock *lock) +void coroutine_fn qemu_co_rwlock_downgrade(CoRwlock *lock) { qemu_co_mutex_lock(&lock->mutex); assert(lock->owners == -1); @@ -XXX,XX +XXX,XX @@ void qemu_co_rwlock_downgrade(CoRwlock *lock) qemu_co_rwlock_maybe_wake_one(lock); } -void qemu_co_rwlock_wrlock(CoRwlock *lock) +void coroutine_fn qemu_co_rwlock_wrlock(CoRwlock *lock) { Coroutine *self = qemu_coroutine_self(); @@ -XXX,XX +XXX,XX @@ void qemu_co_rwlock_wrlock(CoRwlock *lock) self->locks_held++; } -void qemu_co_rwlock_upgrade(CoRwlock *lock) +void coroutine_fn qemu_co_rwlock_upgrade(CoRwlock *lock) { qemu_co_mutex_lock(&lock->mutex); assert(lock->owners > 0); -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-24-pbonzini@redhat.com> [kwolf: Fixed up coding style] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/raw-format.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/raw-format.c b/block/raw-format.c index XXXXXXX..XXXXXXX 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -XXX,XX +XXX,XX @@ static void raw_lock_medium(BlockDriverState *bs, bool locked) bdrv_lock_medium(bs->file->bs, locked); } -static int raw_co_ioctl(BlockDriverState *bs, unsigned long int req, void *buf) +static int coroutine_fn raw_co_ioctl(BlockDriverState *bs, + unsigned long int req, void *buf) { BDRVRawState *s = bs->opaque; if (s->offset || s->has_size) { -- 2.37.3 From: Marc-André Lureau <marcandre.lureau@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Acked-by: Greg Kurz <groug@kaod.org> Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-25-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- hw/9pfs/9p.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/hw/9pfs/9p.h b/hw/9pfs/9p.h index XXXXXXX..XXXXXXX 100644 --- a/hw/9pfs/9p.h +++ b/hw/9pfs/9p.h @@ -XXX,XX +XXX,XX @@ typedef struct V9fsGetlock extern int open_fd_hw; extern int total_open_fd; -static inline void v9fs_path_write_lock(V9fsState *s) +static inline void coroutine_fn +v9fs_path_write_lock(V9fsState *s) { if (s->ctx.export_flags & V9FS_PATHNAME_FSCONTEXT) { qemu_co_rwlock_wrlock(&s->rename_lock); } } -static inline void v9fs_path_read_lock(V9fsState *s) +static inline void coroutine_fn +v9fs_path_read_lock(V9fsState *s) { if (s->ctx.export_flags & V9FS_PATHNAME_FSCONTEXT) { qemu_co_rwlock_rdlock(&s->rename_lock); } } -static inline void v9fs_path_unlock(V9fsState *s) +static inline void coroutine_fn +v9fs_path_unlock(V9fsState *s) { if (s->ctx.export_flags & V9FS_PATHNAME_FSCONTEXT) { qemu_co_rwlock_unlock(&s->rename_lock); -- 2.37.3 From: Marc-André Lureau <marcandre.lureau@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-26-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- migration/migration.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index XXXXXXX..XXXXXXX 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -XXX,XX +XXX,XX @@ static void process_incoming_migration_bh(void *opaque) migration_incoming_state_destroy(); } -static void process_incoming_migration_co(void *opaque) +static void coroutine_fn +process_incoming_migration_co(void *opaque) { MigrationIncomingState *mis = migration_incoming_get_current(); PostcopyState ps; -- 2.37.3 From: Marc-André Lureau <marcandre.lureau@redhat.com> Callers of coroutine_fn must be coroutine_fn themselves, or the call must be within "if (qemu_in_coroutine())". Apply coroutine_fn to functions where this holds. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Alberto Faria <afaria@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220922084924.201610-27-pbonzini@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/unit/test-coroutine.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/unit/test-coroutine.c b/tests/unit/test-coroutine.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-coroutine.c +++ b/tests/unit/test-coroutine.c @@ -XXX,XX +XXX,XX @@ static void perf_baseline(void) g_test_message("Function call %u iterations: %f s", maxcycles, duration); } -static __attribute__((noinline)) void perf_cost_func(void *opaque) +static __attribute__((noinline)) void coroutine_fn perf_cost_func(void *opaque) { qemu_coroutine_yield(); } -- 2.37.3 Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20221006122607.162769-1-kwolf@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/quorum.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/block/quorum.c b/block/quorum.c index XXXXXXX..XXXXXXX 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -XXX,XX +XXX,XX @@ static bool quorum_has_too_much_io_failed(QuorumAIOCB *acb) return false; } -static int read_fifo_child(QuorumAIOCB *acb); - static void quorum_copy_qiov(QEMUIOVector *dest, QEMUIOVector *source) { int i; -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> job mutex will be used to protect the job struct elements and list, replacing AioContext locks. Right now use a shared lock for all jobs, in order to keep things simple. Once the AioContext lock is gone, we can introduce per-job locks. To simplify the switch from aiocontext to job lock, introduce *nop* lock/unlock functions and macros. We want to always call job_lock/unlock outside the AioContext locks, and not vice-versa, otherwise we might get a deadlock. This is not straightforward to do, and that's why we start with nop functions. Once everything is protected by job_lock/unlock, we can change the nop into an actual mutex and remove the aiocontext lock. Since job_mutex is already being used, add static real_job_{lock/unlock} for the existing usage. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-Id: <20220926093214.506243-2-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 24 ++++++++++++++++++++++++ job.c | 35 +++++++++++++++++++++++------------ 2 files changed, 47 insertions(+), 12 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -XXX,XX +XXX,XX @@ typedef enum JobCreateFlags { JOB_MANUAL_DISMISS = 0x04, } JobCreateFlags; +extern QemuMutex job_mutex; + +#define JOB_LOCK_GUARD() /* QEMU_LOCK_GUARD(&job_mutex) */ + +#define WITH_JOB_LOCK_GUARD() /* WITH_QEMU_LOCK_GUARD(&job_mutex) */ + +/** + * job_lock: + * + * Take the mutex protecting the list of jobs and their status. + * Most functions called by the monitor need to call job_lock + * and job_unlock manually. On the other hand, function called + * by the block jobs themselves and by the block layer will take the + * lock for you. + */ +void job_lock(void); + +/** + * job_unlock: + * + * Release the mutex protecting the list of jobs and their status. + */ +void job_unlock(void); + /** * Allocate and return a new job transaction. Jobs can be added to the * transaction using job_txn_add_job(). diff --git a/job.c b/job.c index XXXXXXX..XXXXXXX 100644 --- a/job.c +++ b/job.c @@ -XXX,XX +XXX,XX @@ #include "trace/trace-root.h" #include "qapi/qapi-events-job.h" +/* + * job_mutex protects the jobs list, but also makes the + * struct job fields thread-safe. + */ +QemuMutex job_mutex; + static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs); /* Job State Transition Table */ @@ -XXX,XX +XXX,XX @@ struct JobTxn { int refcnt; }; -/* Right now, this mutex is only needed to synchronize accesses to job->busy - * and job->sleep_timer, such as concurrent calls to job_do_yield and - * job_enter. */ -static QemuMutex job_mutex; +void job_lock(void) +{ + /* nop */ +} + +void job_unlock(void) +{ + /* nop */ +} -static void job_lock(void) +static void real_job_lock(void) { qemu_mutex_lock(&job_mutex); } -static void job_unlock(void) +static void real_job_unlock(void) { qemu_mutex_unlock(&job_mutex); } @@ -XXX,XX +XXX,XX @@ void job_enter_cond(Job *job, bool(*fn)(Job *job)) return; } - job_lock(); + real_job_lock(); if (job->busy) { - job_unlock(); + real_job_unlock(); return; } if (fn && !fn(job)) { - job_unlock(); + real_job_unlock(); return; } assert(!job->deferred_to_main_loop); timer_del(&job->sleep_timer); job->busy = true; - job_unlock(); + real_job_unlock(); aio_co_enter(job->aio_context, job->co); } @@ -XXX,XX +XXX,XX @@ void job_enter(Job *job) * called explicitly. */ static void coroutine_fn job_do_yield(Job *job, uint64_t ns) { - job_lock(); + real_job_lock(); if (ns != -1) { timer_mod(&job->sleep_timer, ns); } job->busy = false; job_event_idle(job); - job_unlock(); + real_job_unlock(); qemu_coroutine_yield(); /* Set by job_enter_cond() before re-entering the coroutine. */ -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Categorize the fields in struct Job to understand which ones need to be protected by the job mutex and which don't. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20220926093214.506243-3-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 61 +++++++++++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 25 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -XXX,XX +XXX,XX @@ typedef struct JobTxn JobTxn; * Long-running operation. */ typedef struct Job { + + /* Fields set at initialization (job_create), and never modified */ + /** The ID of the job. May be NULL for internal jobs. */ char *id; - /** The type of this job. */ + /** + * The type of this job. + * All callbacks are called with job_mutex *not* held. + */ const JobDriver *driver; - /** Reference count of the block job */ - int refcnt; - - /** Current state; See @JobStatus for details. */ - JobStatus status; - - /** AioContext to run the job coroutine in */ - AioContext *aio_context; - /** * The coroutine that executes the job. If not NULL, it is reentered when * busy is false and the job is cancelled. + * Initialized in job_start() */ Coroutine *co; + /** True if this job should automatically finalize itself */ + bool auto_finalize; + + /** True if this job should automatically dismiss itself */ + bool auto_dismiss; + + /** The completion function that will be called when the job completes. */ + BlockCompletionFunc *cb; + + /** The opaque value that is passed to the completion function. */ + void *opaque; + + /* ProgressMeter API is thread-safe */ + ProgressMeter progress; + + + /** Protected by AioContext lock */ + + /** AioContext to run the job coroutine in */ + AioContext *aio_context; + + /** Reference count of the block job */ + int refcnt; + + /** Current state; See @JobStatus for details. */ + JobStatus status; + /** * Timer that is used by @job_sleep_ns. Accessed under job_mutex (in * job.c). @@ -XXX,XX +XXX,XX @@ typedef struct Job { /** Set to true when the job has deferred work to the main loop. */ bool deferred_to_main_loop; - /** True if this job should automatically finalize itself */ - bool auto_finalize; - - /** True if this job should automatically dismiss itself */ - bool auto_dismiss; - - ProgressMeter progress; - /** * Return code from @run and/or @prepare callback(s). * Not final until the job has reached the CONCLUDED status. @@ -XXX,XX +XXX,XX @@ typedef struct Job { */ Error *err; - /** The completion function that will be called when the job completes. */ - BlockCompletionFunc *cb; - - /** The opaque value that is passed to the completion function. */ - void *opaque; - /** Notifiers called when a cancelled job is finalised */ NotifierList on_finalize_cancelled; @@ -XXX,XX +XXX,XX @@ typedef struct Job { /** * Callbacks and other information about a Job driver. + * All callbacks are invoked with job_mutex *not* held. */ struct JobDriver { @@ -XXX,XX +XXX,XX @@ void coroutine_fn job_yield(Job *job); */ void coroutine_fn job_sleep_ns(Job *job, int64_t ns); - /** Returns the JobType of a given Job. */ JobType job_type(const Job *job); -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> job_event_* functions can all be static, as they are not used outside job.c. Same applies for job_txn_add_job(). Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20220926093214.506243-4-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 18 ------------------ job.c | 22 +++++++++++++++++++--- 2 files changed, 19 insertions(+), 21 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -XXX,XX +XXX,XX @@ JobTxn *job_txn_new(void); */ void job_txn_unref(JobTxn *txn); -/** - * @txn: The transaction (may be NULL) - * @job: Job to add to the transaction - * - * Add @job to the transaction. The @job must not already be in a transaction. - * The caller must call either job_txn_unref() or job_completed() to release - * the reference that is automatically grabbed here. - * - * If @txn is NULL, the function does nothing. - */ -void job_txn_add_job(JobTxn *txn, Job *job); - /** * Create a new long-running job and return it. * @@ -XXX,XX +XXX,XX @@ void job_progress_set_remaining(Job *job, uint64_t remaining); */ void job_progress_increase_remaining(Job *job, uint64_t delta); -/** To be called when a cancelled job is finalised. */ -void job_event_cancelled(Job *job); - -/** To be called when a successfully completed job is finalised. */ -void job_event_completed(Job *job); - /** * Conditionally enter the job coroutine if the job is ready to run, not * already busy and fn() returns true. fn() is called while under the job_lock diff --git a/job.c b/job.c index XXXXXXX..XXXXXXX 100644 --- a/job.c +++ b/job.c @@ -XXX,XX +XXX,XX @@ void job_txn_unref(JobTxn *txn) } } -void job_txn_add_job(JobTxn *txn, Job *job) +/** + * @txn: The transaction (may be NULL) + * @job: Job to add to the transaction + * + * Add @job to the transaction. The @job must not already be in a transaction. + * The caller must call either job_txn_unref() or job_completed() to release + * the reference that is automatically grabbed here. + * + * If @txn is NULL, the function does nothing. + */ +static void job_txn_add_job(JobTxn *txn, Job *job) { if (!txn) { return; @@ -XXX,XX +XXX,XX @@ void job_progress_increase_remaining(Job *job, uint64_t delta) progress_increase_remaining(&job->progress, delta); } -void job_event_cancelled(Job *job) +/** + * To be called when a cancelled job is finalised. + */ +static void job_event_cancelled(Job *job) { notifier_list_notify(&job->on_finalize_cancelled, job); } -void job_event_completed(Job *job) +/** + * To be called when a successfully completed job is finalised. + */ +static void job_event_completed(Job *job) { notifier_list_notify(&job->on_finalize_completed, job); } -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Same as AIO_WAIT_WHILE macro, but if we are in the Main loop do not release and then acquire ctx_ 's aiocontext. Once all Aiocontext locks go away, this macro will replace AIO_WAIT_WHILE. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-Id: <20220926093214.506243-5-eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/aio-wait.h | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/aio-wait.h +++ b/include/block/aio-wait.h @@ -XXX,XX +XXX,XX @@ typedef struct { extern AioWait global_aio_wait; /** - * AIO_WAIT_WHILE: + * AIO_WAIT_WHILE_INTERNAL: * @ctx: the aio context, or NULL if multiple aio contexts (for which the * caller does not hold a lock) are involved in the polling condition. * @cond: wait while this conditional expression is true + * @unlock: whether to unlock and then lock again @ctx. This apples + * only when waiting for another AioContext from the main loop. + * Otherwise it's ignored. * * Wait while a condition is true. Use this to implement synchronous * operations that require event loop activity. @@ -XXX,XX +XXX,XX @@ extern AioWait global_aio_wait; * wait on conditions between two IOThreads since that could lead to deadlock, * go via the main loop instead. */ -#define AIO_WAIT_WHILE(ctx, cond) ({ \ +#define AIO_WAIT_WHILE_INTERNAL(ctx, cond, unlock) ({ \ bool waited_ = false; \ AioWait *wait_ = &global_aio_wait; \ AioContext *ctx_ = (ctx); \ @@ -XXX,XX +XXX,XX @@ extern AioWait global_aio_wait; assert(qemu_get_current_aio_context() == \ qemu_get_aio_context()); \ while ((cond)) { \ - if (ctx_) { \ + if (unlock && ctx_) { \ aio_context_release(ctx_); \ } \ aio_poll(qemu_get_aio_context(), true); \ - if (ctx_) { \ + if (unlock && ctx_) { \ aio_context_acquire(ctx_); \ } \ waited_ = true; \ @@ -XXX,XX +XXX,XX @@ extern AioWait global_aio_wait; qatomic_dec(&wait_->num_waiters); \ waited_; }) +#define AIO_WAIT_WHILE(ctx, cond) \ + AIO_WAIT_WHILE_INTERNAL(ctx, cond, true) + +#define AIO_WAIT_WHILE_UNLOCKED(ctx, cond) \ + AIO_WAIT_WHILE_INTERNAL(ctx, cond, false) + /** * aio_wait_kick: * Wake up the main thread if it is waiting on AIO_WAIT_WHILE(). During -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> With "intact" we mean that all job.h functions implicitly take the lock. Therefore API callers are unmodified. This means that: - many static functions that will be always called with job lock held become _locked, and call _locked functions - all public functions take the lock internally if needed, and call _locked functions - all public functions called internally by other functions in job.c will have a _locked counterpart (sometimes public), to avoid deadlocks (job lock already taken). These functions are not used for now. - some public functions called only from exernal files (not job.c) do not have _locked() counterpart and take the lock inside. Others won't need the lock at all because use fields only set at initialization and never modified. job_{lock/unlock} is independent from real_job_{lock/unlock}. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop* Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-Id: <20220926093214.506243-6-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 138 +++++++++- job.c | 610 ++++++++++++++++++++++++++++++++------------- 2 files changed, 561 insertions(+), 187 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -XXX,XX +XXX,XX @@ JobTxn *job_txn_new(void); */ void job_txn_unref(JobTxn *txn); +/* + * Same as job_txn_unref(), but called with job lock held. + * Might release the lock temporarily. + */ +void job_txn_unref_locked(JobTxn *txn); + /** * Create a new long-running job and return it. + * Called with job_mutex *not* held. * * @job_id: The id of the newly-created job, or %NULL for internal jobs * @driver: The class object for the newly-created job. @@ -XXX,XX +XXX,XX @@ void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn, */ void job_ref(Job *job); +/* Same as job_ref(), but called with job lock held. */ +void job_ref_locked(Job *job); + /** * Release a reference that was previously acquired with job_ref() or * job_create(). If it's the last reference to the object, it will be freed. */ void job_unref(Job *job); +/* Same as job_unref(), but called with job lock held. */ +void job_unref_locked(Job *job); + /** * @job: The job that has made progress * @done: How much progress the job made since the last call * * Updates the progress counter of the job. + * + * May be called with mutex held or not held. */ void job_progress_update(Job *job, uint64_t done); @@ -XXX,XX +XXX,XX @@ void job_progress_update(Job *job, uint64_t done); * * Sets the expected end value of the progress counter of a job so that a * completion percentage can be calculated when the progress is updated. + * + * May be called with mutex held or not held. */ void job_progress_set_remaining(Job *job, uint64_t remaining); @@ -XXX,XX +XXX,XX @@ void job_progress_set_remaining(Job *job, uint64_t remaining); * length before, and job_progress_update() afterwards. * (So the operation acts as a parenthesis in regards to the main job * operation running in background.) + * + * May be called with mutex held or not held. */ void job_progress_increase_remaining(Job *job, uint64_t delta); @@ -XXX,XX +XXX,XX @@ void job_progress_increase_remaining(Job *job, uint64_t delta); */ void job_enter_cond(Job *job, bool(*fn)(Job *job)); +/* + * Same as job_enter_cond(), but called with job lock held. + * Might release the lock temporarily. + */ +void job_enter_cond_locked(Job *job, bool(*fn)(Job *job)); + /** * @job: A job that has not yet been started. * * Begins execution of a job. * Takes ownership of one reference to the job object. + * + * Called with job_mutex *not* held. */ void job_start(Job *job); @@ -XXX,XX +XXX,XX @@ void job_start(Job *job); * @job: The job to enter. * * Continue the specified job by entering the coroutine. + * Called with job_mutex *not* held. */ void job_enter(Job *job); @@ -XXX,XX +XXX,XX @@ void job_enter(Job *job); * * Pause now if job_pause() has been called. Jobs that perform lots of I/O * must call this between requests so that the job can be paused. + * + * Called with job_mutex *not* held. */ void coroutine_fn job_pause_point(Job *job); @@ -XXX,XX +XXX,XX @@ void coroutine_fn job_pause_point(Job *job); * @job: The job that calls the function. * * Yield the job coroutine. + * Called with job_mutex *not* held. */ void coroutine_fn job_yield(Job *job); @@ -XXX,XX +XXX,XX @@ void coroutine_fn job_yield(Job *job); * Put the job to sleep (assuming that it wasn't canceled) for @ns * %QEMU_CLOCK_REALTIME nanoseconds. Canceling the job will immediately * interrupt the wait. + * + * Called with job_mutex *not* held. */ void coroutine_fn job_sleep_ns(Job *job, int64_t ns); @@ -XXX,XX +XXX,XX @@ const char *job_type_str(const Job *job); /** Returns true if the job should not be visible to the management layer. */ bool job_is_internal(Job *job); -/** Returns whether the job is being cancelled. */ +/** + * Returns whether the job is being cancelled. + * Called with job_mutex *not* held. + */ bool job_is_cancelled(Job *job); +/* Same as job_is_cancelled(), but called with job lock held. */ +bool job_is_cancelled_locked(Job *job); + /** * Returns whether the job is scheduled for cancellation (at an * indefinite point). + * Called with job_mutex *not* held. */ bool job_cancel_requested(Job *job); -/** Returns whether the job is in a completed state. */ +/** + * Returns whether the job is in a completed state. + * Called with job_mutex *not* held. + */ bool job_is_completed(Job *job); -/** Returns whether the job is ready to be completed. */ +/* Same as job_is_completed(), but called with job lock held. */ +bool job_is_completed_locked(Job *job); + +/** + * Returns whether the job is ready to be completed. + * Called with job_mutex *not* held. + */ bool job_is_ready(Job *job); +/* Same as job_is_ready(), but called with job lock held. */ +bool job_is_ready_locked(Job *job); + /** * Request @job to pause at the next pause point. Must be paired with * job_resume(). If the job is supposed to be resumed by user action, call @@ -XXX,XX +XXX,XX @@ bool job_is_ready(Job *job); */ void job_pause(Job *job); +/* Same as job_pause(), but called with job lock held. */ +void job_pause_locked(Job *job); + /** Resumes a @job paused with job_pause. */ void job_resume(Job *job); +/* + * Same as job_resume(), but called with job lock held. + * Might release the lock temporarily. + */ +void job_resume_locked(Job *job); + /** * Asynchronously pause the specified @job. * Do not allow a resume until a matching call to job_user_resume. */ void job_user_pause(Job *job, Error **errp); +/* Same as job_user_pause(), but called with job lock held. */ +void job_user_pause_locked(Job *job, Error **errp); + /** Returns true if the job is user-paused. */ bool job_user_paused(Job *job); +/* Same as job_user_paused(), but called with job lock held. */ +bool job_user_paused_locked(Job *job); + /** * Resume the specified @job. * Must be paired with a preceding job_user_pause. */ void job_user_resume(Job *job, Error **errp); +/* + * Same as job_user_resume(), but called with job lock held. + * Might release the lock temporarily. + */ +void job_user_resume_locked(Job *job, Error **errp); + /** * Get the next element from the list of block jobs after @job, or the * first one if @job is %NULL. @@ -XXX,XX +XXX,XX @@ void job_user_resume(Job *job, Error **errp); */ Job *job_next(Job *job); +/* Same as job_next(), but called with job lock held. */ +Job *job_next_locked(Job *job); + /** * Get the job identified by @id (which must not be %NULL). * @@ -XXX,XX +XXX,XX @@ Job *job_next(Job *job); */ Job *job_get(const char *id); +/* Same as job_get(), but called with job lock held. */ +Job *job_get_locked(const char *id); + /** * Check whether the verb @verb can be applied to @job in its current state. * Returns 0 if the verb can be applied; otherwise errp is set and -EPERM @@ -XXX,XX +XXX,XX @@ Job *job_get(const char *id); */ int job_apply_verb(Job *job, JobVerb verb, Error **errp); -/** The @job could not be started, free it. */ +/* Same as job_apply_verb, but called with job lock held. */ +int job_apply_verb_locked(Job *job, JobVerb verb, Error **errp); + +/** + * The @job could not be started, free it. + * Called with job_mutex *not* held. + */ void job_early_fail(Job *job); -/** Moves the @job from RUNNING to READY */ +/** + * Moves the @job from RUNNING to READY. + * Called with job_mutex *not* held. + */ void job_transition_to_ready(Job *job); /** Asynchronously complete the specified @job. */ void job_complete(Job *job, Error **errp); +/* + * Same as job_complete(), but called with job lock held. + * Might release the lock temporarily. + */ +void job_complete_locked(Job *job, Error **errp); + /** * Asynchronously cancel the specified @job. If @force is true, the job should * be cancelled immediately without waiting for a consistent state. */ void job_cancel(Job *job, bool force); +/* Same as job_cancel(), but called with job lock held. */ +void job_cancel_locked(Job *job, bool force); + /** * Cancels the specified job like job_cancel(), but may refuse to do so if the * operation isn't meaningful in the current state of the job. */ void job_user_cancel(Job *job, bool force, Error **errp); +/* Same as job_user_cancel(), but called with job lock held. */ +void job_user_cancel_locked(Job *job, bool force, Error **errp); + /** * Synchronously cancel the @job. The completion callback is called * before the function returns. If @force is false, the job may @@ -XXX,XX +XXX,XX @@ void job_user_cancel(Job *job, bool force, Error **errp); */ int job_cancel_sync(Job *job, bool force); -/** Synchronously force-cancels all jobs using job_cancel_sync(). */ +/* Same as job_cancel_sync, but called with job lock held. */ +int job_cancel_sync_locked(Job *job, bool force); + +/** + * Synchronously force-cancels all jobs using job_cancel_sync_locked(). + * + * Called with job_lock *not* held. + */ void job_cancel_sync_all(void); /** @@ -XXX,XX +XXX,XX @@ void job_cancel_sync_all(void); */ int job_complete_sync(Job *job, Error **errp); +/* Same as job_complete_sync, but called with job lock held. */ +int job_complete_sync_locked(Job *job, Error **errp); + /** * For a @job that has finished its work and is pending awaiting explicit * acknowledgement to commit its work, this will commit that work. @@ -XXX,XX +XXX,XX @@ int job_complete_sync(Job *job, Error **errp); */ void job_finalize(Job *job, Error **errp); +/* Same as job_finalize(), but called with job lock held. */ +void job_finalize_locked(Job *job, Error **errp); + /** * Remove the concluded @job from the query list and resets the passed pointer * to %NULL. Returns an error if the job is not actually concluded. */ void job_dismiss(Job **job, Error **errp); +/* Same as job_dismiss(), but called with job lock held. */ +void job_dismiss_locked(Job **job, Error **errp); + /** * Synchronously finishes the given @job. If @finish is given, it is called to * trigger completion or cancellation of the job. @@ -XXX,XX +XXX,XX @@ void job_dismiss(Job **job, Error **errp); * * Callers must hold the AioContext lock of job->aio_context. */ -int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp); +int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), + Error **errp); + +/* + * Same as job_finish_sync(), but called with job lock held. + * Might release the lock temporarily. + */ +int job_finish_sync_locked(Job *job, void (*finish)(Job *, Error **errp), + Error **errp); #endif diff --git a/job.c b/job.c index XXXXXXX..XXXXXXX 100644 --- a/job.c +++ b/job.c @@ -XXX,XX +XXX,XX @@ */ QemuMutex job_mutex; +/* Protected by job_mutex */ static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs); /* Job State Transition Table */ @@ -XXX,XX +XXX,XX @@ JobTxn *job_txn_new(void) return txn; } -static void job_txn_ref(JobTxn *txn) +/* Called with job_mutex held. */ +static void job_txn_ref_locked(JobTxn *txn) { txn->refcnt++; } -void job_txn_unref(JobTxn *txn) +void job_txn_unref_locked(JobTxn *txn) { if (txn && --txn->refcnt == 0) { g_free(txn); } } +void job_txn_unref(JobTxn *txn) +{ + JOB_LOCK_GUARD(); + job_txn_unref_locked(txn); +} + /** * @txn: The transaction (may be NULL) * @job: Job to add to the transaction @@ -XXX,XX +XXX,XX @@ void job_txn_unref(JobTxn *txn) * the reference that is automatically grabbed here. * * If @txn is NULL, the function does nothing. + * + * Called with job_mutex held. */ -static void job_txn_add_job(JobTxn *txn, Job *job) +static void job_txn_add_job_locked(JobTxn *txn, Job *job) { if (!txn) { return; @@ -XXX,XX +XXX,XX @@ static void job_txn_add_job(JobTxn *txn, Job *job) job->txn = txn; QLIST_INSERT_HEAD(&txn->jobs, job, txn_list); - job_txn_ref(txn); + job_txn_ref_locked(txn); } -static void job_txn_del_job(Job *job) +/* Called with job_mutex held. */ +static void job_txn_del_job_locked(Job *job) { if (job->txn) { QLIST_REMOVE(job, txn_list); - job_txn_unref(job->txn); + job_txn_unref_locked(job->txn); job->txn = NULL; } } -static int job_txn_apply(Job *job, int fn(Job *)) +/* Called with job_mutex held, but releases it temporarily. */ +static int job_txn_apply_locked(Job *job, int fn(Job *)) { AioContext *inner_ctx; Job *other_job, *next; @@ -XXX,XX +XXX,XX @@ static int job_txn_apply(Job *job, int fn(Job *)) * we need to release it here to avoid holding the lock twice - which would * break AIO_WAIT_WHILE from within fn. */ - job_ref(job); + job_ref_locked(job); aio_context_release(job->aio_context); QLIST_FOREACH_SAFE(other_job, &txn->jobs, txn_list, next) { @@ -XXX,XX +XXX,XX @@ static int job_txn_apply(Job *job, int fn(Job *)) * can't use a local variable to cache it. */ aio_context_acquire(job->aio_context); - job_unref(job); + job_unref_locked(job); return rc; } @@ -XXX,XX +XXX,XX @@ bool job_is_internal(Job *job) return (job->id == NULL); } -static void job_state_transition(Job *job, JobStatus s1) +/* Called with job_mutex held. */ +static void job_state_transition_locked(Job *job, JobStatus s1) { JobStatus s0 = job->status; assert(s1 >= 0 && s1 < JOB_STATUS__MAX); @@ -XXX,XX +XXX,XX @@ static void job_state_transition(Job *job, JobStatus s1) } } -int job_apply_verb(Job *job, JobVerb verb, Error **errp) +int job_apply_verb_locked(Job *job, JobVerb verb, Error **errp) { JobStatus s0 = job->status; assert(verb >= 0 && verb < JOB_VERB__MAX); @@ -XXX,XX +XXX,XX @@ int job_apply_verb(Job *job, JobVerb verb, Error **errp) return -EPERM; } +int job_apply_verb(Job *job, JobVerb verb, Error **errp) +{ + JOB_LOCK_GUARD(); + return job_apply_verb_locked(job, verb, errp); +} + JobType job_type(const Job *job) { return job->driver->job_type; @@ -XXX,XX +XXX,XX @@ const char *job_type_str(const Job *job) return JobType_str(job_type(job)); } -bool job_is_cancelled(Job *job) +bool job_is_cancelled_locked(Job *job) { /* force_cancel may be true only if cancelled is true, too */ assert(job->cancelled || !job->force_cancel); return job->force_cancel; } -bool job_cancel_requested(Job *job) +bool job_is_cancelled(Job *job) +{ + JOB_LOCK_GUARD(); + return job_is_cancelled_locked(job); +} + +/* Called with job_mutex held. */ +static bool job_cancel_requested_locked(Job *job) { return job->cancelled; } -bool job_is_ready(Job *job) +bool job_cancel_requested(Job *job) +{ + JOB_LOCK_GUARD(); + return job_cancel_requested_locked(job); +} + +bool job_is_ready_locked(Job *job) { switch (job->status) { case JOB_STATUS_UNDEFINED: @@ -XXX,XX +XXX,XX @@ bool job_is_ready(Job *job) return false; } -bool job_is_completed(Job *job) +bool job_is_ready(Job *job) +{ + JOB_LOCK_GUARD(); + return job_is_ready_locked(job); +} + +bool job_is_completed_locked(Job *job) { switch (job->status) { case JOB_STATUS_UNDEFINED: @@ -XXX,XX +XXX,XX @@ bool job_is_completed(Job *job) return false; } -static bool job_started(Job *job) +bool job_is_completed(Job *job) +{ + JOB_LOCK_GUARD(); + return job_is_completed_locked(job); +} + +static bool job_started_locked(Job *job) { return job->co; } -static bool job_should_pause(Job *job) +/* Called with job_mutex held. */ +static bool job_should_pause_locked(Job *job) { return job->pause_count > 0; } -Job *job_next(Job *job) +Job *job_next_locked(Job *job) { if (!job) { return QLIST_FIRST(&jobs); @@ -XXX,XX +XXX,XX @@ Job *job_next(Job *job) return QLIST_NEXT(job, job_list); } -Job *job_get(const char *id) +Job *job_next(Job *job) +{ + JOB_LOCK_GUARD(); + return job_next_locked(job); +} + +Job *job_get_locked(const char *id) { Job *job; @@ -XXX,XX +XXX,XX @@ Job *job_get(const char *id) return NULL; } +Job *job_get(const char *id) +{ + JOB_LOCK_GUARD(); + return job_get_locked(id); +} + +/* Called with job_mutex *not* held. */ static void job_sleep_timer_cb(void *opaque) { Job *job = opaque; @@ -XXX,XX +XXX,XX @@ void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn, { Job *job; + JOB_LOCK_GUARD(); + if (job_id) { if (flags & JOB_INTERNAL) { error_setg(errp, "Cannot specify job ID for internal job"); @@ -XXX,XX +XXX,XX @@ void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn, error_setg(errp, "Invalid job ID '%s'", job_id); return NULL; } - if (job_get(job_id)) { + if (job_get_locked(job_id)) { error_setg(errp, "Job ID '%s' already in use", job_id); return NULL; } @@ -XXX,XX +XXX,XX @@ void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn, notifier_list_init(&job->on_ready); notifier_list_init(&job->on_idle); - job_state_transition(job, JOB_STATUS_CREATED); + job_state_transition_locked(job, JOB_STATUS_CREATED); aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, QEMU_CLOCK_REALTIME, SCALE_NS, job_sleep_timer_cb, job); @@ -XXX,XX +XXX,XX @@ void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn, * consolidating the job management logic */ if (!txn) { txn = job_txn_new(); - job_txn_add_job(txn, job); - job_txn_unref(txn); + job_txn_add_job_locked(txn, job); + job_txn_unref_locked(txn); } else { - job_txn_add_job(txn, job); + job_txn_add_job_locked(txn, job); } return job; } -void job_ref(Job *job) +void job_ref_locked(Job *job) { ++job->refcnt; } -void job_unref(Job *job) +void job_ref(Job *job) +{ + JOB_LOCK_GUARD(); + job_ref_locked(job); +} + +void job_unref_locked(Job *job) { GLOBAL_STATE_CODE(); @@ -XXX,XX +XXX,XX @@ void job_unref(Job *job) assert(!job->txn); if (job->driver->free) { + job_unlock(); job->driver->free(job); + job_lock(); } QLIST_REMOVE(job, job_list); @@ -XXX,XX +XXX,XX @@ void job_unref(Job *job) } } +void job_unref(Job *job) +{ + JOB_LOCK_GUARD(); + job_unref_locked(job); +} + void job_progress_update(Job *job, uint64_t done) { progress_work_done(&job->progress, done); @@ -XXX,XX +XXX,XX @@ void job_progress_increase_remaining(Job *job, uint64_t delta) /** * To be called when a cancelled job is finalised. + * Called with job_mutex held. */ -static void job_event_cancelled(Job *job) +static void job_event_cancelled_locked(Job *job) { notifier_list_notify(&job->on_finalize_cancelled, job); } /** * To be called when a successfully completed job is finalised. + * Called with job_mutex held. */ -static void job_event_completed(Job *job) +static void job_event_completed_locked(Job *job) { notifier_list_notify(&job->on_finalize_completed, job); } -static void job_event_pending(Job *job) +/* Called with job_mutex held. */ +static void job_event_pending_locked(Job *job) { notifier_list_notify(&job->on_pending, job); } -static void job_event_ready(Job *job) +/* Called with job_mutex held. */ +static void job_event_ready_locked(Job *job) { notifier_list_notify(&job->on_ready, job); } -static void job_event_idle(Job *job) +/* Called with job_mutex held. */ +static void job_event_idle_locked(Job *job) { notifier_list_notify(&job->on_idle, job); } -void job_enter_cond(Job *job, bool(*fn)(Job *job)) +void job_enter_cond_locked(Job *job, bool(*fn)(Job *job)) { - if (!job_started(job)) { + if (!job_started_locked(job)) { return; } if (job->deferred_to_main_loop) { @@ -XXX,XX +XXX,XX @@ void job_enter_cond(Job *job, bool(*fn)(Job *job)) timer_del(&job->sleep_timer); job->busy = true; real_job_unlock(); + job_unlock(); aio_co_enter(job->aio_context, job->co); + job_lock(); +} + +void job_enter_cond(Job *job, bool(*fn)(Job *job)) +{ + JOB_LOCK_GUARD(); + job_enter_cond_locked(job, fn); } void job_enter(Job *job) { - job_enter_cond(job, NULL); + JOB_LOCK_GUARD(); + job_enter_cond_locked(job, NULL); } /* Yield, and schedule a timer to reenter the coroutine after @ns nanoseconds. @@ -XXX,XX +XXX,XX @@ void job_enter(Job *job) * is allowed and cancels the timer. * * If @ns is (uint64_t) -1, no timer is scheduled and job_enter() must be - * called explicitly. */ -static void coroutine_fn job_do_yield(Job *job, uint64_t ns) + * called explicitly. + * + * Called with job_mutex held, but releases it temporarily. + */ +static void coroutine_fn job_do_yield_locked(Job *job, uint64_t ns) { real_job_lock(); if (ns != -1) { timer_mod(&job->sleep_timer, ns); } job->busy = false; - job_event_idle(job); + job_event_idle_locked(job); real_job_unlock(); + job_unlock(); qemu_coroutine_yield(); + job_lock(); /* Set by job_enter_cond() before re-entering the coroutine. */ assert(job->busy); } -void coroutine_fn job_pause_point(Job *job) +/* Called with job_mutex held, but releases it temporarily. */ +static void coroutine_fn job_pause_point_locked(Job *job) { - assert(job && job_started(job)); + assert(job && job_started_locked(job)); - if (!job_should_pause(job)) { + if (!job_should_pause_locked(job)) { return; } - if (job_is_cancelled(job)) { + if (job_is_cancelled_locked(job)) { return; } if (job->driver->pause) { + job_unlock(); job->driver->pause(job); + job_lock(); } - if (job_should_pause(job) && !job_is_cancelled(job)) { + if (job_should_pause_locked(job) && !job_is_cancelled_locked(job)) { JobStatus status = job->status; - job_state_transition(job, status == JOB_STATUS_READY - ? JOB_STATUS_STANDBY - : JOB_STATUS_PAUSED); + job_state_transition_locked(job, status == JOB_STATUS_READY + ? JOB_STATUS_STANDBY + : JOB_STATUS_PAUSED); job->paused = true; - job_do_yield(job, -1); + job_do_yield_locked(job, -1); job->paused = false; - job_state_transition(job, status); + job_state_transition_locked(job, status); } if (job->driver->resume) { + job_unlock(); job->driver->resume(job); + job_lock(); } } -void coroutine_fn job_yield(Job *job) +void coroutine_fn job_pause_point(Job *job) +{ + JOB_LOCK_GUARD(); + job_pause_point_locked(job); +} + +static void coroutine_fn job_yield_locked(Job *job) { assert(job->busy); /* Check cancellation *before* setting busy = false, too! */ - if (job_is_cancelled(job)) { + if (job_is_cancelled_locked(job)) { return; } - if (!job_should_pause(job)) { - job_do_yield(job, -1); + if (!job_should_pause_locked(job)) { + job_do_yield_locked(job, -1); } - job_pause_point(job); + job_pause_point_locked(job); +} + +void coroutine_fn job_yield(Job *job) +{ + JOB_LOCK_GUARD(); + job_yield_locked(job); } void coroutine_fn job_sleep_ns(Job *job, int64_t ns) { + JOB_LOCK_GUARD(); assert(job->busy); /* Check cancellation *before* setting busy = false, too! */ - if (job_is_cancelled(job)) { + if (job_is_cancelled_locked(job)) { return; } - if (!job_should_pause(job)) { - job_do_yield(job, qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + ns); + if (!job_should_pause_locked(job)) { + job_do_yield_locked(job, qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + ns); } - job_pause_point(job); + job_pause_point_locked(job); } -/* Assumes the block_job_mutex is held */ -static bool job_timer_not_pending(Job *job) +/* Assumes the job_mutex is held */ +static bool job_timer_not_pending_locked(Job *job) { return !timer_pending(&job->sleep_timer); } -void job_pause(Job *job) +void job_pause_locked(Job *job) { job->pause_count++; if (!job->paused) { - job_enter(job); + job_enter_cond_locked(job, NULL); } } -void job_resume(Job *job) +void job_pause(Job *job) +{ + JOB_LOCK_GUARD(); + job_pause_locked(job); +} + +void job_resume_locked(Job *job) { assert(job->pause_count > 0); job->pause_count--; @@ -XXX,XX +XXX,XX @@ void job_resume(Job *job) } /* kick only if no timer is pending */ - job_enter_cond(job, job_timer_not_pending); + job_enter_cond_locked(job, job_timer_not_pending_locked); } -void job_user_pause(Job *job, Error **errp) +void job_resume(Job *job) { - if (job_apply_verb(job, JOB_VERB_PAUSE, errp)) { + JOB_LOCK_GUARD(); + job_resume_locked(job); +} + +void job_user_pause_locked(Job *job, Error **errp) +{ + if (job_apply_verb_locked(job, JOB_VERB_PAUSE, errp)) { return; } if (job->user_paused) { @@ -XXX,XX +XXX,XX @@ void job_user_pause(Job *job, Error **errp) return; } job->user_paused = true; - job_pause(job); + job_pause_locked(job); } -bool job_user_paused(Job *job) +void job_user_pause(Job *job, Error **errp) +{ + JOB_LOCK_GUARD(); + job_user_pause_locked(job, errp); +} + +bool job_user_paused_locked(Job *job) { return job->user_paused; } -void job_user_resume(Job *job, Error **errp) +bool job_user_paused(Job *job) +{ + JOB_LOCK_GUARD(); + return job_user_paused_locked(job); +} + +void job_user_resume_locked(Job *job, Error **errp) { assert(job); GLOBAL_STATE_CODE(); @@ -XXX,XX +XXX,XX @@ void job_user_resume(Job *job, Error **errp) error_setg(errp, "Can't resume a job that was not paused"); return; } - if (job_apply_verb(job, JOB_VERB_RESUME, errp)) { + if (job_apply_verb_locked(job, JOB_VERB_RESUME, errp)) { return; } if (job->driver->user_resume) { + job_unlock(); job->driver->user_resume(job); + job_lock(); } job->user_paused = false; - job_resume(job); + job_resume_locked(job); } -static void job_do_dismiss(Job *job) +void job_user_resume(Job *job, Error **errp) +{ + JOB_LOCK_GUARD(); + job_user_resume_locked(job, errp); +} + +/* Called with job_mutex held, but releases it temporarily. */ +static void job_do_dismiss_locked(Job *job) { assert(job); job->busy = false; job->paused = false; job->deferred_to_main_loop = true; - job_txn_del_job(job); + job_txn_del_job_locked(job); - job_state_transition(job, JOB_STATUS_NULL); - job_unref(job); + job_state_transition_locked(job, JOB_STATUS_NULL); + job_unref_locked(job); } -void job_dismiss(Job **jobptr, Error **errp) +void job_dismiss_locked(Job **jobptr, Error **errp) { Job *job = *jobptr; /* similarly to _complete, this is QMP-interface only. */ assert(job->id); - if (job_apply_verb(job, JOB_VERB_DISMISS, errp)) { + if (job_apply_verb_locked(job, JOB_VERB_DISMISS, errp)) { return; } - job_do_dismiss(job); + job_do_dismiss_locked(job); *jobptr = NULL; } +void job_dismiss(Job **jobptr, Error **errp) +{ + JOB_LOCK_GUARD(); + job_dismiss_locked(jobptr, errp); +} + void job_early_fail(Job *job) { + JOB_LOCK_GUARD(); assert(job->status == JOB_STATUS_CREATED); - job_do_dismiss(job); + job_do_dismiss_locked(job); } -static void job_conclude(Job *job) +/* Called with job_mutex held. */ +static void job_conclude_locked(Job *job) { - job_state_transition(job, JOB_STATUS_CONCLUDED); - if (job->auto_dismiss || !job_started(job)) { - job_do_dismiss(job); + job_state_transition_locked(job, JOB_STATUS_CONCLUDED); + if (job->auto_dismiss || !job_started_locked(job)) { + job_do_dismiss_locked(job); } } -static void job_update_rc(Job *job) +/* Called with job_mutex held. */ +static void job_update_rc_locked(Job *job) { - if (!job->ret && job_is_cancelled(job)) { + if (!job->ret && job_is_cancelled_locked(job)) { job->ret = -ECANCELED; } if (job->ret) { if (!job->err) { error_setg(&job->err, "%s", strerror(-job->ret)); } - job_state_transition(job, JOB_STATUS_ABORTING); + job_state_transition_locked(job, JOB_STATUS_ABORTING); } } @@ -XXX,XX +XXX,XX @@ static void job_clean(Job *job) } } -static int job_finalize_single(Job *job) +/* Called with job_mutex held, but releases it temporarily */ +static int job_finalize_single_locked(Job *job) { - assert(job_is_completed(job)); + int job_ret; + + assert(job_is_completed_locked(job)); /* Ensure abort is called for late-transactional failures */ - job_update_rc(job); + job_update_rc_locked(job); + + job_ret = job->ret; + job_unlock(); - if (!job->ret) { + if (!job_ret) { job_commit(job); } else { job_abort(job); } job_clean(job); + job_lock(); + if (job->cb) { - job->cb(job->opaque, job->ret); + job_ret = job->ret; + job_unlock(); + job->cb(job->opaque, job_ret); + job_lock(); } /* Emit events only if we actually started */ - if (job_started(job)) { - if (job_is_cancelled(job)) { - job_event_cancelled(job); + if (job_started_locked(job)) { + if (job_is_cancelled_locked(job)) { + job_event_cancelled_locked(job); } else { - job_event_completed(job); + job_event_completed_locked(job); } } - job_txn_del_job(job); - job_conclude(job); + job_txn_del_job_locked(job); + job_conclude_locked(job); return 0; } -static void job_cancel_async(Job *job, bool force) +/* Called with job_mutex held, but releases it temporarily */ +static void job_cancel_async_locked(Job *job, bool force) { GLOBAL_STATE_CODE(); if (job->driver->cancel) { + job_unlock(); force = job->driver->cancel(job, force); + job_lock(); } else { /* No .cancel() means the job will behave as if force-cancelled */ force = true; @@ -XXX,XX +XXX,XX @@ static void job_cancel_async(Job *job, bool force) if (job->user_paused) { /* Do not call job_enter here, the caller will handle it. */ if (job->driver->user_resume) { + job_unlock(); job->driver->user_resume(job); + job_lock(); } job->user_paused = false; assert(job->pause_count > 0); @@ -XXX,XX +XXX,XX @@ static void job_cancel_async(Job *job, bool force) } } -static void job_completed_txn_abort(Job *job) +/* Called with job_mutex held, but releases it temporarily. */ +static void job_completed_txn_abort_locked(Job *job) { AioContext *ctx; JobTxn *txn = job->txn; @@ -XXX,XX +XXX,XX @@ static void job_completed_txn_abort(Job *job) return; } txn->aborting = true; - job_txn_ref(txn); + job_txn_ref_locked(txn); /* * We can only hold the single job's AioContext lock while calling @@ -XXX,XX +XXX,XX @@ static void job_completed_txn_abort(Job *job) * calls of AIO_WAIT_WHILE(), which could deadlock otherwise. * Note that the job's AioContext may change when it is finalized. */ - job_ref(job); + job_ref_locked(job); aio_context_release(job->aio_context); /* Other jobs are effectively cancelled by us, set the status for @@ -XXX,XX +XXX,XX @@ static void job_completed_txn_abort(Job *job) * Therefore, pass force=true to terminate all other jobs as quickly * as possible. */ - job_cancel_async(other_job, true); + job_cancel_async_locked(other_job, true); aio_context_release(ctx); } } @@ -XXX,XX +XXX,XX @@ static void job_completed_txn_abort(Job *job) */ ctx = other_job->aio_context; aio_context_acquire(ctx); - if (!job_is_completed(other_job)) { - assert(job_cancel_requested(other_job)); - job_finish_sync(other_job, NULL, NULL); + if (!job_is_completed_locked(other_job)) { + assert(job_cancel_requested_locked(other_job)); + job_finish_sync_locked(other_job, NULL, NULL); } - job_finalize_single(other_job); + job_finalize_single_locked(other_job); aio_context_release(ctx); } @@ -XXX,XX +XXX,XX @@ static void job_completed_txn_abort(Job *job) * even if the job went away during job_finalize_single(). */ aio_context_acquire(job->aio_context); - job_unref(job); + job_unref_locked(job); - job_txn_unref(txn); + job_txn_unref_locked(txn); } -static int job_prepare(Job *job) +/* Called with job_mutex held, but releases it temporarily */ +static int job_prepare_locked(Job *job) { + int ret; + GLOBAL_STATE_CODE(); if (job->ret == 0 && job->driver->prepare) { - job->ret = job->driver->prepare(job); - job_update_rc(job); + job_unlock(); + ret = job->driver->prepare(job); + job_lock(); + job->ret = ret; + job_update_rc_locked(job); } return job->ret; } -static int job_needs_finalize(Job *job) +/* Called with job_mutex held */ +static int job_needs_finalize_locked(Job *job) { return !job->auto_finalize; } -static void job_do_finalize(Job *job) +/* Called with job_mutex held */ +static void job_do_finalize_locked(Job *job) { int rc; assert(job && job->txn); /* prepare the transaction to complete */ - rc = job_txn_apply(job, job_prepare); + rc = job_txn_apply_locked(job, job_prepare_locked); if (rc) { - job_completed_txn_abort(job); + job_completed_txn_abort_locked(job); } else { - job_txn_apply(job, job_finalize_single); + job_txn_apply_locked(job, job_finalize_single_locked); } } -void job_finalize(Job *job, Error **errp) +void job_finalize_locked(Job *job, Error **errp) { assert(job && job->id); - if (job_apply_verb(job, JOB_VERB_FINALIZE, errp)) { + if (job_apply_verb_locked(job, JOB_VERB_FINALIZE, errp)) { return; } - job_do_finalize(job); + job_do_finalize_locked(job); } -static int job_transition_to_pending(Job *job) +void job_finalize(Job *job, Error **errp) { - job_state_transition(job, JOB_STATUS_PENDING); + JOB_LOCK_GUARD(); + job_finalize_locked(job, errp); +} + +/* Called with job_mutex held. */ +static int job_transition_to_pending_locked(Job *job) +{ + job_state_transition_locked(job, JOB_STATUS_PENDING); if (!job->auto_finalize) { - job_event_pending(job); + job_event_pending_locked(job); } return 0; } void job_transition_to_ready(Job *job) { - job_state_transition(job, JOB_STATUS_READY); - job_event_ready(job); + JOB_LOCK_GUARD(); + job_state_transition_locked(job, JOB_STATUS_READY); + job_event_ready_locked(job); } -static void job_completed_txn_success(Job *job) +/* Called with job_mutex held. */ +static void job_completed_txn_success_locked(Job *job) { JobTxn *txn = job->txn; Job *other_job; - job_state_transition(job, JOB_STATUS_WAITING); + job_state_transition_locked(job, JOB_STATUS_WAITING); /* * Successful completion, see if there are other running jobs in this * txn. */ QLIST_FOREACH(other_job, &txn->jobs, txn_list) { - if (!job_is_completed(other_job)) { + if (!job_is_completed_locked(other_job)) { return; } assert(other_job->ret == 0); } - job_txn_apply(job, job_transition_to_pending); + job_txn_apply_locked(job, job_transition_to_pending_locked); /* If no jobs need manual finalization, automatically do so */ - if (job_txn_apply(job, job_needs_finalize) == 0) { - job_do_finalize(job); + if (job_txn_apply_locked(job, job_needs_finalize_locked) == 0) { + job_do_finalize_locked(job); } } -static void job_completed(Job *job) +/* Called with job_mutex held. */ +static void job_completed_locked(Job *job) { - assert(job && job->txn && !job_is_completed(job)); + assert(job && job->txn && !job_is_completed_locked(job)); - job_update_rc(job); + job_update_rc_locked(job); trace_job_completed(job, job->ret); if (job->ret) { - job_completed_txn_abort(job); + job_completed_txn_abort_locked(job); } else { - job_completed_txn_success(job); + job_completed_txn_success_locked(job); } } -/** Useful only as a type shim for aio_bh_schedule_oneshot. */ +/** + * Useful only as a type shim for aio_bh_schedule_oneshot. + * Called with job_mutex *not* held. + */ static void job_exit(void *opaque) { Job *job = (Job *)opaque; AioContext *ctx; + JOB_LOCK_GUARD(); - job_ref(job); + job_ref_locked(job); aio_context_acquire(job->aio_context); /* This is a lie, we're not quiescent, but still doing the completion @@ -XXX,XX +XXX,XX @@ static void job_exit(void *opaque) * drain block nodes, and if .drained_poll still returned true, we would * deadlock. */ job->busy = false; - job_event_idle(job); + job_event_idle_locked(job); - job_completed(job); + job_completed_locked(job); /* * Note that calling job_completed can move the job to a different @@ -XXX,XX +XXX,XX @@ static void job_exit(void *opaque) * the job underneath us. */ ctx = job->aio_context; - job_unref(job); + job_unref_locked(job); aio_context_release(ctx); } @@ -XXX,XX +XXX,XX @@ static void job_exit(void *opaque) static void coroutine_fn job_co_entry(void *opaque) { Job *job = opaque; + int ret; assert(job && job->driver && job->driver->run); - assert(job->aio_context == qemu_get_current_aio_context()); - job_pause_point(job); - job->ret = job->driver->run(job, &job->err); - job->deferred_to_main_loop = true; - job->busy = true; + WITH_JOB_LOCK_GUARD() { + assert(job->aio_context == qemu_get_current_aio_context()); + job_pause_point_locked(job); + } + ret = job->driver->run(job, &job->err); + WITH_JOB_LOCK_GUARD() { + job->ret = ret; + job->deferred_to_main_loop = true; + job->busy = true; + } aio_bh_schedule_oneshot(qemu_get_aio_context(), job_exit, job); } void job_start(Job *job) { - assert(job && !job_started(job) && job->paused && - job->driver && job->driver->run); - job->co = qemu_coroutine_create(job_co_entry, job); - job->pause_count--; - job->busy = true; - job->paused = false; - job_state_transition(job, JOB_STATUS_RUNNING); + assert(qemu_in_main_thread()); + + WITH_JOB_LOCK_GUARD() { + assert(job && !job_started_locked(job) && job->paused && + job->driver && job->driver->run); + job->co = qemu_coroutine_create(job_co_entry, job); + job->pause_count--; + job->busy = true; + job->paused = false; + job_state_transition_locked(job, JOB_STATUS_RUNNING); + } aio_co_enter(job->aio_context, job->co); } -void job_cancel(Job *job, bool force) +void job_cancel_locked(Job *job, bool force) { if (job->status == JOB_STATUS_CONCLUDED) { - job_do_dismiss(job); + job_do_dismiss_locked(job); return; } - job_cancel_async(job, force); - if (!job_started(job)) { - job_completed(job); + job_cancel_async_locked(job, force); + if (!job_started_locked(job)) { + job_completed_locked(job); } else if (job->deferred_to_main_loop) { /* * job_cancel_async() ignores soft-cancel requests for jobs @@ -XXX,XX +XXX,XX @@ void job_cancel(Job *job, bool force) * choose to call job_is_cancelled() to show that we invoke * job_completed_txn_abort() only for force-cancelled jobs.) */ - if (job_is_cancelled(job)) { - job_completed_txn_abort(job); + if (job_is_cancelled_locked(job)) { + job_completed_txn_abort_locked(job); } } else { - job_enter(job); + job_enter_cond_locked(job, NULL); } } -void job_user_cancel(Job *job, bool force, Error **errp) +void job_cancel(Job *job, bool force) { - if (job_apply_verb(job, JOB_VERB_CANCEL, errp)) { + JOB_LOCK_GUARD(); + job_cancel_locked(job, force); +} + +void job_user_cancel_locked(Job *job, bool force, Error **errp) +{ + if (job_apply_verb_locked(job, JOB_VERB_CANCEL, errp)) { return; } - job_cancel(job, force); + job_cancel_locked(job, force); +} + +void job_user_cancel(Job *job, bool force, Error **errp) +{ + JOB_LOCK_GUARD(); + job_user_cancel_locked(job, force, errp); } /* A wrapper around job_cancel() taking an Error ** parameter so it may be * used with job_finish_sync() without the need for (rather nasty) function - * pointer casts there. */ -static void job_cancel_err(Job *job, Error **errp) + * pointer casts there. + * + * Called with job_mutex held. + */ +static void job_cancel_err_locked(Job *job, Error **errp) { - job_cancel(job, false); + job_cancel_locked(job, false); } /** * Same as job_cancel_err(), but force-cancel. + * Called with job_mutex held. */ -static void job_force_cancel_err(Job *job, Error **errp) +static void job_force_cancel_err_locked(Job *job, Error **errp) { - job_cancel(job, true); + job_cancel_locked(job, true); } -int job_cancel_sync(Job *job, bool force) +int job_cancel_sync_locked(Job *job, bool force) { if (force) { - return job_finish_sync(job, &job_force_cancel_err, NULL); + return job_finish_sync_locked(job, &job_force_cancel_err_locked, NULL); } else { - return job_finish_sync(job, &job_cancel_err, NULL); + return job_finish_sync_locked(job, &job_cancel_err_locked, NULL); } } +int job_cancel_sync(Job *job, bool force) +{ + JOB_LOCK_GUARD(); + return job_cancel_sync_locked(job, force); +} + void job_cancel_sync_all(void) { Job *job; AioContext *aio_context; + JOB_LOCK_GUARD(); - while ((job = job_next(NULL))) { + while ((job = job_next_locked(NULL))) { aio_context = job->aio_context; aio_context_acquire(aio_context); - job_cancel_sync(job, true); + job_cancel_sync_locked(job, true); aio_context_release(aio_context); } } +int job_complete_sync_locked(Job *job, Error **errp) +{ + return job_finish_sync_locked(job, job_complete_locked, errp); +} + int job_complete_sync(Job *job, Error **errp) { - return job_finish_sync(job, job_complete, errp); + JOB_LOCK_GUARD(); + return job_complete_sync_locked(job, errp); } -void job_complete(Job *job, Error **errp) +void job_complete_locked(Job *job, Error **errp) { /* Should not be reachable via external interface for internal jobs */ assert(job->id); GLOBAL_STATE_CODE(); - if (job_apply_verb(job, JOB_VERB_COMPLETE, errp)) { + if (job_apply_verb_locked(job, JOB_VERB_COMPLETE, errp)) { return; } - if (job_cancel_requested(job) || !job->driver->complete) { + if (job_cancel_requested_locked(job) || !job->driver->complete) { error_setg(errp, "The active block job '%s' cannot be completed", job->id); return; } + job_unlock(); job->driver->complete(job, errp); + job_lock(); } -int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) +void job_complete(Job *job, Error **errp) +{ + JOB_LOCK_GUARD(); + job_complete_locked(job, errp); +} + +int job_finish_sync_locked(Job *job, + void (*finish)(Job *, Error **errp), + Error **errp) { Error *local_err = NULL; int ret; - job_ref(job); + job_ref_locked(job); if (finish) { finish(job, &local_err); } if (local_err) { error_propagate(errp, local_err); - job_unref(job); + job_unref_locked(job); return -EBUSY; } + job_unlock(); AIO_WAIT_WHILE(job->aio_context, (job_enter(job), !job_is_completed(job))); + job_lock(); - ret = (job_is_cancelled(job) && job->ret == 0) ? -ECANCELED : job->ret; - job_unref(job); + ret = (job_is_cancelled_locked(job) && job->ret == 0) + ? -ECANCELED : job->ret; + job_unref_locked(job); return ret; } + +int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) +{ + JOB_LOCK_GUARD(); + return job_finish_sync_locked(job, finish, errp); +} -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> This comment applies more on job, it was left in blockjob as in the past the whole job logic was implemented there. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20220926093214.506243-7-eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- blockjob.c | 20 -------------------- job.c | 16 ++++++++++++++++ 2 files changed, 16 insertions(+), 20 deletions(-) diff --git a/blockjob.c b/blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/blockjob.c +++ b/blockjob.c @@ -XXX,XX +XXX,XX @@ #include "qemu/main-loop.h" #include "qemu/timer.h" -/* - * The block job API is composed of two categories of functions. - * - * The first includes functions used by the monitor. The monitor is - * peculiar in that it accesses the block job list with block_job_get, and - * therefore needs consistency across block_job_get and the actual operation - * (e.g. block_job_set_speed). The consistency is achieved with - * aio_context_acquire/release. These functions are declared in blockjob.h. - * - * The second includes functions used by the block job drivers and sometimes - * by the core block layer. These do not care about locking, because the - * whole coroutine runs under the AioContext lock, and are declared in - * blockjob_int.h. - */ - static bool is_block_job(Job *job) { return job_type(job) == JOB_TYPE_BACKUP || @@ -XXX,XX +XXX,XX @@ static void block_job_event_ready(Notifier *n, void *opaque) } -/* - * API for block job drivers and the block layer. These functions are - * declared in blockjob_int.h. - */ - void *block_job_create(const char *job_id, const BlockJobDriver *driver, JobTxn *txn, BlockDriverState *bs, uint64_t perm, uint64_t shared_perm, int64_t speed, int flags, diff --git a/job.c b/job.c index XXXXXXX..XXXXXXX 100644 --- a/job.c +++ b/job.c @@ -XXX,XX +XXX,XX @@ #include "trace/trace-root.h" #include "qapi/qapi-events-job.h" +/* + * The job API is composed of two categories of functions. + * + * The first includes functions used by the monitor. The monitor is + * peculiar in that it accesses the job list with job_get, and + * therefore needs consistency across job_get and the actual operation + * (e.g. job_user_cancel). To achieve this consistency, the caller + * calls job_lock/job_unlock itself around the whole operation. + * + * + * The second includes functions used by the job drivers and sometimes + * by the core block layer. These delegate the locking to the callee instead. + * + * TODO Actually make this true + */ + /* * job_mutex protects the jobs list, but also makes the * struct job fields thread-safe. -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Just as done with job.h, create _locked() functions in blockjob.h These functions will be later useful when caller has already taken the lock. All blockjob _locked functions call job _locked functions. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20220926093214.506243-8-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/blockjob.h | 18 ++++++++++++++ blockjob.c | 52 ++++++++++++++++++++++++++++++++-------- 2 files changed, 60 insertions(+), 10 deletions(-) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -XXX,XX +XXX,XX @@ typedef struct BlockJob { */ BlockJob *block_job_next(BlockJob *job); +/* Same as block_job_next(), but called with job lock held. */ +BlockJob *block_job_next_locked(BlockJob *job); + /** * block_job_get: * @id: The id of the block job. @@ -XXX,XX +XXX,XX @@ BlockJob *block_job_next(BlockJob *job); */ BlockJob *block_job_get(const char *id); +/* Same as block_job_get(), but called with job lock held. */ +BlockJob *block_job_get_locked(const char *id); + /** * block_job_add_bdrv: * @job: A block job @@ -XXX,XX +XXX,XX @@ bool block_job_has_bdrv(BlockJob *job, BlockDriverState *bs); */ bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp); +/* + * Same as block_job_set_speed(), but called with job lock held. + * Might release the lock temporarily. + */ +bool block_job_set_speed_locked(BlockJob *job, int64_t speed, Error **errp); + /** * block_job_query: * @job: The job to get information about. @@ -XXX,XX +XXX,XX @@ bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp); */ BlockJobInfo *block_job_query(BlockJob *job, Error **errp); +/* Same as block_job_query(), but called with job lock held. */ +BlockJobInfo *block_job_query_locked(BlockJob *job, Error **errp); + /** * block_job_iostatus_reset: * @job: The job whose I/O status should be reset. @@ -XXX,XX +XXX,XX @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp); */ void block_job_iostatus_reset(BlockJob *job); +/* Same as block_job_iostatus_reset(), but called with job lock held. */ +void block_job_iostatus_reset_locked(BlockJob *job); + /* * block_job_get_aio_context: * diff --git a/blockjob.c b/blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/blockjob.c +++ b/blockjob.c @@ -XXX,XX +XXX,XX @@ static bool is_block_job(Job *job) job_type(job) == JOB_TYPE_STREAM; } -BlockJob *block_job_next(BlockJob *bjob) +BlockJob *block_job_next_locked(BlockJob *bjob) { Job *job = bjob ? &bjob->job : NULL; GLOBAL_STATE_CODE(); do { - job = job_next(job); + job = job_next_locked(job); } while (job && !is_block_job(job)); return job ? container_of(job, BlockJob, job) : NULL; } -BlockJob *block_job_get(const char *id) +BlockJob *block_job_next(BlockJob *bjob) { - Job *job = job_get(id); + JOB_LOCK_GUARD(); + return block_job_next_locked(bjob); +} + +BlockJob *block_job_get_locked(const char *id) +{ + Job *job = job_get_locked(id); GLOBAL_STATE_CODE(); if (job && is_block_job(job)) { @@ -XXX,XX +XXX,XX @@ BlockJob *block_job_get(const char *id) } } +BlockJob *block_job_get(const char *id) +{ + JOB_LOCK_GUARD(); + return block_job_get_locked(id); +} + void block_job_free(Job *job) { BlockJob *bjob = container_of(job, BlockJob, job); @@ -XXX,XX +XXX,XX @@ static bool job_timer_pending(Job *job) return timer_pending(&job->sleep_timer); } -bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) +bool block_job_set_speed_locked(BlockJob *job, int64_t speed, Error **errp) { const BlockJobDriver *drv = block_job_driver(job); int64_t old_speed = job->speed; GLOBAL_STATE_CODE(); - if (job_apply_verb(&job->job, JOB_VERB_SET_SPEED, errp) < 0) { + if (job_apply_verb_locked(&job->job, JOB_VERB_SET_SPEED, errp) < 0) { return false; } if (speed < 0) { @@ -XXX,XX +XXX,XX @@ bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) job->speed = speed; if (drv->set_speed) { + job_unlock(); drv->set_speed(job, speed); + job_lock(); } if (speed && speed <= old_speed) { @@ -XXX,XX +XXX,XX @@ bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) } /* kick only if a timer is pending */ - job_enter_cond(&job->job, job_timer_pending); + job_enter_cond_locked(&job->job, job_timer_pending); return true; } +bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) +{ + JOB_LOCK_GUARD(); + return block_job_set_speed_locked(job, speed, errp); +} + int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n) { IO_CODE(); return ratelimit_calculate_delay(&job->limit, n); } -BlockJobInfo *block_job_query(BlockJob *job, Error **errp) +BlockJobInfo *block_job_query_locked(BlockJob *job, Error **errp) { BlockJobInfo *info; uint64_t progress_current, progress_total; @@ -XXX,XX +XXX,XX @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) info->len = progress_total; info->speed = job->speed; info->io_status = job->iostatus; - info->ready = job_is_ready(&job->job), + info->ready = job_is_ready_locked(&job->job), info->status = job->job.status; info->auto_finalize = job->job.auto_finalize; info->auto_dismiss = job->job.auto_dismiss; @@ -XXX,XX +XXX,XX @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) return info; } +BlockJobInfo *block_job_query(BlockJob *job, Error **errp) +{ + JOB_LOCK_GUARD(); + return block_job_query_locked(job, errp); +} + static void block_job_iostatus_set_err(BlockJob *job, int error) { if (job->iostatus == BLOCK_DEVICE_IO_STATUS_OK) { @@ -XXX,XX +XXX,XX @@ fail: return NULL; } -void block_job_iostatus_reset(BlockJob *job) +void block_job_iostatus_reset_locked(BlockJob *job) { GLOBAL_STATE_CODE(); if (job->iostatus == BLOCK_DEVICE_IO_STATUS_OK) { @@ -XXX,XX +XXX,XX @@ void block_job_iostatus_reset(BlockJob *job) job->iostatus = BLOCK_DEVICE_IO_STATUS_OK; } +void block_job_iostatus_reset(BlockJob *job) +{ + JOB_LOCK_GUARD(); + block_job_iostatus_reset_locked(job); +} + void block_job_user_resume(Job *job) { BlockJob *bjob = container_of(job, BlockJob, job); -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Both blockdev.c and job-qmp.c have TOC/TOU conditions, because they first search for the job and then perform an action on it. Therefore, we need to do the search + action under the same job mutex critical section. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20220926093214.506243-9-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- blockdev.c | 67 +++++++++++++++++++++++++++++++++++++----------------- job-qmp.c | 57 ++++++++++++++++++++++++++++++++-------------- 2 files changed, 86 insertions(+), 38 deletions(-) diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ out: aio_context_release(aio_context); } -/* Get a block job using its ID and acquire its AioContext */ -static BlockJob *find_block_job(const char *id, AioContext **aio_context, - Error **errp) +/* + * Get a block job using its ID and acquire its AioContext. + * Called with job_mutex held. + */ +static BlockJob *find_block_job_locked(const char *id, + AioContext **aio_context, + Error **errp) { BlockJob *job; @@ -XXX,XX +XXX,XX @@ static BlockJob *find_block_job(const char *id, AioContext **aio_context, *aio_context = NULL; - job = block_job_get(id); + job = block_job_get_locked(id); if (!job) { error_set(errp, ERROR_CLASS_DEVICE_NOT_ACTIVE, @@ -XXX,XX +XXX,XX @@ static BlockJob *find_block_job(const char *id, AioContext **aio_context, void qmp_block_job_set_speed(const char *device, int64_t speed, Error **errp) { AioContext *aio_context; - BlockJob *job = find_block_job(device, &aio_context, errp); + BlockJob *job; + + JOB_LOCK_GUARD(); + job = find_block_job_locked(device, &aio_context, errp); if (!job) { return; } - block_job_set_speed(job, speed, errp); + block_job_set_speed_locked(job, speed, errp); aio_context_release(aio_context); } @@ -XXX,XX +XXX,XX @@ void qmp_block_job_cancel(const char *device, bool has_force, bool force, Error **errp) { AioContext *aio_context; - BlockJob *job = find_block_job(device, &aio_context, errp); + BlockJob *job; + + JOB_LOCK_GUARD(); + job = find_block_job_locked(device, &aio_context, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_block_job_cancel(const char *device, force = false; } - if (job_user_paused(&job->job) && !force) { + if (job_user_paused_locked(&job->job) && !force) { error_setg(errp, "The block job for device '%s' is currently paused", device); goto out; } trace_qmp_block_job_cancel(job); - job_user_cancel(&job->job, force, errp); + job_user_cancel_locked(&job->job, force, errp); out: aio_context_release(aio_context); } @@ -XXX,XX +XXX,XX @@ out: void qmp_block_job_pause(const char *device, Error **errp) { AioContext *aio_context; - BlockJob *job = find_block_job(device, &aio_context, errp); + BlockJob *job; + + JOB_LOCK_GUARD(); + job = find_block_job_locked(device, &aio_context, errp); if (!job) { return; } trace_qmp_block_job_pause(job); - job_user_pause(&job->job, errp); + job_user_pause_locked(&job->job, errp); aio_context_release(aio_context); } void qmp_block_job_resume(const char *device, Error **errp) { AioContext *aio_context; - BlockJob *job = find_block_job(device, &aio_context, errp); + BlockJob *job; + + JOB_LOCK_GUARD(); + job = find_block_job_locked(device, &aio_context, errp); if (!job) { return; } trace_qmp_block_job_resume(job); - job_user_resume(&job->job, errp); + job_user_resume_locked(&job->job, errp); aio_context_release(aio_context); } void qmp_block_job_complete(const char *device, Error **errp) { AioContext *aio_context; - BlockJob *job = find_block_job(device, &aio_context, errp); + BlockJob *job; + + JOB_LOCK_GUARD(); + job = find_block_job_locked(device, &aio_context, errp); if (!job) { return; } trace_qmp_block_job_complete(job); - job_complete(&job->job, errp); + job_complete_locked(&job->job, errp); aio_context_release(aio_context); } void qmp_block_job_finalize(const char *id, Error **errp) { AioContext *aio_context; - BlockJob *job = find_block_job(id, &aio_context, errp); + BlockJob *job; + + JOB_LOCK_GUARD(); + job = find_block_job_locked(id, &aio_context, errp); if (!job) { return; } trace_qmp_block_job_finalize(job); - job_ref(&job->job); - job_finalize(&job->job, errp); + job_ref_locked(&job->job); + job_finalize_locked(&job->job, errp); /* * Job's context might have changed via job_finalize (and job_txn_apply @@ -XXX,XX +XXX,XX @@ void qmp_block_job_finalize(const char *id, Error **errp) * one. */ aio_context = block_job_get_aio_context(job); - job_unref(&job->job); + job_unref_locked(&job->job); aio_context_release(aio_context); } void qmp_block_job_dismiss(const char *id, Error **errp) { AioContext *aio_context; - BlockJob *bjob = find_block_job(id, &aio_context, errp); + BlockJob *bjob; Job *job; + JOB_LOCK_GUARD(); + bjob = find_block_job_locked(id, &aio_context, errp); + if (!bjob) { return; } trace_qmp_block_job_dismiss(bjob); job = &bjob->job; - job_dismiss(&job, errp); + job_dismiss_locked(&job, errp); aio_context_release(aio_context); } diff --git a/job-qmp.c b/job-qmp.c index XXXXXXX..XXXXXXX 100644 --- a/job-qmp.c +++ b/job-qmp.c @@ -XXX,XX +XXX,XX @@ #include "qapi/error.h" #include "trace/trace-root.h" -/* Get a job using its ID and acquire its AioContext */ -static Job *find_job(const char *id, AioContext **aio_context, Error **errp) +/* + * Get a job using its ID and acquire its AioContext. + * Called with job_mutex held. + */ +static Job *find_job_locked(const char *id, + AioContext **aio_context, + Error **errp) { Job *job; *aio_context = NULL; - job = job_get(id); + job = job_get_locked(id); if (!job) { error_setg(errp, "Job not found"); return NULL; @@ -XXX,XX +XXX,XX @@ static Job *find_job(const char *id, AioContext **aio_context, Error **errp) void qmp_job_cancel(const char *id, Error **errp) { AioContext *aio_context; - Job *job = find_job(id, &aio_context, errp); + Job *job; + + JOB_LOCK_GUARD(); + job = find_job_locked(id, &aio_context, errp); if (!job) { return; } trace_qmp_job_cancel(job); - job_user_cancel(job, true, errp); + job_user_cancel_locked(job, true, errp); aio_context_release(aio_context); } void qmp_job_pause(const char *id, Error **errp) { AioContext *aio_context; - Job *job = find_job(id, &aio_context, errp); + Job *job; + + JOB_LOCK_GUARD(); + job = find_job_locked(id, &aio_context, errp); if (!job) { return; } trace_qmp_job_pause(job); - job_user_pause(job, errp); + job_user_pause_locked(job, errp); aio_context_release(aio_context); } void qmp_job_resume(const char *id, Error **errp) { AioContext *aio_context; - Job *job = find_job(id, &aio_context, errp); + Job *job; + + JOB_LOCK_GUARD(); + job = find_job_locked(id, &aio_context, errp); if (!job) { return; } trace_qmp_job_resume(job); - job_user_resume(job, errp); + job_user_resume_locked(job, errp); aio_context_release(aio_context); } void qmp_job_complete(const char *id, Error **errp) { AioContext *aio_context; - Job *job = find_job(id, &aio_context, errp); + Job *job; + + JOB_LOCK_GUARD(); + job = find_job_locked(id, &aio_context, errp); if (!job) { return; } trace_qmp_job_complete(job); - job_complete(job, errp); + job_complete_locked(job, errp); aio_context_release(aio_context); } void qmp_job_finalize(const char *id, Error **errp) { AioContext *aio_context; - Job *job = find_job(id, &aio_context, errp); + Job *job; + + JOB_LOCK_GUARD(); + job = find_job_locked(id, &aio_context, errp); if (!job) { return; } trace_qmp_job_finalize(job); - job_ref(job); - job_finalize(job, errp); + job_ref_locked(job); + job_finalize_locked(job, errp); /* * Job's context might have changed via job_finalize (and job_txn_apply @@ -XXX,XX +XXX,XX @@ void qmp_job_finalize(const char *id, Error **errp) * one. */ aio_context = job->aio_context; - job_unref(job); + job_unref_locked(job); aio_context_release(aio_context); } void qmp_job_dismiss(const char *id, Error **errp) { AioContext *aio_context; - Job *job = find_job(id, &aio_context, errp); + Job *job; + + JOB_LOCK_GUARD(); + job = find_job_locked(id, &aio_context, errp); if (!job) { return; } trace_qmp_job_dismiss(job); - job_dismiss(&job, errp); + job_dismiss_locked(&job, errp); aio_context_release(aio_context); } -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Add missing job synchronization in the unit tests, with explicit locks. We are deliberately using _locked functions wrapped by a guard instead of a normal call because the normal call will be removed in future, as the only usage is limited to the tests. In other words, if a function like job_pause() is/will be only used in tests to avoid: WITH_JOB_LOCK_GUARD(){ job_pause_locked(); } then it is not worth keeping job_pause(), and just use the guard. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20220926093214.506243-10-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/unit/test-bdrv-drain.c | 76 ++++++++++++-------- tests/unit/test-block-iothread.c | 8 ++- tests/unit/test-blockjob-txn.c | 24 ++++--- tests/unit/test-blockjob.c | 115 +++++++++++++++++++------------ 4 files changed, 140 insertions(+), 83 deletions(-) diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-bdrv-drain.c +++ b/tests/unit/test-bdrv-drain.c @@ -XXX,XX +XXX,XX @@ static void test_blockjob_common_drain_node(enum drain_type drain_type, } } - g_assert_cmpint(job->job.pause_count, ==, 0); - g_assert_false(job->job.paused); - g_assert_true(tjob->running); - g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + WITH_JOB_LOCK_GUARD() { + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); + g_assert_true(tjob->running); + g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + } do_drain_begin_unlocked(drain_type, drain_bs); - if (drain_type == BDRV_DRAIN_ALL) { - /* bdrv_drain_all() drains both src and target */ - g_assert_cmpint(job->job.pause_count, ==, 2); - } else { - g_assert_cmpint(job->job.pause_count, ==, 1); + WITH_JOB_LOCK_GUARD() { + if (drain_type == BDRV_DRAIN_ALL) { + /* bdrv_drain_all() drains both src and target */ + g_assert_cmpint(job->job.pause_count, ==, 2); + } else { + g_assert_cmpint(job->job.pause_count, ==, 1); + } + g_assert_true(job->job.paused); + g_assert_false(job->job.busy); /* The job is paused */ } - g_assert_true(job->job.paused); - g_assert_false(job->job.busy); /* The job is paused */ do_drain_end_unlocked(drain_type, drain_bs); if (use_iothread) { - /* paused is reset in the I/O thread, wait for it */ + /* + * Here we are waiting for the paused status to change, + * so don't bother protecting the read every time. + * + * paused is reset in the I/O thread, wait for it + */ while (job->job.paused) { aio_poll(qemu_get_aio_context(), false); } } - g_assert_cmpint(job->job.pause_count, ==, 0); - g_assert_false(job->job.paused); - g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + WITH_JOB_LOCK_GUARD() { + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); + g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + } do_drain_begin_unlocked(drain_type, target); - if (drain_type == BDRV_DRAIN_ALL) { - /* bdrv_drain_all() drains both src and target */ - g_assert_cmpint(job->job.pause_count, ==, 2); - } else { - g_assert_cmpint(job->job.pause_count, ==, 1); + WITH_JOB_LOCK_GUARD() { + if (drain_type == BDRV_DRAIN_ALL) { + /* bdrv_drain_all() drains both src and target */ + g_assert_cmpint(job->job.pause_count, ==, 2); + } else { + g_assert_cmpint(job->job.pause_count, ==, 1); + } + g_assert_true(job->job.paused); + g_assert_false(job->job.busy); /* The job is paused */ } - g_assert_true(job->job.paused); - g_assert_false(job->job.busy); /* The job is paused */ do_drain_end_unlocked(drain_type, target); if (use_iothread) { - /* paused is reset in the I/O thread, wait for it */ + /* + * Here we are waiting for the paused status to change, + * so don't bother protecting the read every time. + * + * paused is reset in the I/O thread, wait for it + */ while (job->job.paused) { aio_poll(qemu_get_aio_context(), false); } } - g_assert_cmpint(job->job.pause_count, ==, 0); - g_assert_false(job->job.paused); - g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + WITH_JOB_LOCK_GUARD() { + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); + g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + } aio_context_acquire(ctx); - ret = job_complete_sync(&job->job, &error_abort); + WITH_JOB_LOCK_GUARD() { + ret = job_complete_sync_locked(&job->job, &error_abort); + } g_assert_cmpint(ret, ==, (result == TEST_JOB_SUCCESS ? 0 : -EIO)); if (use_iothread) { diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothread.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-block-iothread.c +++ b/tests/unit/test-block-iothread.c @@ -XXX,XX +XXX,XX @@ static void test_attach_blockjob(void) } aio_context_acquire(ctx); - job_complete_sync(&tjob->common.job, &error_abort); + WITH_JOB_LOCK_GUARD() { + job_complete_sync_locked(&tjob->common.job, &error_abort); + } blk_set_aio_context(blk, qemu_get_aio_context(), &error_abort); aio_context_release(ctx); @@ -XXX,XX +XXX,XX @@ static void test_propagate_mirror(void) BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, false, "filter_node", MIRROR_COPY_MODE_BACKGROUND, &error_abort); - job = job_get("job0"); + WITH_JOB_LOCK_GUARD() { + job = job_get_locked("job0"); + } filter = bdrv_find_node("filter_node"); /* Change the AioContext of src */ diff --git a/tests/unit/test-blockjob-txn.c b/tests/unit/test-blockjob-txn.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-blockjob-txn.c +++ b/tests/unit/test-blockjob-txn.c @@ -XXX,XX +XXX,XX @@ static void test_single_job(int expected) job = test_block_job_start(1, true, expected, &result, txn); job_start(&job->job); - if (expected == -ECANCELED) { - job_cancel(&job->job, false); + WITH_JOB_LOCK_GUARD() { + if (expected == -ECANCELED) { + job_cancel_locked(&job->job, false); + } } while (result == -EINPROGRESS) { @@ -XXX,XX +XXX,XX @@ static void test_pair_jobs(int expected1, int expected2) /* Release our reference now to trigger as many nice * use-after-free bugs as possible. */ - job_txn_unref(txn); + WITH_JOB_LOCK_GUARD() { + job_txn_unref_locked(txn); - if (expected1 == -ECANCELED) { - job_cancel(&job1->job, false); - } - if (expected2 == -ECANCELED) { - job_cancel(&job2->job, false); + if (expected1 == -ECANCELED) { + job_cancel_locked(&job1->job, false); + } + if (expected2 == -ECANCELED) { + job_cancel_locked(&job2->job, false); + } } while (result1 == -EINPROGRESS || result2 == -EINPROGRESS) { @@ -XXX,XX +XXX,XX @@ static void test_pair_jobs_fail_cancel_race(void) job_start(&job1->job); job_start(&job2->job); - job_cancel(&job1->job, false); + WITH_JOB_LOCK_GUARD() { + job_cancel_locked(&job1->job, false); + } /* Now make job2 finish before the main loop kicks jobs. This simulates * the race between a pending kick and another job completing. diff --git a/tests/unit/test-blockjob.c b/tests/unit/test-blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-blockjob.c +++ b/tests/unit/test-blockjob.c @@ -XXX,XX +XXX,XX @@ static CancelJob *create_common(Job **pjob) bjob = mk_job(blk, "Steve", &test_cancel_driver, true, JOB_MANUAL_FINALIZE | JOB_MANUAL_DISMISS); job = &bjob->job; - job_ref(job); - assert(job->status == JOB_STATUS_CREATED); + WITH_JOB_LOCK_GUARD() { + job_ref_locked(job); + assert(job->status == JOB_STATUS_CREATED); + } + s = container_of(bjob, CancelJob, common); s->blk = blk; @@ -XXX,XX +XXX,XX @@ static void cancel_common(CancelJob *s) aio_context_acquire(ctx); job_cancel_sync(&job->job, true); - if (sts != JOB_STATUS_CREATED && sts != JOB_STATUS_CONCLUDED) { - Job *dummy = &job->job; - job_dismiss(&dummy, &error_abort); + WITH_JOB_LOCK_GUARD() { + if (sts != JOB_STATUS_CREATED && sts != JOB_STATUS_CONCLUDED) { + Job *dummy = &job->job; + job_dismiss_locked(&dummy, &error_abort); + } + assert(job->job.status == JOB_STATUS_NULL); + job_unref_locked(&job->job); } - assert(job->job.status == JOB_STATUS_NULL); - job_unref(&job->job); destroy_blk(blk); aio_context_release(ctx); @@ -XXX,XX +XXX,XX @@ static void test_cancel_created(void) cancel_common(s); } +static void assert_job_status_is(Job *job, int status) +{ + WITH_JOB_LOCK_GUARD() { + assert(job->status == status); + } +} + static void test_cancel_running(void) { Job *job; @@ -XXX,XX +XXX,XX @@ static void test_cancel_running(void) s = create_common(&job); job_start(job); - assert(job->status == JOB_STATUS_RUNNING); + assert_job_status_is(job, JOB_STATUS_RUNNING); cancel_common(s); } @@ -XXX,XX +XXX,XX @@ static void test_cancel_paused(void) s = create_common(&job); job_start(job); - assert(job->status == JOB_STATUS_RUNNING); - - job_user_pause(job, &error_abort); + WITH_JOB_LOCK_GUARD() { + assert(job->status == JOB_STATUS_RUNNING); + job_user_pause_locked(job, &error_abort); + } job_enter(job); - assert(job->status == JOB_STATUS_PAUSED); + assert_job_status_is(job, JOB_STATUS_PAUSED); cancel_common(s); } @@ -XXX,XX +XXX,XX @@ static void test_cancel_ready(void) s = create_common(&job); job_start(job); - assert(job->status == JOB_STATUS_RUNNING); + assert_job_status_is(job, JOB_STATUS_RUNNING); s->should_converge = true; job_enter(job); - assert(job->status == JOB_STATUS_READY); + assert_job_status_is(job, JOB_STATUS_READY); cancel_common(s); } @@ -XXX,XX +XXX,XX @@ static void test_cancel_standby(void) s = create_common(&job); job_start(job); - assert(job->status == JOB_STATUS_RUNNING); + assert_job_status_is(job, JOB_STATUS_RUNNING); s->should_converge = true; job_enter(job); - assert(job->status == JOB_STATUS_READY); - - job_user_pause(job, &error_abort); + WITH_JOB_LOCK_GUARD() { + assert(job->status == JOB_STATUS_READY); + job_user_pause_locked(job, &error_abort); + } job_enter(job); - assert(job->status == JOB_STATUS_STANDBY); + assert_job_status_is(job, JOB_STATUS_STANDBY); cancel_common(s); } @@ -XXX,XX +XXX,XX @@ static void test_cancel_pending(void) s = create_common(&job); job_start(job); - assert(job->status == JOB_STATUS_RUNNING); + assert_job_status_is(job, JOB_STATUS_RUNNING); s->should_converge = true; job_enter(job); - assert(job->status == JOB_STATUS_READY); - - job_complete(job, &error_abort); + WITH_JOB_LOCK_GUARD() { + assert(job->status == JOB_STATUS_READY); + job_complete_locked(job, &error_abort); + } job_enter(job); while (!job->deferred_to_main_loop) { aio_poll(qemu_get_aio_context(), true); } - assert(job->status == JOB_STATUS_READY); + assert_job_status_is(job, JOB_STATUS_READY); aio_poll(qemu_get_aio_context(), true); - assert(job->status == JOB_STATUS_PENDING); + assert_job_status_is(job, JOB_STATUS_PENDING); cancel_common(s); } @@ -XXX,XX +XXX,XX @@ static void test_cancel_concluded(void) s = create_common(&job); job_start(job); - assert(job->status == JOB_STATUS_RUNNING); + assert_job_status_is(job, JOB_STATUS_RUNNING); s->should_converge = true; job_enter(job); - assert(job->status == JOB_STATUS_READY); - - job_complete(job, &error_abort); + WITH_JOB_LOCK_GUARD() { + assert(job->status == JOB_STATUS_READY); + job_complete_locked(job, &error_abort); + } job_enter(job); while (!job->deferred_to_main_loop) { aio_poll(qemu_get_aio_context(), true); } - assert(job->status == JOB_STATUS_READY); + assert_job_status_is(job, JOB_STATUS_READY); aio_poll(qemu_get_aio_context(), true); - assert(job->status == JOB_STATUS_PENDING); + assert_job_status_is(job, JOB_STATUS_PENDING); aio_context_acquire(job->aio_context); - job_finalize(job, &error_abort); + WITH_JOB_LOCK_GUARD() { + job_finalize_locked(job, &error_abort); + } aio_context_release(job->aio_context); - assert(job->status == JOB_STATUS_CONCLUDED); + assert_job_status_is(job, JOB_STATUS_CONCLUDED); cancel_common(s); } @@ -XXX,XX +XXX,XX @@ static void test_complete_in_standby(void) bjob = mk_job(blk, "job", &test_yielding_driver, true, JOB_MANUAL_FINALIZE | JOB_MANUAL_DISMISS); job = &bjob->job; - assert(job->status == JOB_STATUS_CREATED); + assert_job_status_is(job, JOB_STATUS_CREATED); /* Wait for the job to become READY */ job_start(job); aio_context_acquire(ctx); + /* + * Here we are waiting for the status to change, so don't bother + * protecting the read every time. + */ AIO_WAIT_WHILE(ctx, job->status != JOB_STATUS_READY); aio_context_release(ctx); /* Begin the drained section, pausing the job */ bdrv_drain_all_begin(); - assert(job->status == JOB_STATUS_STANDBY); + assert_job_status_is(job, JOB_STATUS_STANDBY); + /* Lock the IO thread to prevent the job from being run */ aio_context_acquire(ctx); /* This will schedule the job to resume it */ bdrv_drain_all_end(); - /* But the job cannot run, so it will remain on standby */ - assert(job->status == JOB_STATUS_STANDBY); + WITH_JOB_LOCK_GUARD() { + /* But the job cannot run, so it will remain on standby */ + assert(job->status == JOB_STATUS_STANDBY); - /* Even though the job is on standby, this should work */ - job_complete(job, &error_abort); + /* Even though the job is on standby, this should work */ + job_complete_locked(job, &error_abort); - /* The test is done now, clean up. */ - job_finish_sync(job, NULL, &error_abort); - assert(job->status == JOB_STATUS_PENDING); + /* The test is done now, clean up. */ + job_finish_sync_locked(job, NULL, &error_abort); + assert(job->status == JOB_STATUS_PENDING); - job_finalize(job, &error_abort); - assert(job->status == JOB_STATUS_CONCLUDED); + job_finalize_locked(job, &error_abort); + assert(job->status == JOB_STATUS_CONCLUDED); - job_dismiss(&job, &error_abort); + job_dismiss_locked(&job, &error_abort); + } destroy_blk(blk); aio_context_release(ctx); -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Once job lock is used and aiocontext is removed, mirror has to perform job operations under the same critical section, Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-Id: <20220926093214.506243-11-eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/mirror.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static void mirror_complete(Job *job, Error **errp) s->should_complete = true; /* If the job is paused, it will be re-entered when it is resumed */ - if (!job->paused) { - job_enter(job); + WITH_JOB_LOCK_GUARD() { + if (!job->paused) { + job_enter_cond_locked(job, NULL); + } } } @@ -XXX,XX +XXX,XX @@ static bool mirror_drained_poll(BlockJob *job) * from one of our own drain sections, to avoid a deadlock waiting for * ourselves. */ - if (!s->common.job.paused && !job_is_cancelled(&job->job) && !s->in_drain) { - return true; + WITH_JOB_LOCK_GUARD() { + if (!s->common.job.paused && !job_is_cancelled_locked(&job->job) + && !s->in_drain) { + return true; + } } return !!s->in_flight; -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Now that the API offers also _locked() functions, take advantage of it and give also the caller control to take the lock and call _locked functions. This makes sense especially when we have for loops, because it makes no sense to have: for(job = job_next(); ...) where each job_next() takes the lock internally. Instead we want JOB_LOCK_GUARD(); for(job = job_next_locked(); ...) In addition, protect also direct field accesses, by either creating a new critical section or widening the existing ones. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-Id: <20220926093214.506243-12-eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block.c | 17 ++++++++++------- blockdev.c | 14 ++++++++++---- blockjob.c | 35 ++++++++++++++++++++++------------- job-qmp.c | 9 ++++++--- monitor/qmp-cmds.c | 7 +++++-- qemu-img.c | 15 ++++++++++----- 6 files changed, 63 insertions(+), 34 deletions(-) diff --git a/block.c b/block.c index XXXXXXX..XXXXXXX 100644 --- a/block.c +++ b/block.c @@ -XXX,XX +XXX,XX @@ static void bdrv_close(BlockDriverState *bs) void bdrv_close_all(void) { - assert(job_next(NULL) == NULL); GLOBAL_STATE_CODE(); + assert(job_next(NULL) == NULL); /* Drop references from requests still in flight, such as canceled block * jobs whose AIO context has not been polled yet */ @@ -XXX,XX +XXX,XX @@ XDbgBlockGraph *bdrv_get_xdbg_block_graph(Error **errp) } } - for (job = block_job_next(NULL); job; job = block_job_next(job)) { - GSList *el; + WITH_JOB_LOCK_GUARD() { + for (job = block_job_next_locked(NULL); job; + job = block_job_next_locked(job)) { + GSList *el; - xdbg_graph_add_node(gr, job, X_DBG_BLOCK_GRAPH_NODE_TYPE_BLOCK_JOB, - job->job.id); - for (el = job->nodes; el; el = el->next) { - xdbg_graph_add_edge(gr, job, (BdrvChild *)el->data); + xdbg_graph_add_node(gr, job, X_DBG_BLOCK_GRAPH_NODE_TYPE_BLOCK_JOB, + job->job.id); + for (el = job->nodes; el; el = el->next) { + xdbg_graph_add_edge(gr, job, (BdrvChild *)el->data); + } } } diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ void blockdev_mark_auto_del(BlockBackend *blk) return; } - for (job = block_job_next(NULL); job; job = block_job_next(job)) { + JOB_LOCK_GUARD(); + + for (job = block_job_next_locked(NULL); job; + job = block_job_next_locked(job)) { if (block_job_has_bdrv(job, blk_bs(blk))) { AioContext *aio_context = job->job.aio_context; aio_context_acquire(aio_context); - job_cancel(&job->job, false); + job_cancel_locked(&job->job, false); aio_context_release(aio_context); } @@ -XXX,XX +XXX,XX @@ BlockJobInfoList *qmp_query_block_jobs(Error **errp) BlockJobInfoList *head = NULL, **tail = &head; BlockJob *job; - for (job = block_job_next(NULL); job; job = block_job_next(job)) { + JOB_LOCK_GUARD(); + + for (job = block_job_next_locked(NULL); job; + job = block_job_next_locked(job)) { BlockJobInfo *value; AioContext *aio_context; @@ -XXX,XX +XXX,XX @@ BlockJobInfoList *qmp_query_block_jobs(Error **errp) } aio_context = block_job_get_aio_context(job); aio_context_acquire(aio_context); - value = block_job_query(job, errp); + value = block_job_query_locked(job, errp); aio_context_release(aio_context); if (!value) { qapi_free_BlockJobInfoList(head); diff --git a/blockjob.c b/blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/blockjob.c +++ b/blockjob.c @@ -XXX,XX +XXX,XX @@ static bool child_job_drained_poll(BdrvChild *c) /* An inactive or completed job doesn't have any pending requests. Jobs * with !job->busy are either already paused or have a pause point after * being reentered, so no job driver code will run before they pause. */ - if (!job->busy || job_is_completed(job)) { - return false; + WITH_JOB_LOCK_GUARD() { + if (!job->busy || job_is_completed_locked(job)) { + return false; + } } /* Otherwise, assume that it isn't fully stopped yet, but allow the job to @@ -XXX,XX +XXX,XX @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->ready_notifier.notify = block_job_event_ready; job->idle_notifier.notify = block_job_on_idle; - notifier_list_add(&job->job.on_finalize_cancelled, - &job->finalize_cancelled_notifier); - notifier_list_add(&job->job.on_finalize_completed, - &job->finalize_completed_notifier); - notifier_list_add(&job->job.on_pending, &job->pending_notifier); - notifier_list_add(&job->job.on_ready, &job->ready_notifier); - notifier_list_add(&job->job.on_idle, &job->idle_notifier); + WITH_JOB_LOCK_GUARD() { + notifier_list_add(&job->job.on_finalize_cancelled, + &job->finalize_cancelled_notifier); + notifier_list_add(&job->job.on_finalize_completed, + &job->finalize_completed_notifier); + notifier_list_add(&job->job.on_pending, &job->pending_notifier); + notifier_list_add(&job->job.on_ready, &job->ready_notifier); + notifier_list_add(&job->job.on_idle, &job->idle_notifier); + } error_setg(&job->blocker, "block device is in use by block job: %s", job_type_str(&job->job)); @@ -XXX,XX +XXX,XX @@ BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, action); } if (action == BLOCK_ERROR_ACTION_STOP) { - if (!job->job.user_paused) { - job_pause(&job->job); - /* make the pause user visible, which will be resumed from QMP. */ - job->job.user_paused = true; + WITH_JOB_LOCK_GUARD() { + if (!job->job.user_paused) { + job_pause_locked(&job->job); + /* + * make the pause user visible, which will be + * resumed from QMP. + */ + job->job.user_paused = true; + } } block_job_iostatus_set_err(job, error); } diff --git a/job-qmp.c b/job-qmp.c index XXXXXXX..XXXXXXX 100644 --- a/job-qmp.c +++ b/job-qmp.c @@ -XXX,XX +XXX,XX @@ void qmp_job_dismiss(const char *id, Error **errp) aio_context_release(aio_context); } -static JobInfo *job_query_single(Job *job, Error **errp) +/* Called with job_mutex held. */ +static JobInfo *job_query_single_locked(Job *job, Error **errp) { JobInfo *info; uint64_t progress_current; @@ -XXX,XX +XXX,XX @@ JobInfoList *qmp_query_jobs(Error **errp) JobInfoList *head = NULL, **tail = &head; Job *job; - for (job = job_next(NULL); job; job = job_next(job)) { + JOB_LOCK_GUARD(); + + for (job = job_next_locked(NULL); job; job = job_next_locked(job)) { JobInfo *value; AioContext *aio_context; @@ -XXX,XX +XXX,XX @@ JobInfoList *qmp_query_jobs(Error **errp) } aio_context = job->aio_context; aio_context_acquire(aio_context); - value = job_query_single(job, errp); + value = job_query_single_locked(job, errp); aio_context_release(aio_context); if (!value) { qapi_free_JobInfoList(head); diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c index XXXXXXX..XXXXXXX 100644 --- a/monitor/qmp-cmds.c +++ b/monitor/qmp-cmds.c @@ -XXX,XX +XXX,XX @@ void qmp_cont(Error **errp) blk_iostatus_reset(blk); } - for (job = block_job_next(NULL); job; job = block_job_next(job)) { - block_job_iostatus_reset(job); + WITH_JOB_LOCK_GUARD() { + for (job = block_job_next_locked(NULL); job; + job = block_job_next_locked(job)) { + block_job_iostatus_reset_locked(job); + } } /* Continuing after completed migration. Images have been inactivated to diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static void run_block_job(BlockJob *job, Error **errp) int ret = 0; aio_context_acquire(aio_context); - job_ref(&job->job); + job_lock(); + job_ref_locked(&job->job); do { float progress = 0.0f; + job_unlock(); aio_poll(aio_context, true); progress_get_snapshot(&job->job.progress, &progress_current, @@ -XXX,XX +XXX,XX @@ static void run_block_job(BlockJob *job, Error **errp) progress = (float)progress_current / progress_total * 100.f; } qemu_progress_print(progress, 0); - } while (!job_is_ready(&job->job) && !job_is_completed(&job->job)); + job_lock(); + } while (!job_is_ready_locked(&job->job) && + !job_is_completed_locked(&job->job)); - if (!job_is_completed(&job->job)) { - ret = job_complete_sync(&job->job, errp); + if (!job_is_completed_locked(&job->job)) { + ret = job_complete_sync_locked(&job->job, errp); } else { ret = job->job.ret; } - job_unref(&job->job); + job_unref_locked(&job->job); + job_unlock(); aio_context_release(aio_context); /* publish completion progress only when success */ -- 2.37.3 From: Paolo Bonzini <pbonzini@redhat.com> We want to make sure access of job->aio_context is always done under either BQL or job_mutex. The problem is that using aio_co_enter(job->aiocontext, job->co) in job_start and job_enter_cond makes the coroutine immediately resume, so we can't hold the job lock. And caching it is not safe either, as it might change. job_start is under BQL, so it can freely read job->aiocontext, but job_enter_cond is not. We want to avoid reading job->aio_context in job_enter_cond, therefore: 1) use aio_co_wake(), since it doesn't want an aiocontext as argument but uses job->co->ctx 2) detect possible discrepancy between job->co->ctx and job->aio_context by checking right after the coroutine resumes back from yielding if job->aio_context has changed. If so, reschedule the coroutine to the new context. Calling bdrv_try_set_aio_context() will issue the following calls (simplified): * in terms of bdrv callbacks: .drained_begin -> .set_aio_context -> .drained_end * in terms of child_job functions: child_job_drained_begin -> child_job_set_aio_context -> child_job_drained_end * in terms of job functions: job_pause_locked -> job_set_aio_context -> job_resume_locked We can see that after setting the new aio_context, job_resume_locked calls again job_enter_cond, which then invokes aio_co_wake(). But while job->aiocontext has been set in job_set_aio_context, job->co->ctx has not changed, so the coroutine would be entering in the wrong aiocontext. Using aio_co_schedule in job_resume_locked() might seem as a valid alternative, but the problem is that the bh resuming the coroutine is not scheduled immediately, and if in the meanwhile another bdrv_try_set_aio_context() is run (see test_propagate_mirror() in test-block-iothread.c), we would have the first schedule in the wrong aiocontext, and the second set of drains won't even manage to schedule the coroutine, as job->busy would still be true from the previous job_resume_locked(). The solution is to stick with aio_co_wake() and detect every time the coroutine resumes back from yielding if job->aio_context has changed. If so, we can reschedule it to the new context. Check for the aiocontext change in job_do_yield_locked because: 1) aio_co_reschedule_self requires to be in the running coroutine 2) since child_job_set_aio_context allows changing the aiocontext only while the job is paused, this is the exact place where the coroutine resumes, before running JobDriver's code. Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220926093214.506243-13-eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- job.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/job.c b/job.c index XXXXXXX..XXXXXXX 100644 --- a/job.c +++ b/job.c @@ -XXX,XX +XXX,XX @@ void job_enter_cond_locked(Job *job, bool(*fn)(Job *job)) job->busy = true; real_job_unlock(); job_unlock(); - aio_co_enter(job->aio_context, job->co); + aio_co_wake(job->co); job_lock(); } @@ -XXX,XX +XXX,XX @@ void job_enter(Job *job) */ static void coroutine_fn job_do_yield_locked(Job *job, uint64_t ns) { + AioContext *next_aio_context; + real_job_lock(); if (ns != -1) { timer_mod(&job->sleep_timer, ns); @@ -XXX,XX +XXX,XX @@ static void coroutine_fn job_do_yield_locked(Job *job, uint64_t ns) qemu_coroutine_yield(); job_lock(); - /* Set by job_enter_cond() before re-entering the coroutine. */ + next_aio_context = job->aio_context; + /* + * Coroutine has resumed, but in the meanwhile the job AioContext + * might have changed via bdrv_try_set_aio_context(), so we need to move + * the coroutine too in the new aiocontext. + */ + while (qemu_get_current_aio_context() != next_aio_context) { + job_unlock(); + aio_co_reschedule_self(next_aio_context); + job_lock(); + next_aio_context = job->aio_context; + } + + /* Set by job_enter_cond_locked() before re-entering the coroutine. */ assert(job->busy); } -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> In order to make it thread safe, implement a "fake rwlock", where we allow reads under BQL *or* job_mutex held, but writes only under BQL *and* job_mutex. The only write we have is in child_job_set_aio_ctx, which always happens under drain (so the job is paused). For this reason, introduce job_set_aio_context and make sure that the context is set under BQL, job_mutex and drain. Also make sure all other places where the aiocontext is read are protected. The reads in commit.c and mirror.c are actually safe, because always done under BQL. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Suggested-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Message-Id: <20220926093214.506243-14-eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 23 ++++++++++++++++++++--- block/replication.c | 1 + blockjob.c | 3 ++- job.c | 12 ++++++++++++ 4 files changed, 35 insertions(+), 4 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -XXX,XX +XXX,XX @@ typedef struct Job { /* ProgressMeter API is thread-safe */ ProgressMeter progress; + /** + * AioContext to run the job coroutine in. + * The job Aiocontext can be read when holding *either* + * the BQL (so we are in the main loop) or the job_mutex. + * It can only be written when we hold *both* BQL + * and the job_mutex. + */ + AioContext *aio_context; - /** Protected by AioContext lock */ - /** AioContext to run the job coroutine in */ - AioContext *aio_context; + /** Protected by AioContext lock */ /** Reference count of the block job */ int refcnt; @@ -XXX,XX +XXX,XX @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), int job_finish_sync_locked(Job *job, void (*finish)(Job *, Error **errp), Error **errp); +/** + * Sets the @job->aio_context. + * Called with job_mutex *not* held. + * + * This function must run in the main thread to protect against + * concurrent read in job_finish_sync_locked(), takes the job_mutex + * lock to protect against the read in job_do_yield_locked(), and must + * be called when the job is quiescent. + */ +void job_set_aio_context(Job *job, AioContext *ctx); + #endif diff --git a/block/replication.c b/block/replication.c index XXXXXXX..XXXXXXX 100644 --- a/block/replication.c +++ b/block/replication.c @@ -XXX,XX +XXX,XX @@ static void replication_close(BlockDriverState *bs) { BDRVReplicationState *s = bs->opaque; Job *commit_job; + GLOBAL_STATE_CODE(); if (s->stage == BLOCK_REPLICATION_RUNNING) { replication_stop(s->rs, false, NULL); diff --git a/blockjob.c b/blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/blockjob.c +++ b/blockjob.c @@ -XXX,XX +XXX,XX @@ static void child_job_set_aio_ctx(BdrvChild *c, AioContext *ctx, bdrv_set_aio_context_ignore(sibling->bs, ctx, ignore); } - job->job.aio_context = ctx; + job_set_aio_context(&job->job, ctx); } static AioContext *child_job_get_parent_aio_context(BdrvChild *c) { BlockJob *job = c->opaque; + GLOBAL_STATE_CODE(); return job->job.aio_context; } diff --git a/job.c b/job.c index XXXXXXX..XXXXXXX 100644 --- a/job.c +++ b/job.c @@ -XXX,XX +XXX,XX @@ Job *job_get(const char *id) return job_get_locked(id); } +void job_set_aio_context(Job *job, AioContext *ctx) +{ + /* protect against read in job_finish_sync_locked and job_start */ + GLOBAL_STATE_CODE(); + /* protect against read in job_do_yield_locked */ + JOB_LOCK_GUARD(); + /* ensure the job is quiescent while the AioContext is changed */ + assert(job->paused || job_is_completed_locked(job)); + job->aio_context = ctx; +} + /* Called with job_mutex *not* held. */ static void job_sleep_timer_cb(void *opaque) { @@ -XXX,XX +XXX,XX @@ int job_finish_sync_locked(Job *job, { Error *local_err = NULL; int ret; + GLOBAL_STATE_CODE(); job_ref_locked(job); -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> The same job lock is being used also to protect some of blockjob fields. Categorize them just as done in job.h. Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Message-Id: <20220926093214.506243-15-eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/blockjob.h | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -XXX,XX +XXX,XX @@ typedef struct BlockJobDriver BlockJobDriver; * Long-running operation on a BlockDriverState. */ typedef struct BlockJob { - /** Data belonging to the generic Job infrastructure */ + /** + * Data belonging to the generic Job infrastructure. + * Protected by job mutex. + */ Job job; - /** Status that is published by the query-block-jobs QMP API */ + /** + * Status that is published by the query-block-jobs QMP API. + * Protected by job mutex. + */ BlockDeviceIoStatus iostatus; - /** Speed that was set with @block_job_set_speed. */ + /** + * Speed that was set with @block_job_set_speed. + * Always modified and read under QEMU global mutex (GLOBAL_STATE_CODE). + */ int64_t speed; - /** Rate limiting data structure for implementing @speed. */ + /** + * Rate limiting data structure for implementing @speed. + * RateLimit API is thread-safe. + */ RateLimit limit; - /** Block other operations when block job is running */ + /** + * Block other operations when block job is running. + * Always modified and read under QEMU global mutex (GLOBAL_STATE_CODE). + */ Error *blocker; + /** All notifiers are set once in block_job_create() and never modified. */ + /** Called when a cancelled job is finalised. */ Notifier finalize_cancelled_notifier; @@ -XXX,XX +XXX,XX @@ typedef struct BlockJob { /** Called when the job coroutine yields or terminates */ Notifier idle_notifier; - /** BlockDriverStates that are involved in this block job */ + /** + * BlockDriverStates that are involved in this block job. + * Always modified and read under QEMU global mutex (GLOBAL_STATE_CODE). + */ GSList *nodes; } BlockJob; -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> They all are called with job_lock held, in job_event_*_locked() Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20220926093214.506243-16-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- blockjob.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/blockjob.c b/blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/blockjob.c +++ b/blockjob.c @@ -XXX,XX +XXX,XX @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, return 0; } -static void block_job_on_idle(Notifier *n, void *opaque) +/* Called with job_mutex lock held. */ +static void block_job_on_idle_locked(Notifier *n, void *opaque) { aio_wait_kick(); } @@ -XXX,XX +XXX,XX @@ static void block_job_iostatus_set_err(BlockJob *job, int error) } } -static void block_job_event_cancelled(Notifier *n, void *opaque) +/* Called with job_mutex lock held. */ +static void block_job_event_cancelled_locked(Notifier *n, void *opaque) { BlockJob *job = opaque; uint64_t progress_current, progress_total; @@ -XXX,XX +XXX,XX @@ static void block_job_event_cancelled(Notifier *n, void *opaque) job->speed); } -static void block_job_event_completed(Notifier *n, void *opaque) +/* Called with job_mutex lock held. */ +static void block_job_event_completed_locked(Notifier *n, void *opaque) { BlockJob *job = opaque; const char *msg = NULL; @@ -XXX,XX +XXX,XX @@ static void block_job_event_completed(Notifier *n, void *opaque) msg); } -static void block_job_event_pending(Notifier *n, void *opaque) +/* Called with job_mutex lock held. */ +static void block_job_event_pending_locked(Notifier *n, void *opaque) { BlockJob *job = opaque; @@ -XXX,XX +XXX,XX @@ static void block_job_event_pending(Notifier *n, void *opaque) job->job.id); } -static void block_job_event_ready(Notifier *n, void *opaque) +/* Called with job_mutex lock held. */ +static void block_job_event_ready_locked(Notifier *n, void *opaque) { BlockJob *job = opaque; uint64_t progress_current, progress_total; @@ -XXX,XX +XXX,XX @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, ratelimit_init(&job->limit); - job->finalize_cancelled_notifier.notify = block_job_event_cancelled; - job->finalize_completed_notifier.notify = block_job_event_completed; - job->pending_notifier.notify = block_job_event_pending; - job->ready_notifier.notify = block_job_event_ready; - job->idle_notifier.notify = block_job_on_idle; + job->finalize_cancelled_notifier.notify = block_job_event_cancelled_locked; + job->finalize_completed_notifier.notify = block_job_event_completed_locked; + job->pending_notifier.notify = block_job_event_pending_locked; + job->ready_notifier.notify = block_job_event_ready_locked; + job->idle_notifier.notify = block_job_on_idle_locked; WITH_JOB_LOCK_GUARD() { notifier_list_add(&job->job.on_finalize_cancelled, -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> iostatus is the only field (together with .job) that needs protection using the job mutex. It is set in the main loop (GLOBAL_STATE functions) but read in I/O code (block_job_error_action). In order to protect it, change block_job_iostatus_set_err to block_job_iostatus_set_err_locked(), always called under job lock. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-Id: <20220926093214.506243-17-eesposit@redhat.com> [kwolf: Fixed up type of iostatus] Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/mirror.c | 6 +++++- blockjob.c | 5 +++-- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index XXXXXXX..XXXXXXX 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -XXX,XX +XXX,XX @@ static int coroutine_fn mirror_run(Job *job, Error **errp) BlockDriverState *bs = s->mirror_top_bs->backing->bs; BlockDriverState *target_bs = blk_bs(s->target); bool need_drain = true; + BlockDeviceIoStatus iostatus; int64_t length; int64_t target_length; BlockDriverInfo bdi; @@ -XXX,XX +XXX,XX @@ static int coroutine_fn mirror_run(Job *job, Error **errp) * We do so every BLKOCK_JOB_SLICE_TIME nanoseconds, or when there is * an error, or when the source is clean, whichever comes first. */ delta = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - s->last_pause_ns; + WITH_JOB_LOCK_GUARD() { + iostatus = s->common.iostatus; + } if (delta < BLOCK_JOB_SLICE_TIME && - s->common.iostatus == BLOCK_DEVICE_IO_STATUS_OK) { + iostatus == BLOCK_DEVICE_IO_STATUS_OK) { if (s->in_flight >= MAX_IN_FLIGHT || s->buf_free_count == 0 || (cnt == 0 && s->in_flight > 0)) { trace_mirror_yield(s, cnt, s->buf_free_count, s->in_flight); diff --git a/blockjob.c b/blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/blockjob.c +++ b/blockjob.c @@ -XXX,XX +XXX,XX @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) return block_job_query_locked(job, errp); } -static void block_job_iostatus_set_err(BlockJob *job, int error) +/* Called with job lock held */ +static void block_job_iostatus_set_err_locked(BlockJob *job, int error) { if (job->iostatus == BLOCK_DEVICE_IO_STATUS_OK) { job->iostatus = error == ENOSPC ? BLOCK_DEVICE_IO_STATUS_NOSPACE : @@ -XXX,XX +XXX,XX @@ BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, */ job->job.user_paused = true; } + block_job_iostatus_set_err_locked(job, error); } - block_job_iostatus_set_err(job, error); } return action; } -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Some callbacks implementation use bdrv_* APIs that assume the AioContext lock is held. Make sure this invariant is documented. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-Id: <20220926093214.506243-18-eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -XXX,XX +XXX,XX @@ typedef struct Job { /** True if this job should automatically dismiss itself */ bool auto_dismiss; - /** The completion function that will be called when the job completes. */ + /** + * The completion function that will be called when the job completes. + * Called with AioContext lock held, since many callback implementations + * use bdrv_* functions that require to hold the lock. + */ BlockCompletionFunc *cb; /** The opaque value that is passed to the completion function. */ @@ -XXX,XX +XXX,XX @@ struct JobDriver { * * This callback will not be invoked if the job has already failed. * If it fails, abort and then clean will be called. + * + * Called with AioContext lock held, since many callbacs implementations + * use bdrv_* functions that require to hold the lock. */ int (*prepare)(Job *job); @@ -XXX,XX +XXX,XX @@ struct JobDriver { * * All jobs will complete with a call to either .commit() or .abort() but * never both. + * + * Called with AioContext lock held, since many callback implementations + * use bdrv_* functions that require to hold the lock. */ void (*commit)(Job *job); @@ -XXX,XX +XXX,XX @@ struct JobDriver { * * All jobs will complete with a call to either .commit() or .abort() but * never both. + * + * Called with AioContext lock held, since many callback implementations + * use bdrv_* functions that require to hold the lock. */ void (*abort)(Job *job); @@ -XXX,XX +XXX,XX @@ struct JobDriver { * .commit() or .abort(). Regardless of which callback is invoked after * completion, .clean() will always be called, even if the job does not * belong to a transaction group. + * + * Called with AioContext lock held, since many callbacs implementations + * use bdrv_* functions that require to hold the lock. */ void (*clean)(Job *job); @@ -XXX,XX +XXX,XX @@ struct JobDriver { * READY). * (If the callback is NULL, the job is assumed to terminate * without I/O.) + * + * Called with AioContext lock held, since many callback implementations + * use bdrv_* functions that require to hold the lock. */ bool (*cancel)(Job *job, bool force); - /** Called when the job is freed */ + /** + * Called when the job is freed. + * Called with AioContext lock held, since many callback implementations + * use bdrv_* functions that require to hold the lock. + */ void (*free)(Job *job); }; -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Change the job_{lock/unlock} and macros to use job_mutex. Now that they are not nop anymore, remove the aiocontext to avoid deadlocks. Therefore: - when possible, remove completely the aiocontext lock/unlock pair - if it is used by some other function too, reduce the locking section as much as possible, leaving the job API outside. - change AIO_WAIT_WHILE in AIO_WAIT_WHILE_UNLOCKED, since we are not using the aiocontext lock anymore The only functions that still need the aiocontext lock are: - the JobDriver callbacks, already documented in job.h - job_cancel_sync() in replication.c is called with aio_context_lock taken, but now job is using AIO_WAIT_WHILE_UNLOCKED so we need to release the lock. Reduce the locking section to only cover the callback invocation and document the functions that take the AioContext lock, to avoid taking it twice. Also remove real_job_{lock/unlock}, as they are replaced by the public functions. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Message-Id: <20220926093214.506243-19-eesposit@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 17 ++--- block/replication.c | 2 + blockdev.c | 72 +++----------------- job-qmp.c | 46 +++---------- job.c | 111 +++++++++---------------------- qemu-img.c | 2 - tests/unit/test-bdrv-drain.c | 4 +- tests/unit/test-block-iothread.c | 2 +- tests/unit/test-blockjob.c | 19 +++--- 9 files changed, 72 insertions(+), 203 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -XXX,XX +XXX,XX @@ typedef struct Job { AioContext *aio_context; - /** Protected by AioContext lock */ + /** Protected by job_mutex */ /** Reference count of the block job */ int refcnt; @@ -XXX,XX +XXX,XX @@ typedef struct Job { /** * Set to false by the job while the coroutine has yielded and may be * re-entered by job_enter(). There may still be I/O or event loop activity - * pending. Accessed under block_job_mutex (in blockjob.c). + * pending. Accessed under job_mutex. * * When the job is deferred to the main loop, busy is true as long as the * bottom half is still pending. @@ -XXX,XX +XXX,XX @@ typedef enum JobCreateFlags { extern QemuMutex job_mutex; -#define JOB_LOCK_GUARD() /* QEMU_LOCK_GUARD(&job_mutex) */ +#define JOB_LOCK_GUARD() QEMU_LOCK_GUARD(&job_mutex) -#define WITH_JOB_LOCK_GUARD() /* WITH_QEMU_LOCK_GUARD(&job_mutex) */ +#define WITH_JOB_LOCK_GUARD() WITH_QEMU_LOCK_GUARD(&job_mutex) /** * job_lock: @@ -XXX,XX +XXX,XX @@ void job_ref_locked(Job *job); /** * Release a reference that was previously acquired with job_ref() or * job_create(). If it's the last reference to the object, it will be freed. + * + * Takes AioContext lock internally to invoke a job->driver callback. */ void job_unref(Job *job); @@ -XXX,XX +XXX,XX @@ void job_user_cancel_locked(Job *job, bool force, Error **errp); * Returns the return value from the job if the job actually completed * during the call, or -ECANCELED if it was canceled. * - * Callers must hold the AioContext lock of job->aio_context. + * Called with job_lock *not* held. */ int job_cancel_sync(Job *job, bool force); @@ -XXX,XX +XXX,XX @@ void job_cancel_sync_all(void); * function). * * Returns the return value from the job. - * - * Callers must hold the AioContext lock of job->aio_context. + * Called with job_lock *not* held. */ int job_complete_sync(Job *job, Error **errp); @@ -XXX,XX +XXX,XX @@ void job_dismiss_locked(Job **job, Error **errp); * Returns 0 if the job is successfully completed, -ECANCELED if the job was * cancelled before completing, and -errno in other error cases. * - * Callers must hold the AioContext lock of job->aio_context. + * Called with job_lock *not* held. */ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp); diff --git a/block/replication.c b/block/replication.c index XXXXXXX..XXXXXXX 100644 --- a/block/replication.c +++ b/block/replication.c @@ -XXX,XX +XXX,XX @@ static void replication_stop(ReplicationState *rs, bool failover, Error **errp) * disk, secondary disk in backup_job_completed(). */ if (s->backup_job) { + aio_context_release(aio_context); job_cancel_sync(&s->backup_job->job, true); + aio_context_acquire(aio_context); } if (!failover) { diff --git a/blockdev.c b/blockdev.c index XXXXXXX..XXXXXXX 100644 --- a/blockdev.c +++ b/blockdev.c @@ -XXX,XX +XXX,XX @@ void blockdev_mark_auto_del(BlockBackend *blk) for (job = block_job_next_locked(NULL); job; job = block_job_next_locked(job)) { if (block_job_has_bdrv(job, blk_bs(blk))) { - AioContext *aio_context = job->job.aio_context; - aio_context_acquire(aio_context); - job_cancel_locked(&job->job, false); - - aio_context_release(aio_context); } } @@ -XXX,XX +XXX,XX @@ static void drive_backup_abort(BlkActionState *common) DriveBackupState *state = DO_UPCAST(DriveBackupState, common, common); if (state->job) { - AioContext *aio_context; - - aio_context = bdrv_get_aio_context(state->bs); - aio_context_acquire(aio_context); - job_cancel_sync(&state->job->job, true); - - aio_context_release(aio_context); } } @@ -XXX,XX +XXX,XX @@ static void blockdev_backup_abort(BlkActionState *common) BlockdevBackupState *state = DO_UPCAST(BlockdevBackupState, common, common); if (state->job) { - AioContext *aio_context; - - aio_context = bdrv_get_aio_context(state->bs); - aio_context_acquire(aio_context); - job_cancel_sync(&state->job->job, true); - - aio_context_release(aio_context); } } @@ -XXX,XX +XXX,XX @@ out: } /* - * Get a block job using its ID and acquire its AioContext. - * Called with job_mutex held. + * Get a block job using its ID. Called with job_mutex held. */ -static BlockJob *find_block_job_locked(const char *id, - AioContext **aio_context, - Error **errp) +static BlockJob *find_block_job_locked(const char *id, Error **errp) { BlockJob *job; assert(id != NULL); - *aio_context = NULL; - job = block_job_get_locked(id); if (!job) { @@ -XXX,XX +XXX,XX @@ static BlockJob *find_block_job_locked(const char *id, return NULL; } - *aio_context = block_job_get_aio_context(job); - aio_context_acquire(*aio_context); - return job; } void qmp_block_job_set_speed(const char *device, int64_t speed, Error **errp) { - AioContext *aio_context; BlockJob *job; JOB_LOCK_GUARD(); - job = find_block_job_locked(device, &aio_context, errp); + job = find_block_job_locked(device, errp); if (!job) { return; } block_job_set_speed_locked(job, speed, errp); - aio_context_release(aio_context); } void qmp_block_job_cancel(const char *device, bool has_force, bool force, Error **errp) { - AioContext *aio_context; BlockJob *job; JOB_LOCK_GUARD(); - job = find_block_job_locked(device, &aio_context, errp); + job = find_block_job_locked(device, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_block_job_cancel(const char *device, if (job_user_paused_locked(&job->job) && !force) { error_setg(errp, "The block job for device '%s' is currently paused", device); - goto out; + return; } trace_qmp_block_job_cancel(job); job_user_cancel_locked(&job->job, force, errp); -out: - aio_context_release(aio_context); } void qmp_block_job_pause(const char *device, Error **errp) { - AioContext *aio_context; BlockJob *job; JOB_LOCK_GUARD(); - job = find_block_job_locked(device, &aio_context, errp); + job = find_block_job_locked(device, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_block_job_pause(const char *device, Error **errp) trace_qmp_block_job_pause(job); job_user_pause_locked(&job->job, errp); - aio_context_release(aio_context); } void qmp_block_job_resume(const char *device, Error **errp) { - AioContext *aio_context; BlockJob *job; JOB_LOCK_GUARD(); - job = find_block_job_locked(device, &aio_context, errp); + job = find_block_job_locked(device, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_block_job_resume(const char *device, Error **errp) trace_qmp_block_job_resume(job); job_user_resume_locked(&job->job, errp); - aio_context_release(aio_context); } void qmp_block_job_complete(const char *device, Error **errp) { - AioContext *aio_context; BlockJob *job; JOB_LOCK_GUARD(); - job = find_block_job_locked(device, &aio_context, errp); + job = find_block_job_locked(device, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_block_job_complete(const char *device, Error **errp) trace_qmp_block_job_complete(job); job_complete_locked(&job->job, errp); - aio_context_release(aio_context); } void qmp_block_job_finalize(const char *id, Error **errp) { - AioContext *aio_context; BlockJob *job; JOB_LOCK_GUARD(); - job = find_block_job_locked(id, &aio_context, errp); + job = find_block_job_locked(id, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_block_job_finalize(const char *id, Error **errp) job_ref_locked(&job->job); job_finalize_locked(&job->job, errp); - /* - * Job's context might have changed via job_finalize (and job_txn_apply - * automatically acquires the new one), so make sure we release the correct - * one. - */ - aio_context = block_job_get_aio_context(job); job_unref_locked(&job->job); - aio_context_release(aio_context); } void qmp_block_job_dismiss(const char *id, Error **errp) { - AioContext *aio_context; BlockJob *bjob; Job *job; JOB_LOCK_GUARD(); - bjob = find_block_job_locked(id, &aio_context, errp); + bjob = find_block_job_locked(id, errp); if (!bjob) { return; @@ -XXX,XX +XXX,XX @@ void qmp_block_job_dismiss(const char *id, Error **errp) trace_qmp_block_job_dismiss(bjob); job = &bjob->job; job_dismiss_locked(&job, errp); - aio_context_release(aio_context); } void qmp_change_backing_file(const char *device, @@ -XXX,XX +XXX,XX @@ BlockJobInfoList *qmp_query_block_jobs(Error **errp) for (job = block_job_next_locked(NULL); job; job = block_job_next_locked(job)) { BlockJobInfo *value; - AioContext *aio_context; if (block_job_is_internal(job)) { continue; } - aio_context = block_job_get_aio_context(job); - aio_context_acquire(aio_context); value = block_job_query_locked(job, errp); - aio_context_release(aio_context); if (!value) { qapi_free_BlockJobInfoList(head); return NULL; diff --git a/job-qmp.c b/job-qmp.c index XXXXXXX..XXXXXXX 100644 --- a/job-qmp.c +++ b/job-qmp.c @@ -XXX,XX +XXX,XX @@ #include "trace/trace-root.h" /* - * Get a job using its ID and acquire its AioContext. - * Called with job_mutex held. + * Get a job using its ID. Called with job_mutex held. */ -static Job *find_job_locked(const char *id, - AioContext **aio_context, - Error **errp) +static Job *find_job_locked(const char *id, Error **errp) { Job *job; - *aio_context = NULL; - job = job_get_locked(id); if (!job) { error_setg(errp, "Job not found"); return NULL; } - *aio_context = job->aio_context; - aio_context_acquire(*aio_context); - return job; } void qmp_job_cancel(const char *id, Error **errp) { - AioContext *aio_context; Job *job; JOB_LOCK_GUARD(); - job = find_job_locked(id, &aio_context, errp); + job = find_job_locked(id, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_job_cancel(const char *id, Error **errp) trace_qmp_job_cancel(job); job_user_cancel_locked(job, true, errp); - aio_context_release(aio_context); } void qmp_job_pause(const char *id, Error **errp) { - AioContext *aio_context; Job *job; JOB_LOCK_GUARD(); - job = find_job_locked(id, &aio_context, errp); + job = find_job_locked(id, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_job_pause(const char *id, Error **errp) trace_qmp_job_pause(job); job_user_pause_locked(job, errp); - aio_context_release(aio_context); } void qmp_job_resume(const char *id, Error **errp) { - AioContext *aio_context; Job *job; JOB_LOCK_GUARD(); - job = find_job_locked(id, &aio_context, errp); + job = find_job_locked(id, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_job_resume(const char *id, Error **errp) trace_qmp_job_resume(job); job_user_resume_locked(job, errp); - aio_context_release(aio_context); } void qmp_job_complete(const char *id, Error **errp) { - AioContext *aio_context; Job *job; JOB_LOCK_GUARD(); - job = find_job_locked(id, &aio_context, errp); + job = find_job_locked(id, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_job_complete(const char *id, Error **errp) trace_qmp_job_complete(job); job_complete_locked(job, errp); - aio_context_release(aio_context); } void qmp_job_finalize(const char *id, Error **errp) { - AioContext *aio_context; Job *job; JOB_LOCK_GUARD(); - job = find_job_locked(id, &aio_context, errp); + job = find_job_locked(id, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_job_finalize(const char *id, Error **errp) job_ref_locked(job); job_finalize_locked(job, errp); - /* - * Job's context might have changed via job_finalize (and job_txn_apply - * automatically acquires the new one), so make sure we release the correct - * one. - */ - aio_context = job->aio_context; job_unref_locked(job); - aio_context_release(aio_context); } void qmp_job_dismiss(const char *id, Error **errp) { - AioContext *aio_context; Job *job; JOB_LOCK_GUARD(); - job = find_job_locked(id, &aio_context, errp); + job = find_job_locked(id, errp); if (!job) { return; @@ -XXX,XX +XXX,XX @@ void qmp_job_dismiss(const char *id, Error **errp) trace_qmp_job_dismiss(job); job_dismiss_locked(&job, errp); - aio_context_release(aio_context); } /* Called with job_mutex held. */ @@ -XXX,XX +XXX,XX @@ JobInfoList *qmp_query_jobs(Error **errp) for (job = job_next_locked(NULL); job; job = job_next_locked(job)) { JobInfo *value; - AioContext *aio_context; if (job_is_internal(job)) { continue; } - aio_context = job->aio_context; - aio_context_acquire(aio_context); value = job_query_single_locked(job, errp); - aio_context_release(aio_context); if (!value) { qapi_free_JobInfoList(head); return NULL; diff --git a/job.c b/job.c index XXXXXXX..XXXXXXX 100644 --- a/job.c +++ b/job.c @@ -XXX,XX +XXX,XX @@ * * The second includes functions used by the job drivers and sometimes * by the core block layer. These delegate the locking to the callee instead. - * - * TODO Actually make this true */ /* @@ -XXX,XX +XXX,XX @@ struct JobTxn { }; void job_lock(void) -{ - /* nop */ -} - -void job_unlock(void) -{ - /* nop */ -} - -static void real_job_lock(void) { qemu_mutex_lock(&job_mutex); } -static void real_job_unlock(void) +void job_unlock(void) { qemu_mutex_unlock(&job_mutex); } @@ -XXX,XX +XXX,XX @@ static void job_txn_del_job_locked(Job *job) /* Called with job_mutex held, but releases it temporarily. */ static int job_txn_apply_locked(Job *job, int fn(Job *)) { - AioContext *inner_ctx; Job *other_job, *next; JobTxn *txn = job->txn; int rc = 0; @@ -XXX,XX +XXX,XX @@ static int job_txn_apply_locked(Job *job, int fn(Job *)) * break AIO_WAIT_WHILE from within fn. */ job_ref_locked(job); - aio_context_release(job->aio_context); QLIST_FOREACH_SAFE(other_job, &txn->jobs, txn_list, next) { - inner_ctx = other_job->aio_context; - aio_context_acquire(inner_ctx); rc = fn(other_job); - aio_context_release(inner_ctx); if (rc) { break; } } - /* - * Note that job->aio_context might have been changed by calling fn, so we - * can't use a local variable to cache it. - */ - aio_context_acquire(job->aio_context); job_unref_locked(job); return rc; } @@ -XXX,XX +XXX,XX @@ void job_unref_locked(Job *job) assert(!job->txn); if (job->driver->free) { + AioContext *aio_context = job->aio_context; job_unlock(); + /* FIXME: aiocontext lock is required because cb calls blk_unref */ + aio_context_acquire(aio_context); job->driver->free(job); + aio_context_release(aio_context); job_lock(); } @@ -XXX,XX +XXX,XX @@ void job_enter_cond_locked(Job *job, bool(*fn)(Job *job)) return; } - real_job_lock(); if (job->busy) { - real_job_unlock(); return; } if (fn && !fn(job)) { - real_job_unlock(); return; } assert(!job->deferred_to_main_loop); timer_del(&job->sleep_timer); job->busy = true; - real_job_unlock(); job_unlock(); aio_co_wake(job->co); job_lock(); @@ -XXX,XX +XXX,XX @@ static void coroutine_fn job_do_yield_locked(Job *job, uint64_t ns) { AioContext *next_aio_context; - real_job_lock(); if (ns != -1) { timer_mod(&job->sleep_timer, ns); } job->busy = false; job_event_idle_locked(job); - real_job_unlock(); job_unlock(); qemu_coroutine_yield(); job_lock(); @@ -XXX,XX +XXX,XX @@ static void job_clean(Job *job) } } -/* Called with job_mutex held, but releases it temporarily */ +/* + * Called with job_mutex held, but releases it temporarily. + * Takes AioContext lock internally to invoke a job->driver callback. + */ static int job_finalize_single_locked(Job *job) { int job_ret; + AioContext *ctx = job->aio_context; assert(job_is_completed_locked(job)); @@ -XXX,XX +XXX,XX @@ static int job_finalize_single_locked(Job *job) job_ret = job->ret; job_unlock(); + aio_context_acquire(ctx); if (!job_ret) { job_commit(job); @@ -XXX,XX +XXX,XX @@ static int job_finalize_single_locked(Job *job) } job_clean(job); - job_lock(); - if (job->cb) { - job_ret = job->ret; - job_unlock(); job->cb(job->opaque, job_ret); - job_lock(); } + aio_context_release(ctx); + job_lock(); + /* Emit events only if we actually started */ if (job_started_locked(job)) { if (job_is_cancelled_locked(job)) { @@ -XXX,XX +XXX,XX @@ static int job_finalize_single_locked(Job *job) return 0; } -/* Called with job_mutex held, but releases it temporarily */ +/* + * Called with job_mutex held, but releases it temporarily. + * Takes AioContext lock internally to invoke a job->driver callback. + */ static void job_cancel_async_locked(Job *job, bool force) { + AioContext *ctx = job->aio_context; GLOBAL_STATE_CODE(); if (job->driver->cancel) { job_unlock(); + aio_context_acquire(ctx); force = job->driver->cancel(job, force); + aio_context_release(ctx); job_lock(); } else { /* No .cancel() means the job will behave as if force-cancelled */ @@ -XXX,XX +XXX,XX @@ static void job_cancel_async_locked(Job *job, bool force) } } -/* Called with job_mutex held, but releases it temporarily. */ +/* + * Called with job_mutex held, but releases it temporarily. + * Takes AioContext lock internally to invoke a job->driver callback. + */ static void job_completed_txn_abort_locked(Job *job) { - AioContext *ctx; JobTxn *txn = job->txn; Job *other_job; @@ -XXX,XX +XXX,XX @@ static void job_completed_txn_abort_locked(Job *job) txn->aborting = true; job_txn_ref_locked(txn); - /* - * We can only hold the single job's AioContext lock while calling - * job_finalize_single() because the finalization callbacks can involve - * calls of AIO_WAIT_WHILE(), which could deadlock otherwise. - * Note that the job's AioContext may change when it is finalized. - */ job_ref_locked(job); - aio_context_release(job->aio_context); /* Other jobs are effectively cancelled by us, set the status for * them; this job, however, may or may not be cancelled, depending * on the caller, so leave it. */ QLIST_FOREACH(other_job, &txn->jobs, txn_list) { if (other_job != job) { - ctx = other_job->aio_context; - aio_context_acquire(ctx); /* * This is a transaction: If one job failed, no result will matter. * Therefore, pass force=true to terminate all other jobs as quickly * as possible. */ job_cancel_async_locked(other_job, true); - aio_context_release(ctx); } } while (!QLIST_EMPTY(&txn->jobs)) { other_job = QLIST_FIRST(&txn->jobs); - /* - * The job's AioContext may change, so store it in @ctx so we - * release the same context that we have acquired before. - */ - ctx = other_job->aio_context; - aio_context_acquire(ctx); if (!job_is_completed_locked(other_job)) { assert(job_cancel_requested_locked(other_job)); job_finish_sync_locked(other_job, NULL, NULL); } job_finalize_single_locked(other_job); - aio_context_release(ctx); } - /* - * Use job_ref()/job_unref() so we can read the AioContext here - * even if the job went away during job_finalize_single(). - */ - aio_context_acquire(job->aio_context); job_unref_locked(job); - job_txn_unref_locked(txn); } @@ -XXX,XX +XXX,XX @@ static void job_completed_txn_abort_locked(Job *job) static int job_prepare_locked(Job *job) { int ret; + AioContext *ctx = job->aio_context; GLOBAL_STATE_CODE(); + if (job->ret == 0 && job->driver->prepare) { job_unlock(); + aio_context_acquire(ctx); ret = job->driver->prepare(job); + aio_context_release(ctx); job_lock(); job->ret = ret; job_update_rc_locked(job); } + return job->ret; } @@ -XXX,XX +XXX,XX @@ static void job_completed_locked(Job *job) static void job_exit(void *opaque) { Job *job = (Job *)opaque; - AioContext *ctx; JOB_LOCK_GUARD(); - job_ref_locked(job); - aio_context_acquire(job->aio_context); /* This is a lie, we're not quiescent, but still doing the completion * callbacks. However, completion callbacks tend to involve operations that @@ -XXX,XX +XXX,XX @@ static void job_exit(void *opaque) job_event_idle_locked(job); job_completed_locked(job); - - /* - * Note that calling job_completed can move the job to a different - * aio_context, so we cannot cache from above. job_txn_apply takes care of - * acquiring the new lock, and we ref/unref to avoid job_completed freeing - * the job underneath us. - */ - ctx = job->aio_context; job_unref_locked(job); - aio_context_release(ctx); } /** @@ -XXX,XX +XXX,XX @@ int job_cancel_sync(Job *job, bool force) void job_cancel_sync_all(void) { Job *job; - AioContext *aio_context; JOB_LOCK_GUARD(); while ((job = job_next_locked(NULL))) { - aio_context = job->aio_context; - aio_context_acquire(aio_context); job_cancel_sync_locked(job, true); - aio_context_release(aio_context); } } @@ -XXX,XX +XXX,XX @@ int job_finish_sync_locked(Job *job, } job_unlock(); - AIO_WAIT_WHILE(job->aio_context, - (job_enter(job), !job_is_completed(job))); + AIO_WAIT_WHILE_UNLOCKED(job->aio_context, + (job_enter(job), !job_is_completed(job))); job_lock(); ret = (job_is_cancelled_locked(job) && job->ret == 0) diff --git a/qemu-img.c b/qemu-img.c index XXXXXXX..XXXXXXX 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -XXX,XX +XXX,XX @@ static void run_block_job(BlockJob *job, Error **errp) AioContext *aio_context = block_job_get_aio_context(job); int ret = 0; - aio_context_acquire(aio_context); job_lock(); job_ref_locked(&job->job); do { @@ -XXX,XX +XXX,XX @@ static void run_block_job(BlockJob *job, Error **errp) } job_unref_locked(&job->job); job_unlock(); - aio_context_release(aio_context); /* publish completion progress only when success */ if (!ret) { diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-bdrv-drain.c +++ b/tests/unit/test-bdrv-drain.c @@ -XXX,XX +XXX,XX @@ static void test_blockjob_common_drain_node(enum drain_type drain_type, tjob->prepare_ret = -EIO; break; } + aio_context_release(ctx); job_start(&job->job); - aio_context_release(ctx); if (use_iothread) { /* job_co_entry() is run in the I/O thread, wait for the actual job @@ -XXX,XX +XXX,XX @@ static void test_blockjob_common_drain_node(enum drain_type drain_type, g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ } - aio_context_acquire(ctx); WITH_JOB_LOCK_GUARD() { ret = job_complete_sync_locked(&job->job, &error_abort); } g_assert_cmpint(ret, ==, (result == TEST_JOB_SUCCESS ? 0 : -EIO)); + aio_context_acquire(ctx); if (use_iothread) { blk_set_aio_context(blk_src, qemu_get_aio_context(), &error_abort); assert(blk_get_aio_context(blk_target) == qemu_get_aio_context()); diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothread.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-block-iothread.c +++ b/tests/unit/test-block-iothread.c @@ -XXX,XX +XXX,XX @@ static void test_attach_blockjob(void) aio_poll(qemu_get_aio_context(), false); } - aio_context_acquire(ctx); WITH_JOB_LOCK_GUARD() { job_complete_sync_locked(&tjob->common.job, &error_abort); } + aio_context_acquire(ctx); blk_set_aio_context(blk, qemu_get_aio_context(), &error_abort); aio_context_release(ctx); diff --git a/tests/unit/test-blockjob.c b/tests/unit/test-blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-blockjob.c +++ b/tests/unit/test-blockjob.c @@ -XXX,XX +XXX,XX @@ static void cancel_common(CancelJob *s) BlockJob *job = &s->common; BlockBackend *blk = s->blk; JobStatus sts = job->job.status; - AioContext *ctx; - - ctx = job->job.aio_context; - aio_context_acquire(ctx); + AioContext *ctx = job->job.aio_context; job_cancel_sync(&job->job, true); WITH_JOB_LOCK_GUARD() { @@ -XXX,XX +XXX,XX @@ static void cancel_common(CancelJob *s) assert(job->job.status == JOB_STATUS_NULL); job_unref_locked(&job->job); } - destroy_blk(blk); + aio_context_acquire(ctx); + destroy_blk(blk); aio_context_release(ctx); + } static void test_cancel_created(void) @@ -XXX,XX +XXX,XX @@ static void test_cancel_concluded(void) aio_poll(qemu_get_aio_context(), true); assert_job_status_is(job, JOB_STATUS_PENDING); - aio_context_acquire(job->aio_context); WITH_JOB_LOCK_GUARD() { job_finalize_locked(job, &error_abort); + assert(job->status == JOB_STATUS_CONCLUDED); } - aio_context_release(job->aio_context); - assert_job_status_is(job, JOB_STATUS_CONCLUDED); cancel_common(s); } @@ -XXX,XX +XXX,XX @@ static void test_complete_in_standby(void) /* Wait for the job to become READY */ job_start(job); - aio_context_acquire(ctx); /* * Here we are waiting for the status to change, so don't bother * protecting the read every time. */ - AIO_WAIT_WHILE(ctx, job->status != JOB_STATUS_READY); - aio_context_release(ctx); + AIO_WAIT_WHILE_UNLOCKED(ctx, job->status != JOB_STATUS_READY); /* Begin the drained section, pausing the job */ bdrv_drain_all_begin(); @@ -XXX,XX +XXX,XX @@ static void test_complete_in_standby(void) aio_context_acquire(ctx); /* This will schedule the job to resume it */ bdrv_drain_all_end(); + aio_context_release(ctx); WITH_JOB_LOCK_GUARD() { /* But the job cannot run, so it will remain on standby */ @@ -XXX,XX +XXX,XX @@ static void test_complete_in_standby(void) job_dismiss_locked(&job, &error_abort); } + aio_context_acquire(ctx); destroy_blk(blk); aio_context_release(ctx); iothread_join(iothread); -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> Not sure what the atomic here was supposed to do, since job.busy is protected by the job lock. Since the whole function is called under job_mutex, just remove the atomic. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20220926093214.506243-20-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blockjob.c b/blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/blockjob.c +++ b/blockjob.c @@ -XXX,XX +XXX,XX @@ BlockJobInfo *block_job_query_locked(BlockJob *job, Error **errp) info = g_new0(BlockJobInfo, 1); info->type = g_strdup(job_type_str(&job->job)); info->device = g_strdup(job->job.id); - info->busy = qatomic_read(&job->job.busy); + info->busy = job->job.busy; info->paused = job->job.pause_count > 0; info->offset = progress_current; info->len = progress_total; -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> These public functions are not used anywhere, thus can be dropped. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-Id: <20220926093214.506243-21-eesposit@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/block/blockjob.h | 31 ++++++++++++------------------- blockjob.c | 16 ++-------------- 2 files changed, 14 insertions(+), 33 deletions(-) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index XXXXXXX..XXXXXXX 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -XXX,XX +XXX,XX @@ typedef struct BlockJob { */ /** - * block_job_next: + * block_job_next_locked: * @job: A block job, or %NULL. * * Get the next element from the list of block jobs after @job, or the * first one if @job is %NULL. * * Returns the requested job, or %NULL if there are no more jobs left. + * Called with job lock held. */ -BlockJob *block_job_next(BlockJob *job); - -/* Same as block_job_next(), but called with job lock held. */ BlockJob *block_job_next_locked(BlockJob *job); /** @@ -XXX,XX +XXX,XX @@ BlockJob *block_job_next_locked(BlockJob *job); * Get the block job identified by @id (which must not be %NULL). * * Returns the requested job, or %NULL if it doesn't exist. + * Called with job lock *not* held. */ BlockJob *block_job_get(const char *id); @@ -XXX,XX +XXX,XX @@ void block_job_remove_all_bdrv(BlockJob *job); bool block_job_has_bdrv(BlockJob *job, BlockDriverState *bs); /** - * block_job_set_speed: + * block_job_set_speed_locked: * @job: The job to set the speed for. * @speed: The new value * @errp: Error object. * * Set a rate-limiting parameter for the job; the actual meaning may * vary depending on the job type. - */ -bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp); - -/* - * Same as block_job_set_speed(), but called with job lock held. - * Might release the lock temporarily. + * + * Called with job lock held, but might release it temporarily. */ bool block_job_set_speed_locked(BlockJob *job, int64_t speed, Error **errp); /** - * block_job_query: + * block_job_query_locked: * @job: The job to get information about. * * Return information about a job. + * + * Called with job lock held. */ -BlockJobInfo *block_job_query(BlockJob *job, Error **errp); - -/* Same as block_job_query(), but called with job lock held. */ BlockJobInfo *block_job_query_locked(BlockJob *job, Error **errp); /** - * block_job_iostatus_reset: + * block_job_iostatus_reset_locked: * @job: The job whose I/O status should be reset. * * Reset I/O status on @job and on BlockDriverState objects it uses, * other than job->blk. + * + * Called with job lock held. */ -void block_job_iostatus_reset(BlockJob *job); - -/* Same as block_job_iostatus_reset(), but called with job lock held. */ void block_job_iostatus_reset_locked(BlockJob *job); /* diff --git a/blockjob.c b/blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/blockjob.c +++ b/blockjob.c @@ -XXX,XX +XXX,XX @@ BlockJob *block_job_next_locked(BlockJob *bjob) return job ? container_of(job, BlockJob, job) : NULL; } -BlockJob *block_job_next(BlockJob *bjob) -{ - JOB_LOCK_GUARD(); - return block_job_next_locked(bjob); -} - BlockJob *block_job_get_locked(const char *id) { Job *job = job_get_locked(id); @@ -XXX,XX +XXX,XX @@ bool block_job_set_speed_locked(BlockJob *job, int64_t speed, Error **errp) return true; } -bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) +static bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) { JOB_LOCK_GUARD(); return block_job_set_speed_locked(job, speed, errp); @@ -XXX,XX +XXX,XX @@ BlockJobInfo *block_job_query_locked(BlockJob *job, Error **errp) return info; } -BlockJobInfo *block_job_query(BlockJob *job, Error **errp) -{ - JOB_LOCK_GUARD(); - return block_job_query_locked(job, errp); -} - /* Called with job lock held */ static void block_job_iostatus_set_err_locked(BlockJob *job, int error) { @@ -XXX,XX +XXX,XX @@ void block_job_iostatus_reset_locked(BlockJob *job) job->iostatus = BLOCK_DEVICE_IO_STATUS_OK; } -void block_job_iostatus_reset(BlockJob *job) +static void block_job_iostatus_reset(BlockJob *job) { JOB_LOCK_GUARD(); block_job_iostatus_reset_locked(job); -- 2.37.3 From: Emanuele Giuseppe Esposito <eesposit@redhat.com> These public functions are not used anywhere, thus can be dropped. Also, since this is the final job API that doesn't use AioContext lock and replaces it with job_lock, adjust all remaining function documentation to clearly specify if the job lock is taken or not. Also document the locking requirements for a few functions where the second version is not removed. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20220926093214.506243-22-eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 110 +++++++++++++------------------------ job.c | 107 ++---------------------------------- tests/unit/test-blockjob.c | 4 +- 3 files changed, 46 insertions(+), 175 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -XXX,XX +XXX,XX @@ JobTxn *job_txn_new(void); /** * Release a reference that was previously acquired with job_txn_add_job or * job_txn_new. If it's the last reference to the object, it will be freed. + * + * Called with job lock *not* held. */ void job_txn_unref(JobTxn *txn); @@ -XXX,XX +XXX,XX @@ void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn, /** * Add a reference to Job refcnt, it will be decreased with job_unref, and then * be freed if it comes to be the last reference. + * + * Called with job lock held. */ -void job_ref(Job *job); - -/* Same as job_ref(), but called with job lock held. */ void job_ref_locked(Job *job); /** - * Release a reference that was previously acquired with job_ref() or + * Release a reference that was previously acquired with job_ref_locked() or * job_create(). If it's the last reference to the object, it will be freed. * * Takes AioContext lock internally to invoke a job->driver callback. + * Called with job lock held. */ -void job_unref(Job *job); - -/* Same as job_unref(), but called with job lock held. */ void job_unref_locked(Job *job); /** @@ -XXX,XX +XXX,XX @@ void job_progress_increase_remaining(Job *job, uint64_t delta); * Conditionally enter the job coroutine if the job is ready to run, not * already busy and fn() returns true. fn() is called while under the job_lock * critical section. - */ -void job_enter_cond(Job *job, bool(*fn)(Job *job)); - -/* - * Same as job_enter_cond(), but called with job lock held. - * Might release the lock temporarily. + * + * Called with job lock held, but might release it temporarily. */ void job_enter_cond_locked(Job *job, bool(*fn)(Job *job)); @@ -XXX,XX +XXX,XX @@ bool job_cancel_requested(Job *job); /** * Returns whether the job is in a completed state. - * Called with job_mutex *not* held. + * Called with job lock held. */ -bool job_is_completed(Job *job); - -/* Same as job_is_completed(), but called with job lock held. */ bool job_is_completed_locked(Job *job); /** @@ -XXX,XX +XXX,XX @@ bool job_is_ready_locked(Job *job); /** * Request @job to pause at the next pause point. Must be paired with * job_resume(). If the job is supposed to be resumed by user action, call - * job_user_pause() instead. + * job_user_pause_locked() instead. + * + * Called with job lock *not* held. */ void job_pause(Job *job); /* Same as job_pause(), but called with job lock held. */ void job_pause_locked(Job *job); -/** Resumes a @job paused with job_pause. */ +/** Resumes a @job paused with job_pause. Called with job lock *not* held. */ void job_resume(Job *job); /* @@ -XXX,XX +XXX,XX @@ void job_resume_locked(Job *job); /** * Asynchronously pause the specified @job. * Do not allow a resume until a matching call to job_user_resume. + * Called with job lock held. */ -void job_user_pause(Job *job, Error **errp); - -/* Same as job_user_pause(), but called with job lock held. */ void job_user_pause_locked(Job *job, Error **errp); -/** Returns true if the job is user-paused. */ -bool job_user_paused(Job *job); - -/* Same as job_user_paused(), but called with job lock held. */ +/** + * Returns true if the job is user-paused. + * Called with job lock held. + */ bool job_user_paused_locked(Job *job); /** * Resume the specified @job. - * Must be paired with a preceding job_user_pause. - */ -void job_user_resume(Job *job, Error **errp); - -/* - * Same as job_user_resume(), but called with job lock held. - * Might release the lock temporarily. + * Must be paired with a preceding job_user_pause_locked. + * Called with job lock held, but might release it temporarily. */ void job_user_resume_locked(Job *job, Error **errp); @@ -XXX,XX +XXX,XX @@ void job_user_resume_locked(Job *job, Error **errp); * first one if @job is %NULL. * * Returns the requested job, or %NULL if there are no more jobs left. + * Called with job lock *not* held. */ Job *job_next(Job *job); @@ -XXX,XX +XXX,XX @@ Job *job_next_locked(Job *job); * Get the job identified by @id (which must not be %NULL). * * Returns the requested job, or %NULL if it doesn't exist. + * Called with job lock held. */ -Job *job_get(const char *id); - -/* Same as job_get(), but called with job lock held. */ Job *job_get_locked(const char *id); /** * Check whether the verb @verb can be applied to @job in its current state. * Returns 0 if the verb can be applied; otherwise errp is set and -EPERM * returned. + * + * Called with job lock held. */ -int job_apply_verb(Job *job, JobVerb verb, Error **errp); - -/* Same as job_apply_verb, but called with job lock held. */ int job_apply_verb_locked(Job *job, JobVerb verb, Error **errp); /** @@ -XXX,XX +XXX,XX @@ void job_early_fail(Job *job); */ void job_transition_to_ready(Job *job); -/** Asynchronously complete the specified @job. */ -void job_complete(Job *job, Error **errp); - -/* - * Same as job_complete(), but called with job lock held. - * Might release the lock temporarily. +/** + * Asynchronously complete the specified @job. + * Called with job lock held, but might release it temporarily. */ void job_complete_locked(Job *job, Error **errp); /** * Asynchronously cancel the specified @job. If @force is true, the job should * be cancelled immediately without waiting for a consistent state. + * Called with job lock held. */ -void job_cancel(Job *job, bool force); - -/* Same as job_cancel(), but called with job lock held. */ void job_cancel_locked(Job *job, bool force); /** - * Cancels the specified job like job_cancel(), but may refuse to do so if the - * operation isn't meaningful in the current state of the job. + * Cancels the specified job like job_cancel_locked(), but may refuse + * to do so if the operation isn't meaningful in the current state of the job. + * Called with job lock held. */ -void job_user_cancel(Job *job, bool force, Error **errp); - -/* Same as job_user_cancel(), but called with job lock held. */ void job_user_cancel_locked(Job *job, bool force, Error **errp); /** @@ -XXX,XX +XXX,XX @@ void job_cancel_sync_all(void); /** * @job: The job to be completed. - * @errp: Error object which may be set by job_complete(); this is not + * @errp: Error object which may be set by job_complete_locked(); this is not * necessarily set on every error, the job return value has to be * checked as well. * @@ -XXX,XX +XXX,XX @@ void job_cancel_sync_all(void); * function). * * Returns the return value from the job. - * Called with job_lock *not* held. + * Called with job_lock held. */ -int job_complete_sync(Job *job, Error **errp); - -/* Same as job_complete_sync, but called with job lock held. */ int job_complete_sync_locked(Job *job, Error **errp); /** @@ -XXX,XX +XXX,XX @@ int job_complete_sync_locked(Job *job, Error **errp); * FIXME: Make the below statement universally true: * For jobs that support the manual workflow mode, all graph changes that occur * as a result will occur after this command and before a successful reply. + * + * Called with job lock held. */ -void job_finalize(Job *job, Error **errp); - -/* Same as job_finalize(), but called with job lock held. */ void job_finalize_locked(Job *job, Error **errp); /** * Remove the concluded @job from the query list and resets the passed pointer * to %NULL. Returns an error if the job is not actually concluded. + * + * Called with job lock held. */ -void job_dismiss(Job **job, Error **errp); - -/* Same as job_dismiss(), but called with job lock held. */ void job_dismiss_locked(Job **job, Error **errp); /** @@ -XXX,XX +XXX,XX @@ void job_dismiss_locked(Job **job, Error **errp); * Returns 0 if the job is successfully completed, -ECANCELED if the job was * cancelled before completing, and -errno in other error cases. * - * Called with job_lock *not* held. - */ -int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), - Error **errp); - -/* - * Same as job_finish_sync(), but called with job lock held. - * Might release the lock temporarily. + * Called with job_lock held, but might release it temporarily. */ int job_finish_sync_locked(Job *job, void (*finish)(Job *, Error **errp), Error **errp); diff --git a/job.c b/job.c index XXXXXXX..XXXXXXX 100644 --- a/job.c +++ b/job.c @@ -XXX,XX +XXX,XX @@ int job_apply_verb_locked(Job *job, JobVerb verb, Error **errp) return -EPERM; } -int job_apply_verb(Job *job, JobVerb verb, Error **errp) -{ - JOB_LOCK_GUARD(); - return job_apply_verb_locked(job, verb, errp); -} - JobType job_type(const Job *job) { return job->driver->job_type; @@ -XXX,XX +XXX,XX @@ bool job_is_completed_locked(Job *job) return false; } -bool job_is_completed(Job *job) +static bool job_is_completed(Job *job) { JOB_LOCK_GUARD(); return job_is_completed_locked(job); @@ -XXX,XX +XXX,XX @@ Job *job_get_locked(const char *id) return NULL; } -Job *job_get(const char *id) -{ - JOB_LOCK_GUARD(); - return job_get_locked(id); -} - void job_set_aio_context(Job *job, AioContext *ctx) { /* protect against read in job_finish_sync_locked and job_start */ @@ -XXX,XX +XXX,XX @@ void job_ref_locked(Job *job) ++job->refcnt; } -void job_ref(Job *job) -{ - JOB_LOCK_GUARD(); - job_ref_locked(job); -} - void job_unref_locked(Job *job) { GLOBAL_STATE_CODE(); @@ -XXX,XX +XXX,XX @@ void job_unref_locked(Job *job) } } -void job_unref(Job *job) -{ - JOB_LOCK_GUARD(); - job_unref_locked(job); -} - void job_progress_update(Job *job, uint64_t done) { progress_work_done(&job->progress, done); @@ -XXX,XX +XXX,XX @@ void job_enter_cond_locked(Job *job, bool(*fn)(Job *job)) job_lock(); } -void job_enter_cond(Job *job, bool(*fn)(Job *job)) -{ - JOB_LOCK_GUARD(); - job_enter_cond_locked(job, fn); -} - void job_enter(Job *job) { JOB_LOCK_GUARD(); @@ -XXX,XX +XXX,XX @@ void coroutine_fn job_pause_point(Job *job) job_pause_point_locked(job); } -static void coroutine_fn job_yield_locked(Job *job) +void coroutine_fn job_yield(Job *job) { + JOB_LOCK_GUARD(); assert(job->busy); /* Check cancellation *before* setting busy = false, too! */ @@ -XXX,XX +XXX,XX @@ static void coroutine_fn job_yield_locked(Job *job) job_pause_point_locked(job); } -void coroutine_fn job_yield(Job *job) -{ - JOB_LOCK_GUARD(); - job_yield_locked(job); -} - void coroutine_fn job_sleep_ns(Job *job, int64_t ns) { JOB_LOCK_GUARD(); @@ -XXX,XX +XXX,XX @@ void job_user_pause_locked(Job *job, Error **errp) job_pause_locked(job); } -void job_user_pause(Job *job, Error **errp) -{ - JOB_LOCK_GUARD(); - job_user_pause_locked(job, errp); -} - bool job_user_paused_locked(Job *job) { return job->user_paused; } -bool job_user_paused(Job *job) -{ - JOB_LOCK_GUARD(); - return job_user_paused_locked(job); -} - void job_user_resume_locked(Job *job, Error **errp) { assert(job); @@ -XXX,XX +XXX,XX @@ void job_user_resume_locked(Job *job, Error **errp) job_resume_locked(job); } -void job_user_resume(Job *job, Error **errp) -{ - JOB_LOCK_GUARD(); - job_user_resume_locked(job, errp); -} - /* Called with job_mutex held, but releases it temporarily. */ static void job_do_dismiss_locked(Job *job) { @@ -XXX,XX +XXX,XX @@ void job_dismiss_locked(Job **jobptr, Error **errp) *jobptr = NULL; } -void job_dismiss(Job **jobptr, Error **errp) -{ - JOB_LOCK_GUARD(); - job_dismiss_locked(jobptr, errp); -} - void job_early_fail(Job *job) { JOB_LOCK_GUARD(); @@ -XXX,XX +XXX,XX @@ void job_finalize_locked(Job *job, Error **errp) job_do_finalize_locked(job); } -void job_finalize(Job *job, Error **errp) -{ - JOB_LOCK_GUARD(); - job_finalize_locked(job, errp); -} - /* Called with job_mutex held. */ static int job_transition_to_pending_locked(Job *job) { @@ -XXX,XX +XXX,XX @@ void job_cancel_locked(Job *job, bool force) } } -void job_cancel(Job *job, bool force) -{ - JOB_LOCK_GUARD(); - job_cancel_locked(job, force); -} - void job_user_cancel_locked(Job *job, bool force, Error **errp) { if (job_apply_verb_locked(job, JOB_VERB_CANCEL, errp)) { @@ -XXX,XX +XXX,XX @@ void job_user_cancel_locked(Job *job, bool force, Error **errp) job_cancel_locked(job, force); } -void job_user_cancel(Job *job, bool force, Error **errp) -{ - JOB_LOCK_GUARD(); - job_user_cancel_locked(job, force, errp); -} - -/* A wrapper around job_cancel() taking an Error ** parameter so it may be - * used with job_finish_sync() without the need for (rather nasty) function - * pointer casts there. +/* A wrapper around job_cancel_locked() taking an Error ** parameter so it may + * be used with job_finish_sync_locked() without the need for (rather nasty) + * function pointer casts there. * * Called with job_mutex held. */ @@ -XXX,XX +XXX,XX @@ int job_complete_sync_locked(Job *job, Error **errp) return job_finish_sync_locked(job, job_complete_locked, errp); } -int job_complete_sync(Job *job, Error **errp) -{ - JOB_LOCK_GUARD(); - return job_complete_sync_locked(job, errp); -} - void job_complete_locked(Job *job, Error **errp) { /* Should not be reachable via external interface for internal jobs */ @@ -XXX,XX +XXX,XX @@ void job_complete_locked(Job *job, Error **errp) job_lock(); } -void job_complete(Job *job, Error **errp) -{ - JOB_LOCK_GUARD(); - job_complete_locked(job, errp); -} - int job_finish_sync_locked(Job *job, void (*finish)(Job *, Error **errp), Error **errp) @@ -XXX,XX +XXX,XX @@ int job_finish_sync_locked(Job *job, job_unref_locked(job); return ret; } - -int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) -{ - JOB_LOCK_GUARD(); - return job_finish_sync_locked(job, finish, errp); -} diff --git a/tests/unit/test-blockjob.c b/tests/unit/test-blockjob.c index XXXXXXX..XXXXXXX 100644 --- a/tests/unit/test-blockjob.c +++ b/tests/unit/test-blockjob.c @@ -XXX,XX +XXX,XX @@ static const BlockJobDriver test_yielding_driver = { }; /* - * Test that job_complete() works even on jobs that are in a paused + * Test that job_complete_locked() works even on jobs that are in a paused * state (i.e., STANDBY). * * To do this, run YieldingJob in an IO thread, get it into the READY @@ -XXX,XX +XXX,XX @@ static const BlockJobDriver test_yielding_driver = { * acquire the context so the job will not be entered and will thus * remain on STANDBY. * - * job_complete() should still work without error. + * job_complete_locked() should still work without error. * * Note that on the QMP interface, it is impossible to lock an IO * thread before a drained section ends. In practice, the -- 2.37.3 The field is unused (only ever set, but never read) since commit ac9185603. Additionally, the commit message of commit 34fa110e already explained earlier why it's unreliable. Remove it. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20220923142838.91043-1-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/file-posix.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c index XXXXXXX..XXXXXXX 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -XXX,XX +XXX,XX @@ typedef struct BDRVRawState { bool has_discard:1; bool has_write_zeroes:1; - bool discard_zeroes:1; bool use_linux_aio:1; bool use_linux_io_uring:1; int page_cache_inconsistent; /* errno from fdatasync failure */ @@ -XXX,XX +XXX,XX @@ static int raw_open_common(BlockDriverState *bs, QDict *options, ret = -EINVAL; goto fail; } else { - s->discard_zeroes = true; s->has_fallocate = true; } } else { @@ -XXX,XX +XXX,XX @@ static int raw_open_common(BlockDriverState *bs, QDict *options, } if (S_ISBLK(st.st_mode)) { -#ifdef BLKDISCARDZEROES - unsigned int arg; - if (ioctl(s->fd, BLKDISCARDZEROES, &arg) == 0 && arg) { - s->discard_zeroes = true; - } -#endif #ifdef __linux__ /* On Linux 3.10, BLKDISCARD leaves stale data in the page cache. Do * not rely on the contents of discarded blocks unless using O_DIRECT. * Same for BLKZEROOUT. */ if (!(bs->open_flags & BDRV_O_NOCACHE)) { - s->discard_zeroes = false; s->has_write_zeroes = false; } #endif -- 2.37.3