[Qemu-devel] [PATCH v9 0/9] qemu-img: add measure sub-command

Stefan Hajnoczi posted 9 patches 6 years, 8 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20170705125738.8777-1-stefanha@redhat.com
Test FreeBSD passed
Test checkpatch passed
Test docker passed
Test s390x passed
qapi/block-core.json             |  25 +++
include/block/block.h            |   2 +
include/block/block_int.h        |   2 +
block.c                          |  35 ++++
block/qcow2.c                    | 383 +++++++++++++++++++++++++++++----------
block/raw-format.c               |  26 +++
qemu-img.c                       | 234 ++++++++++++++++++++++++
qemu-img-cmds.hx                 |   6 +
qemu-img.texi                    |  30 +++
tests/qemu-iotests/178           | 170 +++++++++++++++++
tests/qemu-iotests/178.out.qcow2 | 286 +++++++++++++++++++++++++++++
tests/qemu-iotests/178.out.raw   | 158 ++++++++++++++++
tests/qemu-iotests/check         |   5 +
tests/qemu-iotests/group         |   1 +
14 files changed, 1268 insertions(+), 95 deletions(-)
create mode 100755 tests/qemu-iotests/178
create mode 100644 tests/qemu-iotests/178.out.qcow2
create mode 100644 tests/qemu-iotests/178.out.raw
[Qemu-devel] [PATCH v9 0/9] qemu-img: add measure sub-command
Posted by Stefan Hajnoczi 6 years, 8 months ago
v9:
 * Fix swapped fmt <-> output_fmt parameters in documentation [Max]
 * Clarify that backing file clusters also available in the new image
   file *may* be ommitted from the calculation [Max]

v8:
 * Check 2P file size in qemu-iotests 178 [Berto]
 * Use QCOW_MAX_L1_SIZE to check maximum virtual disk size [Berto]

v7:
 * Check max file size with 7 exabytes [Berto]
 * Really use UINT64_MAX everywhere instead of ~0ULL [Berto]

v6:
 * Change bdrv_measure() return type to BlockMeasureInfo * [Eric]
 * Clarify that holes in sparse POSIX files are still counted [Eric]

v5:
 * Use UINT64_MAX instead of ~0ULL [Berto]
 * Document qemu-img measure ofmt, fmt, output_fmt, and snapshot_param
   [Berto]

v4:
 * Make qcow2 refcount calculation conservative [Maor]
 * Include actual qemu-img convert image size in test cases

v3:
 * Drop RFC, this is ready to go for QEMU 2.10
 * Use "required size" instead of "required bytes" in qemu-img output for
   consistency [Nir]
 * Clarify BlockMeasureInfo semantics [Max]
 * Clarify bdrv_measure() opts argument and error handling [Nir]
 * Handle -o backing_file= for qcow2 [Max]
 * Handle snapshot options in qemu-img measure
 * Probe input image for allocated data clusters for qcow2.  Didn't centralize
   this because there are format-specific aspects such as the cluster_size.  It
   may make sense to centralize it later (with a bit more complexity) if
   support is added to more formats.
 * Add qemu-img(1) man page section for 'measure' sub-command [Max]
 * Extend test case to cover additional scenarios [Nir]

RFCv2:
 * Publishing RFC again to discuss the new user-visible interfaces.  Code has
   changed quite a bit, I have not kept any Reviewed-by tags.
 * Rename qemu-img sub-command "measure" and API bdrv_measure() [Nir]
 * Report both "required bytes" and "fully allocated bytes" to handle the empty
   image file and prealloc use cases [Nir and Dan]
 * Use bdrv_getlength() instead of bdrv_nb_sectors() [Berto]
 * Rename "err" label "out" in qemu-img-cmds.c [Nir]
 * Add basic qcow2 support, doesn't support qemu-img convert from existing files yet

RFCv1:
 * Publishing patch series with just raw support, no qcow2 yet.  Please review
   the command-line interface and let me know if you are happy with this
   approach.

Users and management tools sometimes need to know the size required for a new
disk image so that an LVM volume, SAN LUN, etc can be allocated ahead of time.
Image formats like qcow2 have non-trivial metadata that makes it hard to
estimate the exact size without knowledge of file format internals.

This patch series introduces a new qemu-img sub-command that calculates the
required size for both image creation and conversion scenarios.

The conversion scenario is:

  $ qemu-img measure -f raw -O qcow2 input.img
  required size: 1327680
  fully allocated size: 1074069504

Here an existing image file is taken and the output includes the space required
for data from the input image file.

The creation scenario is:

  $ qemu-img measure -O qcow2 --size 5G
  required size: 327680
  fully allocated size: 1074069504

Stefan Hajnoczi (9):
  block: add bdrv_measure() API
  raw-format: add bdrv_measure() support
  qcow2: extract preallocation calculation function
  qcow2: make refcount size calculation conservative
  qcow2: extract image creation option parsing
  qcow2: add bdrv_measure() support
  qemu-img: add measure subcommand
  qemu-iotests: support per-format golden output files
  iotests: add test 178 for qemu-img measure

 qapi/block-core.json             |  25 +++
 include/block/block.h            |   2 +
 include/block/block_int.h        |   2 +
 block.c                          |  35 ++++
 block/qcow2.c                    | 383 +++++++++++++++++++++++++++++----------
 block/raw-format.c               |  26 +++
 qemu-img.c                       | 234 ++++++++++++++++++++++++
 qemu-img-cmds.hx                 |   6 +
 qemu-img.texi                    |  30 +++
 tests/qemu-iotests/178           | 170 +++++++++++++++++
 tests/qemu-iotests/178.out.qcow2 | 286 +++++++++++++++++++++++++++++
 tests/qemu-iotests/178.out.raw   | 158 ++++++++++++++++
 tests/qemu-iotests/check         |   5 +
 tests/qemu-iotests/group         |   1 +
 14 files changed, 1268 insertions(+), 95 deletions(-)
 create mode 100755 tests/qemu-iotests/178
 create mode 100644 tests/qemu-iotests/178.out.qcow2
 create mode 100644 tests/qemu-iotests/178.out.raw

-- 
2.9.4


Re: [Qemu-devel] [PATCH v9 0/9] qemu-img: add measure sub-command
Posted by Max Reitz 6 years, 8 months ago
On 2017-07-05 14:57, Stefan Hajnoczi wrote:
> v9:
>  * Fix swapped fmt <-> output_fmt parameters in documentation [Max]

Oh, it was just the wrong way around there.

>  * Clarify that backing file clusters also available in the new image
>    file *may* be ommitted from the calculation [Max]

[...]

> Users and management tools sometimes need to know the size required for a new
> disk image so that an LVM volume, SAN LUN, etc can be allocated ahead of time.
> Image formats like qcow2 have non-trivial metadata that makes it hard to
> estimate the exact size without knowledge of file format internals.
> 
> This patch series introduces a new qemu-img sub-command that calculates the
> required size for both image creation and conversion scenarios.
> 
> The conversion scenario is:
> 
>   $ qemu-img measure -f raw -O qcow2 input.img
>   required size: 1327680
>   fully allocated size: 1074069504
> 
> Here an existing image file is taken and the output includes the space required
> for data from the input image file.
> 
> The creation scenario is:
> 
>   $ qemu-img measure -O qcow2 --size 5G
>   required size: 327680
>   fully allocated size: 1074069504
> 
> Stefan Hajnoczi (9):
>   block: add bdrv_measure() API
>   raw-format: add bdrv_measure() support
>   qcow2: extract preallocation calculation function
>   qcow2: make refcount size calculation conservative
>   qcow2: extract image creation option parsing
>   qcow2: add bdrv_measure() support
>   qemu-img: add measure subcommand
>   qemu-iotests: support per-format golden output files
>   iotests: add test 178 for qemu-img measure
> 
>  qapi/block-core.json             |  25 +++
>  include/block/block.h            |   2 +
>  include/block/block_int.h        |   2 +
>  block.c                          |  35 ++++
>  block/qcow2.c                    | 383 +++++++++++++++++++++++++++++----------
>  block/raw-format.c               |  26 +++
>  qemu-img.c                       | 234 ++++++++++++++++++++++++
>  qemu-img-cmds.hx                 |   6 +
>  qemu-img.texi                    |  30 +++
>  tests/qemu-iotests/178           | 170 +++++++++++++++++
>  tests/qemu-iotests/178.out.qcow2 | 286 +++++++++++++++++++++++++++++
>  tests/qemu-iotests/178.out.raw   | 158 ++++++++++++++++
>  tests/qemu-iotests/check         |   5 +
>  tests/qemu-iotests/group         |   1 +
>  14 files changed, 1268 insertions(+), 95 deletions(-)
>  create mode 100755 tests/qemu-iotests/178
>  create mode 100644 tests/qemu-iotests/178.out.qcow2
>  create mode 100644 tests/qemu-iotests/178.out.raw

Thanks, applied to my block tree:

https://github.com/XanClic/qemu/commits/block

Max