[PATCH v6 0/4] Add full zoned storage emulation to qcow2 driver

Sam Li posted 4 patches 8 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20231127043703.49489-1-faithilikerun@gmail.com
Maintainers: Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Eric Blake <eblake@redhat.com>, Markus Armbruster <armbru@redhat.com>
There is a newer version of this series
block/qcow2.c                            | 972 ++++++++++++++++++++++-
block/qcow2.h                            |  36 +-
block/trace-events                       |   2 +
docs/interop/qcow2.txt                   |  99 ++-
docs/system/qemu-block-drivers.rst.inc   |  35 +
include/block/block_int-common.h         |  13 +
include/qemu/queue.h                     |   1 +
qapi/block-core.json                     |  63 +-
tests/qemu-iotests/tests/zoned-qcow2     | 126 +++
tests/qemu-iotests/tests/zoned-qcow2.out | 118 +++
10 files changed, 1460 insertions(+), 5 deletions(-)
create mode 100755 tests/qemu-iotests/tests/zoned-qcow2
create mode 100644 tests/qemu-iotests/tests/zoned-qcow2.out
[PATCH v6 0/4] Add full zoned storage emulation to qcow2 driver
Posted by Sam Li 8 months ago
This patch series add a new extension - zoned format - to the
qcow2 driver thereby allowing full zoned storage emulation on
the qcow2 img file. Users can attach such a qcow2 file to the
guest as a zoned device.

Write pointer are preserved in the zoned metadata. It will be
recovered after power cycle. Meanwhile, any open (implicit or
explicit) zone will show up as closed.

Zone states are in memory. Read-only and offline states are
device-internal events, which are not considerred in qcow2
emulation for simplicity. The other zone states
(closed, empty, full) can be inferred from write poiner
values, presistent across QEMU reboots. The open states are
kept in memory using open zone lists.

Zoned extension feature is optional. We only set it to host-manged 
when emulating a zoned device. For non-zoned devices, it does not 
need to consider setting this option.

To create a qcow2 image with zoned format feature, use command like
this:
$path/to/qemu-img create -f qcow2 zbc.qcow2 -o size=768M
-o zone.size=64M -o zone.capacity=64M -o zone.conventional_zones=0
-o zone.max_append_bytes=4096 -o zone.max_open_zones=10
-o zone.max_active_zones=12 -o zone.mode=host-managed


Then add it to the QEMU command line:
    -blockdev node-name=drive1,driver=qcow2,file.driver=file,file.filename=../qemu/test.qcow2 \
    -device virtio-blk-pci,drive=drive1 \

v5->v6:
- fix docs and specs [Eric, Markus, Stefan]
- add general sanity checks for zoned device configurations while creation and opening [Eric]
- fix LRU when implicitly open a zone for a long time [Stefan]

v4->v5:
- add incompatible bit for zoned format [Eric]
- fix and manage zone resources via LRU [Damien]
- renaming functions and fields, spec changes [Markus, Damien]
- add closed zone list
- make qemu iotests for zoned device consecutive [Stefan]

v3->v4:
- use QLIST for implicit, explicit open zones management [Stefan]
- keep zone states in memory and drop state bits in wp metadata structure [Damien, Stefan]
- change zone resource management and iotests accordingly
- add tracing for number of implicit zones
- address review comments [Stefan, Markus]:
  * documentation, config, style

v2->v3:
- drop zoned_profile option [Klaus]
- reformat doc comments of qcow2 [Markus]
- add input validation and checks for zoned information [Stefan]
- code style: format, comments, documentation, naming [Stefan]
- add tracing function for wp tracking [Stefan]
- reconstruct io path in check_zone_resources [Stefan]

v1->v2:
- add more tests to qemu-io zoned commands
- make zone append change state to full when wp reaches end
- add documentation to qcow2 zoned extension header
- address review comments (Stefan):
  * fix zoned_mata allocation size
  * use bitwise or than addition
  * fix wp index overflow and locking
  * cleanups: comments, naming

Sam Li (4):
  docs/qcow2: add the zoned format feature
  qcow2: add configurations for zoned format extension
  qcow2: add zoned emulation capability
  iotests: test the zoned format feature for qcow2 file

 block/qcow2.c                            | 972 ++++++++++++++++++++++-
 block/qcow2.h                            |  36 +-
 block/trace-events                       |   2 +
 docs/interop/qcow2.txt                   |  99 ++-
 docs/system/qemu-block-drivers.rst.inc   |  35 +
 include/block/block_int-common.h         |  13 +
 include/qemu/queue.h                     |   1 +
 qapi/block-core.json                     |  63 +-
 tests/qemu-iotests/tests/zoned-qcow2     | 126 +++
 tests/qemu-iotests/tests/zoned-qcow2.out | 118 +++
 10 files changed, 1460 insertions(+), 5 deletions(-)
 create mode 100755 tests/qemu-iotests/tests/zoned-qcow2
 create mode 100644 tests/qemu-iotests/tests/zoned-qcow2.out

-- 
2.40.1
Re: [PATCH v6 0/4] Add full zoned storage emulation to qcow2 driver
Posted by Markus Armbruster 7 months, 4 weeks ago
Clang reports

../block/qcow2.c:4066:5: error: mutex 'graph_lock' is not held on every path through here [-Werror,-Wthread-safety-analysis]
    blk_co_unref(blk);
    ^
../block/qcow2.c:3928:5: note: mutex acquired here
    bdrv_graph_co_rdlock();
    ^
../block/qcow2.c:4066:5: error: mutex 'graph_lock' is not held on every path through here [-Werror,-Wthread-safety-analysis]
    blk_co_unref(blk);
    ^
../block/qcow2.c:3928:5: note: mutex acquired here
    bdrv_graph_co_rdlock();
    ^
2 errors generated.
Re: [PATCH v6 0/4] Add full zoned storage emulation to qcow2 driver
Posted by Sam Li 7 months, 1 week ago
Markus Armbruster <armbru@redhat.com> 于2023年11月30日周四 09:25写道:
>
> Clang reports
>
> ../block/qcow2.c:4066:5: error: mutex 'graph_lock' is not held on every path through here [-Werror,-Wthread-safety-analysis]
>     blk_co_unref(blk);
>     ^
> ../block/qcow2.c:3928:5: note: mutex acquired here
>     bdrv_graph_co_rdlock();
>     ^
> ../block/qcow2.c:4066:5: error: mutex 'graph_lock' is not held on every path through here [-Werror,-Wthread-safety-analysis]
>     blk_co_unref(blk);
>     ^
> ../block/qcow2.c:3928:5: note: mutex acquired here
>     bdrv_graph_co_rdlock();
>     ^
> 2 errors generated.
>

Turns out that my gcc 12.0 does not support -Wthread-safety-analysis
flag. Need to use --cc=clang to reproduce it. Thanks!

Sam