This patch series add a new extension - zoned format - to the
qcow2 driver thereby allowing full zoned storage emulation on
the qcow2 img file. Users can attach such a qcow2 file to the
guest as a zoned device.

To create a qcow2 file with zoned format, use command like this:
$ qemu-img create -f qcow2 test.qcow2 -o size=768M -o
zone_size=64M -o zone_capacity=64M -o zone_nr_conv=0 -o
max_append_sectors=512 -o max_open_zones=0 -o max_active_zones=0
-o zoned_profile=zbc

Then add it to the QEMU command line:
    -blockdev node-name=drive1,driver=qcow2,file.driver=file,file.filename=../qemu/test.qcow2 \
    -device virtio-blk-pci,drive=drive1 \

v1->v2:
- add more tests to qemu-io zoned commands
- make zone append change state to full when wp reaches end
- add documentation to qcow2 zoned extension header
- address review comments (Stefan):
  * fix zoned_mata allocation size
  * use bitwise or than addition
  * fix wp index overflow and locking
  * cleanups: comments, naming

Sam Li (4):
  docs/qcow2: add the zoned format feature
  qcow2: add configurations for zoned format extension
  qcow2: add zoned emulation capability
  iotests: test the zoned format feature for qcow2 file

 block/qcow2.c                            | 799 ++++++++++++++++++++++-
 block/qcow2.h                            |  23 +
 docs/interop/qcow2.txt                   |  26 +
 docs/system/qemu-block-drivers.rst.inc   |  39 ++
 include/block/block-common.h             |   5 +
 include/block/block_int-common.h         |  16 +
 qapi/block-core.json                     |  46 +-
 tests/qemu-iotests/tests/zoned-qcow2     | 135 ++++
 tests/qemu-iotests/tests/zoned-qcow2.out | 140 ++++
 9 files changed, 1214 insertions(+), 15 deletions(-)
 create mode 100755 tests/qemu-iotests/tests/zoned-qcow2
 create mode 100644 tests/qemu-iotests/tests/zoned-qcow2.out

-- 
2.40.1

On Aug 14 16:57, Sam Li wrote:
> This patch series add a new extension - zoned format - to the
> qcow2 driver thereby allowing full zoned storage emulation on
> the qcow2 img file. Users can attach such a qcow2 file to the
> guest as a zoned device.
> 
> To create a qcow2 file with zoned format, use command like this:
> $ qemu-img create -f qcow2 test.qcow2 -o size=768M -o
> zone_size=64M -o zone_capacity=64M -o zone_nr_conv=0 -o
> max_append_sectors=512 -o max_open_zones=0 -o max_active_zones=0
> -o zoned_profile=zbc
> 
> Then add it to the QEMU command line:
>     -blockdev node-name=drive1,driver=qcow2,file.driver=file,file.filename=../qemu/test.qcow2 \
>     -device virtio-blk-pci,drive=drive1 \
> 
> v1->v2:
> - add more tests to qemu-io zoned commands
> - make zone append change state to full when wp reaches end
> - add documentation to qcow2 zoned extension header
> - address review comments (Stefan):
>   * fix zoned_mata allocation size
>   * use bitwise or than addition
>   * fix wp index overflow and locking
>   * cleanups: comments, naming
> 
> Sam Li (4):
>   docs/qcow2: add the zoned format feature
>   qcow2: add configurations for zoned format extension
>   qcow2: add zoned emulation capability
>   iotests: test the zoned format feature for qcow2 file
> 
>  block/qcow2.c                            | 799 ++++++++++++++++++++++-
>  block/qcow2.h                            |  23 +
>  docs/interop/qcow2.txt                   |  26 +
>  docs/system/qemu-block-drivers.rst.inc   |  39 ++
>  include/block/block-common.h             |   5 +
>  include/block/block_int-common.h         |  16 +
>  qapi/block-core.json                     |  46 +-
>  tests/qemu-iotests/tests/zoned-qcow2     | 135 ++++
>  tests/qemu-iotests/tests/zoned-qcow2.out | 140 ++++
>  9 files changed, 1214 insertions(+), 15 deletions(-)
>  create mode 100755 tests/qemu-iotests/tests/zoned-qcow2
>  create mode 100644 tests/qemu-iotests/tests/zoned-qcow2.out
> 

Hi Sam,

Thanks for this and for the RFC for hw/nvme - this is an awesome
improvement.

Can you explain the need for the zoned_profile? I understand that only
ZNS requires potentially setting zone_capacity and configuring extended
descriptors. When an image is hooked up to a block emulation device that
doesnt understand cap < size or extended descriptors, it could just
would fail on the cap < size and just ignore the extended descriptor
space. Do we really need to add the complexity of the user explicitly
having to set the profile? I also think it is fair for the QEMU zoned
block api to accomodate both variations - if a particular configuration
is supported or not is up to the emulating device.

Checking the profile from hw/nvme or hw/block/virtio is the same as
checking if cap < size or possibly the presence of extended descriptors.


Thanks,
Klaus

Klaus Jensen <its@irrelevant.dk> 于2023年8月16日周三 15:37写道：
>
> On Aug 14 16:57, Sam Li wrote:
> > This patch series add a new extension - zoned format - to the
> > qcow2 driver thereby allowing full zoned storage emulation on
> > the qcow2 img file. Users can attach such a qcow2 file to the
> > guest as a zoned device.
> >
> > To create a qcow2 file with zoned format, use command like this:
> > $ qemu-img create -f qcow2 test.qcow2 -o size=768M -o
> > zone_size=64M -o zone_capacity=64M -o zone_nr_conv=0 -o
> > max_append_sectors=512 -o max_open_zones=0 -o max_active_zones=0
> > -o zoned_profile=zbc
> >
> > Then add it to the QEMU command line:
> >     -blockdev node-name=drive1,driver=qcow2,file.driver=file,file.filename=../qemu/test.qcow2 \
> >     -device virtio-blk-pci,drive=drive1 \
> >
> > v1->v2:
> > - add more tests to qemu-io zoned commands
> > - make zone append change state to full when wp reaches end
> > - add documentation to qcow2 zoned extension header
> > - address review comments (Stefan):
> >   * fix zoned_mata allocation size
> >   * use bitwise or than addition
> >   * fix wp index overflow and locking
> >   * cleanups: comments, naming
> >
> > Sam Li (4):
> >   docs/qcow2: add the zoned format feature
> >   qcow2: add configurations for zoned format extension
> >   qcow2: add zoned emulation capability
> >   iotests: test the zoned format feature for qcow2 file
> >
> >  block/qcow2.c                            | 799 ++++++++++++++++++++++-
> >  block/qcow2.h                            |  23 +
> >  docs/interop/qcow2.txt                   |  26 +
> >  docs/system/qemu-block-drivers.rst.inc   |  39 ++
> >  include/block/block-common.h             |   5 +
> >  include/block/block_int-common.h         |  16 +
> >  qapi/block-core.json                     |  46 +-
> >  tests/qemu-iotests/tests/zoned-qcow2     | 135 ++++
> >  tests/qemu-iotests/tests/zoned-qcow2.out | 140 ++++
> >  9 files changed, 1214 insertions(+), 15 deletions(-)
> >  create mode 100755 tests/qemu-iotests/tests/zoned-qcow2
> >  create mode 100644 tests/qemu-iotests/tests/zoned-qcow2.out
> >
>
> Hi Sam,
>
> Thanks for this and for the RFC for hw/nvme - this is an awesome
> improvement.
>
> Can you explain the need for the zoned_profile? I understand that only
> ZNS requires potentially setting zone_capacity and configuring extended
> descriptors. When an image is hooked up to a block emulation device that
> doesnt understand cap < size or extended descriptors, it could just
> would fail on the cap < size and just ignore the extended descriptor
> space. Do we really need to add the complexity of the user explicitly
> having to set the profile? I also think it is fair for the QEMU zoned
> block api to accomodate both variations - if a particular configuration
> is supported or not is up to the emulating device.
>
> Checking the profile from hw/nvme or hw/block/virtio is the same as
> checking if cap < size or possibly the presence of extended descriptors.

Hi Klaus,

Thanks for your feedback.

The zoned_profile is for users to choose the emulating device type,
either zbc or zns. It implies using virtio-blk or nvme pass through.
The zoned block api does accommodate both variations. Since the cap <
size and extended descriptor config can also infer zoned_profile, this
option can be dropped. Then the device type is determined by the
configurations. When cap = size and no extended descriptor, the img
can be used both in virtio-blk and nvme zns depending on the QEMU
command line.


Best regards,
Sam

On Wed, Aug 16, 2023 at 04:14:08PM +0800, Sam Li wrote:
> Klaus Jensen <its@irrelevant.dk> 于2023年8月16日周三 15:37写道：
> >
> > On Aug 14 16:57, Sam Li wrote:
> > > This patch series add a new extension - zoned format - to the
> > > qcow2 driver thereby allowing full zoned storage emulation on
> > > the qcow2 img file. Users can attach such a qcow2 file to the
> > > guest as a zoned device.
> > >
> > > To create a qcow2 file with zoned format, use command like this:
> > > $ qemu-img create -f qcow2 test.qcow2 -o size=768M -o
> > > zone_size=64M -o zone_capacity=64M -o zone_nr_conv=0 -o
> > > max_append_sectors=512 -o max_open_zones=0 -o max_active_zones=0
> > > -o zoned_profile=zbc
> > >
> > > Then add it to the QEMU command line:
> > >     -blockdev node-name=drive1,driver=qcow2,file.driver=file,file.filename=../qemu/test.qcow2 \
> > >     -device virtio-blk-pci,drive=drive1 \
> > >
> > > v1->v2:
> > > - add more tests to qemu-io zoned commands
> > > - make zone append change state to full when wp reaches end
> > > - add documentation to qcow2 zoned extension header
> > > - address review comments (Stefan):
> > >   * fix zoned_mata allocation size
> > >   * use bitwise or than addition
> > >   * fix wp index overflow and locking
> > >   * cleanups: comments, naming
> > >
> > > Sam Li (4):
> > >   docs/qcow2: add the zoned format feature
> > >   qcow2: add configurations for zoned format extension
> > >   qcow2: add zoned emulation capability
> > >   iotests: test the zoned format feature for qcow2 file
> > >
> > >  block/qcow2.c                            | 799 ++++++++++++++++++++++-
> > >  block/qcow2.h                            |  23 +
> > >  docs/interop/qcow2.txt                   |  26 +
> > >  docs/system/qemu-block-drivers.rst.inc   |  39 ++
> > >  include/block/block-common.h             |   5 +
> > >  include/block/block_int-common.h         |  16 +
> > >  qapi/block-core.json                     |  46 +-
> > >  tests/qemu-iotests/tests/zoned-qcow2     | 135 ++++
> > >  tests/qemu-iotests/tests/zoned-qcow2.out | 140 ++++
> > >  9 files changed, 1214 insertions(+), 15 deletions(-)
> > >  create mode 100755 tests/qemu-iotests/tests/zoned-qcow2
> > >  create mode 100644 tests/qemu-iotests/tests/zoned-qcow2.out
> > >
> >
> > Hi Sam,
> >
> > Thanks for this and for the RFC for hw/nvme - this is an awesome
> > improvement.
> >
> > Can you explain the need for the zoned_profile? I understand that only
> > ZNS requires potentially setting zone_capacity and configuring extended
> > descriptors. When an image is hooked up to a block emulation device that
> > doesnt understand cap < size or extended descriptors, it could just
> > would fail on the cap < size and just ignore the extended descriptor
> > space. Do we really need to add the complexity of the user explicitly
> > having to set the profile? I also think it is fair for the QEMU zoned
> > block api to accomodate both variations - if a particular configuration
> > is supported or not is up to the emulating device.
> >
> > Checking the profile from hw/nvme or hw/block/virtio is the same as
> > checking if cap < size or possibly the presence of extended descriptors.
> 
> Hi Klaus,
> 
> Thanks for your feedback.
> 
> The zoned_profile is for users to choose the emulating device type,
> either zbc or zns. It implies using virtio-blk or nvme pass through.
> The zoned block api does accommodate both variations. Since the cap <
> size and extended descriptor config can also infer zoned_profile, this
> option can be dropped. Then the device type is determined by the
> configurations. When cap = size and no extended descriptor, the img
> can be used both in virtio-blk and nvme zns depending on the QEMU
> command line.

Dropping zoned_profile would be a nice simplification.

Stefan