On Thu, Jan 31, 2019 at 7:57 PM Kevin Wolf <kwolf@redhat.com> wrote:
This will be very useful for new oVirt Cinder based storage. Thanks for
working on this!
I did not see any discussion about this here, but I did not follow this
list closely
lately. Do we have more info on this? a feature page describing the use
cases
and the limitations?
There are use cases where raw images are given (e.g. existing physical
> disks), but advanced features like dirty bitmaps or backing files are
> wanted that require use of a proper image format like qcow2.
>
> This series adds an incompatible feature bit to qcow2 which allows to
> use an external data file: Metadata is kept in the qcow2 file like
> usual, but guest data is written to an external file. Clusters in the
> data file are not reference counted, instead we use a flat layout where
> host cluster offset == guest cluster offset. The external data file is
> therefore readable as a raw image (though writing to it invalidates the
> associated qcow2 metadata). Features that require refcounting such as
> internal snapshots or compression are not supposed in such setups.
>
Makes sense. In oVirt we would like to use this only for raw images on
legacy
storage (no snapshot), or on LUNs managed via cinderlib, when the underlying
storage provides snapshots and thin provisioning.
If a user wants snapshot on legacy storage, they can add a qcow2 layer and
use incremental backup in this layer.
>
> There are a few reasons why this is still RFC:
>
> - The resulting code passes qemu-iotests, so we don't regress on normal
> qcow2 files, but testing with external data files is still minimal
> (converting an existing image and confirming it reads back unmodified;
> installing a guest OS and making sure it boots). We need at least some
> qemu-iotests cases.
>
> - QAPI documentation is missing
>
> - Discard isn't passed through to the data file yet
>
> - s->image_data_file isn't correct, it gets the value from the attached
> node rather than just from the image file. This means that on header
> updates we might be writing "back" a path that wasn't there before.
>
> - Probably something else I just can't remember now :-)
>
I think the most interesting questions regarding this is how we bind a raw
image
to the metadata qcow2 image, so the image cannot be used outside of a
management
system. I don't think this would safe enough without such mechanism.
I think Fam suggested in the past to encrypt the some part of the image,
e.g the first
cluster, so it is not usable without the qcow2 image. The management
system should
be able to detach a raw image from the qcow2 image using qemu-img, which
will invalidate
the metadata in the attached qcow2 image, and make the image usable outside
of the system.
If an attached image is replicated to another storage, the the original
qcow2 image is lost,
a user should be able to use qemu-img to detach the image, making it usable
outside the
system, or use qemu-img to attach is to a new qcow2 layer, allowing the
system to use the
image again.
Another question is how you attach existing raw image to qcow2 layer,
enabling incremental
backup with this volume. Do start recording changes from the point of the
attachment, so
anything not in the qcow2 layer will be available when using full backup?
Nir
>
> Kevin Wolf (11):
> qcow2: Extend spec for external data files
> qcow2: Basic definitions for external data files
> qcow2: Pass bs to qcow2_get_cluster_type()
> qcow2: Prepare qcow2_get_cluster_type() for external data file
> qcow2: Prepare count_contiguous_clusters() for external data file
> qcow2: Don't assume 0 is an invalid cluster offset
> qcow2: External file I/O
> qcow2: Add basic data-file infrastructure
> qcow2: Creating images with external data file
> qcow2: Store data file name in the image
> qcow2: Add data file to ImageInfoSpecificQCow2
>
> qapi/block-core.json | 5 +-
> docs/interop/qcow2.txt | 19 ++++-
> block/qcow2.h | 40 +++++++--
> include/block/block_int.h | 1 +
> block/qcow2-bitmap.c | 7 +-
> block/qcow2-cache.c | 6 +-
> block/qcow2-cluster.c | 144 +++++++++++++++++++-------------
> block/qcow2-refcount.c | 40 ++++++---
> block/qcow2-snapshot.c | 7 +-
> block/qcow2.c | 166 ++++++++++++++++++++++++++++++++++---
> tests/qemu-iotests/031.out | 8 +-
> tests/qemu-iotests/036.out | 4 +-
> tests/qemu-iotests/061.out | 14 ++--
> tests/qemu-iotests/082.out | 27 ++++++
> 14 files changed, 372 insertions(+), 116 deletions(-)
>
> --
> 2.20.1
>
>
>