block/qcow2-cache.c | 8 +++++ block/qcow2-cluster.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++ block/qcow2-refcount.c | 65 ++++++++++++++++++++++++++++++++++++ block/qcow2.c | 40 ++++++++++++++++------ block/qcow2.h | 4 +++ qapi/block-core.json | 4 ++- tests/qemu-iotests/025 | 19 +++++++++-- tests/qemu-iotests/025.out | 12 ++++++- 8 files changed, 221 insertions(+), 14 deletions(-)
This patch adds the reduction of the image file for qcow2. As a result, this
allows us to reduce the virtual image size and free up space on the disk without
copying the image. Image can be fragmented and reduction is done by punching
holes in the image file.
# ./qemu-img create -f qcow2 -o size=4G image.qcow2
Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
# ./qemu-io -c "write -P 0x22 0 1G" image.qcow2
wrote 1073741824/1073741824 bytes at offset 0
1 GiB, 1 ops; 0:00:02.59 (395.180 MiB/sec and 0.3859 ops/sec)
# ./qemu-img resize image.qcow2 128M
Image resized.
# qemu-img info image.qcow2
image: image.qcow2
file format: qcow2
virtual size: 128M (134217728 bytes)
disk size: 128M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
# du -h image.qcow2
129M image.qcow2
Pavel Butsykin (2):
qcow2: add reduce image support
qemu-iotests: add reducing image test in 025
block/qcow2-cache.c | 8 +++++
block/qcow2-cluster.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++
block/qcow2-refcount.c | 65 ++++++++++++++++++++++++++++++++++++
block/qcow2.c | 40 ++++++++++++++++------
block/qcow2.h | 4 +++
qapi/block-core.json | 4 ++-
tests/qemu-iotests/025 | 19 +++++++++--
tests/qemu-iotests/025.out | 12 ++++++-
8 files changed, 221 insertions(+), 14 deletions(-)
--
2.11.0
On 05/31/2017 09:43 AM, Pavel Butsykin wrote: > This patch adds the reduction of the image file for qcow2. As a result, this > allows us to reduce the virtual image size and free up space on the disk without > copying the image. Image can be fragmented and reduction is done by punching > holes in the image file. > > # ./qemu-img create -f qcow2 -o size=4G image.qcow2 > Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 > > # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 So this is 1G of guest-visible data... > # ./qemu-img resize image.qcow2 128M > Image resized. ...and you are truncating the image by throwing away guest-visible content, with no warning or double-checking (such as requiring a -f force parameter or something) about the data loss. Shrinking images is something we should allow, but it feels like is a rare enough operation that you don't want it to be casually easy to throw away data. Is it feasible to require that a shrink operation will not be performed unless all clusters being eliminated have been previously discarded (or maybe written to zero), as an assurance that the guest did not care about the tail of the image? -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On 31.05.2017 18:03, Eric Blake wrote: > On 05/31/2017 09:43 AM, Pavel Butsykin wrote: >> This patch adds the reduction of the image file for qcow2. As a result, this >> allows us to reduce the virtual image size and free up space on the disk without >> copying the image. Image can be fragmented and reduction is done by punching >> holes in the image file. >> >> # ./qemu-img create -f qcow2 -o size=4G image.qcow2 >> Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 >> >> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 > > So this is 1G of guest-visible data... > >> # ./qemu-img resize image.qcow2 128M >> Image resized. > > ...and you are truncating the image by throwing away guest-visible > content, with no warning or double-checking (such as requiring a -f > force parameter or something) about the data loss. Shrinking images is > something we should allow, but it feels like is a rare enough operation > that you don't want it to be casually easy to throw away data. It is assumed that the user has already made a preparatory with the image: 1. freeing space at the end of the image 2. reducing the last partition on the disk 3. rebuilding fs Only after these steps, the user can reduce the image by qemu-img. I think it's not so rare case, sometimes people run out of disk space and this is another way to solve the problem (along with the use of trim). We already have all the interfaces, left them only to support :) > Is it feasible to require that a shrink operation will not be performed > unless all clusters being eliminated have been previously discarded (or > maybe written to zero), as an assurance that the guest did not care > about the tail of the image? > Yes. # ./qemu-img create -f qcow2 -o size=4G image.qcow2 # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 # ./qemu-io -c "write -P 0x22 1G 1G" image.qcow2 # qemu-img map ./image.qcow2 Offset Length Mapped to File 0 0x20000000 0x50000 ./image.qcow2 0x20000000 0x20000000 0x20060000 ./image.qcow2 0x40000000 0x20000000 0x40070000 ./image.qcow2 0x60000000 0x20000000 0x60080000 ./image.qcow2 # ./qemu-io -c "discard 1G 1G" ./image.qcow2 # qemu-img map ./image.qcow2 Offset Length Mapped to File0 0x20000000 0x50000 ./image.qcow2 0x20000000 0x20000000 0x20060000 ./image.qcow2
On 2017-05-31 17:54, Pavel Butsykin wrote: > On 31.05.2017 18:03, Eric Blake wrote: >> On 05/31/2017 09:43 AM, Pavel Butsykin wrote: >>> This patch adds the reduction of the image file for qcow2. As a >>> result, this >>> allows us to reduce the virtual image size and free up space on the >>> disk without >>> copying the image. Image can be fragmented and reduction is done by >>> punching >>> holes in the image file. >>> >>> # ./qemu-img create -f qcow2 -o size=4G image.qcow2 >>> Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off >>> cluster_size=65536 lazy_refcounts=off refcount_bits=16 >>> >>> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 >> >> So this is 1G of guest-visible data... >> >>> # ./qemu-img resize image.qcow2 128M >>> Image resized. >> >> ...and you are truncating the image by throwing away guest-visible >> content, with no warning or double-checking (such as requiring a -f >> force parameter or something) about the data loss. Shrinking images is >> something we should allow, but it feels like is a rare enough operation >> that you don't want it to be casually easy to throw away data. > > It is assumed that the user has already made a preparatory with the > image: > 1. freeing space at the end of the image > 2. reducing the last partition on the disk > 3. rebuilding fs > Only after these steps, the user can reduce the image by qemu-img. But your implementation allows the user to reduce it anyway, even if these steps have not been performed. I very much agree that we have to be careful because otherwise you might ruin all your data if you hand-type a resize command and drop a digit. > I think it's not so rare case, sometimes people run out of disk space > and this is another way to solve the problem (along with the use of > trim). > > We already have all the interfaces, left them only to support :) > >> Is it feasible to require that a shrink operation will not be performed >> unless all clusters being eliminated have been previously discarded (or >> maybe written to zero), as an assurance that the guest did not care >> about the tail of the image? >> > > Yes. > > # ./qemu-img create -f qcow2 -o size=4G image.qcow2 > > # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 > # ./qemu-io -c "write -P 0x22 1G 1G" image.qcow2 > > # qemu-img map ./image.qcow2 > Offset Length Mapped to File > 0 0x20000000 0x50000 ./image.qcow2 > 0x20000000 0x20000000 0x20060000 ./image.qcow2 > 0x40000000 0x20000000 0x40070000 ./image.qcow2 > 0x60000000 0x20000000 0x60080000 ./image.qcow2 > > # ./qemu-io -c "discard 1G 1G" ./image.qcow2 > > # qemu-img map ./image.qcow2 > Offset Length Mapped to File0 0x20000000 > 0x50000 ./image.qcow2 > 0x20000000 0x20000000 0x20060000 ./image.qcow2 No, it isn't, because qemu-io is a debugging tool and a debugging tool only. We could require the user to perform a trim operation or something in the guest instead -- but I'd prefer a plain new flag for qemu-img resize that says the user is OK with shrinking the image and thus throwing data way. I think it's fine to have this flag only as part of the qemu-img interface, not e.g. for the block-resize QMP command. I think it's reasonable to assume someone sending a QMP command (i.e. usually the management layer) to know exactly what they are doing. OTOH, I wouldn't oppose a flag there, though, I just don't think it's absolutely necessary. Max
On 31.05.2017 19:03, Max Reitz wrote: > On 2017-05-31 17:54, Pavel Butsykin wrote: >> On 31.05.2017 18:03, Eric Blake wrote: >>> On 05/31/2017 09:43 AM, Pavel Butsykin wrote: >>>> This patch adds the reduction of the image file for qcow2. As a >>>> result, this >>>> allows us to reduce the virtual image size and free up space on the >>>> disk without >>>> copying the image. Image can be fragmented and reduction is done by >>>> punching >>>> holes in the image file. >>>> >>>> # ./qemu-img create -f qcow2 -o size=4G image.qcow2 >>>> Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off >>>> cluster_size=65536 lazy_refcounts=off refcount_bits=16 >>>> >>>> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 >>> >>> So this is 1G of guest-visible data... >>> >>>> # ./qemu-img resize image.qcow2 128M >>>> Image resized. >>> >>> ...and you are truncating the image by throwing away guest-visible >>> content, with no warning or double-checking (such as requiring a -f >>> force parameter or something) about the data loss. Shrinking images is >>> something we should allow, but it feels like is a rare enough operation >>> that you don't want it to be casually easy to throw away data. >> >> It is assumed that the user has already made a preparatory with the >> image: >> 1. freeing space at the end of the image >> 2. reducing the last partition on the disk >> 3. rebuilding fs >> Only after these steps, the user can reduce the image by qemu-img. > > But your implementation allows the user to reduce it anyway, even if > these steps have not been performed. > > I very much agree that we have to be careful because otherwise you might > ruin all your data if you hand-type a resize command and drop a digit. > We could check that the shrinking part of the image doesn't contain non-zero clusters and print just a warning. But on the other hand, if the user has not performed trim, at the end of the disk will still be dirty cluster and we can't force users to do trim :) We can add a flag --force and without flag just print a warning. >> I think it's not so rare case, sometimes people run out of disk space >> and this is another way to solve the problem (along with the use of >> trim). >> >> We already have all the interfaces, left them only to support :) >> >>> Is it feasible to require that a shrink operation will not be performed >>> unless all clusters being eliminated have been previously discarded (or >>> maybe written to zero), as an assurance that the guest did not care >>> about the tail of the image? >>> >> >> Yes. >> >> # ./qemu-img create -f qcow2 -o size=4G image.qcow2 >> >> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 >> # ./qemu-io -c "write -P 0x22 1G 1G" image.qcow2 >> >> # qemu-img map ./image.qcow2 >> Offset Length Mapped to File >> 0 0x20000000 0x50000 ./image.qcow2 >> 0x20000000 0x20000000 0x20060000 ./image.qcow2 >> 0x40000000 0x20000000 0x40070000 ./image.qcow2 >> 0x60000000 0x20000000 0x60080000 ./image.qcow2 >> >> # ./qemu-io -c "discard 1G 1G" ./image.qcow2 >> >> # qemu-img map ./image.qcow2 >> Offset Length Mapped to File0 0x20000000 >> 0x50000 ./image.qcow2 >> 0x20000000 0x20000000 0x20060000 ./image.qcow2 > > No, it isn't, because qemu-io is a debugging tool and a debugging tool only. > > We could require the user to perform a trim operation or something in > the guest instead -- but I'd prefer a plain new flag for qemu-img resize > that says the user is OK with shrinking the image and thus throwing data > way. > > I think it's fine to have this flag only as part of the qemu-img > interface, not e.g. for the block-resize QMP command. I think it's > reasonable to assume someone sending a QMP command (i.e. usually the > management layer) to know exactly what they are doing. OTOH, I wouldn't > oppose a flag there, though, I just don't think it's absolutely necessary. > I agree that the flag can be only as protection from the human factor. > Max >
On Wed, May 31, 2017 at 06:54:33PM +0300, Pavel Butsykin wrote: > It is assumed that the user has already made a preparatory with the > image: > 1. freeing space at the end of the image > 2. reducing the last partition on the disk > 3. rebuilding fs > Only after these steps, the user can reduce the image by qemu-img. It's tricky with GPT, as GPT puts a second copy of the header in the last few sectors of the disk. I always advise people to use trim (or virt-sparsify) instead of trying to adjust virtual disk size. Nevertheless I don't think we should prevent shrinking qcow2 virtual size. It's likely useful to someone. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On 31.05.2017 19:10, Richard W.M. Jones wrote: > On Wed, May 31, 2017 at 06:54:33PM +0300, Pavel Butsykin wrote: >> It is assumed that the user has already made a preparatory with the >> image: >> 1. freeing space at the end of the image >> 2. reducing the last partition on the disk >> 3. rebuilding fs >> Only after these steps, the user can reduce the image by qemu-img. > > It's tricky with GPT, as GPT puts a second copy of the header in the > last few sectors of the disk. I always advise people to use trim (or > virt-sparsify) instead of trying to adjust virtual disk size. virt-sparsify is a very useful utility, I use it too :) I agree that the trim is much better if the goal is only to free up space on the host disk. But there's another goal is to limit further growth of the image. > > Nevertheless I don't think we should prevent shrinking qcow2 virtual > size. It's likely useful to someone. > > Rich. >
Am 31.05.2017 um 17:03 hat Eric Blake geschrieben: > On 05/31/2017 09:43 AM, Pavel Butsykin wrote: > > This patch adds the reduction of the image file for qcow2. As a result, this > > allows us to reduce the virtual image size and free up space on the disk without > > copying the image. Image can be fragmented and reduction is done by punching > > holes in the image file. > > > > # ./qemu-img create -f qcow2 -o size=4G image.qcow2 > > Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 > > > > # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 > > So this is 1G of guest-visible data... > > > # ./qemu-img resize image.qcow2 128M > > Image resized. > > ...and you are truncating the image by throwing away guest-visible > content, with no warning or double-checking (such as requiring a -f > force parameter or something) about the data loss. Shrinking images is > something we should allow, but it feels like is a rare enough operation > that you don't want it to be casually easy to throw away data. > > Is it feasible to require that a shrink operation will not be performed > unless all clusters being eliminated have been previously discarded (or > maybe written to zero), as an assurance that the guest did not care > about the tail of the image? I think that ship has sailed long ago because raw images have always supported shrinking images without any special checks or options. We want consistency between raw and qcow2, and we also need to provide backwards compatibility. The only thing I can imagine we could do now is to introduce a --shrink option in qemu-img, print a deprecation warning for shrinking without this option, and then make it mandatory in a few years. If we want to distinguish based on the block driver, so that we can require --shrink for all drivers that didn't support shrinking until now, we'd have to check the .bdrv_truncate() implementations of all drivers to see whether it already allowed shrinking. Kevin
On 06/01/2017 12:12 PM, Kevin Wolf wrote: > Am 31.05.2017 um 17:03 hat Eric Blake geschrieben: >> On 05/31/2017 09:43 AM, Pavel Butsykin wrote: >>> This patch adds the reduction of the image file for qcow2. As a result, this >>> allows us to reduce the virtual image size and free up space on the disk without >>> copying the image. Image can be fragmented and reduction is done by punching >>> holes in the image file. >>> >>> # ./qemu-img create -f qcow2 -o size=4G image.qcow2 >>> Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 >>> >>> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 >> So this is 1G of guest-visible data... >> >>> # ./qemu-img resize image.qcow2 128M >>> Image resized. >> ...and you are truncating the image by throwing away guest-visible >> content, with no warning or double-checking (such as requiring a -f >> force parameter or something) about the data loss. Shrinking images is >> something we should allow, but it feels like is a rare enough operation >> that you don't want it to be casually easy to throw away data. >> >> Is it feasible to require that a shrink operation will not be performed >> unless all clusters being eliminated have been previously discarded (or >> maybe written to zero), as an assurance that the guest did not care >> about the tail of the image? > I think that ship has sailed long ago because raw images have always > supported shrinking images without any special checks or options. We > want consistency between raw and qcow2, and we also need to provide > backwards compatibility. > > The only thing I can imagine we could do now is to introduce a --shrink > option in qemu-img, print a deprecation warning for shrinking without > this option, and then make it mandatory in a few years. > > If we want to distinguish based on the block driver, so that we can > require --shrink for all drivers that didn't support shrinking until > now, we'd have to check the .bdrv_truncate() implementations of all > drivers to see whether it already allowed shrinking. > > Kevin Will the solution proposed by Pavel in the answer to Max work: - if there are no data blocks in the truncated space, truncate is allowed without additional options - if there are data blocks in the truncated space, truncate is allowed with the flag --force or error is generated Den
Am 01.06.2017 um 13:11 hat Denis V. Lunev geschrieben: > On 06/01/2017 12:12 PM, Kevin Wolf wrote: > > Am 31.05.2017 um 17:03 hat Eric Blake geschrieben: > >> On 05/31/2017 09:43 AM, Pavel Butsykin wrote: > >>> This patch adds the reduction of the image file for qcow2. As a result, this > >>> allows us to reduce the virtual image size and free up space on the disk without > >>> copying the image. Image can be fragmented and reduction is done by punching > >>> holes in the image file. > >>> > >>> # ./qemu-img create -f qcow2 -o size=4G image.qcow2 > >>> Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 > >>> > >>> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 > >> So this is 1G of guest-visible data... > >> > >>> # ./qemu-img resize image.qcow2 128M > >>> Image resized. > >> ...and you are truncating the image by throwing away guest-visible > >> content, with no warning or double-checking (such as requiring a -f > >> force parameter or something) about the data loss. Shrinking images is > >> something we should allow, but it feels like is a rare enough operation > >> that you don't want it to be casually easy to throw away data. > >> > >> Is it feasible to require that a shrink operation will not be performed > >> unless all clusters being eliminated have been previously discarded (or > >> maybe written to zero), as an assurance that the guest did not care > >> about the tail of the image? > > I think that ship has sailed long ago because raw images have always > > supported shrinking images without any special checks or options. We > > want consistency between raw and qcow2, and we also need to provide > > backwards compatibility. > > > > The only thing I can imagine we could do now is to introduce a --shrink > > option in qemu-img, print a deprecation warning for shrinking without > > this option, and then make it mandatory in a few years. > > > > If we want to distinguish based on the block driver, so that we can > > require --shrink for all drivers that didn't support shrinking until > > now, we'd have to check the .bdrv_truncate() implementations of all > > drivers to see whether it already allowed shrinking. > > > > Kevin > Will the solution proposed by Pavel in the answer to Max work: > - if there are no data blocks in the truncated space, truncate is allowed > without additional options > - if there are data blocks in the truncated space, truncate is allowed > with the flag --force or error is generated I don't think that's a great user interface because it means that you can only shrink images without --force if you use a guest device that supports discard (i.e. not virtio-blk) and your guest OS supports it, too. In fact, even with a SCSI device and Linux, I wouldn't be sure offhand how to discard unused space after the last partition. I'd rather call the switch --shrink and require it always. In any case, however, I think 'qemu-img resize' should work consistently across different formats. That is, if we require that there be no data blocks in the truncated space for qcow2, we must require the same for raw. And that's where we come back to backwards compatibility, so it's the same case as I mentioned above: We would have to print a deprecation warning for some years and only then we could make it mandatory. Kevin
On 2017-06-01 13:11, Denis V. Lunev wrote: > On 06/01/2017 12:12 PM, Kevin Wolf wrote: >> Am 31.05.2017 um 17:03 hat Eric Blake geschrieben: >>> On 05/31/2017 09:43 AM, Pavel Butsykin wrote: >>>> This patch adds the reduction of the image file for qcow2. As a result, this >>>> allows us to reduce the virtual image size and free up space on the disk without >>>> copying the image. Image can be fragmented and reduction is done by punching >>>> holes in the image file. >>>> >>>> # ./qemu-img create -f qcow2 -o size=4G image.qcow2 >>>> Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 >>>> >>>> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 >>> So this is 1G of guest-visible data... >>> >>>> # ./qemu-img resize image.qcow2 128M >>>> Image resized. >>> ...and you are truncating the image by throwing away guest-visible >>> content, with no warning or double-checking (such as requiring a -f >>> force parameter or something) about the data loss. Shrinking images is >>> something we should allow, but it feels like is a rare enough operation >>> that you don't want it to be casually easy to throw away data. >>> >>> Is it feasible to require that a shrink operation will not be performed >>> unless all clusters being eliminated have been previously discarded (or >>> maybe written to zero), as an assurance that the guest did not care >>> about the tail of the image? >> I think that ship has sailed long ago because raw images have always >> supported shrinking images without any special checks or options. We >> want consistency between raw and qcow2, and we also need to provide >> backwards compatibility. >> >> The only thing I can imagine we could do now is to introduce a --shrink >> option in qemu-img, print a deprecation warning for shrinking without >> this option, and then make it mandatory in a few years. Do I hear a "3.0"? >> If we want to distinguish based on the block driver, so that we can >> require --shrink for all drivers that didn't support shrinking until >> now, we'd have to check the .bdrv_truncate() implementations of all >> drivers to see whether it already allowed shrinking. We could have an ugly raw-only hack directly in qemu-img (and qmp_block_resize) until 3.0. I'm really concerned about someone mistyping something and accidentally dropping a digit. >> Kevin > Will the solution proposed by Pavel in the answer to Max work: > - if there are no data blocks in the truncated space, truncate is allowed > without additional options > - if there are data blocks in the truncated space, truncate is allowed > with the flag --force or error is generated I'd be OK with that, but I'm not sure we really need it. It would be nice to have for convenience, but I don't think it solves the backwards-compatibility problem (as said by Kevin), and I'm not sure it makes things that much more convenient: Just specifying --force is easier than to manually trim the device in the guest (and then maybe having to specify --force anyway because you didn't do it right). Max
Am 07.06.2017 um 15:37 hat Max Reitz geschrieben: > On 2017-06-01 13:11, Denis V. Lunev wrote: > > On 06/01/2017 12:12 PM, Kevin Wolf wrote: > >> Am 31.05.2017 um 17:03 hat Eric Blake geschrieben: > >>> On 05/31/2017 09:43 AM, Pavel Butsykin wrote: > >>>> This patch adds the reduction of the image file for qcow2. As a result, this > >>>> allows us to reduce the virtual image size and free up space on the disk without > >>>> copying the image. Image can be fragmented and reduction is done by punching > >>>> holes in the image file. > >>>> > >>>> # ./qemu-img create -f qcow2 -o size=4G image.qcow2 > >>>> Formatting 'image.qcow2', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 > >>>> > >>>> # ./qemu-io -c "write -P 0x22 0 1G" image.qcow2 > >>> So this is 1G of guest-visible data... > >>> > >>>> # ./qemu-img resize image.qcow2 128M > >>>> Image resized. > >>> ...and you are truncating the image by throwing away guest-visible > >>> content, with no warning or double-checking (such as requiring a -f > >>> force parameter or something) about the data loss. Shrinking images is > >>> something we should allow, but it feels like is a rare enough operation > >>> that you don't want it to be casually easy to throw away data. > >>> > >>> Is it feasible to require that a shrink operation will not be performed > >>> unless all clusters being eliminated have been previously discarded (or > >>> maybe written to zero), as an assurance that the guest did not care > >>> about the tail of the image? > >> I think that ship has sailed long ago because raw images have always > >> supported shrinking images without any special checks or options. We > >> want consistency between raw and qcow2, and we also need to provide > >> backwards compatibility. > >> > >> The only thing I can imagine we could do now is to introduce a --shrink > >> option in qemu-img, print a deprecation warning for shrinking without > >> this option, and then make it mandatory in a few years. > > Do I hear a "3.0"? You do. > >> If we want to distinguish based on the block driver, so that we can > >> require --shrink for all drivers that didn't support shrinking until > >> now, we'd have to check the .bdrv_truncate() implementations of all > >> drivers to see whether it already allowed shrinking. > > We could have an ugly raw-only hack directly in qemu-img (and > qmp_block_resize) until 3.0. > > I'm really concerned about someone mistyping something and accidentally > dropping a digit. Me too. But still we can't break working command lines (at least before 3.0). I'm okay with temporary hacks in qemu-img, but did you check whether it's really raw-only? We know that raw can shrink currently and qcow2 can't, but there are 12 drivers implementing .bdrv_truncate, not only two. > > Will the solution proposed by Pavel in the answer to Max work: > > - if there are no data blocks in the truncated space, truncate is allowed > > without additional options > > - if there are data blocks in the truncated space, truncate is allowed > > with the flag --force or error is generated > > I'd be OK with that, but I'm not sure we really need it. Agreed. It would add a lot of complexity for little use. As I wrote in a previous mail: I don't even know how I would discard the unpartitioned space after shrinking my guest filesystem. > It would be nice to have for convenience, but I don't think it solves > the backwards-compatibility problem (as said by Kevin), and I'm not > sure it makes things that much more convenient: Just specifying > --force is easier than to manually trim the device in the guest (and > then maybe having to specify --force anyway because you didn't do it > right). I think it should be specifically --shrink rather than an unspecific --force that could mean anything. Kevin
© 2016 - 2024 Red Hat, Inc.