[Qemu-devel] [PATCH 0/3] block/file-posix: File locking during creation

Max Reitz posted 3 patches 5 years, 11 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20180420220913.27000-1-mreitz@redhat.com
Test checkpatch passed
Test docker-build@min-glib passed
Test docker-mingw@fedora passed
Test s390x passed
There is a newer version of this series
block/file-posix.c         | 68 ++++++++++++++++++++++++++++++++++++----------
tests/qemu-iotests/153     | 18 ++++++++++++
tests/qemu-iotests/153.out | 13 +++++++++
3 files changed, 84 insertions(+), 15 deletions(-)
[Qemu-devel] [PATCH 0/3] block/file-posix: File locking during creation
Posted by Max Reitz 5 years, 11 months ago
Currently we do not take permissions on a file while it is being
created.  That is a bit sad.  The simplest way to test this is the
following:

    $ qemu-img create -f qcow2 foo.qcow2 64M
    Formatting 'foo.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16
    $ qemu-img convert -f qcow2 -O qcow2 foo.qcow2 foo.qcow2
    qemu-img: foo.qcow2: error while converting qcow2: Failed to get "write" lock
    Is another process using the image?
    $ qemu-img info foo.qcow2
    image: foo.qcow2
    file format: raw
    virtual size: 0 (0 bytes)
    disk size: 0

(See also https://bugzilla.redhat.com/show_bug.cgi?id=1519144)

Here is what's happening: file-posix opens the file with
O_CREAT | O_TRUNC, thus truncating the file to 0.  Then qcow2 tries to
format it, but fails because it needs the WRITE permission to do so.  So
it cannot format it and the file stays empty.

We should actually take a WRITE and a RESIZE permission before we
truncate the file, and this is what this series does.

I personally consider the solution taken here a bit of a hack, but it
works and we don't have any locking in any drivers but file-posix
anyway, so it isn't lacking anything in that regard.  Integrating it in
blockdev-create might be possible, but there are two issues:
(1) It would be harder.
(2) What would we do anyway?  We'd advise protocol drivers to take WRITE
    and RESIZE permissions before they truncate a file to be empty...
    Well, and then they'd do exactly what file-posix is made to do in
    this series.

So basically what I consider a hack could be seen as exactly the right
way to tackle the issue: Protocol drivers have to ensure the correct
permissions (and they have to choose what those are) are applied before
changing any file -- which is what this series implements.


Max Reitz (3):
  block/file-posix: Pass FD to locking helpers
  block/file-posix: File locking during creation
  iotests: Add creation test to 153

 block/file-posix.c         | 68 ++++++++++++++++++++++++++++++++++++----------
 tests/qemu-iotests/153     | 18 ++++++++++++
 tests/qemu-iotests/153.out | 13 +++++++++
 3 files changed, 84 insertions(+), 15 deletions(-)

-- 
2.14.3


Re: [Qemu-devel] [Qemu-block] [PATCH 0/3] block/file-posix: File locking during creation
Posted by Alberto Garcia 5 years, 11 months ago
On Sat 21 Apr 2018 12:09:10 AM CEST, Max Reitz wrote:
> Currently we do not take permissions on a file while it is being
> created.  That is a bit sad.  The simplest way to test this is the
> following:
>
>     $ qemu-img create -f qcow2 foo.qcow2 64M
>     Formatting 'foo.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16
>     $ qemu-img convert -f qcow2 -O qcow2 foo.qcow2 foo.qcow2
>     qemu-img: foo.qcow2: error while converting qcow2: Failed to get "write" lock
>     Is another process using the image?
>     $ qemu-img info foo.qcow2
>     image: foo.qcow2
>     file format: raw
>     virtual size: 0 (0 bytes)
>     disk size: 0

Not quite the same problem, but here's another example of QEMU creating
an image before checking if it can be created:

$ qemu-img create -o cluster_size=2M -f qcow2 img.qcow2 7E
Formatting 'img.qcow2', fmt=qcow2 size=8070450532247928832 encryption=off cluster_size=2097152 lazy_refcounts=off refcount_bits=16
qemu-img: img.qcow2: The image size is too large for file format 'qcow2' (try using a larger cluster size)

$ qemu-img info img.qcow2 
image: img.qcow2
file format: qcow2
virtual size: 0 (0 bytes)
disk size: 6.0M
cluster_size: 2097152
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Berto

Re: [Qemu-devel] [Qemu-block] [PATCH 0/3] block/file-posix: File locking during creation
Posted by Max Reitz 5 years, 11 months ago
On 2018-04-23 15:19, Alberto Garcia wrote:
> On Sat 21 Apr 2018 12:09:10 AM CEST, Max Reitz wrote:
>> Currently we do not take permissions on a file while it is being
>> created.  That is a bit sad.  The simplest way to test this is the
>> following:
>>
>>     $ qemu-img create -f qcow2 foo.qcow2 64M
>>     Formatting 'foo.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16
>>     $ qemu-img convert -f qcow2 -O qcow2 foo.qcow2 foo.qcow2
>>     qemu-img: foo.qcow2: error while converting qcow2: Failed to get "write" lock
>>     Is another process using the image?
>>     $ qemu-img info foo.qcow2
>>     image: foo.qcow2
>>     file format: raw
>>     virtual size: 0 (0 bytes)
>>     disk size: 0
> 
> Not quite the same problem, but here's another example of QEMU creating
> an image before checking if it can be created:
> 
> $ qemu-img create -o cluster_size=2M -f qcow2 img.qcow2 7E
> Formatting 'img.qcow2', fmt=qcow2 size=8070450532247928832 encryption=off cluster_size=2097152 lazy_refcounts=off refcount_bits=16
> qemu-img: img.qcow2: The image size is too large for file format 'qcow2' (try using a larger cluster size)
> 
> $ qemu-img info img.qcow2 
> image: img.qcow2
> file format: qcow2
> virtual size: 0 (0 bytes)
> disk size: 6.0M
> cluster_size: 2097152
> Format specific information:
>     compat: 1.1
>     lazy refcounts: false
>     refcount bits: 16
>     corrupt: false
> 
> Berto

Actually a very different problem, and I think we are not going to
change this behavior -- especially considering the direction
blockdev-create is taking (that is, separating protocol file creation
and formatting).

The main reason why it's a different problem, though, is because it
doesn't mean any data loss.  You just get a stale file lying around.

(You could argue that in any case the file should at least be a raw file
of size 0, because the qcow2 driver should not start formatting until it
has checked all options, but, well...  I don't know how that would help
anyone.)

Max