[Qemu-devel] [PATCH 0/3] proper locking on bitmap add/remove paths

Vladimir Sementsov-Ogievskiy posted 3 patches 4 years, 7 months ago
Test docker-clang@ubuntu failed
Test FreeBSD passed
Test checkpatch passed
Test docker-quick@centos7 failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20190910162724.79574-1-vsementsov@virtuozzo.com
Maintainers: Max Reitz <mreitz@redhat.com>, John Snow <jsnow@redhat.com>, Markus Armbruster <armbru@redhat.com>, Fam Zheng <fam@euphon.net>, Kevin Wolf <kwolf@redhat.com>
There is a newer version of this series
block/qcow2.h                |  14 ++---
include/block/block_int.h    |  14 ++---
include/block/dirty-bitmap.h |   5 +-
block.c                      |  22 -------
block/dirty-bitmap.c         | 119 +++++++++++++++++++++++++++++++++--
block/qcow2-bitmap.c         |  36 +++++++----
block/qcow2.c                |   5 +-
blockdev.c                   |  28 +++------
8 files changed, 163 insertions(+), 80 deletions(-)
[Qemu-devel] [PATCH 0/3] proper locking on bitmap add/remove paths
Posted by Vladimir Sementsov-Ogievskiy 4 years, 7 months ago
We need to lock qcow2 mutex on accessing in-image metadata, especially
on updating this metadata. Let's implement it.

Vladimir Sementsov-Ogievskiy (3):
  block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c
  block/dirty-bitmap: return int from
    bdrv_remove_persistent_dirty_bitmap
  block/qcow2: proper locking on bitmap add/remove paths

 block/qcow2.h                |  14 ++---
 include/block/block_int.h    |  14 ++---
 include/block/dirty-bitmap.h |   5 +-
 block.c                      |  22 -------
 block/dirty-bitmap.c         | 119 +++++++++++++++++++++++++++++++++--
 block/qcow2-bitmap.c         |  36 +++++++----
 block/qcow2.c                |   5 +-
 blockdev.c                   |  28 +++------
 8 files changed, 163 insertions(+), 80 deletions(-)

-- 
2.18.0


Re: [Qemu-devel] [PATCH 0/3] proper locking on bitmap add/remove paths
Posted by no-reply@patchew.org 4 years, 7 months ago
Patchew URL: https://patchew.org/QEMU/20190910162724.79574-1-vsementsov@virtuozzo.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

libudev           no
default devices   yes

warning: Python 2 support is deprecated
warning: Python 3 will be required for building future versions of QEMU

NOTE: cross-compilers enabled:  'cc'
  GEN     x86_64-softmmu/config-devices.mak.tmp
---
  CC      block/qed-cluster.o
  CC      block/qed-check.o
/tmp/qemu-test/src/block/qcow2-bitmap.c: In function 'qcow2_co_remove_persistent_dirty_bitmap':
/tmp/qemu-test/src/block/qcow2-bitmap.c:502:8: error: 'bm' may be used uninitialized in this function [-Werror=maybe-uninitialized]
     if (bm == NULL) {
        ^
/tmp/qemu-test/src/block/qcow2-bitmap.c:1413:18: note: 'bm' was declared here


The full log is available at
http://patchew.org/logs/20190910162724.79574-1-vsementsov@virtuozzo.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [Qemu-devel] [PATCH 0/3] proper locking on bitmap add/remove paths
Posted by Vladimir Sementsov-Ogievskiy 4 years, 7 months ago
10.09.2019 23:37, no-reply@patchew.org wrote:
> Patchew URL: https://patchew.org/QEMU/20190910162724.79574-1-vsementsov@virtuozzo.com/
> 
> 
> 
> Hi,
> 
> This series failed the docker-quick@centos7 build test. Please find the testing commands and
> their output below. If you have Docker installed, you can probably reproduce it
> locally.
> 
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> make docker-image-centos7 V=1 NETWORK=1
> time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
> === TEST SCRIPT END ===
> 
> libudev           no
> default devices   yes
> 
> warning: Python 2 support is deprecated
> warning: Python 3 will be required for building future versions of QEMU
> 
> NOTE: cross-compilers enabled:  'cc'
>    GEN     x86_64-softmmu/config-devices.mak.tmp
> ---
>    CC      block/qed-cluster.o
>    CC      block/qed-check.o
> /tmp/qemu-test/src/block/qcow2-bitmap.c: In function 'qcow2_co_remove_persistent_dirty_bitmap':
> /tmp/qemu-test/src/block/qcow2-bitmap.c:502:8: error: 'bm' may be used uninitialized in this function [-Werror=maybe-uninitialized]
>       if (bm == NULL) {
>          ^
> /tmp/qemu-test/src/block/qcow2-bitmap.c:1413:18: note: 'bm' was declared here
> 
> 
> The full log is available at
> http://patchew.org/logs/20190910162724.79574-1-vsementsov@virtuozzo.com/testing.docker-quick@centos7/?type=message.
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to patchew-devel@redhat.com
> 

Who knows, how to clang Qemu?

I try with
./configure --target-list=x86_64-softmmu --enable-debug --disable-virtfs --enable-werror --audio-drv-list=oss --extra-cflags=-Wall --enable-sanitizers --cc=clang --cxx=clang++
make -j9

but get the following error:

/tmp/linuxboot_dma-6712c0.s: Assembler messages:
/tmp/linuxboot_dma-6712c0.s:473: Error: inconsistent uses of .cfi_sections
clang: error: assembler command failed with exit code 1 (use -v to see invocation)
make[1]: *** [linuxboot_dma.o] Error 1
make[1]: *** Waiting for unfinished jobs....
   SIGN    pc-bios/optionrom/linuxboot.bin
   LINK    ivshmem-client
/tmp/pvh_main-3bbc75.s: Assembler messages:
/tmp/pvh_main-3bbc75.s:651: Error: inconsistent uses of .cfi_sections
clang: error: assembler command failed with exit code 1 (use -v to see invocation)
make[1]: *** [pvh_main.o] Error 1
make: *** [pc-bios/optionrom/all] Error 2
make: *** Waiting for unfinished jobs....
make: *** wait: No child processes.  Stop.



-- 
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH 0/3] proper locking on bitmap add/remove paths
Posted by John Snow 4 years, 7 months ago

On 9/11/19 10:09 AM, Vladimir Sementsov-Ogievskiy wrote:
> 10.09.2019 23:37, no-reply@patchew.org wrote:
>> Patchew URL: https://patchew.org/QEMU/20190910162724.79574-1-vsementsov@virtuozzo.com/
>>
>>
>>
>> Hi,
>>
>> This series failed the docker-quick@centos7 build test. Please find the testing commands and
>> their output below. If you have Docker installed, you can probably reproduce it
>> locally.
>>
>> === TEST SCRIPT BEGIN ===
>> #!/bin/bash
>> make docker-image-centos7 V=1 NETWORK=1
>> time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1

(Was patchew even using clang?)

>> === TEST SCRIPT END ===
>>
>> libudev           no
>> default devices   yes
>>
>> warning: Python 2 support is deprecated
>> warning: Python 3 will be required for building future versions of QEMU
>>
>> NOTE: cross-compilers enabled:  'cc'
>>    GEN     x86_64-softmmu/config-devices.mak.tmp
>> ---
>>    CC      block/qed-cluster.o
>>    CC      block/qed-check.o
>> /tmp/qemu-test/src/block/qcow2-bitmap.c: In function 'qcow2_co_remove_persistent_dirty_bitmap':
>> /tmp/qemu-test/src/block/qcow2-bitmap.c:502:8: error: 'bm' may be used uninitialized in this function [-Werror=maybe-uninitialized]
>>       if (bm == NULL) {
>>          ^
>> /tmp/qemu-test/src/block/qcow2-bitmap.c:1413:18: note: 'bm' was declared here
>>
>>
>> The full log is available at
>> http://patchew.org/logs/20190910162724.79574-1-vsementsov@virtuozzo.com/testing.docker-quick@centos7/?type=message.
>> ---
>> Email generated automatically by Patchew [https://patchew.org/].
>> Please send your feedback to patchew-devel@redhat.com
>>
> 
> Who knows, how to clang Qemu?
> 
> I try with
> ./configure --target-list=x86_64-softmmu --enable-debug --disable-virtfs --enable-werror --audio-drv-list=oss --extra-cflags=-Wall --enable-sanitizers --cc=clang --cxx=clang++
> make -j9
> 

../../configure --target-list="x86_64-softmmu" --cc=clang --cxx=clang++
--host-cc=clang

works OK for me in fedora 30.

--target-list="x86_64-softmmu" --cc=clang --cxx=clang++ --host-cc=clang
--enable-werror --extra-cflags=-Wall

Seems OK too, and finally adding

--enable-sanitizers

also appears to work alright.


clang version 8.0.0 (Fedora 8.0.0-1.fc30)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin


Maybe something bad in your ccache or some intermediate state in your
build dir. I use separate build directories for gcc and clang just in case.

--js


Re: [Qemu-devel] [PATCH 0/3] proper locking on bitmap add/remove paths
Posted by Vladimir Sementsov-Ogievskiy 4 years, 7 months ago
12.09.2019 0:46, John Snow wrote:
> 
> 
> On 9/11/19 10:09 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 10.09.2019 23:37, no-reply@patchew.org wrote:
>>> Patchew URL: https://patchew.org/QEMU/20190910162724.79574-1-vsementsov@virtuozzo.com/
>>>
>>>
>>>
>>> Hi,
>>>
>>> This series failed the docker-quick@centos7 build test. Please find the testing commands and
>>> their output below. If you have Docker installed, you can probably reproduce it
>>> locally.
>>>
>>> === TEST SCRIPT BEGIN ===
>>> #!/bin/bash
>>> make docker-image-centos7 V=1 NETWORK=1
>>> time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
> 
> (Was patchew even using clang?)

hmm gcc 9.2 don't show such error but clang does.

> 
>>> === TEST SCRIPT END ===
>>>
>>> libudev           no
>>> default devices   yes
>>>
>>> warning: Python 2 support is deprecated
>>> warning: Python 3 will be required for building future versions of QEMU
>>>
>>> NOTE: cross-compilers enabled:  'cc'
>>>     GEN     x86_64-softmmu/config-devices.mak.tmp
>>> ---
>>>     CC      block/qed-cluster.o
>>>     CC      block/qed-check.o
>>> /tmp/qemu-test/src/block/qcow2-bitmap.c: In function 'qcow2_co_remove_persistent_dirty_bitmap':
>>> /tmp/qemu-test/src/block/qcow2-bitmap.c:502:8: error: 'bm' may be used uninitialized in this function [-Werror=maybe-uninitialized]
>>>        if (bm == NULL) {
>>>           ^
>>> /tmp/qemu-test/src/block/qcow2-bitmap.c:1413:18: note: 'bm' was declared here
>>>
>>>
>>> The full log is available at
>>> http://patchew.org/logs/20190910162724.79574-1-vsementsov@virtuozzo.com/testing.docker-quick@centos7/?type=message.
>>> ---
>>> Email generated automatically by Patchew [https://patchew.org/].
>>> Please send your feedback to patchew-devel@redhat.com
>>>
>>
>> Who knows, how to clang Qemu?
>>
>> I try with
>> ./configure --target-list=x86_64-softmmu --enable-debug --disable-virtfs --enable-werror --audio-drv-list=oss --extra-cflags=-Wall --enable-sanitizers --cc=clang --cxx=clang++
>> make -j9
>>
> 
> ../../configure --target-list="x86_64-softmmu" --cc=clang --cxx=clang++
> --host-cc=clang
> 
> works OK for me in fedora 30.

So, now I've moved to fedora from rhel-based virtuozzo, and it just works :)

> 
> --target-list="x86_64-softmmu" --cc=clang --cxx=clang++ --host-cc=clang
> --enable-werror --extra-cflags=-Wall
> 
> Seems OK too, and finally adding
> 
> --enable-sanitizers
> 
> also appears to work alright.
> 
> 
> clang version 8.0.0 (Fedora 8.0.0-1.fc30)
> Target: x86_64-unknown-linux-gnu
> Thread model: posix
> InstalledDir: /usr/bin
> 
> 
> Maybe something bad in your ccache or some intermediate state in your
> build dir. I use separate build directories for gcc and clang just in case.
> 

Thanks for help!


-- 
Best regards,
Vladimir