[Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4

Max Reitz posted 7 patches 4 years, 8 months ago
Test checkpatch passed
Test s390x passed
Test asan passed
Test docker-mingw@fedora passed
Test FreeBSD passed
Test docker-clang@ubuntu passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20190805163740.23616-1-mreitz@redhat.com
Maintainers: Max Reitz <mreitz@redhat.com>, John Snow <jsnow@redhat.com>, Fam Zheng <fam@euphon.net>, Kevin Wolf <kwolf@redhat.com>
There is a newer version of this series
block/backup.c             | 15 ++++++++++++---
block/mirror.c             | 29 ++++++++++++++++++++++++++++
util/hbitmap.c             |  6 +++++-
tests/qemu-iotests/056     | 39 ++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/056.out |  4 ++--
tests/qemu-iotests/124     | 38 +++++++++++++++++++++++++++++++++----
tests/qemu-iotests/124.out |  4 ++--
tests/qemu-iotests/151     | 25 ++++++++++++++++++++++++
tests/qemu-iotests/151.out |  4 ++--
9 files changed, 150 insertions(+), 14 deletions(-)
[Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Max Reitz 4 years, 8 months ago
The following changes since commit 9bb68d34dda9be60335e73e65c8fb61bca035362:

  Merge remote-tracking branch 'remotes/philmd-gitlab/tags/edk2-next-20190803' into staging (2019-08-05 11:05:36 +0100)

are available in the Git repository at:

  https://github.com/XanClic/qemu.git tags/pull-block-2019-08-05

for you to fetch changes up to 07b0851c592efe188a87259adbda26a63c61dc92:

  block/backup: disable copy_range for compressed backup (2019-08-05 18:05:05 +0200)

----------------------------------------------------------------
Block patches for 4.1.0-rc4:
- Fix the backup block job when using copy offloading
- Fix the mirror block job when using the write-blocking copy mode
- Fix incremental backups after the image has been grown with the
  respective bitmap attached to it

----------------------------------------------------------------
Max Reitz (5):
  backup: Copy only dirty areas
  iotests: Test backup job with two guest writes
  iotests: Test incremental backup after truncation
  mirror: Only mirror granularity-aligned chunks
  iotests: Test unaligned blocking mirror write

Vladimir Sementsov-Ogievskiy (2):
  util/hbitmap: update orig_size on truncate
  block/backup: disable copy_range for compressed backup

 block/backup.c             | 15 ++++++++++++---
 block/mirror.c             | 29 ++++++++++++++++++++++++++++
 util/hbitmap.c             |  6 +++++-
 tests/qemu-iotests/056     | 39 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/056.out |  4 ++--
 tests/qemu-iotests/124     | 38 +++++++++++++++++++++++++++++++++----
 tests/qemu-iotests/124.out |  4 ++--
 tests/qemu-iotests/151     | 25 ++++++++++++++++++++++++
 tests/qemu-iotests/151.out |  4 ++--
 9 files changed, 150 insertions(+), 14 deletions(-)

-- 
2.21.0


Re: [Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Vladimir Sementsov-Ogievskiy 4 years, 8 months ago
05.08.2019 19:37, Max Reitz wrote:
> The following changes since commit 9bb68d34dda9be60335e73e65c8fb61bca035362:
> 
>    Merge remote-tracking branch 'remotes/philmd-gitlab/tags/edk2-next-20190803' into staging (2019-08-05 11:05:36 +0100)
> 
> are available in the Git repository at:
> 
>    https://github.com/XanClic/qemu.git tags/pull-block-2019-08-05
> 
> for you to fetch changes up to 07b0851c592efe188a87259adbda26a63c61dc92:
> 
>    block/backup: disable copy_range for compressed backup (2019-08-05 18:05:05 +0200)
> 
> ----------------------------------------------------------------
> Block patches for 4.1.0-rc4:
> - Fix the backup block job when using copy offloading
> - Fix the mirror block job when using the write-blocking copy mode
> - Fix incremental backups after the image has been grown with the
>    respective bitmap attached to it
> 
> ----------------------------------------------------------------
> Max Reitz (5):
>    backup: Copy only dirty areas
>    iotests: Test backup job with two guest writes
>    iotests: Test incremental backup after truncation
>    mirror: Only mirror granularity-aligned chunks
>    iotests: Test unaligned blocking mirror write
> 
> Vladimir Sementsov-Ogievskiy (2):
>    util/hbitmap: update orig_size on truncate
>    block/backup: disable copy_range for compressed backup
> 

As I understand, this all should go to stable too? CC it.

-- 
Best regards,
Vladimir
Re: [Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Max Reitz 4 years, 8 months ago
On 05.08.19 18:59, Vladimir Sementsov-Ogievskiy wrote:
> 05.08.2019 19:37, Max Reitz wrote:
>> The following changes since commit 9bb68d34dda9be60335e73e65c8fb61bca035362:
>>
>>    Merge remote-tracking branch 'remotes/philmd-gitlab/tags/edk2-next-20190803' into staging (2019-08-05 11:05:36 +0100)
>>
>> are available in the Git repository at:
>>
>>    https://github.com/XanClic/qemu.git tags/pull-block-2019-08-05
>>
>> for you to fetch changes up to 07b0851c592efe188a87259adbda26a63c61dc92:
>>
>>    block/backup: disable copy_range for compressed backup (2019-08-05 18:05:05 +0200)
>>
>> ----------------------------------------------------------------
>> Block patches for 4.1.0-rc4:
>> - Fix the backup block job when using copy offloading
>> - Fix the mirror block job when using the write-blocking copy mode
>> - Fix incremental backups after the image has been grown with the
>>    respective bitmap attached to it
>>
>> ----------------------------------------------------------------
>> Max Reitz (5):
>>    backup: Copy only dirty areas
>>    iotests: Test backup job with two guest writes
>>    iotests: Test incremental backup after truncation
>>    mirror: Only mirror granularity-aligned chunks
>>    iotests: Test unaligned blocking mirror write
>>
>> Vladimir Sementsov-Ogievskiy (2):
>>    util/hbitmap: update orig_size on truncate
>>    block/backup: disable copy_range for compressed backup
>>
> 
> As I understand, this all should go to stable too? CC it.
Ah, yes.  Thanks.

Max

Re: [Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Peter Maydell 4 years, 8 months ago
On Mon, 5 Aug 2019 at 18:01, Max Reitz <mreitz@redhat.com> wrote:
>
> On 05.08.19 18:59, Vladimir Sementsov-Ogievskiy wrote:
> > 05.08.2019 19:37, Max Reitz wrote:
> >> The following changes since commit 9bb68d34dda9be60335e73e65c8fb61bca035362:
> >>
> >>    Merge remote-tracking branch 'remotes/philmd-gitlab/tags/edk2-next-20190803' into staging (2019-08-05 11:05:36 +0100)
> >>
> >> are available in the Git repository at:
> >>
> >>    https://github.com/XanClic/qemu.git tags/pull-block-2019-08-05
> >>
> >> for you to fetch changes up to 07b0851c592efe188a87259adbda26a63c61dc92:
> >>
> >>    block/backup: disable copy_range for compressed backup (2019-08-05 18:05:05 +0200)
> >>
> >> ----------------------------------------------------------------
> >> Block patches for 4.1.0-rc4:
> >> - Fix the backup block job when using copy offloading
> >> - Fix the mirror block job when using the write-blocking copy mode
> >> - Fix incremental backups after the image has been grown with the
> >>    respective bitmap attached to it
> >>
> >> ----------------------------------------------------------------
> >> Max Reitz (5):
> >>    backup: Copy only dirty areas
> >>    iotests: Test backup job with two guest writes
> >>    iotests: Test incremental backup after truncation
> >>    mirror: Only mirror granularity-aligned chunks
> >>    iotests: Test unaligned blocking mirror write
> >>
> >> Vladimir Sementsov-Ogievskiy (2):
> >>    util/hbitmap: update orig_size on truncate
> >>    block/backup: disable copy_range for compressed backup
> >>
> >
> > As I understand, this all should go to stable too? CC it.
> Ah, yes.  Thanks.

Are you planning to send a respin with the CC:stable tags?
(I did a test merge of this version which all passed OK.)

thanks
-- PMM

Re: [Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Max Reitz 4 years, 8 months ago
On 06.08.19 12:12, Peter Maydell wrote:
> On Mon, 5 Aug 2019 at 18:01, Max Reitz <mreitz@redhat.com> wrote:
>>
>> On 05.08.19 18:59, Vladimir Sementsov-Ogievskiy wrote:
>>> 05.08.2019 19:37, Max Reitz wrote:
>>>> The following changes since commit 9bb68d34dda9be60335e73e65c8fb61bca035362:
>>>>
>>>>    Merge remote-tracking branch 'remotes/philmd-gitlab/tags/edk2-next-20190803' into staging (2019-08-05 11:05:36 +0100)
>>>>
>>>> are available in the Git repository at:
>>>>
>>>>    https://github.com/XanClic/qemu.git tags/pull-block-2019-08-05
>>>>
>>>> for you to fetch changes up to 07b0851c592efe188a87259adbda26a63c61dc92:
>>>>
>>>>    block/backup: disable copy_range for compressed backup (2019-08-05 18:05:05 +0200)
>>>>
>>>> ----------------------------------------------------------------
>>>> Block patches for 4.1.0-rc4:
>>>> - Fix the backup block job when using copy offloading
>>>> - Fix the mirror block job when using the write-blocking copy mode
>>>> - Fix incremental backups after the image has been grown with the
>>>>    respective bitmap attached to it
>>>>
>>>> ----------------------------------------------------------------
>>>> Max Reitz (5):
>>>>    backup: Copy only dirty areas
>>>>    iotests: Test backup job with two guest writes
>>>>    iotests: Test incremental backup after truncation
>>>>    mirror: Only mirror granularity-aligned chunks
>>>>    iotests: Test unaligned blocking mirror write
>>>>
>>>> Vladimir Sementsov-Ogievskiy (2):
>>>>    util/hbitmap: update orig_size on truncate
>>>>    block/backup: disable copy_range for compressed backup
>>>>
>>>
>>> As I understand, this all should go to stable too? CC it.
>> Ah, yes.  Thanks.
> 
> Are you planning to send a respin with the CC:stable tags?
> (I did a test merge of this version which all passed OK.)

I thought Vladimir just meant to physically CC qemu-stable on the series
(which he did).  Should I respin with the tags?

Max

Re: [Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Peter Maydell 4 years, 8 months ago
On Tue, 6 Aug 2019 at 12:12, Max Reitz <mreitz@redhat.com> wrote:
>
> On 06.08.19 12:12, Peter Maydell wrote:
> > On Mon, 5 Aug 2019 at 18:01, Max Reitz <mreitz@redhat.com> wrote:
> >>
> >> On 05.08.19 18:59, Vladimir Sementsov-Ogievskiy wrote:
> >>> 05.08.2019 19:37, Max Reitz wrote:
> >>>> The following changes since commit 9bb68d34dda9be60335e73e65c8fb61bca035362:
> >>>>
> >>>>    Merge remote-tracking branch 'remotes/philmd-gitlab/tags/edk2-next-20190803' into staging (2019-08-05 11:05:36 +0100)
> >>>>
> >>>> are available in the Git repository at:
> >>>>
> >>>>    https://github.com/XanClic/qemu.git tags/pull-block-2019-08-05
> >>>>
> >>>> for you to fetch changes up to 07b0851c592efe188a87259adbda26a63c61dc92:
> >>>>
> >>>>    block/backup: disable copy_range for compressed backup (2019-08-05 18:05:05 +0200)
> >>>>
> >>>> ----------------------------------------------------------------
> >>>> Block patches for 4.1.0-rc4:
> >>>> - Fix the backup block job when using copy offloading
> >>>> - Fix the mirror block job when using the write-blocking copy mode
> >>>> - Fix incremental backups after the image has been grown with the
> >>>>    respective bitmap attached to it
> >>>>
> >>>> ----------------------------------------------------------------
> >>>> Max Reitz (5):
> >>>>    backup: Copy only dirty areas
> >>>>    iotests: Test backup job with two guest writes
> >>>>    iotests: Test incremental backup after truncation
> >>>>    mirror: Only mirror granularity-aligned chunks
> >>>>    iotests: Test unaligned blocking mirror write
> >>>>
> >>>> Vladimir Sementsov-Ogievskiy (2):
> >>>>    util/hbitmap: update orig_size on truncate
> >>>>    block/backup: disable copy_range for compressed backup
> >>>>
> >>>
> >>> As I understand, this all should go to stable too? CC it.
> >> Ah, yes.  Thanks.
> >
> > Are you planning to send a respin with the CC:stable tags?
> > (I did a test merge of this version which all passed OK.)
>
> I thought Vladimir just meant to physically CC qemu-stable on the series
> (which he did).  Should I respin with the tags?

If you could do a quick respin that's probably most reliable --
I'm not sure exactly how the qemu-stable process works, though.

thanks
-- PMM

Re: [Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Peter Maydell 4 years, 8 months ago
On Mon, 5 Aug 2019 at 17:37, Max Reitz <mreitz@redhat.com> wrote:
>
> The following changes since commit 9bb68d34dda9be60335e73e65c8fb61bca035362:
>
>   Merge remote-tracking branch 'remotes/philmd-gitlab/tags/edk2-next-20190803' into staging (2019-08-05 11:05:36 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/XanClic/qemu.git tags/pull-block-2019-08-05
>
> for you to fetch changes up to 07b0851c592efe188a87259adbda26a63c61dc92:
>
>   block/backup: disable copy_range for compressed backup (2019-08-05 18:05:05 +0200)
>
> ----------------------------------------------------------------
> Block patches for 4.1.0-rc4:
> - Fix the backup block job when using copy offloading
> - Fix the mirror block job when using the write-blocking copy mode
> - Fix incremental backups after the image has been grown with the
>   respective bitmap attached to it
>
> ----------------------------------------------------------------
> Max Reitz (5):
>   backup: Copy only dirty areas
>   iotests: Test backup job with two guest writes
>   iotests: Test incremental backup after truncation
>   mirror: Only mirror granularity-aligned chunks
>   iotests: Test unaligned blocking mirror write
>
> Vladimir Sementsov-Ogievskiy (2):
>   util/hbitmap: update orig_size on truncate
>   block/backup: disable copy_range for compressed backup
>
>  block/backup.c             | 15 ++++++++++++---
>  block/mirror.c             | 29 ++++++++++++++++++++++++++++
>  util/hbitmap.c             |  6 +++++-
>  tests/qemu-iotests/056     | 39 ++++++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/056.out |  4 ++--
>  tests/qemu-iotests/124     | 38 +++++++++++++++++++++++++++++++++----
>  tests/qemu-iotests/124.out |  4 ++--
>  tests/qemu-iotests/151     | 25 ++++++++++++++++++++++++
>  tests/qemu-iotests/151.out |  4 ++--
>  9 files changed, 150 insertions(+), 14 deletions(-)

This is quite a lot of changes for rc4 -- how confident are
you about them ? I suppose 3 out of 4 commits are updating
the test suite...

thanks
-- PMM

Re: [Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Peter Maydell 4 years, 8 months ago
On Mon, 5 Aug 2019 at 19:05, Peter Maydell <peter.maydell@linaro.org> wrote:

> This is quite a lot of changes for rc4 -- how confident are
> you about them ? I suppose 3 out of 4 commits are updating
> the test suite...

3 out of 7, I meant :-)

thanks
-- PMM

Re: [Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Max Reitz 4 years, 8 months ago
On 05.08.19 20:05, Peter Maydell wrote:
> On Mon, 5 Aug 2019 at 17:37, Max Reitz <mreitz@redhat.com> wrote:
>>
>> The following changes since commit 9bb68d34dda9be60335e73e65c8fb61bca035362:
>>
>>   Merge remote-tracking branch 'remotes/philmd-gitlab/tags/edk2-next-20190803' into staging (2019-08-05 11:05:36 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/XanClic/qemu.git tags/pull-block-2019-08-05
>>
>> for you to fetch changes up to 07b0851c592efe188a87259adbda26a63c61dc92:
>>
>>   block/backup: disable copy_range for compressed backup (2019-08-05 18:05:05 +0200)
>>
>> ----------------------------------------------------------------
>> Block patches for 4.1.0-rc4:
>> - Fix the backup block job when using copy offloading
>> - Fix the mirror block job when using the write-blocking copy mode
>> - Fix incremental backups after the image has been grown with the
>>   respective bitmap attached to it
>>
>> ----------------------------------------------------------------
>> Max Reitz (5):
>>   backup: Copy only dirty areas
>>   iotests: Test backup job with two guest writes
>>   iotests: Test incremental backup after truncation
>>   mirror: Only mirror granularity-aligned chunks
>>   iotests: Test unaligned blocking mirror write
>>
>> Vladimir Sementsov-Ogievskiy (2):
>>   util/hbitmap: update orig_size on truncate
>>   block/backup: disable copy_range for compressed backup
>>
>>  block/backup.c             | 15 ++++++++++++---
>>  block/mirror.c             | 29 ++++++++++++++++++++++++++++
>>  util/hbitmap.c             |  6 +++++-
>>  tests/qemu-iotests/056     | 39 ++++++++++++++++++++++++++++++++++++++
>>  tests/qemu-iotests/056.out |  4 ++--
>>  tests/qemu-iotests/124     | 38 +++++++++++++++++++++++++++++++++----
>>  tests/qemu-iotests/124.out |  4 ++--
>>  tests/qemu-iotests/151     | 25 ++++++++++++++++++++++++
>>  tests/qemu-iotests/151.out |  4 ++--
>>  9 files changed, 150 insertions(+), 14 deletions(-)
> 
> This is quite a lot of changes for rc4 -- how confident are
> you about them ? I suppose 3 out of 4 commits are updating
> the test suite...

Would dropping the test patches make it better? :-)

I am reasonably (i.e., rc4-levels of) confident that the patches don’t
break anything that wasn’t broken before.

(I’m least confident about the test patches working for everyone and
everywhere, as with all new test cases.  But it was my impression that
it’s always fine to include test case additions.)

Patch 1 is very important.  I’m very confident about it.
It fixes a silent corruption in the backup job, so I’m not too surprised
people haven’t noticed.  I would be surprised if really noone was
affected so far.

Patch 3 is not that important, but it is sufficiently simple, so I think
we should take it, even into rc4.

Patch 5 is very important for a specific mirror copying mode.  It can be
argued that nobody really uses this mode because otherwise somebody
should have noticed the corruption, because if you hit it, you will
simply lose data (as opposed to the backup case, where you will simply
get the wrong version of the data in the output image).
But that is why it’s so important.  I really don’t want anyone to hit it.
It is probably the most complicated patch here, but at any other point,
it would still be considered a simple patch.  (Just not quite trivial.)
I think it is worth taking it.

Patch 7 is actually not important.  But it’s an obvious trivial
one-liner.  I thought I might as well.

Max

Re: [Qemu-devel] [PULL 0/7] Block patches for 4.1.0-rc4
Posted by Peter Maydell 4 years, 8 months ago
On Mon, 5 Aug 2019 at 19:21, Max Reitz <mreitz@redhat.com> wrote:
> Would dropping the test patches make it better? :-)
>
> I am reasonably (i.e., rc4-levels of) confident that the patches don’t
> break anything that wasn’t broken before.
>
> (I’m least confident about the test patches working for everyone and
> everywhere, as with all new test cases.  But it was my impression that
> it’s always fine to include test case additions.)
>
> Patch 1 is very important.  I’m very confident about it.
> It fixes a silent corruption in the backup job, so I’m not too surprised
> people haven’t noticed.  I would be surprised if really noone was
> affected so far.
>
> Patch 3 is not that important, but it is sufficiently simple, so I think
> we should take it, even into rc4.
>
> Patch 5 is very important for a specific mirror copying mode.  It can be
> argued that nobody really uses this mode because otherwise somebody
> should have noticed the corruption, because if you hit it, you will
> simply lose data (as opposed to the backup case, where you will simply
> get the wrong version of the data in the output image).
> But that is why it’s so important.  I really don’t want anyone to hit it.
> It is probably the most complicated patch here, but at any other point,
> it would still be considered a simple patch.  (Just not quite trivial.)
> I think it is worth taking it.
>
> Patch 7 is actually not important.  But it’s an obvious trivial
> one-liner.  I thought I might as well.

Thanks for the clarifications -- these all sound worth taking.
The thing about rc4 is that we don't really have much chance
to find any problems with patches we put in at this point,
so I like to be pretty cautious.

thanks
-- PMM