[PATCH v2 13/20] iotests: 129: prepare for backup over block-copy

Vladimir Sementsov-Ogievskiy posted 20 patches 5 years, 8 months ago
There is a newer version of this series
[PATCH v2 13/20] iotests: 129: prepare for backup over block-copy
Posted by Vladimir Sementsov-Ogievskiy 5 years, 8 months ago
After introducing parallel async copy requests instead of plain
cluster-by-cluster copying loop, backup job may finish earlier than
final assertion in do_test_stop. Let's require slow backup explicitly
by specifying speed parameter.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/129 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index 4db5eca441..bca56b589d 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -76,7 +76,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
     def test_drive_backup(self):
         self.do_test_stop("drive-backup", device="drive0",
                           target=self.target_img,
-                          sync="full")
+                          sync="full", speed=1024)
 
     def test_block_commit(self):
         self.do_test_stop("block-commit", device="drive0")
-- 
2.21.0


Re: [PATCH v2 13/20] iotests: 129: prepare for backup over block-copy
Posted by Max Reitz 5 years, 6 months ago
On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote:
> After introducing parallel async copy requests instead of plain
> cluster-by-cluster copying loop, backup job may finish earlier than
> final assertion in do_test_stop. Let's require slow backup explicitly
> by specifying speed parameter.

Isn’t the problem really that block_set_io_throttle does absolutely
nothing?  (Which is a long-standing problem with 129.  I personally just
never run it, honestly.)

> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  tests/qemu-iotests/129 | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
> index 4db5eca441..bca56b589d 100755
> --- a/tests/qemu-iotests/129
> +++ b/tests/qemu-iotests/129
> @@ -76,7 +76,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
>      def test_drive_backup(self):
>          self.do_test_stop("drive-backup", device="drive0",
>                            target=self.target_img,
> -                          sync="full")
> +                          sync="full", speed=1024)
>  
>      def test_block_commit(self):
>          self.do_test_stop("block-commit", device="drive0")
> 


Re: [PATCH v2 13/20] iotests: 129: prepare for backup over block-copy
Posted by Vladimir Sementsov-Ogievskiy 5 years, 3 months ago
23.07.2020 11:03, Max Reitz wrote:
> On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote:
>> After introducing parallel async copy requests instead of plain
>> cluster-by-cluster copying loop, backup job may finish earlier than
>> final assertion in do_test_stop. Let's require slow backup explicitly
>> by specifying speed parameter.
> 
> Isn’t the problem really that block_set_io_throttle does absolutely
> nothing?  (Which is a long-standing problem with 129.  I personally just
> never run it, honestly.)

Hmm.. is it better to drop test_drive_backup() from here ?

> 
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   tests/qemu-iotests/129 | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
>> index 4db5eca441..bca56b589d 100755
>> --- a/tests/qemu-iotests/129
>> +++ b/tests/qemu-iotests/129
>> @@ -76,7 +76,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
>>       def test_drive_backup(self):
>>           self.do_test_stop("drive-backup", device="drive0",
>>                             target=self.target_img,
>> -                          sync="full")
>> +                          sync="full", speed=1024)
>>   
>>       def test_block_commit(self):
>>           self.do_test_stop("block-commit", device="drive0")
>>
> 
> 


-- 
Best regards,
Vladimir

Re: [PATCH v2 13/20] iotests: 129: prepare for backup over block-copy
Posted by Max Reitz 5 years, 3 months ago
On 22.10.20 23:10, Vladimir Sementsov-Ogievskiy wrote:
> 23.07.2020 11:03, Max Reitz wrote:
>> On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote:
>>> After introducing parallel async copy requests instead of plain
>>> cluster-by-cluster copying loop, backup job may finish earlier than
>>> final assertion in do_test_stop. Let's require slow backup explicitly
>>> by specifying speed parameter.
>>
>> Isn’t the problem really that block_set_io_throttle does absolutely
>> nothing?  (Which is a long-standing problem with 129.  I personally just
>> never run it, honestly.)
> 
> Hmm.. is it better to drop test_drive_backup() from here ?

I think the best would be to revisit this:

https://lists.nongnu.org/archive/html/qemu-block/2019-06/msg00499.html

Max


Re: [PATCH v2 13/20] iotests: 129: prepare for backup over block-copy
Posted by Vladimir Sementsov-Ogievskiy 5 years, 3 months ago
04.11.2020 20:30, Max Reitz wrote:
> On 22.10.20 23:10, Vladimir Sementsov-Ogievskiy wrote:
>> 23.07.2020 11:03, Max Reitz wrote:
>>> On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote:
>>>> After introducing parallel async copy requests instead of plain
>>>> cluster-by-cluster copying loop, backup job may finish earlier than
>>>> final assertion in do_test_stop. Let's require slow backup explicitly
>>>> by specifying speed parameter.
>>>
>>> Isn’t the problem really that block_set_io_throttle does absolutely
>>> nothing?  (Which is a long-standing problem with 129.  I personally just
>>> never run it, honestly.)
>>
>> Hmm.. is it better to drop test_drive_backup() from here ?
> 
> I think the best would be to revisit this:
> 
> https://lists.nongnu.org/archive/html/qemu-block/2019-06/msg00499.html
> 
> Max
> 

I've checked, no, that doesn't help. I suppose that throttling has same defect as
original block-job speed: it calculates throttling after request, and if we issue
simultaneously many requests they all will be handled (and after it we'll have long
throttling pause). New solution for backup in these series works better with parallel
requests (see patch [PATCH v2 07/20] block/block-copy: add ratelimit to block-copy).

So, I'd keep this patch for now.

-- 
Best regards,
Vladimir