1. Patchew
  2. QEMU
  3. View series

[PATCH v3 00/11] Record/replay acceptance tests

Pavel Dovgalyuk posted 11 patches 3 years, 11 months ago
Test docker-mingw@fedora passed
Test checkpatch passed
Test asan passed
Test docker-quick@centos7 passed
Test FreeBSD passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/159073587336.20809.5404476664125786279.stgit@pasha-ThinkPad-X280
Maintainers: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>, Paolo Bonzini <pbonzini@redhat.com>
0 files changed
[PATCH v3 00/11] Record/replay acceptance tests
Posted by Pavel Dovgalyuk 3 years, 11 months ago 
The following series adds record/replay tests to the acceptance group.
Test pass successfully with the latest submitted record/replay fixes:
 - replay: notify the main loop when there are no instructions
 - replay: synchronize on every virtual timer callback

The provided tests perform kernel boot and disk image boot scenarios.
For all of them recording and replaying phases are executed.
Tests were borrowed from existing boot_linux*.py tests. But some
of the platforms and images were excluded, because icount for them
still has some issues.

Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>

v3 changes:
 - Added record/replay logging (suggested by Philippe Mathieu-Daudé)
 - Changed the console pattern to get longer kernel execution (suggested by Alex Bennée)
 - Coding style fixes
 - Other minor changes
v2 changes:
 - Some test structure refactoring (suggested by Willian Rampazzo)

---

Pavel Dovgaluk (11):
      tests/acceptance: allow console interaction with specific VMs
      tests/acceptance: refactor boot_linux_console test to allow code reuse
      tests/acceptance: add base class record/replay kernel tests
      tests/acceptance: add kernel record/replay test for x86_64
      tests/acceptance: add record/replay test for aarch64
      tests/acceptance: add record/replay test for arm
      tests/acceptance: add record/replay test for ppc64
      tests/acceptance: add record/replay test for m68k
      tests/acceptance: record/replay tests with advcal images
      tests/acceptance: refactor boot_linux to allow code reuse
      tests/acceptance: Linux boot test for record/replay


 0 files changed

--
Pavel Dovgalyuk
Re: [PATCH v3 00/11] Record/replay acceptance tests
Posted by Philippe Mathieu-Daudé 3 years, 11 months ago 
On 5/29/20 9:04 AM, Pavel Dovgalyuk wrote:
> The following series adds record/replay tests to the acceptance group.
> Test pass successfully with the latest submitted record/replay fixes:
>  - replay: notify the main loop when there are no instructions
>  - replay: synchronize on every virtual timer callback
> 
> The provided tests perform kernel boot and disk image boot scenarios.
> For all of them recording and replaying phases are executed.
> Tests were borrowed from existing boot_linux*.py tests. But some
> of the platforms and images were excluded, because icount for them
> still has some issues.
> 
> Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
> 
> v3 changes:
>  - Added record/replay logging (suggested by Philippe Mathieu-Daudé)
>  - Changed the console pattern to get longer kernel execution (suggested by Alex Bennée)
>  - Coding style fixes
>  - Other minor changes
> v2 changes:
>  - Some test structure refactoring (suggested by Willian Rampazzo)
> 
> ---
> 
> Pavel Dovgaluk (11):
>       tests/acceptance: allow console interaction with specific VMs
>       tests/acceptance: refactor boot_linux_console test to allow code reuse
>       tests/acceptance: add base class record/replay kernel tests
>       tests/acceptance: add kernel record/replay test for x86_64
>       tests/acceptance: add record/replay test for aarch64
>       tests/acceptance: add record/replay test for arm
>       tests/acceptance: add record/replay test for ppc64
>       tests/acceptance: add record/replay test for m68k
>       tests/acceptance: record/replay tests with advcal images
>       tests/acceptance: refactor boot_linux to allow code reuse
>       tests/acceptance: Linux boot test for record/replay

Thanks, patches 1/2/10 applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next

- tests/acceptance: allow console interaction with specific VMs
- tests/acceptance: refactor boot_linux_console test to allow code reuse
- tests/acceptance: refactor boot_linux to allow code reuse
Re: [PATCH v3 00/11] Record/replay acceptance tests
Posted by Philippe Mathieu-Daudé 3 years, 10 months ago 
Hi Pavel,

On 5/31/20 5:09 PM, Philippe Mathieu-Daudé wrote:
> On 5/29/20 9:04 AM, Pavel Dovgalyuk wrote:
>> The following series adds record/replay tests to the acceptance group.
>> Test pass successfully with the latest submitted record/replay fixes:
>>  - replay: notify the main loop when there are no instructions
>>  - replay: synchronize on every virtual timer callback
>>
>> The provided tests perform kernel boot and disk image boot scenarios.
>> For all of them recording and replaying phases are executed.
>> Tests were borrowed from existing boot_linux*.py tests. But some
>> of the platforms and images were excluded, because icount for them
>> still has some issues.
>>
>> Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
>>
>> v3 changes:
>>  - Added record/replay logging (suggested by Philippe Mathieu-Daudé)
>>  - Changed the console pattern to get longer kernel execution (suggested by Alex Bennée)
>>  - Coding style fixes
>>  - Other minor changes
>> v2 changes:
>>  - Some test structure refactoring (suggested by Willian Rampazzo)
>>
>> ---
>>
>> Pavel Dovgaluk (11):
>>       tests/acceptance: allow console interaction with specific VMs
>>       tests/acceptance: refactor boot_linux_console test to allow code reuse
>>       tests/acceptance: add base class record/replay kernel tests
>>       tests/acceptance: add kernel record/replay test for x86_64
>>       tests/acceptance: add record/replay test for aarch64
>>       tests/acceptance: add record/replay test for arm
>>       tests/acceptance: add record/replay test for ppc64
>>       tests/acceptance: add record/replay test for m68k

You forgot to Cc the maintainers for each target/machine tested.

Anyway this series has been on the list for 3 weeks, so I'm
queuing it for next acceptance-testing pull request.

Since the x86 tests generate some timeout errors on Travis-CI,
I squashed the following changes:

-- >8 --
diff --git a/tests/acceptance/replay_kernel.py
b/tests/acceptance/replay_kernel.py
index a25eb3d3ee..8fa6b517c5 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -12,6 +12,7 @@
 import logging
 import time

+from avocado import skipIf
 from avocado_qemu import wait_for_console_pattern
 from avocado.utils import archive
 from avocado.utils import process
@@ -72,6 +73,7 @@ def run_rr(self, kernel_path, kernel_command_line,
console_pattern,
         logger = logging.getLogger('replay')
         logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))

+    @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
     def test_x86_64_pc(self):
         """
         :avocado: tags=arch:x86_64
diff --git a/tests/acceptance/replay_linux.py
b/tests/acceptance/replay_linux.py
index 472ab730cd..8e43e282a7 100644
--- a/tests/acceptance/replay_linux.py
+++ b/tests/acceptance/replay_linux.py
@@ -12,6 +12,7 @@
 import logging
 import time

+from avocado import skipIf
 from avocado.utils import cloudinit
 from avocado.utils import datadrainer
 from avocado.utils import network
@@ -92,6 +93,7 @@ def run_rr(self, args=None, shift=7):
         logger = logging.getLogger('replay')
         logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))

+@skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
 class ReplayLinuxX8664(ReplayLinux):
     """
     :avocado: tags=arch:x86_64
---

Regards,

Phil.

>>       tests/acceptance: record/replay tests with advcal images
>>       tests/acceptance: refactor boot_linux to allow code reuse
>>       tests/acceptance: Linux boot test for record/replay
> 
> Thanks, patches 1/2/10 applied to my python-next tree:
> https://gitlab.com/philmd/qemu/commits/python-next
> 
> - tests/acceptance: allow console interaction with specific VMs
> - tests/acceptance: refactor boot_linux_console test to allow code reuse
> - tests/acceptance: refactor boot_linux to allow code reuse
>
Re: [PATCH v3 00/11] Record/replay acceptance tests
Posted by Pavel Dovgalyuk 3 years, 10 months ago 
On 21.06.2020 02:36, Philippe Mathieu-Daudé wrote:
> Hi Pavel,
> 
> On 5/31/20 5:09 PM, Philippe Mathieu-Daudé wrote:
>> On 5/29/20 9:04 AM, Pavel Dovgalyuk wrote:
>>> The following series adds record/replay tests to the acceptance group.
>>> Test pass successfully with the latest submitted record/replay fixes:
>>>   - replay: notify the main loop when there are no instructions
>>>   - replay: synchronize on every virtual timer callback
>>>
>>> The provided tests perform kernel boot and disk image boot scenarios.
>>> For all of them recording and replaying phases are executed.
>>> Tests were borrowed from existing boot_linux*.py tests. But some
>>> of the platforms and images were excluded, because icount for them
>>> still has some issues.
>>>
>>> Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
>>>
>>> v3 changes:
>>>   - Added record/replay logging (suggested by Philippe Mathieu-Daudé)
>>>   - Changed the console pattern to get longer kernel execution (suggested by Alex Bennée)
>>>   - Coding style fixes
>>>   - Other minor changes
>>> v2 changes:
>>>   - Some test structure refactoring (suggested by Willian Rampazzo)
>>>
>>> ---
>>>
>>> Pavel Dovgaluk (11):
>>>        tests/acceptance: allow console interaction with specific VMs
>>>        tests/acceptance: refactor boot_linux_console test to allow code reuse
>>>        tests/acceptance: add base class record/replay kernel tests
>>>        tests/acceptance: add kernel record/replay test for x86_64
>>>        tests/acceptance: add record/replay test for aarch64
>>>        tests/acceptance: add record/replay test for arm
>>>        tests/acceptance: add record/replay test for ppc64
>>>        tests/acceptance: add record/replay test for m68k
> 
> You forgot to Cc the maintainers for each target/machine tested.
> 
> Anyway this series has been on the list for 3 weeks, so I'm
> queuing it for next acceptance-testing pull request.
> 
> Since the x86 tests generate some timeout errors on Travis-CI,

Errors occur with the commits that were not queued yet?

> I squashed the following changes:
> 
> -- >8 --
> diff --git a/tests/acceptance/replay_kernel.py
> b/tests/acceptance/replay_kernel.py
> index a25eb3d3ee..8fa6b517c5 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -12,6 +12,7 @@
>   import logging
>   import time
> 
> +from avocado import skipIf
>   from avocado_qemu import wait_for_console_pattern
>   from avocado.utils import archive
>   from avocado.utils import process
> @@ -72,6 +73,7 @@ def run_rr(self, kernel_path, kernel_command_line,
> console_pattern,
>           logger = logging.getLogger('replay')
>           logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))
> 
> +    @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
>       def test_x86_64_pc(self):
>           """
>           :avocado: tags=arch:x86_64
> diff --git a/tests/acceptance/replay_linux.py
> b/tests/acceptance/replay_linux.py
> index 472ab730cd..8e43e282a7 100644
> --- a/tests/acceptance/replay_linux.py
> +++ b/tests/acceptance/replay_linux.py
> @@ -12,6 +12,7 @@
>   import logging
>   import time
> 
> +from avocado import skipIf
>   from avocado.utils import cloudinit
>   from avocado.utils import datadrainer
>   from avocado.utils import network
> @@ -92,6 +93,7 @@ def run_rr(self, args=None, shift=7):
>           logger = logging.getLogger('replay')
>           logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))
> 
> +@skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
>   class ReplayLinuxX8664(ReplayLinux):
>       """
>       :avocado: tags=arch:x86_64
> ---
> 
> Regards,
> 
> Phil.
> 
>>>        tests/acceptance: record/replay tests with advcal images
>>>        tests/acceptance: refactor boot_linux to allow code reuse
>>>        tests/acceptance: Linux boot test for record/replay
>>
>> Thanks, patches 1/2/10 applied to my python-next tree:
>> https://gitlab.com/philmd/qemu/commits/python-next
>>
>> - tests/acceptance: allow console interaction with specific VMs
>> - tests/acceptance: refactor boot_linux_console test to allow code reuse
>> - tests/acceptance: refactor boot_linux to allow code reuse
>>
>

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.