[PATCH v2 00/29] tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86

Philippe Mathieu-Daudé posted 29 patches 4 years, 3 months ago
Only 28 patches received!
accel/accel.c                                 |   1 +
hw/core/machine.c                             |   3 +-
MAINTAINERS                                   |   6 +
python/qemu/accel.py                          |  26 +----
python/qemu/binutils.py                       | 107 ++++++++++++++++++
tests/acceptance/avocado_qemu/__init__.py     |  51 ++++++++-
tests/acceptance/boot_linux_console.py        |   2 +-
tests/acceptance/core_scripts.py              |  79 +++++++++++++
tests/acceptance/migration.py                 |   4 +-
tests/acceptance/version.py                   |   1 +
...g_max_adjust.py => virtio_check_params.py} |  70 +++++++++---
tests/acceptance/x86_cpu_model_versions.py    |   1 +
12 files changed, 306 insertions(+), 45 deletions(-)
create mode 100644 python/qemu/binutils.py
create mode 100644 tests/acceptance/core_scripts.py
rename tests/acceptance/{virtio_seg_max_adjust.py => virtio_check_params.py} (64%)
[PATCH v2 00/29] tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86
Posted by Philippe Mathieu-Daudé 4 years, 3 months ago
Currently our CI is mostly restricted to Linux/X86.
We want to have more, but until there we have to run these tests
manually, which is why we didn't noticed earlier this test was
failing on OSX.

This series contains the patches I used to debug why the
virtio_seg_max_adjust had broken the OSX CI, but then I
kept testing on other archs, thus added more patches...

Currently our CI is mostly restricted to Linux/X86.
We want to have more, but until there we have to run these tests
manually, which is why we didn't noticed earlier some tests are
failing (on OSX, using --without-default-devices, ...).

Tested on Linux/FreeBSD/OSX, X86/Aarch64.

Regards,

Phil.

Supersedes: <20200122223247.30419-1-philmd@redhat.com>
"tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86"

Denis Plotnikov (1):
  tests: rename virtio_seg_max_adjust to virtio_check_params

Philippe Mathieu-Daudé (28):
  hw/core: Allow setting 'virtio-blk-device.scsi' property on OSX host
  accel/accel: Make TYPE_ACCEL abstract
  python/qemu: Add binutils::binary_get_version()
  tests/acceptance: Use 'version-min' tag to verify QEMU binary version
  tests/acceptance: Restrict X86CPUModelAliases test to QEMU >= 4.1
  python/qemu: Add binutils::binary_get_arch()
  tests/acceptance: Use the 'arch' tag to verify QEMU binary target
  python/qemu: Add binutils::binary_get_machines()
  tests/acceptance: Use 'machine' tag to check if available in QEMU
    binary
  python/qemu: Add binutils::binary_get_qom_implementations()
  python/qemu: Add binutils::binary_get_accels()
  python/qemu/accel: Use binutils::binary_get_accels()
  python/qemu: Add binutils::binary_get_devices()
  tests/acceptance: Use 'device' tags to check availability in QEMU
    binary
  tests/acceptance/virtio_check_params: Only remove listed machines
  tests/acceptance/virtio_check_params: Improve exception logging
  tests/acceptance/virtio_check_params: List machine being tested
  tests/acceptance/virtio_check_params: Only test Xen as superuser
  tests/acceptance/virtio_check_params: Skip test if arch is not
    supported
  tests/acceptance/virtio_check_params: Kludge to skip tests on MIPS
  tests/acceptance/virtio_check_params: Support the s390x architecture
  tests/acceptance/virtio_check_params: Support the ppc architecture
  tests/acceptance/virtio_check_params: Default to -nodefaults
  tests/acceptance/virtio_check_params: Require a virtio-scsi-pci device
  tests/acceptance/boot_linux_console: Do not use VGA on Clipper machine
  tests/acceptance/migration: Default to -nodefaults
  tests/acceptance/version: Default to -nodefaults
  MAINTAINERS: Add Acceptance tests reviewers

 accel/accel.c                                 |   1 +
 hw/core/machine.c                             |   3 +-
 MAINTAINERS                                   |   6 +
 python/qemu/accel.py                          |  26 +----
 python/qemu/binutils.py                       | 107 ++++++++++++++++++
 tests/acceptance/avocado_qemu/__init__.py     |  51 ++++++++-
 tests/acceptance/boot_linux_console.py        |   2 +-
 tests/acceptance/core_scripts.py              |  79 +++++++++++++
 tests/acceptance/migration.py                 |   4 +-
 tests/acceptance/version.py                   |   1 +
 ...g_max_adjust.py => virtio_check_params.py} |  70 +++++++++---
 tests/acceptance/x86_cpu_model_versions.py    |   1 +
 12 files changed, 306 insertions(+), 45 deletions(-)
 create mode 100644 python/qemu/binutils.py
 create mode 100644 tests/acceptance/core_scripts.py
 rename tests/acceptance/{virtio_seg_max_adjust.py => virtio_check_params.py} (64%)

-- 
2.21.1


Re: [PATCH v2 00/29] tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86
Posted by Philippe Mathieu-Daudé 4 years, 3 months ago
Wrong subject... should be:

   "Improve multi-arch/OS support of acceptance tests"

On 1/29/20 10:23 PM, Philippe Mathieu-Daudé wrote:
> Currently our CI is mostly restricted to Linux/X86.
> We want to have more, but until there we have to run these tests
> manually, which is why we didn't noticed earlier this test was
> failing on OSX.
> 
> This series contains the patches I used to debug why the
> virtio_seg_max_adjust had broken the OSX CI, but then I
> kept testing on other archs, thus added more patches...
> 
> Currently our CI is mostly restricted to Linux/X86.
> We want to have more, but until there we have to run these tests
> manually, which is why we didn't noticed earlier some tests are
> failing (on OSX, using --without-default-devices, ...).
> 
> Tested on Linux/FreeBSD/OSX, X86/Aarch64.
> 
> Regards,
> 
> Phil.
> 
> Supersedes: <20200122223247.30419-1-philmd@redhat.com>
> "tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86"
> 
> Denis Plotnikov (1):
>    tests: rename virtio_seg_max_adjust to virtio_check_params
> 
> Philippe Mathieu-Daudé (28):
>    hw/core: Allow setting 'virtio-blk-device.scsi' property on OSX host
>    accel/accel: Make TYPE_ACCEL abstract
>    python/qemu: Add binutils::binary_get_version()
>    tests/acceptance: Use 'version-min' tag to verify QEMU binary version
>    tests/acceptance: Restrict X86CPUModelAliases test to QEMU >= 4.1
>    python/qemu: Add binutils::binary_get_arch()
>    tests/acceptance: Use the 'arch' tag to verify QEMU binary target
>    python/qemu: Add binutils::binary_get_machines()
>    tests/acceptance: Use 'machine' tag to check if available in QEMU
>      binary
>    python/qemu: Add binutils::binary_get_qom_implementations()
>    python/qemu: Add binutils::binary_get_accels()
>    python/qemu/accel: Use binutils::binary_get_accels()
>    python/qemu: Add binutils::binary_get_devices()
>    tests/acceptance: Use 'device' tags to check availability in QEMU
>      binary
>    tests/acceptance/virtio_check_params: Only remove listed machines
>    tests/acceptance/virtio_check_params: Improve exception logging
>    tests/acceptance/virtio_check_params: List machine being tested
>    tests/acceptance/virtio_check_params: Only test Xen as superuser
>    tests/acceptance/virtio_check_params: Skip test if arch is not
>      supported
>    tests/acceptance/virtio_check_params: Kludge to skip tests on MIPS
>    tests/acceptance/virtio_check_params: Support the s390x architecture
>    tests/acceptance/virtio_check_params: Support the ppc architecture
>    tests/acceptance/virtio_check_params: Default to -nodefaults
>    tests/acceptance/virtio_check_params: Require a virtio-scsi-pci device
>    tests/acceptance/boot_linux_console: Do not use VGA on Clipper machine
>    tests/acceptance/migration: Default to -nodefaults
>    tests/acceptance/version: Default to -nodefaults
>    MAINTAINERS: Add Acceptance tests reviewers
> 
>   accel/accel.c                                 |   1 +
>   hw/core/machine.c                             |   3 +-
>   MAINTAINERS                                   |   6 +
>   python/qemu/accel.py                          |  26 +----
>   python/qemu/binutils.py                       | 107 ++++++++++++++++++
>   tests/acceptance/avocado_qemu/__init__.py     |  51 ++++++++-
>   tests/acceptance/boot_linux_console.py        |   2 +-
>   tests/acceptance/core_scripts.py              |  79 +++++++++++++
>   tests/acceptance/migration.py                 |   4 +-
>   tests/acceptance/version.py                   |   1 +
>   ...g_max_adjust.py => virtio_check_params.py} |  70 +++++++++---
>   tests/acceptance/x86_cpu_model_versions.py    |   1 +
>   12 files changed, 306 insertions(+), 45 deletions(-)
>   create mode 100644 python/qemu/binutils.py
>   create mode 100644 tests/acceptance/core_scripts.py
>   rename tests/acceptance/{virtio_seg_max_adjust.py => virtio_check_params.py} (64%)
> 


Re: [PATCH v2 00/29] tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86
Posted by Philippe Mathieu-Daudé 4 years, 2 months ago
On 1/29/20 10:23 PM, Philippe Mathieu-Daudé wrote:
> Currently our CI is mostly restricted to Linux/X86.
> We want to have more, but until there we have to run these tests
> manually, which is why we didn't noticed earlier this test was
> failing on OSX.
> 
> This series contains the patches I used to debug why the
> virtio_seg_max_adjust had broken the OSX CI, but then I
> kept testing on other archs, thus added more patches...
> 
> Currently our CI is mostly restricted to Linux/X86.
> We want to have more, but until there we have to run these tests
> manually, which is why we didn't noticed earlier some tests are
> failing (on OSX, using --without-default-devices, ...).
> 
> Tested on Linux/FreeBSD/OSX, X86/Aarch64.
> 
> Regards,
> 
> Phil.
> 
> Supersedes: <20200122223247.30419-1-philmd@redhat.com>
> "tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86"
> 
> Denis Plotnikov (1):
>    tests: rename virtio_seg_max_adjust to virtio_check_params
> 
> Philippe Mathieu-Daudé (28):
>    hw/core: Allow setting 'virtio-blk-device.scsi' property on OSX host
>    accel/accel: Make TYPE_ACCEL abstract
>    python/qemu: Add binutils::binary_get_version()
>    tests/acceptance: Use 'version-min' tag to verify QEMU binary version
>    tests/acceptance: Restrict X86CPUModelAliases test to QEMU >= 4.1
>    python/qemu: Add binutils::binary_get_arch()
>    tests/acceptance: Use the 'arch' tag to verify QEMU binary target
>    python/qemu: Add binutils::binary_get_machines()
>    tests/acceptance: Use 'machine' tag to check if available in QEMU
>      binary
>    python/qemu: Add binutils::binary_get_qom_implementations()
>    python/qemu: Add binutils::binary_get_accels()
>    python/qemu/accel: Use binutils::binary_get_accels()
>    python/qemu: Add binutils::binary_get_devices()
>    tests/acceptance: Use 'device' tags to check availability in QEMU
>      binary
>    tests/acceptance/virtio_check_params: Only remove listed machines
>    tests/acceptance/virtio_check_params: Improve exception logging
>    tests/acceptance/virtio_check_params: List machine being tested
>    tests/acceptance/virtio_check_params: Only test Xen as superuser
>    tests/acceptance/virtio_check_params: Skip test if arch is not
>      supported
>    tests/acceptance/virtio_check_params: Kludge to skip tests on MIPS
>    tests/acceptance/virtio_check_params: Support the s390x architecture
>    tests/acceptance/virtio_check_params: Support the ppc architecture
>    tests/acceptance/virtio_check_params: Default to -nodefaults
>    tests/acceptance/virtio_check_params: Require a virtio-scsi-pci device
>    tests/acceptance/boot_linux_console: Do not use VGA on Clipper machine
>    tests/acceptance/migration: Default to -nodefaults
>    tests/acceptance/version: Default to -nodefaults
>    MAINTAINERS: Add Acceptance tests reviewers

I have applied the following patches on my python-next tree [*]:
15/29 tests: rename virtio_seg_max_adjust to virtio_check_params
17/29 tests/acceptance/virtio_check_params: Improve exception logging
18/29 tests/acceptance/virtio_check_params: List machine being tested
24/29 tests/acceptance/virtio_check_params: Default to -nodefaults
26/29 tests/acceptance/boot_linux_console: Do not use VGA on Clipper machine
27/29 tests/acceptance/migration: Default to -nodefaults
28/29 tests/acceptance/version: Default to -nodefaults

I plan to disable the virtio_check_params.py test for now, because it 
broke the CI and the Virtuozzo folks didn't reply to this series 
intending to fix it. We will restore it once it later.

[*] https://gitlab.com/philmd/qemu/commits/python-next


[PATCH v2 15/29] tests: rename virtio_seg_max_adjust to virtio_check_params
Posted by Philippe Mathieu-Daudé 4 years, 3 months ago
From: Denis Plotnikov <dplotnikov@virtuozzo.com>

Since, virtio_seg_max_adjust checks not only seg_max, but also
virtqueue_size parameter, let's make the test more general and
add new parameters to be checked there in the future.

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
Message-Id: <20200129140702.5411-5-dplotnikov@virtuozzo.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 .../{virtio_seg_max_adjust.py => virtio_check_params.py}          | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename tests/acceptance/{virtio_seg_max_adjust.py => virtio_check_params.py} (100%)

diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_check_params.py
similarity index 100%
rename from tests/acceptance/virtio_seg_max_adjust.py
rename to tests/acceptance/virtio_check_params.py
-- 
2.21.1


Re: [PATCH v2 15/29] tests: rename virtio_seg_max_adjust to virtio_check_params
Posted by Philippe Mathieu-Daudé 4 years, 3 months ago
On 1/29/20 10:23 PM, Philippe Mathieu-Daudé wrote:
> From: Denis Plotnikov <dplotnikov@virtuozzo.com>
> 
> Since, virtio_seg_max_adjust checks not only seg_max, but also
> virtqueue_size parameter, let's make the test more general and
> add new parameters to be checked there in the future.
> 
> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> Message-Id: <20200129140702.5411-5-dplotnikov@virtuozzo.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>   .../{virtio_seg_max_adjust.py => virtio_check_params.py}          | 0
>   1 file changed, 0 insertions(+), 0 deletions(-)
>   rename tests/acceptance/{virtio_seg_max_adjust.py => virtio_check_params.py} (100%)
> 
> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_check_params.py
> similarity index 100%
> rename from tests/acceptance/virtio_seg_max_adjust.py
> rename to tests/acceptance/virtio_check_params.py
> 

Thanks, applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next


Re: [PATCH v2 15/29] tests: rename virtio_seg_max_adjust to virtio_check_params
Posted by Cornelia Huck 4 years, 3 months ago
On Wed, 29 Jan 2020 22:23:31 +0100
Philippe Mathieu-Daudé <philmd@redhat.com> wrote:

> From: Denis Plotnikov <dplotnikov@virtuozzo.com>
> 
> Since, virtio_seg_max_adjust checks not only seg_max, but also
> virtqueue_size parameter, let's make the test more general and
> add new parameters to be checked there in the future.
> 
> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
> Message-Id: <20200129140702.5411-5-dplotnikov@virtuozzo.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  .../{virtio_seg_max_adjust.py => virtio_check_params.py}          | 0
>  1 file changed, 0 insertions(+), 0 deletions(-)
>  rename tests/acceptance/{virtio_seg_max_adjust.py => virtio_check_params.py} (100%)
> 
> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_check_params.py
> similarity index 100%
> rename from tests/acceptance/virtio_seg_max_adjust.py
> rename to tests/acceptance/virtio_check_params.py

Makes sense, especially if we want to add non-blk/scsi checks in there.

Acked-by: Cornelia Huck <cohuck@redhat.com>


python/qemu: Refactor QemuBinaryInfo
Posted by Philippe Mathieu-Daudé 3 years, 11 months ago
On 1/29/20 10:23 PM, Philippe Mathieu-Daudé wrote:
[...]
>   accel/accel: Make TYPE_ACCEL abstract
>   python/qemu: Add binutils::binary_get_version()
>   tests/acceptance: Use 'version-min' tag to verify QEMU binary version
>   tests/acceptance: Restrict X86CPUModelAliases test to QEMU >= 4.1
>   python/qemu: Add binutils::binary_get_arch()
>   tests/acceptance: Use the 'arch' tag to verify QEMU binary target
>   python/qemu: Add binutils::binary_get_machines()
>   tests/acceptance: Use 'machine' tag to check if available in QEMU
>     binary
>   python/qemu: Add binutils::binary_get_qom_implementations()
>   python/qemu: Add binutils::binary_get_accels()
>   python/qemu/accel: Use binutils::binary_get_accels()
>   python/qemu: Add binutils::binary_get_devices()

I just noticed various of these methods are already implemented in
QemuBinaryInfo (see scripts/device-crash-test), so I guess it'd be
better to start refactoring from there (to the recent QEMU package).

> 
>  accel/accel.c                                 |   1 +
>  hw/core/machine.c                             |   3 +-
>  MAINTAINERS                                   |   6 +
>  python/qemu/accel.py                          |  26 +----
>  python/qemu/binutils.py                       | 107 ++++++++++++++++++
[...]