One of the Avocado features relevant to virtualization testing is the
ability to reuse tests in different scenarios, known as variants.
This adds a JSON based variants file, that can be used to run most
tests in a number of different architectures. It can be run with:
$ avocado run \
--json-variants-load=tests/acceptance/variants/arch.json \
--filter-by-tags='-x86_64' -- tests/acceptance/
Currently this covers 5 architectures, resulting in the execution
of 25 different combinations.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
tests/acceptance/variants/arch.json | 92 +++++++++++++++++++++++++++++
1 file changed, 92 insertions(+)
create mode 100644 tests/acceptance/variants/arch.json
diff --git a/tests/acceptance/variants/arch.json b/tests/acceptance/variants/arch.json
new file mode 100644
index 0000000000..64ff6ad278
--- /dev/null
+++ b/tests/acceptance/variants/arch.json
@@ -0,0 +1,92 @@
+[
+ {
+ "paths": [
+ "/run/*"
+ ],
+ "variant": [
+ [
+ "/run/aarch64",
+ [
+ [
+ "/run/aarch64",
+ "arch",
+ "aarch64"
+ ]
+ ]
+ ]
+ ],
+ "variant_id": "aarch64"
+ },
+ {
+ "paths": [
+ "/run/*"
+ ],
+ "variant": [
+ [
+ "/run/ppc",
+ [
+ [
+ "/run/ppc",
+ "arch",
+ "ppc"
+ ]
+ ]
+ ]
+ ],
+ "variant_id": "ppc"
+ },
+ {
+ "paths": [
+ "/run/*"
+ ],
+ "variant": [
+ [
+ "/run/ppc64",
+ [
+ [
+ "/run/ppc64",
+ "arch",
+ "ppc64"
+ ]
+ ]
+ ]
+ ],
+ "variant_id": "ppc64"
+ },
+ {
+ "paths": [
+ "/run/*"
+ ],
+ "variant": [
+ [
+ "/run/s390x",
+ [
+ [
+ "/run/s390x",
+ "arch",
+ "s390x"
+ ]
+ ]
+ ]
+ ],
+ "variant_id": "s390x"
+ },
+ {
+ "paths": [
+ "/run/*"
+ ],
+ "variant": [
+ [
+ "/run/x86_64",
+ [
+ [
+ "/run/x86_64",
+ "arch",
+ "x86_64"
+ ]
+ ]
+ ]
+ ],
+ "variant_id": "x86_64"
+ }
+]
--
2.17.1
Hi Cleber,
On 10/10/2018 01:26, Cleber Rosa wrote:
> One of the Avocado features relevant to virtualization testing is the
> ability to reuse tests in different scenarios, known as variants.
> This adds a JSON based variants file, that can be used to run most
> tests in a number of different architectures. It can be run with:
>
> $ avocado run \
> --json-variants-load=tests/acceptance/variants/arch.json \
> --filter-by-tags='-x86_64' -- tests/acceptance/
>
> Currently this covers 5 architectures, resulting in the execution
> of 25 different combinations.
Can you provide here the command you used to generate this JSON?
Thanks,
Phil.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> tests/acceptance/variants/arch.json | 92 +++++++++++++++++++++++++++++
> 1 file changed, 92 insertions(+)
> create mode 100644 tests/acceptance/variants/arch.json
>
> diff --git a/tests/acceptance/variants/arch.json b/tests/acceptance/variants/arch.json
> new file mode 100644
> index 0000000000..64ff6ad278
> --- /dev/null
> +++ b/tests/acceptance/variants/arch.json
> @@ -0,0 +1,92 @@
> +[
> + {
> + "paths": [
> + "/run/*"
> + ],
> + "variant": [
> + [
> + "/run/aarch64",
> + [
> + [
> + "/run/aarch64",
> + "arch",
> + "aarch64"
> + ]
> + ]
> + ]
> + ],
> + "variant_id": "aarch64"
> + },
> + {
> + "paths": [
> + "/run/*"
> + ],
> + "variant": [
> + [
> + "/run/ppc",
> + [
> + [
> + "/run/ppc",
> + "arch",
> + "ppc"
> + ]
> + ]
> + ]
> + ],
> + "variant_id": "ppc"
> + },
> + {
> + "paths": [
> + "/run/*"
> + ],
> + "variant": [
> + [
> + "/run/ppc64",
> + [
> + [
> + "/run/ppc64",
> + "arch",
> + "ppc64"
> + ]
> + ]
> + ]
> + ],
> + "variant_id": "ppc64"
> + },
> + {
> + "paths": [
> + "/run/*"
> + ],
> + "variant": [
> + [
> + "/run/s390x",
> + [
> + [
> + "/run/s390x",
> + "arch",
> + "s390x"
> + ]
> + ]
> + ]
> + ],
> + "variant_id": "s390x"
> + },
> + {
> + "paths": [
> + "/run/*"
> + ],
> + "variant": [
> + [
> + "/run/x86_64",
> + [
> + [
> + "/run/x86_64",
> + "arch",
> + "x86_64"
> + ]
> + ]
> + ]
> + ],
> + "variant_id": "x86_64"
> + }
> +]
>
On 10/10/18 6:51 AM, Philippe Mathieu-Daudé wrote:
> Hi Cleber,
>
> On 10/10/2018 01:26, Cleber Rosa wrote:
>> One of the Avocado features relevant to virtualization testing is the
>> ability to reuse tests in different scenarios, known as variants.
>> This adds a JSON based variants file, that can be used to run most
>> tests in a number of different architectures. It can be run with:
>>
>> $ avocado run \
>> --json-variants-load=tests/acceptance/variants/arch.json \
>> --filter-by-tags='-x86_64' -- tests/acceptance/
>>
>> Currently this covers 5 architectures, resulting in the execution
>> of 25 different combinations.
>
> Can you provide here the command you used to generate this JSON?
>
Yep, there are many ways to do it, one is to have the
varianter-yaml-to-mux plugin installed, and create a YAML file such as:
---- archs.yaml START -----
!mux
x86_64:
arch: x86_64
aarch64:
arch: aarch64
ppc64:
arch: ppc64
ppc:
arch: ppc
s390x:
arch: s390x
---- archs.yaml END -----
And run:
$ avocado run -m archs.yaml -- /bin/true
$ python -m json.tool ~/avocado/job-results/latest/jobdata/variants.json
I did tweak the "variant_id" value of the resulting JSON file, because
the hashes are somewhat specific to yaml-to-mux, and don't add anything
to the meaning of the variants.
- Cleber.
> Thanks,
>
> Phil.
>
>>
>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>> ---
>> tests/acceptance/variants/arch.json | 92 +++++++++++++++++++++++++++++
>> 1 file changed, 92 insertions(+)
>> create mode 100644 tests/acceptance/variants/arch.json
>>
>> diff --git a/tests/acceptance/variants/arch.json b/tests/acceptance/variants/arch.json
>> new file mode 100644
>> index 0000000000..64ff6ad278
>> --- /dev/null
>> +++ b/tests/acceptance/variants/arch.json
>> @@ -0,0 +1,92 @@
>> +[
>> + {
>> + "paths": [
>> + "/run/*"
>> + ],
>> + "variant": [
>> + [
>> + "/run/aarch64",
>> + [
>> + [
>> + "/run/aarch64",
>> + "arch",
>> + "aarch64"
>> + ]
>> + ]
>> + ]
>> + ],
>> + "variant_id": "aarch64"
>> + },
>> + {
>> + "paths": [
>> + "/run/*"
>> + ],
>> + "variant": [
>> + [
>> + "/run/ppc",
>> + [
>> + [
>> + "/run/ppc",
>> + "arch",
>> + "ppc"
>> + ]
>> + ]
>> + ]
>> + ],
>> + "variant_id": "ppc"
>> + },
>> + {
>> + "paths": [
>> + "/run/*"
>> + ],
>> + "variant": [
>> + [
>> + "/run/ppc64",
>> + [
>> + [
>> + "/run/ppc64",
>> + "arch",
>> + "ppc64"
>> + ]
>> + ]
>> + ]
>> + ],
>> + "variant_id": "ppc64"
>> + },
>> + {
>> + "paths": [
>> + "/run/*"
>> + ],
>> + "variant": [
>> + [
>> + "/run/s390x",
>> + [
>> + [
>> + "/run/s390x",
>> + "arch",
>> + "s390x"
>> + ]
>> + ]
>> + ]
>> + ],
>> + "variant_id": "s390x"
>> + },
>> + {
>> + "paths": [
>> + "/run/*"
>> + ],
>> + "variant": [
>> + [
>> + "/run/x86_64",
>> + [
>> + [
>> + "/run/x86_64",
>> + "arch",
>> + "x86_64"
>> + ]
>> + ]
>> + ]
>> + ],
>> + "variant_id": "x86_64"
>> + }
>> +]
>>
--
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ]
On 10/10/2018 01:26, Cleber Rosa wrote: > One of the Avocado features relevant to virtualization testing is the > ability to reuse tests in different scenarios, known as variants. > This adds a JSON based variants file, that can be used to run most > tests in a number of different architectures. It can be run with: > > $ avocado run \ > --json-variants-load=tests/acceptance/variants/arch.json \ > --filter-by-tags='-x86_64' -- tests/acceptance/ > > Currently this covers 5 architectures, resulting in the execution > of 25 different combinations. IMHO we should add this file with all supported archs, and CANCEL a job if there is no preferred machine in Qemu::MACHINE_TYPES. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > --- > tests/acceptance/variants/arch.json | 92 +++++++++++++++++++++++++++++ > 1 file changed, 92 insertions(+) > create mode 100644 tests/acceptance/variants/arch.json
On 10/10/18 6:59 AM, Philippe Mathieu-Daudé wrote: > On 10/10/2018 01:26, Cleber Rosa wrote: >> One of the Avocado features relevant to virtualization testing is the >> ability to reuse tests in different scenarios, known as variants. >> This adds a JSON based variants file, that can be used to run most >> tests in a number of different architectures. It can be run with: >> >> $ avocado run \ >> --json-variants-load=tests/acceptance/variants/arch.json \ >> --filter-by-tags='-x86_64' -- tests/acceptance/ >> >> Currently this covers 5 architectures, resulting in the execution >> of 25 different combinations. > > IMHO we should add this file with all supported archs, and CANCEL a job > if there is no preferred machine in Qemu::MACHINE_TYPES. > While your proposal works, I see two issues: 1) It defines a policy, which will affect all tests, existing and new - even for tests that don't launch a VM and don't care about the arch. 2) CANCEL is mostly silent, and the amount of noise in test results would be quite large. So, the real problem is that there are other issues when making basic use of QEMUMachine with other archs. The idea is to incrementally test other archs, make the modifications necessary to scripts/qemu.py, and end up with all architectures covered. What do you think? - Cleber. >> >> Signed-off-by: Cleber Rosa <crosa@redhat.com> >> --- >> tests/acceptance/variants/arch.json | 92 +++++++++++++++++++++++++++++ >> 1 file changed, 92 insertions(+) >> create mode 100644 tests/acceptance/variants/arch.json
© 2016 - 2026 Red Hat, Inc.