[PATCH v2 00/10] Acceptance Test: introduce base class for Linux based tests

Cleber Rosa posted 10 patches 3 years ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20210323221539.3532660-1-crosa@redhat.com
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <philmd@redhat.com>, Willian Rampazzo <willianr@redhat.com>, Aurelien Jarno <aurelien@aurel32.net>, John Snow <jsnow@redhat.com>, Wainer dos Santos Moschetta <wainersm@redhat.com>, Thomas Huth <thuth@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>, Cleber Rosa <crosa@redhat.com>
There is a newer version of this series
docs/devel/testing.rst                    | 25 ++++++++
python/qemu/utils.py                      | 35 ++++++++++++
tests/acceptance/avocado_qemu/__init__.py | 63 +++++++++++++++++++--
tests/acceptance/hotplug_cpu.py           | 37 ++++++++++++
tests/acceptance/info_usernet.py          | 29 ++++++++++
tests/acceptance/linux_ssh_mips_malta.py  | 44 ++-------------
tests/acceptance/virtiofs_submounts.py    | 69 +++--------------------
tests/vm/basevm.py                        |  7 +--
8 files changed, 198 insertions(+), 111 deletions(-)
create mode 100644 python/qemu/utils.py
create mode 100644 tests/acceptance/hotplug_cpu.py
create mode 100644 tests/acceptance/info_usernet.py
[PATCH v2 00/10] Acceptance Test: introduce base class for Linux based tests
Posted by Cleber Rosa 3 years ago
This introduces a base class for tests that need to interact with a
Linux guest.  It generalizes the "boot_linux.py" code, already been
used by the "virtiofs_submounts.py" and also SSH related code being
used by that and "linux_ssh_mips_malta.py".

While at it, a number of fixes on hopeful improvements to those tests
were added.

Changes from v1:

* Majority of v1 patches have been merged.

* New patches:
  - Acceptance Tests: make username/password configurable
  - Acceptance Tests: set up SSH connection by default after boot for LinuxTest
  - tests/acceptance/virtiofs_submounts.py: remove launch_vm()

* Allowed for the configuration of the network device type (defaulting
  to virtio-net) [Phil]

* Fix module name typo (s/qemu.util/qemu.utils/) in the commit message
  [John]

* Tests based on LinuxTest will have the SSH connection already prepared

Cleber Rosa (10):
  tests/acceptance/virtiofs_submounts.py: add missing accel tag
  tests/acceptance/virtiofs_submounts.py: evaluate string not length
  Python: add utility function for retrieving port redirection
  Acceptance Tests: move useful ssh methods to base class
  Acceptance Tests: add port redirection for ssh by default
  Acceptance Tests: make username/password configurable
  Acceptance Tests: set up SSH connection by default after boot for
    LinuxTest
  tests/acceptance/virtiofs_submounts.py: remove launch_vm()
  Acceptance Tests: add basic documentation on LinuxTest base class
  Acceptance Tests: introduce CPU hotplug test

 docs/devel/testing.rst                    | 25 ++++++++
 python/qemu/utils.py                      | 35 ++++++++++++
 tests/acceptance/avocado_qemu/__init__.py | 63 +++++++++++++++++++--
 tests/acceptance/hotplug_cpu.py           | 37 ++++++++++++
 tests/acceptance/info_usernet.py          | 29 ++++++++++
 tests/acceptance/linux_ssh_mips_malta.py  | 44 ++-------------
 tests/acceptance/virtiofs_submounts.py    | 69 +++--------------------
 tests/vm/basevm.py                        |  7 +--
 8 files changed, 198 insertions(+), 111 deletions(-)
 create mode 100644 python/qemu/utils.py
 create mode 100644 tests/acceptance/hotplug_cpu.py
 create mode 100644 tests/acceptance/info_usernet.py

-- 
2.25.4



Re: [PATCH v2 00/10] Acceptance Test: introduce base class for Linux based tests
Posted by Wainer dos Santos Moschetta 3 years ago
Hi,

On 3/23/21 7:15 PM, Cleber Rosa wrote:
> This introduces a base class for tests that need to interact with a
> Linux guest.  It generalizes the "boot_linux.py" code, already been
> used by the "virtiofs_submounts.py" and also SSH related code being
> used by that and "linux_ssh_mips_malta.py".

I ran the linux_ssh_mips_malta.py tests, they all passed:

(11/34) 
tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta32eb_kernel3_2_0: 
PASS (64.41 s)
(12/34) 
tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta32el_kernel3_2_0: 
PASS (63.43 s)
(13/34) 
tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta64eb_kernel3_2_0: 
PASS (63.76 s)
(14/34) 
tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta64el_kernel3_2_0: 
PASS (62.52 s)

Then I tried the virtiofs_submounts.py tests, it finishes with error. 
Something like that fixes it:

diff --git a/tests/acceptance/virtiofs_submounts.py 
b/tests/acceptance/virtiofs_submounts.py
index d77ee35674..21ad7d792e 100644
--- a/tests/acceptance/virtiofs_submounts.py
+++ b/tests/acceptance/virtiofs_submounts.py
@@ -195,7 +195,7 @@ def setUp(self):

          self.run(('ssh-keygen', '-N', '', '-t', 'ed25519', '-f', 
self.ssh_key))

-        pubkey = open(self.ssh_key + '.pub').read()
+        pubkey = self.ssh_key + '.pub'

          super(VirtiofsSubmountsTest, self).setUp(pubkey)


>
> While at it, a number of fixes on hopeful improvements to those tests
> were added.
>
> Changes from v1:
>
> * Majority of v1 patches have been merged.
>
> * New patches:
>    - Acceptance Tests: make username/password configurable
>    - Acceptance Tests: set up SSH connection by default after boot for LinuxTest
>    - tests/acceptance/virtiofs_submounts.py: remove launch_vm()
>
> * Allowed for the configuration of the network device type (defaulting
>    to virtio-net) [Phil]
>
> * Fix module name typo (s/qemu.util/qemu.utils/) in the commit message
>    [John]
>
> * Tests based on LinuxTest will have the SSH connection already prepared
>
> Cleber Rosa (10):
>    tests/acceptance/virtiofs_submounts.py: add missing accel tag
>    tests/acceptance/virtiofs_submounts.py: evaluate string not length
>    Python: add utility function for retrieving port redirection
>    Acceptance Tests: move useful ssh methods to base class
>    Acceptance Tests: add port redirection for ssh by default
>    Acceptance Tests: make username/password configurable
>    Acceptance Tests: set up SSH connection by default after boot for
>      LinuxTest
>    tests/acceptance/virtiofs_submounts.py: remove launch_vm()
>    Acceptance Tests: add basic documentation on LinuxTest base class
>    Acceptance Tests: introduce CPU hotplug test
>
>   docs/devel/testing.rst                    | 25 ++++++++
>   python/qemu/utils.py                      | 35 ++++++++++++
>   tests/acceptance/avocado_qemu/__init__.py | 63 +++++++++++++++++++--
>   tests/acceptance/hotplug_cpu.py           | 37 ++++++++++++
>   tests/acceptance/info_usernet.py          | 29 ++++++++++
>   tests/acceptance/linux_ssh_mips_malta.py  | 44 ++-------------
>   tests/acceptance/virtiofs_submounts.py    | 69 +++--------------------
>   tests/vm/basevm.py                        |  7 +--
>   8 files changed, 198 insertions(+), 111 deletions(-)
>   create mode 100644 python/qemu/utils.py
>   create mode 100644 tests/acceptance/hotplug_cpu.py
>   create mode 100644 tests/acceptance/info_usernet.py
>


Re: [PATCH v2 00/10] Acceptance Test: introduce base class for Linux based tests
Posted by Auger Eric 3 years ago
Hi Wainer,

On 3/25/21 8:45 PM, Wainer dos Santos Moschetta wrote:
> Hi,
> 
> On 3/23/21 7:15 PM, Cleber Rosa wrote:
>> This introduces a base class for tests that need to interact with a
>> Linux guest.  It generalizes the "boot_linux.py" code, already been
>> used by the "virtiofs_submounts.py" and also SSH related code being
>> used by that and "linux_ssh_mips_malta.py".
> 
> I ran the linux_ssh_mips_malta.py tests, they all passed:
> 
> (11/34)
> tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta32eb_kernel3_2_0:
> PASS (64.41 s)
> (12/34)
> tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta32el_kernel3_2_0:
> PASS (63.43 s)
> (13/34)
> tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta64eb_kernel3_2_0:
> PASS (63.76 s)
> (14/34)
> tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta64el_kernel3_2_0:
> PASS (62.52 s)
> 
> Then I tried the virtiofs_submounts.py tests, it finishes with error.
> Something like that fixes it:
> 
> diff --git a/tests/acceptance/virtiofs_submounts.py
> b/tests/acceptance/virtiofs_submounts.py
> index d77ee35674..21ad7d792e 100644
> --- a/tests/acceptance/virtiofs_submounts.py
> +++ b/tests/acceptance/virtiofs_submounts.py
> @@ -195,7 +195,7 @@ def setUp(self):
> 
>          self.run(('ssh-keygen', '-N', '', '-t', 'ed25519', '-f',
> self.ssh_key))
> 
> -        pubkey = open(self.ssh_key + '.pub').read()
> +        pubkey = self.ssh_key + '.pub'

Yes I discovered that too when developping the SMMU test. Thanks for
mentionning

Eric
> 
>          super(VirtiofsSubmountsTest, self).setUp(pubkey)
> 
> 
>>
>> While at it, a number of fixes on hopeful improvements to those tests
>> were added.
>>
>> Changes from v1:
>>
>> * Majority of v1 patches have been merged.
>>
>> * New patches:
>>    - Acceptance Tests: make username/password configurable
>>    - Acceptance Tests: set up SSH connection by default after boot for
>> LinuxTest
>>    - tests/acceptance/virtiofs_submounts.py: remove launch_vm()
>>
>> * Allowed for the configuration of the network device type (defaulting
>>    to virtio-net) [Phil]
>>
>> * Fix module name typo (s/qemu.util/qemu.utils/) in the commit message
>>    [John]
>>
>> * Tests based on LinuxTest will have the SSH connection already prepared
>>
>> Cleber Rosa (10):
>>    tests/acceptance/virtiofs_submounts.py: add missing accel tag
>>    tests/acceptance/virtiofs_submounts.py: evaluate string not length
>>    Python: add utility function for retrieving port redirection
>>    Acceptance Tests: move useful ssh methods to base class
>>    Acceptance Tests: add port redirection for ssh by default
>>    Acceptance Tests: make username/password configurable
>>    Acceptance Tests: set up SSH connection by default after boot for
>>      LinuxTest
>>    tests/acceptance/virtiofs_submounts.py: remove launch_vm()
>>    Acceptance Tests: add basic documentation on LinuxTest base class
>>    Acceptance Tests: introduce CPU hotplug test
>>
>>   docs/devel/testing.rst                    | 25 ++++++++
>>   python/qemu/utils.py                      | 35 ++++++++++++
>>   tests/acceptance/avocado_qemu/__init__.py | 63 +++++++++++++++++++--
>>   tests/acceptance/hotplug_cpu.py           | 37 ++++++++++++
>>   tests/acceptance/info_usernet.py          | 29 ++++++++++
>>   tests/acceptance/linux_ssh_mips_malta.py  | 44 ++-------------
>>   tests/acceptance/virtiofs_submounts.py    | 69 +++--------------------
>>   tests/vm/basevm.py                        |  7 +--
>>   8 files changed, 198 insertions(+), 111 deletions(-)
>>   create mode 100644 python/qemu/utils.py
>>   create mode 100644 tests/acceptance/hotplug_cpu.py
>>   create mode 100644 tests/acceptance/info_usernet.py
>>


Re: [PATCH v2 00/10] Acceptance Test: introduce base class for Linux based tests
Posted by Cleber Rosa 2 years, 11 months ago
On Thu, Mar 25, 2021 at 04:45:51PM -0300, Wainer dos Santos Moschetta wrote:
> Hi,
> 
> On 3/23/21 7:15 PM, Cleber Rosa wrote:
> > This introduces a base class for tests that need to interact with a
> > Linux guest.  It generalizes the "boot_linux.py" code, already been
> > used by the "virtiofs_submounts.py" and also SSH related code being
> > used by that and "linux_ssh_mips_malta.py".
> 
> I ran the linux_ssh_mips_malta.py tests, they all passed:
> 
> (11/34) tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta32eb_kernel3_2_0:
> PASS (64.41 s)
> (12/34) tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta32el_kernel3_2_0:
> PASS (63.43 s)
> (13/34) tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta64eb_kernel3_2_0:
> PASS (63.76 s)
> (14/34) tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta64el_kernel3_2_0:
> PASS (62.52 s)
> 
> Then I tried the virtiofs_submounts.py tests, it finishes with error.
> Something like that fixes it:
> 
> diff --git a/tests/acceptance/virtiofs_submounts.py
> b/tests/acceptance/virtiofs_submounts.py
> index d77ee35674..21ad7d792e 100644
> --- a/tests/acceptance/virtiofs_submounts.py
> +++ b/tests/acceptance/virtiofs_submounts.py
> @@ -195,7 +195,7 @@ def setUp(self):
> 
>          self.run(('ssh-keygen', '-N', '', '-t', 'ed25519', '-f',
> self.ssh_key))
> 
> -        pubkey = open(self.ssh_key + '.pub').read()
> +        pubkey = self.ssh_key + '.pub'
> 
>          super(VirtiofsSubmountsTest, self).setUp(pubkey)
> 

Hi Wainer,

Yes, thank you so much for catching that and proposing a fix.  I'm
adding to the v3 of this series.

Thanks again!
- Cleber.