tests/acceptance: Do not install paramiko module by default

[Qemu-devel] [PATCH] tests/acceptance: Do not install paramiko module by default

Posted by Philippe Mathieu-Daudé 4 years, 7 months ago

The paramiko Python module has many dependencies. Some of them
are not pure Python, such cryptography module which requires to
be built and linked with OpenSSL.

When native libraries and header are missing on the host, the
error reported is not very helpful:

  $ make check-venv
  VENV    tests/venv
  PIP     tests/requirements.txt
  Command "tests/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-la4el5r5/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-1efs22iz-record/install-record.txt --single-version-externally-managed --compile --install-headers tests/venv/include/site/python3.6/cryptography" failed with error code 1 in /tmp/pip-build-la4el5r5/cryptography/

Since currently the tests depending on paramiko are targetting
very specific uses, we can avoid the strong dependency on the
paramiko module, to let systems where it does not build properly
work the rest of the tests.
If paramiko is manually installed, the tests using it will be
automatically run.

Fixes: c47c336e870
Reported-by: David Gibson <david@gibson.dropbear.id.au>
Suggested-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/acceptance/linux_ssh_mips_malta.py | 13 +++++++++++--
 tests/requirements.txt                   |  1 -
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
index aafb0c39f6..dbcdc03a80 100644
--- a/tests/acceptance/linux_ssh_mips_malta.py
+++ b/tests/acceptance/linux_ssh_mips_malta.py
@@ -9,14 +9,19 @@ import os
 import re
 import base64
 import logging
-import paramiko
 import time
 
-from avocado import skipIf
+from avocado import skipIf, skipUnless
 from avocado_qemu import Test
 from avocado.utils import process
 from avocado.utils import archive
 
+PARAMIKO_AVAILABLE = True
+try:
+    import paramiko
+except ImportError:
+    PARAMIKO_AVAILABLE = False
+
 
 class LinuxSSH(Test):
 
@@ -171,6 +176,7 @@ class LinuxSSH(Test):
         self.run_common_commands()
         self.shutdown_via_ssh()
 
+    @skipUnless(PARAMIKO_AVAILABLE, "paramiko module not available")
     @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
     def test_mips_malta32eb_kernel3_2_0(self):
         """
@@ -186,6 +192,7 @@ class LinuxSSH(Test):
 
         self.do_test_mips_malta('be', kernel_path, 'mips')
 
+    @skipUnless(PARAMIKO_AVAILABLE, "paramiko module not available")
     @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
     def test_mips_malta32el_kernel3_2_0(self):
         """
@@ -201,6 +208,7 @@ class LinuxSSH(Test):
 
         self.do_test_mips_malta('le', kernel_path, 'mips')
 
+    @skipUnless(PARAMIKO_AVAILABLE, "paramiko module not available")
     @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
     def test_mips_malta64eb_kernel3_2_0(self):
         """
@@ -215,6 +223,7 @@ class LinuxSSH(Test):
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
         self.do_test_mips_malta('be', kernel_path, 'mips64')
 
+    @skipUnless(PARAMIKO_AVAILABLE, "paramiko module not available")
     @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
     def test_mips_malta64el_kernel3_2_0(self):
         """
diff --git a/tests/requirements.txt b/tests/requirements.txt
index 3ae0e29ad7..002ded6a22 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -2,4 +2,3 @@
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
 avocado-framework==68.0
-paramiko
-- 
2.20.1

Re: [Qemu-devel] [PATCH] tests/acceptance: Do not install paramiko module by default

Posted by Eduardo Habkost 4 years, 7 months ago

On Fri, Aug 30, 2019 at 08:40:33PM +0200, Philippe Mathieu-Daudé wrote:
> The paramiko Python module has many dependencies. Some of them
> are not pure Python, such cryptography module which requires to
> be built and linked with OpenSSL.
> 
> When native libraries and header are missing on the host, the
> error reported is not very helpful:
> 
>   $ make check-venv
>   VENV    tests/venv
>   PIP     tests/requirements.txt
>   Command "tests/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-la4el5r5/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-1efs22iz-record/install-record.txt --single-version-externally-managed --compile --install-headers tests/venv/include/site/python3.6/cryptography" failed with error code 1 in /tmp/pip-build-la4el5r5/cryptography/
> 
> Since currently the tests depending on paramiko are targetting
> very specific uses, we can avoid the strong dependency on the
> paramiko module, to let systems where it does not build properly
> work the rest of the tests.
> If paramiko is manually installed, the tests using it will be
> automatically run.
> 
> Fixes: c47c336e870
> Reported-by: David Gibson <david@gibson.dropbear.id.au>
> Suggested-by: Cleber Rosa <crosa@redhat.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Thanks!  Queued on python-next.

-- 
Eduardo

Re: [Qemu-devel] [PATCH] tests/acceptance: Do not install paramiko module by default

Posted by David Gibson 4 years, 7 months ago

On Fri, Aug 30, 2019 at 03:51:39PM -0300, Eduardo Habkost wrote:
> On Fri, Aug 30, 2019 at 08:40:33PM +0200, Philippe Mathieu-Daudé wrote:
> > The paramiko Python module has many dependencies. Some of them
> > are not pure Python, such cryptography module which requires to
> > be built and linked with OpenSSL.
> > 
> > When native libraries and header are missing on the host, the
> > error reported is not very helpful:
> > 
> >   $ make check-venv
> >   VENV    tests/venv
> >   PIP     tests/requirements.txt
> >   Command "tests/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-la4el5r5/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-1efs22iz-record/install-record.txt --single-version-externally-managed --compile --install-headers tests/venv/include/site/python3.6/cryptography" failed with error code 1 in /tmp/pip-build-la4el5r5/cryptography/
> > 
> > Since currently the tests depending on paramiko are targetting
> > very specific uses, we can avoid the strong dependency on the
> > paramiko module, to let systems where it does not build properly
> > work the rest of the tests.
> > If paramiko is manually installed, the tests using it will be
> > automatically run.
> > 
> > Fixes: c47c336e870
> > Reported-by: David Gibson <david@gibson.dropbear.id.au>
> > Suggested-by: Cleber Rosa <crosa@redhat.com>
> > Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> 
> Thanks!  Queued on python-next.

Yay!

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson