This enables the execution of the acceptance tests on Travis.
Because the Travis environment is based on Ubuntu Trusty, it requires
the python3-pip.
Note: while another supposedely required component on newer versions
(such as on Bionic) split the Python 3 installation further on the
python3-venv package.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
.travis.yml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 95be6ec59f..db1a31ea51 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -36,6 +36,7 @@ addons:
- liburcu-dev
- libusb-1.0-0-dev
- libvte-2.90-dev
+ - python3-pip
- sparse
- uuid-dev
- gcovr
@@ -117,6 +118,11 @@ matrix:
- env: CONFIG="--target-list=x86_64-softmmu"
python:
- "3.6"
+ # Acceptance (Functional) tests
+ - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
+ TEST_CMD="make check-acceptance"
+ python:
+ - "3.6"
# Using newer GCC with sanitizers
- addons:
apt:
--
2.17.1
Hi Cleber,
On 09/10/2018 06:18, Cleber Rosa wrote:
> This enables the execution of the acceptance tests on Travis.
Did you test this? =)
>
> Because the Travis environment is based on Ubuntu Trusty, it requires
> the python3-pip.
>
> Note: while another supposedely required component on newer versions
> (such as on Bionic) split the Python 3 installation further on the
> python3-venv package.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> .travis.yml | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 95be6ec59f..db1a31ea51 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -36,6 +36,7 @@ addons:
> - liburcu-dev
> - libusb-1.0-0-dev
> - libvte-2.90-dev
> + - python3-pip
> - sparse
> - uuid-dev
> - gcovr
> @@ -117,6 +118,11 @@ matrix:
> - env: CONFIG="--target-list=x86_64-softmmu"
> python:
> - "3.6"
> + # Acceptance (Functional) tests
> + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
> + TEST_CMD="make check-acceptance"
> + python:
> + - "3.6"
> # Using newer GCC with sanitizers
> - addons:
> apt:
>
Using the following patch:
-- >8 --
diff --git a/.travis.yml b/.travis.yml
index 95be6ec59f..87e0c9a13f 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -118,4 +118,15 @@ matrix:
python:
- "3.6"
+ # Acceptance (Functional) tests
+ - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
+ TEST_CMD="make check-acceptance"
+ # sudo rm /usr/local/bin/pip*
+ python:
+ - "3.6"
+ addons:
+ apt:
+ packages:
+ - python3-pip
+ - python3.4-venv
# Using newer GCC with sanitizers
- addons:
---
I got some improvements until:
VENV /home/travis/build/philmd/qemu/tests/venv
MKDIR /home/travis/build/philmd/qemu/tests/results
PIP /home/travis/build/philmd/qemu/tests/venv-requirements.txt
Exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in
main
status = self.run(options, args)
File "/usr/lib/python3/dist-packages/pip/commands/install.py", line
283, in run
requirement_set.install(install_options, global_options,
root=options.root_path)
File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install
requirement.install(install_options, global_options, *args, **kwargs)
File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install
self.move_wheel_files(self.source_dir, root=root)
File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in
move_wheel_files
pycompile=self.pycompile,
File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in
move_wheel_files
clobber(source, lib_dir, True)
File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber
os.makedirs(destsubdir)
File "/usr/lib/python3.4/os.py", line 237, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied:
'/usr/local/lib/python3.4/dist-packages/avocado'
See: https://travis-ci.org/philmd/qemu/jobs/439138706
Philippe Mathieu-Daudé <philmd@redhat.com> writes: > Hi Cleber, > > On 09/10/2018 06:18, Cleber Rosa wrote: >> This enables the execution of the acceptance tests on Travis. > > Did you test this? =) > >> >> Because the Travis environment is based on Ubuntu Trusty, it requires >> the python3-pip. >> >> Note: while another supposedely required component on newer versions >> (such as on Bionic) split the Python 3 installation further on the >> python3-venv package. >> >> Signed-off-by: Cleber Rosa <crosa@redhat.com> >> --- >> .travis.yml | 6 ++++++ >> 1 file changed, 6 insertions(+) >> >> diff --git a/.travis.yml b/.travis.yml >> index 95be6ec59f..db1a31ea51 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -36,6 +36,7 @@ addons: >> - liburcu-dev >> - libusb-1.0-0-dev >> - libvte-2.90-dev >> + - python3-pip >> - sparse >> - uuid-dev >> - gcovr >> @@ -117,6 +118,11 @@ matrix: >> - env: CONFIG="--target-list=x86_64-softmmu" >> python: >> - "3.6" >> + # Acceptance (Functional) tests >> + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu" >> + TEST_CMD="make check-acceptance" >> + python: >> + - "3.6" >> # Using newer GCC with sanitizers >> - addons: >> apt: >> > > Using the following patch: > > -- >8 -- > diff --git a/.travis.yml b/.travis.yml > index 95be6ec59f..87e0c9a13f 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -118,4 +118,15 @@ matrix: > python: > - "3.6" > + # Acceptance (Functional) tests > + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu" > + TEST_CMD="make check-acceptance" > + # sudo rm /usr/local/bin/pip* That snuck in ^ > + python: > + - "3.6" > + addons: > + apt: > + packages: > + - python3-pip > + - python3.4-venv > # Using newer GCC with sanitizers > - addons: > --- > > I got some improvements until: > > VENV /home/travis/build/philmd/qemu/tests/venv > MKDIR /home/travis/build/philmd/qemu/tests/results > PIP /home/travis/build/philmd/qemu/tests/venv-requirements.txt > Exception: > Traceback (most recent call last): > File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in > main > status = self.run(options, args) > File "/usr/lib/python3/dist-packages/pip/commands/install.py", line > 283, in run > requirement_set.install(install_options, global_options, > root=options.root_path) > File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install > requirement.install(install_options, global_options, *args, **kwargs) > File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install > self.move_wheel_files(self.source_dir, root=root) > File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in > move_wheel_files > pycompile=self.pycompile, > File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in > move_wheel_files > clobber(source, lib_dir, True) > File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber > os.makedirs(destsubdir) > File "/usr/lib/python3.4/os.py", line 237, in makedirs > mkdir(name, mode) > PermissionError: [Errno 13] Permission denied: > '/usr/local/lib/python3.4/dist-packages/avocado' > > See: https://travis-ci.org/philmd/qemu/jobs/439138706 Don't we need to do a pip install or is avocado already included? -- Alex Bennée
On 09/10/2018 16:15, Alex Bennée wrote: > > Philippe Mathieu-Daudé <philmd@redhat.com> writes: > >> Hi Cleber, >> >> On 09/10/2018 06:18, Cleber Rosa wrote: >>> This enables the execution of the acceptance tests on Travis. >> >> Did you test this? =) >> >>> >>> Because the Travis environment is based on Ubuntu Trusty, it requires >>> the python3-pip. >>> >>> Note: while another supposedely required component on newer versions >>> (such as on Bionic) split the Python 3 installation further on the >>> python3-venv package. >>> >>> Signed-off-by: Cleber Rosa <crosa@redhat.com> >>> --- >>> .travis.yml | 6 ++++++ >>> 1 file changed, 6 insertions(+) >>> >>> diff --git a/.travis.yml b/.travis.yml >>> index 95be6ec59f..db1a31ea51 100644 >>> --- a/.travis.yml >>> +++ b/.travis.yml >>> @@ -36,6 +36,7 @@ addons: >>> - liburcu-dev >>> - libusb-1.0-0-dev >>> - libvte-2.90-dev >>> + - python3-pip >>> - sparse >>> - uuid-dev >>> - gcovr >>> @@ -117,6 +118,11 @@ matrix: >>> - env: CONFIG="--target-list=x86_64-softmmu" >>> python: >>> - "3.6" >>> + # Acceptance (Functional) tests >>> + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu" >>> + TEST_CMD="make check-acceptance" >>> + python: >>> + - "3.6" >>> # Using newer GCC with sanitizers >>> - addons: >>> apt: >>> >> >> Using the following patch: >> >> -- >8 -- >> diff --git a/.travis.yml b/.travis.yml >> index 95be6ec59f..87e0c9a13f 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -118,4 +118,15 @@ matrix: >> python: >> - "3.6" >> + # Acceptance (Functional) tests >> + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu" >> + TEST_CMD="make check-acceptance" >> + # sudo rm /usr/local/bin/pip* > > That snuck in ^ =) Travis-ci image comes with this version which is Python2. >> + python: >> + - "3.6" >> + addons: >> + apt: >> + packages: >> + - python3-pip >> + - python3.4-venv >> # Using newer GCC with sanitizers >> - addons: >> --- >> >> I got some improvements until: >> >> VENV /home/travis/build/philmd/qemu/tests/venv >> MKDIR /home/travis/build/philmd/qemu/tests/results >> PIP /home/travis/build/philmd/qemu/tests/venv-requirements.txt >> Exception: >> Traceback (most recent call last): >> File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in >> main >> status = self.run(options, args) >> File "/usr/lib/python3/dist-packages/pip/commands/install.py", line >> 283, in run >> requirement_set.install(install_options, global_options, >> root=options.root_path) >> File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install >> requirement.install(install_options, global_options, *args, **kwargs) >> File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install >> self.move_wheel_files(self.source_dir, root=root) >> File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in >> move_wheel_files >> pycompile=self.pycompile, >> File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in >> move_wheel_files >> clobber(source, lib_dir, True) >> File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber >> os.makedirs(destsubdir) >> File "/usr/lib/python3.4/os.py", line 237, in makedirs >> mkdir(name, mode) >> PermissionError: [Errno 13] Permission denied: >> '/usr/local/lib/python3.4/dist-packages/avocado' >> >> See: https://travis-ci.org/philmd/qemu/jobs/439138706 > > Don't we need to do a pip install or is avocado already included? Yes I think we do. This is why I asked Cleber on patch #1 of this series why he choose to use "venv ... --system-site-packages".
On 10/9/18 9:46 AM, Philippe Mathieu-Daudé wrote: > Hi Cleber, > > On 09/10/2018 06:18, Cleber Rosa wrote: >> This enables the execution of the acceptance tests on Travis. > > Did you test this? =) > I did have some jobs on Travis that looked promising. But yeah, later I found some issues. :/ >> >> Because the Travis environment is based on Ubuntu Trusty, it requires >> the python3-pip. >> >> Note: while another supposedely required component on newer versions >> (such as on Bionic) split the Python 3 installation further on the >> python3-venv package. >> >> Signed-off-by: Cleber Rosa <crosa@redhat.com> >> --- >> .travis.yml | 6 ++++++ >> 1 file changed, 6 insertions(+) >> >> diff --git a/.travis.yml b/.travis.yml >> index 95be6ec59f..db1a31ea51 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -36,6 +36,7 @@ addons: >> - liburcu-dev >> - libusb-1.0-0-dev >> - libvte-2.90-dev >> + - python3-pip >> - sparse >> - uuid-dev >> - gcovr >> @@ -117,6 +118,11 @@ matrix: >> - env: CONFIG="--target-list=x86_64-softmmu" >> python: >> - "3.6" >> + # Acceptance (Functional) tests >> + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu" >> + TEST_CMD="make check-acceptance" >> + python: >> + - "3.6" >> # Using newer GCC with sanitizers >> - addons: >> apt: >> > > Using the following patch: > > -- >8 -- > diff --git a/.travis.yml b/.travis.yml > index 95be6ec59f..87e0c9a13f 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -118,4 +118,15 @@ matrix: > python: > - "3.6" > + # Acceptance (Functional) tests > + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu" > + TEST_CMD="make check-acceptance" > + # sudo rm /usr/local/bin/pip* > + python: > + - "3.6" > + addons: > + apt: > + packages: > + - python3-pip > + - python3.4-venv This patch revealed to me that, even though we're asking Travis for a Python 3.6 stack, we're manually pointing to the primary Python (3.4) installation. This is in itself a source of problems. Read on... > # Using newer GCC with sanitizers > - addons: > --- > > I got some improvements until: > > VENV /home/travis/build/philmd/qemu/tests/venv > MKDIR /home/travis/build/philmd/qemu/tests/results > PIP /home/travis/build/philmd/qemu/tests/venv-requirements.txt > Exception: > Traceback (most recent call last): > File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in > main > status = self.run(options, args) > File "/usr/lib/python3/dist-packages/pip/commands/install.py", line > 283, in run > requirement_set.install(install_options, global_options, > root=options.root_path) > File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install > requirement.install(install_options, global_options, *args, **kwargs) > File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install > self.move_wheel_files(self.source_dir, root=root) > File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in > move_wheel_files > pycompile=self.pycompile, > File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in > move_wheel_files > clobber(source, lib_dir, True) > File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber > os.makedirs(destsubdir) > File "/usr/lib/python3.4/os.py", line 237, in makedirs > mkdir(name, mode) > PermissionError: [Errno 13] Permission denied: > '/usr/local/lib/python3.4/dist-packages/avocado' > This looks to me like it's caused by the use of the system wide Python 3.4 installation, as mentioned above. That, and possibly the "--system-wide-packages" flag to the venv. So, in order to understand if, in those environments, we should let "configure" pick up the right Python, I did the following experiment: $ python3 -m venv /tmp/py3-build $ . /tmp/py3-build/bin/activate $ which python /tmp/py3-build/bin/python $ /tmp/py3-build/bin/python --version Python 3.6.6 This shows that when a venv is activated, the "right" Python should be available in the $PATH. So, building QEMU without pointint to a specific Python binary (when inside a venv) gives you: $ mkdir -p /tmp/qemu-build $ cd /tmp/qemu-build $ ~/src/qemu/configure --target-list="x86_64-softmmu" $ make print-PYTHON PYTHON=python -B $ which python /tmp/py3-build/bin/python The whole question now is how/where Travis puts the requested Python version (python: - "3.6"). Is it on a venv? Is it on a specific location? Is that reliable enough? I'm going to look for those answers now. - Cleber. > See: https://travis-ci.org/philmd/qemu/jobs/439138706 >
Cleber Rosa <crosa@redhat.com> writes: > On 10/9/18 9:46 AM, Philippe Mathieu-Daudé wrote: >> Hi Cleber, >> >> On 09/10/2018 06:18, Cleber Rosa wrote: >>> This enables the execution of the acceptance tests on Travis. >> >> Did you test this? =) >> > > I did have some jobs on Travis that looked promising. But yeah, later I > found some issues. :/ > >>> >>> Because the Travis environment is based on Ubuntu Trusty, it requires >>> the python3-pip. >>> >>> Note: while another supposedely required component on newer versions >>> (such as on Bionic) split the Python 3 installation further on the >>> python3-venv package. >>> >>> Signed-off-by: Cleber Rosa <crosa@redhat.com> >>> --- >>> .travis.yml | 6 ++++++ >>> 1 file changed, 6 insertions(+) >>> >>> diff --git a/.travis.yml b/.travis.yml >>> index 95be6ec59f..db1a31ea51 100644 >>> --- a/.travis.yml >>> +++ b/.travis.yml >>> @@ -36,6 +36,7 @@ addons: >>> - liburcu-dev >>> - libusb-1.0-0-dev >>> - libvte-2.90-dev >>> + - python3-pip >>> - sparse >>> - uuid-dev >>> - gcovr >>> @@ -117,6 +118,11 @@ matrix: >>> - env: CONFIG="--target-list=x86_64-softmmu" >>> python: >>> - "3.6" >>> + # Acceptance (Functional) tests >>> + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu" >>> + TEST_CMD="make check-acceptance" >>> + python: >>> + - "3.6" >>> # Using newer GCC with sanitizers >>> - addons: >>> apt: >>> >> >> Using the following patch: >> >> -- >8 -- >> diff --git a/.travis.yml b/.travis.yml >> index 95be6ec59f..87e0c9a13f 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -118,4 +118,15 @@ matrix: >> python: >> - "3.6" >> + # Acceptance (Functional) tests >> + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu" >> + TEST_CMD="make check-acceptance" >> + # sudo rm /usr/local/bin/pip* >> + python: >> + - "3.6" >> + addons: >> + apt: >> + packages: >> + - python3-pip >> + - python3.4-venv > > This patch revealed to me that, even though we're asking Travis for a > Python 3.6 stack, we're manually pointing to the primary Python (3.4) > installation. This is in itself a source of problems. > > Read on... > >> # Using newer GCC with sanitizers >> - addons: >> --- >> >> I got some improvements until: >> >> VENV /home/travis/build/philmd/qemu/tests/venv >> MKDIR /home/travis/build/philmd/qemu/tests/results >> PIP /home/travis/build/philmd/qemu/tests/venv-requirements.txt >> Exception: >> Traceback (most recent call last): >> File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in >> main >> status = self.run(options, args) >> File "/usr/lib/python3/dist-packages/pip/commands/install.py", line >> 283, in run >> requirement_set.install(install_options, global_options, >> root=options.root_path) >> File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install >> requirement.install(install_options, global_options, *args, **kwargs) >> File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install >> self.move_wheel_files(self.source_dir, root=root) >> File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in >> move_wheel_files >> pycompile=self.pycompile, >> File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in >> move_wheel_files >> clobber(source, lib_dir, True) >> File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber >> os.makedirs(destsubdir) >> File "/usr/lib/python3.4/os.py", line 237, in makedirs >> mkdir(name, mode) >> PermissionError: [Errno 13] Permission denied: >> '/usr/local/lib/python3.4/dist-packages/avocado' >> > > This looks to me like it's caused by the use of the system wide Python > 3.4 installation, as mentioned above. That, and possibly the > "--system-wide-packages" flag to the venv. > > So, in order to understand if, in those environments, we should let > "configure" pick up the right Python, I did the following experiment: > > $ python3 -m venv /tmp/py3-build > $ . /tmp/py3-build/bin/activate > $ which python > /tmp/py3-build/bin/python > $ /tmp/py3-build/bin/python --version > Python 3.6.6 > > This shows that when a venv is activated, the "right" Python should be > available in the $PATH. So, building QEMU without pointint to a > specific Python binary (when inside a venv) gives you: > > $ mkdir -p /tmp/qemu-build > $ cd /tmp/qemu-build > $ ~/src/qemu/configure --target-list="x86_64-softmmu" > $ make print-PYTHON > PYTHON=python -B > $ which python > /tmp/py3-build/bin/python > > The whole question now is how/where Travis puts the requested Python > version (python: - "3.6"). Is it on a venv? Is it on a specific > location? Is that reliable enough? > > I'm going to look for those answers now. You can experiment with the Travis environment on your own system using: make docker-test-build@travis DEBUG=1 > > - Cleber. > >> See: https://travis-ci.org/philmd/qemu/jobs/439138706 >> -- Alex Bennée
© 2016 - 2025 Red Hat, Inc.