1. Patchew
  2. Libvirt
  3. View series

[libvirt] [jenkins-ci PATCH] jobs: Set $PYTHONPATH for python-distutils jobs

Andrea Bolognani posted 1 patch 6 years, 2 months ago
Failed in applying to current master (apply log)
jobs/python-distutils.yaml | 3 +++
1 file changed, 3 insertions(+)
[libvirt] [jenkins-ci PATCH] jobs: Set $PYTHONPATH for python-distutils jobs
Posted by Andrea Bolognani 6 years, 2 months ago 
Since we install Python modules under $VIRT_PREFIX, we need to set
$PYTHONPATH or the interpreter won't be able to locate them. This is
currently being done per-worker in the Jenkins Web interface.

However, now that we've introduced Python 3 builds, depending on the
project and the OS, we might be using any of Python 2.7, 3.4, 3.5
and 3.6, which means we can't have a single $PYTHONPATH anymore: in
particular, while it's okay to have non-esisting directories in
$PYTHONPATH, we have to make sure that a Python 3 interpreter will
never try to use a Python 2 module and vice versa.

To solve the issue, we use a fairly large hammer: we set $PYTHONPATH
at the job level, and include all reasonable minor versions for the
Python major version (pyver) the job is using, even if they don't
yet exist.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
Another approach would be to use something like

  T=$VIRT_PREFIX/lib/python{pyver}.4/site-packages
  T=$T:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages
  T=$T:$VIRT_PREFIX/lib/python{pyver}.5/site-packages
  T=$T:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages
  T=$T:$VIRT_PREFIX/lib/python{pyver}.6/site-packages
  T=$T:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages
  T=$T:$VIRT_PREFIX/lib/python{pyver}.7/site-packages
  T=$T:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages
  export PYTHONPATH=$T

but that's just a different kind of ugly :/

 jobs/python-distutils.yaml | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/jobs/python-distutils.yaml b/jobs/python-distutils.yaml
index ff68c29..510769e 100644
--- a/jobs/python-distutils.yaml
+++ b/jobs/python-distutils.yaml
@@ -42,6 +42,7 @@
       - shell: |
           {global_env}
           {local_env}
+          export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages
           {command_pre_build}
           python{pyver} ./setup.py build
           python{pyver} ./setup.py install --prefix=$VIRT_PREFIX
@@ -83,6 +84,7 @@
       - shell: |
           {global_env}
           {local_env}
+          export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages
           python{pyver} ./setup.py test
     publishers:
       - email:
@@ -121,6 +123,7 @@
       - shell: |
           {global_env}
           {local_env}
+          export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages
           sed -i -e 's/BuildRequires: libvirt.*devel.*//' *.spec.in
           python{pyver} ./setup.py rpm
     publishers:
-- 
2.14.3

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [jenkins-ci PATCH] jobs: Set $PYTHONPATH for python-distutils jobs
Posted by Daniel P. Berrangé 6 years, 2 months ago 
On Wed, Feb 14, 2018 at 05:53:57PM +0100, Andrea Bolognani wrote:
> Since we install Python modules under $VIRT_PREFIX, we need to set
> $PYTHONPATH or the interpreter won't be able to locate them. This is
> currently being done per-worker in the Jenkins Web interface.
> 
> However, now that we've introduced Python 3 builds, depending on the
> project and the OS, we might be using any of Python 2.7, 3.4, 3.5
> and 3.6, which means we can't have a single $PYTHONPATH anymore: in
> particular, while it's okay to have non-esisting directories in
> $PYTHONPATH, we have to make sure that a Python 3 interpreter will
> never try to use a Python 2 module and vice versa.
> 
> To solve the issue, we use a fairly large hammer: we set $PYTHONPATH
> at the job level, and include all reasonable minor versions for the
> Python major version (pyver) the job is using, even if they don't
> yet exist.
> 
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
> Another approach would be to use something like
> 
>   T=$VIRT_PREFIX/lib/python{pyver}.4/site-packages
>   T=$T:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages
>   T=$T:$VIRT_PREFIX/lib/python{pyver}.5/site-packages
>   T=$T:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages
>   T=$T:$VIRT_PREFIX/lib/python{pyver}.6/site-packages
>   T=$T:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages
>   T=$T:$VIRT_PREFIX/lib/python{pyver}.7/site-packages
>   T=$T:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages
>   export PYTHONPATH=$T
> 
> but that's just a different kind of ugly :/
> 
>  jobs/python-distutils.yaml | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/jobs/python-distutils.yaml b/jobs/python-distutils.yaml
> index ff68c29..510769e 100644
> --- a/jobs/python-distutils.yaml
> +++ b/jobs/python-distutils.yaml
> @@ -42,6 +42,7 @@
>        - shell: |
>            {global_env}
>            {local_env}
> +          export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages
>            {command_pre_build}
>            python{pyver} ./setup.py build
>            python{pyver} ./setup.py install --prefix=$VIRT_PREFIX
> @@ -83,6 +84,7 @@
>        - shell: |
>            {global_env}
>            {local_env}
> +          export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages
>            python{pyver} ./setup.py test
>      publishers:
>        - email:
> @@ -121,6 +123,7 @@
>        - shell: |
>            {global_env}
>            {local_env}
> +          export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages
>            sed -i -e 's/BuildRequires: libvirt.*devel.*//' *.spec.in
>            python{pyver} ./setup.py rpm
>      publishers:

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


Might as well kill PYTHONPATH from the nodes after activating this, to
avoid accidents elswhere.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [jenkins-ci PATCH] jobs: Set $PYTHONPATH for python-distutils jobs
Posted by Andrea Bolognani 6 years, 2 months ago 
On Wed, 2018-02-14 at 17:03 +0000, Daniel P. Berrangé wrote:
> Might as well kill PYTHONPATH from the nodes after activating this, to
> avoid accidents elswhere.

Absolutely. In the slightly longer term, the plan is to move all
environment definitions from the nodes to the jobs, so that we can
track changes along with everything else.

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.