[PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts

Peter Krempa posted 1 patch 1 year, 5 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/953779142fb30a0f763671e6bb0f5b4e8df5dfbd.1665395203.git.pkrempa@redhat.com
ci/integration-template.yml |  2 +-
ci/integration.yml          | 40 +++++++++++++++++++++++++++++++++++++
2 files changed, 41 insertions(+), 1 deletion(-)
[PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts
Posted by Peter Krempa 1 year, 5 months ago
After addition of the new libvirt-client-qemu sub-package which is using
python bindings (thus creating a circular dependency between the libvirt
and libvirt-python projects) the integration jobs fail with:

  Error:
   Problem: conflicting requests
    - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64

The libvirt-python project now provides the RPMs in artifacts:
 https://gitlab.com/libvirt/libvirt-python/-/merge_requests/96

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
---
 ci/integration-template.yml |  2 +-
 ci/integration.yml          | 40 +++++++++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/ci/integration-template.yml b/ci/integration-template.yml
index 3347bb03ed..acb51a0b27 100644
--- a/ci/integration-template.yml
+++ b/ci/integration-template.yml
@@ -25,7 +25,7 @@


 .install-deps: &install-deps
-  - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/*
+  - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/* libvirt-python-rpms/*
   - sudo pip3 install --prefix=/usr avocado-framework


diff --git a/ci/integration.yml b/ci/integration.yml
index 19d227faad..b79d2e0f40 100644
--- a/ci/integration.yml
+++ b/ci/integration.yml
@@ -20,6 +20,10 @@ centos-stream-8-tests-prebuilt-env:
       job: x86_64-centos-stream-8-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-centos-stream-8-prebuilt-env
+      ref: master
+      artifacts: true

 centos-stream-8-tests-local-env:
   extends:
@@ -31,6 +35,10 @@ centos-stream-8-tests-local-env:
       job: x86_64-centos-stream-8-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-centos-stream-8-prebuilt-env
+      ref: master
+      artifacts: true


 .centos-stream-9-tests:
@@ -52,6 +60,10 @@ centos-stream-9-tests-prebuilt-env:
       job: x86_64-centos-stream-9-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-centos-stream-9-prebuilt-env
+      ref: master
+      artifacts: true

 centos-stream-9-tests-local-env:
   extends:
@@ -63,6 +75,10 @@ centos-stream-9-tests-local-env:
       job: x86_64-centos-stream-9-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-centos-stream-9-prebuilt-env
+      ref: master
+      artifacts: true


 .fedora-35-tests:
@@ -84,6 +100,10 @@ fedora-35-tests-prebuilt-env:
       job: x86_64-fedora-35-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-fedora-35-prebuilt-env
+      ref: master
+      artifacts: true

 fedora-35-tests-local-env:
   extends:
@@ -95,6 +115,10 @@ fedora-35-tests-local-env:
       job: x86_64-fedora-35-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-fedora-35-prebuilt-env
+      ref: master
+      artifacts: true


 .fedora-36-tests:
@@ -116,6 +140,10 @@ fedora-36-tests-prebuilt-env:
       job: x86_64-fedora-36-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-fedora-36-prebuilt-env
+      ref: master
+      artifacts: true

 fedora-36-tests-local-env:
   extends:
@@ -127,6 +155,10 @@ fedora-36-tests-local-env:
       job: x86_64-fedora-36-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-fedora-36-prebuilt-env
+      ref: master
+      artifacts: true


 .fedora-36-upstream-qemu-tests:
@@ -148,6 +180,10 @@ fedora-36-upstream-qemu-tests-prebuilt-env:
       job: x86_64-fedora-36-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-fedora-36-prebuilt-env
+      ref: master
+      artifacts: true

 fedora-36-upstream-qemu-tests-local-env:
   extends:
@@ -159,3 +195,7 @@ fedora-36-upstream-qemu-tests-local-env:
       job: x86_64-fedora-36-prebuilt-env
       ref: master
       artifacts: true
+    - project: libvirt/libvirt-python
+      job: x86_64-fedora-36-prebuilt-env
+      ref: master
+      artifacts: true
-- 
2.37.3
Re: [PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts
Posted by Daniel P. Berrangé 1 year, 5 months ago
On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote:
> After addition of the new libvirt-client-qemu sub-package which is using
> python bindings (thus creating a circular dependency between the libvirt
> and libvirt-python projects) the integration jobs fail with:
> 
>   Error:
>    Problem: conflicting requests
>     - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64

Hmm, I wonder, should we really be trying to install *all* RPMs ?

IIUC, we're only running integration tests for QEMU, so it ought
to be sufficient to do only have the QEMU driver and virsh client
and Perl APIs. eg can we change ci/integrate-templates.yml to
do

   createrepo libvirt-rpms
   createrepo libvirt-perl
   dnf install libvirt-daemon-qemu libvirt-client perl-Sys-Virt


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts
Posted by Erik Skultety 1 year, 5 months ago
On Mon, Oct 10, 2022 at 11:09:51AM +0100, Daniel P. Berrangé wrote:
> On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote:
> > After addition of the new libvirt-client-qemu sub-package which is using
> > python bindings (thus creating a circular dependency between the libvirt
> > and libvirt-python projects) the integration jobs fail with:
> > 
> >   Error:
> >    Problem: conflicting requests
> >     - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
> 
> Hmm, I wonder, should we really be trying to install *all* RPMs ?
> 
> IIUC, we're only running integration tests for QEMU, so it ought
> to be sufficient to do only have the QEMU driver and virsh client
> and Perl APIs. eg can we change ci/integrate-templates.yml to
> do
> 
>    createrepo libvirt-rpms
>    createrepo libvirt-perl
>    dnf install libvirt-daemon-qemu libvirt-client perl-Sys-Virt

Sorry for the delay, I was actually integrating ^this suggestion and GitLab is,
well, slow to test these changes with. Anyhow, I vote for Peter's original
patch as it's the simplest and cleanest way to fix the problem, why? Here's
what I ended up with on my branch:

diff --git a/ci/integration-template.yml b/ci/integration-template.yml
index d4799b2ca8..9c58a1b71c 100644
--- a/ci/integration-template.yml
+++ b/ci/integration-template.yml
@@ -25,7 +25,31 @@


 .install-deps: &install-deps
-  - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/*
+  - sudo dnf install -y createrepo
+  - createrepo libvirt-rpms
+  - createrepo libvirt-perl-rpms
+  - |-
+    for repo in libvirt-rpms libvirt-perl-rpms; do
+    cat << EOF >> "$SCRATCH_DIR/ci.repo"
+    [CI-$repo]
+    name=ci-$repo
+    baseurl=file://$(pwd)/$repo
+    enabled=1
+    gpgcheck=0
+    EOF
+    done
+  - sudo mv "$SCRATCH_DIR/ci.repo" /etc/yum.repos.d/
+  - sudo dnf install -y
+          libvirt-daemon-config-nwfilter
+          libvirt-daemon-config-network
+          libvirt-daemon-driver-interface
+          libvirt-daemon-driver-network
+          libvirt-daemon-driver-nodedev
+          libvirt-daemon-driver-nwfilter
+          libvirt-daemon-driver-qemu
+          libvirt-daemon-driver-secret
+          libvirt-daemon-driver-storage
+          perl-Sys-Virt
   - sudo pip3 install --prefix=/usr avocado-framework


Not only does it require more steps, is ugly because of the heredoc combined
with YAML's explicit line break preservation (|-), libvirt-daemon-driver-qemu
also doesn't pull all the needed bits as it was suggested, e.g.
libvirt-daemon-config-nwfilter which is needed by the tests. Although the tests
don't need virtinterfaced, I still had to include it because the logic which
restarts all the daemons would have to be adjusted as well; and then one day we
add a test for virtinterfaced we'd have to add it back, so IMO installing and
restarting all the daemons is the cleanest and most transparent way of setting
up the CI env even though we're not making use of all of it now (and so I ended
up naming all the secondary drivers explicitly).

The other huge inconvenience with the proposed approach compared to Peter's is
debugging the failures. What I currently do with any integration CI failures is
that I connect to the runner machine and spin up an instance of one of the
images from which gitlab provisions the environment, download the RPM artifacts
from libvirt and libvirt-perl, install and then debug - again, why? Because
unfortunately trying to do this locally is still less convenient even though
we've done plenty of work in lcitool to improve the status quo. So, back to
your suggestion - I'd have to create the repositories manually, install a few
explicit packages and then test; sorry, that maybe fine for the CI, but doesn't
beat the following in any way:

wget <libvirt-rpms-uri> -O libvirt_rpms.zip
wget <libvirt-perl-rpms-uri> -O libvirt_perl_rpms.zip
wget <libvirt-python-rpms-uri> -O libvirt_python_rpms.zip
unzip <yada yada>

dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/*


Regards,
Erik

Re: [PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts
Posted by Daniel P. Berrangé 1 year, 5 months ago
On Thu, Oct 13, 2022 at 09:43:50AM +0200, Erik Skultety wrote:
> On Mon, Oct 10, 2022 at 11:09:51AM +0100, Daniel P. Berrangé wrote:
> > On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote:
> > > After addition of the new libvirt-client-qemu sub-package which is using
> > > python bindings (thus creating a circular dependency between the libvirt
> > > and libvirt-python projects) the integration jobs fail with:
> > > 
> > >   Error:
> > >    Problem: conflicting requests
> > >     - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
> > 
> > Hmm, I wonder, should we really be trying to install *all* RPMs ?
> > 
> > IIUC, we're only running integration tests for QEMU, so it ought
> > to be sufficient to do only have the QEMU driver and virsh client
> > and Perl APIs. eg can we change ci/integrate-templates.yml to
> > do
> > 
> >    createrepo libvirt-rpms
> >    createrepo libvirt-perl
> >    dnf install libvirt-daemon-qemu libvirt-client perl-Sys-Virt
> 
> Sorry for the delay, I was actually integrating ^this suggestion and GitLab is,
> well, slow to test these changes with. Anyhow, I vote for Peter's original
> patch as it's the simplest and cleanest way to fix the problem, why? Here's
> what I ended up with on my branch:
> 
> diff --git a/ci/integration-template.yml b/ci/integration-template.yml
> index d4799b2ca8..9c58a1b71c 100644
> --- a/ci/integration-template.yml
> +++ b/ci/integration-template.yml
> @@ -25,7 +25,31 @@
> 
> 
>  .install-deps: &install-deps
> -  - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/*
> +  - sudo dnf install -y createrepo
> +  - createrepo libvirt-rpms
> +  - createrepo libvirt-perl-rpms
> +  - |-
> +    for repo in libvirt-rpms libvirt-perl-rpms; do
> +    cat << EOF >> "$SCRATCH_DIR/ci.repo"
> +    [CI-$repo]
> +    name=ci-$repo
> +    baseurl=file://$(pwd)/$repo
> +    enabled=1
> +    gpgcheck=0
> +    EOF
> +    done
> +  - sudo mv "$SCRATCH_DIR/ci.repo" /etc/yum.repos.d/
> +  - sudo dnf install -y
> +          libvirt-daemon-config-nwfilter
> +          libvirt-daemon-config-network
> +          libvirt-daemon-driver-interface
> +          libvirt-daemon-driver-network
> +          libvirt-daemon-driver-nodedev
> +          libvirt-daemon-driver-nwfilter
> +          libvirt-daemon-driver-qemu
> +          libvirt-daemon-driver-secret
> +          libvirt-daemon-driver-storage
> +          perl-Sys-Virt
>    - sudo pip3 install --prefix=/usr avocado-framework

FYI, can be simplified to:

  - sudo dnf install -y createrepo
  - createrepo libvirt-rpms
  - createrepo libvirt-perl-rpms
  - dnf config-manager --add-repo libvirt-rpms
  - dnf config-manager --add-repo libvirt-perl-rpms
  - sudo dnf install -y
          libvirt-daemon-config-nwfilter
          libvirt-daemon-config-network
          libvirt-daemon-qemu
          perl-Sys-Virt

> Not only does it require more steps, is ugly because of the heredoc combined
> with YAML's explicit line break preservation (|-), libvirt-daemon-driver-qemu
> also doesn't pull all the needed bits as it was suggested, e.g.
> libvirt-daemon-config-nwfilter which is needed by the tests. Although the tests
> don't need virtinterfaced, I still had to include it because the logic which
> restarts all the daemons would have to be adjusted as well; and then one day we
> add a test for virtinterfaced we'd have to add it back, so IMO installing and
> restarting all the daemons is the cleanest and most transparent way of setting
> up the CI env even though we're not making use of all of it now (and so I ended
> up naming all the secondary drivers explicitly).

NB, I suggested libvirt-daemon-qemu, not libvirt-daemon-driver-qemu,
because the former pulls in all the secondary driver packages, so
would only miss the config packages.

> The other huge inconvenience with the proposed approach compared to Peter's is
> debugging the failures. What I currently do with any integration CI failures is
> that I connect to the runner machine and spin up an instance of one of the
> images from which gitlab provisions the environment, download the RPM artifacts
> from libvirt and libvirt-perl, install and then debug - again, why? Because
> unfortunately trying to do this locally is still less convenient even though
> we've done plenty of work in lcitool to improve the status quo. So, back to
> your suggestion - I'd have to create the repositories manually, install a few
> explicit packages and then test; sorry, that maybe fine for the CI, but doesn't
> beat the following in any way:
> 
> wget <libvirt-rpms-uri> -O libvirt_rpms.zip
> wget <libvirt-perl-rpms-uri> -O libvirt_perl_rpms.zip
> wget <libvirt-python-rpms-uri> -O libvirt_python_rpms.zip
> unzip <yada yada>
> 
> dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/*

That's fine as an approach for reproducing locally. Does it matter
that you're installing more packages that the CI system would have ?
I guess it could affect the results, but fairly slim liklihood IMHO.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts
Posted by Erik Skultety 1 year, 5 months ago
On Thu, Oct 13, 2022 at 09:06:53AM +0100, Daniel P. Berrangé wrote:
> On Thu, Oct 13, 2022 at 09:43:50AM +0200, Erik Skultety wrote:
> > On Mon, Oct 10, 2022 at 11:09:51AM +0100, Daniel P. Berrangé wrote:
> > > On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote:
> > > > After addition of the new libvirt-client-qemu sub-package which is using
> > > > python bindings (thus creating a circular dependency between the libvirt
> > > > and libvirt-python projects) the integration jobs fail with:
> > > > 
> > > >   Error:
> > > >    Problem: conflicting requests
> > > >     - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
> > > 
> > > Hmm, I wonder, should we really be trying to install *all* RPMs ?
> > > 
> > > IIUC, we're only running integration tests for QEMU, so it ought
> > > to be sufficient to do only have the QEMU driver and virsh client
> > > and Perl APIs. eg can we change ci/integrate-templates.yml to
> > > do
> > > 
> > >    createrepo libvirt-rpms
> > >    createrepo libvirt-perl
> > >    dnf install libvirt-daemon-qemu libvirt-client perl-Sys-Virt
> > 
> > Sorry for the delay, I was actually integrating ^this suggestion and GitLab is,
> > well, slow to test these changes with. Anyhow, I vote for Peter's original
> > patch as it's the simplest and cleanest way to fix the problem, why? Here's
> > what I ended up with on my branch:
> > 
> > diff --git a/ci/integration-template.yml b/ci/integration-template.yml
> > index d4799b2ca8..9c58a1b71c 100644
> > --- a/ci/integration-template.yml
> > +++ b/ci/integration-template.yml
> > @@ -25,7 +25,31 @@
> > 
> > 
> >  .install-deps: &install-deps
> > -  - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/*
> > +  - sudo dnf install -y createrepo
> > +  - createrepo libvirt-rpms
> > +  - createrepo libvirt-perl-rpms
> > +  - |-
> > +    for repo in libvirt-rpms libvirt-perl-rpms; do
> > +    cat << EOF >> "$SCRATCH_DIR/ci.repo"
> > +    [CI-$repo]
> > +    name=ci-$repo
> > +    baseurl=file://$(pwd)/$repo
> > +    enabled=1
> > +    gpgcheck=0
> > +    EOF
> > +    done
> > +  - sudo mv "$SCRATCH_DIR/ci.repo" /etc/yum.repos.d/
> > +  - sudo dnf install -y
> > +          libvirt-daemon-config-nwfilter
> > +          libvirt-daemon-config-network
> > +          libvirt-daemon-driver-interface
> > +          libvirt-daemon-driver-network
> > +          libvirt-daemon-driver-nodedev
> > +          libvirt-daemon-driver-nwfilter
> > +          libvirt-daemon-driver-qemu
> > +          libvirt-daemon-driver-secret
> > +          libvirt-daemon-driver-storage
> > +          perl-Sys-Virt
> >    - sudo pip3 install --prefix=/usr avocado-framework
> 
> FYI, can be simplified to:
> 
>   - sudo dnf install -y createrepo
>   - createrepo libvirt-rpms
>   - createrepo libvirt-perl-rpms
>   - dnf config-manager --add-repo libvirt-rpms
>   - dnf config-manager --add-repo libvirt-perl-rpms

Oh, I didn't know DNF could do this^^^, that's neat. I still prefer the
simplicity of Peter's patch though.

>   - sudo dnf install -y
>           libvirt-daemon-config-nwfilter
>           libvirt-daemon-config-network
>           libvirt-daemon-qemu
>           perl-Sys-Virt
> 
> > Not only does it require more steps, is ugly because of the heredoc combined
> > with YAML's explicit line break preservation (|-), libvirt-daemon-driver-qemu
> > also doesn't pull all the needed bits as it was suggested, e.g.
> > libvirt-daemon-config-nwfilter which is needed by the tests. Although the tests
> > don't need virtinterfaced, I still had to include it because the logic which
> > restarts all the daemons would have to be adjusted as well; and then one day we
> > add a test for virtinterfaced we'd have to add it back, so IMO installing and
> > restarting all the daemons is the cleanest and most transparent way of setting
> > up the CI env even though we're not making use of all of it now (and so I ended
> > up naming all the secondary drivers explicitly).
> 
> NB, I suggested libvirt-daemon-qemu, not libvirt-daemon-driver-qemu,
> because the former pulls in all the secondary driver packages, so
> would only miss the config packages.

We don't build libvirt-daemon-qemu on CS9, I presume that's a bug.

> 
> > The other huge inconvenience with the proposed approach compared to Peter's is
> > debugging the failures. What I currently do with any integration CI failures is
> > that I connect to the runner machine and spin up an instance of one of the
> > images from which gitlab provisions the environment, download the RPM artifacts
> > from libvirt and libvirt-perl, install and then debug - again, why? Because
> > unfortunately trying to do this locally is still less convenient even though
> > we've done plenty of work in lcitool to improve the status quo. So, back to
> > your suggestion - I'd have to create the repositories manually, install a few
> > explicit packages and then test; sorry, that maybe fine for the CI, but doesn't
> > beat the following in any way:
> > 
> > wget <libvirt-rpms-uri> -O libvirt_rpms.zip
> > wget <libvirt-perl-rpms-uri> -O libvirt_perl_rpms.zip
> > wget <libvirt-python-rpms-uri> -O libvirt_python_rpms.zip
> > unzip <yada yada>
> > 
> > dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/*
> 
> That's fine as an approach for reproducing locally. Does it matter
> that you're installing more packages that the CI system would have ?
> I guess it could affect the results, but fairly slim liklihood IMHO.

No, the point I was trying to make is that for ^this use case we still need
Peters libvirt-python MR (which I merged in the meantime btw). The other
argument I have is that we shouldn't deviate in the CI job transcript from what
you would done naturally locally in a manual manner - sure, you can always
copy-paste, but copy pasting

    dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/*

will always be faster than copying

    sudo dnf install -y createrepo
    createrepo libvirt-rpms
    createrepo libvirt-perl-rpms
    createrepo libvirt-python-rpms
    dnf config-manager --add-repo libvirt-rpms
    dnf config-manager --add-repo libvirt-perl-rpms
    dnf config-manager --add-repo libvirt-python-rpms
    sudo dnf install -y
           libvirt-daemon-config-nwfilter
           libvirt-daemon-config-network
           libvirt-daemon-qemu
           perl-Sys-Virt

Erik

Re: [PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts
Posted by Daniel P. Berrangé 1 year, 5 months ago
On Thu, Oct 13, 2022 at 11:42:06AM +0200, Erik Skultety wrote:
> On Thu, Oct 13, 2022 at 09:06:53AM +0100, Daniel P. Berrangé wrote:
> > On Thu, Oct 13, 2022 at 09:43:50AM +0200, Erik Skultety wrote:
> > > On Mon, Oct 10, 2022 at 11:09:51AM +0100, Daniel P. Berrangé wrote:
> > > > On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote:
> > > > > After addition of the new libvirt-client-qemu sub-package which is using
> > > > > python bindings (thus creating a circular dependency between the libvirt
> > > > > and libvirt-python projects) the integration jobs fail with:
> > > > > 
> > > > >   Error:
> > > > >    Problem: conflicting requests
> > > > >     - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
> > > > 
> > > > Hmm, I wonder, should we really be trying to install *all* RPMs ?
> > > > 
> > > > IIUC, we're only running integration tests for QEMU, so it ought
> > > > to be sufficient to do only have the QEMU driver and virsh client
> > > > and Perl APIs. eg can we change ci/integrate-templates.yml to
> > > > do
> > > > 
> > > >    createrepo libvirt-rpms
> > > >    createrepo libvirt-perl
> > > >    dnf install libvirt-daemon-qemu libvirt-client perl-Sys-Virt
> > > 
> > > Sorry for the delay, I was actually integrating ^this suggestion and GitLab is,
> > > well, slow to test these changes with. Anyhow, I vote for Peter's original
> > > patch as it's the simplest and cleanest way to fix the problem, why? Here's
> > > what I ended up with on my branch:
> > > 
> > > diff --git a/ci/integration-template.yml b/ci/integration-template.yml
> > > index d4799b2ca8..9c58a1b71c 100644
> > > --- a/ci/integration-template.yml
> > > +++ b/ci/integration-template.yml
> > > @@ -25,7 +25,31 @@
> > > 
> > > 
> > >  .install-deps: &install-deps
> > > -  - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/*
> > > +  - sudo dnf install -y createrepo
> > > +  - createrepo libvirt-rpms
> > > +  - createrepo libvirt-perl-rpms
> > > +  - |-
> > > +    for repo in libvirt-rpms libvirt-perl-rpms; do
> > > +    cat << EOF >> "$SCRATCH_DIR/ci.repo"
> > > +    [CI-$repo]
> > > +    name=ci-$repo
> > > +    baseurl=file://$(pwd)/$repo
> > > +    enabled=1
> > > +    gpgcheck=0
> > > +    EOF
> > > +    done
> > > +  - sudo mv "$SCRATCH_DIR/ci.repo" /etc/yum.repos.d/
> > > +  - sudo dnf install -y
> > > +          libvirt-daemon-config-nwfilter
> > > +          libvirt-daemon-config-network
> > > +          libvirt-daemon-driver-interface
> > > +          libvirt-daemon-driver-network
> > > +          libvirt-daemon-driver-nodedev
> > > +          libvirt-daemon-driver-nwfilter
> > > +          libvirt-daemon-driver-qemu
> > > +          libvirt-daemon-driver-secret
> > > +          libvirt-daemon-driver-storage
> > > +          perl-Sys-Virt
> > >    - sudo pip3 install --prefix=/usr avocado-framework
> > 
> > FYI, can be simplified to:
> > 
> >   - sudo dnf install -y createrepo
> >   - createrepo libvirt-rpms
> >   - createrepo libvirt-perl-rpms
> >   - dnf config-manager --add-repo libvirt-rpms
> >   - dnf config-manager --add-repo libvirt-perl-rpms
> 
> Oh, I didn't know DNF could do this^^^, that's neat. I still prefer the
> simplicity of Peter's patch though.

Ok, I don't mind.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts
Posted by Peter Krempa 1 year, 5 months ago
On Mon, Oct 10, 2022 at 11:09:51 +0100, Daniel P. Berrangé wrote:
> On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote:
> > After addition of the new libvirt-client-qemu sub-package which is using
> > python bindings (thus creating a circular dependency between the libvirt
> > and libvirt-python projects) the integration jobs fail with:
> > 
> >   Error:
> >    Problem: conflicting requests
> >     - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
> 
> Hmm, I wonder, should we really be trying to install *all* RPMs ?
> 
> IIUC, we're only running integration tests for QEMU, so it ought
> to be sufficient to do only have the QEMU driver and virsh client
> and Perl APIs. eg can we change ci/integrate-templates.yml to
> do

That probably makes sense. I'll drop this patch and close the
corresponding libvirt-python MR.
Re: [PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts
Posted by Erik Skultety 1 year, 5 months ago
On Mon, Oct 10, 2022 at 05:53:23PM +0200, Peter Krempa wrote:
> On Mon, Oct 10, 2022 at 11:09:51 +0100, Daniel P. Berrangé wrote:
> > On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote:
> > > After addition of the new libvirt-client-qemu sub-package which is using
> > > python bindings (thus creating a circular dependency between the libvirt
> > > and libvirt-python projects) the integration jobs fail with:
> > > 
> > >   Error:
> > >    Problem: conflicting requests
> > >     - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
> > 
> > Hmm, I wonder, should we really be trying to install *all* RPMs ?
> > 
> > IIUC, we're only running integration tests for QEMU, so it ought
> > to be sufficient to do only have the QEMU driver and virsh client
> > and Perl APIs. eg can we change ci/integrate-templates.yml to
> > do
> 
> That probably makes sense. I'll drop this patch and close the
> corresponding libvirt-python MR.
> 

I vote for this patch and given my response to Dan's review, FWIW you have my

Reviewed-by: Erik Skultety <eskultet@redhat.com>

I have a couple more fixes to the CI configuration, so if we all agree that
we'll go with your original patch I'd include it in my series. I'll also
happily review the python MR if you decide to re-open it.

Erik