[v1] ci: Paint the pipeline green

[libvirt PATCH 5/6] ci: Add temporary workaround for Fedora Rawhide

Posted by Andrea Bolognani 4 years, 12 months ago

The .repo files for Fedora Rawhide are already pointing to the
Fedora 35 key, but all RPMs are still signed with the Fedora 34
key, resulting in

  GPG key at file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64 (0x9867C58F) is already installed
  The GPG keys listed for the "Fedora - Rawhide - Developmental packages for the next Fedora release" repository
  are already installed but they are not correct for this package.
  Check that the correct key URLs are configured for this repository.. Failing package is: nosync-1.1-10.fc34.x86_64
   GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64
  The downloaded packages were saved in cache until the next successful transaction.
  You can remove cached packages by executing 'dnf clean packages'.
  Error: GPG check FAILED

Temporarily tweak the .repo files so that the Fedora 34 key is
used for validation. We should be able to revert this in a few
days.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
 ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile | 3 ++-
 ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile | 3 ++-
 ci/containers/ci-fedora-rawhide.Dockerfile               | 3 ++-
 3 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile b/ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile
index 553dfd47f0..da0a6cc02c 100644
--- a/ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile
+++ b/ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile
@@ -5,7 +5,8 @@
 # https://gitlab.com/libvirt/libvirt-ci/-/commit/824b894aa3ca40e9d2807b693765e213cb8aa832
 FROM registry.fedoraproject.org/fedora:rawhide
 
-RUN dnf install -y nosync && \
+RUN sed -Ei 's|^gpgkey=.*$|gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-34-x86_64|g' /etc/yum.repos.d/*.repo && \
+    dnf install -y nosync && \
     echo -e '#!/bin/sh\n\
 if test -d /usr/lib64\n\
 then\n\
diff --git a/ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile b/ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile
index 0183b15b28..a1e50a34b2 100644
--- a/ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile
+++ b/ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile
@@ -5,7 +5,8 @@
 # https://gitlab.com/libvirt/libvirt-ci/-/commit/824b894aa3ca40e9d2807b693765e213cb8aa832
 FROM registry.fedoraproject.org/fedora:rawhide
 
-RUN dnf install -y nosync && \
+RUN sed -Ei 's|^gpgkey=.*$|gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-34-x86_64|g' /etc/yum.repos.d/*.repo && \
+    dnf install -y nosync && \
     echo -e '#!/bin/sh\n\
 if test -d /usr/lib64\n\
 then\n\
diff --git a/ci/containers/ci-fedora-rawhide.Dockerfile b/ci/containers/ci-fedora-rawhide.Dockerfile
index f331c8d74c..e3968b2199 100644
--- a/ci/containers/ci-fedora-rawhide.Dockerfile
+++ b/ci/containers/ci-fedora-rawhide.Dockerfile
@@ -5,7 +5,8 @@
 # https://gitlab.com/libvirt/libvirt-ci/-/commit/824b894aa3ca40e9d2807b693765e213cb8aa832
 FROM registry.fedoraproject.org/fedora:rawhide
 
-RUN dnf install -y nosync && \
+RUN sed -Ei 's|^gpgkey=.*$|gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-34-x86_64|g' /etc/yum.repos.d/*.repo && \
+    dnf install -y nosync && \
     echo -e '#!/bin/sh\n\
 if test -d /usr/lib64\n\
 then\n\
-- 
2.26.2

Re: [libvirt PATCH 5/6] ci: Add temporary workaround for Fedora Rawhide

Posted by Daniel P. Berrangé 4 years, 12 months ago

On Thu, Feb 11, 2021 at 02:06:45PM +0100, Andrea Bolognani wrote:
> The .repo files for Fedora Rawhide are already pointing to the
> Fedora 35 key, but all RPMs are still signed with the Fedora 34
> key, resulting in
> 
>   GPG key at file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64 (0x9867C58F) is already installed
>   The GPG keys listed for the "Fedora - Rawhide - Developmental packages for the next Fedora release" repository
>   are already installed but they are not correct for this package.
>   Check that the correct key URLs are configured for this repository.. Failing package is: nosync-1.1-10.fc34.x86_64
>    GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64
>   The downloaded packages were saved in cache until the next successful transaction.
>   You can remove cached packages by executing 'dnf clean packages'.
>   Error: GPG check FAILED
> 
> Temporarily tweak the .repo files so that the Fedora 34 key is
> used for validation. We should be able to revert this in a few
> days.

Hmm, isn't the real bug the ordering:

    dnf install -y nosync && \
    nosync dnf update -y --nogpgcheck fedora-gpg-keys && \
    nosync dnf update -y && \

We installed nosync too early. We need to update fedora-gpg-keys
as the *first* thing we do in rawhide images, and only then
attempt to install nosync.

> 
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
>  ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile | 3 ++-
>  ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile | 3 ++-
>  ci/containers/ci-fedora-rawhide.Dockerfile               | 3 ++-
>  3 files changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile b/ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile
> index 553dfd47f0..da0a6cc02c 100644
> --- a/ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile
> +++ b/ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile
> @@ -5,7 +5,8 @@
>  # https://gitlab.com/libvirt/libvirt-ci/-/commit/824b894aa3ca40e9d2807b693765e213cb8aa832
>  FROM registry.fedoraproject.org/fedora:rawhide
>  
> -RUN dnf install -y nosync && \
> +RUN sed -Ei 's|^gpgkey=.*$|gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-34-x86_64|g' /etc/yum.repos.d/*.repo && \
> +    dnf install -y nosync && \
>      echo -e '#!/bin/sh\n\
>  if test -d /usr/lib64\n\
>  then\n\
> diff --git a/ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile b/ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile
> index 0183b15b28..a1e50a34b2 100644
> --- a/ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile
> +++ b/ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile
> @@ -5,7 +5,8 @@
>  # https://gitlab.com/libvirt/libvirt-ci/-/commit/824b894aa3ca40e9d2807b693765e213cb8aa832
>  FROM registry.fedoraproject.org/fedora:rawhide
>  
> -RUN dnf install -y nosync && \
> +RUN sed -Ei 's|^gpgkey=.*$|gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-34-x86_64|g' /etc/yum.repos.d/*.repo && \
> +    dnf install -y nosync && \
>      echo -e '#!/bin/sh\n\
>  if test -d /usr/lib64\n\
>  then\n\
> diff --git a/ci/containers/ci-fedora-rawhide.Dockerfile b/ci/containers/ci-fedora-rawhide.Dockerfile
> index f331c8d74c..e3968b2199 100644
> --- a/ci/containers/ci-fedora-rawhide.Dockerfile
> +++ b/ci/containers/ci-fedora-rawhide.Dockerfile
> @@ -5,7 +5,8 @@
>  # https://gitlab.com/libvirt/libvirt-ci/-/commit/824b894aa3ca40e9d2807b693765e213cb8aa832
>  FROM registry.fedoraproject.org/fedora:rawhide
>  
> -RUN dnf install -y nosync && \
> +RUN sed -Ei 's|^gpgkey=.*$|gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-34-x86_64|g' /etc/yum.repos.d/*.repo && \
> +    dnf install -y nosync && \
>      echo -e '#!/bin/sh\n\
>  if test -d /usr/lib64\n\
>  then\n\
> -- 
> 2.26.2
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [libvirt PATCH 5/6] ci: Add temporary workaround for Fedora Rawhide

Posted by Andrea Bolognani 4 years, 12 months ago

On Thu, 2021-02-11 at 14:48 +0000, Daniel P. Berrangé wrote:
> On Thu, Feb 11, 2021 at 02:06:45PM +0100, Andrea Bolognani wrote:
> > The .repo files for Fedora Rawhide are already pointing to the
> > Fedora 35 key, but all RPMs are still signed with the Fedora 34
> > key, resulting in
> > 
> >   GPG key at file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64 (0x9867C58F) is already installed
> >   The GPG keys listed for the "Fedora - Rawhide - Developmental packages for the next Fedora release" repository
> >   are already installed but they are not correct for this package.
> >   Check that the correct key URLs are configured for this repository.. Failing package is: nosync-1.1-10.fc34.x86_64
> >    GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64
> >   The downloaded packages were saved in cache until the next successful transaction.
> >   You can remove cached packages by executing 'dnf clean packages'.
> >   Error: GPG check FAILED
> > 
> > Temporarily tweak the .repo files so that the Fedora 34 key is
> > used for validation. We should be able to revert this in a few
> > days.
> 
> Hmm, isn't the real bug the ordering:
> 
>     dnf install -y nosync && \
>     nosync dnf update -y --nogpgcheck fedora-gpg-keys && \
>     nosync dnf update -y && \
> 
> We installed nosync too early. We need to update fedora-gpg-keys
> as the *first* thing we do in rawhide images, and only then
> attempt to install nosync.

Yeah, I thought of that as well at first, and it's definitely
something that we need to fix in lcitool. I'll post a patch later.

That said, in this case the problem can't be solved by simply
inverting the commands: from the output above, you'll see that the
signing key for Fedora 35 is already present on the system; however,
since the RPMs that are currently in the Rawhide repos are still
signed with the Fedora 34 key, we need to convince dnf to use the
latter for validation. That's what the hack in this patch does.

-- 
Andrea Bolognani / Red Hat / Virtualization

[libvirt PATCH 1/6] ci: Shuffle cross-building jobs around
[libvirt PATCH 2/6] ci: Mark container build jobs as required/optional correctly
[libvirt PATCH 3/6] ci: Move ppc64le build from Debian sid to Debian 10
[libvirt PATCH 4/6] ci: Refresh Dockerfiles
[libvirt PATCH 5/6] ci: Add temporary workaround for Fedora Rawhide
[libvirt PATCH 6/6] ci: Build on FreeBSD 12.2