[PATCH v2 2/3] tests/avocado: Add Fedora 34 distro, including kernel/initrd checksums

David Woodhouse posted 3 patches 2 years, 11 months ago
Maintainers: "Michael S. Tsirkin" <mst@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Cleber Rosa <crosa@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Wainer dos Santos Moschetta <wainersm@redhat.com>, Beraldo Leal <bleal@redhat.com>
[PATCH v2 2/3] tests/avocado: Add Fedora 34 distro, including kernel/initrd checksums
Posted by David Woodhouse 2 years, 11 months ago
From: David Woodhouse <dwmw@amazon.co.uk>

The kernel in Fedora 31 doesn't support 'xen_no_vector_callback' on
its command line, so add a slightly newer version as a prelude to
enabling avocado tests for Xen guests.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 tests/avocado/avocado_qemu/__init__.py | 27 ++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
index a313e88c07..49e414e267 100644
--- a/tests/avocado/avocado_qemu/__init__.py
+++ b/tests/avocado/avocado_qemu/__init__.py
@@ -485,6 +485,23 @@ class LinuxDistro:
                                   ' console=tty0'),
                  },
             },
+            '34': {
+                'x86_64':
+                {'checksum': ('b9b621b26725ba95442d9a56cbaa0547'
+                              '84e0779a9522ec6eafff07c6e6f717ea'),
+                 'pxeboot_url': ('https://archives.fedoraproject.org/'
+                                 'pub/archive/fedora/linux/releases/34/'
+                                 'Everything/x86_64/os/images/pxeboot/'),
+                 'kernel_hash': ('085fc6e47f2e3a271b591f3e56739ca9'
+                                 '4c16718837a5f431ab95468e1e95f9eb'),
+                 'initrd_hash': ('d6cd2e03e8188eed6c896fd65ff05f81'
+                                 '2c4c1c8777d630b5909e9a1a4627e337'),
+                 'kernel_params': ('root=UUID=386769a3-cfa5-47c8-8797-'
+                                   'd5ec58c9cb6c ro no_timer_check '
+                                   'net.ifnames=0 console=tty1 '
+                                   'console=ttyS0,115200n8'),
+                },
+            },
         }
     }
 
@@ -513,6 +530,16 @@ def pxeboot_url(self):
         """Gets the repository url where pxeboot files can be found"""
         return self._info.get('pxeboot_url', None)
 
+    @property
+    def kernel_hash(self):
+        """Gets checksum of the pxeboot kernel image"""
+        return self._info.get('kernel_hash', None)
+
+    @property
+    def initrd_hash(self):
+        """Gets checksum of the pxeboot initrd image"""
+        return self._info.get('initrd_hash', None)
+
     @property
     def default_kernel_params(self):
         """Gets the default kernel parameters"""
-- 
2.39.0
Re: [PATCH v2 2/3] tests/avocado: Add Fedora 34 distro, including kernel/initrd checksums
Posted by Alex Bennée 2 years, 11 months ago
David Woodhouse <dwmw2@infradead.org> writes:

> From: David Woodhouse <dwmw@amazon.co.uk>
>
> The kernel in Fedora 31 doesn't support 'xen_no_vector_callback' on
> its command line, so add a slightly newer version as a prelude to
> enabling avocado tests for Xen guests.

Why slightly newer rather than current release?

Our existing Fedora guest builds cause all sorts of timeout issues
running under TCG and seem particularly heavyweight considering the
coverage we manage to get.

>
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> ---
>  tests/avocado/avocado_qemu/__init__.py | 27 ++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>
> diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
> index a313e88c07..49e414e267 100644
> --- a/tests/avocado/avocado_qemu/__init__.py
> +++ b/tests/avocado/avocado_qemu/__init__.py
> @@ -485,6 +485,23 @@ class LinuxDistro:
>                                    ' console=tty0'),
>                   },
>              },
> +            '34': {
> +                'x86_64':
> +                {'checksum': ('b9b621b26725ba95442d9a56cbaa0547'
> +                              '84e0779a9522ec6eafff07c6e6f717ea'),
> +                 'pxeboot_url': ('https://archives.fedoraproject.org/'
> +                                 'pub/archive/fedora/linux/releases/34/'
> +                                 'Everything/x86_64/os/images/pxeboot/'),
> +                 'kernel_hash': ('085fc6e47f2e3a271b591f3e56739ca9'
> +                                 '4c16718837a5f431ab95468e1e95f9eb'),
> +                 'initrd_hash': ('d6cd2e03e8188eed6c896fd65ff05f81'
> +                                 '2c4c1c8777d630b5909e9a1a4627e337'),
> +                 'kernel_params': ('root=UUID=386769a3-cfa5-47c8-8797-'
> +                                   'd5ec58c9cb6c ro no_timer_check '
> +                                   'net.ifnames=0 console=tty1 '
> +                                   'console=ttyS0,115200n8'),
> +                },
> +            },
>          }
>      }
>  
> @@ -513,6 +530,16 @@ def pxeboot_url(self):
>          """Gets the repository url where pxeboot files can be found"""
>          return self._info.get('pxeboot_url', None)
>  
> +    @property
> +    def kernel_hash(self):
> +        """Gets checksum of the pxeboot kernel image"""
> +        return self._info.get('kernel_hash', None)
> +
> +    @property
> +    def initrd_hash(self):
> +        """Gets checksum of the pxeboot initrd image"""
> +        return self._info.get('initrd_hash', None)
> +
>      @property
>      def default_kernel_params(self):
>          """Gets the default kernel parameters"""


-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro
Re: [PATCH v2 2/3] tests/avocado: Add Fedora 34 distro, including kernel/initrd checksums
Posted by David Woodhouse 2 years, 11 months ago
On Wed, 2023-03-08 at 17:42 +0000, Alex Bennée wrote:
> 
> David Woodhouse <dwmw2@infradead.org> writes:
> 
> > From: David Woodhouse <dwmw@amazon.co.uk>
> > 
> > The kernel in Fedora 31 doesn't support 'xen_no_vector_callback' on
> > its command line, so add a slightly newer version as a prelude to
> > enabling avocado tests for Xen guests.
> 
> Why slightly newer rather than current release?
> 
> Our existing Fedora guest builds cause all sorts of timeout issues
> running under TCG and seem particularly heavyweight considering the
> coverage we manage to get.

Yeah, I spent a happy afternoon playing with this; I think you may have
observed some of it on IRC. Although I kept some of the naughtier words
to my side of the keyboard. From memory...

I tried Fedora 37 and eventually got that to *boot* (hint:
rootflags=subvol=root) although it concerned me that it wasn't yet at
the URL containing /archive/ so I wondered if adding it at its current
URL might not work in the long term. And then cloudinit didn't seem to
work on it (the finalize step failed in the guest IIRC).

I tried Fedora 36 and cloudinit *did* work there, and it was in
basically the same /pub/archive/fedora/ location so all that made me
happier... but 'dnf install xen-runtime' didn't work, which I was using
in my tests at that point to install the `xenstore-ls` binary. And in
fact dnf didn't seem to find *anything* useful at all in the
repositories which were configured in the Fedora Cloud 36 image.

So I tried Fedora 34, which is the first to have the
'xen_no_vector_callback' option that I'd added in the 5.12 kernel.
And when that worked, I declared all the above to be "Someone Else's
Problem".

The fact that it didn't boot with `-smp 4` was clearly *my* problem,
but the code was already merged upstream at that point, so fixing what
turned out to be the I/OAPIC migration bug ended up being my top
priority. 

Now, I've dropped the dnf invocation because now the second PR has 
been submitted I've switched it to use xen-disk anyway, and that's a
good enough exercise for both xenstore and event channel interrupts.

So it probably *could* move to Fedora 36, and the fact that the intel-
iommu test wouldn't work on Fedora 36 could remain Someone Else's
Problem. :)

Please advise.
Re: [PATCH v2 2/3] tests/avocado: Add Fedora 34 distro, including kernel/initrd checksums
Posted by David Woodhouse 2 years, 11 months ago
On Wed, 2023-03-08 at 19:20 +0000, David Woodhouse wrote:
> On Wed, 2023-03-08 at 17:42 +0000, Alex Bennée wrote:
> > Why slightly newer rather than current release?
> >
> > Our existing Fedora guest builds cause all sorts of timeout issues
> > running under TCG and seem particularly heavyweight considering the
> > coverage we manage to get.
>
> Yeah, I spent a happy afternoon playing with this; I think you may have
> observed some of it on IRC. Although I kept some of the naughtier words
> to my side of the keyboard. From memory...

So...

            '34': {
                'x86_64':
                {'checksum': ('b9b621b26725ba95442d9a56cbaa0547'
                              '84e0779a9522ec6eafff07c6e6f717ea'),
                 'pxeboot_url': ('https://archives.fedoraproject.org/'
                                 'pub/archive/fedora/linux/releases/34/'
                                 'Everything/x86_64/os/images/pxeboot/'),
                 'kernel_hash': ('085fc6e47f2e3a271b591f3e56739ca9'
                                 '4c16718837a5f431ab95468e1e95f9eb'),
                 'initrd_hash': ('d6cd2e03e8188eed6c896fd65ff05f81'
                                 '2c4c1c8777d630b5909e9a1a4627e337'),
                 'kernel_params': ('root=UUID=386769a3-cfa5-47c8-8797-'
                                   'd5ec58c9cb6c ro no_timer_check '
                                   'net.ifnames=0 console=tty1 '
                                   'console=ttyS0,115200n8'),
                },
            },
            '36': {
                'x86_64':
                {'checksum': ('ca9e514cc2f4a7a0188e7c68af60eb4e'
                              '573d2e6850cc65b464697223f46b4605'),
                 'pxeboot_url': ('https://archives.fedoraproject.org/'
                                 'pub/archive/fedora/linux/releases/36/'
                                 'Everything/x86_64/os/images/pxeboot/'),
                 'kernel_hash': ('f5d8162b1e086f1a9d09d3d2c6decdbe'
                                 'c84d7020cde4b6f32a98f21eacc8cf3b'),
                 'initrd_hash': ('2c8a9d3f0b607f86422a0df8227106c0'
                                 '847eb4a4f15bade77b1abbc3228533c6'),
                 'kernel_params': ('root=UUID=737459fa-eab9-4d8f-9a14-'
                                   'c8c32d403c08 ro rootflags=subvol=root '
                                   'no_timer_check net.ifnames=0 '
                                   'console=tty1 console=ttyS0,115200n8'),
                },
            },
            '37': {
                'x86_64':
                {'checksum': ('b5b9bec91eee65489a5745f6ee620573'
                              'b23337cbb1eb4501ce200b157a01f3a0'),
                 'pxeboot_url': ('https://archives.fedoraproject.org/'
                                 'pub/fedora/linux/releases/37/'
                                 'Everything/x86_64/os/images/pxeboot/'),
                 'kernel_hash': ('2bd5f834349313d10a9dbeb41d9f71d9'
                                 '85f7b9c4f7e2ba750a1dd6558f082ba9'),
                 'initrd_hash': ('8b7e91d6f14353217ed44a98802b0fef'
                                 'b190ab1ce38a96166da87d629ab39bef'),
                 'kernel_params': ('root=UUID=3975d5e5-e66a-44c3-90f7-'
                                   'dc2d1c7859d1 ro rootflags=subvol=root '
                                   'no_timer_check net.ifnames=0 '
                                   'console=tty1 console=ttyS0,115200n8'),
                },
            },
        }

Fedora 36 does work (since I'm not actually trying to install anything
inside the guest any more, and I'm sure that *would* be a surmountable
problem if we bothered to investigate and fix up the repo config). It's
about 10 seconds slower per test than Fedora 34 was.

Fedora 37 fails thus:

2023-03-09 08:25:17,072 datadrainer      L0193 DEBUG| -----END SSH HOST KEY KEYS-----
2023-03-09 08:25:17,081 datadrainer      L0193 DEBUG| [   49.036621] cloud-init[1550]: 2023-03-09 08:25:16,989 - util.py[WARNING]: Running module phone-home (<module 'cloudinit.config.cc_phone_home' from '/usr/lib/python3.11/site-packages/cloudinit/config/cc_phone_home.py'>) failed
2023-03-09 08:25:17,101 datadrainer      L0193 DEBUG| [   49.055528] cloud-init[1550]: Cloud-init v. 22.2 finished at Thu, 09 Mar 2023 08:25:16 +0000. Datasource DataSourceNoCloud [seed=/dev/sda][dsmode=net].  Up 49.04 seconds
2023-03-09 08:25:17,203 datadrainer      L0193 DEBUG| [FAILED] Failed to start cloud-fina… Execute cloud user/final scripts.

Full log at http://david.woodhou.se/job-2023-03-09T08.20-d57f30d.log

I've made the other changes (mostly xen_guest → kvm_xen_guest) and
pushed to my branch at 
https://git.infradead.org/users/dwmw2/qemu.git/shortlog/refs/heads/xenfv