[PULL 32/38] tests: add FreeBSD tests for aarch64/sbsa-ref

There is a newer version of this series
[PULL 32/38] tests: add FreeBSD tests for aarch64/sbsa-ref
Posted by Peter Maydell 1 month ago
From: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>

FreeBSD has longer support cycle for stable release (14.x EoL in 2028)
than OpenBSD (7.3 we use is already EoL). Also bugfixes are backported
so we can stay on 14.x for longer.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20240910-b4-move-to-freebsd-v5-2-0fb66d803c93@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 tests/functional/test_aarch64_sbsaref.py | 43 +++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/tests/functional/test_aarch64_sbsaref.py b/tests/functional/test_aarch64_sbsaref.py
index 528882f4fa9..89d0b9587a8 100755
--- a/tests/functional/test_aarch64_sbsaref.py
+++ b/tests/functional/test_aarch64_sbsaref.py
@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 #
-# Functional test that boots a Linux kernel and checks the console
+# Functional test that boots a kernel and checks the console
 #
 # SPDX-FileCopyrightText: 2023-2024 Linaro Ltd.
 # SPDX-FileContributor: Philippe Mathieu-Daudé <philmd@linaro.org>
@@ -184,5 +184,46 @@ def test_sbsaref_openbsd73_max(self):
         self.boot_openbsd73("max")
 
 
+    ASSET_FREEBSD_ISO = Asset(
+        ('https://download.freebsd.org/releases/arm64/aarch64/ISO-IMAGES/'
+         '14.1/FreeBSD-14.1-RELEASE-arm64-aarch64-bootonly.iso'),
+        '44cdbae275ef1bb6dab1d5fbb59473d4f741e1c8ea8a80fd9e906b531d6ad461')
+
+    # This tests the whole boot chain from EFI to Userspace
+    # We only boot a whole OS for the current top level CPU and GIC
+    # Other test profiles should use more minimal boots
+    def boot_freebsd14(self, cpu=None):
+        self.fetch_firmware()
+
+        img_path = self.ASSET_FREEBSD_ISO.fetch()
+
+        self.vm.set_console()
+        self.vm.add_args(
+            "-drive", f"file={img_path},format=raw,snapshot=on",
+        )
+        if cpu:
+            self.vm.add_args("-cpu", cpu)
+
+        self.vm.launch()
+        wait_for_console_pattern(self, 'Welcome to FreeBSD!')
+
+    def test_sbsaref_freebsd14_cortex_a57(self):
+        self.boot_freebsd14("cortex-a57")
+
+    def test_sbsaref_freebsd14_default_cpu(self):
+        self.boot_freebsd14()
+
+    def test_sbsaref_freebsd14_max_pauth_off(self):
+        self.boot_freebsd14("max,pauth=off")
+
+    @skipUnless(os.getenv('QEMU_TEST_TIMEOUT_EXPECTED'), 'Test might timeout')
+    def test_sbsaref_freebsd14_max_pauth_impdef(self):
+        self.boot_freebsd14("max,pauth-impdef=on")
+
+    @skipUnless(os.getenv('QEMU_TEST_TIMEOUT_EXPECTED'), 'Test might timeout')
+    def test_sbsaref_freebsd14_max(self):
+        self.boot_freebsd14("max")
+
+
 if __name__ == '__main__':
     QemuSystemTest.main()
-- 
2.34.1


Re: [PULL 32/38] tests: add FreeBSD tests for aarch64/sbsa-ref
Posted by Thomas Huth 3 weeks, 5 days ago
On 19/09/2024 15.11, Peter Maydell wrote:
> From: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
> 
> FreeBSD has longer support cycle for stable release (14.x EoL in 2028)
> than OpenBSD (7.3 we use is already EoL). Also bugfixes are backported
> so we can stay on 14.x for longer.
> 
> Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
> Message-id: 20240910-b4-move-to-freebsd-v5-2-0fb66d803c93@linaro.org
> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
>   tests/functional/test_aarch64_sbsaref.py | 43 +++++++++++++++++++++++-
>   1 file changed, 42 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/functional/test_aarch64_sbsaref.py b/tests/functional/test_aarch64_sbsaref.py
> index 528882f4fa9..89d0b9587a8 100755
> --- a/tests/functional/test_aarch64_sbsaref.py
> +++ b/tests/functional/test_aarch64_sbsaref.py
> @@ -1,6 +1,6 @@
>   #!/usr/bin/env python3
>   #
> -# Functional test that boots a Linux kernel and checks the console
> +# Functional test that boots a kernel and checks the console
>   #
>   # SPDX-FileCopyrightText: 2023-2024 Linaro Ltd.
>   # SPDX-FileContributor: Philippe Mathieu-Daudé <philmd@linaro.org>
> @@ -184,5 +184,46 @@ def test_sbsaref_openbsd73_max(self):
>           self.boot_openbsd73("max")
>   
>   
> +    ASSET_FREEBSD_ISO = Asset(
> +        ('https://download.freebsd.org/releases/arm64/aarch64/ISO-IMAGES/'
> +         '14.1/FreeBSD-14.1-RELEASE-arm64-aarch64-bootonly.iso'),
> +        '44cdbae275ef1bb6dab1d5fbb59473d4f741e1c8ea8a80fd9e906b531d6ad461')
> +
> +    # This tests the whole boot chain from EFI to Userspace
> +    # We only boot a whole OS for the current top level CPU and GIC
> +    # Other test profiles should use more minimal boots
> +    def boot_freebsd14(self, cpu=None):
> +        self.fetch_firmware()
> +
> +        img_path = self.ASSET_FREEBSD_ISO.fetch()
> +
> +        self.vm.set_console()
> +        self.vm.add_args(
> +            "-drive", f"file={img_path},format=raw,snapshot=on",
> +        )
> +        if cpu:
> +            self.vm.add_args("-cpu", cpu)
> +
> +        self.vm.launch()
> +        wait_for_console_pattern(self, 'Welcome to FreeBSD!')
> +
> +    def test_sbsaref_freebsd14_cortex_a57(self):
> +        self.boot_freebsd14("cortex-a57")
> +
> +    def test_sbsaref_freebsd14_default_cpu(self):
> +        self.boot_freebsd14()
> +
> +    def test_sbsaref_freebsd14_max_pauth_off(self):
> +        self.boot_freebsd14("max,pauth=off")

  Hi Marcin,

these new tests cause the CI to time out in my fork of the QEMU repository:

  https://gitlab.com/thuth/qemu/-/jobs/7915380405
  https://gitlab.com/thuth/qemu/-/jobs/7916009309
  https://gitlab.com/thuth/qemu/-/jobs/7914930038

Looking at the console.log artifacts, it seems like this 
test_sbsaref_freebsd14_max_pauth_off is hanging at arbitrary points in time?

Could this somehow be fixed, or if not, shall we add a @skipUnless decorator 
here, too?

  Thomas


> +    @skipUnless(os.getenv('QEMU_TEST_TIMEOUT_EXPECTED'), 'Test might timeout')
> +    def test_sbsaref_freebsd14_max_pauth_impdef(self):
> +        self.boot_freebsd14("max,pauth-impdef=on")
> +
> +    @skipUnless(os.getenv('QEMU_TEST_TIMEOUT_EXPECTED'), 'Test might timeout')
> +    def test_sbsaref_freebsd14_max(self):
> +        self.boot_freebsd14("max")
> +
> +
>   if __name__ == '__main__':
>       QemuSystemTest.main()