[PATCH] tests/acceptance: Add tests for the Pegasos2 machine

Cleber Rosa posted 1 patch 2 years, 9 months ago
Test checkpatch failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20210713014342.3086855-1-crosa@redhat.com
tests/acceptance/machine_ppc_pegasos.py | 103 ++++++++++++++++++++++++
1 file changed, 103 insertions(+)
create mode 100644 tests/acceptance/machine_ppc_pegasos.py
[PATCH] tests/acceptance: Add tests for the Pegasos2 machine
Posted by Cleber Rosa 2 years, 9 months ago
From: Philippe Mathieu-Daudé <f4bug@amsat.org>

Add a pair of tests for the Pegasos2 machine following the steps from:
https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg01553.html

  $ AVOCADO_ALLOW_UNTRUSTED_CODE=1 avocado --show=app,console,tesseract \
      run -t machine:pegasos2 tests/acceptance/
   (1/2) tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_rom_serial_console:
  console: PegasosII Boot Strap (c) 2002-2003 bplan GmbH
  console: Running on CPU PVR:000C0209
  console: Enable L1 ICache...                                                    Done.
  console: Reading W83194 :                                                       FAILED.
  console: Setting Front Side Bus to 133MHz...                                    FAILED.
  console: Configuring DDR...                                                     Done.
  console: Configuring PCI0...                                                    Done.
  console: Configuring PCI1...                                                    Done.
  console: Configuring ETH...                                                     Done.
  console: Releasing IDE reset ...                                                Done.
  console: Configuring Legacy Devices
  console: Initializing KBD...                                                    Done.
  console: Testing 10000000 Bytes, Pass: 00000000 Failed: 00000000
  console: RAM TEST (fill linear)...                                              Done.
  console: FFFFFFFF
  console: SmartFirmware:
  console: cpu0: PowerPC,G4 CPUClock 599 Mhz BUSClock 133 Mhz (Version 0x000C,0x0209)
  console: no/bad nvramrc - performing default startup script
  console: channel 1 unit 0 : atapi | QEMU DVD-ROM                             | 2.5+
  console: ATA device not present or not responding
  console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 (20040405172512)
  PASS (5.23 s)
   (2/2) tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_morphos_cdrom_vga:
  ...
  console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 (20040405172512)
  console: SmartFirmware(tm) Copyright 1996-2001 by CodeGen, Inc.
  console: All Rights Reserved.
  console: Pegasos BIOS Extensions Copyright 2001-2003 by bplan GmbH.
  console: All Rights Reserved.
  console: entering main read/eval loop...
  console: ok boot cd boot.img
  console: ISO-9660 filesystem:  System-ID: "MORPHOS"  Volume-ID: "MorphOSBoot"
  console: " flags=0x2 extent=0x20 size=0x1800
  console: Memory used before SYS_Init: 9MB
  console: PCI ATA/ATAPI Driver@2: PIO Mode 4
  console: PCI ATA/ATAPI Driver@2: UDMA Mode 5
  console: ide.device@2: QEMU     QEMU DVD-ROM     <CDROM>
  console: ide.device@2:  CDRom <CD001>,<MORPHOS > found, bootable
  tesseract: Ambient Screen 4: Saturday, 15 May 2021, 13:36:06 &
  tesseract: keymap
  tesseract: Albanian keyboard with 101/104 keys
  tesseract: ‘American keyboard with Greek input extension, 105 keys
  tesseract: Belarusian keyboard with 105 keys
  tesseract: Belgian keyboard with 105 keys J
  tesseract: British Apple keyboard
  tesseract: British keyboard with 105 keys
  tesseract: Bulgarian keyboard with 104 keys
  tesseract: Canadian keyboard with 105 keys
  tesseract: Colemak layout for keyboards with 101/104 keys
  tesseract: Croatian keyboard with 101/108 keys
  tesseract: Czech keyboard (QWERTY) with 101/104 keys
  tesseract: Czech keyboard (QWERTZ) with 101/104 keys
  tesseract: Danish keyboard with 105 keys
  PASS (28.56 s)
  RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
  JOB TIME   : 34.42 s

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/acceptance/machine_ppc_pegasos.py | 103 ++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100644 tests/acceptance/machine_ppc_pegasos.py

diff --git a/tests/acceptance/machine_ppc_pegasos.py b/tests/acceptance/machine_ppc_pegasos.py
new file mode 100644
index 0000000000..e890a3a539
--- /dev/null
+++ b/tests/acceptance/machine_ppc_pegasos.py
@@ -0,0 +1,103 @@
+# Functional tests for the Pegasos2 machine.
+#
+# Copyright (c) 2021 Philippe Mathieu-Daudé <f4bug@amsat.org>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import time
+
+from avocado import skipUnless
+from avocado_qemu import Test
+from avocado_qemu import exec_command_and_wait_for_pattern
+from avocado_qemu import wait_for_console_pattern
+from tesseract_utils import tesseract_available, tesseract_ocr
+
+PIL_AVAILABLE = True
+try:
+    from PIL import Image
+except ImportError:
+    PIL_AVAILABLE = False
+
+
+@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+class PPCPegasos2(Test):
+    """These tests require a Pegasos2 ROM.  You need to perform a one
+    time manual set up of the ROM, registering it with the Avocado
+    assets system.  Please follow the steps bellow from a QEMU build
+    directory, replaceing $MY_ROMS_DIR with a suitable directory.
+
+    1. make check-venv
+    2. curl http://web.archive.org/web/20071021223056/http://www.bplan-gmbh.de/up050404/up050404 -o $MY_ROMS_DIR/up050404
+    3. tail -c +85581 /tmp/up050404 | head -c 524288 > $MY_ROMS_DIR/pegasos2.rom
+    4. ./tests/venv/bin/avocado assets register --hash 08dc28afb3d10fb223376a28eebfd07c9f8df9fa pegasos2.rom $MY_ROMS_DIR/pegasos2.rom
+    """
+    timeout = 60
+    rom_hash = '08dc28afb3d10fb223376a28eebfd07c9f8df9fa'
+
+    def test_rom_serial_console(self):
+        """
+        :avocado: tags=arch:ppc
+        :avocado: tags=machine:pegasos2
+        :avocado: tags=device:mv64361
+        :avocado: tags=device:vt8231
+        """
+        rom_path = self.fetch_asset('pegasos2.rom', asset_hash=self.rom_hash)
+        self.vm.set_console()
+        self.vm.add_args('-bios', rom_path)
+        self.vm.launch()
+        msg = 'PegasosII Boot Strap (c) 2002-2003 bplan GmbH'
+        wait_for_console_pattern(self, msg)
+        msg = 'Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1'
+        wait_for_console_pattern(self, msg)
+
+    def test_morphos_cdrom_vga(self):
+        """
+        :avocado: tags=arch:ppc
+        :avocado: tags=machine:pegasos2
+        :avocado: tags=device:mv64361
+        :avocado: tags=device:vt8231
+        :avocado: tags=device:ati-vga
+        """
+        rom_path = self.fetch_asset('pegasos2.rom', asset_hash=self.rom_hash)
+        iso_url = 'https://www.morphos-team.net/morphos-3.15.iso'
+        iso_hash = 'a19dbfbbc4728e0ba9ceb6335db69ca4'
+        iso_path = self.fetch_asset(iso_url,
+                                    asset_hash=iso_hash, algorithm='md5')
+
+        self.vm.set_console()
+        self.vm.add_args('-bios', rom_path,
+                         '-device', 'ati-vga,romfile=',
+                         '-cdrom', iso_path)
+        self.vm.launch()
+        msg = 'Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1'
+        wait_for_console_pattern(self, msg)
+        wait_for_console_pattern(self, 'entering main read/eval loop...')
+        msg = 'filesystem:  System-ID: "MORPHOS"  Volume-ID: "MorphOSBoot"'
+        exec_command_and_wait_for_pattern(self, 'boot cd boot.img', msg)
+
+        msg = 'ide.device@2:  CDRom <CD001>,<MORPHOS > found, bootable'
+        wait_for_console_pattern(self, msg)
+        if PIL_AVAILABLE:
+            delay_s = 20 if tesseract_available(4) else 8
+
+            self.log.info('VM launched, waiting for display')
+            # TODO: Use avocado.utils.wait.wait_for to catch the
+            #       'displaysurface_create 1120x832' trace-event.
+            time.sleep(delay_s)
+
+            screenshot_path = os.path.join(self.workdir, "dump.ppm")
+            self.vm.command('human-monitor-command',
+                            command_line='screendump %s' % screenshot_path)
+            width, height = Image.open(screenshot_path).size
+            self.assertEqual(width, 2048)
+            self.assertEqual(height, 1152)
+
+            if tesseract_available(4):
+                lines = tesseract_ocr(screenshot_path, tesseract_version=4)
+                text = '\n'.join(lines)
+                msg = 'American keyboard with Greek input extension, 105 keys'
+                self.assertIn(msg, text)
-- 
2.31.1


Re: [PATCH] tests/acceptance: Add tests for the Pegasos2 machine
Posted by Philippe Mathieu-Daudé 2 years, 9 months ago
Hi Cleber,

On 7/13/21 3:43 AM, Cleber Rosa wrote:
> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
> 
> Add a pair of tests for the Pegasos2 machine following the steps from:
> https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg01553.html
> 
>   $ AVOCADO_ALLOW_UNTRUSTED_CODE=1 avocado --show=app,console,tesseract \
>       run -t machine:pegasos2 tests/acceptance/
>    (1/2) tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_rom_serial_console:
>   console: PegasosII Boot Strap (c) 2002-2003 bplan GmbH
>   console: Running on CPU PVR:000C0209
>   console: Enable L1 ICache...                                                    Done.
>   console: Reading W83194 :                                                       FAILED.
>   console: Setting Front Side Bus to 133MHz...                                    FAILED.
>   console: Configuring DDR...                                                     Done.
>   console: Configuring PCI0...                                                    Done.
>   console: Configuring PCI1...                                                    Done.
>   console: Configuring ETH...                                                     Done.
>   console: Releasing IDE reset ...                                                Done.
>   console: Configuring Legacy Devices
>   console: Initializing KBD...                                                    Done.
>   console: Testing 10000000 Bytes, Pass: 00000000 Failed: 00000000
>   console: RAM TEST (fill linear)...                                              Done.
>   console: FFFFFFFF
>   console: SmartFirmware:
>   console: cpu0: PowerPC,G4 CPUClock 599 Mhz BUSClock 133 Mhz (Version 0x000C,0x0209)
>   console: no/bad nvramrc - performing default startup script
>   console: channel 1 unit 0 : atapi | QEMU DVD-ROM                             | 2.5+
>   console: ATA device not present or not responding
>   console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 (20040405172512)
>   PASS (5.23 s)
>    (2/2) tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_morphos_cdrom_vga:
>   ...
>   console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 (20040405172512)
>   console: SmartFirmware(tm) Copyright 1996-2001 by CodeGen, Inc.
>   console: All Rights Reserved.
>   console: Pegasos BIOS Extensions Copyright 2001-2003 by bplan GmbH.
>   console: All Rights Reserved.
>   console: entering main read/eval loop...
>   console: ok boot cd boot.img
>   console: ISO-9660 filesystem:  System-ID: "MORPHOS"  Volume-ID: "MorphOSBoot"
>   console: " flags=0x2 extent=0x20 size=0x1800
>   console: Memory used before SYS_Init: 9MB
>   console: PCI ATA/ATAPI Driver@2: PIO Mode 4
>   console: PCI ATA/ATAPI Driver@2: UDMA Mode 5
>   console: ide.device@2: QEMU     QEMU DVD-ROM     <CDROM>
>   console: ide.device@2:  CDRom <CD001>,<MORPHOS > found, bootable
>   tesseract: Ambient Screen 4: Saturday, 15 May 2021, 13:36:06 &
>   tesseract: keymap
>   tesseract: Albanian keyboard with 101/104 keys
>   tesseract: ‘American keyboard with Greek input extension, 105 keys
>   tesseract: Belarusian keyboard with 105 keys
>   tesseract: Belgian keyboard with 105 keys J
>   tesseract: British Apple keyboard
>   tesseract: British keyboard with 105 keys
>   tesseract: Bulgarian keyboard with 104 keys
>   tesseract: Canadian keyboard with 105 keys
>   tesseract: Colemak layout for keyboards with 101/104 keys
>   tesseract: Croatian keyboard with 101/108 keys
>   tesseract: Czech keyboard (QWERTY) with 101/104 keys
>   tesseract: Czech keyboard (QWERTZ) with 101/104 keys
>   tesseract: Danish keyboard with 105 keys
>   PASS (28.56 s)
>   RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>   JOB TIME   : 34.42 s
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---

So this is v2.

Here would be nice to have the list of changes.

>  tests/acceptance/machine_ppc_pegasos.py | 103 ++++++++++++++++++++++++
>  1 file changed, 103 insertions(+)
>  create mode 100644 tests/acceptance/machine_ppc_pegasos.py
> 
> diff --git a/tests/acceptance/machine_ppc_pegasos.py b/tests/acceptance/machine_ppc_pegasos.py
> new file mode 100644
> index 0000000000..e890a3a539
> --- /dev/null
> +++ b/tests/acceptance/machine_ppc_pegasos.py
> @@ -0,0 +1,103 @@
> +# Functional tests for the Pegasos2 machine.
> +#
> +# Copyright (c) 2021 Philippe Mathieu-Daudé <f4bug@amsat.org>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> +# See the COPYING file in the top-level directory.
> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +import os
> +import time
> +
> +from avocado import skipUnless
> +from avocado_qemu import Test
> +from avocado_qemu import exec_command_and_wait_for_pattern
> +from avocado_qemu import wait_for_console_pattern
> +from tesseract_utils import tesseract_available, tesseract_ocr
> +
> +PIL_AVAILABLE = True
> +try:
> +    from PIL import Image
> +except ImportError:
> +    PIL_AVAILABLE = False
> +
> +
> +@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
> +class PPCPegasos2(Test):
> +    """These tests require a Pegasos2 ROM.  You need to perform a one
> +    time manual set up of the ROM, registering it with the Avocado
> +    assets system.  Please follow the steps bellow from a QEMU build
> +    directory, replaceing $MY_ROMS_DIR with a suitable directory.
> +
> +    1. make check-venv
> +    2. curl http://web.archive.org/web/20071021223056/http://www.bplan-gmbh.de/up050404/up050404 -o $MY_ROMS_DIR/up050404
> +    3. tail -c +85581 /tmp/up050404 | head -c 524288 > $MY_ROMS_DIR/pegasos2.rom

Sorry but I am not signing for this patch description. I don't think
the project should link non-free software like this; which is why I
only listed the recipe link in the commit description. Maybe we can
simply add this link here again.

Stefan do you know what is recommended?

> +    4. ./tests/venv/bin/avocado assets register --hash 08dc28afb3d10fb223376a28eebfd07c9f8df9fa pegasos2.rom $MY_ROMS_DIR/pegasos2.rom
> +    """
> +    timeout = 60
> +    rom_hash = '08dc28afb3d10fb223376a28eebfd07c9f8df9fa'
> +
> +    def test_rom_serial_console(self):
> +        """
> +        :avocado: tags=arch:ppc
> +        :avocado: tags=machine:pegasos2
> +        :avocado: tags=device:mv64361
> +        :avocado: tags=device:vt8231
> +        """
> +        rom_path = self.fetch_asset('pegasos2.rom', asset_hash=self.rom_hash)
> +        self.vm.set_console()
> +        self.vm.add_args('-bios', rom_path)
> +        self.vm.launch()
> +        msg = 'PegasosII Boot Strap (c) 2002-2003 bplan GmbH'
> +        wait_for_console_pattern(self, msg)
> +        msg = 'Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1'
> +        wait_for_console_pattern(self, msg)
> +
> +    def test_morphos_cdrom_vga(self):
> +        """
> +        :avocado: tags=arch:ppc
> +        :avocado: tags=machine:pegasos2
> +        :avocado: tags=device:mv64361
> +        :avocado: tags=device:vt8231
> +        :avocado: tags=device:ati-vga
> +        """
> +        rom_path = self.fetch_asset('pegasos2.rom', asset_hash=self.rom_hash)
> +        iso_url = 'https://www.morphos-team.net/morphos-3.15.iso'
> +        iso_hash = 'a19dbfbbc4728e0ba9ceb6335db69ca4'
> +        iso_path = self.fetch_asset(iso_url,
> +                                    asset_hash=iso_hash, algorithm='md5')
> +
> +        self.vm.set_console()
> +        self.vm.add_args('-bios', rom_path,
> +                         '-device', 'ati-vga,romfile=',
> +                         '-cdrom', iso_path)
> +        self.vm.launch()
> +        msg = 'Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1'
> +        wait_for_console_pattern(self, msg)
> +        wait_for_console_pattern(self, 'entering main read/eval loop...')
> +        msg = 'filesystem:  System-ID: "MORPHOS"  Volume-ID: "MorphOSBoot"'
> +        exec_command_and_wait_for_pattern(self, 'boot cd boot.img', msg)
> +
> +        msg = 'ide.device@2:  CDRom <CD001>,<MORPHOS > found, bootable'
> +        wait_for_console_pattern(self, msg)
> +        if PIL_AVAILABLE:
> +            delay_s = 20 if tesseract_available(4) else 8
> +
> +            self.log.info('VM launched, waiting for display')
> +            # TODO: Use avocado.utils.wait.wait_for to catch the
> +            #       'displaysurface_create 1120x832' trace-event.
> +            time.sleep(delay_s)
> +
> +            screenshot_path = os.path.join(self.workdir, "dump.ppm")
> +            self.vm.command('human-monitor-command',
> +                            command_line='screendump %s' % screenshot_path)
> +            width, height = Image.open(screenshot_path).size
> +            self.assertEqual(width, 2048)
> +            self.assertEqual(height, 1152)
> +
> +            if tesseract_available(4):
> +                lines = tesseract_ocr(screenshot_path, tesseract_version=4)
> +                text = '\n'.join(lines)
> +                msg = 'American keyboard with Greek input extension, 105 keys'
> +                self.assertIn(msg, text)
> 

For the test code part it looks better indeed, thanks for the help here.

Regards,

Phil.

Re: [PATCH] tests/acceptance: Add tests for the Pegasos2 machine
Posted by BALATON Zoltan 2 years, 9 months ago
On Wed, 14 Jul 2021, Philippe Mathieu-Daudé wrote:
> Hi Cleber,
>
> On 7/13/21 3:43 AM, Cleber Rosa wrote:
>> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>
>> Add a pair of tests for the Pegasos2 machine following the steps from:
>> https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg01553.html
>>
>>   $ AVOCADO_ALLOW_UNTRUSTED_CODE=1 avocado --show=app,console,tesseract \
>>       run -t machine:pegasos2 tests/acceptance/
>>    (1/2) tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_rom_serial_console:
>>   console: PegasosII Boot Strap (c) 2002-2003 bplan GmbH
>>   console: Running on CPU PVR:000C0209
>>   console: Enable L1 ICache...                                                    Done.
>>   console: Reading W83194 :                                                       FAILED.
>>   console: Setting Front Side Bus to 133MHz...                                    FAILED.
>>   console: Configuring DDR...                                                     Done.
>>   console: Configuring PCI0...                                                    Done.
>>   console: Configuring PCI1...                                                    Done.
>>   console: Configuring ETH...                                                     Done.
>>   console: Releasing IDE reset ...                                                Done.
>>   console: Configuring Legacy Devices
>>   console: Initializing KBD...                                                    Done.
>>   console: Testing 10000000 Bytes, Pass: 00000000 Failed: 00000000
>>   console: RAM TEST (fill linear)...                                              Done.
>>   console: FFFFFFFF
>>   console: SmartFirmware:
>>   console: cpu0: PowerPC,G4 CPUClock 599 Mhz BUSClock 133 Mhz (Version 0x000C,0x0209)
>>   console: no/bad nvramrc - performing default startup script
>>   console: channel 1 unit 0 : atapi | QEMU DVD-ROM                             | 2.5+
>>   console: ATA device not present or not responding
>>   console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 (20040405172512)
>>   PASS (5.23 s)
>>    (2/2) tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_morphos_cdrom_vga:
>>   ...
>>   console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 (20040405172512)
>>   console: SmartFirmware(tm) Copyright 1996-2001 by CodeGen, Inc.
>>   console: All Rights Reserved.
>>   console: Pegasos BIOS Extensions Copyright 2001-2003 by bplan GmbH.
>>   console: All Rights Reserved.
>>   console: entering main read/eval loop...
>>   console: ok boot cd boot.img
>>   console: ISO-9660 filesystem:  System-ID: "MORPHOS"  Volume-ID: "MorphOSBoot"
>>   console: " flags=0x2 extent=0x20 size=0x1800
>>   console: Memory used before SYS_Init: 9MB
>>   console: PCI ATA/ATAPI Driver@2: PIO Mode 4
>>   console: PCI ATA/ATAPI Driver@2: UDMA Mode 5
>>   console: ide.device@2: QEMU     QEMU DVD-ROM     <CDROM>
>>   console: ide.device@2:  CDRom <CD001>,<MORPHOS > found, bootable
>>   tesseract: Ambient Screen 4: Saturday, 15 May 2021, 13:36:06 &
>>   tesseract: keymap
>>   tesseract: Albanian keyboard with 101/104 keys
>>   tesseract: ‘American keyboard with Greek input extension, 105 keys
>>   tesseract: Belarusian keyboard with 105 keys
>>   tesseract: Belgian keyboard with 105 keys J
>>   tesseract: British Apple keyboard
>>   tesseract: British keyboard with 105 keys
>>   tesseract: Bulgarian keyboard with 104 keys
>>   tesseract: Canadian keyboard with 105 keys
>>   tesseract: Colemak layout for keyboards with 101/104 keys
>>   tesseract: Croatian keyboard with 101/108 keys
>>   tesseract: Czech keyboard (QWERTY) with 101/104 keys
>>   tesseract: Czech keyboard (QWERTZ) with 101/104 keys
>>   tesseract: Danish keyboard with 105 keys
>>   PASS (28.56 s)
>>   RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>>   JOB TIME   : 34.42 s
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>> ---
>
> So this is v2.
>
> Here would be nice to have the list of changes.
>
>>  tests/acceptance/machine_ppc_pegasos.py | 103 ++++++++++++++++++++++++
>>  1 file changed, 103 insertions(+)
>>  create mode 100644 tests/acceptance/machine_ppc_pegasos.py
>>
>> diff --git a/tests/acceptance/machine_ppc_pegasos.py b/tests/acceptance/machine_ppc_pegasos.py
>> new file mode 100644
>> index 0000000000..e890a3a539
>> --- /dev/null
>> +++ b/tests/acceptance/machine_ppc_pegasos.py
>> @@ -0,0 +1,103 @@
>> +# Functional tests for the Pegasos2 machine.
>> +#
>> +# Copyright (c) 2021 Philippe Mathieu-Daudé <f4bug@amsat.org>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
>> +# See the COPYING file in the top-level directory.
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +import os
>> +import time
>> +
>> +from avocado import skipUnless
>> +from avocado_qemu import Test
>> +from avocado_qemu import exec_command_and_wait_for_pattern
>> +from avocado_qemu import wait_for_console_pattern
>> +from tesseract_utils import tesseract_available, tesseract_ocr
>> +
>> +PIL_AVAILABLE = True
>> +try:
>> +    from PIL import Image
>> +except ImportError:
>> +    PIL_AVAILABLE = False
>> +
>> +
>> +@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
>> +class PPCPegasos2(Test):
>> +    """These tests require a Pegasos2 ROM.  You need to perform a one
>> +    time manual set up of the ROM, registering it with the Avocado
>> +    assets system.  Please follow the steps bellow from a QEMU build
>> +    directory, replaceing $MY_ROMS_DIR with a suitable directory.
>> +
>> +    1. make check-venv
>> +    2. curl http://web.archive.org/web/20071021223056/http://www.bplan-gmbh.de/up050404/up050404 -o $MY_ROMS_DIR/up050404
>> +    3. tail -c +85581 /tmp/up050404 | head -c 524288 > $MY_ROMS_DIR/pegasos2.rom
>
> Sorry but I am not signing for this patch description. I don't think
> the project should link non-free software like this; which is why I
> only listed the recipe link in the commit description. Maybe we can
> simply add this link here again.
>
> Stefan do you know what is recommended?
>
>> +    4. ./tests/venv/bin/avocado assets register --hash 08dc28afb3d10fb223376a28eebfd07c9f8df9fa pegasos2.rom $MY_ROMS_DIR/pegasos2.rom
>> +    """
>> +    timeout = 60
>> +    rom_hash = '08dc28afb3d10fb223376a28eebfd07c9f8df9fa'
>> +
>> +    def test_rom_serial_console(self):
>> +        """
>> +        :avocado: tags=arch:ppc
>> +        :avocado: tags=machine:pegasos2
>> +        :avocado: tags=device:mv64361
>> +        :avocado: tags=device:vt8231
>> +        """
>> +        rom_path = self.fetch_asset('pegasos2.rom', asset_hash=self.rom_hash)
>> +        self.vm.set_console()
>> +        self.vm.add_args('-bios', rom_path)
>> +        self.vm.launch()
>> +        msg = 'PegasosII Boot Strap (c) 2002-2003 bplan GmbH'
>> +        wait_for_console_pattern(self, msg)
>> +        msg = 'Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1'
>> +        wait_for_console_pattern(self, msg)
>> +
>> +    def test_morphos_cdrom_vga(self):
>> +        """
>> +        :avocado: tags=arch:ppc
>> +        :avocado: tags=machine:pegasos2
>> +        :avocado: tags=device:mv64361
>> +        :avocado: tags=device:vt8231
>> +        :avocado: tags=device:ati-vga
>> +        """
>> +        rom_path = self.fetch_asset('pegasos2.rom', asset_hash=self.rom_hash)
>> +        iso_url = 'https://www.morphos-team.net/morphos-3.15.iso'
>> +        iso_hash = 'a19dbfbbc4728e0ba9ceb6335db69ca4'
>> +        iso_path = self.fetch_asset(iso_url,
>> +                                    asset_hash=iso_hash, algorithm='md5')
>> +
>> +        self.vm.set_console()
>> +        self.vm.add_args('-bios', rom_path,
>> +                         '-device', 'ati-vga,romfile=',
>> +                         '-cdrom', iso_path)
>> +        self.vm.launch()
>> +        msg = 'Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1'
>> +        wait_for_console_pattern(self, msg)
>> +        wait_for_console_pattern(self, 'entering main read/eval loop...')
>> +        msg = 'filesystem:  System-ID: "MORPHOS"  Volume-ID: "MorphOSBoot"'

This should wait until these messages instead:

ide.device@2: QEMU     QEMU DVD-ROM     <CDROM>
ide.device@2:  CDRom <CD001>,<MORPHOS > found, bootable

otherwise it could happen this test passes when cdrom somewhat works so it 
can find the iso but DMA is not working so it can't actually read it. I've 
seen this happening during development and when via-ide was in wrong mode 
so waiting for the above lines confirm MorphOS could also read the CD not 
just find it. (Although you seem to go further and try to parse the 
graphical screen too. I think that's probably an overkill but could detect 
if MorphOS booted and maybe test ati-vga too.)

Regards,
BALATON Zoltan

>> +        exec_command_and_wait_for_pattern(self, 'boot cd boot.img', msg)
>> +
>> +        msg = 'ide.device@2:  CDRom <CD001>,<MORPHOS > found, bootable'
>> +        wait_for_console_pattern(self, msg)
>> +        if PIL_AVAILABLE:
>> +            delay_s = 20 if tesseract_available(4) else 8
>> +
>> +            self.log.info('VM launched, waiting for display')
>> +            # TODO: Use avocado.utils.wait.wait_for to catch the
>> +            #       'displaysurface_create 1120x832' trace-event.
>> +            time.sleep(delay_s)
>> +
>> +            screenshot_path = os.path.join(self.workdir, "dump.ppm")
>> +            self.vm.command('human-monitor-command',
>> +                            command_line='screendump %s' % screenshot_path)
>> +            width, height = Image.open(screenshot_path).size
>> +            self.assertEqual(width, 2048)
>> +            self.assertEqual(height, 1152)
>> +
>> +            if tesseract_available(4):
>> +                lines = tesseract_ocr(screenshot_path, tesseract_version=4)
>> +                text = '\n'.join(lines)
>> +                msg = 'American keyboard with Greek input extension, 105 keys'
>> +                self.assertIn(msg, text)
>>
>
> For the test code part it looks better indeed, thanks for the help here.
>
> Regards,
>
> Phil.
>
>
Re: [PATCH] tests/acceptance: Add tests for the Pegasos2 machine
Posted by BALATON Zoltan 2 years, 9 months ago
On Wed, 14 Jul 2021, Philippe Mathieu-Daudé wrote:
> Hi Cleber,
>
> On 7/13/21 3:43 AM, Cleber Rosa wrote:
>> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>
>> Add a pair of tests for the Pegasos2 machine following the steps from:
>> https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg01553.html
>>
>>   $ AVOCADO_ALLOW_UNTRUSTED_CODE=1 avocado --show=app,console,tesseract \
>>       run -t machine:pegasos2 tests/acceptance/
>>    (1/2) tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_rom_serial_console:
>>   console: PegasosII Boot Strap (c) 2002-2003 bplan GmbH
>>   console: Running on CPU PVR:000C0209
>>   console: Enable L1 ICache...                                                    Done.
>>   console: Reading W83194 :                                                       FAILED.
>>   console: Setting Front Side Bus to 133MHz...                                    FAILED.
>>   console: Configuring DDR...                                                     Done.
>>   console: Configuring PCI0...                                                    Done.
>>   console: Configuring PCI1...                                                    Done.
>>   console: Configuring ETH...                                                     Done.
>>   console: Releasing IDE reset ...                                                Done.
>>   console: Configuring Legacy Devices
>>   console: Initializing KBD...                                                    Done.
>>   console: Testing 10000000 Bytes, Pass: 00000000 Failed: 00000000
>>   console: RAM TEST (fill linear)...                                              Done.
>>   console: FFFFFFFF
>>   console: SmartFirmware:
>>   console: cpu0: PowerPC,G4 CPUClock 599 Mhz BUSClock 133 Mhz (Version 0x000C,0x0209)
>>   console: no/bad nvramrc - performing default startup script
>>   console: channel 1 unit 0 : atapi | QEMU DVD-ROM                             | 2.5+
>>   console: ATA device not present or not responding
>>   console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 (20040405172512)
>>   PASS (5.23 s)
>>    (2/2) tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_morphos_cdrom_vga:
>>   ...
>>   console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 (20040405172512)
>>   console: SmartFirmware(tm) Copyright 1996-2001 by CodeGen, Inc.
>>   console: All Rights Reserved.
>>   console: Pegasos BIOS Extensions Copyright 2001-2003 by bplan GmbH.
>>   console: All Rights Reserved.
>>   console: entering main read/eval loop...
>>   console: ok boot cd boot.img
>>   console: ISO-9660 filesystem:  System-ID: "MORPHOS"  Volume-ID: "MorphOSBoot"
>>   console: " flags=0x2 extent=0x20 size=0x1800
>>   console: Memory used before SYS_Init: 9MB
>>   console: PCI ATA/ATAPI Driver@2: PIO Mode 4
>>   console: PCI ATA/ATAPI Driver@2: UDMA Mode 5
>>   console: ide.device@2: QEMU     QEMU DVD-ROM     <CDROM>
>>   console: ide.device@2:  CDRom <CD001>,<MORPHOS > found, bootable
>>   tesseract: Ambient Screen 4: Saturday, 15 May 2021, 13:36:06 &
>>   tesseract: keymap
>>   tesseract: Albanian keyboard with 101/104 keys
>>   tesseract: ‘American keyboard with Greek input extension, 105 keys
>>   tesseract: Belarusian keyboard with 105 keys
>>   tesseract: Belgian keyboard with 105 keys J
>>   tesseract: British Apple keyboard
>>   tesseract: British keyboard with 105 keys
>>   tesseract: Bulgarian keyboard with 104 keys
>>   tesseract: Canadian keyboard with 105 keys
>>   tesseract: Colemak layout for keyboards with 101/104 keys
>>   tesseract: Croatian keyboard with 101/108 keys
>>   tesseract: Czech keyboard (QWERTY) with 101/104 keys
>>   tesseract: Czech keyboard (QWERTZ) with 101/104 keys
>>   tesseract: Danish keyboard with 105 keys
>>   PASS (28.56 s)
>>   RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>>   JOB TIME   : 34.42 s
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>> ---
>
> So this is v2.
>
> Here would be nice to have the list of changes.
>
>>  tests/acceptance/machine_ppc_pegasos.py | 103 ++++++++++++++++++++++++
>>  1 file changed, 103 insertions(+)
>>  create mode 100644 tests/acceptance/machine_ppc_pegasos.py
>>
>> diff --git a/tests/acceptance/machine_ppc_pegasos.py b/tests/acceptance/machine_ppc_pegasos.py
>> new file mode 100644
>> index 0000000000..e890a3a539
>> --- /dev/null
>> +++ b/tests/acceptance/machine_ppc_pegasos.py
>> @@ -0,0 +1,103 @@
>> +# Functional tests for the Pegasos2 machine.
>> +#
>> +# Copyright (c) 2021 Philippe Mathieu-Daudé <f4bug@amsat.org>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
>> +# See the COPYING file in the top-level directory.
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +import os
>> +import time
>> +
>> +from avocado import skipUnless
>> +from avocado_qemu import Test
>> +from avocado_qemu import exec_command_and_wait_for_pattern
>> +from avocado_qemu import wait_for_console_pattern
>> +from tesseract_utils import tesseract_available, tesseract_ocr
>> +
>> +PIL_AVAILABLE = True
>> +try:
>> +    from PIL import Image
>> +except ImportError:
>> +    PIL_AVAILABLE = False
>> +
>> +
>> +@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
>> +class PPCPegasos2(Test):
>> +    """These tests require a Pegasos2 ROM.  You need to perform a one
>> +    time manual set up of the ROM, registering it with the Avocado
>> +    assets system.  Please follow the steps bellow from a QEMU build
>> +    directory, replaceing $MY_ROMS_DIR with a suitable directory.
>> +
>> +    1. make check-venv
>> +    2. curl http://web.archive.org/web/20071021223056/http://www.bplan-gmbh.de/up050404/up050404 -o $MY_ROMS_DIR/up050404
>> +    3. tail -c +85581 /tmp/up050404 | head -c 524288 > $MY_ROMS_DIR/pegasos2.rom
>
> Sorry but I am not signing for this patch description. I don't think
> the project should link non-free software like this; which is why I
> only listed the recipe link in the commit description. Maybe we can
> simply add this link here again.
>
> Stefan do you know what is recommended?
>
>> +    4. ./tests/venv/bin/avocado assets register --hash 08dc28afb3d10fb223376a28eebfd07c9f8df9fa pegasos2.rom $MY_ROMS_DIR/pegasos2.rom
>> +    """
>> +    timeout = 60
>> +    rom_hash = '08dc28afb3d10fb223376a28eebfd07c9f8df9fa'
>> +
>> +    def test_rom_serial_console(self):
>> +        """
>> +        :avocado: tags=arch:ppc
>> +        :avocado: tags=machine:pegasos2
>> +        :avocado: tags=device:mv64361
>> +        :avocado: tags=device:vt8231
>> +        """
>> +        rom_path = self.fetch_asset('pegasos2.rom', asset_hash=self.rom_hash)
>> +        self.vm.set_console()
>> +        self.vm.add_args('-bios', rom_path)
>> +        self.vm.launch()
>> +        msg = 'PegasosII Boot Strap (c) 2002-2003 bplan GmbH'
>> +        wait_for_console_pattern(self, msg)
>> +        msg = 'Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1'
>> +        wait_for_console_pattern(self, msg)
>> +
>> +    def test_morphos_cdrom_vga(self):
>> +        """
>> +        :avocado: tags=arch:ppc
>> +        :avocado: tags=machine:pegasos2
>> +        :avocado: tags=device:mv64361
>> +        :avocado: tags=device:vt8231
>> +        :avocado: tags=device:ati-vga
>> +        """
>> +        rom_path = self.fetch_asset('pegasos2.rom', asset_hash=self.rom_hash)
>> +        iso_url = 'https://www.morphos-team.net/morphos-3.15.iso'
>> +        iso_hash = 'a19dbfbbc4728e0ba9ceb6335db69ca4'
>> +        iso_path = self.fetch_asset(iso_url,
>> +                                    asset_hash=iso_hash, algorithm='md5')
>> +
>> +        self.vm.set_console()
>> +        self.vm.add_args('-bios', rom_path,
>> +                         '-device', 'ati-vga,romfile=',
>> +                         '-cdrom', iso_path)
>> +        self.vm.launch()
>> +        msg = 'Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1'
>> +        wait_for_console_pattern(self, msg)
>> +        wait_for_console_pattern(self, 'entering main read/eval loop...')
>> +        msg = 'filesystem:  System-ID: "MORPHOS"  Volume-ID: "MorphOSBoot"'
>> +        exec_command_and_wait_for_pattern(self, 'boot cd boot.img', msg)
>> +
>> +        msg = 'ide.device@2:  CDRom <CD001>,<MORPHOS > found, bootable'
>> +        wait_for_console_pattern(self, msg)

Now I see it does wait for that here too. I don't know why you have so 
many wait's instead of just two: "entering main read/eval loop\nok" and 
this one but I don't know how these tests work so that's OK. Then 
disregard my previous comment.

Regards,
BALATON Zoltan

>> +        if PIL_AVAILABLE:
>> +            delay_s = 20 if tesseract_available(4) else 8
>> +
>> +            self.log.info('VM launched, waiting for display')
>> +            # TODO: Use avocado.utils.wait.wait_for to catch the
>> +            #       'displaysurface_create 1120x832' trace-event.
>> +            time.sleep(delay_s)
>> +
>> +            screenshot_path = os.path.join(self.workdir, "dump.ppm")
>> +            self.vm.command('human-monitor-command',
>> +                            command_line='screendump %s' % screenshot_path)
>> +            width, height = Image.open(screenshot_path).size
>> +            self.assertEqual(width, 2048)
>> +            self.assertEqual(height, 1152)
>> +
>> +            if tesseract_available(4):
>> +                lines = tesseract_ocr(screenshot_path, tesseract_version=4)
>> +                text = '\n'.join(lines)
>> +                msg = 'American keyboard with Greek input extension, 105 keys'
>> +                self.assertIn(msg, text)
>>
>
> For the test code part it looks better indeed, thanks for the help here.
>
> Regards,
>
> Phil.
>
>
Re: [PATCH] tests/acceptance: Add tests for the Pegasos2 machine
Posted by Stefan Hajnoczi 2 years, 9 months ago
On Wed, Jul 14, 2021 at 11:37:54AM +0200, Philippe Mathieu-Daudé wrote:
> On 7/13/21 3:43 AM, Cleber Rosa wrote:
> > +@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
> > +class PPCPegasos2(Test):
> > +    """These tests require a Pegasos2 ROM.  You need to perform a one
> > +    time manual set up of the ROM, registering it with the Avocado
> > +    assets system.  Please follow the steps bellow from a QEMU build
> > +    directory, replaceing $MY_ROMS_DIR with a suitable directory.
> > +
> > +    1. make check-venv
> > +    2. curl http://web.archive.org/web/20071021223056/http://www.bplan-gmbh.de/up050404/up050404 -o $MY_ROMS_DIR/up050404
> > +    3. tail -c +85581 /tmp/up050404 | head -c 524288 > $MY_ROMS_DIR/pegasos2.rom
> 
> Sorry but I am not signing for this patch description. I don't think
> the project should link non-free software like this; which is why I
> only listed the recipe link in the commit description. Maybe we can
> simply add this link here again.
> 
> Stefan do you know what is recommended?

I think that it's okay to include instructions for manually adding
non-free software, but I'm not a lawyer.

I don't know what the origin of this up050404 file is and under what
license it is available. Could you add that information so users can
make their own decision on whether or not to manually install it?

Thanks,
Stefan