This tests boot a full VM and check the serial console until
the SSH daemon is running, then start a SSH session and run
some commands.

This test can be run using:

  $ avocado --show=ssh run -t arch:mips tests/acceptance/linux_ssh_mips_malta.py
  ssh: Entering interactive session.
  ssh: # uname -a
  ssh: Linux debian-mips 3.2.0-4-4kc-malta #1 Debian 3.2.51-1 mips GNU/Linux
  ssh: # lspci -d 11ab:4620
  ssh: 00:00.0 Host bridge: Marvell Technology Group Ltd. GT-64120/64120A/64121A System Controller (rev 10)
  ssh: # cat /sys/bus/i2c/devices/i2c-0/name
  ssh: SMBus PIIX4 adapter at 1100
  ssh: # cat /proc/mtd
  ssh: dev:    size   erasesize  name
  ssh: mtd0: 00100000 00010000 "YAMON"
  ssh: mtd1: 002e0000 00010000 "User FS"
  ssh: mtd2: 00020000 00010000 "Board Config"
  ssh: # md5sum /dev/mtd2ro
  ssh: 0dfbe8aa4c20b52e1b8bf3cb6cbdf193  /dev/mtd2ro
  ssh: # poweroff

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
TODO: do not run this tests by default, use the 'slow' tag
---
 MAINTAINERS                              |   1 +
 tests/acceptance/linux_ssh_mips_malta.py | 229 +++++++++++++++++++++++
 tests/requirements.txt                   |   1 +
 3 files changed, 231 insertions(+)
 create mode 100644 tests/acceptance/linux_ssh_mips_malta.py

diff --git a/MAINTAINERS b/MAINTAINERS
index 9424a490d6..69fa4b3abc 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -934,6 +934,7 @@ M: Aurelien Jarno <aurelien@aurel32.net>
 R: Aleksandar Rikalo <arikalo@wavecomp.com>
 S: Maintained
 F: hw/mips/mips_malta.c
+F: tests/acceptance/linux_ssh_mips_malta.py
 
 Mipssim
 M: Aleksandar Markovic <amarkovic@wavecomp.com>
diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
new file mode 100644
index 0000000000..ceb530ff88
--- /dev/null
+++ b/tests/acceptance/linux_ssh_mips_malta.py
@@ -0,0 +1,229 @@
+# Functional test that boots a VM and run commands via a SSH session
+#
+# Copyright (c) Philippe Mathieu-Daudé <f4bug@amsat.org>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import os
+import re
+import base64
+import logging
+import paramiko
+import time
+
+from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
+
+
+class LinuxSSH(Test):
+
+    timeout = 150 # Not for 'configure --enable-debug --enable-debug-tcg'
+
+    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
+    VM_IP = '127.0.0.1'
+
+    IMAGE_INFO = {
+        'be': {
+            'image_url': 'https://people.debian.org/~aurel32/qemu/mips/'
+                         'debian_wheezy_mips_standard.qcow2',
+            'image_hash': '8987a63270df67345b2135a6b7a4885a35e392d5',
+            'rsa_hostkey': b'AAAAB3NzaC1yc2EAAAADAQABAAABAQCca1VitiyLAdQOld'
+                           b'zT43IOEVJZ0wHD78GJi8wDAjMiYWUzNSSn0rXGQsINHuH5'
+                           b'IlF+kBZsHinb/FtKCAyS9a8uCHhQI4SuB4QhAb0+39MlUw'
+                           b'Mm0CLkctgM2eUUZ6MQMQvDlqnue6CCkxN62EZYbaxmby7j'
+                           b'CQa1125o1HRKBvdGm2zrJWxXAfA+f1v6jHLyE8Jnu83eQ+'
+                           b'BFY25G+Vzx1PVc3zQBwJ8r0NGTRqy2//oWQP0h+bMsgeFe'
+                           b'KH/J3RJM22vg6+I4JAdBFcxnK+l781h1FuRxOn4O/Xslbg'
+                           b'go6WtB4V4TOsw2E/KfxI5IZ/icxF+swVcnvF46Hf3uQc/0'
+                           b'BBqb',
+        },
+        'le': {
+            'image_url': 'https://people.debian.org/~aurel32/qemu/mipsel/'
+                         'debian_wheezy_mipsel_standard.qcow2',
+            'image_hash': '7866764d9de3ef536ffca24c9fb9f04ffdb45802',
+            'rsa_hostkey': b'AAAAB3NzaC1yc2EAAAADAQABAAABAQClXJlBT71HL5yKvv'
+                           b'gfC7jmxSWx5zSBCzET6CLZczwAafSIs7YKfNOy/dQTxhuk'
+                           b'yIGFUugZFoF3E9PzdhunuyvyTd56MPoNIqFbb5rGokwU5I'
+                           b'TOx3dBHZR0mClypL6MVrwe0bsiIb8GhF1zioNwcsaAZnAi'
+                           b'KfXStVDtXvn/kLLq+xLABYt48CC5KYWoFaCoICskLAY+qo'
+                           b'L+LWyAnQisj4jAH8VSaSKIImFpfkHWEXPhHcC4ZBlDKtnH'
+                           b'po9vhfCHgnfW3Pzrqmk8BI4HysqPFVmJWkJGlGUL+sGeg3'
+                           b'ZZolAYuDXGuBrw8ooPJq2v2dOH+z6dyD2q/ypmAbyPqj5C'
+                           b'rc8H',
+        },
+    }
+
+    def wait_for_console_pattern(self, success_message,
+                                 failure_message='Oops'):
+        console = self.vm.console_socket.makefile()
+        console_logger = logging.getLogger('console')
+        while True:
+            msg = console.readline()
+            console_logger.debug(msg.strip())
+            if success_message in msg:
+                break
+            if failure_message in msg:
+                fail = 'Failure message found in console: %s' % failure_message
+                self.fail(fail)
+
+    def get_portfwd(self):
+        res = self.vm.command('human-monitor-command',
+                              command_line='info usernet')
+        line = res.split('\r\n')[2]
+        port = re.split(r'.*TCP.HOST_FORWARD.*127\.0\.0\.1 (\d+)\s+10\..*',
+                        line)[1]
+        self.log.debug("sshd listening on port:" + port)
+        return port
+
+    def ssh_connect(self, username, password, rsa_hostkey_b64=None):
+        self.ssh_logger = logging.getLogger('ssh')
+        self.ssh_username = username
+        self.ssh_ps1 = '# ' if username is 'root' else '$ '
+        self.ssh_client = paramiko.SSHClient()
+        port = self.get_portfwd()
+        if rsa_hostkey_b64:
+            rsa_hostkey_bin = base64.b64decode(rsa_hostkey_b64)
+            rsa_hostkey = paramiko.RSAKey(data = rsa_hostkey_bin)
+            ipport = '[%s]:%s' % (self.VM_IP, port)
+            self.ssh_logger.debug('ipport ' + ipport)
+            self.ssh_client.get_host_keys().add(ipport, 'ssh-rsa', rsa_hostkey)
+        for i in range(10):
+            try:
+                self.ssh_client.connect(self.VM_IP, int(port),
+                                        username, password, banner_timeout=90)
+                self.ssh_logger.info("Entering interactive session.")
+                return
+            except:
+                time.sleep(4)
+                pass
+        self.fail("sshd timeout")
+
+    def ssh_disconnect_vm(self):
+        self.ssh_client.close()
+
+    def ssh_command(self, command, is_root=True):
+        self.ssh_logger.info(self.ssh_ps1 + command)
+        stdin, stdout, stderr = self.ssh_client.exec_command(command)
+        stdout_lines = [line.strip('\n') for line in stdout]
+        for line in stdout_lines:
+            self.ssh_logger.info(line)
+        stderr_lines = [line.strip('\n') for line in stderr]
+        for line in stderr_lines:
+            self.ssh_logger.warning(line)
+        return stdout_lines, stderr_lines
+
+    def boot_debian_wheezy_image_and_ssh_login(self, endianess, kernel_path):
+        image_url = self.IMAGE_INFO[endianess]['image_url']
+        image_hash = self.IMAGE_INFO[endianess]['image_hash']
+        image_path = self.fetch_asset(image_url, asset_hash=image_hash)
+        rsa_hostkey_b64 = self.IMAGE_INFO[endianess]['rsa_hostkey']
+
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+                               + 'console=ttyS0 root=/dev/sda1')
+        self.vm.add_args('-no-reboot',
+                         '-kernel', kernel_path,
+                         '-append', kernel_command_line,
+                         '-hda', image_path,
+                         '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+                         '-device', 'pcnet,netdev=vnet')
+        self.vm.launch()
+
+        self.log.info('VM launched, waiting for sshd')
+        console_pattern = 'Starting OpenBSD Secure Shell server: sshd'
+        self.wait_for_console_pattern(console_pattern)
+        self.log.info('sshd ready')
+
+        self.ssh_connect('root', 'root', rsa_hostkey_b64=rsa_hostkey_b64)
+
+    def shutdown_via_ssh(self):
+        self.ssh_command('poweroff')
+        self.ssh_disconnect_vm()
+        self.wait_for_console_pattern('Power down')
+
+    def run_common_commands(self):
+        stdout, stderr = self.ssh_command('lspci -d 11ab:4620')
+        self.assertIn(True, ["GT-64120" in line for line in stdout])
+
+        stdout, stderr = self.ssh_command('cat /sys/bus/i2c/devices/i2c-0/name')
+        self.assertIn(True, ["SMBus PIIX4 adapter" in line
+                             for line in stdout])
+
+        stdout, stderr = self.ssh_command('cat /proc/mtd')
+        self.assertIn(True, ["YAMON" in line
+                             for line in stdout])
+
+        # Empty 'Board Config'
+        stdout, stderr = self.ssh_command('md5sum /dev/mtd2ro')
+        self.assertIn(True, ["0dfbe8aa4c20b52e1b8bf3cb6cbdf193" in line
+                             for line in stdout])
+
+    def do_test_mips_malta(self, endianess, kernel_path, uname_m):
+        self.boot_debian_wheezy_image_and_ssh_login(endianess, kernel_path)
+
+        stdout, stderr = self.ssh_command('uname -a')
+        self.assertIn(True, [uname_m + " GNU/Linux" in line for line in stdout])
+
+        self.run_common_commands()
+        self.shutdown_via_ssh()
+
+    def test_mips_malta32eb_kernel3_2_0(self):
+        """
+        :avocado: tags=slow
+        :avocado: tags=arch:mips
+        :avocado: tags=machine:malta
+        :avocado: tags=endian:big
+        :avocado: tags=device:pcnet32
+        """
+        kernel_url = ('https://people.debian.org/~aurel32/qemu/mips/'
+                      'vmlinux-3.2.0-4-4kc-malta')
+        kernel_hash = '592e384a4edc16dade52a6cd5c785c637bcbc9ad'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.do_test_mips_malta('be', kernel_path, 'mips')
+
+    def test_mips_malta32el_kernel3_2_0(self):
+        """
+        :avocado: tags=slow
+        :avocado: tags=arch:mipsel
+        :avocado: tags=machine:malta
+        :avocado: tags=endian:little
+        :avocado: tags=device:pcnet32
+        """
+        kernel_url = ('https://people.debian.org/~aurel32/qemu/mipsel/'
+                      'vmlinux-3.2.0-4-4kc-malta')
+        kernel_hash = 'a66bea5a8adaa2cb3d36a1d4e0ccdb01be8f6c2a'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.do_test_mips_malta('le', kernel_path, 'mips')
+
+    def test_mips_malta64eb_kernel3_2_0(self):
+        """
+        :avocado: tags=slow
+        :avocado: tags=arch:mips64
+        :avocado: tags=machine:malta
+        :avocado: tags=endian:big
+        :avocado: tags=device:pcnet32
+        """
+        kernel_url = ('https://people.debian.org/~aurel32/qemu/mips/'
+                      'vmlinux-3.2.0-4-5kc-malta')
+        kernel_hash = 'db6eea7de35d36c77d8c165b6bcb222e16eb91db'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+        self.do_test_mips_malta('be', kernel_path, 'mips64')
+
+    def test_mips_malta64el_kernel3_2_0(self):
+        """
+        :avocado: tags=slow
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        :avocado: tags=endian:little
+        :avocado: tags=device:pcnet32
+        """
+        kernel_url = ('https://people.debian.org/~aurel32/qemu/mipsel/'
+                      'vmlinux-3.2.0-4-5kc-malta')
+        kernel_hash = '6a7f77245acf231415a0e8b725d91ed2f3487794'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+        self.do_test_mips_malta('le', kernel_path, 'mips64')
diff --git a/tests/requirements.txt b/tests/requirements.txt
index 002ded6a22..3ae0e29ad7 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -2,3 +2,4 @@
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
 avocado-framework==68.0
+paramiko
-- 
2.19.1

On May 21, 2019 1:19 AM, "Philippe Mathieu-Daudé" <f4bug@amsat.org> wrote:
>
> This tests boot a full VM and check the serial console until
> the SSH daemon is running, then start a SSH session and run
> some commands.
>
> This test can be run using:
>
>   $ avocado --show=ssh run -t arch:mips
tests/acceptance/linux_ssh_mips_malta.py
>   ssh: Entering interactive session.
>   ssh: # uname -a
>   ssh: Linux debian-mips 3.2.0-4-4kc-malta #1 Debian 3.2.51-1 mips
GNU/Linux
>   ssh: # lspci -d 11ab:4620
>   ssh: 00:00.0 Host bridge: Marvell Technology Group Ltd.
GT-64120/64120A/64121A System Controller (rev 10)
>   ssh: # cat /sys/bus/i2c/devices/i2c-0/name
>   ssh: SMBus PIIX4 adapter at 1100
>   ssh: # cat /proc/mtd
>   ssh: dev:    size   erasesize  name
>   ssh: mtd0: 00100000 00010000 "YAMON"
>   ssh: mtd1: 002e0000 00010000 "User FS"
>   ssh: mtd2: 00020000 00010000 "Board Config"
>   ssh: # md5sum /dev/mtd2ro
>   ssh: 0dfbe8aa4c20b52e1b8bf3cb6cbdf193  /dev/mtd2ro
>   ssh: # poweroff
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---

Excelent! Frankly, this was something that we in MIPS needed and missed
very much for a long time.

I liked the idea that this test does not run as a default, giving us
opportunity to extend and adjust it in future as we deem appropriate for
our MIPS needs, but without affecting people running default test
execution. In other words, this patch achieves “best of both worlds”.

Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>

> TODO: do not run this tests by default, use the 'slow' tag
> ---
>  MAINTAINERS                              |   1 +
>  tests/acceptance/linux_ssh_mips_malta.py | 229 +++++++++++++++++++++++
>  tests/requirements.txt                   |   1 +
>  3 files changed, 231 insertions(+)
>  create mode 100644 tests/acceptance/linux_ssh_mips_malta.py
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 9424a490d6..69fa4b3abc 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -934,6 +934,7 @@ M: Aurelien Jarno <aurelien@aurel32.net>
>  R: Aleksandar Rikalo <arikalo@wavecomp.com>
>  S: Maintained
>  F: hw/mips/mips_malta.c
> +F: tests/acceptance/linux_ssh_mips_malta.py
>
>  Mipssim
>  M: Aleksandar Markovic <amarkovic@wavecomp.com>
> diff --git a/tests/acceptance/linux_ssh_mips_malta.py
b/tests/acceptance/linux_ssh_mips_malta.py
> new file mode 100644
> index 0000000000..ceb530ff88
> --- /dev/null
> +++ b/tests/acceptance/linux_ssh_mips_malta.py
> @@ -0,0 +1,229 @@
> +# Functional test that boots a VM and run commands via a SSH session
> +#
> +# Copyright (c) Philippe Mathieu-Daudé <f4bug@amsat.org>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.
> +
> +import os
> +import re
> +import base64
> +import logging
> +import paramiko
> +import time
> +
> +from avocado_qemu import Test
> +from avocado.utils import process
> +from avocado.utils import archive
> +
> +
> +class LinuxSSH(Test):
> +
> +    timeout = 150 # Not for 'configure --enable-debug --enable-debug-tcg'
> +
> +    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
> +    VM_IP = '127.0.0.1'
> +
> +    IMAGE_INFO = {
> +        'be': {
> +            'image_url': 'https://people.debian.org/~aurel32/qemu/mips/'
> +                         'debian_wheezy_mips_standard.qcow2',
> +            'image_hash': '8987a63270df67345b2135a6b7a4885a35e392d5',
> +            'rsa_hostkey':
b'AAAAB3NzaC1yc2EAAAADAQABAAABAQCca1VitiyLAdQOld'
> +
 b'zT43IOEVJZ0wHD78GJi8wDAjMiYWUzNSSn0rXGQsINHuH5'
> +
 b'IlF+kBZsHinb/FtKCAyS9a8uCHhQI4SuB4QhAb0+39MlUw'
> +
 b'Mm0CLkctgM2eUUZ6MQMQvDlqnue6CCkxN62EZYbaxmby7j'
> +
 b'CQa1125o1HRKBvdGm2zrJWxXAfA+f1v6jHLyE8Jnu83eQ+'
> +
 b'BFY25G+Vzx1PVc3zQBwJ8r0NGTRqy2//oWQP0h+bMsgeFe'
> +
 b'KH/J3RJM22vg6+I4JAdBFcxnK+l781h1FuRxOn4O/Xslbg'
> +
 b'go6WtB4V4TOsw2E/KfxI5IZ/icxF+swVcnvF46Hf3uQc/0'
> +                           b'BBqb',
> +        },
> +        'le': {
> +            'image_url': 'https://people.debian.org/~aurel32/qemu/mipsel/
'
> +                         'debian_wheezy_mipsel_standard.qcow2',
> +            'image_hash': '7866764d9de3ef536ffca24c9fb9f04ffdb45802',
> +            'rsa_hostkey':
b'AAAAB3NzaC1yc2EAAAADAQABAAABAQClXJlBT71HL5yKvv'
> +
 b'gfC7jmxSWx5zSBCzET6CLZczwAafSIs7YKfNOy/dQTxhuk'
> +
 b'yIGFUugZFoF3E9PzdhunuyvyTd56MPoNIqFbb5rGokwU5I'
> +
 b'TOx3dBHZR0mClypL6MVrwe0bsiIb8GhF1zioNwcsaAZnAi'
> +
 b'KfXStVDtXvn/kLLq+xLABYt48CC5KYWoFaCoICskLAY+qo'
> +
 b'L+LWyAnQisj4jAH8VSaSKIImFpfkHWEXPhHcC4ZBlDKtnH'
> +
 b'po9vhfCHgnfW3Pzrqmk8BI4HysqPFVmJWkJGlGUL+sGeg3'
> +
 b'ZZolAYuDXGuBrw8ooPJq2v2dOH+z6dyD2q/ypmAbyPqj5C'
> +                           b'rc8H',
> +        },
> +    }
> +
> +    def wait_for_console_pattern(self, success_message,
> +                                 failure_message='Oops'):
> +        console = self.vm.console_socket.makefile()
> +        console_logger = logging.getLogger('console')
> +        while True:
> +            msg = console.readline()
> +            console_logger.debug(msg.strip())
> +            if success_message in msg:
> +                break
> +            if failure_message in msg:
> +                fail = 'Failure message found in console: %s' %
failure_message
> +                self.fail(fail)
> +
> +    def get_portfwd(self):
> +        res = self.vm.command('human-monitor-command',
> +                              command_line='info usernet')
> +        line = res.split('\r\n')[2]
> +        port = re.split(r'.*TCP.HOST_FORWARD.*127\.0\.0\.1
(\d+)\s+10\..*',
> +                        line)[1]
> +        self.log.debug("sshd listening on port:" + port)
> +        return port
> +
> +    def ssh_connect(self, username, password, rsa_hostkey_b64=None):
> +        self.ssh_logger = logging.getLogger('ssh')
> +        self.ssh_username = username
> +        self.ssh_ps1 = '# ' if username is 'root' else '$ '
> +        self.ssh_client = paramiko.SSHClient()
> +        port = self.get_portfwd()
> +        if rsa_hostkey_b64:
> +            rsa_hostkey_bin = base64.b64decode(rsa_hostkey_b64)
> +            rsa_hostkey = paramiko.RSAKey(data = rsa_hostkey_bin)
> +            ipport = '[%s]:%s' % (self.VM_IP, port)
> +            self.ssh_logger.debug('ipport ' + ipport)
> +            self.ssh_client.get_host_keys().add(ipport, 'ssh-rsa',
rsa_hostkey)
> +        for i in range(10):
> +            try:
> +                self.ssh_client.connect(self.VM_IP, int(port),
> +                                        username, password,
banner_timeout=90)
> +                self.ssh_logger.info("Entering interactive session.")
> +                return
> +            except:
> +                time.sleep(4)
> +                pass
> +        self.fail("sshd timeout")
> +
> +    def ssh_disconnect_vm(self):
> +        self.ssh_client.close()
> +
> +    def ssh_command(self, command, is_root=True):
> +        self.ssh_logger.info(self.ssh_ps1 + command)
> +        stdin, stdout, stderr = self.ssh_client.exec_command(command)
> +        stdout_lines = [line.strip('\n') for line in stdout]
> +        for line in stdout_lines:
> +            self.ssh_logger.info(line)
> +        stderr_lines = [line.strip('\n') for line in stderr]
> +        for line in stderr_lines:
> +            self.ssh_logger.warning(line)
> +        return stdout_lines, stderr_lines
> +
> +    def boot_debian_wheezy_image_and_ssh_login(self, endianess,
kernel_path):
> +        image_url = self.IMAGE_INFO[endianess]['image_url']
> +        image_hash = self.IMAGE_INFO[endianess]['image_hash']
> +        image_path = self.fetch_asset(image_url, asset_hash=image_hash)
> +        rsa_hostkey_b64 = self.IMAGE_INFO[endianess]['rsa_hostkey']
> +
> +        self.vm.set_machine('malta')
> +        self.vm.set_console()
> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> +                               + 'console=ttyS0 root=/dev/sda1')
> +        self.vm.add_args('-no-reboot',
> +                         '-kernel', kernel_path,
> +                         '-append', kernel_command_line,
> +                         '-hda', image_path,
> +                         '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
> +                         '-device', 'pcnet,netdev=vnet')
> +        self.vm.launch()
> +
> +        self.log.info('VM launched, waiting for sshd')
> +        console_pattern = 'Starting OpenBSD Secure Shell server: sshd'
> +        self.wait_for_console_pattern(console_pattern)
> +        self.log.info('sshd ready')
> +
> +        self.ssh_connect('root', 'root', rsa_hostkey_b64=rsa_hostkey_b64)
> +
> +    def shutdown_via_ssh(self):
> +        self.ssh_command('poweroff')
> +        self.ssh_disconnect_vm()
> +        self.wait_for_console_pattern('Power down')
> +
> +    def run_common_commands(self):
> +        stdout, stderr = self.ssh_command('lspci -d 11ab:4620')
> +        self.assertIn(True, ["GT-64120" in line for line in stdout])
> +
> +        stdout, stderr = self.ssh_command('cat
/sys/bus/i2c/devices/i2c-0/name')
> +        self.assertIn(True, ["SMBus PIIX4 adapter" in line
> +                             for line in stdout])
> +
> +        stdout, stderr = self.ssh_command('cat /proc/mtd')
> +        self.assertIn(True, ["YAMON" in line
> +                             for line in stdout])
> +
> +        # Empty 'Board Config'
> +        stdout, stderr = self.ssh_command('md5sum /dev/mtd2ro')
> +        self.assertIn(True, ["0dfbe8aa4c20b52e1b8bf3cb6cbdf193" in line
> +                             for line in stdout])
> +
> +    def do_test_mips_malta(self, endianess, kernel_path, uname_m):
> +        self.boot_debian_wheezy_image_and_ssh_login(endianess,
kernel_path)
> +
> +        stdout, stderr = self.ssh_command('uname -a')
> +        self.assertIn(True, [uname_m + " GNU/Linux" in line for line in
stdout])
> +
> +        self.run_common_commands()
> +        self.shutdown_via_ssh()
> +
> +    def test_mips_malta32eb_kernel3_2_0(self):
> +        """
> +        :avocado: tags=slow
> +        :avocado: tags=arch:mips
> +        :avocado: tags=machine:malta
> +        :avocado: tags=endian:big
> +        :avocado: tags=device:pcnet32
> +        """
> +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mips/'
> +                      'vmlinux-3.2.0-4-4kc-malta')
> +        kernel_hash = '592e384a4edc16dade52a6cd5c785c637bcbc9ad'
> +        kernel_path = self.fetch_asset(kernel_url,
asset_hash=kernel_hash)
> +
> +        self.do_test_mips_malta('be', kernel_path, 'mips')
> +
> +    def test_mips_malta32el_kernel3_2_0(self):
> +        """
> +        :avocado: tags=slow
> +        :avocado: tags=arch:mipsel
> +        :avocado: tags=machine:malta
> +        :avocado: tags=endian:little
> +        :avocado: tags=device:pcnet32
> +        """
> +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mipsel/'
> +                      'vmlinux-3.2.0-4-4kc-malta')
> +        kernel_hash = 'a66bea5a8adaa2cb3d36a1d4e0ccdb01be8f6c2a'
> +        kernel_path = self.fetch_asset(kernel_url,
asset_hash=kernel_hash)
> +
> +        self.do_test_mips_malta('le', kernel_path, 'mips')
> +
> +    def test_mips_malta64eb_kernel3_2_0(self):
> +        """
> +        :avocado: tags=slow
> +        :avocado: tags=arch:mips64
> +        :avocado: tags=machine:malta
> +        :avocado: tags=endian:big
> +        :avocado: tags=device:pcnet32
> +        """
> +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mips/'
> +                      'vmlinux-3.2.0-4-5kc-malta')
> +        kernel_hash = 'db6eea7de35d36c77d8c165b6bcb222e16eb91db'
> +        kernel_path = self.fetch_asset(kernel_url,
asset_hash=kernel_hash)
> +        self.do_test_mips_malta('be', kernel_path, 'mips64')
> +
> +    def test_mips_malta64el_kernel3_2_0(self):
> +        """
> +        :avocado: tags=slow
> +        :avocado: tags=arch:mips64el
> +        :avocado: tags=machine:malta
> +        :avocado: tags=endian:little
> +        :avocado: tags=device:pcnet32
> +        """
> +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mipsel/'
> +                      'vmlinux-3.2.0-4-5kc-malta')
> +        kernel_hash = '6a7f77245acf231415a0e8b725d91ed2f3487794'
> +        kernel_path = self.fetch_asset(kernel_url,
asset_hash=kernel_hash)
> +        self.do_test_mips_malta('le', kernel_path, 'mips64')
> diff --git a/tests/requirements.txt b/tests/requirements.txt
> index 002ded6a22..3ae0e29ad7 100644
> --- a/tests/requirements.txt
> +++ b/tests/requirements.txt
> @@ -2,3 +2,4 @@
>  # in the tests/venv Python virtual environment. For more info,
>  # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
>  avocado-framework==68.0
> +paramiko
> --
> 2.19.1
>

On Tue, May 21, 2019 at 10:18:10AM +0200, Aleksandar Markovic wrote:
> On May 21, 2019 1:19 AM, "Philippe Mathieu-Daudé" <f4bug@amsat.org> wrote:
> >
> > This tests boot a full VM and check the serial console until
> > the SSH daemon is running, then start a SSH session and run
> > some commands.
> >
> > This test can be run using:
> >
> >   $ avocado --show=ssh run -t arch:mips
> tests/acceptance/linux_ssh_mips_malta.py
> >   ssh: Entering interactive session.
> >   ssh: # uname -a
> >   ssh: Linux debian-mips 3.2.0-4-4kc-malta #1 Debian 3.2.51-1 mips
> GNU/Linux
> >   ssh: # lspci -d 11ab:4620
> >   ssh: 00:00.0 Host bridge: Marvell Technology Group Ltd.
> GT-64120/64120A/64121A System Controller (rev 10)
> >   ssh: # cat /sys/bus/i2c/devices/i2c-0/name
> >   ssh: SMBus PIIX4 adapter at 1100
> >   ssh: # cat /proc/mtd
> >   ssh: dev:    size   erasesize  name
> >   ssh: mtd0: 00100000 00010000 "YAMON"
> >   ssh: mtd1: 002e0000 00010000 "User FS"
> >   ssh: mtd2: 00020000 00010000 "Board Config"
> >   ssh: # md5sum /dev/mtd2ro
> >   ssh: 0dfbe8aa4c20b52e1b8bf3cb6cbdf193  /dev/mtd2ro
> >   ssh: # poweroff
> >
> > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> > ---
> 
> Excelent! Frankly, this was something that we in MIPS needed and missed
> very much for a long time.
> 
> I liked the idea that this test does not run as a default, giving us
> opportunity to extend and adjust it in future as we deem appropriate for
> our MIPS needs, but without affecting people running default test
> execution. In other words, this patch achieves “best of both worlds”.
> 

Actually, the test is being run by default, and is timing out on
travis-ci:

https://travis-ci.org/ehabkost/qemu/jobs/535468057#L3297

-- 
Eduardo

On May 21, 2019 10:18 AM, "Aleksandar Markovic" <aleksandar.m.mail@gmail.com>
wrote:
>
>
> On May 21, 2019 1:19 AM, "Philippe Mathieu-Daudé" <f4bug@amsat.org> wrote:
> >
> > This tests boot a full VM and check the serial console until
> > the SSH daemon is running, then start a SSH session and run
> > some commands.
> >

If there is no objection from Cleber or others, I would like to apply this
patch to my next MIPS pull request. We in Wave just started regular
regression tests of various nature for QEMU 4.1 for MIPS, and would really
like to have this test integrated as soon as possible, that is why I am
rushing a bit. I am talking here about this patch only, since it is a sort
of an independant unit - I will not do anything with other remaining
patches from this series.

Please let me know if you have any concerns.

Sincerely, Aleksandar

> > This test can be run using:
> >
> >   $ avocado --show=ssh run -t arch:mips
tests/acceptance/linux_ssh_mips_malta.py
> >   ssh: Entering interactive session.
> >   ssh: # uname -a
> >   ssh: Linux debian-mips 3.2.0-4-4kc-malta #1 Debian 3.2.51-1 mips
GNU/Linux
> >   ssh: # lspci -d 11ab:4620
> >   ssh: 00:00.0 Host bridge: Marvell Technology Group Ltd.
GT-64120/64120A/64121A System Controller (rev 10)
> >   ssh: # cat /sys/bus/i2c/devices/i2c-0/name
> >   ssh: SMBus PIIX4 adapter at 1100
> >   ssh: # cat /proc/mtd
> >   ssh: dev:    size   erasesize  name
> >   ssh: mtd0: 00100000 00010000 "YAMON"
> >   ssh: mtd1: 002e0000 00010000 "User FS"
> >   ssh: mtd2: 00020000 00010000 "Board Config"
> >   ssh: # md5sum /dev/mtd2ro
> >   ssh: 0dfbe8aa4c20b52e1b8bf3cb6cbdf193  /dev/mtd2ro
> >   ssh: # poweroff
> >
> > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> > ---
>
> Excelent! Frankly, this was something that we in MIPS needed and missed
very much for a long time.
>
> I liked the idea that this test does not run as a default, giving us
opportunity to extend and adjust it in future as we deem appropriate for
our MIPS needs, but without affecting people running default test
execution. In other words, this patch achieves “best of both worlds”.
>
> Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
>
> > TODO: do not run this tests by default, use the 'slow' tag
> > ---
> >  MAINTAINERS                              |   1 +
> >  tests/acceptance/linux_ssh_mips_malta.py | 229 +++++++++++++++++++++++
> >  tests/requirements.txt                   |   1 +
> >  3 files changed, 231 insertions(+)
> >  create mode 100644 tests/acceptance/linux_ssh_mips_malta.py
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 9424a490d6..69fa4b3abc 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -934,6 +934,7 @@ M: Aurelien Jarno <aurelien@aurel32.net>
> >  R: Aleksandar Rikalo <arikalo@wavecomp.com>
> >  S: Maintained
> >  F: hw/mips/mips_malta.c
> > +F: tests/acceptance/linux_ssh_mips_malta.py
> >
> >  Mipssim
> >  M: Aleksandar Markovic <amarkovic@wavecomp.com>
> > diff --git a/tests/acceptance/linux_ssh_mips_malta.py
b/tests/acceptance/linux_ssh_mips_malta.py
> > new file mode 100644
> > index 0000000000..ceb530ff88
> > --- /dev/null
> > +++ b/tests/acceptance/linux_ssh_mips_malta.py
> > @@ -0,0 +1,229 @@
> > +# Functional test that boots a VM and run commands via a SSH session
> > +#
> > +# Copyright (c) Philippe Mathieu-Daudé <f4bug@amsat.org>
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or
> > +# later.  See the COPYING file in the top-level directory.
> > +
> > +import os
> > +import re
> > +import base64
> > +import logging
> > +import paramiko
> > +import time
> > +
> > +from avocado_qemu import Test
> > +from avocado.utils import process
> > +from avocado.utils import archive
> > +
> > +
> > +class LinuxSSH(Test):
> > +
> > +    timeout = 150 # Not for 'configure --enable-debug
--enable-debug-tcg'
> > +
> > +    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
> > +    VM_IP = '127.0.0.1'
> > +
> > +    IMAGE_INFO = {
> > +        'be': {
> > +            'image_url': 'https://people.debian.org/~aurel32/qemu/mips/
'
> > +                         'debian_wheezy_mips_standard.qcow2',
> > +            'image_hash': '8987a63270df67345b2135a6b7a4885a35e392d5',
> > +            'rsa_hostkey':
b'AAAAB3NzaC1yc2EAAAADAQABAAABAQCca1VitiyLAdQOld'
> > +
 b'zT43IOEVJZ0wHD78GJi8wDAjMiYWUzNSSn0rXGQsINHuH5'
> > +
 b'IlF+kBZsHinb/FtKCAyS9a8uCHhQI4SuB4QhAb0+39MlUw'
> > +
 b'Mm0CLkctgM2eUUZ6MQMQvDlqnue6CCkxN62EZYbaxmby7j'
> > +
 b'CQa1125o1HRKBvdGm2zrJWxXAfA+f1v6jHLyE8Jnu83eQ+'
> > +
 b'BFY25G+Vzx1PVc3zQBwJ8r0NGTRqy2//oWQP0h+bMsgeFe'
> > +
 b'KH/J3RJM22vg6+I4JAdBFcxnK+l781h1FuRxOn4O/Xslbg'
> > +
 b'go6WtB4V4TOsw2E/KfxI5IZ/icxF+swVcnvF46Hf3uQc/0'
> > +                           b'BBqb',
> > +        },
> > +        'le': {
> > +            'image_url': '
https://people.debian.org/~aurel32/qemu/mipsel/'
> > +                         'debian_wheezy_mipsel_standard.qcow2',
> > +            'image_hash': '7866764d9de3ef536ffca24c9fb9f04ffdb45802',
> > +            'rsa_hostkey':
b'AAAAB3NzaC1yc2EAAAADAQABAAABAQClXJlBT71HL5yKvv'
> > +
 b'gfC7jmxSWx5zSBCzET6CLZczwAafSIs7YKfNOy/dQTxhuk'
> > +
 b'yIGFUugZFoF3E9PzdhunuyvyTd56MPoNIqFbb5rGokwU5I'
> > +
 b'TOx3dBHZR0mClypL6MVrwe0bsiIb8GhF1zioNwcsaAZnAi'
> > +
 b'KfXStVDtXvn/kLLq+xLABYt48CC5KYWoFaCoICskLAY+qo'
> > +
 b'L+LWyAnQisj4jAH8VSaSKIImFpfkHWEXPhHcC4ZBlDKtnH'
> > +
 b'po9vhfCHgnfW3Pzrqmk8BI4HysqPFVmJWkJGlGUL+sGeg3'
> > +
 b'ZZolAYuDXGuBrw8ooPJq2v2dOH+z6dyD2q/ypmAbyPqj5C'
> > +                           b'rc8H',
> > +        },
> > +    }
> > +
> > +    def wait_for_console_pattern(self, success_message,
> > +                                 failure_message='Oops'):
> > +        console = self.vm.console_socket.makefile()
> > +        console_logger = logging.getLogger('console')
> > +        while True:
> > +            msg = console.readline()
> > +            console_logger.debug(msg.strip())
> > +            if success_message in msg:
> > +                break
> > +            if failure_message in msg:
> > +                fail = 'Failure message found in console: %s' %
failure_message
> > +                self.fail(fail)
> > +
> > +    def get_portfwd(self):
> > +        res = self.vm.command('human-monitor-command',
> > +                              command_line='info usernet')
> > +        line = res.split('\r\n')[2]
> > +        port = re.split(r'.*TCP.HOST_FORWARD.*127\.0\.0\.1
(\d+)\s+10\..*',
> > +                        line)[1]
> > +        self.log.debug("sshd listening on port:" + port)
> > +        return port
> > +
> > +    def ssh_connect(self, username, password, rsa_hostkey_b64=None):
> > +        self.ssh_logger = logging.getLogger('ssh')
> > +        self.ssh_username = username
> > +        self.ssh_ps1 = '# ' if username is 'root' else '$ '
> > +        self.ssh_client = paramiko.SSHClient()
> > +        port = self.get_portfwd()
> > +        if rsa_hostkey_b64:
> > +            rsa_hostkey_bin = base64.b64decode(rsa_hostkey_b64)
> > +            rsa_hostkey = paramiko.RSAKey(data = rsa_hostkey_bin)
> > +            ipport = '[%s]:%s' % (self.VM_IP, port)
> > +            self.ssh_logger.debug('ipport ' + ipport)
> > +            self.ssh_client.get_host_keys().add(ipport, 'ssh-rsa',
rsa_hostkey)
> > +        for i in range(10):
> > +            try:
> > +                self.ssh_client.connect(self.VM_IP, int(port),
> > +                                        username, password,
banner_timeout=90)
> > +                self.ssh_logger.info("Entering interactive session.")
> > +                return
> > +            except:
> > +                time.sleep(4)
> > +                pass
> > +        self.fail("sshd timeout")
> > +
> > +    def ssh_disconnect_vm(self):
> > +        self.ssh_client.close()
> > +
> > +    def ssh_command(self, command, is_root=True):
> > +        self.ssh_logger.info(self.ssh_ps1 + command)
> > +        stdin, stdout, stderr = self.ssh_client.exec_command(command)
> > +        stdout_lines = [line.strip('\n') for line in stdout]
> > +        for line in stdout_lines:
> > +            self.ssh_logger.info(line)
> > +        stderr_lines = [line.strip('\n') for line in stderr]
> > +        for line in stderr_lines:
> > +            self.ssh_logger.warning(line)
> > +        return stdout_lines, stderr_lines
> > +
> > +    def boot_debian_wheezy_image_and_ssh_login(self, endianess,
kernel_path):
> > +        image_url = self.IMAGE_INFO[endianess]['image_url']
> > +        image_hash = self.IMAGE_INFO[endianess]['image_hash']
> > +        image_path = self.fetch_asset(image_url, asset_hash=image_hash)
> > +        rsa_hostkey_b64 = self.IMAGE_INFO[endianess]['rsa_hostkey']
> > +
> > +        self.vm.set_machine('malta')
> > +        self.vm.set_console()
> > +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> > +                               + 'console=ttyS0 root=/dev/sda1')
> > +        self.vm.add_args('-no-reboot',
> > +                         '-kernel', kernel_path,
> > +                         '-append', kernel_command_line,
> > +                         '-hda', image_path,
> > +                         '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
> > +                         '-device', 'pcnet,netdev=vnet')
> > +        self.vm.launch()
> > +
> > +        self.log.info('VM launched, waiting for sshd')
> > +        console_pattern = 'Starting OpenBSD Secure Shell server: sshd'
> > +        self.wait_for_console_pattern(console_pattern)
> > +        self.log.info('sshd ready')
> > +
> > +        self.ssh_connect('root', 'root',
rsa_hostkey_b64=rsa_hostkey_b64)
> > +
> > +    def shutdown_via_ssh(self):
> > +        self.ssh_command('poweroff')
> > +        self.ssh_disconnect_vm()
> > +        self.wait_for_console_pattern('Power down')
> > +
> > +    def run_common_commands(self):
> > +        stdout, stderr = self.ssh_command('lspci -d 11ab:4620')
> > +        self.assertIn(True, ["GT-64120" in line for line in stdout])
> > +
> > +        stdout, stderr = self.ssh_command('cat
/sys/bus/i2c/devices/i2c-0/name')
> > +        self.assertIn(True, ["SMBus PIIX4 adapter" in line
> > +                             for line in stdout])
> > +
> > +        stdout, stderr = self.ssh_command('cat /proc/mtd')
> > +        self.assertIn(True, ["YAMON" in line
> > +                             for line in stdout])
> > +
> > +        # Empty 'Board Config'
> > +        stdout, stderr = self.ssh_command('md5sum /dev/mtd2ro')
> > +        self.assertIn(True, ["0dfbe8aa4c20b52e1b8bf3cb6cbdf193" in line
> > +                             for line in stdout])
> > +
> > +    def do_test_mips_malta(self, endianess, kernel_path, uname_m):
> > +        self.boot_debian_wheezy_image_and_ssh_login(endianess,
kernel_path)
> > +
> > +        stdout, stderr = self.ssh_command('uname -a')
> > +        self.assertIn(True, [uname_m + " GNU/Linux" in line for line
in stdout])
> > +
> > +        self.run_common_commands()
> > +        self.shutdown_via_ssh()
> > +
> > +    def test_mips_malta32eb_kernel3_2_0(self):
> > +        """
> > +        :avocado: tags=slow
> > +        :avocado: tags=arch:mips
> > +        :avocado: tags=machine:malta
> > +        :avocado: tags=endian:big
> > +        :avocado: tags=device:pcnet32
> > +        """
> > +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mips/'
> > +                      'vmlinux-3.2.0-4-4kc-malta')
> > +        kernel_hash = '592e384a4edc16dade52a6cd5c785c637bcbc9ad'
> > +        kernel_path = self.fetch_asset(kernel_url,
asset_hash=kernel_hash)
> > +
> > +        self.do_test_mips_malta('be', kernel_path, 'mips')
> > +
> > +    def test_mips_malta32el_kernel3_2_0(self):
> > +        """
> > +        :avocado: tags=slow
> > +        :avocado: tags=arch:mipsel
> > +        :avocado: tags=machine:malta
> > +        :avocado: tags=endian:little
> > +        :avocado: tags=device:pcnet32
> > +        """
> > +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mipsel/'
> > +                      'vmlinux-3.2.0-4-4kc-malta')
> > +        kernel_hash = 'a66bea5a8adaa2cb3d36a1d4e0ccdb01be8f6c2a'
> > +        kernel_path = self.fetch_asset(kernel_url,
asset_hash=kernel_hash)
> > +
> > +        self.do_test_mips_malta('le', kernel_path, 'mips')
> > +
> > +    def test_mips_malta64eb_kernel3_2_0(self):
> > +        """
> > +        :avocado: tags=slow
> > +        :avocado: tags=arch:mips64
> > +        :avocado: tags=machine:malta
> > +        :avocado: tags=endian:big
> > +        :avocado: tags=device:pcnet32
> > +        """
> > +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mips/'
> > +                      'vmlinux-3.2.0-4-5kc-malta')
> > +        kernel_hash = 'db6eea7de35d36c77d8c165b6bcb222e16eb91db'
> > +        kernel_path = self.fetch_asset(kernel_url,
asset_hash=kernel_hash)
> > +        self.do_test_mips_malta('be', kernel_path, 'mips64')
> > +
> > +    def test_mips_malta64el_kernel3_2_0(self):
> > +        """
> > +        :avocado: tags=slow
> > +        :avocado: tags=arch:mips64el
> > +        :avocado: tags=machine:malta
> > +        :avocado: tags=endian:little
> > +        :avocado: tags=device:pcnet32
> > +        """
> > +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mipsel/'
> > +                      'vmlinux-3.2.0-4-5kc-malta')
> > +        kernel_hash = '6a7f77245acf231415a0e8b725d91ed2f3487794'
> > +        kernel_path = self.fetch_asset(kernel_url,
asset_hash=kernel_hash)
> > +        self.do_test_mips_malta('le', kernel_path, 'mips64')
> > diff --git a/tests/requirements.txt b/tests/requirements.txt
> > index 002ded6a22..3ae0e29ad7 100644
> > --- a/tests/requirements.txt
> > +++ b/tests/requirements.txt
> > @@ -2,3 +2,4 @@
> >  # in the tests/venv Python virtual environment. For more info,
> >  # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
> >  avocado-framework==68.0
> > +paramiko
> > --
> > 2.19.1
> >

Hi Aleksandar,

On 5/21/19 11:26 AM, Aleksandar Markovic wrote:
> On May 21, 2019 10:18 AM, "Aleksandar Markovic" <aleksandar.m.mail@gmail.com>
> wrote:
>>
>>
>> On May 21, 2019 1:19 AM, "Philippe Mathieu-Daudé" <f4bug@amsat.org> wrote:
>>>
>>> This tests boot a full VM and check the serial console until
>>> the SSH daemon is running, then start a SSH session and run
>>> some commands.
>>>
> 
> If there is no objection from Cleber or others, I would like to apply this
> patch to my next MIPS pull request. We in Wave just started regular
> regression tests of various nature for QEMU 4.1 for MIPS, and would really
> like to have this test integrated as soon as possible, that is why I am
> rushing a bit. I am talking here about this patch only, since it is a sort
> of an independant unit - I will not do anything with other remaining
> patches from this series.

Thanks for the quick review :)

> 
> Please let me know if you have any concerns.

I threw this as it because it is working enough for me, but I'm secretly
hoping Cleber would take it and improve it, because I don't feel
confident with Python, and he better know Avocado ;)
This is why I also split this out of the other BootLinuxConsole test.

Anyway if it doesn't break anything and is already useful, it can get
improved later.

> Sincerely, Aleksandar
> 
>>> This test can be run using:
>>>
>>>   $ avocado --show=ssh run -t arch:mips
> tests/acceptance/linux_ssh_mips_malta.py
>>>   ssh: Entering interactive session.
>>>   ssh: # uname -a
>>>   ssh: Linux debian-mips 3.2.0-4-4kc-malta #1 Debian 3.2.51-1 mips
> GNU/Linux
>>>   ssh: # lspci -d 11ab:4620
>>>   ssh: 00:00.0 Host bridge: Marvell Technology Group Ltd.
> GT-64120/64120A/64121A System Controller (rev 10)
>>>   ssh: # cat /sys/bus/i2c/devices/i2c-0/name
>>>   ssh: SMBus PIIX4 adapter at 1100
>>>   ssh: # cat /proc/mtd
>>>   ssh: dev:    size   erasesize  name
>>>   ssh: mtd0: 00100000 00010000 "YAMON"
>>>   ssh: mtd1: 002e0000 00010000 "User FS"
>>>   ssh: mtd2: 00020000 00010000 "Board Config"
>>>   ssh: # md5sum /dev/mtd2ro
>>>   ssh: 0dfbe8aa4c20b52e1b8bf3cb6cbdf193  /dev/mtd2ro
>>>   ssh: # poweroff

Aleksandar, do you think of other commands we should run? What commands
do you usually try?

I plan to later install more packages in the VM (in particular test
packages, FPU extensive, device tests and so on).

My rule about device testing is "if the device can be tested in a KVM
guest, don't test it on TCG guest" for obvious reason of speed.

>>>
>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>> ---
>>
>> Excelent! Frankly, this was something that we in MIPS needed and missed
> very much for a long time.
>>
>> I liked the idea that this test does not run as a default, giving us
> opportunity to extend and adjust it in future as we deem appropriate for
> our MIPS needs, but without affecting people running default test

Note, the tests have the "slow" avocado tag, but it is not used because
I'm not sure how to modify the 'check-acceptance' rule of
tests/Makefile.include but Cleber will probably fix this ;)

> execution. In other words, this patch achieves “best of both worlds”.
>>
>> Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
>>
>>> TODO: do not run this tests by default, use the 'slow' tag
>>> ---
>>>  MAINTAINERS                              |   1 +
>>>  tests/acceptance/linux_ssh_mips_malta.py | 229 +++++++++++++++++++++++
>>>  tests/requirements.txt                   |   1 +
>>>  3 files changed, 231 insertions(+)
>>>  create mode 100644 tests/acceptance/linux_ssh_mips_malta.py
>>>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 9424a490d6..69fa4b3abc 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -934,6 +934,7 @@ M: Aurelien Jarno <aurelien@aurel32.net>
>>>  R: Aleksandar Rikalo <arikalo@wavecomp.com>
>>>  S: Maintained
>>>  F: hw/mips/mips_malta.c
>>> +F: tests/acceptance/linux_ssh_mips_malta.py
>>>
>>>  Mipssim
>>>  M: Aleksandar Markovic <amarkovic@wavecomp.com>
>>> diff --git a/tests/acceptance/linux_ssh_mips_malta.py
> b/tests/acceptance/linux_ssh_mips_malta.py
>>> new file mode 100644
>>> index 0000000000..ceb530ff88
>>> --- /dev/null
>>> +++ b/tests/acceptance/linux_ssh_mips_malta.py
>>> @@ -0,0 +1,229 @@
>>> +# Functional test that boots a VM and run commands via a SSH session
>>> +#
>>> +# Copyright (c) Philippe Mathieu-Daudé <f4bug@amsat.org>
>>> +#
>>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>>> +# later.  See the COPYING file in the top-level directory.
>>> +
>>> +import os
>>> +import re
>>> +import base64
>>> +import logging
>>> +import paramiko
>>> +import time
>>> +
>>> +from avocado_qemu import Test
>>> +from avocado.utils import process
>>> +from avocado.utils import archive
>>> +
>>> +
>>> +class LinuxSSH(Test):
>>> +
>>> +    timeout = 150 # Not for 'configure --enable-debug
> --enable-debug-tcg'
>>> +
>>> +    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>>> +    VM_IP = '127.0.0.1'
>>> +
>>> +    IMAGE_INFO = {
>>> +        'be': {
>>> +            'image_url': 'https://people.debian.org/~aurel32/qemu/mips/
> '
>>> +                         'debian_wheezy_mips_standard.qcow2',
>>> +            'image_hash': '8987a63270df67345b2135a6b7a4885a35e392d5',
>>> +            'rsa_hostkey':
> b'AAAAB3NzaC1yc2EAAAADAQABAAABAQCca1VitiyLAdQOld'
>>> +
>  b'zT43IOEVJZ0wHD78GJi8wDAjMiYWUzNSSn0rXGQsINHuH5'
>>> +
>  b'IlF+kBZsHinb/FtKCAyS9a8uCHhQI4SuB4QhAb0+39MlUw'
>>> +
>  b'Mm0CLkctgM2eUUZ6MQMQvDlqnue6CCkxN62EZYbaxmby7j'
>>> +
>  b'CQa1125o1HRKBvdGm2zrJWxXAfA+f1v6jHLyE8Jnu83eQ+'
>>> +
>  b'BFY25G+Vzx1PVc3zQBwJ8r0NGTRqy2//oWQP0h+bMsgeFe'
>>> +
>  b'KH/J3RJM22vg6+I4JAdBFcxnK+l781h1FuRxOn4O/Xslbg'
>>> +
>  b'go6WtB4V4TOsw2E/KfxI5IZ/icxF+swVcnvF46Hf3uQc/0'
>>> +                           b'BBqb',
>>> +        },
>>> +        'le': {
>>> +            'image_url': '
> https://people.debian.org/~aurel32/qemu/mipsel/'
>>> +                         'debian_wheezy_mipsel_standard.qcow2',
>>> +            'image_hash': '7866764d9de3ef536ffca24c9fb9f04ffdb45802',
>>> +            'rsa_hostkey':
> b'AAAAB3NzaC1yc2EAAAADAQABAAABAQClXJlBT71HL5yKvv'
>>> +
>  b'gfC7jmxSWx5zSBCzET6CLZczwAafSIs7YKfNOy/dQTxhuk'
>>> +
>  b'yIGFUugZFoF3E9PzdhunuyvyTd56MPoNIqFbb5rGokwU5I'
>>> +
>  b'TOx3dBHZR0mClypL6MVrwe0bsiIb8GhF1zioNwcsaAZnAi'
>>> +
>  b'KfXStVDtXvn/kLLq+xLABYt48CC5KYWoFaCoICskLAY+qo'
>>> +
>  b'L+LWyAnQisj4jAH8VSaSKIImFpfkHWEXPhHcC4ZBlDKtnH'
>>> +
>  b'po9vhfCHgnfW3Pzrqmk8BI4HysqPFVmJWkJGlGUL+sGeg3'
>>> +
>  b'ZZolAYuDXGuBrw8ooPJq2v2dOH+z6dyD2q/ypmAbyPqj5C'
>>> +                           b'rc8H',
>>> +        },
>>> +    }
>>> +
>>> +    def wait_for_console_pattern(self, success_message,
>>> +                                 failure_message='Oops'):

Reviewing myself:

This method might go in a generic class to handle buffered IO with QEMU
chardev.

>>> +        console = self.vm.console_socket.makefile()
>>> +        console_logger = logging.getLogger('console')
>>> +        while True:
>>> +            msg = console.readline()
>>> +            console_logger.debug(msg.strip())
>>> +            if success_message in msg:
>>> +                break
>>> +            if failure_message in msg:
>>> +                fail = 'Failure message found in console: %s' %
> failure_message
>>> +                self.fail(fail)
>>> +
>>> +    def get_portfwd(self):
>>> +        res = self.vm.command('human-monitor-command',
>>> +                              command_line='info usernet')
>>> +        line = res.split('\r\n')[2]
>>> +        port = re.split(r'.*TCP.HOST_FORWARD.*127\.0\.0\.1
> (\d+)\s+10\..*',
>>> +                        line)[1]
>>> +        self.log.debug("sshd listening on port:" + port)
>>> +        return port
>>> +
>>> +    def ssh_connect(self, username, password, rsa_hostkey_b64=None):
>>> +        self.ssh_logger = logging.getLogger('ssh')
>>> +        self.ssh_username = username
>>> +        self.ssh_ps1 = '# ' if username is 'root' else '$ '
>>> +        self.ssh_client = paramiko.SSHClient()
>>> +        port = self.get_portfwd()
>>> +        if rsa_hostkey_b64:
>>> +            rsa_hostkey_bin = base64.b64decode(rsa_hostkey_b64)
>>> +            rsa_hostkey = paramiko.RSAKey(data = rsa_hostkey_bin)
>>> +            ipport = '[%s]:%s' % (self.VM_IP, port)
>>> +            self.ssh_logger.debug('ipport ' + ipport)
>>> +            self.ssh_client.get_host_keys().add(ipport, 'ssh-rsa',
> rsa_hostkey)
>>> +        for i in range(10):
>>> +            try:
>>> +                self.ssh_client.connect(self.VM_IP, int(port),
>>> +                                        username, password,
> banner_timeout=90)
>>> +                self.ssh_logger.info("Entering interactive session.")
>>> +                return
>>> +            except:
>>> +                time.sleep(4)
>>> +                pass
>>> +        self.fail("sshd timeout")
>>> +
>>> +    def ssh_disconnect_vm(self):
>>> +        self.ssh_client.close()
>>> +
>>> +    def ssh_command(self, command, is_root=True):
>>> +        self.ssh_logger.info(self.ssh_ps1 + command)
>>> +        stdin, stdout, stderr = self.ssh_client.exec_command(command)
>>> +        stdout_lines = [line.strip('\n') for line in stdout]
>>> +        for line in stdout_lines:
>>> +            self.ssh_logger.info(line)
>>> +        stderr_lines = [line.strip('\n') for line in stderr]
>>> +        for line in stderr_lines:
>>> +            self.ssh_logger.warning(line)
>>> +        return stdout_lines, stderr_lines
>>> +
>>> +    def boot_debian_wheezy_image_and_ssh_login(self, endianess,
> kernel_path):
>>> +        image_url = self.IMAGE_INFO[endianess]['image_url']
>>> +        image_hash = self.IMAGE_INFO[endianess]['image_hash']
>>> +        image_path = self.fetch_asset(image_url, asset_hash=image_hash)
>>> +        rsa_hostkey_b64 = self.IMAGE_INFO[endianess]['rsa_hostkey']
>>> +
>>> +        self.vm.set_machine('malta')
>>> +        self.vm.set_console()
>>> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
>>> +                               + 'console=ttyS0 root=/dev/sda1')
>>> +        self.vm.add_args('-no-reboot',
>>> +                         '-kernel', kernel_path,
>>> +                         '-append', kernel_command_line,
>>> +                         '-hda', image_path,

We should probably use 'snapshot=on' here.

>>> +                         '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
> -:22',
>>> +                         '-device', 'pcnet,netdev=vnet')
>>> +        self.vm.launch()
>>> +
>>> +        self.log.info('VM launched, waiting for sshd')
>>> +        console_pattern = 'Starting OpenBSD Secure Shell server: sshd'
>>> +        self.wait_for_console_pattern(console_pattern)
>>> +        self.log.info('sshd ready')
>>> +
>>> +        self.ssh_connect('root', 'root',
> rsa_hostkey_b64=rsa_hostkey_b64)
>>> +
>>> +    def shutdown_via_ssh(self):
>>> +        self.ssh_command('poweroff')
>>> +        self.ssh_disconnect_vm()
>>> +        self.wait_for_console_pattern('Power down')
>>> +
>>> +    def run_common_commands(self):
>>> +        stdout, stderr = self.ssh_command('lspci -d 11ab:4620')
>>> +        self.assertIn(True, ["GT-64120" in line for line in stdout])
>>> +
>>> +        stdout, stderr = self.ssh_command('cat
> /sys/bus/i2c/devices/i2c-0/name')
>>> +        self.assertIn(True, ["SMBus PIIX4 adapter" in line
>>> +                             for line in stdout])
>>> +
>>> +        stdout, stderr = self.ssh_command('cat /proc/mtd')
>>> +        self.assertIn(True, ["YAMON" in line
>>> +                             for line in stdout])
>>> +
>>> +        # Empty 'Board Config'
>>> +        stdout, stderr = self.ssh_command('md5sum /dev/mtd2ro')
>>> +        self.assertIn(True, ["0dfbe8aa4c20b52e1b8bf3cb6cbdf193" in line
>>> +                             for line in stdout])
>>> +
>>> +    def do_test_mips_malta(self, endianess, kernel_path, uname_m):
>>> +        self.boot_debian_wheezy_image_and_ssh_login(endianess,
> kernel_path)
>>> +
>>> +        stdout, stderr = self.ssh_command('uname -a')
>>> +        self.assertIn(True, [uname_m + " GNU/Linux" in line for line
> in stdout])
>>> +
>>> +        self.run_common_commands()
>>> +        self.shutdown_via_ssh()
>>> +
>>> +    def test_mips_malta32eb_kernel3_2_0(self):
>>> +        """
>>> +        :avocado: tags=slow
>>> +        :avocado: tags=arch:mips
>>> +        :avocado: tags=machine:malta
>>> +        :avocado: tags=endian:big
>>> +        :avocado: tags=device:pcnet32
>>> +        """
>>> +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mips/'
>>> +                      'vmlinux-3.2.0-4-4kc-malta')
>>> +        kernel_hash = '592e384a4edc16dade52a6cd5c785c637bcbc9ad'
>>> +        kernel_path = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
>>> +
>>> +        self.do_test_mips_malta('be', kernel_path, 'mips')
>>> +
>>> +    def test_mips_malta32el_kernel3_2_0(self):
>>> +        """
>>> +        :avocado: tags=slow
>>> +        :avocado: tags=arch:mipsel
>>> +        :avocado: tags=machine:malta
>>> +        :avocado: tags=endian:little
>>> +        :avocado: tags=device:pcnet32
>>> +        """
>>> +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mipsel/'
>>> +                      'vmlinux-3.2.0-4-4kc-malta')
>>> +        kernel_hash = 'a66bea5a8adaa2cb3d36a1d4e0ccdb01be8f6c2a'
>>> +        kernel_path = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
>>> +
>>> +        self.do_test_mips_malta('le', kernel_path, 'mips')
>>> +
>>> +    def test_mips_malta64eb_kernel3_2_0(self):
>>> +        """
>>> +        :avocado: tags=slow
>>> +        :avocado: tags=arch:mips64
>>> +        :avocado: tags=machine:malta
>>> +        :avocado: tags=endian:big
>>> +        :avocado: tags=device:pcnet32
>>> +        """
>>> +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mips/'
>>> +                      'vmlinux-3.2.0-4-5kc-malta')
>>> +        kernel_hash = 'db6eea7de35d36c77d8c165b6bcb222e16eb91db'
>>> +        kernel_path = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
>>> +        self.do_test_mips_malta('be', kernel_path, 'mips64')
>>> +
>>> +    def test_mips_malta64el_kernel3_2_0(self):
>>> +        """
>>> +        :avocado: tags=slow
>>> +        :avocado: tags=arch:mips64el
>>> +        :avocado: tags=machine:malta
>>> +        :avocado: tags=endian:little
>>> +        :avocado: tags=device:pcnet32
>>> +        """
>>> +        kernel_url = ('https://people.debian.org/~aurel32/qemu/mipsel/'
>>> +                      'vmlinux-3.2.0-4-5kc-malta')
>>> +        kernel_hash = '6a7f77245acf231415a0e8b725d91ed2f3487794'
>>> +        kernel_path = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
>>> +        self.do_test_mips_malta('le', kernel_path, 'mips64')
>>> diff --git a/tests/requirements.txt b/tests/requirements.txt
>>> index 002ded6a22..3ae0e29ad7 100644
>>> --- a/tests/requirements.txt
>>> +++ b/tests/requirements.txt
>>> @@ -2,3 +2,4 @@
>>>  # in the tests/venv Python virtual environment. For more info,
>>>  # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
>>>  avocado-framework==68.0
>>> +paramiko
>>> --
>>> 2.19.1
>>>

> >>> This test can be run using:
> >>>
> >>>   $ avocado --show=ssh run -t arch:mips
> > tests/acceptance/linux_ssh_mips_malta.py
> >>>   ssh: Entering interactive session.
> >>>   ssh: # uname -a
> >>>   ssh: Linux debian-mips 3.2.0-4-4kc-malta #1 Debian 3.2.51-1 mips
> > GNU/Linux
> >>>   ssh: # lspci -d 11ab:4620
> >>>   ssh: 00:00.0 Host bridge: Marvell Technology Group Ltd.
> > GT-64120/64120A/64121A System Controller (rev 10)
> >>>   ssh: # cat /sys/bus/i2c/devices/i2c-0/name
> >>>   ssh: SMBus PIIX4 adapter at 1100
> >>>   ssh: # cat /proc/mtd
> >>>   ssh: dev:    size   erasesize  name
> >>>   ssh: mtd0: 00100000 00010000 "YAMON"
> >>>   ssh: mtd1: 002e0000 00010000 "User FS"
> >>>   ssh: mtd2: 00020000 00010000 "Board Config"
> >>>   ssh: # md5sum /dev/mtd2ro
> >>>   ssh: 0dfbe8aa4c20b52e1b8bf3cb6cbdf193  /dev/mtd2ro
> >>>   ssh: # poweroff
>
> Aleksandar, do you think of other commands we should run? What commands
> do you usually try?
>

Philippe, I will leave to you the choice of commands. You know that I
have a very high opinion on this test that you designed and also
implemented, I am more than happy even without any additions. But,
by no means, add something whenever you think it is interesting
and useful.

However, I want to bring to your attention something that could also
be of tremendous importance: Can we try communicating with the
newly booted machine via gdb? And do some simple commands,
like load a binary, set a breakpoint, display register content, etc?

Merci mille fois.

Aleksandar

> I plan to later install more packages in the VM (in particular test
> packages, FPU extensive, device tests and so on).
>
> My rule about device testing is "if the device can be tested in a KVM
> guest, don't test it on TCG guest" for obvious reason of speed.

On 6/3/19 3:41 PM, Aleksandar Markovic wrote:
>>>>> This test can be run using:
>>>>>
>>>>>   $ avocado --show=ssh run -t arch:mips
>>> tests/acceptance/linux_ssh_mips_malta.py
>>>>>   ssh: Entering interactive session.
>>>>>   ssh: # uname -a
>>>>>   ssh: Linux debian-mips 3.2.0-4-4kc-malta #1 Debian 3.2.51-1 mips
>>> GNU/Linux
>>>>>   ssh: # lspci -d 11ab:4620
>>>>>   ssh: 00:00.0 Host bridge: Marvell Technology Group Ltd.
>>> GT-64120/64120A/64121A System Controller (rev 10)
>>>>>   ssh: # cat /sys/bus/i2c/devices/i2c-0/name
>>>>>   ssh: SMBus PIIX4 adapter at 1100
>>>>>   ssh: # cat /proc/mtd
>>>>>   ssh: dev:    size   erasesize  name
>>>>>   ssh: mtd0: 00100000 00010000 "YAMON"
>>>>>   ssh: mtd1: 002e0000 00010000 "User FS"
>>>>>   ssh: mtd2: 00020000 00010000 "Board Config"
>>>>>   ssh: # md5sum /dev/mtd2ro
>>>>>   ssh: 0dfbe8aa4c20b52e1b8bf3cb6cbdf193  /dev/mtd2ro
>>>>>   ssh: # poweroff
>>
>> Aleksandar, do you think of other commands we should run? What commands
>> do you usually try?
>>
> 
> Philippe, I will leave to you the choice of commands. You know that I
> have a very high opinion on this test that you designed and also
> implemented, I am more than happy even without any additions. But,
> by no means, add something whenever you think it is interesting
> and useful.
> 
> However, I want to bring to your attention something that could also
> be of tremendous importance: Can we try communicating with the
> newly booted machine via gdb? And do some simple commands,
> like load a binary, set a breakpoint, display register content, etc?

Yes! This is my next step, I'm working on some draft series. It might
takes a while. My idea is to have some prototype working, then send as
RFC and ask for help. I'll Cc you then.

Regards,

Phil.

On Tue, May 21, 2019 at 11:26:35AM +0200, Aleksandar Markovic wrote:
> On May 21, 2019 10:18 AM, "Aleksandar Markovic" <aleksandar.m.mail@gmail.com>
> wrote:
> >
> >
> > On May 21, 2019 1:19 AM, "Philippe Mathieu-Daudé" <f4bug@amsat.org> wrote:
> > >
> > > This tests boot a full VM and check the serial console until
> > > the SSH daemon is running, then start a SSH session and run
> > > some commands.
> > >
> 
> If there is no objection from Cleber or others, I would like to apply this
> patch to my next MIPS pull request. We in Wave just started regular
> regression tests of various nature for QEMU 4.1 for MIPS, and would really
> like to have this test integrated as soon as possible, that is why I am
> rushing a bit. I am talking here about this patch only, since it is a sort
> of an independant unit - I will not do anything with other remaining
> patches from this series.
> 
> Please let me know if you have any concerns.

No problem to me.  I don't expect any merge conflicts caused by
the same patch being merged through both trees.

Acked-by: Eduardo Habkost <ehabkost@redhat.com>

-- 
Eduardo