1. Patchew
  2. QEMU
  3. View series

[PATCH] tests/vm: Add Haiku test based on their vagrant images

Alexander von Gluck IV posted 1 patch 3 years, 7 months ago
Test docker-quick@centos7 failed
Test docker-mingw@fedora failed
Test checkpatch failed
Test FreeBSD failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200906143439.1924930-1-kallisti5@unixzen.com
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <philmd@redhat.com>, Fam Zheng <fam@euphon.net>
tests/keys/vagrant     |  27 +++++++++
tests/keys/vagrant.pub |   1 +
tests/vm/basevm.py     |   5 +-
tests/vm/haiku.x86_64  | 121 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 152 insertions(+), 2 deletions(-)
create mode 100644 tests/keys/vagrant
create mode 100644 tests/keys/vagrant.pub
create mode 100755 tests/vm/haiku.x86_64
[PATCH] tests/vm: Add Haiku test based on their vagrant images
Posted by Alexander von Gluck IV 3 years, 7 months ago 
Signed-off-by: Alexander von Gluck IV <kallisti5@unixzen.com>
---
 tests/keys/vagrant     |  27 +++++++++
 tests/keys/vagrant.pub |   1 +
 tests/vm/basevm.py     |   5 +-
 tests/vm/haiku.x86_64  | 121 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 152 insertions(+), 2 deletions(-)
 create mode 100644 tests/keys/vagrant
 create mode 100644 tests/keys/vagrant.pub
 create mode 100755 tests/vm/haiku.x86_64

diff --git a/tests/keys/vagrant b/tests/keys/vagrant
new file mode 100644
index 0000000000..7d6a083909
--- /dev/null
+++ b/tests/keys/vagrant
@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEogIBAAKCAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzI
+w+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoP
+kcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2
+hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NO
+Td0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcW
+yLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQIBIwKCAQEA4iqWPJXtzZA68mKd
+ELs4jJsdyky+ewdZeNds5tjcnHU5zUYE25K+ffJED9qUWICcLZDc81TGWjHyAqD1
+Bw7XpgUwFgeUJwUlzQurAv+/ySnxiwuaGJfhFM1CaQHzfXphgVml+fZUvnJUTvzf
+TK2Lg6EdbUE9TarUlBf/xPfuEhMSlIE5keb/Zz3/LUlRg8yDqz5w+QWVJ4utnKnK
+iqwZN0mwpwU7YSyJhlT4YV1F3n4YjLswM5wJs2oqm0jssQu/BT0tyEXNDYBLEF4A
+sClaWuSJ2kjq7KhrrYXzagqhnSei9ODYFShJu8UWVec3Ihb5ZXlzO6vdNQ1J9Xsf
+4m+2ywKBgQD6qFxx/Rv9CNN96l/4rb14HKirC2o/orApiHmHDsURs5rUKDx0f9iP
+cXN7S1uePXuJRK/5hsubaOCx3Owd2u9gD6Oq0CsMkE4CUSiJcYrMANtx54cGH7Rk
+EjFZxK8xAv1ldELEyxrFqkbE4BKd8QOt414qjvTGyAK+OLD3M2QdCQKBgQDtx8pN
+CAxR7yhHbIWT1AH66+XWN8bXq7l3RO/ukeaci98JfkbkxURZhtxV/HHuvUhnPLdX
+3TwygPBYZFNo4pzVEhzWoTtnEtrFueKxyc3+LjZpuo+mBlQ6ORtfgkr9gBVphXZG
+YEzkCD3lVdl8L4cw9BVpKrJCs1c5taGjDgdInQKBgHm/fVvv96bJxc9x1tffXAcj
+3OVdUN0UgXNCSaf/3A/phbeBQe9xS+3mpc4r6qvx+iy69mNBeNZ0xOitIjpjBo2+
+dBEjSBwLk5q5tJqHmy/jKMJL4n9ROlx93XS+njxgibTvU6Fp9w+NOFD/HvxB3Tcz
+6+jJF85D5BNAG3DBMKBjAoGBAOAxZvgsKN+JuENXsST7F89Tck2iTcQIT8g5rwWC
+P9Vt74yboe2kDT531w8+egz7nAmRBKNM751U/95P9t88EDacDI/Z2OwnuFQHCPDF
+llYOUI+SpLJ6/vURRbHSnnn8a/XG+nzedGH5JGqEJNQsz+xT2axM0/W/CRknmGaJ
+kda/AoGANWrLCz708y7VYgAtW2Uf1DPOIYMdvo6fxIB5i9ZfISgcJ/bbCUkFrhoH
++vq/5CIWxCPp0f85R4qxxQ5ihxJ0YDQT9Jpx4TMss4PSavPaBH3RXow5Ohe+bYoQ
+NE5OgEXk2wVfZczCZpigBKbKZHNYcelXtTt/nP3rsCuGcM4h53s=
+-----END RSA PRIVATE KEY-----
diff --git a/tests/keys/vagrant.pub b/tests/keys/vagrant.pub
new file mode 100644
index 0000000000..18a9c00fd5
--- /dev/null
+++ b/tests/keys/vagrant.pub
@@ -0,0 +1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 3fac20e929..00f1d5ca8d 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -44,6 +44,7 @@ DEFAULT_CONFIG = {
     'machine'         : 'pc',
     'guest_user'      : "qemu",
     'guest_pass'      : "qemupass",
+    'root_user'       : "root",
     'root_pass'       : "qemupass",
     'ssh_key_file'    : SSH_KEY_FILE,
     'ssh_pub_key_file': SSH_PUB_KEY_FILE,
@@ -245,13 +246,13 @@ class BaseVM(object):
         return self._ssh_do(self._config["guest_user"], cmd, False)
 
     def ssh_root(self, *cmd):
-        return self._ssh_do("root", cmd, False)
+        return self._ssh_do(self._config["root_user"], cmd, False)
 
     def ssh_check(self, *cmd):
         self._ssh_do(self._config["guest_user"], cmd, True)
 
     def ssh_root_check(self, *cmd):
-        self._ssh_do("root", cmd, True)
+        self._ssh_do(self._config["root_user"], cmd, True)
 
     def build_image(self, img):
         raise NotImplementedError
diff --git a/tests/vm/haiku.x86_64 b/tests/vm/haiku.x86_64
new file mode 100755
index 0000000000..9777722f51
--- /dev/null
+++ b/tests/vm/haiku.x86_64
@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+#
+# Haiku VM image
+#
+# Copyright 2020 Haiku, Inc.
+#
+# Authors:
+#  Alexander von Gluck IV <kallisti5@unixzen.com>
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import re
+import sys
+import time
+import socket
+import subprocess
+import basevm
+
+VAGRANT_KEY_FILE = os.path.join(os.path.dirname(__file__),
+    "..", "keys", "vagrant")
+
+VAGRANT_PUB_KEY_FILE = os.path.join(os.path.dirname(__file__),
+    "..", "keys", "vagrant.pub")
+
+HAIKU_CONFIG = {
+    'cpu'             : "max",
+    'machine'         : 'pc',
+    'guest_user'      : "vagrant",
+    'guest_pass'      : "",
+    'root_user'       : "vagrant",
+    'root_pass'       : "",
+    'ssh_key_file'    : VAGRANT_KEY_FILE,
+    'ssh_pub_key_file': VAGRANT_PUB_KEY_FILE,
+    'memory'          : "4G",
+    'extra_args'      : [],
+    'qemu_args'       : "-device VGA",
+    'dns'             : "",
+    'ssh_port'        : 0,
+    'install_cmds'    : "",
+    'boot_dev_type'   : "block",
+    'ssh_timeout'     : 1,
+}
+
+class HaikuVM(basevm.BaseVM):
+    name = "haiku"
+    arch = "x86_64"
+
+    link = "https://app.vagrantup.com/haiku-os/boxes/r1beta2-x86_64/versions/20200702/providers/libvirt.box"
+    csum = "41c38b316e0cbdbc66b5dbaf3612b866700a4f35807cb1eb266a5bf83e9e68d5"
+
+    poweroff = "shutdown"
+
+    requirements = [
+        "devel:libbz2",
+        "devel:libcapstone",
+        "devel:libcurl",
+        "devel:libfdt",
+        "devel:libgcrypt",
+        "devel:libgl",
+        "devel:libglib_2.0",
+        "devel:libgnutls",
+        "devel:libgpg_error",
+        "devel:libintl",
+        "devel:libjpeg",
+        "devel:liblzo2",
+        "devel:libncursesw",
+        "devel:libnettle",
+        "devel:libpixman_1",
+        "devel:libpng16",
+        "devel:libsdl2_2.0",
+        "devel:libsnappy",
+        "devel:libssh2",
+        "devel:libtasn1",
+        "devel:libusb_1.0",
+        "devel:libz",
+        "setuptools_python3"
+    ]
+
+    # https://dev.haiku-os.org/ticket/16512 virtio disk1 shows up as 0 (reversed order)
+    BUILD_SCRIPT = """
+        set -e;
+        rm -rf /tmp/qemu-test.*
+        cd $(mktemp -d /tmp/qemu-test.XXXXXX);
+        mkdir src build; cd src;
+        tar -xf /dev/disk/virtual/virtio_block/0/raw;
+        cd ../build
+        ../src/configure --python=python3 {configure_opts};
+        make --output-sync -j{jobs} {target} {verbose};
+    """
+
+    def build_image(self, img):
+        self.print_step("Downloading disk image")
+        cvg = self._download_with_cache(self.link, sha256sum=self.csum)
+        cgz = cvg + ".tar.gz"
+        img_tmp = "./box.img"
+
+        self.print_step("Preparing disk image")
+
+        subprocess.check_call(["cp", "-f", cvg, cgz])
+        subprocess.check_call(["tar", "xvzf", cgz, img_tmp])
+        subprocess.check_call(["chmod", "666", img_tmp])
+
+        self.boot(img_tmp)
+
+        # Wait for ssh to be available.
+        self.wait_ssh(wait_root=True, cmd="exit 0")
+
+        # Install packages
+        self.ssh_root("pkgman install -y %s" % " ".join(self.requirements))
+        self.graceful_shutdown()
+
+        if os.path.exists(cgz):
+            os.remove(cgz)
+        subprocess.check_call(["mv", img_tmp, img])
+        self.print_step("All done")
+
+if __name__ == "__main__":
+    sys.exit(basevm.main(HaikuVM, config=HAIKU_CONFIG))
-- 
2.26.2
Re: [PATCH] tests/vm: Add Haiku test based on their vagrant images
Posted by Alexander von Gluck IV 3 years, 7 months ago 
September 6, 2020 9:35 AM, "Alexander von Gluck IV" <kallisti5@unixzen.com> wrote:
> Signed-off-by: Alexander von Gluck IV <kallisti5@unixzen.com>
> ---
> tests/keys/vagrant | 27 +++++++++
> tests/keys/vagrant.pub | 1 +
> tests/vm/basevm.py | 5 +-
> tests/vm/haiku.x86_64 | 121 +++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 152 insertions(+), 2 deletions(-)
> create mode 100644 tests/keys/vagrant
> create mode 100644 tests/keys/vagrant.pub
> create mode 100755 tests/vm/haiku.x86_64
> 
> diff --git a/tests/keys/vagrant b/tests/keys/vagrant
> new file mode 100644
> index 0000000000..7d6a083909
> --- /dev/null
> +++ b/tests/keys/vagrant
> @@ -0,0 +1,27 @@
> +-----BEGIN RSA PRIVATE KEY-----
> +MIIEogIBAAKCAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzI
>
> diff --git a/tests/keys/vagrant.pub b/tests/keys/vagrant.pub
> new file mode 100644
> index 0000000000..18a9c00fd5
> --- /dev/null
> +++ b/tests/keys/vagrant.pub
> @@ -0,0 +1 @@
> +ssh-rsa
> AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oX

A little background information for context. These are the Vagrant SSH keys which are packed with every vagrant OS image and allow OS access for automation.  The python vm tester knowing of these lets it leverage Vagrant OS images for testing without much work.



> --- a/tests/vm/basevm.py
> +++ b/tests/vm/basevm.py
> @@ -44,6 +44,7 @@ DEFAULT_CONFIG = {
> 'machine' : 'pc',
> 'guest_user' : "qemu",
> 'guest_pass' : "qemupass",
> + 'root_user' : "root",
> 'root_pass' : "qemupass",
> 'ssh_key_file' : SSH_KEY_FILE,
> 'ssh_pub_key_file': SSH_PUB_KEY_FILE,
> @@ -245,13 +246,13 @@ class BaseVM(object):
> return self._ssh_do(self._config["guest_user"], cmd, False)
> 
> def ssh_root(self, *cmd):
> - return self._ssh_do("root", cmd, False)
> + return self._ssh_do(self._config["root_user"], cmd, False)
> 
> def ssh_check(self, *cmd):
> self._ssh_do(self._config["guest_user"], cmd, True)
> 
> def ssh_root_check(self, *cmd):
> - self._ssh_do("root", cmd, True)
> + self._ssh_do(self._config["root_user"], cmd, True)
> 
> def build_image(self, img):
> raise NotImplementedError


Haiku's user is UID 0, so essentially our root user isn't named root.
This adds the (optional) ability to override the root username.


> diff --git a/tests/vm/haiku.x86_64 b/tests/vm/haiku.x86_64
> new file mode 100755
> index 0000000000..9777722f51
> --- /dev/null
> +++ b/tests/vm/haiku.x86_64
> @@ -0,0 +1,121 @@
> +#!/usr/bin/env python3
> +#
> +# Haiku VM image
> +#
> +# Copyright 2020 Haiku, Inc.
> +#
> +# Authors:
> +# Alexander von Gluck IV <kallisti5@unixzen.com>
> +#
> +# This code is licensed under the GPL version 2 or later. See
> +# the COPYING file in the top-level directory.
> +#


This build script works as expected, transferring the qemu archive over
via the virtio block device and building it.

More information here (including output of tools):
https://bugs.launchpad.net/qemu/+bug/1715203

This purpose of this is trying to prevent the need to remove
upstream qemu support for Haiku.

We have some out-of-tree patches to fix the error seen in our ports, i'll
work on upstreaming these.


 -- Alex
Re: [PATCH] tests/vm: Add Haiku test based on their vagrant images
Posted by Philippe Mathieu-Daudé 3 years, 7 months ago 
Hi Alexander,

On 9/6/20 4:41 PM, Alexander von Gluck IV wrote:
> September 6, 2020 9:35 AM, "Alexander von Gluck IV" <kallisti5@unixzen.com> wrote:
>> Signed-off-by: Alexander von Gluck IV <kallisti5@unixzen.com>
>> ---
>> tests/keys/vagrant | 27 +++++++++
>> tests/keys/vagrant.pub | 1 +
>> tests/vm/basevm.py | 5 +-
>> tests/vm/haiku.x86_64 | 121 +++++++++++++++++++++++++++++++++++++++++
>> 4 files changed, 152 insertions(+), 2 deletions(-)
>> create mode 100644 tests/keys/vagrant
>> create mode 100644 tests/keys/vagrant.pub
>> create mode 100755 tests/vm/haiku.x86_64
>>
>> diff --git a/tests/keys/vagrant b/tests/keys/vagrant
>> new file mode 100644
>> index 0000000000..7d6a083909
>> --- /dev/null
>> +++ b/tests/keys/vagrant
>> @@ -0,0 +1,27 @@
>> +-----BEGIN RSA PRIVATE KEY-----
>> +MIIEogIBAAKCAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzI
>>
>> diff --git a/tests/keys/vagrant.pub b/tests/keys/vagrant.pub
>> new file mode 100644
>> index 0000000000..18a9c00fd5
>> --- /dev/null
>> +++ b/tests/keys/vagrant.pub
>> @@ -0,0 +1 @@
>> +ssh-rsa
>> AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oX
> 
> A little background information for context. These are the Vagrant SSH keys which are packed with every vagrant OS image and allow OS access for automation.  The python vm tester knowing of these lets it leverage Vagrant OS images for testing without much work.

Please add this information as comment in the source files.

> 
> 
> 
>> --- a/tests/vm/basevm.py
>> +++ b/tests/vm/basevm.py
>> @@ -44,6 +44,7 @@ DEFAULT_CONFIG = {
>> 'machine' : 'pc',
>> 'guest_user' : "qemu",
>> 'guest_pass' : "qemupass",
>> + 'root_user' : "root",
>> 'root_pass' : "qemupass",
>> 'ssh_key_file' : SSH_KEY_FILE,
>> 'ssh_pub_key_file': SSH_PUB_KEY_FILE,
>> @@ -245,13 +246,13 @@ class BaseVM(object):
>> return self._ssh_do(self._config["guest_user"], cmd, False)
>>
>> def ssh_root(self, *cmd):
>> - return self._ssh_do("root", cmd, False)
>> + return self._ssh_do(self._config["root_user"], cmd, False)
>>
>> def ssh_check(self, *cmd):
>> self._ssh_do(self._config["guest_user"], cmd, True)
>>
>> def ssh_root_check(self, *cmd):
>> - self._ssh_do("root", cmd, True)
>> + self._ssh_do(self._config["root_user"], cmd, True)
>>
>> def build_image(self, img):
>> raise NotImplementedError
> 
> 
> Haiku's user is UID 0, so essentially our root user isn't named root.
> This adds the (optional) ability to override the root username.

Ditto.

> 
> 
>> diff --git a/tests/vm/haiku.x86_64 b/tests/vm/haiku.x86_64
>> new file mode 100755
>> index 0000000000..9777722f51
>> --- /dev/null
>> +++ b/tests/vm/haiku.x86_64
>> @@ -0,0 +1,121 @@
>> +#!/usr/bin/env python3
>> +#
>> +# Haiku VM image
>> +#
>> +# Copyright 2020 Haiku, Inc.
>> +#
>> +# Authors:
>> +# Alexander von Gluck IV <kallisti5@unixzen.com>
>> +#
>> +# This code is licensed under the GPL version 2 or later. See
>> +# the COPYING file in the top-level directory.
>> +#
> 
> 
> This build script works as expected, transferring the qemu archive over
> via the virtio block device and building it.
> 
> More information here (including output of tools):
> https://bugs.launchpad.net/qemu/+bug/1715203
> 
> This purpose of this is trying to prevent the need to remove
> upstream qemu support for Haiku.
> 
> We have some out-of-tree patches to fix the error seen in our ports, i'll
> work on upstreaming these.

Please do, because so far this fail before compiling the 10th file:

slirp/src/tftp.c: In function 'tftp_read_data':
slirp/src/tftp.c:113:50: error: 'O_BINARY' undeclared (first use in this
function); did you mean 'L_INCR'?
         spt->fd = open(spt->filename, O_RDONLY | O_BINARY);
                                                  ^~~~~~~~
                                                  L_INCR

To avoid using 4GB of temporary storage in my HOMEDIR I had
to do this change:

-- >8 --
--- a/tests/vm/haiku.x86_64
+++ b/tests/vm/haiku.x86_64
@@ -93,17 +93,15 @@ class HaikuVM(basevm.BaseVM):

     def build_image(self, img):
         self.print_step("Downloading disk image")
-        cvg = self._download_with_cache(self.link, sha256sum=self.csum)
-        cgz = cvg + ".tar.gz"
-        img_tmp = "./box.img"
+        tarball = self._download_with_cache(self.link, sha256sum=self.csum)
+
+        self.print_step("Extracting disk image")
+
+        subprocess.check_call(["tar", "xzf", tarball, "./box.img", "-O"],
+                              stdout=open(img, 'wb'))

         self.print_step("Preparing disk image")
-
-        subprocess.check_call(["cp", "-f", cvg, cgz])
-        subprocess.check_call(["tar", "xvzf", cgz, img_tmp])
-        subprocess.check_call(["chmod", "666", img_tmp])
-
-        self.boot(img_tmp)
+        self.boot(img)

         # Wait for ssh to be available.
         self.wait_ssh(wait_root=True, cmd="exit 0")
@@ -112,9 +110,6 @@ class HaikuVM(basevm.BaseVM):
         self.ssh_root("pkgman install -y %s" % " ".join(self.requirements))
         self.graceful_shutdown()

-        if os.path.exists(cgz):
-            os.remove(cgz)
-        subprocess.check_call(["mv", img_tmp, img])
         self.print_step("All done")

 if __name__ == "__main__":
---

Note that something is not working well in your script because
the image is extracted/reinstalled each time. This has to be
done once, then we reuse the image for the builds.

Also, please address Thomas comment.

Thanks,

Phil.
Re: [PATCH] tests/vm: Add Haiku test based on their vagrant images
Posted by Philippe Mathieu-Daudé 3 years, 7 months ago 
On 9/25/20 10:49 AM, Philippe Mathieu-Daudé wrote:
> Hi Alexander,
> 
> On 9/6/20 4:41 PM, Alexander von Gluck IV wrote:
>> September 6, 2020 9:35 AM, "Alexander von Gluck IV" <kallisti5@unixzen.com> wrote:
>>> Signed-off-by: Alexander von Gluck IV <kallisti5@unixzen.com>
>>> ---
>>> tests/keys/vagrant | 27 +++++++++
>>> tests/keys/vagrant.pub | 1 +
>>> tests/vm/basevm.py | 5 +-
>>> tests/vm/haiku.x86_64 | 121 +++++++++++++++++++++++++++++++++++++++++
>>> 4 files changed, 152 insertions(+), 2 deletions(-)
>>> create mode 100644 tests/keys/vagrant
>>> create mode 100644 tests/keys/vagrant.pub
>>> create mode 100755 tests/vm/haiku.x86_64
>>>
>>> diff --git a/tests/keys/vagrant b/tests/keys/vagrant
>>> new file mode 100644
>>> index 0000000000..7d6a083909
>>> --- /dev/null
>>> +++ b/tests/keys/vagrant
>>> @@ -0,0 +1,27 @@
>>> +-----BEGIN RSA PRIVATE KEY-----
>>> +MIIEogIBAAKCAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzI
>>>
>>> diff --git a/tests/keys/vagrant.pub b/tests/keys/vagrant.pub
>>> new file mode 100644
>>> index 0000000000..18a9c00fd5
>>> --- /dev/null
>>> +++ b/tests/keys/vagrant.pub
>>> @@ -0,0 +1 @@
>>> +ssh-rsa
>>> AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oX
>>
>> A little background information for context. These are the Vagrant SSH keys which are packed with every vagrant OS image and allow OS access for automation.  The python vm tester knowing of these lets it leverage Vagrant OS images for testing without much work.
> 
> Please add this information as comment in the source files.
> 
>>
>>
>>
>>> --- a/tests/vm/basevm.py
>>> +++ b/tests/vm/basevm.py
>>> @@ -44,6 +44,7 @@ DEFAULT_CONFIG = {
>>> 'machine' : 'pc',
>>> 'guest_user' : "qemu",
>>> 'guest_pass' : "qemupass",
>>> + 'root_user' : "root",
>>> 'root_pass' : "qemupass",
>>> 'ssh_key_file' : SSH_KEY_FILE,
>>> 'ssh_pub_key_file': SSH_PUB_KEY_FILE,
>>> @@ -245,13 +246,13 @@ class BaseVM(object):
>>> return self._ssh_do(self._config["guest_user"], cmd, False)
>>>
>>> def ssh_root(self, *cmd):
>>> - return self._ssh_do("root", cmd, False)
>>> + return self._ssh_do(self._config["root_user"], cmd, False)
>>>
>>> def ssh_check(self, *cmd):
>>> self._ssh_do(self._config["guest_user"], cmd, True)
>>>
>>> def ssh_root_check(self, *cmd):
>>> - self._ssh_do("root", cmd, True)
>>> + self._ssh_do(self._config["root_user"], cmd, True)
>>>
>>> def build_image(self, img):
>>> raise NotImplementedError
>>
>>
>> Haiku's user is UID 0, so essentially our root user isn't named root.
>> This adds the (optional) ability to override the root username.
> 
> Ditto.
> 
>>
>>
>>> diff --git a/tests/vm/haiku.x86_64 b/tests/vm/haiku.x86_64
>>> new file mode 100755
>>> index 0000000000..9777722f51
>>> --- /dev/null
>>> +++ b/tests/vm/haiku.x86_64
>>> @@ -0,0 +1,121 @@
>>> +#!/usr/bin/env python3
>>> +#
>>> +# Haiku VM image
>>> +#
>>> +# Copyright 2020 Haiku, Inc.
>>> +#
>>> +# Authors:
>>> +# Alexander von Gluck IV <kallisti5@unixzen.com>
>>> +#
>>> +# This code is licensed under the GPL version 2 or later. See
>>> +# the COPYING file in the top-level directory.
>>> +#
>>
>>
>> This build script works as expected, transferring the qemu archive over
>> via the virtio block device and building it.
>>
>> More information here (including output of tools):
>> https://bugs.launchpad.net/qemu/+bug/1715203
>>
>> This purpose of this is trying to prevent the need to remove
>> upstream qemu support for Haiku.
>>
>> We have some out-of-tree patches to fix the error seen in our ports, i'll
>> work on upstreaming these.
> 
> Please do, because so far this fail before compiling the 10th file:
> 
> slirp/src/tftp.c: In function 'tftp_read_data':
> slirp/src/tftp.c:113:50: error: 'O_BINARY' undeclared (first use in this
> function); did you mean 'L_INCR'?
>          spt->fd = open(spt->filename, O_RDONLY | O_BINARY);
>                                                   ^~~~~~~~
>                                                   L_INCR
> 
> To avoid using 4GB of temporary storage in my HOMEDIR I had
> to do this change:
> 
> -- >8 --
> --- a/tests/vm/haiku.x86_64
> +++ b/tests/vm/haiku.x86_64
> @@ -93,17 +93,15 @@ class HaikuVM(basevm.BaseVM):
> 
>      def build_image(self, img):
>          self.print_step("Downloading disk image")
> -        cvg = self._download_with_cache(self.link, sha256sum=self.csum)
> -        cgz = cvg + ".tar.gz"
> -        img_tmp = "./box.img"
> +        tarball = self._download_with_cache(self.link, sha256sum=self.csum)
> +
> +        self.print_step("Extracting disk image")
> +
> +        subprocess.check_call(["tar", "xzf", tarball, "./box.img", "-O"],
> +                              stdout=open(img, 'wb'))
> 
>          self.print_step("Preparing disk image")
> -
> -        subprocess.check_call(["cp", "-f", cvg, cgz])
> -        subprocess.check_call(["tar", "xvzf", cgz, img_tmp])
> -        subprocess.check_call(["chmod", "666", img_tmp])
> -
> -        self.boot(img_tmp)
> +        self.boot(img)
> 
>          # Wait for ssh to be available.
>          self.wait_ssh(wait_root=True, cmd="exit 0")
> @@ -112,9 +110,6 @@ class HaikuVM(basevm.BaseVM):
>          self.ssh_root("pkgman install -y %s" % " ".join(self.requirements))
>          self.graceful_shutdown()
> 
> -        if os.path.exists(cgz):
> -            os.remove(cgz)
> -        subprocess.check_call(["mv", img_tmp, img])
>          self.print_step("All done")
> 
>  if __name__ == "__main__":
> ---
> 
> Note that something is not working well in your script because
> the image is extracted/reinstalled each time. This has to be
> done once, then we reuse the image for the builds.

I figured out make was removing the image as a temporary file,
because you didn't added it in the IMAGES variable, so this is
the fix:

-- >8 --
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -4,7 +4,7 @@

 EFI_AARCH64 = $(wildcard $(BUILD_DIR)/pc-bios/edk2-aarch64-code.fd)

-IMAGES := freebsd netbsd openbsd centos fedora
+IMAGES := freebsd netbsd openbsd centos fedora haiku.x86_64
 ifneq ($(GENISOIMAGE),)
 IMAGES += ubuntu.i386 centos
 ifneq ($(EFI_AARCH64),)
---

> 
> Also, please address Thomas comment.

Which is:

-- >8 --
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -41,6 +41,7 @@ endif
 else
        @echo "  (install genisoimage to build centos/ubuntu images)"
 endif
+       @echo "  vm-build-haiku.x86_64           - Build QEMU in Haiku VM"
        @echo ""
        @echo "  vm-build-all                    - Build QEMU in all VMs"
        @echo "  vm-clean-all                    - Clean up VM images"
---
> 
> Thanks,
> 
> Phil.
>
Re: [PATCH] tests/vm: Add Haiku test based on their vagrant images
Posted by Thomas Huth 3 years, 7 months ago 
On 06/09/2020 16.34, Alexander von Gluck IV wrote:
> Signed-off-by: Alexander von Gluck IV <kallisti5@unixzen.com>
> ---
>  tests/keys/vagrant     |  27 +++++++++
>  tests/keys/vagrant.pub |   1 +
>  tests/vm/basevm.py     |   5 +-
>  tests/vm/haiku.x86_64  | 121 +++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 152 insertions(+), 2 deletions(-)
>  create mode 100644 tests/keys/vagrant
>  create mode 100644 tests/keys/vagrant.pub
>  create mode 100755 tests/vm/haiku.x86_64

In case you respin this patch: Please also add a line to the output of
"make vm-help" in tests/vm/Makefile.include.

Thanks,
 Thomas

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.