From nobody Mon Feb 9 17:24:08 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593612226; cv=none; d=zohomail.com; s=zohoarc; b=gDSEDnbPmyPkHidq/3G+uam4Uws40eVsMTwKE4mb+XDmuBPXMXGkEEtC2fhZ2hGyKcJHD/XBHUqyesLW+eJ43R4srlkLz9R0aJxRCKuyNRuspCQ8cPe7642xiah6o4thXSnyw9vuHpNHPGJRbSTEdAjYJh5hKXgFSVzEO7tEhK8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593612226; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lPp8XkbnsM3c17W7WUUEalp68Xx8V1JQ2Yf70AfovqA=; b=KVOI3jz/sFPycM+lKyrbk21e1x/YBSEnSjaJyOl7Q0Ets4BYfuVeuk5aTRgyPQcbhzt6FFlgSDpMTm7rFyP5xt30P9jI0+AVM6VuB/1qn5vpqbhRf7iULvjDHBrLRSJZcA2FHErXdmi7vAsS6nZdCqs3a4WS5H8K79ITg3EhjAk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593612226710437.68443827105716; Wed, 1 Jul 2020 07:03:46 -0700 (PDT) Received: from localhost ([::1]:51052 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jqdKz-0004L8-9g for importer@patchew.org; Wed, 01 Jul 2020 10:03:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58922) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jqdEa-0002RS-Ii for qemu-devel@nongnu.org; Wed, 01 Jul 2020 09:57:08 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]:35786) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jqdEX-0005rS-JA for qemu-devel@nongnu.org; Wed, 01 Jul 2020 09:57:08 -0400 Received: by mail-wr1-x431.google.com with SMTP id z2so1691298wrp.2 for ; Wed, 01 Jul 2020 06:57:05 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 207sm8165108wme.13.2020.07.01.06.56.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Jul 2020 06:57:00 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id B60331FF98; Wed, 1 Jul 2020 14:56:53 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lPp8XkbnsM3c17W7WUUEalp68Xx8V1JQ2Yf70AfovqA=; b=hs6B33ZuIJP/corK1jB5ucTI9w3eJccTMvFHqd+Wbq40qTGa7ed8yo+pkq7kV2KPuZ dAEGGemtJ43yhk9IDG9vqLODV5/afVYJUlmy/RS+US028CU9Rt8pN8jr4g0XW314ejqh 56T1dy030+ENVNHcUYCT/xEsPH5KubVKH5bubg3J7PY/eDaow2Aku/BNEvAbdno1bJzM FL2D8Lgj9HAPnJezUG6EVXUl7f3hpgEwIsUhroz1WCfpUsOiizn2VG3bl9PiHxCsGovT h0aKfZ6IJWABO3FVCTlTMH0Qw+PdA+t9/XXFJILbCw+opBhEDHInW3Z01+yXJpsiNGJx 3Bhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lPp8XkbnsM3c17W7WUUEalp68Xx8V1JQ2Yf70AfovqA=; b=RmdGBPU9D7BfZs/R2LGAOMQrxFELXIeKwGKxdhmMx21Et8ylDwctqxZUuRW6CY0Klg fDNmugj5OcIkDpqJIhFToyUqClvge1oBSd6W4ju7O6I9WNvMN+A+bgT+JjiuTans0MDF 2J9VUT+sWoPkUqjtft25l8KcFLSxefABOAQAUA6PDkKXmtsAeI+YVQEZkq9ZVlVoo033 aVyc9MXpp+mPVvKMjWDnvkWg8xHGQv3xVxE6kPc/1FB0DexZlnaPNXzWzbYGDc0MBVFU gElW9W1JKlajZbhVqSxiEmNOQMWlD7ZuX1Sxau95L39YAVLlFVlYZsTU51FROBUAh2K4 ls4A== X-Gm-Message-State: AOAM530igDKYAKfAgmVxC3bNYzDw9psSas3h7Up7ex+aAhLoBXlHs+Ou pO8F1v3IGKLO3NuoKd+1kliBxw== X-Google-Smtp-Source: ABdhPJwNzyRXyKa71o1J9mM+/4urUsWcyziLA29ZMC+U5MmN74a3eckER20fy7k9LpPGYIPgYT6stQ== X-Received: by 2002:a5d:484b:: with SMTP id n11mr25751128wrs.320.1593611823980; Wed, 01 Jul 2020 06:57:03 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v4 09/40] tests/vm: Added a new script for ubuntu.aarch64. Date: Wed, 1 Jul 2020 14:56:21 +0100 Message-Id: <20200701135652.1366-10-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200701135652.1366-1-alex.bennee@linaro.org> References: <20200701135652.1366-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x431.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley ubuntu.aarch64 provides a script to create an Ubuntu 18.04 VM. Another new file is also added aarch64vm.py, which is a module with common methods used by aarch64 VMs, such as how to create the flash images. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-6-robert.foley@linaro.org> --- configure | 20 +++++++ tests/vm/Makefile.include | 11 ++++ tests/vm/aarch64vm.py | 106 ++++++++++++++++++++++++++++++++++++++ tests/vm/basevm.py | 12 +++++ tests/vm/ubuntu.aarch64 | 68 ++++++++++++++++++++++++ 5 files changed, 217 insertions(+) create mode 100644 tests/vm/aarch64vm.py create mode 100755 tests/vm/ubuntu.aarch64 diff --git a/configure b/configure index 96256be057a..1fb4d5b61e4 100755 --- a/configure +++ b/configure @@ -418,6 +418,7 @@ prefix=3D"/usr/local" mandir=3D"\${prefix}/share/man" datadir=3D"\${prefix}/share" firmwarepath=3D"\${prefix}/share/qemu-firmware" +efi_aarch64=3D"" qemu_docdir=3D"\${prefix}/share/doc/qemu" bindir=3D"\${prefix}/bin" libdir=3D"\${prefix}/lib" @@ -1109,6 +1110,8 @@ for opt do ;; --firmwarepath=3D*) firmwarepath=3D"$optarg" ;; + --efi-aarch64=3D*) efi_aarch64=3D"$optarg" + ;; --host=3D*|--build=3D*|\ --disable-dependency-tracking|\ --sbindir=3D*|--sharedstatedir=3D*|\ @@ -1787,6 +1790,7 @@ Advanced options (experts only): --sysconfdir=3DPATH install config in PATH$confsuffix --localstatedir=3DPATH install local state in PATH (set at runtime o= n win32) --firmwarepath=3DPATH search PATH for firmware files + --efi-aarch64=3DPATH PATH of efi file to use for aarch64 VMs. --with-confsuffix=3DSUFFIX suffix for QEMU data inside datadir/libdir/sy= sconfdir [$confsuffix] --with-pkgversion=3DVERS use specified string as sub-version of the pa= ckage --enable-debug enable common debug build options @@ -3606,6 +3610,20 @@ EOF fi fi =20 +############################################ +# efi-aarch64 probe +# Check for efi files needed by aarch64 VMs. +# By default we will use the efi included with QEMU. +# Allow user to override the path for efi also. +if ! test -f "$efi_aarch64"; then + if test -f $source_path/pc-bios/edk2-aarch64-code.fd.bz2; then + # valid after build + efi_aarch64=3D$PWD/pc-bios/edk2-aarch64-code.fd + else + efi_aarch64=3D"" + fi +fi + ########################################## # libcap-ng library probe if test "$cap_ng" !=3D "no" ; then @@ -6850,6 +6868,7 @@ if test "$docs" !=3D "no"; then echo "sphinx-build $sphinx_build" fi echo "genisoimage $genisoimage" +echo "efi_aarch64 $efi_aarch64" echo "python_yaml $python_yaml" echo "slirp support $slirp $(echo_version $slirp $slirp_version)" if test "$slirp" !=3D "no" ; then @@ -7936,6 +7955,7 @@ echo "PYTHON=3D$python" >> $config_host_mak echo "SPHINX_BUILD=3D$sphinx_build" >> $config_host_mak echo "SPHINX_WERROR=3D$sphinx_werror" >> $config_host_mak echo "GENISOIMAGE=3D$genisoimage" >> $config_host_mak +echo "EFI_AARCH64=3D$efi_aarch64" >> $config_host_mak echo "PYTHON_YAML=3D$python_yaml" >> $config_host_mak echo "CC=3D$cc" >> $config_host_mak if $iasl -h > /dev/null 2>&1; then diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index f6c3892bb28..4fa292765d5 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -5,6 +5,9 @@ IMAGES :=3D freebsd netbsd openbsd centos fedora ifneq ($(GENISOIMAGE),) IMAGES +=3D ubuntu.i386 centos +ifneq ($(EFI_AARCH64),) +IMAGES +=3D ubuntu.aarch64 +endif endif =20 IMAGES_DIR :=3D $(HOME)/.cache/qemu-vm/images @@ -23,6 +26,11 @@ vm-help vm-test: ifneq ($(GENISOIMAGE),) @echo " vm-build-centos - Build QEMU in CentOS VM, with = Docker" @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM" +ifneq ($(EFI_AARCH64),) + @echo " vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 V= M" +else + @echo " (to build centos/ubuntu aarch64 images use configure --efi-aarch= 64)" +endif else @echo " (install genisoimage to build centos/ubuntu images)" endif @@ -65,6 +73,7 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \ $(if $(V)$(DEBUG), --debug) \ $(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$@" \ --force \ --build-image $@, \ @@ -80,6 +89,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(J),--jobs $(J)) \ $(if $(V),--verbose) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ --snapshot \ @@ -102,6 +112,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(if $(J),--jobs $(J)) \ $(if $(V)$(DEBUG), --debug) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$<" \ --interactive \ false, \ diff --git a/tests/vm/aarch64vm.py b/tests/vm/aarch64vm.py new file mode 100644 index 00000000000..bb04cb19c91 --- /dev/null +++ b/tests/vm/aarch64vm.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +# +# VM testing aarch64 library +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# +import os +import sys +import subprocess +import basevm +from qemu.accel import kvm_available + +# This is the config needed for current version of QEMU. +# This works for both kvm and tcg. +CURRENT_CONFIG =3D { + 'cpu' : "max", + 'machine' : "virt,gic-version=3Dmax", +} + +# The minimum minor version of QEMU we will support with aarch64 VMs is 3. +# QEMU versions less than 3 have various issues running these VMs. +QEMU_AARCH64_MIN_VERSION =3D 3 + +# The DEFAULT_CONFIG will default to a version of +# parameters that works for backwards compatibility. +DEFAULT_CONFIG =3D {'kvm' : {'cpu' : "host", + 'machine' : "virt,gic-version=3Dhost"}, + 'tcg' : {'cpu' : "cortex-a57", + 'machine' : "virt"}, +} + +def get_config_defaults(vmcls, default_config): + """Fetch the configuration defaults for this VM, + taking into consideration the defaults for + aarch64 first, followed by the defaults for this VM.""" + config =3D default_config + config.update(aarch_get_config_defaults(vmcls)) + return config + +def aarch_get_config_defaults(vmcls): + """Set the defaults for current version of QEMU.""" + config =3D CURRENT_CONFIG + args, argv =3D basevm.parse_args(vmcls) + qemu_path =3D basevm.get_qemu_path(vmcls.arch, args.build_path) + qemu_version =3D basevm.get_qemu_version(qemu_path) + if qemu_version < QEMU_AARCH64_MIN_VERSION: + error =3D "\nThis major version of QEMU {} is to old for aarch64 V= Ms.\n"\ + "The major version must be at least {}.\n"\ + "To continue with the current build of QEMU, "\ + "please restart with QEMU_LOCAL=3D1 .\n" + print(error.format(qemu_version, QEMU_AARCH64_MIN_VERSION)) + exit(1) + if qemu_version =3D=3D QEMU_AARCH64_MIN_VERSION: + # We have an older version of QEMU, + # set the config values for backwards compatibility. + if kvm_available('aarch64'): + config.update(DEFAULT_CONFIG['kvm']) + else: + config.update(DEFAULT_CONFIG['tcg']) + return config + +def create_flash_images(flash_dir=3D"./", efi_img=3D""): + """Creates the appropriate pflash files + for an aarch64 VM.""" + flash0_path =3D get_flash_path(flash_dir, "flash0") + flash1_path =3D get_flash_path(flash_dir, "flash1") + fd_null =3D open(os.devnull, 'w') + subprocess.check_call(["dd", "if=3D/dev/zero", "of=3D{}".format(flash0= _path), + "bs=3D1M", "count=3D64"], + stdout=3Dfd_null, stderr=3Dsubprocess.STDOUT) + # A reliable way to get the QEMU EFI image is via an installed package= or + # via the bios included with qemu. + if not os.path.exists(efi_img): + sys.stderr.write("*** efi argument is invalid ({})\n".format(efi_i= mg)) + sys.stderr.write("*** please check --efi-aarch64 argument or "\ + "install qemu-efi-aarch64 package\n") + exit(3) + subprocess.check_call(["dd", "if=3D{}".format(efi_img), + "of=3D{}".format(flash0_path), + "conv=3Dnotrunc"], + stdout=3Dfd_null, stderr=3Dsubprocess.STDOUT) + subprocess.check_call(["dd", "if=3D/dev/zero", + "of=3D{}".format(flash1_path), + "bs=3D1M", "count=3D64"], + stdout=3Dfd_null, stderr=3Dsubprocess.STDOUT) + fd_null.close() + +def get_pflash_args(flash_dir=3D"./"): + """Returns a string that can be used to + boot qemu using the appropriate pflash files + for aarch64.""" + flash0_path =3D get_flash_path(flash_dir, "flash0") + flash1_path =3D get_flash_path(flash_dir, "flash1") + pflash_args_str =3D "-drive file=3D{},format=3Draw,if=3Dpflash "\ + "-drive file=3D{},format=3Draw,if=3Dpflash" + pflash_args =3D pflash_args_str.format(flash0_path, flash1_path) + return pflash_args.split(" ") + +def get_flash_path(flash_dir, name): + return os.path.join(flash_dir, "{}.img".format(name)) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index fa56fbbb4b6..5fd66f6b26a 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -92,6 +92,7 @@ class BaseVM(object): self._guest =3D None self._genisoimage =3D args.genisoimage self._build_path =3D args.build_path + self._efi_aarch64 =3D args.efi_aarch64 # Allow input config to override defaults. self._config =3D DEFAULT_CONFIG.copy() if config !=3D None: @@ -496,6 +497,14 @@ def get_qemu_path(arch, build_path=3DNone): qemu_path =3D "qemu-system-" + arch return qemu_path =20 +def get_qemu_version(qemu_path): + """Get the version number from the current QEMU, + and return the major number.""" + output =3D subprocess.check_output([qemu_path, '--version']) + version_line =3D output.decode("utf-8") + version_num =3D re.split(' |\(', version_line)[3].split('.')[0] + return int(version_num) + def parse_config(config, args): """ Parse yaml config and populate our config structure. The yaml config allows the user to override the @@ -573,6 +582,9 @@ def parse_args(vmcls): parser.add_option("--config", "-c", default=3DNone, help=3D"Provide config yaml for configuration. "\ "See config_example.yaml for example.") + parser.add_option("--efi-aarch64", + default=3D"/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", + help=3D"Path to efi image for aarch64 VMs.") parser.disable_interspersed_args() return parser.parse_args() =20 diff --git a/tests/vm/ubuntu.aarch64 b/tests/vm/ubuntu.aarch64 new file mode 100755 index 00000000000..21d454c27f1 --- /dev/null +++ b/tests/vm/ubuntu.aarch64 @@ -0,0 +1,68 @@ +#!/usr/bin/env python3 +# +# Ubuntu aarch64 image +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.i386 Fam Zheng +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# + +import sys +import basevm +import aarch64vm +import ubuntuvm + +DEFAULT_CONFIG =3D { + 'cpu' : "cortex-a57", + 'machine' : "virt,gic-version=3D3", + 'install_cmds' : "apt-get update,"\ + "apt-get build-dep -y --arch-only qemu,"\ + "apt-get install -y libfdt-dev pkg-config language-pa= ck-en", + # We increase beyond the default time since during boot + # it can take some time (many seconds) to log into the VM + # especially using softmmu. + 'ssh_timeout' : 60, +} + +class UbuntuAarch64VM(ubuntuvm.UbuntuVM): + name =3D "ubuntu.aarch64" + arch =3D "aarch64" + image_name =3D "ubuntu-18.04-server-cloudimg-arm64.img" + image_link =3D "https://cloud-images.ubuntu.com/releases/18.04/release= /" + image_name + image_sha256=3D"0fdcba761965735a8a903d8b88df8e47f156f48715c00508e4315c= 506d7d3cb1" + BUILD_SCRIPT =3D """ + set -e; + cd $(mktemp -d); + sudo chmod a+r /dev/vdb; + tar --checkpoint=3D.10 -xf /dev/vdb; + ./configure {configure_opts}; + make --output-sync {target} -j{jobs} {verbose}; + """ + def boot(self, img, extra_args=3DNone): + aarch64vm.create_flash_images(self._tmpdir, self._efi_aarch64) + default_args =3D aarch64vm.get_pflash_args(self._tmpdir) + if extra_args: + extra_args.extend(default_args) + else: + extra_args =3D default_args + # We always add these performance tweaks + # because without them, we boot so slowly that we + # can time out finding the boot efi device. + if '-smp' not in extra_args and \ + '-smp' not in self._config['extra_args'] and \ + '-smp' not in self._args: + # Only add if not already there to give caller option to chang= e it. + extra_args.extend(["-smp", "8"]) + + # We have overridden boot() since aarch64 has additional parameter= s. + # Call down to the base class method. + super(UbuntuAarch64VM, self).boot(img, extra_args=3Dextra_args) + +if __name__ =3D=3D "__main__": + defaults =3D aarch64vm.get_config_defaults(UbuntuAarch64VM, DEFAULT_CO= NFIG) + sys.exit(basevm.main(UbuntuAarch64VM, defaults)) --=20 2.20.1