From nobody Mon Feb 9 11:06:34 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1589312112; cv=none; d=zohomail.com; s=zohoarc; b=bK6BmElSzmigwroe99CqcOWJkqYwtzekkXv8EwaBE0Faixq2Jp+aiY1GAnQpvJIv7dRaIzFRuC53+ZDH+KB5eP9dGSVfHx2dCLOWtbLHPeL0x0SgWiwS8Yn+OYLQwTHgIYRPZAKnpDqoj+PatA+tuMPyZ+QeYOWUPJ9ehl+GCtk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589312112; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=IHVxQvHLGmuAsOe7Our5n+V2vriioUviLGm8LCdzEg4=; b=dvSWcljoQLPcsD2pgXqwVTIfjV7kzd6I8SeuRjs9AHWy7ffdCm3VPC3Jqv+4NrgjSGQHA7p52O7qJbsn33RTstiDt1TTcxrjGi6h4pr+mwtAAIajjiT5FclraUAJkRdU0OlafPL6O2EwCkS2He9vxGljCsx2ynNVKtotSTlOFD0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1589312112815465.14612252549193; Tue, 12 May 2020 12:35:12 -0700 (PDT) Received: from localhost ([::1]:41378 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYagJ-0001s7-9J for importer@patchew.org; Tue, 12 May 2020 15:35:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47242) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYaej-0000K8-L0 for qemu-devel@nongnu.org; Tue, 12 May 2020 15:33:34 -0400 Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d]:41612) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jYaeh-0001q7-Pm for qemu-devel@nongnu.org; Tue, 12 May 2020 15:33:33 -0400 Received: by mail-qk1-x72d.google.com with SMTP id n14so14913763qke.8 for ; Tue, 12 May 2020 12:33:31 -0700 (PDT) Received: from Rfoley-MA01.hsd1.ma.comcast.net ([2601:199:4480:60c0:d31:9f53:b3eb:9b53]) by smtp.gmail.com with ESMTPSA id j45sm2100998qtk.14.2020.05.12.12.33.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2020 12:33:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IHVxQvHLGmuAsOe7Our5n+V2vriioUviLGm8LCdzEg4=; b=VNobTaSht+DGyk/GTwbQcYUVC/3h5zk4hkLIhzb/lhD65yDyrAKQEKPXImz3Kp4B3u CuLulFQLxYm6FPaKukYJESwODCK+Jwaz8QwDvk8VhpJ6ipun7vuz+miAqegMNNLIrwak uIU1j8gou1XkaJjnV3GUSDlmOJMpeSvUYwDdNzRTeQeZiuivEE7dUQHgOh01WC5i902R KIYGMhoZPsMw06gDKwltZfKRW/bdbVpcIiVHluNINTAebvNqTk+NTZ1gruOa55v8VQIr kZbHyWNHDJGCN6ENnRK66VCDhLDS3DBGC6h4JPcsTTa96lhYI2ha/GwdlpyC2x6npIE8 frMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IHVxQvHLGmuAsOe7Our5n+V2vriioUviLGm8LCdzEg4=; b=QXkic65z4RgQSwYfiKZc7ei202LkjBp5LrgJIZeo28HgxNdmmLsf971Y1qATcenHxg kynqGNJ4HUQzl0zhJa/nHVq9erDJH3pcQVn8jm5V736db0BKuIBgF95ncuyMuKlzUNGN N59Ivm6mmz7fI7i20VzmTWN6FL1qeZHV42ul0yhJcviNH/eQd0oBVqBe3++sFMwzx/s/ Xhi4WqEk9Ncj/zauPx4PGyfaHTeVGP+5ww9g9sfC2BafxWgLwHQP2evPIu07w9FySQba Q+co6QdEIJ/xjOmSgbMdGeYxyA44LT65WOV3j+8ExmjDWDlhdmA5udA8vj6tnZU32X3c y8aA== X-Gm-Message-State: AGi0PuY9a3/+J1oIpCv1Sz5r0WL9ULIQ4WcoG4lXQgten3pgxsoe188k kO/lRD7/2fmdu5MwEbi3MP66zdA2X/Vy7w== X-Google-Smtp-Source: APiQypIjxhT8M4F13SFKY/4mGL8MggqFy9blAYcZM+h+fH1GShI48rfl9zQaoKcb446LBwsoVqN0YA== X-Received: by 2002:a37:8902:: with SMTP id l2mr21610049qkd.127.1589312009833; Tue, 12 May 2020 12:33:29 -0700 (PDT) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v6 1/9] tests/vm: Add configuration to basevm.py Date: Tue, 12 May 2020 15:33:32 -0400 Message-Id: <20200512193340.265-2-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200512193340.265-1-robert.foley@linaro.org> References: <20200512193340.265-1-robert.foley@linaro.org> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::72d; envelope-from=robert.foley@linaro.org; helo=mail-qk1-x72d.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Added use of a configuration to tests/vm/basevm.py. The configuration provides parameters used to configure a VM. This allows for providing alternate configurations to the VM being created/launched. cpu, machine, memory, and NUMA configuration are all examples of configuration which we might want to vary on the VM being creat= ed or launched. This will for example allow for creating an aarch64 vm. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov --- tests/vm/basevm.py | 170 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 131 insertions(+), 39 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 756ccf7aca..fb93a111c1 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -29,16 +29,41 @@ import tempfile import shutil import multiprocessing import traceback - -SSH_KEY =3D open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa")).read() -SSH_PUB_KEY =3D open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa.pub")).read() - +import shlex + +SSH_KEY_FILE =3D os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa") +SSH_PUB_KEY_FILE =3D os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa.pub") + +# This is the standard configuration. +# Any or all of these can be overridden by +# passing in a config argument to the VM constructor. +DEFAULT_CONFIG =3D { + 'cpu' : "max", + 'machine' : 'pc', + 'guest_user' : "qemu", + 'guest_pass' : "qemupass", + 'root_pass' : "qemupass", + 'ssh_key_file' : SSH_KEY_FILE, + 'ssh_pub_key_file': SSH_PUB_KEY_FILE, + 'memory' : "4G", + 'extra_args' : [], + 'qemu_args' : "", + 'dns' : "", + 'ssh_port' : 0, + 'install_cmds' : "", + 'boot_dev_type' : "block", + 'ssh_timeout' : 1, +} +BOOT_DEVICE =3D { + 'block' : "-drive file=3D{},if=3Dnone,id=3Ddrive0,cache=3Dwriteback "\ + "-device virtio-blk,drive=3Ddrive0,bootindex=3D0", + 'scsi' : "-device virtio-scsi-device,id=3Dscsi "\ + "-drive file=3D{},format=3Draw,if=3Dnone,id=3Dhd0 "\ + "-device scsi-hd,drive=3Dhd0,bootindex=3D0", +} class BaseVM(object): - GUEST_USER =3D "qemu" - GUEST_PASS =3D "qemupass" - ROOT_PASS =3D "qemupass" =20 envvars =3D [ "https_proxy", @@ -61,19 +86,30 @@ class BaseVM(object): # 4 is arbitrary, but greater than 2, # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier =3D 4 - def __init__(self, debug=3DFalse, vcpus=3DNone): + def __init__(self, debug=3DFalse, vcpus=3DNone, config=3DNone): self._guest =3D None + # Allow input config to override defaults. + self._config =3D DEFAULT_CONFIG.copy() + if config !=3D None: + self._config.update(config) + self.validate_ssh_keys() self._tmpdir =3D os.path.realpath(tempfile.mkdtemp(prefix=3D"vm-te= st-", suffix=3D".tmp", dir=3D".")) atexit.register(shutil.rmtree, self._tmpdir) - - self._ssh_key_file =3D os.path.join(self._tmpdir, "id_rsa") - open(self._ssh_key_file, "w").write(SSH_KEY) - subprocess.check_call(["chmod", "600", self._ssh_key_file]) - - self._ssh_pub_key_file =3D os.path.join(self._tmpdir, "id_rsa.pub") - open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY) + # Copy the key files to a temporary directory. + # Also chmod the key file to agree with ssh requirements. + self._config['ssh_key'] =3D \ + open(self._config['ssh_key_file']).read().rstrip() + self._config['ssh_pub_key'] =3D \ + open(self._config['ssh_pub_key_file']).read().rstrip() + self._ssh_tmp_key_file =3D os.path.join(self._tmpdir, "id_rsa") + open(self._ssh_tmp_key_file, "w").write(self._config['ssh_key']) + subprocess.check_call(["chmod", "600", self._ssh_tmp_key_file]) + + self._ssh_tmp_pub_key_file =3D os.path.join(self._tmpdir, "id_rsa.= pub") + open(self._ssh_tmp_pub_key_file, + "w").write(self._config['ssh_pub_key']) =20 self.debug =3D debug self._stderr =3D sys.stderr @@ -82,11 +118,14 @@ class BaseVM(object): self._stdout =3D sys.stdout else: self._stdout =3D self._devnull + netdev =3D "user,id=3Dvnet,hostfwd=3D:127.0.0.1:{}-:22" self._args =3D [ \ - "-nodefaults", "-m", "4G", - "-cpu", "max", - "-netdev", "user,id=3Dvnet,hostfwd=3D:127.0.0.1:0-:22" + - (",ipv6=3Dno" if not self.ipv6 else ""), + "-nodefaults", "-m", self._config['memory'], + "-cpu", self._config['cpu'], + "-netdev", + netdev.format(self._config['ssh_port']) + + (",ipv6=3Dno" if not self.ipv6 else "") + + (",dns=3D" + self._config['dns'] if self._config['dns'] else "= "), "-device", "virtio-net-pci,netdev=3Dvnet", "-vnc", "127.0.0.1:0,to=3D20"] if vcpus and vcpus > 1: @@ -97,6 +136,55 @@ class BaseVM(object): logging.info("KVM not available, not using -enable-kvm") self._data_args =3D [] =20 + if self._config['qemu_args'] !=3D None: + qemu_args =3D self._config['qemu_args'] + qemu_args =3D qemu_args.replace('\n',' ').replace('\r','') + # shlex groups quoted arguments together + # we need this to keep the quoted args together for when + # the QEMU command is issued later. + args =3D shlex.split(qemu_args) + self._config['extra_args'] =3D [] + for arg in args: + if arg: + # Preserve quotes around arguments. + # shlex above takes them out, so add them in. + if " " in arg: + arg =3D '"{}"'.format(arg) + self._config['extra_args'].append(arg) + + def validate_ssh_keys(self): + """Check to see if the ssh key files exist.""" + if 'ssh_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_key_file']): + raise Exception("ssh key file not found.") + if 'ssh_pub_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_pub_key_file']): + raise Exception("ssh pub key file not found.") + + def wait_boot(self, wait_string=3DNone): + """Wait for the standard string we expect + on completion of a normal boot. + The user can also choose to override with an + alternate string to wait for.""" + if wait_string is None: + if self.login_prompt is None: + raise Exception("self.login_prompt not defined") + wait_string =3D self.login_prompt + # Intentionally bump up the default timeout under TCG, + # since the console wait below takes longer. + timeout =3D self.socket_timeout + if not kvm_available(self.arch): + timeout *=3D 8 + self.console_init(timeout=3Dtimeout) + self.console_wait(wait_string) + + def __getattr__(self, name): + # Support direct access to config by key. + # for example, access self._config['cpu'] by self.cpu + if name.lower() in self._config.keys(): + return self._config[name.lower()] + return object.__getattribute__(self, name) + def _download_with_cache(self, url, sha256sum=3DNone, sha512sum=3DNone= ): def check_sha256sum(fname): if not sha256sum: @@ -128,8 +216,9 @@ class BaseVM(object): "-t", "-o", "StrictHostKeyChecking=3Dno", "-o", "UserKnownHostsFile=3D" + os.devnull, - "-o", "ConnectTimeout=3D1", - "-p", self.ssh_port, "-i", self._ssh_key_file] + "-o", + "ConnectTimeout=3D{}".format(self._config["ssh_timeout"= ]), + "-p", self.ssh_port, "-i", self._ssh_tmp_key_file] # If not in debug mode, set ssh to quiet mode to # avoid printing the results of commands. if not self.debug: @@ -178,14 +267,14 @@ class BaseVM(object): "virtio-blk,drive=3D%s,serial=3D%s,bootindex= =3D1" % (name, name)] =20 def boot(self, img, extra_args=3D[]): - args =3D self._args + [ - "-drive", "file=3D%s,if=3Dnone,id=3Ddrive0,cache=3Dwriteback" = % img, - "-device", "virtio-blk,drive=3Ddrive0,bootindex=3D0"] - args +=3D self._data_args + extra_args + boot_dev =3D BOOT_DEVICE[self._config['boot_dev_type']] + boot_params =3D boot_dev.format(img) + args =3D self._args + boot_params.split(' ') + args +=3D self._data_args + extra_args + self._config['extra_args'] logging.debug("QEMU args: %s", " ".join(args)) qemu_bin =3D os.environ.get("QEMU", "qemu-system-" + self.arch) guest =3D QEMUMachine(binary=3Dqemu_bin, args=3Dargs) - guest.set_machine('pc') + guest.set_machine(self._config['machine']) guest.set_console() try: guest.launch() @@ -299,7 +388,8 @@ class BaseVM(object): self.console_send(command) =20 def console_ssh_init(self, prompt, user, pw): - sshkey_cmd =3D "echo '%s' > .ssh/authorized_keys\n" % SSH_PUB_KEY.= rstrip() + sshkey_cmd =3D "echo '%s' > .ssh/authorized_keys\n" \ + % self._config['ssh_pub_key'].rstrip() self.console_wait_send("login:", "%s\n" % user) self.console_wait_send("Password:", "%s\n" % pw) self.console_wait_send(prompt, "mkdir .ssh\n") @@ -358,23 +448,23 @@ class BaseVM(object): "local-hostname: {}-guest\n".format(name)]) mdata.close() udata =3D open(os.path.join(cidir, "user-data"), "w") - print("guest user:pw {}:{}".format(self.GUEST_USER, - self.GUEST_PASS)) + print("guest user:pw {}:{}".format(self._config['guest_user'], + self._config['guest_pass'])) udata.writelines(["#cloud-config\n", "chpasswd:\n", " list: |\n", - " root:%s\n" % self.ROOT_PASS, - " %s:%s\n" % (self.GUEST_USER, - self.GUEST_PASS), + " root:%s\n" % self._config['root_pass'], + " %s:%s\n" % (self._config['guest_user'], + self._config['guest_pass']), " expire: False\n", "users:\n", - " - name: %s\n" % self.GUEST_USER, + " - name: %s\n" % self._config['guest_user'], " sudo: ALL=3D(ALL) NOPASSWD:ALL\n", " ssh-authorized-keys:\n", - " - %s\n" % SSH_PUB_KEY, + " - %s\n" % self._config['ssh_pub_key'], " - name: root\n", " ssh-authorized-keys:\n", - " - %s\n" % SSH_PUB_KEY, + " - %s\n" % self._config['ssh_pub_key'], "locale: en_US.UTF-8\n"]) proxy =3D os.environ.get("http_proxy") if not proxy is None: @@ -427,15 +517,17 @@ def parse_args(vmcls): parser.disable_interspersed_args() return parser.parse_args() =20 -def main(vmcls): +def main(vmcls, config=3DNone): try: + if config =3D=3D None: + config =3D {} args, argv =3D parse_args(vmcls) if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 logging.basicConfig(level=3D(logging.DEBUG if args.debug else logging.WARN)) - vm =3D vmcls(debug=3Dargs.debug, vcpus=3Dargs.jobs) + vm =3D vmcls(debug=3Dargs.debug, vcpus=3Dargs.jobs, config=3Dconfi= g) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.im= age, --=20 2.17.1