From nobody Mon Feb 9 17:36:11 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1579885199; cv=none; d=zohomail.com; s=zohoarc; b=FZRZR7icW0dpmpBMWEmJLM9Yie9Vlk1mwQB0+Z9OOlNkONPWSwj0ZL4WrJIbg2zHJTV/qKrp/zm9Pz/eCUgEg7zXNNjm/7EfZ1m1C0wP2uo5x0px7KZ/V9oRdM1YQ148tYTIOxGm24dtc4CqIes/L6XH4SB6lMzz1VRIE4851H4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579885199; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=MwX8tXxSDd35fuy08I41Y0xncP6DuvbsklPwCAUTvwA=; b=OE77HLXwkA8XglndNxTpmDCMC7Ib5YkkM3xk9ra0nTUt8REvYKvq/4IAcQ0TCfVY050aojJnl06hTkSpEL0yzXAbh0/X9N7sM3mTbFm3fy25WasxZmoqK1Y2Y/kO+7GCqY/GL5rCQ5W/zQzUOYHDnMLYjFtWNlvohA+aCYlCH58= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1579885199794457.28109307116654; Fri, 24 Jan 2020 08:59:59 -0800 (PST) Received: from localhost ([::1]:45026 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iv2JK-0002L8-Ge for importer@patchew.org; Fri, 24 Jan 2020 11:59:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45623) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iv2G1-0003sO-0D for qemu-devel@nongnu.org; Fri, 24 Jan 2020 11:56:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iv2Fz-00034o-Ii for qemu-devel@nongnu.org; Fri, 24 Jan 2020 11:56:32 -0500 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]:34659) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iv2Fz-00034B-CP for qemu-devel@nongnu.org; Fri, 24 Jan 2020 11:56:31 -0500 Received: by mail-pg1-x544.google.com with SMTP id r11so1409434pgf.1 for ; Fri, 24 Jan 2020 08:56:31 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id l10sm6969953pjy.5.2020.01.24.08.56.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2020 08:56:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MwX8tXxSDd35fuy08I41Y0xncP6DuvbsklPwCAUTvwA=; b=MwOm5ktlM7+HK+md2zbDxrdFdZfbx2+qbTdvOyBSQA1OBh3wIeUNrRPCaZDSm9C9L6 /Nv1MerRi0T/0gl9mKv3FDe178tLcWOrgP3vLhpd7Qdj1D4Pim+MzjAEnm8aJb1oZsKT b34Ql4pzMWQDi33+s+zVzSe5u2eUJnZoK/DtiH5ozn8XpFjE7l8ovrNlCCRLKZsoVdNQ iwsP+oiAdhJ2eEr5QSfUJx0zyaQZwZVgUUkYEMB0EtQuPZtfPkWcFSw+1biFW9NNZC9P X84hDb9+D+wzMliptZf+lpqEPt3lDM9BTnuSVpslfdHLIcExtGh1t4bhipLG2Tetud8v +Fqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MwX8tXxSDd35fuy08I41Y0xncP6DuvbsklPwCAUTvwA=; b=oq9oXS0voZg9PXUU8R0uKQjBcYOCghbzQxu0mqc83pFLn1LJ987mV9Km5NxeU3U7JW Tnwg5Ow5UHoIkVF/6YFP5FprGN3LBK+rP/bgqr28nCQAG1gpCOH1Gb0xNVW+2mAAEYD/ iZGQ7pL8E2Po3zcoXk00Zk7ypHA7sl7pMHZEFfDVg+H5syOzJqWBSSWHcSJkAPqetz24 fToOq7p0QvkIHk9ui4R5MEicntEa5m5FqhfqLXmV9Ufn/S+55elVXFYwdE6mcpbvqBGu 7MjELj2H4H0UhivacZJlSvIQUcSizKvP8X/Pwc3oxUULwlqdRa5AhDe+OUG5rPtbfgDa B4dg== X-Gm-Message-State: APjAAAVGR6MfT74Z89Rlpp6Px/bWzrhfeCw6u1NNef2xvYkKxcjYrrLz 8jMo4CMT0B/dECj8w6Hyhg9tHZAbmi4= X-Google-Smtp-Source: APXvYqz6XTm6Q7cEUEXsTcSRzl3A9hx23I9QsHtea3slJvyXe3SiKJcRDeHVXLRku/Y1UDxsdZE6mA== X-Received: by 2002:a63:364f:: with SMTP id d76mr5175232pga.215.1579884990017; Fri, 24 Jan 2020 08:56:30 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH 5/8] tests/vm: Added configuration file support Date: Fri, 24 Jan 2020 11:53:32 -0500 Message-Id: <20200124165335.422-6-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200124165335.422-1-robert.foley@linaro.org> References: <20200124165335.422-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::544 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Changes to tests/vm/basevm.py to allow accepting a configuration file as a parameter. Allows for specifying VM options such as cpu, machine, memory, and arbitrary qemu arguments for specifying options such as NUMA configuration. Also added an example config_example.yml. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov --- tests/vm/basevm.py | 60 +++++++++++++++++++++++++++++++++++++ tests/vm/config_example.yml | 52 ++++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) create mode 100644 tests/vm/config_example.yml diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index ec92c8f105..08a8989ac0 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -31,6 +31,7 @@ import tempfile import shutil import multiprocessing import traceback +import yaml =20 SSH_KEY_FILE =3D os.path.join(os.path.dirname(__file__), "..", "keys", "id_rsa") @@ -396,6 +397,61 @@ class BaseVM(object): def qmp(self, *args, **kwargs): return self._guest.qmp(*args, **kwargs) =20 + +def parse_config(config, args): + """ Parse yaml config and populate our config structure. + The yaml config allows the user to override the + defaults for VM parameters. In many cases these + defaults can be overridden without rebuilding the VM.""" + if args.config: + config_file =3D args.config + elif 'QEMU_CONFIG' in os.environ: + config_file =3D os.environ['QEMU_CONFIG'] + else: + return config + if not os.path.exists(config_file): + raise Exception("config file {} does not exist".format(config_file= )) + with open(config_file) as f: + yaml_dict =3D yaml.safe_load(f) + if 'target-conf' in yaml_dict: + target_dict =3D yaml_dict['target-conf'] + if 'username' in target_dict and target_dict['username'] !=3D 'roo= t': + config['guest_user'] =3D target_dict['username'] + if 'password' in target_dict: + config['root_pass'] =3D target_dict['password'] + config['guest_pass'] =3D target_dict['password'] + if any (k in target_dict for k in ("ssh_key","ssh_pub_key")) and \ + not all (k in target_dict for k in ("ssh_key","ssh_pub_key")): + missing_key =3D "ssh_pub_key" \ + if 'ssh_key' in target_dict else "ssh_key" + raise Exception("both ssh_key and ssh_pub_key required. " + "{} key is missing.".format(missing_key)) + if 'ssh_key' in target_dict: + config['ssh_key_file'] =3D target_dict['ssh_key'] + if not os.path.exists(config['ssh_key_file']): + raise Exception("ssh key file not found.") + if 'ssh_pub_key' in target_dict: + config['ssh_pub_key_file'] =3D target_dict['ssh_pub_key'] + if not os.path.exists(config['ssh_pub_key_file']): + raise Exception("ssh pub key file not found.") + if 'machine' in target_dict: + config['machine'] =3D target_dict['machine'] + if 'qemu_args' in target_dict: + qemu_args =3D target_dict['qemu_args'] + qemu_args =3D qemu_args.replace('\n', ' ').replace('\r', '') + config['extra_args'] =3D qemu_args.split(' ') + if 'memory' in target_dict: + config['memory'] =3D target_dict['memory'] + if 'dns' in target_dict: + config['dns'] =3D target_dict['dns'] + if 'cpu' in target_dict: + config['cpu'] =3D target_dict['cpu'] + if 'ssh_port' in target_dict: + config['ssh_port'] =3D target_dict['ssh_port'] + if 'install_cmds' in target_dict: + config['install_cmds'] =3D target_dict['install_cmds'] + return config + def parse_args(vmcls): =20 def get_default_jobs(): @@ -430,6 +486,9 @@ def parse_args(vmcls): help=3D"Interactively run command") parser.add_option("--snapshot", "-s", action=3D"store_true", help=3D"run tests with a snapshot") + parser.add_option("--config", "-c", default=3DNone, + help=3D"Provide config yaml for configuration. "\ + "See config_example.yaml for example.") parser.disable_interspersed_args() return parser.parse_args() =20 @@ -441,6 +500,7 @@ def main(vmcls, config=3DNone): if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 + config =3D parse_config(config, args) logging.basicConfig(level=3D(logging.DEBUG if args.debug else logging.WARN)) vm =3D vmcls(debug=3Dargs.debug, vcpus=3Dargs.jobs, config=3Dconfi= g) diff --git a/tests/vm/config_example.yml b/tests/vm/config_example.yml new file mode 100644 index 0000000000..0a1fec3824 --- /dev/null +++ b/tests/vm/config_example.yml @@ -0,0 +1,52 @@ +# +# Example yaml for use by any of the scripts in tests/vm. +# Can be provided as an environment variable QEMU_CONFIG +# +target-conf: + + # If any of the below are not provided, we will just use the qemu defa= ults. + + # Login username (has to be sudo enabled) + #username: qemu + + # Password is used by root and default login user. + #password: "qemupass" + + # If one key is provided, both must be provided. + #ssh_key: /complete/path/of/your/keyfile/id_rsa + #ssh_pub_key: /complete/path/of/your/keyfile/id_rsa.pub + + cpu: max + machine: virt,gic_version=3D3 + memory: 16G + + # The below is an example for how to configure NUMA topology with + # 4 NUMA nodes and 2 different NUMA distances. + qemu_args: "-smp cpus=3D16,sockets=3D2,cores=3D8 + -object memory-backend-ram,size=3D4G,policy=3Dbind,host-no= des=3D0,id=3Dram-node0 + -object memory-backend-ram,size=3D4G,policy=3Dbind,host-no= des=3D0,id=3Dram-node1 + -object memory-backend-ram,size=3D4G,policy=3Dbind,host-no= des=3D1,id=3Dram-node2 + -object memory-backend-ram,size=3D4G,policy=3Dbind,host-no= des=3D1,id=3Dram-node3 + -numa node,memdev=3Dram-node0,cpus=3D0-3,nodeid=3D0 -numa = node,memdev=3Dram-node1,cpus=3D4-7,nodeid=3D1 + -numa node,memdev=3Dram-node2,cpus=3D8-11,nodeid=3D2 -numa= node,memdev=3Dram-node3,cpus=3D12-15,nodeid=3D3 + -numa dist,src=3D0,dst=3D1,val=3D15 -numa dist,src=3D2,dst= =3D3,val=3D15 + -numa dist,src=3D0,dst=3D2,val=3D20 -numa dist,src=3D0,dst= =3D3,val=3D20 + -numa dist,src=3D1,dst=3D2,val=3D20 -numa dist,src=3D1,dst= =3D3,val=3D20" + + # By default we do not set the DNS. + # You override the defaults by setting the below. + #dns: 1.234.567.89 + + # By default we will use a "block" device, but + # you can also boot from a "scsi" device. + # Just keep in mind your scripts might need to change + # As you will have /dev/sda instead of /dev/vda (for block device) + #boot_dev_type: "scsi" + + # By default the ssh port is not fixed. + # A fixed ssh port makes it easier for automated tests. + #ssh_port: 5555 + + # To install a different set of packages, provide a command to issue + #install_cmds: "apt-get update ; apt-get build-dep -y qemu" + --=20 2.17.1