From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845342344139.59792168331285; Tue, 17 Jul 2018 09:35:42 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6319930001EE; Tue, 17 Jul 2018 16:35:40 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CDC3820155E0; Tue, 17 Jul 2018 16:35:39 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id DCB2718037F0; Tue, 17 Jul 2018 16:35:37 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZZZU021032 for ; Tue, 17 Jul 2018 12:35:36 -0400 Received: by smtp.corp.redhat.com (Postfix) id B6D8F16878; Tue, 17 Jul 2018 16:35:35 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3738E16875 for ; Tue, 17 Jul 2018 16:35:35 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:17 +0200 Message-Id: <20180717163528.4591-2-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 01/12] lcitool: Drop shell implementation X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.25 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Tue, 17 Jul 2018 16:35:41 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" We're going to rewrite the script completely, and getting the current implementation out of the way firts will make the diffs more reasonable. Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 227 ------------------------------------------------- 1 file changed, 227 deletions(-) delete mode 100755 guests/lcitool diff --git a/guests/lcitool b/guests/lcitool deleted file mode 100755 index 0c1520e..0000000 --- a/guests/lcitool +++ /dev/null @@ -1,227 +0,0 @@ -#!/bin/sh - -# ------------------- -# Utility functions -# ------------------- - -# die MESSAGE -# -# Abort the program after displaying $MESSAGE on standard error. -die() { - echo "$1" >&2 - exit 1 -} - -# hash_file PASS_FILE -# -# Generate a password hash from the contents of PASS_FILE. -hash_file() { - PASS_FILE=3D"$1" - - perl -le ' - my @chars =3D ("A".."Z", "a".."z", "0".."9"); - my $salt; $salt .=3D $chars[rand @chars] for 1..16; - my $handle; open($handle, "'"$PASS_FILE"'"); - my $pass =3D <$handle>; chomp($pass); - print crypt("$pass", "\$6\$$salt\$");' -} - -# yaml_var FILE VAR -# -# Read $FILE and output the value of YAML variable $VAR. Only trivial YAML -# values are supported, eg. strings and numbers that don't depend on the -# value of other variables. That's enough for our use case. -yaml_var() { - grep "^$2:\\s*" "$1" 2>/dev/null | tail -1 | sed "s/$2:\\s*//g" -} - -# load_install_config FILE -# -# Read all known configuration variables from $FILE and set them in the -# environment. Configuration variables that have already been set in -# the environment will not be updated. -load_install_config() { - INSTALL_URL=3D${INSTALL_URL:-$(yaml_var "$1" install_url)} - INSTALL_CONFIG=3D${INSTALL_CONFIG:-$(yaml_var "$1" install_config)} - INSTALL_VIRT_TYPE=3D${INSTALL_VIRT_TYPE:-$(yaml_var "$1" install_virt_= type)} - INSTALL_ARCH=3D${INSTALL_ARCH:-$(yaml_var "$1" install_arch)} - INSTALL_MACHINE=3D${INSTALL_MACHINE:-$(yaml_var "$1" install_machine)} - INSTALL_CPU_MODEL=3D${INSTALL_CPU_MODEL:-$(yaml_var "$1" install_cpu_m= odel)} - INSTALL_VCPUS=3D${INSTALL_VCPUS:-$(yaml_var "$1" install_vcpus)} - INSTALL_MEMORY_SIZE=3D${INSTALL_MEMORY_SIZE:-$(yaml_var "$1" install_m= emory_size)} - INSTALL_DISK_SIZE=3D${INSTALL_DISK_SIZE:-$(yaml_var "$1" install_disk_= size)} - INSTALL_STORAGE_POOL=3D${INSTALL_STORAGE_POOL:-$(yaml_var "$1" install= _storage_pool)} - INSTALL_NETWORK=3D${INSTALL_NETWORK:-$(yaml_var "$1" install_network)} -} - -# load_config -# -# Read tool configuration and perform the necessary validation. -load_config() { - CONFIG_DIR=3D"$HOME/.config/$PROGRAM_NAME" - - mkdir -p "$CONFIG_DIR" >/dev/null 2>&1 || { - die "$PROGRAM_NAME: $CONFIG_DIR: Unable to create config directory" - } - - FLAVOR_FILE=3D"$CONFIG_DIR/flavor" - VAULT_PASS_FILE=3D"$CONFIG_DIR/vault-password" - ROOT_PASS_FILE=3D"$CONFIG_DIR/root-password" - - # Two flavors are supported: test (default) and jenkins. Read the - # flavor from configuration, validate it and write it back in case - # it was not present - FLAVOR=3D"$(cat "$FLAVOR_FILE" 2>/dev/null)" - FLAVOR=3D${FLAVOR:-test} - test "$FLAVOR" =3D test || test "$FLAVOR" =3D jenkins || { - die "$PROGRAM_NAME: Invalid flavor '$FLAVOR'" - } - echo "$FLAVOR" >"$FLAVOR_FILE" || { - die "$PROGRAM_NAME: $FLAVOR_FILE: Unable to save flavor" - } - - test "$FLAVOR" =3D jenkins && { - # The vault password is only needed for the jenkins flavor, so only - # validate it in that case - test -f "$VAULT_PASS_FILE" && test "$(cat "$VAULT_PASS_FILE")" || { - die "$PROGRAM_NAME: $VAULT_PASS_FILE: Missing or invalid passw= ord" - } - } || { - # For other flavors, undefine the variable so that Ansible - # will not try to read the file at all - VAULT_PASS_FILE=3D - } - - # Make sure the root password has been configured properly - test -f "$ROOT_PASS_FILE" && test "$(cat "$ROOT_PASS_FILE")" || { - die "$PROGRAM_NAME: $ROOT_PASS_FILE: Missing or invalid password" - } - - ROOT_HASH_FILE=3D"$CONFIG_DIR/.root-password.hash" - - # Regenerate root password hash. Ansible expects passwords as hashes b= ut - # doesn't provide a built-in facility to generate one from plain text - hash_file "$ROOT_PASS_FILE" >"$ROOT_HASH_FILE" || { - die "$PROGRAM_NAME: Failure while hashing root password" - } -} - -# ---------------------- -# User-visible actions -# ---------------------- - -do_help() { - echo "\ -Usage: $CALL_NAME ACTION [OPTIONS] - -Actions: - list List known guests - install GUEST Install GUEST - prepare GUEST|all Prepare or update GUEST. Can be run multiple times - update GUEST|all Alias for prepare - help Display this help" -} - -do_list() { - # List all guests present in the inventory. Skip group names, - # comments and empty lines - grep -vE '^#|^\[|^$' inventory | sort -u -} - -do_install() -{ - GUEST=3D"$1" - - test "$GUEST" || { - die "$(do_help)" - } - do_list | grep -q "$GUEST" || { - die "$PROGRAM_NAME: $GUEST: Unknown guest" - } - test -f "host_vars/$GUEST/install.yml" || { - die "$PROGRAM_NAME: $GUEST: Missing configuration, guest must be i= nstalled manually" - } - - load_config - - # Load configuration files. Values don't get overwritten after being - # set the first time, so loading the host-specific configuration before - # the group configuration ensures overrides work as expected - load_install_config "host_vars/$GUEST/install.yml" - load_install_config "group_vars/all/install.yml" - - # Both memory size and disk size use GiB as unit, but virt-install wan= ts - # disk size in GiB and memory size in *MiB*, so perform conversion here - INSTALL_MEMORY_SIZE=3D$(expr "$INSTALL_MEMORY_SIZE" \* 1024 2>/dev/nul= l) - - # preseed files must use a well-known name to be picked up by d-i; - # for kickstart files, we can use whatever name we please but we need - # to point anaconda in the right direction through a kernel argument - case "$INSTALL_CONFIG" in - *kickstart*|*ks*) EXTRA_ARGS=3D"ks=3Dfile:/${INSTALL_CONFIG##*/}" = ;; - esac - - # Only configure autostart for the guest for the jenkins flavor - test "$FLAVOR" =3D jenkins && { - AUTOSTART=3D"--autostart" - } - - virt-install \ - --name "$GUEST" \ - --location "$INSTALL_URL" \ - --virt-type "$INSTALL_VIRT_TYPE" \ - --arch "$INSTALL_ARCH" \ - --machine "$INSTALL_MACHINE" \ - --cpu "$INSTALL_CPU_MODEL" \ - --vcpus "$INSTALL_VCPUS" \ - --memory "$INSTALL_MEMORY_SIZE" \ - --disk "size=3D$INSTALL_DISK_SIZE,pool=3D$INSTALL_STORAGE_POOL,bus= =3Dvirtio" \ - --network "network=3D$INSTALL_NETWORK,model=3Dvirtio" \ - --graphics none \ - --console pty \ - --sound none \ - --initrd-inject "$INSTALL_CONFIG" \ - --extra-args "console=3DttyS0 $EXTRA_ARGS" \ - $AUTOSTART \ - --wait 0 -} - -do_prepare() { - GUEST=3D"$1" - - test "$GUEST" || { - die "$(do_help)" - } - do_list | grep -q "$GUEST" || test "$GUEST" =3D all || { - die "$PROGRAM_NAME: $GUEST: Unknown guest" - } - - load_config - - EXTRA_VARS=3D"flavor=3D$FLAVOR root_password_file=3D$ROOT_HASH_FILE" - - ansible-playbook \ - --vault-password-file "$VAULT_PASS_FILE" \ - --extra-vars "$EXTRA_VARS" \ - -l "$GUEST" \ - site.yml -} - -# --------------------- -# Program entry point -# --------------------- - -CALL_NAME=3D"$0" -PROGRAM_NAME=3D"${0##*/}" - -test -f "$PROGRAM_NAME" || { - die "$PROGRAM_NAME: Must be run from the source directory" -} - -case "$1" in - list) do_list ;; - install) do_install "$2" ;; - prepare|update) do_prepare "$2" ;; - *help) do_help ;; - *) die "$(do_help)" ;; -esac --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845354461568.0654061120824; Tue, 17 Jul 2018 09:35:54 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8EB8D30E6860; Tue, 17 Jul 2018 16:35:52 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 45CFC308BDB2; Tue, 17 Jul 2018 16:35:52 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B34664BB78; Tue, 17 Jul 2018 16:35:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZajH021039 for ; Tue, 17 Jul 2018 12:35:36 -0400 Received: by smtp.corp.redhat.com (Postfix) id 5DBEB16878; Tue, 17 Jul 2018 16:35:36 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F397C16875 for ; Tue, 17 Jul 2018 16:35:35 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:18 +0200 Message-Id: <20180717163528.4591-3-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 02/12] lcitool: Stub out Python implementation X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 17 Jul 2018 16:35:53 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Doesn't do much right now, but it's a start :) Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100755 guests/lcitool diff --git a/guests/lcitool b/guests/lcitool new file mode 100755 index 0000000..5ca885f --- /dev/null +++ b/guests/lcitool @@ -0,0 +1,46 @@ +#!/usr/bin/env python + +# lcitool - libvirt CI guest management tool +# Copyright (C) 2017-2018 Andrea Bolognani +# +# This program is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by the +# Free Software Foundation; either version 2 of the License, or (at your +# option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . + +import argparse +import sys + + +class Error(Exception): + + def __init__(self, message): + super(Error, self).__init__() + self.message =3D message + + +class Application: + + def __init__(self): + self._parser =3D argparse.ArgumentParser( + description=3D"libvirt CI guest management tool", + ) + + def run(self): + self._parser.parse_args() + + +if __name__ =3D=3D "__main__": + try: + Application().run() + except Error as err: + sys.stderr.write("{}: {}\n".format(sys.argv[0], err.message)) + sys.exit(1) --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845354810820.0088180998558; Tue, 17 Jul 2018 09:35:54 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7ACD1307D85A; Tue, 17 Jul 2018 16:35:52 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3C79F20155E0; Tue, 17 Jul 2018 16:35:52 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B3B1024F64; Tue, 17 Jul 2018 16:35:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZbxD021044 for ; Tue, 17 Jul 2018 12:35:37 -0400 Received: by smtp.corp.redhat.com (Postfix) id 06D5416875; Tue, 17 Jul 2018 16:35:37 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9B4331687B for ; Tue, 17 Jul 2018 16:35:36 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:19 +0200 Message-Id: <20180717163528.4591-4-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 03/12] lcitool: Add tool configuration handling X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.25 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Tue, 17 Jul 2018 16:35:53 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" The on-disk configuration format and its behavior are fully backwards compatible with the previous implementation. Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 115 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) diff --git a/guests/lcitool b/guests/lcitool index 5ca885f..70dcaff 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -17,6 +17,10 @@ # with this program. If not, see . =20 import argparse +import crypt +import os +import random +import string import sys =20 =20 @@ -27,9 +31,120 @@ class Error(Exception): self.message =3D message =20 =20 +class Util: + + @staticmethod + def mksalt(): + alphabeth =3D string.ascii_letters + string.digits + salt =3D "".join(random.choice(alphabeth) for x in range(0, 16)) + return "$6${}$".format(salt) + + +class Config: + + @staticmethod + def _get_config_file(name): + try: + config_dir =3D os.environ["XDG_CONFIG_HOME"] + except KeyError: + config_dir =3D os.path.join(os.environ["HOME"], ".config/") + config_dir =3D os.path.join(config_dir, "lcitool/") + + # Create the directory if it doesn't already exist + if not os.path.exists(config_dir): + try: + os.mkdir(config_dir) + except Exception: + raise Error( + "Can't create configuration directory ({})".format( + config_dir, + ) + ) + + return os.path.join(config_dir, name) + + def get_flavor(self): + flavor_file =3D self._get_config_file("flavor") + + try: + with open(flavor_file, "r") as infile: + flavor =3D infile.readline().strip() + except Exception: + # If the flavor has not been configured, we choose the default + # and store it on disk to ensure consistent behavior from now = on + flavor =3D "test" + try: + with open(flavor_file, "w") as infile: + infile.write("{}\n".format(flavor)) + except Exception: + raise Error( + "Can't write flavor file ({})".format( + flavor_file, + ) + ) + + if flavor not in ["test", "jenkins"]: + raise Error("Invalid flavor '{}'".format(flavor)) + + return flavor + + def get_vault_password_file(self): + vault_pass_file =3D None + + # The vault password is only needed for the jenkins flavor, but in + # that case we want to make sure there's *something* in there + if self.get_flavor() !=3D "test": + vault_pass_file =3D self._get_config_file("vault-password") + + try: + with open(vault_pass_file, "r") as infile: + if not infile.readline().strip(): + raise ValueError + except Exception: + raise Error( + "Missing or invalid vault password file ({})".format( + vault_pass_file, + ) + ) + + return vault_pass_file + + def get_root_password_file(self): + root_pass_file =3D self._get_config_file("root-password") + root_hash_file =3D self._get_config_file(".root-password.hash") + + try: + with open(root_pass_file, "r") as infile: + root_pass =3D infile.readline().strip() + except Exception: + raise Error( + "Missing or invalid root password file ({})".format( + root_pass_file, + ) + ) + + # The hash will be different every time we run, but that doesn't + # matter - it will still validate the correct root password + root_hash =3D crypt.crypt(root_pass, Util.mksalt()) + + try: + with open(root_hash_file, "w") as infile: + infile.write("{}\n".format(root_hash)) + except Exception: + raise Error( + "Can't write hashed root password file ({})".format( + root_hash_file, + ) + ) + + return root_hash_file + + class Application: =20 def __init__(self): + self._config =3D Config() + self._parser =3D argparse.ArgumentParser( description=3D"libvirt CI guest management tool", ) --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845363499675.8987355748727; Tue, 17 Jul 2018 09:36:03 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1791930820E1; Tue, 17 Jul 2018 16:36:00 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A62C430012C2; Tue, 17 Jul 2018 16:35:59 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3296124F59; Tue, 17 Jul 2018 16:35:59 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZbKN021052 for ; Tue, 17 Jul 2018 12:35:37 -0400 Received: by smtp.corp.redhat.com (Postfix) id A333116875; Tue, 17 Jul 2018 16:35:37 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4469816878 for ; Tue, 17 Jul 2018 16:35:37 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:20 +0200 Message-Id: <20180717163528.4591-5-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 04/12] lcitool: Add inventory handling X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Tue, 17 Jul 2018 16:36:02 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" We use an actual YAML parser this time around, and bring the behavior more in line with what Ansible is doing, so interoperability should be more solid overall. New in this implementation is more flexibility in defining host lists, including support for explicit lists as well as glob patterns. Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) diff --git a/guests/lcitool b/guests/lcitool index 70dcaff..ffea969 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -18,10 +18,18 @@ =20 import argparse import crypt +import fnmatch import os import random import string import sys +import yaml + +# This is necessary to maintain Python 2.7 compatibility +try: + import configparser +except ImportError: + import ConfigParser as configparser =20 =20 class Error(Exception): @@ -39,6 +47,32 @@ class Util: salt =3D "".join(random.choice(alphabeth) for x in range(0, 16)) return "$6${}$".format(salt) =20 + @staticmethod + def expand_pattern(pattern, source, name): + if pattern is None: + raise Error("Missing {} list".format(name)) + + if pattern =3D=3D "all": + pattern =3D "*" + + # This works correctly for single items as well as more complex + # cases such as explicit lists, glob patterns and any combination + # of the above + matches =3D [] + for partial_pattern in pattern.split(","): + + partial_matches =3D [] + for item in source: + if fnmatch.fnmatch(item, partial_pattern): + partial_matches +=3D [item] + + if not partial_matches: + raise Error("Invalid {} list '{}'".format(name, pattern)) + + matches +=3D partial_matches + + return sorted(set(matches)) + =20 class Config: =20 @@ -140,10 +174,75 @@ class Config: return root_hash_file =20 =20 +class Inventory: + + def __init__(self): + try: + parser =3D configparser.SafeConfigParser() + parser.read("./ansible.cfg") + inventory_path =3D parser.get("defaults", "inventory") + except Exception: + raise Error("Can't find inventory location in ansible.cfg") + + self._facts =3D {} + try: + # We can only deal with trivial inventories, but that's + # all we need right now and we can expand support further + # later on if necessary + with open(inventory_path, "r") as infile: + for line in infile: + host =3D line.strip() + self._facts[host] =3D {} + except Exception: + raise Error( + "Missing or invalid inventory ({})".format( + inventory_path, + ) + ) + + for host in self._facts: + try: + self._facts[host] =3D self._read_all_facts(host) + self._facts[host]["inventory_hostname"] =3D host + except Exception: + raise Error("Can't load facts for '{}'".format(host)) + + @staticmethod + def _add_facts_from_file(facts, yaml_path): + with open(yaml_path, "r") as infile: + some_facts =3D yaml.load(infile) + for fact in some_facts: + facts[fact] =3D some_facts[fact] + + def _read_all_facts(self, host): + facts =3D {} + + # We load from group_vars/ first and host_vars/ second, sorting + # files alphabetically; doing so should result in our view of + # the facts matching Ansible's + for source in ["./group_vars/all/", "./host_vars/{}/".format(host)= ]: + for item in sorted(os.listdir(source)): + yaml_path =3D os.path.join(source, item) + if not os.path.isfile(yaml_path): + continue + if not yaml_path.endswith(".yml"): + continue + self._add_facts_from_file(facts, yaml_path) + + return facts + + def expand_pattern(self, pattern): + return Util.expand_pattern(pattern, self._facts, "host") + + def get_facts(self, host): + return self._facts[host] + + class Application: =20 def __init__(self): self._config =3D Config() + self._inventory =3D Inventory() =20 self._parser =3D argparse.ArgumentParser( description=3D"libvirt CI guest management tool", --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845343161527.0378820042379; Tue, 17 Jul 2018 09:35:43 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2A19C308403F; Tue, 17 Jul 2018 16:35:41 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D37CD5C1A1; Tue, 17 Jul 2018 16:35:40 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 7D77424F60; Tue, 17 Jul 2018 16:35:40 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZc2T021061 for ; Tue, 17 Jul 2018 12:35:38 -0400 Received: by smtp.corp.redhat.com (Postfix) id 4CC6E16875; Tue, 17 Jul 2018 16:35:38 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E0F8016878 for ; Tue, 17 Jul 2018 16:35:37 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:21 +0200 Message-Id: <20180717163528.4591-6-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 05/12] lcitool: Implement the 'hosts' action X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Tue, 17 Jul 2018 16:35:42 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This replaces the 'list' action from the original implementation. We're going to list more than just hosts over time, so a more specific name is warranted. Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/guests/lcitool b/guests/lcitool index ffea969..2df231a 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -23,6 +23,7 @@ import os import random import string import sys +import textwrap import yaml =20 # This is necessary to maintain Python 2.7 compatibility @@ -245,11 +246,34 @@ class Application: self._inventory =3D Inventory() =20 self._parser =3D argparse.ArgumentParser( + formatter_class=3Dargparse.RawDescriptionHelpFormatter, description=3D"libvirt CI guest management tool", + epilog=3Dtextwrap.dedent(""" + informational actions: + hosts list all known hosts + """), ) + self._parser.add_argument( + "-a", + metavar=3D"ACTION", + required=3DTrue, + help=3D"action to perform (see below)", + ) + + def _action_hosts(self): + for host in self._inventory.expand_pattern("all"): + print(host) =20 def run(self): - self._parser.parse_args() + cmdline =3D self._parser.parse_args() + action =3D cmdline.a + + method =3D "_action_{}".format(action.replace("-", "_")) + + if hasattr(self, method): + getattr(self, method).__call__() + else: + raise Error("Invalid action '{}'".format(action)) =20 =20 if __name__ =3D=3D "__main__": --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845370832705.8694622155169; Tue, 17 Jul 2018 09:36:10 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B3E7230820E6; Tue, 17 Jul 2018 16:36:07 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5E97A5D6A5; Tue, 17 Jul 2018 16:36:07 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 00E7718037FA; Tue, 17 Jul 2018 16:36:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZddA021066 for ; Tue, 17 Jul 2018 12:35:39 -0400 Received: by smtp.corp.redhat.com (Postfix) id E98E916875; Tue, 17 Jul 2018 16:35:38 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8A31016878 for ; Tue, 17 Jul 2018 16:35:38 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:22 +0200 Message-Id: <20180717163528.4591-7-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 06/12] lcitool: Implement the 'install' action X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Tue, 17 Jul 2018 16:36:10 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 75 ++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 73 insertions(+), 2 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 2df231a..5156869 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -22,6 +22,7 @@ import fnmatch import os import random import string +import subprocess import sys import textwrap import yaml @@ -246,11 +247,17 @@ class Application: self._inventory =3D Inventory() =20 self._parser =3D argparse.ArgumentParser( + conflict_handler=3D"resolve", formatter_class=3Dargparse.RawDescriptionHelpFormatter, description=3D"libvirt CI guest management tool", epilog=3Dtextwrap.dedent(""" + common actions: + install perform unattended host installation + informational actions: hosts list all known hosts + + glob patterns are supported for HOSTS """), ) self._parser.add_argument( @@ -259,19 +266,83 @@ class Application: required=3DTrue, help=3D"action to perform (see below)", ) + self._parser.add_argument( + "-h", + metavar=3D"HOSTS", + help=3D"list of hosts to act on", + ) =20 - def _action_hosts(self): + def _action_hosts(self, _hosts): for host in self._inventory.expand_pattern("all"): print(host) =20 + def _action_install(self, hosts): + flavor =3D self._config.get_flavor() + + for host in self._inventory.expand_pattern(hosts): + facts =3D self._inventory.get_facts(host) + + # Both memory size and disk size are stored as GiB in the + # inventory, but virt-install expects the disk size in GiB + # and the memory size in *MiB*, so perform conversion here + memory_arg =3D str(int(facts["install_memory_size"]) * 1024) + + vcpus_arg =3D str(facts["install_vcpus"]) + + disk_arg =3D "size=3D{},pool=3D{},bus=3Dvirtio".format( + facts["install_disk_size"], + facts["install_storage_pool"], + ) + network_arg =3D "network=3D{},model=3Dvirtio".format( + facts["install_network"], + ) + + # preseed files must use a well-known name to be picked up by + # d-i; for kickstart files, we can use whatever name we please + # but we need to point anaconda in the right direction through + # a kernel argument + extra_arg =3D "console=3DttyS0 ks=3Dfile:/{}".format( + facts["install_config"], + ) + + cmd =3D [ + "virt-install", + "--name", host, + "--location", facts["install_url"], + "--virt-type", facts["install_virt_type"], + "--arch", facts["install_arch"], + "--machine", facts["install_machine"], + "--cpu", facts["install_cpu_model"], + "--vcpus", vcpus_arg, + "--memory", memory_arg, + "--disk", disk_arg, + "--network", network_arg, + "--graphics", "none", + "--console", "pty", + "--sound", "none", + "--initrd-inject", facts["install_config"], + "--extra-args", extra_arg, + "--wait", "0", + ] + + # Only configure autostart for the guest for the jenkins flavor + if flavor =3D=3D "jenkins": + cmd +=3D ["--autostart"] + + try: + subprocess.check_call(cmd) + except Exception: + raise Error("Failed to install '{}'".format(host)) + def run(self): cmdline =3D self._parser.parse_args() action =3D cmdline.a + hosts =3D cmdline.h =20 method =3D "_action_{}".format(action.replace("-", "_")) =20 if hasattr(self, method): - getattr(self, method).__call__() + getattr(self, method).__call__(hosts) else: raise Error("Invalid action '{}'".format(action)) =20 --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845375840580.1161385376342; Tue, 17 Jul 2018 09:36:15 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F0FA9308A970; Tue, 17 Jul 2018 16:36:13 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A68FB5D6B5; Tue, 17 Jul 2018 16:36:13 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 32B6D18037F4; Tue, 17 Jul 2018 16:36:13 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZdVT021074 for ; Tue, 17 Jul 2018 12:35:39 -0400 Received: by smtp.corp.redhat.com (Postfix) id 91C6D16875; Tue, 17 Jul 2018 16:35:39 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 326381687B for ; Tue, 17 Jul 2018 16:35:39 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:23 +0200 Message-Id: <20180717163528.4591-8-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 07/12] lcitool: Implement the 'update' action X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Tue, 17 Jul 2018 16:36:15 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" The 'prepare' alias was kinda redundant and offered dubious value, so it has been dropped. Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/guests/lcitool b/guests/lcitool index 5156869..ba5ed5d 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -253,6 +253,7 @@ class Application: epilog=3Dtextwrap.dedent(""" common actions: install perform unattended host installation + update prepare hosts and keep them updated =20 informational actions: hosts list all known hosts @@ -334,6 +335,35 @@ class Application: except Exception: raise Error("Failed to install '{}'".format(host)) =20 + def _action_update(self, hosts): + flavor =3D self._config.get_flavor() + vault_pass_file =3D self._config.get_vault_password_file() + root_pass_file =3D self._config.get_root_password_file() + + ansible_hosts =3D ",".join(self._inventory.expand_pattern(hosts)) + + extra_vars =3D "flavor=3D{} root_password_file=3D{}".format( + flavor, + root_pass_file, + ) + + cmd =3D ["ansible-playbook"] + + # Provide the vault password if available + if vault_pass_file is not None: + cmd +=3D ["--vault-password-file", vault_pass_file] + + cmd +=3D [ + "--limit", ansible_hosts, + "--extra-vars", extra_vars, + "./site.yml", + ] + + try: + subprocess.check_call(cmd) + except Exception: + raise Error("Failed to update '{}'".format(hosts)) + def run(self): cmdline =3D self._parser.parse_args() action =3D cmdline.a --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845381863843.6906481447072; Tue, 17 Jul 2018 09:36:21 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 04D9A4E358; Tue, 17 Jul 2018 16:36:20 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 97D0360175; Tue, 17 Jul 2018 16:36:19 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 460E418037ED; Tue, 17 Jul 2018 16:36:19 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZeOR021084 for ; Tue, 17 Jul 2018 12:35:40 -0400 Received: by smtp.corp.redhat.com (Postfix) id 3A66D16875; Tue, 17 Jul 2018 16:35:40 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CE47116878 for ; Tue, 17 Jul 2018 16:35:39 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:24 +0200 Message-Id: <20180717163528.4591-9-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 08/12] guests: Update documentation X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 17 Jul 2018 16:36:21 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/README.markdown | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/guests/README.markdown b/guests/README.markdown index bc780f3..4a40619 100644 --- a/guests/README.markdown +++ b/guests/README.markdown @@ -6,16 +6,16 @@ of the guests used by the Jenkins-based libvirt CI enviro= nment. =20 There are two steps to bringing up a guest: =20 -* `./lcitool install $guest` will perform an unattended installation +* `./lcitool -a install -h $guest` will perform an unattended installation of `$guest`. Not all guests can be installed this way: see the "FreeBSD" section below; =20 -* `./lcitool prepare $guest` will go through all the post-installation +* `./lcitool -a update -h $guest` will go through all the post-installation configuration steps required to make the newly-created guest usable; =20 Once those steps have been performed, maintainance will involve running: =20 -* `./lcitool update $guest` +* `./lcitool -a update -h $guest` =20 periodically to ensure the guest configuration is sane and all installed packages are updated. @@ -40,7 +40,7 @@ you'll want to use the `libvirt_guest` variant of the plu= gin. To keep guests up to date over time, it's recommended to have an entry along the lines of =20 - 0 0 * * * cd ~/libvirt-jenkins-ci/guests && ./lcitool update all + 0 0 * * * cd ~/libvirt-jenkins-ci/guests && ./lcitool -a update -h all =20 in your crontab. =20 --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845387410128.42184549290937; Tue, 17 Jul 2018 09:36:27 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2220D30E6870; Tue, 17 Jul 2018 16:36:26 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D888360172; Tue, 17 Jul 2018 16:36:25 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 7086D18037F1; Tue, 17 Jul 2018 16:36:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZeWu021094 for ; Tue, 17 Jul 2018 12:35:41 -0400 Received: by smtp.corp.redhat.com (Postfix) id D57EB16875; Tue, 17 Jul 2018 16:35:40 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 76C4916878 for ; Tue, 17 Jul 2018 16:35:40 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:25 +0200 Message-Id: <20180717163528.4591-10-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 09/12] guests: Add Docker-related information to the inventory X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 17 Jul 2018 16:36:26 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/host_vars/libvirt-centos-7/docker.yml | 2 ++ guests/host_vars/libvirt-debian-8/docker.yml | 2 ++ guests/host_vars/libvirt-debian-9/docker.yml | 2 ++ guests/host_vars/libvirt-debian-sid/docker.yml | 2 ++ guests/host_vars/libvirt-fedora-27/docker.yml | 2 ++ guests/host_vars/libvirt-fedora-28/docker.yml | 2 ++ guests/host_vars/libvirt-fedora-rawhide/docker.yml | 2 ++ guests/host_vars/libvirt-ubuntu-16/docker.yml | 2 ++ guests/host_vars/libvirt-ubuntu-18/docker.yml | 2 ++ 9 files changed, 18 insertions(+) create mode 100644 guests/host_vars/libvirt-centos-7/docker.yml create mode 100644 guests/host_vars/libvirt-debian-8/docker.yml create mode 100644 guests/host_vars/libvirt-debian-9/docker.yml create mode 100644 guests/host_vars/libvirt-debian-sid/docker.yml create mode 100644 guests/host_vars/libvirt-fedora-27/docker.yml create mode 100644 guests/host_vars/libvirt-fedora-28/docker.yml create mode 100644 guests/host_vars/libvirt-fedora-rawhide/docker.yml create mode 100644 guests/host_vars/libvirt-ubuntu-16/docker.yml create mode 100644 guests/host_vars/libvirt-ubuntu-18/docker.yml diff --git a/guests/host_vars/libvirt-centos-7/docker.yml b/guests/host_var= s/libvirt-centos-7/docker.yml new file mode 100644 index 0000000..59f7f12 --- /dev/null +++ b/guests/host_vars/libvirt-centos-7/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: centos:centos7 diff --git a/guests/host_vars/libvirt-debian-8/docker.yml b/guests/host_var= s/libvirt-debian-8/docker.yml new file mode 100644 index 0000000..235f0fd --- /dev/null +++ b/guests/host_vars/libvirt-debian-8/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: debian:8 diff --git a/guests/host_vars/libvirt-debian-9/docker.yml b/guests/host_var= s/libvirt-debian-9/docker.yml new file mode 100644 index 0000000..0b4ccee --- /dev/null +++ b/guests/host_vars/libvirt-debian-9/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: debian:9 diff --git a/guests/host_vars/libvirt-debian-sid/docker.yml b/guests/host_v= ars/libvirt-debian-sid/docker.yml new file mode 100644 index 0000000..e20a37e --- /dev/null +++ b/guests/host_vars/libvirt-debian-sid/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: debian:sid diff --git a/guests/host_vars/libvirt-fedora-27/docker.yml b/guests/host_va= rs/libvirt-fedora-27/docker.yml new file mode 100644 index 0000000..dcada18 --- /dev/null +++ b/guests/host_vars/libvirt-fedora-27/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: fedora:27 diff --git a/guests/host_vars/libvirt-fedora-28/docker.yml b/guests/host_va= rs/libvirt-fedora-28/docker.yml new file mode 100644 index 0000000..a874018 --- /dev/null +++ b/guests/host_vars/libvirt-fedora-28/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: fedora:28 diff --git a/guests/host_vars/libvirt-fedora-rawhide/docker.yml b/guests/ho= st_vars/libvirt-fedora-rawhide/docker.yml new file mode 100644 index 0000000..39dda1c --- /dev/null +++ b/guests/host_vars/libvirt-fedora-rawhide/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: fedora:rawhide diff --git a/guests/host_vars/libvirt-ubuntu-16/docker.yml b/guests/host_va= rs/libvirt-ubuntu-16/docker.yml new file mode 100644 index 0000000..2d4eb25 --- /dev/null +++ b/guests/host_vars/libvirt-ubuntu-16/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: ubuntu:16.04 diff --git a/guests/host_vars/libvirt-ubuntu-18/docker.yml b/guests/host_va= rs/libvirt-ubuntu-18/docker.yml new file mode 100644 index 0000000..13d6cc1 --- /dev/null +++ b/guests/host_vars/libvirt-ubuntu-18/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: ubuntu:18.04 --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845367035859.9982683754479; Tue, 17 Jul 2018 09:36:07 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 58FFE3680A; Tue, 17 Jul 2018 16:36:03 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E652B10694C0; Tue, 17 Jul 2018 16:36:02 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6D85E24F5C; Tue, 17 Jul 2018 16:36:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZfOx021099 for ; Tue, 17 Jul 2018 12:35:41 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7D28616875; Tue, 17 Jul 2018 16:35:41 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1EA5716878 for ; Tue, 17 Jul 2018 16:35:40 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:26 +0200 Message-Id: <20180717163528.4591-11-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 10/12] lcitool: Add projects information handling X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 17 Jul 2018 16:36:05 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" The original tool's limited scope meant loading this information was not needed, but we're going to start making use of it pretty soon. Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/guests/lcitool b/guests/lcitool index ba5ed5d..95d16b3 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -240,11 +240,58 @@ class Inventory: return self._facts[host] =20 =20 +class Projects: + + def __init__(self): + try: + with open("./vars/mappings.yml", "r") as infile: + mappings =3D yaml.load(infile) + self._mappings =3D mappings["mappings"] + except Exception: + raise Error("Can't load mappings") + + self._packages =3D {} + source =3D "./vars/projects/" + for item in os.listdir(source): + yaml_path =3D os.path.join(source, item) + if not os.path.isfile(yaml_path): + continue + if not yaml_path.endswith(".yml"): + continue + + project =3D os.path.splitext(item)[0] + + try: + with open(yaml_path, "r") as infile: + packages =3D yaml.load(infile) + self._packages[project] =3D packages["packages"] + except Exception: + raise Error("Can't load packages for '{}'".format(project)) + + def expand_pattern(self, pattern): + projects =3D Util.expand_pattern(pattern, self._packages, "project= ") + + # Some projects are internal implementation details and should + # not be exposed to the user + for project in ["base", "blacklist", "jenkins"]: + if project in projects: + projects.remove(project) + + return projects + + def get_mappings(self): + return self._mappings + + def get_packages(self, project): + return self._packages[project] + + class Application: =20 def __init__(self): self._config =3D Config() self._inventory =3D Inventory() + self._projects =3D Projects() =20 self._parser =3D argparse.ArgumentParser( conflict_handler=3D"resolve", --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845354138918.5374297516692; Tue, 17 Jul 2018 09:35:54 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5D3453084036; Tue, 17 Jul 2018 16:35:52 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 195CC5D6A5; Tue, 17 Jul 2018 16:35:52 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id AE8D718037F1; Tue, 17 Jul 2018 16:35:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZgsD021109 for ; Tue, 17 Jul 2018 12:35:42 -0400 Received: by smtp.corp.redhat.com (Postfix) id 2500116875; Tue, 17 Jul 2018 16:35:42 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BA8641687B for ; Tue, 17 Jul 2018 16:35:41 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:27 +0200 Message-Id: <20180717163528.4591-12-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 11/12] lcitool: Implement the 'projects' action X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Tue, 17 Jul 2018 16:35:53 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/guests/lcitool b/guests/lcitool index 95d16b3..cfea077 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -303,7 +303,8 @@ class Application: update prepare hosts and keep them updated =20 informational actions: - hosts list all known hosts + hosts list all known hosts + projects list all known projects =20 glob patterns are supported for HOSTS """), @@ -324,6 +325,10 @@ class Application: for host in self._inventory.expand_pattern("all"): print(host) =20 + def _action_projects(self, _hosts): + for project in self._projects.expand_pattern("all"): + print(project) + def _action_install(self, hosts): flavor =3D self._config.get_flavor() =20 --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri May 3 17:31:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1531845361118259.9895794948761; Tue, 17 Jul 2018 09:36:01 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 84D4D8110F; Tue, 17 Jul 2018 16:35:58 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 22732308BDB2; Tue, 17 Jul 2018 16:35:58 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9F90E18037EC; Tue, 17 Jul 2018 16:35:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w6HGZgJ2021116 for ; Tue, 17 Jul 2018 12:35:43 -0400 Received: by smtp.corp.redhat.com (Postfix) id C100B16875; Tue, 17 Jul 2018 16:35:42 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 61D5D16878 for ; Tue, 17 Jul 2018 16:35:42 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Tue, 17 Jul 2018 18:35:28 +0200 Message-Id: <20180717163528.4591-13-abologna@redhat.com> In-Reply-To: <20180717163528.4591-1-abologna@redhat.com> References: <20180717163528.4591-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 12/12] lcitool: Implement the 'dockerfile' action X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 17 Jul 2018 16:36:00 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This is basically the exact same algorithm used by the Ansible playbooks to process package mappings, implemented in pure Python. Signed-off-by: Andrea Bolognani Reviewed-by: Katerina Koukiou --- guests/lcitool | 86 ++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 80 insertions(+), 6 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index cfea077..22b08dd 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -306,7 +306,10 @@ class Application: hosts list all known hosts projects list all known projects =20 - glob patterns are supported for HOSTS + uncommon actions: + dockerfile generate Dockerfile (doesn't access the host) + + glob patterns are supported for HOSTS and PROJECTS """), ) self._parser.add_argument( @@ -320,16 +323,21 @@ class Application: metavar=3D"HOSTS", help=3D"list of hosts to act on", ) + self._parser.add_argument( + "-p", + metavar=3D"PROJECTS", + help=3D"list of projects to consider", + ) =20 - def _action_hosts(self, _hosts): + def _action_hosts(self, _hosts, _projects): for host in self._inventory.expand_pattern("all"): print(host) =20 - def _action_projects(self, _hosts): + def _action_projects(self, _hosts, _projects): for project in self._projects.expand_pattern("all"): print(project) =20 - def _action_install(self, hosts): + def _action_install(self, hosts, _projects): flavor =3D self._config.get_flavor() =20 for host in self._inventory.expand_pattern(hosts): @@ -387,7 +395,7 @@ class Application: except Exception: raise Error("Failed to install '{}'".format(host)) =20 - def _action_update(self, hosts): + def _action_update(self, hosts, _projects): flavor =3D self._config.get_flavor() vault_pass_file =3D self._config.get_vault_password_file() root_pass_file =3D self._config.get_root_password_file() @@ -416,15 +424,81 @@ class Application: except Exception: raise Error("Failed to update '{}'".format(hosts)) =20 + def _action_dockerfile(self, hosts, projects): + mappings =3D self._projects.get_mappings() + + hosts =3D self._inventory.expand_pattern(hosts) + if len(hosts) > 1: + raise Error("Can't generate Dockerfile for multiple hosts") + host =3D hosts[0] + + facts =3D self._inventory.get_facts(host) + package_format =3D facts["package_format"] + os_name =3D facts["os_name"] + os_full =3D os_name + str(facts["os_version"]) + + if package_format not in ["deb", "rpm"]: + raise Error("Host {} doesn't support Dockerfiles".format(host)) + + projects =3D self._projects.expand_pattern(projects) + for project in projects: + if project not in facts["projects"]: + raise Error( + "Host {} doesn't support project {}".format( + host, + project, + ) + ) + + temp =3D {} + + # We need to add the base project manually here: the standard + # machinery hides it because it's an implementation detail + for project in projects + ["base"]: + for package in self._projects.get_packages(project): + if "default" in mappings[package]: + temp[package] =3D mappings[package]["default"] + if package_format in mappings[package]: + temp[package] =3D mappings[package][package_format] + if os_name in mappings[package]: + temp[package] =3D mappings[package][os_name] + if os_full in mappings[package]: + temp[package] =3D mappings[package][os_full] + + flattened =3D [] + for item in temp: + if temp[item] is not None and temp[item] not in flattened: + flattened +=3D [temp[item]] + + print("FROM {}".format(facts["docker_base"])) + + sys.stdout.write("ENV PACKAGES ") + sys.stdout.write(" \\\n ".join(sorted(flattened))) + + if package_format =3D=3D "deb": + sys.stdout.write(textwrap.dedent(""" + RUN apt-get update && \\ + apt-get install -y ${PACKAGES} && \\ + apt-get autoremove -y && \\ + apt-get autoclean -y + """)) + elif package_format =3D=3D "rpm": + sys.stdout.write(textwrap.dedent(""" + RUN yum install -y ${PACKAGES} && \\ + yum autoremove -y && \\ + yum clean all -y + """)) + def run(self): cmdline =3D self._parser.parse_args() action =3D cmdline.a hosts =3D cmdline.h + projects =3D cmdline.p =20 method =3D "_action_{}".format(action.replace("-", "_")) =20 if hasattr(self, method): - getattr(self, method).__call__(hosts) + getattr(self, method).__call__(hosts, projects) else: raise Error("Invalid action '{}'".format(action)) =20 --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list