From nobody Tue Nov 26 17:39:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1570567737; cv=none; d=zoho.com; s=zohoarc; b=LEXaIpxrG5yWn3ohYbWc63XfY/cswWv/oJE91h8UtrvQ1e5/BCzazg5T7+MCngDFHosi2u1OowaXofDahFEWo1h9vWpyaRZOeAqAgPF8veVdKWa+a8NtdaInf7VuO6r1zOSRGMWzQ25PpZsPbZqiwOET0TN495t+QAeEh7+SZz4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1570567737; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=TQIqgYH6yHfi53pokgSDYnON4aX/9PmFDk4LoCtvkhc=; b=gqTpbY7PJb5SGcFQ4q1f4d//ZDxkAOi0kn1Wa6W9KyPeoI0QtV9I+MjK6WGIAOLA7e/Cm7iXdMwkuSN5FOC+mMfbYAhqvUp4YKT4O2cwu9f3o/YZgoxCLtFZ0OVcRG3HGuXvjz6KiRW3dCbacxNzdGxVPwicLFjD3Vd0SpdzvE8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1570567737914116.57254359318347; Tue, 8 Oct 2019 13:48:57 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5D6EC10CC206; Tue, 8 Oct 2019 20:48:56 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2F9D25D772; Tue, 8 Oct 2019 20:48:56 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id E85B54EE6E; Tue, 8 Oct 2019 20:48:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x98KmtcU024133 for ; Tue, 8 Oct 2019 16:48:55 -0400 Received: by smtp.corp.redhat.com (Postfix) id 37FC761559; Tue, 8 Oct 2019 20:48:55 +0000 (UTC) Received: from laerte.brq.redhat.com (unknown [10.43.2.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id F397F5D772; Tue, 8 Oct 2019 20:48:44 +0000 (UTC) From: =?UTF-8?q?Fabiano=20Fid=C3=AAncio?= To: libvir-list@redhat.com Date: Tue, 8 Oct 2019 22:48:18 +0200 Message-Id: <20191008204824.1323758-7-fidencio@redhat.com> In-Reply-To: <20191008204824.1323758-1-fidencio@redhat.com> References: <20191008204824.1323758-1-fidencio@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Cc: =?UTF-8?q?Fabiano=20Fid=C3=AAncio?= , abologna@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 06/12] guests: Install meson via pip X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.65]); Tue, 08 Oct 2019 20:48:56 +0000 (UTC) Let's install meson via pip whenever meson is not available at all or does not match the minimum required version. Signed-off-by: Fabiano Fid=C3=AAncio --- guests/lcitool | 25 ++++++++++++ guests/playbooks/update/tasks/packages.yml | 44 ++++++++++++++++++++++ guests/vars/mappings.yml | 17 +++++++++ 3 files changed, 86 insertions(+) diff --git a/guests/lcitool b/guests/lcitool index 5b5b622..2c08455 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -303,6 +303,7 @@ class Projects: with open(mappings_path, "r") as infile: mappings =3D yaml.safe_load(infile) self._mappings =3D mappings["mappings"] + self._pip_mappings =3D mappings["pip_mappings"] except Exception as ex: raise Exception("Can't load mappings: {}".format(ex)) =20 @@ -340,6 +341,9 @@ class Projects: def get_mappings(self): return self._mappings =20 + def get_pip_mappings(self): + return self._pip_mappings + def get_packages(self, project): return self._packages[project] =20 @@ -583,6 +587,7 @@ class Application: =20 def _action_dockerfile(self, args): mappings =3D self._projects.get_mappings() + pip_mappings =3D self._projects.get_pip_mappings() =20 hosts =3D self._inventory.expand_pattern(args.hosts) if len(hosts) > 1: @@ -617,6 +622,7 @@ class Application: =20 pkgs =3D {} cross_pkgs =3D {} + pip_pkgs =3D {} base_keys =3D ["default", package_format, os_name, os_full] cross_keys =3D [] if args.cross_arch: @@ -641,21 +647,28 @@ class Application: for key in keys: if key in mappings[package]: pkgs[package] =3D mappings[package][key] + if package in pip_mappings and key in pip_mappings[pac= kage]: + pip_pkgs[package] =3D pip_mappings[package][key] =20 if package not in pkgs: continue + if package in pip_pkgs and pkgs[package] is not None: + del pip_pkgs[package] if cross_policy =3D=3D "foreign" and pkgs[package] is not = None: cross_pkgs[package] =3D pkgs[package] if pkgs[package] is None or cross_policy in ["skip", "fore= ign"]: del pkgs[package] =20 pkg_align =3D " \\\n" + (" " * len("RUN " + package_manager + " ")) + pip_pkg_align =3D " \\\n" + (" " * len("RUN pip3 ")) =20 print("FROM {}".format(facts["docker_base"])) =20 varmap =3D {} varmap["package_manager"] =3D package_manager varmap["pkgs"] =3D pkg_align[1:] + pkg_align.join(sorted(set(pkgs.= values()))) + varmap["pip_pkgs"] =3D pip_pkg_align[1:] + pip_pkg_align.join(sort= ed(set(pip_pkgs.values()))) + if package_format =3D=3D "deb": if args.cross_arch: deb_arch =3D Util.native_arch_to_deb_arch(args.cross_arch) @@ -691,7 +704,14 @@ class Application: {package_manager} install --no-install-recommends = -y {cross_pkgs} && \\ {package_manager} autoremove -y && \\ {package_manager} autoclean -y + """).format(**varmap)) + + if pip_pkgs: + sys.stdout.write(textwrap.dedent(""" + RUN pip3 install {pip_pkgs} + """).format(**varmap)) =20 + sys.stdout.write(textwrap.dedent(""" ENV ABI "{cross_abi}" ENV CONFIGURE_OPTS "--host=3D{cross_abi} \\ --target=3D{cross_abi}" @@ -714,6 +734,11 @@ class Application: {package_manager} clean all -y """).format(**varmap)) =20 + if pip_pkgs: + sys.stdout.write(textwrap.dedent(""" + RUN pip3 install {pip_pkgs} + """).format(**varmap)) + def run(self): args =3D self._parser.parse_args() if args.debug: diff --git a/guests/playbooks/update/tasks/packages.yml b/guests/playbooks/= update/tasks/packages.yml index ec8a4c4..c041eab 100644 --- a/guests/playbooks/update/tasks/packages.yml +++ b/guests/playbooks/update/tasks/packages.yml @@ -100,3 +100,47 @@ package: name: '{{ flattened|sort }}' state: '{{ state }}' + +- set_fact: + pip_temp: {} + +- name: '{{ project }}: Verify pip mappings' + fail: + msg: 'No mappings defined for {{ item }}' + with_items: + '{{ packages }}' + when: + - pip_mappings[item] is undefined + +- name: '{{ project }}: Look up pip mappings (default)' + set_fact: + pip_temp: '{{ pip_temp|combine({ item: pip_mappings[item]["default"] }= ) }}' + with_items: + '{{ packages }}' + when: + - pip_mappings[item]["default"] is defined + +- set_fact: + pip_flattened: [] + +- name: '{{ project }}: Flatten pip package list' + set_fact: + pip_flattened: '{{ pip_flattened }} + [ "{{ pip_temp[item] }}" ]' + with_items: + '{{ temp }}' + when: + - pip_temp[item] !=3D None + - pip_temp[item] not in pip_flattened + +- name: '{{ project }}: Install packages from pip (state=3D{{ state }})' + pip: + name: '{{ pip_flattened|sort }}' + executable: pip3 + state: '{{ state }}' + with_items: + '{{ packages }}' + when: + - temp[item] is defined + - temp[item] =3D=3D None + - pip_temp[item] is defined + - pip_temp[item] !=3D None diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 524cff2..ba0f1cf 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -1007,3 +1007,20 @@ mappings: deb: zlib1g-dev rpm: zlib-static cross-policy-deb: foreign + + +# Pip mappings are used to map the generic packages name used to define +# projects to the specific package name and version which will be used when +# installing them via pip. +# +# pip packages' name are not different between different OSes, thus only +# the default "priority" is taken into account. +# +# In case we need to stick to a specific version of a pip package, it can +# be represented by adding "=3D=3D$version" to the pip package name, as do= ne +# with the meson package. + +pip_mappings: + + meson: + default: meson=3D=3D0.49.0 --=20 2.23.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list