From nobody Mon Feb 9 16:13:05 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1581355151790582.540397422154; Mon, 10 Feb 2020 09:19:11 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-366-xrc0ofPzN022ZJ2U6t7Mvg-1; Mon, 10 Feb 2020 12:18:53 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DA6A9108BD10; Mon, 10 Feb 2020 17:18:47 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B79955D9CA; Mon, 10 Feb 2020 17:18:47 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 70DA018089D6; Mon, 10 Feb 2020 17:18:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 01AHIREv028034 for ; Mon, 10 Feb 2020 12:18:27 -0500 Received: by smtp.corp.redhat.com (Postfix) id 040345C28F; Mon, 10 Feb 2020 17:18:27 +0000 (UTC) Received: from kinshicho.usersys.redhat.com (unknown [10.43.2.246]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 80C805C1D6 for ; Mon, 10 Feb 2020 17:18:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581355150; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=oKwTRJpK01PxVllSOn5jB7t4f8+HsgV4muM4obqjUus=; b=NlpyGbyO+iZTJRHTMsuryHnXgEsn54OOwPTxZt45nb+j4G+66s9akvuot9NYsLFEPYqMFC sDADb5Qk+DIkR285mNWAzegF1YSTr4D2i45h7PvUMgd0YZLCcoqU4yITAynsSUz6dazou2 Z1IQXLsFp86jWgye09/yrHY2rsRJbdc= From: Andrea Bolognani To: libvir-list@redhat.com Subject: [jenkins-ci PATCH 5/8] lcitool: Introduce _dockerfile_build_varmap() Date: Mon, 10 Feb 2020 18:18:14 +0100 Message-Id: <20200210171817.53126-6-abologna@redhat.com> In-Reply-To: <20200210171817.53126-1-abologna@redhat.com> References: <20200210171817.53126-1-abologna@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: xrc0ofPzN022ZJ2U6t7Mvg-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This function takes care of most of what's still handled by _action_dockerfile(), namely populating the dictionary that _dockerfile_format() will later consume. Signed-off-by: Andrea Bolognani --- guests/lcitool | 132 ++++++++++++++++++++++++++----------------------- 1 file changed, 71 insertions(+), 61 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index f216910..82e98c5 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -642,6 +642,76 @@ class Application: with open(keyfile, "r") as r: return r.read().rstrip() =20 + def _dockerfile_build_varmap(self, facts, mappings, pip_mappings, proj= ects, cross_arch): + package_format =3D facts["package_format"] + package_manager =3D facts["package_manager"] + os_name =3D facts["os_name"] + os_version =3D facts["os_version"] + os_full =3D os_name + os_version + + pkgs =3D {} + cross_pkgs =3D {} + pip_pkgs =3D {} + base_keys =3D ["default", package_format, os_name, os_full] + cross_keys =3D [] + if cross_arch: + keys =3D base_keys + [cross_arch + "-" + k for k in base_keys] + cross_keys =3D ["cross-policy-" + k for k in base_keys] + else: + keys =3D base_keys + [self._native_arch + "-" + k for k in bas= e_keys] + + # We need to add the base project manually here: the standard + # machinery hides it because it's an implementation detail + for project in projects + ["base"]: + for package in self._projects.get_packages(project): + cross_policy =3D "native" + for key in cross_keys: + if key in mappings[package]: + cross_policy =3D mappings[package][key] + if cross_policy not in ["native", "foreign", "skip"]: + raise Exception( + "Unexpected cross arch policy {} for {}".format + (cross_policy, package)) + + for key in keys: + if key in mappings[package]: + pkgs[package] =3D mappings[package][key] + if package in pip_mappings and key in pip_mappings[pac= kage]: + pip_pkgs[package] =3D pip_mappings[package][key] + + if package not in pkgs: + continue + if package in pip_pkgs and pkgs[package] is not None: + del pip_pkgs[package] + if cross_policy =3D=3D "foreign" and pkgs[package] is not = None: + cross_pkgs[package] =3D pkgs[package] + if pkgs[package] is None or cross_policy in ["skip", "fore= ign"]: + del pkgs[package] + + pkg_align =3D " \\\n" + (" " * len("RUN " + package_manager + " ")) + pip_pkg_align =3D " \\\n" + (" " * len("RUN pip3 ")) + + varmap =3D {} + varmap["package_manager"] =3D package_manager + varmap["pkgs"] =3D pkg_align[1:] + pkg_align.join(sorted(set(pkgs.= values()))) + + if package_format =3D=3D "deb" and cross_arch: + deb_arch =3D Util.native_arch_to_deb_arch(cross_arch) + abi =3D Util.native_arch_to_abi(cross_arch) + lib =3D Util.native_arch_to_lib(cross_arch) + gcc =3D "gcc-" + abi + varmap["cross_arch"] =3D deb_arch + pkg_names =3D [p + ":" + deb_arch for p in cross_pkgs.values()] + pkg_names.append(gcc) + varmap["cross_pkgs"] =3D pkg_align[1:] + pkg_align.join(sorted= (set(pkg_names))) + varmap["cross_abi"] =3D abi + varmap["cross_lib"] =3D lib + + if pip_pkgs: + varmap["pip_pkgs"] =3D pip_pkg_align[1:] + pip_pkg_align.join(= sorted(set(pip_pkgs.values()))) + + return varmap + def _dockerfile_format(self, facts, cross_arch, varmap): package_format =3D facts["package_format"] package_manager =3D facts["package_manager"] @@ -787,67 +857,7 @@ class Application: ) ) =20 - pkgs =3D {} - cross_pkgs =3D {} - pip_pkgs =3D {} - base_keys =3D ["default", package_format, os_name, os_full] - cross_keys =3D [] - if cross_arch: - keys =3D base_keys + [cross_arch + "-" + k for k in base_keys] - cross_keys =3D ["cross-policy-" + k for k in base_keys] - else: - keys =3D base_keys + [self._native_arch + "-" + k for k in bas= e_keys] - - # We need to add the base project manually here: the standard - # machinery hides it because it's an implementation detail - for project in projects + ["base"]: - for package in self._projects.get_packages(project): - cross_policy =3D "native" - for key in cross_keys: - if key in mappings[package]: - cross_policy =3D mappings[package][key] - if cross_policy not in ["native", "foreign", "skip"]: - raise Exception( - "Unexpected cross arch policy {} for {}".format - (cross_policy, package)) - - for key in keys: - if key in mappings[package]: - pkgs[package] =3D mappings[package][key] - if package in pip_mappings and key in pip_mappings[pac= kage]: - pip_pkgs[package] =3D pip_mappings[package][key] - - if package not in pkgs: - continue - if package in pip_pkgs and pkgs[package] is not None: - del pip_pkgs[package] - if cross_policy =3D=3D "foreign" and pkgs[package] is not = None: - cross_pkgs[package] =3D pkgs[package] - if pkgs[package] is None or cross_policy in ["skip", "fore= ign"]: - del pkgs[package] - - pkg_align =3D " \\\n" + (" " * len("RUN " + package_manager + " ")) - pip_pkg_align =3D " \\\n" + (" " * len("RUN pip3 ")) - - varmap =3D {} - varmap["package_manager"] =3D package_manager - varmap["pkgs"] =3D pkg_align[1:] + pkg_align.join(sorted(set(pkgs.= values()))) - - if package_format =3D=3D "deb" and cross_arch: - deb_arch =3D Util.native_arch_to_deb_arch(cross_arch) - abi =3D Util.native_arch_to_abi(cross_arch) - lib =3D Util.native_arch_to_lib(cross_arch) - gcc =3D "gcc-" + abi - varmap["cross_arch"] =3D deb_arch - pkg_names =3D [p + ":" + deb_arch for p in cross_pkgs.values()] - pkg_names.append(gcc) - varmap["cross_pkgs"] =3D pkg_align[1:] + pkg_align.join(sorted= (set(pkg_names))) - varmap["cross_abi"] =3D abi - varmap["cross_lib"] =3D lib - - if pip_pkgs: - varmap["pip_pkgs"] =3D pip_pkg_align[1:] + pip_pkg_align.join(= sorted(set(pip_pkgs.values()))) - + varmap =3D self._dockerfile_build_varmap(facts, mappings, pip_mapp= ings, projects, cross_arch) self._dockerfile_format(facts, cross_arch, varmap) =20 def run(self): --=20 2.24.1