From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084598795517.5260515455077; Wed, 13 Feb 2019 11:03:18 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A6033A7898; Wed, 13 Feb 2019 19:03:16 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6FAB15D6AA; Wed, 13 Feb 2019 19:03:16 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 13969181A00D; Wed, 13 Feb 2019 19:03:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3El6009148 for ; Wed, 13 Feb 2019 14:03:14 -0500 Received: by smtp.corp.redhat.com (Postfix) id 9E8FC60863; Wed, 13 Feb 2019 19:03:14 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA1A660856; Wed, 13 Feb 2019 19:03:12 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:02:56 +0000 Message-Id: <20190213190305.10926-2-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani Subject: [libvirt] [jenkins-ci PATCH v3 01/10] guests: use libpcap0.8-dev package on Debian X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 13 Feb 2019 19:03:17 +0000 (UTC) The libpcap-dev package is a temporary backcompat package until everything switches to the new libpcap0.8-dev pacakge name. Reviewed-by: Andrea Bolognani Signed-off-by: Daniel P. Berrang=C3=A9 --- guests/vars/mappings.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 84543c4..f211169 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -281,7 +281,7 @@ mappings: rpm: parted-devel =20 libpcap: - deb: libpcap-dev + deb: libpcap0.8-dev pkg: libpcap rpm: libpcap-devel =20 --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084607701925.0399516726734; Wed, 13 Feb 2019 11:03:27 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EFBB4C0624A0; Wed, 13 Feb 2019 19:03:24 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BB46719C7E; Wed, 13 Feb 2019 19:03:24 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 77F653F603; Wed, 13 Feb 2019 19:03:24 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3G1u009154 for ; Wed, 13 Feb 2019 14:03:16 -0500 Received: by smtp.corp.redhat.com (Postfix) id 7866C60863; Wed, 13 Feb 2019 19:03:16 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1F43C60856; Wed, 13 Feb 2019 19:03:14 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:02:57 +0000 Message-Id: <20190213190305.10926-3-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani Subject: [libvirt] [jenkins-ci PATCH v3 02/10] guests: add xfsprogs development package for libvirt X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 13 Feb 2019 19:03:25 +0000 (UTC) Reviewed-by: Andrea Bolognani Signed-off-by: Daniel P. Berrang=C3=A9 --- guests/vars/mappings.yml | 4 ++++ guests/vars/projects/libvirt.yml | 1 + 2 files changed, 5 insertions(+) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index f211169..e10e3da 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -815,6 +815,10 @@ mappings: deb: libxen-dev Fedora: xen-devel =20 + xfsprogs: + deb: xfslibs-dev + rpm: xfsprogs-devel + xmllint: default: libxml2 deb: libxml2-utils diff --git a/guests/vars/projects/libvirt.yml b/guests/vars/projects/libvir= t.yml index 605d201..b19d91f 100644 --- a/guests/vars/projects/libvirt.yml +++ b/guests/vars/projects/libvirt.yml @@ -53,6 +53,7 @@ packages: - tc - wireshark - xen + - xfsprogs - xmllint - xsltproc - yajl --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084611258647.9122526788158; Wed, 13 Feb 2019 11:03:31 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 21625C0C4280; Wed, 13 Feb 2019 19:03:29 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 68B9C5D973; Wed, 13 Feb 2019 19:03:28 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 28E53181A12A; Wed, 13 Feb 2019 19:03:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3ITS009166 for ; Wed, 13 Feb 2019 14:03:18 -0500 Received: by smtp.corp.redhat.com (Postfix) id 374ED60863; Wed, 13 Feb 2019 19:03:18 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id E61D360856; Wed, 13 Feb 2019 19:03:16 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:02:58 +0000 Message-Id: <20190213190305.10926-4-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani Subject: [libvirt] [jenkins-ci PATCH v3 03/10] guests: fix glusterfs package name on Debian X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 13 Feb 2019 19:03:29 +0000 (UTC) We want the development headers not the client binary Reviewed-by: Andrea Bolognani Signed-off-by: Daniel P. Berrang=C3=A9 --- guests/vars/mappings.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index e10e3da..0945e36 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -141,8 +141,12 @@ mappings: rpm: glibc-static =20 glusterfs: - deb: glusterfs-client + deb: libglusterfs-dev rpm: glusterfs-api-devel + Debian8: glusterfs-common + Debian9: glusterfs-common + Ubuntu16: glusterfs-common + Ubuntu18: glusterfs-common =20 gnome-common: default: gnome-common --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084610771866.3497552648223; Wed, 13 Feb 2019 11:03:30 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D65A61174F; Wed, 13 Feb 2019 19:03:28 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AE1FA60C6B; Wed, 13 Feb 2019 19:03:28 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6D0D03F603; Wed, 13 Feb 2019 19:03:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3J2a009176 for ; Wed, 13 Feb 2019 14:03:19 -0500 Received: by smtp.corp.redhat.com (Postfix) id C162060857; Wed, 13 Feb 2019 19:03:19 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id AF8B260856; Wed, 13 Feb 2019 19:03:18 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:02:59 +0000 Message-Id: <20190213190305.10926-5-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 04/10] guests: Debian SID has dropped the sheepdog package X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 13 Feb 2019 19:03:29 +0000 (UTC) Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Andrea Bolognani --- guests/vars/mappings.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 0945e36..f31b460 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -774,6 +774,7 @@ mappings: default: sheepdog CentOS: FreeBSD: + DebianSid: =20 showmount: deb: nfs-common --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084614775979.3708651432748; Wed, 13 Feb 2019 11:03:34 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C87A689AE1; Wed, 13 Feb 2019 19:03:32 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9D1095D9C6; Wed, 13 Feb 2019 19:03:32 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 517C6181A13B; Wed, 13 Feb 2019 19:03:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3LbT009185 for ; Wed, 13 Feb 2019 14:03:21 -0500 Received: by smtp.corp.redhat.com (Postfix) id 83185608A6; Wed, 13 Feb 2019 19:03:21 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 789ED60856; Wed, 13 Feb 2019 19:03:20 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:03:00 +0000 Message-Id: <20190213190305.10926-6-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 05/10] lcitool: include root cause when reporting errors X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 13 Feb 2019 19:03:33 +0000 (UTC) The root cause exception contains the useful information about what really failed during loading of some resource, or running of a command. Signed-off-by: Daniel P. Berrang=C3=A9 --- guests/lcitool | 54 +++++++++++++++++++++++++------------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 759eff6..bd32d1f 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -95,10 +95,10 @@ class Config: if not os.path.exists(config_dir): try: os.mkdir(config_dir) - except Exception: + except Exception as ex: raise Error( - "Can't create configuration directory ({})".format( - config_dir, + "Can't create configuration directory ({}): {}".format( + config_dir, ex, ) ) =20 @@ -117,10 +117,10 @@ class Config: try: with open(flavor_file, "w") as infile: infile.write("{}\n".format(flavor)) - except Exception: + except Exception as ex: raise Error( - "Can't write flavor file ({})".format( - flavor_file, + "Can't write flavor file ({}): {}".format( + flavor_file, ex ) ) =20 @@ -141,10 +141,10 @@ class Config: with open(vault_pass_file, "r") as infile: if not infile.readline().strip(): raise ValueError - except Exception: + except Exception as ex: raise Error( - "Missing or invalid vault password file ({})".format( - vault_pass_file, + "Missing or invalid vault password file ({}): {}".form= at( + vault_pass_file, ex ) ) =20 @@ -157,10 +157,10 @@ class Config: with open(root_pass_file, "r") as infile: if not infile.readline().strip(): raise ValueError - except Exception: + except Exception as ex: raise Error( - "Missing or invalid root password file ({})".format( - root_pass_file, + "Missing or invalid root password file ({}): {}".format( + root_pass_file, ex ) ) =20 @@ -177,8 +177,8 @@ class Inventory: parser =3D configparser.SafeConfigParser() parser.read(ansible_cfg_path) inventory_path =3D parser.get("defaults", "inventory") - except Exception: - raise Error("Can't find inventory location in ansible.cfg") + except Exception as ex: + raise Error("Can't read inventory location in ansible.cfg: {}"= .format(ex)) =20 inventory_path =3D os.path.join(base, inventory_path) =20 @@ -191,10 +191,10 @@ class Inventory: for line in infile: host =3D line.strip() self._facts[host] =3D {} - except Exception: + except Exception as ex: raise Error( - "Missing or invalid inventory ({})".format( - inventory_path, + "Missing or invalid inventory ({}): {}".format( + inventory_path, ex ) ) =20 @@ -202,8 +202,8 @@ class Inventory: try: self._facts[host] =3D self._read_all_facts(host) self._facts[host]["inventory_hostname"] =3D host - except Exception: - raise Error("Can't load facts for '{}'".format(host)) + except Exception as ex: + raise Error("Can't load facts for '{}': {}".format(host, e= x)) =20 @staticmethod def _add_facts_from_file(facts, yaml_path): @@ -254,8 +254,8 @@ class Projects: with open(mappings_path, "r") as infile: mappings =3D yaml.load(infile) self._mappings =3D mappings["mappings"] - except Exception: - raise Error("Can't load mappings") + except Exception as ex: + raise Error("Can't load mappings: {}".format(ex)) =20 source =3D os.path.join(base, "vars", "projects") =20 @@ -273,8 +273,8 @@ class Projects: with open(yaml_path, "r") as infile: packages =3D yaml.load(infile) self._packages[project] =3D packages["packages"] - except Exception: - raise Error("Can't load packages for '{}'".format(project)) + except Exception as ex: + raise Error("Can't load packages for '{}': {}".format(proj= ect, ex)) =20 def expand_pattern(self, pattern): projects =3D Util.expand_pattern(pattern, self._packages, "project= ") @@ -398,8 +398,8 @@ class Application: =20 try: subprocess.check_call(cmd) - except Exception: - raise Error("Failed to run {} on '{}'".format(playbook, hosts)) + except Exception as ex: + raise Error("Failed to run {} on '{}': {}".format(playbook, ho= sts), ex) =20 def _action_hosts(self, _hosts, _projects, _revision): for host in self._inventory.expand_pattern("all"): @@ -478,8 +478,8 @@ class Application: =20 try: subprocess.check_call(cmd) - except Exception: - raise Error("Failed to install '{}'".format(host)) + except Exception as ex: + raise Error("Failed to install '{}': {}".format(host, ex)) =20 def _action_update(self, hosts, projects, git_revision): self._execute_playbook("update", hosts, projects, git_revision) --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084607671864.0264896944899; Wed, 13 Feb 2019 11:03:27 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 60DD3B8364; Wed, 13 Feb 2019 19:03:25 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2EB5D5C221; Wed, 13 Feb 2019 19:03:25 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D698D3F605; Wed, 13 Feb 2019 19:03:24 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3NPj009198 for ; Wed, 13 Feb 2019 14:03:23 -0500 Received: by smtp.corp.redhat.com (Postfix) id 3052060863; Wed, 13 Feb 2019 19:03:23 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1DAA460856; Wed, 13 Feb 2019 19:03:21 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:03:01 +0000 Message-Id: <20190213190305.10926-7-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 06/10] lcitool: force non-interactive apt-get frontend X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 13 Feb 2019 19:03:26 +0000 (UTC) Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Andrea Bolognani --- guests/lcitool | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/guests/lcitool b/guests/lcitool index bd32d1f..a7bcae3 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -541,7 +541,8 @@ class Application: =20 if package_format =3D=3D "deb": sys.stdout.write(textwrap.dedent(""" - RUN apt-get update && \\ + RUN export DEBIAN_FRONTEND=3Dnoninteractive && \\ + apt-get update && \\ apt-get dist-upgrade -y && \\ apt-get install -y ${PACKAGES} && \\ apt-get autoremove -y && \\ --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084618772226.0601239343448; Wed, 13 Feb 2019 11:03:38 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AF17CC0C057C; Wed, 13 Feb 2019 19:03:36 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7963F5D9C6; Wed, 13 Feb 2019 19:03:36 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 29C55181A13E; Wed, 13 Feb 2019 19:03:36 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3P28009211 for ; Wed, 13 Feb 2019 14:03:26 -0500 Received: by smtp.corp.redhat.com (Postfix) id EEF9D608A4; Wed, 13 Feb 2019 19:03:25 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id C79F660856; Wed, 13 Feb 2019 19:03:23 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:03:02 +0000 Message-Id: <20190213190305.10926-8-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani Subject: [libvirt] [jenkins-ci PATCH v3 07/10] lcitool: avoid installing recommended packages X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 13 Feb 2019 19:03:37 +0000 (UTC) We know exactly which packages we need and don't want apt picking extra "recommended" ones for us. Reviewed-by: Andrea Bolognani Signed-off-by: Daniel P. Berrang=C3=A9 --- guests/lcitool | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guests/lcitool b/guests/lcitool index a7bcae3..1271954 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -544,7 +544,7 @@ class Application: RUN export DEBIAN_FRONTEND=3Dnoninteractive && \\ apt-get update && \\ apt-get dist-upgrade -y && \\ - apt-get install -y ${PACKAGES} && \\ + apt-get install --no-install-recommends -y ${PACKAGES}= && \\ apt-get autoremove -y && \\ apt-get autoclean -y """)) --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084612686205.71093540854372; Wed, 13 Feb 2019 11:03:32 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 49683B0ABB; Wed, 13 Feb 2019 19:03:30 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0BD1319C7C; Wed, 13 Feb 2019 19:03:30 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B5BDF3F60A; Wed, 13 Feb 2019 19:03:29 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3Ske009222 for ; Wed, 13 Feb 2019 14:03:28 -0500 Received: by smtp.corp.redhat.com (Postfix) id 2060460865; Wed, 13 Feb 2019 19:03:28 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id B844A60856; Wed, 13 Feb 2019 19:03:26 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:03:03 +0000 Message-Id: <20190213190305.10926-9-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 08/10] lcitool: refactor logic for building package list X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 13 Feb 2019 19:03:31 +0000 (UTC) Signed-off-by: Daniel P. Berrang=C3=A9 --- guests/lcitool | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 1271954..0978c40 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -529,15 +529,18 @@ class Application: if os_full in mappings[package]: temp[package] =3D mappings[package][os_full] =20 - flattened =3D [] + pkgs =3D [] for item in temp: - if temp[item] is not None and temp[item] not in flattened: - flattened +=3D [temp[item]] + pkgname =3D temp[item] + if pkgname is None: + continue + if pkgname not in pkgs: + pkgs.append(pkgname) =20 print("FROM {}".format(facts["docker_base"])) =20 sys.stdout.write("ENV PACKAGES ") - sys.stdout.write(" \\\n ".join(sorted(flattened))) + sys.stdout.write(" \\\n ".join(sorted(pkgs))) =20 if package_format =3D=3D "deb": sys.stdout.write(textwrap.dedent(""" --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084617450110.24135635956372; Wed, 13 Feb 2019 11:03:37 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 57889C78F7; Wed, 13 Feb 2019 19:03:35 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2398519C7C; Wed, 13 Feb 2019 19:03:35 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D43023F60D; Wed, 13 Feb 2019 19:03:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3XiF009238 for ; Wed, 13 Feb 2019 14:03:33 -0500 Received: by smtp.corp.redhat.com (Postfix) id 62F5560865; Wed, 13 Feb 2019 19:03:33 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 179F3608A4; Wed, 13 Feb 2019 19:03:28 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:03:04 +0000 Message-Id: <20190213190305.10926-10-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 09/10] lcitool: avoid using an env var to store package list X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 13 Feb 2019 19:03:36 +0000 (UTC) Every statement in a dockerfile results in a new layer in the image. There is no need for an env var to store the package list when it can be included inline. This avoids the env variable being later exposed to the container at runtime. Signed-off-by: Daniel P. Berrang=C3=A9 --- guests/lcitool | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 0978c40..8252dc2 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -539,34 +539,34 @@ class Application: =20 print("FROM {}".format(facts["docker_base"])) =20 - sys.stdout.write("ENV PACKAGES ") - sys.stdout.write(" \\\n ".join(sorted(pkgs))) - + varmap =3D {} + varmap["pkgs"] =3D " \\\n ".join(sorted(pkgs)) if package_format =3D=3D "deb": sys.stdout.write(textwrap.dedent(""" RUN export DEBIAN_FRONTEND=3Dnoninteractive && \\ apt-get update && \\ apt-get dist-upgrade -y && \\ - apt-get install --no-install-recommends -y ${PACKAGES}= && \\ + apt-get install --no-install-recommends -y \\ + {pkgs} && \\ apt-get autoremove -y && \\ apt-get autoclean -y - """)) + """).format(**varmap)) elif package_format =3D=3D "rpm": if os_name =3D=3D "Fedora" and os_version =3D=3D "Rawhide": sys.stdout.write(textwrap.dedent(""" RUN yum update -y --nogpgcheck fedora-gpg-keys && \\ yum update -y && \\ - yum install -y ${PACKAGES} && \\ + yum install -y %(pkgs)s && \\ yum autoremove -y && \\ yum clean all -y - """)) + """).format(**varmap)) else: sys.stdout.write(textwrap.dedent(""" RUN yum update -y && \\ - yum install -y ${PACKAGES} && \\ + yum install -y %(pkgs)s && \\ yum autoremove -y && \\ yum clean all -y - """)) + """).format(**varmap)) =20 def run(self): cmdline =3D self._parser.parse_args() --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 11:15:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1550084618918278.94489004812215; Wed, 13 Feb 2019 11:03:38 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DDCFCC7A06; Wed, 13 Feb 2019 19:03:36 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A77E75D962; Wed, 13 Feb 2019 19:03:36 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 5A271181A13D; Wed, 13 Feb 2019 19:03:36 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x1DJ3ZQd009247 for ; Wed, 13 Feb 2019 14:03:35 -0500 Received: by smtp.corp.redhat.com (Postfix) id 1D36960863; Wed, 13 Feb 2019 19:03:35 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0E85460865; Wed, 13 Feb 2019 19:03:33 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 13 Feb 2019 19:03:05 +0000 Message-Id: <20190213190305.10926-11-berrange@redhat.com> In-Reply-To: <20190213190305.10926-1-berrange@redhat.com> References: <20190213190305.10926-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Subject: [libvirt] [jenkins-ci PATCH v3 10/10] lcitool: support generating cross compiler dockerfiles X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 13 Feb 2019 19:03:37 +0000 (UTC) Debian's filesystem layout has a nice advantage over Fedora which is that it can install non-native RPMs in the main root filesystem. It is thus possible to prepare an x86_64 filesystem containing -dev packages for a foreign architecture, along with a GCC cross compiler. QEMU has used this technique to facilitate developer build testing of non-x86 architectures, since few people have access to physical hardware for most of these architectures. For the same reason it would be helpful to libvirt developers. This patch extends the 'dockerfile' command to 'lcitool' so that it accepts a '-x $ARCH' argument. $ lcitool -a dockerfile -h libvirt-debian-9 -p libvirt -x s390x This is only valid when using a 'deb' based distro. With the Debian 9 distro, this supports arm64, armel, armhf, mips, mipsel, mips64el, ppc64el, s390x, which are all the official archs that Debian maintainers currently build libvirt for. With Debian Sid, this is extended to include i386. Debian also builds libvirt on hppa, powerpcspe, sparc64 and x32 which are unofficial ports. Unfortunately it is not possible to reliably add foreign arch packages from the unofficial ports in parallel with those from the native arch, without hitting dependency problems from apt. When an architecture is given, any package name ending in '-dev' will be installed using that architecture variant, while all remaining packages will have their native variant installed. For various reasons (commented inline) a few blacklists are required on a per-arch basis. Signed-off-by: Daniel P. Berrang=C3=A9 --- .../libvirt-debian-9/cross-build.yml | 50 ++++++++++++++ .../libvirt-debian-sid/cross-build.yml | 52 +++++++++++++++ guests/lcitool | 66 ++++++++++++++++--- 3 files changed, 159 insertions(+), 9 deletions(-) create mode 100644 guests/host_vars/libvirt-debian-9/cross-build.yml create mode 100644 guests/host_vars/libvirt-debian-sid/cross-build.yml diff --git a/guests/host_vars/libvirt-debian-9/cross-build.yml b/guests/hos= t_vars/libvirt-debian-9/cross-build.yml new file mode 100644 index 0000000..9c6e511 --- /dev/null +++ b/guests/host_vars/libvirt-debian-9/cross-build.yml @@ -0,0 +1,50 @@ +--- +cross_build: + blacklist: + # apt doesn't properly resolve from foreign arch package + # to the native package of augeas-lenses + - libnetcf-dev + # apt fails to resolve deps from foreign arch + - wireshark-dev + # dtrace tool doesn't know how to cross-compile + - systemtap-sdt-dev + arches: + arm64: + target: aarch64-linux-gnu + armel: + target: arm-linux-gnueabi + blacklist: + # Not built on this arch + - libxen-dev + # Arch does not support NUMA concept + - libnuma-dev + armhf: + target: arm-linux-gnueabihf + blacklist: + # Arch does not support NUMA concept + - libnuma-dev + mips64el: + target: mips64el-linux-gnuabi64 + blacklist: + # Not built on this arch + - libxen-dev + mips: + target: mips-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + mipsel: + target: mipsel-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + ppc64el: + target: powerpc64le-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + s390x: + target: s390x-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev diff --git a/guests/host_vars/libvirt-debian-sid/cross-build.yml b/guests/h= ost_vars/libvirt-debian-sid/cross-build.yml new file mode 100644 index 0000000..7184083 --- /dev/null +++ b/guests/host_vars/libvirt-debian-sid/cross-build.yml @@ -0,0 +1,52 @@ +--- +cross_build: + blacklist: + # apt doesn't properly resolve from foreign arch package + # to the native package of augeas-lenses + - libnetcf-dev + # apt fails to resolve deps from foreign arch + - wireshark-dev + # dtrace tool doesn't know how to cross-compile + - systemtap-sdt-dev + arches: + arm64: + target: aarch64-linux-gnu + armel: + target: arm-linux-gnueabi + blacklist: + # Not built on this arch + - libxen-dev + # Arch does not support NUMA concept + - libnuma-dev + armhf: + target: arm-linux-gnueabihf + blacklist: + # Arch does not support NUMA concept + - libnuma-dev + mips64el: + target: mips64el-linux-gnuabi64 + blacklist: + # Not built on this arch + - libxen-dev + mips: + target: mips-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + mipsel: + target: mipsel-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + ppc64el: + target: powerpc64le-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + s390x: + target: s390x-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + i386: + target: i686-linux-gnu diff --git a/guests/lcitool b/guests/lcitool index 8252dc2..cc123a0 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -342,6 +342,11 @@ class Application: metavar=3D"GIT_REVISION", help=3D"git revision to build (remote/branch)", ) + self._parser.add_argument( + "-x", + metavar=3D"CROSS_ARCH", + help=3D"cross compiler architecture (dockerfile action only)", + ) =20 def _execute_playbook(self, playbook, hosts, projects, git_revision): base =3D Util.get_base() @@ -401,15 +406,15 @@ class Application: except Exception as ex: raise Error("Failed to run {} on '{}': {}".format(playbook, ho= sts), ex) =20 - def _action_hosts(self, _hosts, _projects, _revision): + def _action_hosts(self, _hosts, _projects, _revision, _cross_arch): for host in self._inventory.expand_pattern("all"): print(host) =20 - def _action_projects(self, _hosts, _projects, _revision): + def _action_projects(self, _hosts, _projects, _revision, _cross_arch): for project in self._projects.expand_pattern("all"): print(project) =20 - def _action_install(self, hosts, _projects, _revision): + def _action_install(self, hosts, _projects, _revision, _cross_arch): base =3D Util.get_base() =20 flavor =3D self._config.get_flavor() @@ -481,13 +486,13 @@ class Application: except Exception as ex: raise Error("Failed to install '{}': {}".format(host, ex)) =20 - def _action_update(self, hosts, projects, git_revision): + def _action_update(self, hosts, projects, git_revision, _cross_arch): self._execute_playbook("update", hosts, projects, git_revision) =20 - def _action_build(self, hosts, projects, git_revision): + def _action_build(self, hosts, projects, git_revision, _cross_arch): self._execute_playbook("build", hosts, projects, git_revision) =20 - def _action_dockerfile(self, hosts, projects, _revision): + def _action_dockerfile(self, hosts, projects, _revision, cross_arch): mappings =3D self._projects.get_mappings() =20 hosts =3D self._inventory.expand_pattern(hosts) @@ -500,10 +505,22 @@ class Application: os_name =3D facts["os_name"] os_version =3D facts["os_version"] os_full =3D os_name + os_version + blacklist =3D [] + cross_build_facts =3D None =20 if package_format not in ["deb", "rpm"]: raise Error("Host {} doesn't support Dockerfiles".format(host)) =20 + if cross_arch is not None: + if "cross_build" not in facts: + raise Error("cross compilers not supported for this host") + if cross_arch not in facts["cross_build"]["arches"]: + raise Error("unsupported cross compiler architecture, use = one of {}".format( + ", ".join(facts["cross_build"]["arches"].keys()))) + cross_build_facts =3D facts["cross_build"]["arches"][cross_arc= h] + blacklist.extend(facts["cross_build"].get("blacklist", [])) + blacklist.extend(cross_build_facts.get("blacklist", [])) + projects =3D self._projects.expand_pattern(projects) for project in projects: if project not in facts["projects"]: @@ -530,17 +547,29 @@ class Application: temp[package] =3D mappings[package][os_full] =20 pkgs =3D [] + cross_pkgs =3D [] for item in temp: pkgname =3D temp[item] if pkgname is None: continue - if pkgname not in pkgs: - pkgs.append(pkgname) + if pkgname in blacklist: + continue + if cross_arch and pkgname[-4:] =3D=3D "-dev": + if pkgname not in cross_pkgs: + cross_pkgs.append(pkgname + ":" + cross_arch) + else: + if pkgname not in pkgs: + pkgs.append(pkgname) =20 print("FROM {}".format(facts["docker_base"])) =20 varmap =3D {} varmap["pkgs"] =3D " \\\n ".join(sorted(pkgs)) + if cross_arch: + varmap["cross_arch"] =3D cross_arch + cross_pkgs.append("gcc-" + cross_build_facts["target"]) + varmap["cross_pkgs"] =3D " \\\n ".join(sorted(cross= _pkgs)) + varmap["cross_target"] =3D cross_build_facts["target"] if package_format =3D=3D "deb": sys.stdout.write(textwrap.dedent(""" RUN export DEBIAN_FRONTEND=3Dnoninteractive && \\ @@ -551,6 +580,24 @@ class Application: apt-get autoremove -y && \\ apt-get autoclean -y """).format(**varmap)) + if cross_arch: + # Intentionally a separate RUN command from the above + # so that the common packages of all cross-built images + # share a docker image layer. + sys.stdout.write(textwrap.dedent(""" + RUN export DEBIAN_FRONTEND=3Dnoninteractive && \\ + dpkg --add-architecture {cross_arch} && \\ + apt-get update && \\ + apt-get dist-upgrade -y && \\ + apt-get install --no-install-recommends -y \\ + {cross_pkgs} && \\ + apt-get autoremove -y && \\ + apt-get autoclean -y + + ENV TARGET "{cross_target}" + ENV CONFIGURE_OPTS "--host=3D{cross_target} --target= =3D{cross_target}" + ENV PKG_CONFIG_LIBDIR "/usr/lib/{cross_target}/pkgconf= ig" + """).format(**varmap)) elif package_format =3D=3D "rpm": if os_name =3D=3D "Fedora" and os_version =3D=3D "Rawhide": sys.stdout.write(textwrap.dedent(""" @@ -574,11 +621,12 @@ class Application: hosts =3D cmdline.h projects =3D cmdline.p git_revision =3D cmdline.g + cross_arch =3D cmdline.x =20 method =3D "_action_{}".format(action.replace("-", "_")) =20 if hasattr(self, method): - getattr(self, method).__call__(hosts, projects, git_revision) + getattr(self, method).__call__(hosts, projects, git_revision, = cross_arch) else: raise Error("Invalid action '{}'".format(action)) =20 --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list