From nobody Fri May 3 01:54:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616054976; cv=none; d=zohomail.com; s=zohoarc; b=NmLn5OpYN4DoVwQrRMYOH1o6fei3UnFUKXFFjfYcptvzHAff/NGQUIygGz/Bpp8mslsr24mVpRkqyPoCAZUJKW3Yo3HRsZ0IrI5VHqBxWqxknAC4TPxutL4BtdxmG/HuxZQIZLR9aGO3lBFHxyJhck+NGHIgMNh5PnW2zM38ej0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616054976; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=zj/vw9M3vwH+Bg/RgzwCwSPG5HF+zkaYga5Vvl+SUqI=; b=W8Z0/4UTYM7DrIlaUsmXm3xAaVumJxCSE3CN1n1XMUq+6BIIPRB8j6aJI+YhRINrFsWmsLPymWYJYCEh2GE3uNAuaekruKOftjNgJoDgQZFRqJWnWXxfuaMWeY3ADGDuPkY3B1Cf/nS/wcDANyRlUObqGsUwA72luB9C82SpDVs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1616054976179985.891576848636; Thu, 18 Mar 2021 01:09:36 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-153-OPurwyAPOw-yQcuLMFQpgw-1; Thu, 18 Mar 2021 04:09:32 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7A50FCC631; Thu, 18 Mar 2021 08:09:26 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F0F1017195; Thu, 18 Mar 2021 08:09:25 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B03831809C83; Thu, 18 Mar 2021 08:09:24 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12I89NYH030358 for ; Thu, 18 Mar 2021 04:09:23 -0400 Received: by smtp.corp.redhat.com (Postfix) id 5C6B21F41B; Thu, 18 Mar 2021 08:09:23 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id AA4FC19C45; Thu, 18 Mar 2021 08:09:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616054974; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=zj/vw9M3vwH+Bg/RgzwCwSPG5HF+zkaYga5Vvl+SUqI=; b=OEjPhetsTNx+nSufdgx4F7BiCT399oTJmDa87tlOC2M4DhjD7wyIKUA30J7Yz1eRkFvC62 Hx06I8mbM0PJkDCaqxd2ErBWphdE+FGXiK3BxcMuCUNnTh5Rj2n/j8GCk3Kp1xf4fkoJih ENfW+IQx/qDPOh3sN7BqIh1mi10GhTY= X-MC-Unique: OPurwyAPOw-yQcuLMFQpgw-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v3 1/6] ci: helper: Use the ArgumentDefaultsHelpFormatter help formatter Date: Thu, 18 Mar 2021 09:09:12 +0100 Message-Id: <5f3358ec1369ddf6022b34d02c6377b99d5c7009.1616054919.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This help formatter class reports the defaults we use for options taking an argument. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/helper | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/ci/helper b/ci/helper index 73a3f729da..8f34f4b59d 100755 --- a/ci/helper +++ b/ci/helper @@ -79,6 +79,7 @@ class Parser: "build", help=3D"run a build in a container", parents=3D[containerparser, mesonparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) buildparser.set_defaults(func=3DApplication.action_build) =20 @@ -87,6 +88,7 @@ class Parser: "test", help=3D"run a build in a container (including tests)", parents=3D[containerparser, mesonparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) testparser.set_defaults(func=3DApplication.action_test) =20 @@ -95,6 +97,7 @@ class Parser: "shell", help=3D"start a shell in a container", parents=3D[containerparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) shellparser.set_defaults(func=3DApplication.action_shell) =20 @@ -102,6 +105,7 @@ class Parser: listimagesparser =3D subparsers.add_parser( "list-images", help=3D"list known container images", + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) listimagesparser.set_defaults(func=3DApplication.action_list_image= s) =20 @@ -110,6 +114,7 @@ class Parser: "refresh", help=3D"refresh data generated with lcitool", parents=3D[lcitoolparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) refreshparser.set_defaults(func=3DApplication.action_refresh) =20 --=20 2.29.2 From nobody Fri May 3 01:54:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) client-ip=63.128.21.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616054979; cv=none; d=zohomail.com; s=zohoarc; b=FISBzvl6RZhnstJSwXvOXFmqIo+RCCsHLwdxew7hVQFpKlvqtG+Kf0PIjtCC16GNiODN3h6A5Aa2rUTb2J4HH+V/W4TScfPcEN6h2lo8ZpEvKqT/ecf3fKCuOjJrZTlSJQKamzbT6+kq56FXbNIsR3ehaqP2jiJHPHNjZPUelSM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616054979; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CXCaXRWqaSmSJ3oVMwfPcE5D08FQAZnTLpiO9jVjVCk=; b=XCCD4zlTv7Vw6FlbYTi+sLdlxJm0AbQpUC4b8MOW0JE5yQgjW5057fZhpTNq/eJd8uOvAd0/WiAGc8v+dcS78SRjJ8ziFP4JDr+dqSB+US43jVPlfi6+9RLwg0nPI7PXcpQKJktTJEfz0djYj6pZDMxWv/x6FcPGJmUhJdemtTY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mx.zohomail.com with SMTPS id 1616054979828615.8863180287624; Thu, 18 Mar 2021 01:09:39 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-320-FYhWl6NKPP-9ii72D81zPg-1; Thu, 18 Mar 2021 04:09:36 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4313C107B784; Thu, 18 Mar 2021 08:09:31 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1E27B5E1A4; Thu, 18 Mar 2021 08:09:31 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id CF0F5180B452; Thu, 18 Mar 2021 08:09:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12I89OFW030366 for ; Thu, 18 Mar 2021 04:09:24 -0400 Received: by smtp.corp.redhat.com (Postfix) id 5F0AE1F410; Thu, 18 Mar 2021 08:09:24 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id ADEBF19C45; Thu, 18 Mar 2021 08:09:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616054978; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=CXCaXRWqaSmSJ3oVMwfPcE5D08FQAZnTLpiO9jVjVCk=; b=ibtOmfuhjG6+hyH/tmZc/7g4cwnBAPaF2+CKsnGJrtxktQ6y6LLIevZ1Ilwj5NeUUsOf/D S4SF2nwQf7+Vq2N1USgiDpm/z6dIgjnRBgHwyYynd2MIweWrBxsWxbDXy7J2df+Yz+RyLr 10k/QvghtRc37cKTYeY4/SJbZd2S39E= X-MC-Unique: FYhWl6NKPP-9ii72D81zPg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v3 2/6] ci: helper: Introduce --quiet Date: Thu, 18 Mar 2021 09:09:13 +0100 Message-Id: <9100c64bc3cd8838c39264ed83fbf137eb96b860.1616054919.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Offer an option to silence all output to stdout, e.g. when generating dockerfiles/varsfiles. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/helper | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/ci/helper b/ci/helper index 8f34f4b59d..53baef3cb1 100755 --- a/ci/helper +++ b/ci/helper @@ -116,6 +116,12 @@ class Parser: parents=3D[lcitoolparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) + refreshparser.add_argument( + "--quiet", + action=3D"store_true", + default=3DFalse, + help=3D"refresh data silently" + ) refreshparser.set_defaults(func=3DApplication.action_refresh) =20 def parse(self): @@ -173,7 +179,8 @@ class Application: outfile =3D f"ci-{host}-cross-{cross}.Dockerfile" =20 outpath =3D outdir.joinpath(outfile) - print(outpath) + if not self.args.quiet: + print(outpath) =20 output =3D self.lcitool_run(args) with open(outpath, "w") as f: @@ -185,7 +192,8 @@ class Application: outfile =3D f"{host}.vars" =20 outpath =3D outdir.joinpath(outfile) - print(outpath) + if not self.args.quiet: + print(outpath) =20 output =3D self.lcitool_run(args) with open(outpath, "w") as f: --=20 2.29.2 From nobody Fri May 3 01:54:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616054984; cv=none; d=zohomail.com; s=zohoarc; b=ev83VISBF/PkPzL5Q8gJloGYPCwxewV2zToiXeCXlCe0wtzrSFIpNqxDzDIqw/9/UPVHpsFFjQaj2kdQF60hDu6nV/ZKmz/UQKik3bToeId6qeR2Xqr8YUbyJjllaQoeV7gBNswVIjo4QP3fHMlSnRTDelpNe5jrOofjHAyDHqQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616054984; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VlMw45/ataMAlQiS77H/ZJ5a7l6xnYnJkDU4IHq3LSM=; b=HDoIRyyKISLzwiLzMoH0YeEmQZKyYk+MjC5f1FOEoeu3qIfBUSnfjacVFBOPu+yNQGuY1WUKFsoT7VnNhK7aLybnUwbUtGBgeeMkhdY4X2nEAvSvJsCnTuZKfFSVD+pUvWxQUxCxKHW/XANFTcRbOA7NWaNqfNQLpUSGmviUYhQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 161605498425254.13403905538348; Thu, 18 Mar 2021 01:09:44 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-232-XTGIvcVENYyXoLlGZzSGdQ-1; Thu, 18 Mar 2021 04:09:41 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2B6988018A2; Thu, 18 Mar 2021 08:09:35 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 01E5960C13; Thu, 18 Mar 2021 08:09:35 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B88575003B; Thu, 18 Mar 2021 08:09:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12I89PlB030377 for ; Thu, 18 Mar 2021 04:09:25 -0400 Received: by smtp.corp.redhat.com (Postfix) id 6174B1F410; Thu, 18 Mar 2021 08:09:25 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id AF6FA19C45; Thu, 18 Mar 2021 08:09:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616054983; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=VlMw45/ataMAlQiS77H/ZJ5a7l6xnYnJkDU4IHq3LSM=; b=Gc75qFwR+ESK055tE+IE8ymY97I/BKcQ68wDUlbqIyiwo1DiB5RpSee1e18hm7TLTOyqyO lTRzIG3gCiDIuagSTil5UiSKM0H5ucgmLAOjRDc5IfKXQr7JuiJMRGBZnZDVhuSwdPEEiM wZc81HkZlz8clnqPtxv6IVKP3eXOc60= X-MC-Unique: XTGIvcVENYyXoLlGZzSGdQ-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v3 3/6] ci: Introduce a util module Date: Thu, 18 Mar 2021 09:09:14 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" With the gradual rewrite of the Makefile to the 'helper' script will require helper functions that would better live in a separate util module. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/util.py | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 ci/util.py diff --git a/ci/util.py b/ci/util.py new file mode 100644 index 0000000000..f9f8320276 --- /dev/null +++ b/ci/util.py @@ -0,0 +1,40 @@ +import json +import urllib.request +import urllib.parse + +from typing import Dict, List + + +def get_registry_uri(namespace: str, + gitlab_uri: str =3D "https://gitlab.com") -> str: + """ + Construct a v4 API URI pointing the namespaced project's image registr= y. + + :param namespace: GitLab project namespace, e.g. "libvirt/libvirt" + :param gitlab_uri: GitLab base URI, can be a private deployment + :param api_version: GitLab REST API version number + :return: URI pointing to a namespaced project's image registry + """ + + # this converts something like "libvirt/libvirt" to "libvirt%2Flibvirt" + namespace_urlenc =3D urllib.parse.quote_plus(namespace) + + project_uri =3D f"{gitlab_uri}/api/v4/projects/{namespace_urlenc}" + + uri =3D project_uri + "/registry/repositories" + return uri + + +def get_registry_images(uri: str) -> List[Dict]: + """ + List all container images that are currently available in the given Gi= tLab + project. + + :param uri: URI pointing to a GitLab instance's image registry + :return: list of container image names + """ + + r =3D urllib.request.urlopen(uri + "?per_page=3D100") + + # read the HTTP response and load the JSON part of it + return json.loads(r.read().decode()) --=20 2.29.2 From nobody Fri May 3 01:54:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616055069; cv=none; d=zohomail.com; s=zohoarc; b=LcegtDho6WJ9rh7gNMEEsICpLcXB1qrkV9cbofx8Y/pQBa5lcwlcM9WVj83wNhY4vg1BAA2t5mfda8j/yB7zlIitktYWzTuwSVKP3FlCt2aVVjj6zwsz97Q3Qm92x3iwRNrvvI7xy0Wa8eQegl2WtIuZ73Aj7o+C4/3CoEziHLY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616055069; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0eerKRorRZhus/mOT210mzNCdYVN6fc0aw1AqbBpd/o=; b=hNJWSb1ahkGvO07IsoO9ObENJ4DHjfK5Ci3gOV/DG+jNFc3U4WfIYeN8BI8J3nVyozQu8zSnvmfOH5HB/X/W3g+Z/cmUViRa0Lfzviq0xYcObTOZUw3RfFtKHN9IP+vBP0t2lpG6z6uY3qYlOkW3ciMOAmZcb0TAP2MWKnrIxws= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1616055069525673.9926985245608; Thu, 18 Mar 2021 01:11:09 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-519-IhmzprI0Pc6Lnpf6vVpFtQ-1; Thu, 18 Mar 2021 04:09:44 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 846CE84B9A7; Thu, 18 Mar 2021 08:09:38 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5FE2917D04; Thu, 18 Mar 2021 08:09:38 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 1EF465003F; Thu, 18 Mar 2021 08:09:38 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12I89QOC030385 for ; Thu, 18 Mar 2021 04:09:26 -0400 Received: by smtp.corp.redhat.com (Postfix) id 67D4C1F041; Thu, 18 Mar 2021 08:09:26 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id B1A2A1F41B; Thu, 18 Mar 2021 08:09:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616055068; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=0eerKRorRZhus/mOT210mzNCdYVN6fc0aw1AqbBpd/o=; b=fz01BSWZ/qPNhOnTJIPsgF3CWpD1W36U2I+qVILUoPDkM40vR7fgBSHn3WdfZrTx4bGWeV IKnFGs1216nWZscpGllFP0dkeExWtk3oxYD7Dwlyc82ysND5q0/maNw/4Ppb+Hc1V6O1rZ tagRkVGZoOf2Q3iQJfeIrguiDh4VDp4= X-MC-Unique: IhmzprI0Pc6Lnpf6vVpFtQ-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v3 4/6] ci: helper: Rewrite image listing to Python Date: Thu, 18 Mar 2021 09:09:15 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" The corresponding Bash script is dropped. After this patch's rewrite, the Makefile's original image listing target remains intact only to notify the user to use the Python helper instead. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/Makefile | 12 ------------ ci/helper | 38 +++++++++++++++++++++++++++++++++++++- ci/list-images.sh | 14 -------------- 3 files changed, 37 insertions(+), 27 deletions(-) delete mode 100644 ci/list-images.sh diff --git a/ci/Makefile b/ci/Makefile index 72f5bda942..02ce0df7ee 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -211,17 +211,6 @@ ci-build@%: ci-test@%: $(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_NINJA_ARGS=3Dtest =20 -ci-list-images: - @echo - @echo "Available x86 container images:" - @echo - @sh list-images.sh "$(CI_IMAGE_PREFIX)" | grep -v cross - @echo - @echo "Available cross-compiler container images:" - @echo - @sh list-images.sh "$(CI_IMAGE_PREFIX)" | grep cross - @echo - ci-help: @echo @echo @@ -241,7 +230,6 @@ ci-help: @echo " ci-build@\$$IMAGE - run a default 'ninja' build" @echo " ci-test@\$$IMAGE - run a 'ninja test'" @echo " ci-shell@\$$IMAGE - run an interactive shell" - @echo " ci-list-images - list available images" @echo " ci-help - show this help message" @echo @echo "Available make variables:" diff --git a/ci/helper b/ci/helper index 53baef3cb1..31cf72fbdf 100755 --- a/ci/helper +++ b/ci/helper @@ -11,6 +11,8 @@ import shutil import subprocess import sys =20 +import util + =20 class Parser: def __init__(self): @@ -66,6 +68,20 @@ class Parser: help=3D"path to lcitool binary", ) =20 + # Options that are common to actions communicating with a GitLab + # instance + gitlabparser =3D argparse.ArgumentParser(add_help=3DFalse) + gitlabparser.add_argument( + "--namespace", + default=3D"libvirt/libvirt", + help=3D"GitLab project namespace" + ) + gitlabparser.add_argument( + "--gitlab-uri", + default=3D"https://gitlab.com", + help=3D"base GitLab URI" + ) + # Main parser self.parser =3D argparse.ArgumentParser() subparsers =3D self.parser.add_subparsers( @@ -105,6 +121,7 @@ class Parser: listimagesparser =3D subparsers.add_parser( "list-images", help=3D"list known container images", + parents=3D[gitlabparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) listimagesparser.set_defaults(func=3DApplication.action_list_image= s) @@ -249,7 +266,26 @@ class Application: self.make_run(f"ci-shell@{self.args.target}") =20 def action_list_images(self): - self.make_run(f"ci-list-images") + registry_uri =3D util.get_registry_uri(self.args.namespace, + self.args.gitlab_uri) + images =3D util.get_registry_images(registry_uri) + + # skip the "ci-" prefix each of our container images' name has + name_prefix =3D "ci-" + names =3D [i["name"][len(name_prefix):] for i in images] + names.sort() + + native =3D [name for name in names if "-cross-" not in name] + cross =3D [name for name in names if "-cross-" in name] + + spacing =3D 4 * " " + print("Available x86 container images:\n") + print(spacing + ("\n" + spacing).join(native)) + + if cross: + print() + print("Available cross-compiler container images:\n") + print(spacing + ("\n" + spacing).join(cross)) =20 def action_refresh(self): self.refresh_containers() diff --git a/ci/list-images.sh b/ci/list-images.sh deleted file mode 100644 index b85b132253..0000000000 --- a/ci/list-images.sh +++ /dev/null @@ -1,14 +0,0 @@ -#!/bin/sh - -prefix=3D"${1##registry.gitlab.com/}" - -PROJECT_ID=3D192693 - -all_repos() { - curl -s "https://gitlab.com/api/v4/projects/$PROJECT_ID/registry/reposit= ories?per_page=3D100" \ - | tr , '\n' | grep '"path":' | sed 's,"path":",,g;s,"$,,g' -} - -all_repos | grep "^$prefix" | sed "s,^$prefix,,g" | while read repo; do - echo " $repo" -done | sort -u --=20 2.29.2 From nobody Fri May 3 01:54:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) client-ip=63.128.21.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616054981; cv=none; d=zohomail.com; s=zohoarc; b=d44yYgkOZ+MD5EiFhwxO6dHiH5Bl1csGLLYpXYYmURy5uxpKzhKcKjNcfZd9nMLWARuX6Q2cZ0Eg2EQS9CpHiP/ufiW8ROwXPyCTkPBIUvb2fg5datx+stipaICgYAXUT2e21EAhZIGHdFiST4pRveLN+ulz+ep0rzHdPHPjmvc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616054981; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=c8oIH2dNbEM9/Cqou9HU7XSS8j6M0b0jkJZA6O4sYuM=; b=DXi348c9MHcKvHrkS3mTO/c2TeoI0+zlrRxDp0LfcrJ56fxYcdziQwZ9PxZXYAOdOXuN6OcglBrlT85ku990DDR6YQJpDEyGVZWu+KiL0AsHMCfAFn6KUvSRGXBA+d+EmMONB/HAwqfuNbdx5CT4GViXhwW5Y/zR7HHbPmTtsJ8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mx.zohomail.com with SMTPS id 1616054981114711.4102630937356; Thu, 18 Mar 2021 01:09:41 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-576-CipzZ61hO02aLhiR7-7mOg-1; Thu, 18 Mar 2021 04:09:38 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B065B84B9A2; Thu, 18 Mar 2021 08:09:30 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8B49A1378D; Thu, 18 Mar 2021 08:09:30 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 4B842180B450; Thu, 18 Mar 2021 08:09:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12I89RJV030393 for ; Thu, 18 Mar 2021 04:09:27 -0400 Received: by smtp.corp.redhat.com (Postfix) id 67E091F041; Thu, 18 Mar 2021 08:09:27 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id B6D7319C45; Thu, 18 Mar 2021 08:09:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616054980; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=c8oIH2dNbEM9/Cqou9HU7XSS8j6M0b0jkJZA6O4sYuM=; b=DmcuD9FGG0IrT5R+yXnO6DK5VMaDegaHPidpSuBru5YtXWJ4mgSJWv4Dgwpv7uOIG3mbyp ygyQtItZDZlliggPaJoswTGHnfzJESSdJv2kRIlTMjCIiLW+nNp6O9VGpf8IS6DQoFxpux PQYVirDjkKZ76/aJDz/V5zqQ+67KslU= X-MC-Unique: CipzZ61hO02aLhiR7-7mOg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v3 5/6] ci: util: Add a registry checker for stale images Date: Thu, 18 Mar 2021 09:09:16 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This function checks whether there are any stale Docker images in the registry that can be purged. Since we're pulling available container images from our GitLab registry with the 'list-images' action, it could happen that we'd list old (already unsupported) images and make them available for the user to consume and run a build in them. Naturally, the build will most likely fail leaving the user confused. Signed-off-by: Erik Skultety --- ci/helper | 45 ++++++++++++++++++++++++++++++++++++++++++++- ci/util.py | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+), 1 deletion(-) diff --git a/ci/helper b/ci/helper index 31cf72fbdf..b5255db835 100755 --- a/ci/helper +++ b/ci/helper @@ -130,7 +130,7 @@ class Parser: refreshparser =3D subparsers.add_parser( "refresh", help=3D"refresh data generated with lcitool", - parents=3D[lcitoolparser], + parents=3D[lcitoolparser, gitlabparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) refreshparser.add_argument( @@ -139,6 +139,13 @@ class Parser: default=3DFalse, help=3D"refresh data silently" ) + refreshparser.add_argument( + "--check-stale", + action=3D"store", + choices=3D["yes", "no"], + default=3D"yes", + help=3D"check for existence of stale images on the GitLab inst= ance" + ) refreshparser.set_defaults(func=3DApplication.action_refresh) =20 def parse(self): @@ -287,10 +294,46 @@ class Application: print("Available cross-compiler container images:\n") print(spacing + ("\n" + spacing).join(cross)) =20 + def check_stale_images(self): + if self.args.check_stale !=3D "yes" or self.args.quiet: + return + + namespace =3D self.args.namespace + gitlab_uri =3D self.args.gitlab_uri + registry_uri =3D util.get_registry_uri(namespace, gitlab_uri) + lcitool_hosts =3D self.lcitool_get_hosts() + + stale_images =3D util.get_registry_stale_images(registry_uri, + lcitool_hosts) + if stale_images: + spacing =3D "\n" + 4 * " " + stale_fmt =3D [f"{k} (ID: {v})" for k, v in stale_images.items= ()] + stale_details =3D spacing + spacing.join(stale_fmt) + stale_ids =3D ' '.join([str(id) for id in stale_images.values(= )]) + registry_uri =3D util.get_registry_uri(namespace, gitlab_uri) + + print(f""" +The following images are stale and can be purged from the registry: +{stale_details} + +You can remove the above images over the API with the following code snipp= et: + + $ for image_id in {stale_ids}; do \\ + curl --request DELETE --header "PRIVATE-TOKEN: " \\ + {registry_uri}/$image_id \\ + done + +You can generate a personal access token here: + {gitlab_uri}/-/profile/personal_access_tokens""") + def action_refresh(self): + # refresh Dockerfiles and vars files self.refresh_containers() self.refresh_cirrus() =20 + # check for stale images + self.check_stale_images() + def run(self): self.args.func(self) =20 diff --git a/ci/util.py b/ci/util.py index f9f8320276..d69c246872 100644 --- a/ci/util.py +++ b/ci/util.py @@ -38,3 +38,55 @@ def get_registry_images(uri: str) -> List[Dict]: =20 # read the HTTP response and load the JSON part of it return json.loads(r.read().decode()) + + +def get_image_distro(image_name: str) -> str: + """ + Extract the name of the distro in the GitLab image registry name, e.g. + ci-debian-9-cross-mipsel --> debian-9 + + :param image_name: name of the GitLab registry image + :return: distro name as a string + """ + name_prefix =3D "ci-" + name_suffix =3D "-cross-" + + distro =3D image_name[len(name_prefix):] + + index =3D distro.find(name_suffix) + if index > 0: + distro =3D distro[:index] + + return distro + + +def get_registry_stale_images(registry_uri: str, + supported_distros: List[str]) -> Dict[str, i= nt]: + """ + Check the GitLab image registry for images that we no longer support a= nd + which should be deleted. + + :param uri: URI pointing to a GitLab instance's image registry + :param supported_distros: list of hosts supported by lcitool + :return: dictionary formatted as: {: } + """ + + images =3D get_registry_images(registry_uri) + + # extract distro names from the list of registry images + registry_distros =3D [get_image_distro(i["name"]) for i in images] + + # - compare the distros powering the images in GitLab registry with + # the list of host available from lcitool + # - @unsupported is a set containing the distro names which we no long= er + # support; we need to map these back to registry image names + unsupported =3D set(registry_distros) - set(supported_distros) + if unsupported: + stale_images =3D {} + for distro in unsupported: + for img in images: + # gitlab images are named like "ci--?" + if distro in img["name"]: + stale_images[img["name"]] =3D img["id"] + + return stale_images --=20 2.29.2 From nobody Fri May 3 01:54:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616054984; cv=none; d=zohomail.com; s=zohoarc; b=XCwio/5hnQkrR7XdBKdOir4JzmoToorGOugQsgNNDdAUrWs29gnyRvMHq392sRKr0fYJHVsWVkp/89hkhKAZDzeXvE5KUstNla6YpS1LjUlEPEjDqKts+l0c4NsS+qt3GhXlMaVCSw11I/1YwhTyGTB5NasSrZIySblphaF5ykQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616054984; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vOMLccc3SbcjAWpAhvgl63D8TBFm8YWlB8rB0TGATls=; b=O2+Z7z0Ge3s6VvkXnOmB8Ee8paS5QWymRdM3R0jvMriPLfsp0nMdiSCqZY+mc3e1kCEECkDaFr2ZV74mXs870eiAfrBFLbUayOXSlj2hpH4M49z3o/bBYV2o+A2oBPOeTZBg6shAwaBrIfrtjRS+QlnI8Hd0BBF5uUj1Hfmo1Rw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1616054984264736.9301451889; Thu, 18 Mar 2021 01:09:44 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-541-Hap5FMFVNdKYKJMupHkP_Q-1; Thu, 18 Mar 2021 04:09:41 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2C64F881280; Thu, 18 Mar 2021 08:09:34 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 03C4350446; Thu, 18 Mar 2021 08:09:34 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B877C50039; Thu, 18 Mar 2021 08:09:33 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12I89ShT030405 for ; Thu, 18 Mar 2021 04:09:28 -0400 Received: by smtp.corp.redhat.com (Postfix) id 6B44F1F041; Thu, 18 Mar 2021 08:09:28 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9D7619C45; Thu, 18 Mar 2021 08:09:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616054983; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=vOMLccc3SbcjAWpAhvgl63D8TBFm8YWlB8rB0TGATls=; b=dI4ZXm3a9PaBYnvK9jOJBf7T8hk9fETyTVzJtFEyD+//LWx2LjsNECLHvnya2U0tMht8ao hDCKRzqT2MgbqbVSPHyjJp5iWMpbdk1Lq4M8+RU5EuzRpZV4mdhbJvwbjR7e7YkmS8hoy+ v+ZncfbzQMyv458qsHDD61mzIOQ6nUc= X-MC-Unique: Hap5FMFVNdKYKJMupHkP_Q-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v3 6/6] ci: helper: Convert private methods to Python common naming practice Date: Thu, 18 Mar 2021 09:09:17 +0100 Message-Id: <6d7f494310d198840c956b3c094fcec2ce1e16a2.1616054919.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As documented at [1], the common practice wrt to private attributes/methods naming is to prefix them with an underscore. [1] https://docs.python.org/3/tutorial/classes.html#private-variables Signed-off-by: Erik Skultety --- ci/helper | 68 +++++++++++++++++++++++++++---------------------------- 1 file changed, 34 insertions(+), 34 deletions(-) diff --git a/ci/helper b/ci/helper index b5255db835..3d2e0397f5 100755 --- a/ci/helper +++ b/ci/helper @@ -97,7 +97,7 @@ class Parser: parents=3D[containerparser, mesonparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) - buildparser.set_defaults(func=3DApplication.action_build) + buildparser.set_defaults(func=3DApplication._action_build) =20 # test action testparser =3D subparsers.add_parser( @@ -106,7 +106,7 @@ class Parser: parents=3D[containerparser, mesonparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) - testparser.set_defaults(func=3DApplication.action_test) + testparser.set_defaults(func=3DApplication._action_test) =20 # shell action shellparser =3D subparsers.add_parser( @@ -115,7 +115,7 @@ class Parser: parents=3D[containerparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) - shellparser.set_defaults(func=3DApplication.action_shell) + shellparser.set_defaults(func=3DApplication._action_shell) =20 # list-images action listimagesparser =3D subparsers.add_parser( @@ -124,7 +124,7 @@ class Parser: parents=3D[gitlabparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) - listimagesparser.set_defaults(func=3DApplication.action_list_image= s) + listimagesparser.set_defaults(func=3DApplication._action_list_imag= es) =20 # refresh action refreshparser =3D subparsers.add_parser( @@ -146,7 +146,7 @@ class Parser: default=3D"yes", help=3D"check for existence of stale images on the GitLab inst= ance" ) - refreshparser.set_defaults(func=3DApplication.action_refresh) + refreshparser.set_defaults(func=3DApplication._action_refresh) =20 def parse(self): return self.parser.parse_args() @@ -161,7 +161,7 @@ class Application: if not shutil.which(self.args.lcitool): sys.exit("error: 'lcitool' not installed") =20 - def make_run(self, target): + def _make_run(self, target): args =3D [ "-C", self.basedir, @@ -185,15 +185,15 @@ class Application: if pty.spawn(["make"] + args) !=3D 0: sys.exit("error: 'make' failed") =20 - def lcitool_run(self, args): + def _lcitool_run(self, args): output =3D subprocess.check_output([self.args.lcitool] + args) return output.decode("utf-8") =20 - def lcitool_get_hosts(self): - output =3D self.lcitool_run(["hosts"]) + def _lcitool_get_hosts(self): + output =3D self._lcitool_run(["hosts"]) return output.splitlines() =20 - def generate_dockerfile(self, host, cross=3DNone): + def _generate_dockerfile(self, host, cross=3DNone): args =3D ["dockerfile", host, "libvirt"] outdir =3D self.basedir.joinpath("containers") outfile =3D f"ci-{host}.Dockerfile" @@ -206,11 +206,11 @@ class Application: if not self.args.quiet: print(outpath) =20 - output =3D self.lcitool_run(args) + output =3D self._lcitool_run(args) with open(outpath, "w") as f: f.write(output) =20 - def generate_vars(self, host): + def _generate_vars(self, host): args =3D ["variables", host, "libvirt"] outdir =3D self.basedir.joinpath("cirrus") outfile =3D f"{host}.vars" @@ -219,11 +219,11 @@ class Application: if not self.args.quiet: print(outpath) =20 - output =3D self.lcitool_run(args) + output =3D self._lcitool_run(args) with open(outpath, "w") as f: f.write(output) =20 - def refresh_containers(self): + def _refresh_containers(self): debian_cross =3D [ "aarch64", "armv6l", @@ -240,39 +240,39 @@ class Application: "mingw64", ] =20 - for host in self.lcitool_get_hosts(): + for host in self._lcitool_get_hosts(): if host.startswith("freebsd-") or host.startswith("macos-"): continue =20 - self.generate_dockerfile(host) + self._generate_dockerfile(host) =20 if host =3D=3D "fedora-rawhide": for cross in fedora_cross: - self.generate_dockerfile(host, cross) + self._generate_dockerfile(host, cross) =20 if host.startswith("debian-"): for cross in debian_cross: if host =3D=3D "debian-sid" and cross =3D=3D "mips": continue - self.generate_dockerfile(host, cross) + self._generate_dockerfile(host, cross) =20 - def refresh_cirrus(self): - for host in self.lcitool_get_hosts(): + def _refresh_cirrus(self): + for host in self._lcitool_get_hosts(): if not (host.startswith("freebsd-") or host.startswith("macos-= ")): continue =20 - self.generate_vars(host) + self._generate_vars(host) =20 - def action_build(self): - self.make_run(f"ci-build@{self.args.target}") + def _action_build(self): + self._make_run(f"ci-build@{self.args.target}") =20 - def action_test(self): - self.make_run(f"ci-test@{self.args.target}") + def _action_test(self): + self._make_run(f"ci-test@{self.args.target}") =20 - def action_shell(self): - self.make_run(f"ci-shell@{self.args.target}") + def _action_shell(self): + self._make_run(f"ci-shell@{self.args.target}") =20 - def action_list_images(self): + def _action_list_images(self): registry_uri =3D util.get_registry_uri(self.args.namespace, self.args.gitlab_uri) images =3D util.get_registry_images(registry_uri) @@ -294,14 +294,14 @@ class Application: print("Available cross-compiler container images:\n") print(spacing + ("\n" + spacing).join(cross)) =20 - def check_stale_images(self): + def _check_stale_images(self): if self.args.check_stale !=3D "yes" or self.args.quiet: return =20 namespace =3D self.args.namespace gitlab_uri =3D self.args.gitlab_uri registry_uri =3D util.get_registry_uri(namespace, gitlab_uri) - lcitool_hosts =3D self.lcitool_get_hosts() + lcitool_hosts =3D self._lcitool_get_hosts() =20 stale_images =3D util.get_registry_stale_images(registry_uri, lcitool_hosts) @@ -326,13 +326,13 @@ You can remove the above images over the API with the= following code snippet: You can generate a personal access token here: {gitlab_uri}/-/profile/personal_access_tokens""") =20 - def action_refresh(self): + def _action_refresh(self): # refresh Dockerfiles and vars files - self.refresh_containers() - self.refresh_cirrus() + self._refresh_containers() + self._refresh_cirrus() =20 # check for stale images - self.check_stale_images() + self._check_stale_images() =20 def run(self): self.args.func(self) --=20 2.29.2