From nobody Thu May 9 13:40:29 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616083923; cv=none; d=zohomail.com; s=zohoarc; b=BJKQ3qlIXBgB01ykliRJxcXUuCW1YfFYs9h0JKPbJfODbsJZ13ccP/YjGYNm546goHAF+YPgsXI9KbncowOWQFkYRd3C3fceIzeGpM4rADf24TrD8sCvvvb7hcUCzSNPpzEbiMolyBn0Aygq3DxG/9G0LbuxOu8jmLsnf///rtI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616083923; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=G4TNT2dj8TenYC+I0ZEykm3B0zcMKFG2Jngw7jJCE1A=; b=Jwc/nYLvQ9drn2oJvYYoIylFYU6ntTb5JclJCwu2wG4cUBrsgDAeOGvyzxMAUgoTDPJKaEGijdVe64E4gY0V3jjagv5aotIDv6kKSFfbtxNOI0br2Z97dP79sqnz2bad05vGF3TZaa4L99k8sSKe4gcS2d0GFO4MwEkHy3GYQHI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1616083923414854.4526439165973; Thu, 18 Mar 2021 09:12:03 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-60-QlrAVaNLOt6C4VtUB_wNew-1; Thu, 18 Mar 2021 12:11:59 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BAF0E84BA6C; Thu, 18 Mar 2021 16:11:54 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9268110589A2; Thu, 18 Mar 2021 16:11:54 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 5326718155DD; Thu, 18 Mar 2021 16:11:54 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12IGBpZX024179 for ; Thu, 18 Mar 2021 12:11:51 -0400 Received: by smtp.corp.redhat.com (Postfix) id E757A60CE6; Thu, 18 Mar 2021 16:11:51 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4338F60CD7; Thu, 18 Mar 2021 16:11:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616083922; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=G4TNT2dj8TenYC+I0ZEykm3B0zcMKFG2Jngw7jJCE1A=; b=RE6iQjc1FafimmK0qXSOljb8dEaQPGkOj+c4G1zP4TYY/lPKC/bLP0QSJfFT5u+rr5QXkg QCxcLxwAfxVKlpAkscqHQkgaeIhv8lffs/cVTtP8G35yhO5ZzuWyeARkFIS0wJAoOV+kXZ 779vteum/+ntlT//GksVDdlLzQLcPps= X-MC-Unique: QlrAVaNLOt6C4VtUB_wNew-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v4 1/6] ci: helper: Use the ArgumentDefaultsHelpFormatter help formatter Date: Thu, 18 Mar 2021 17:11:41 +0100 Message-Id: <5f3358ec1369ddf6022b34d02c6377b99d5c7009.1616083899.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This help formatter class reports the defaults we use for options taking an argument. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/helper | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/ci/helper b/ci/helper index 73a3f729da..8f34f4b59d 100755 --- a/ci/helper +++ b/ci/helper @@ -79,6 +79,7 @@ class Parser: "build", help=3D"run a build in a container", parents=3D[containerparser, mesonparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) buildparser.set_defaults(func=3DApplication.action_build) =20 @@ -87,6 +88,7 @@ class Parser: "test", help=3D"run a build in a container (including tests)", parents=3D[containerparser, mesonparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) testparser.set_defaults(func=3DApplication.action_test) =20 @@ -95,6 +97,7 @@ class Parser: "shell", help=3D"start a shell in a container", parents=3D[containerparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) shellparser.set_defaults(func=3DApplication.action_shell) =20 @@ -102,6 +105,7 @@ class Parser: listimagesparser =3D subparsers.add_parser( "list-images", help=3D"list known container images", + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) listimagesparser.set_defaults(func=3DApplication.action_list_image= s) =20 @@ -110,6 +114,7 @@ class Parser: "refresh", help=3D"refresh data generated with lcitool", parents=3D[lcitoolparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) refreshparser.set_defaults(func=3DApplication.action_refresh) =20 --=20 2.30.2 From nobody Thu May 9 13:40:29 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616083933; cv=none; d=zohomail.com; s=zohoarc; b=hQ3B4KoChz/6s1TCP8FVNOL3QbguG+uMkYHxD/LtICAhZhhV8vq40ozahtFK3f+gpmUGD7QfHG1Lr26yIIXLcKfJVGJX90BWL/FS9uh9YblVkAhb/27FBBE2SX+0nILZ+4jcHWaUjIGCoj/HhvrFEauDobS7TSp19eYvPHtboqc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616083933; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CWBajPV4m4L5yCj5ITd0Rur/S8MhZvHN4n5YM26tRwo=; b=BpdsHtmw/8Y8iD6HSEctHrYFGHREgoi2fWc0vAgOAU9uf0//pPCmpia2C4ljpDImddfXu2+pM4qg+tuwo1EoQG6WXeWtFaa4Lmk0JE/YeANu261VBHgSXkD2wdpeUyXiz++hCNRns8sv8vocWrBJJrbmqHQjmesx0N6yMFSwkPg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1616083933138511.3769537196863; Thu, 18 Mar 2021 09:12:13 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-50-wNCGySU2NJqZm_a8DEAWiA-1; Thu, 18 Mar 2021 12:12:09 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8FFAB108BD07; Thu, 18 Mar 2021 16:12:03 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 651A75D761; Thu, 18 Mar 2021 16:12:03 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 22B584A7CD; Thu, 18 Mar 2021 16:12:03 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12IGBqct024191 for ; Thu, 18 Mar 2021 12:11:52 -0400 Received: by smtp.corp.redhat.com (Postfix) id E76D360CE6; Thu, 18 Mar 2021 16:11:52 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id 429C460CD7; Thu, 18 Mar 2021 16:11:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616083931; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=CWBajPV4m4L5yCj5ITd0Rur/S8MhZvHN4n5YM26tRwo=; b=NrYaTWt4lMV1xPEpQ+rINy1B9hQMxUN9R9OVlLIZzfyCdkrOHySu043K0zo2ohrAi0b4vk lKBMou08Q0pNZiayBil5RlapwAW+4gPUtIEJpIxY5DteuCENwHgxVsPIv8zQZ/WOJX+gO7 Zxivxhq6+Nht1MVKMxLGo/JxJIJYSg0= X-MC-Unique: wNCGySU2NJqZm_a8DEAWiA-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v4 2/6] ci: helper: Introduce --quiet Date: Thu, 18 Mar 2021 17:11:42 +0100 Message-Id: <9100c64bc3cd8838c39264ed83fbf137eb96b860.1616083899.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Offer an option to silence all output to stdout, e.g. when generating dockerfiles/varsfiles. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/helper | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/ci/helper b/ci/helper index 8f34f4b59d..53baef3cb1 100755 --- a/ci/helper +++ b/ci/helper @@ -116,6 +116,12 @@ class Parser: parents=3D[lcitoolparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) + refreshparser.add_argument( + "--quiet", + action=3D"store_true", + default=3DFalse, + help=3D"refresh data silently" + ) refreshparser.set_defaults(func=3DApplication.action_refresh) =20 def parse(self): @@ -173,7 +179,8 @@ class Application: outfile =3D f"ci-{host}-cross-{cross}.Dockerfile" =20 outpath =3D outdir.joinpath(outfile) - print(outpath) + if not self.args.quiet: + print(outpath) =20 output =3D self.lcitool_run(args) with open(outpath, "w") as f: @@ -185,7 +192,8 @@ class Application: outfile =3D f"{host}.vars" =20 outpath =3D outdir.joinpath(outfile) - print(outpath) + if not self.args.quiet: + print(outpath) =20 output =3D self.lcitool_run(args) with open(outpath, "w") as f: --=20 2.30.2 From nobody Thu May 9 13:40:29 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616083927; cv=none; d=zohomail.com; s=zohoarc; b=L9eYyOtX//K47QeBV3tYdoApEauHA47UsmbFflj4nBeUMirc6cwEDawOWRBxcofZ3dLfofAf7hnJ1TD8hx8R973XoaYpq6Yh8rqz0r7zMCOjjR4Ael3pTn54eRjJ3ZgK14aTL8EDjUg9zMjeNEdd7QF2LvpVIfvqXfht38mfTHg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616083927; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7Qqx+IuMwpZEkuWCbzlmnEsb3NiVzGK2PLCaLgM1ySc=; b=M7MHfncpUQpJc9VHpEpRs3asNh+hEhMoyd3AQe2g5TqGpdskxxgaumqLRARw9TgSpTBZKO6RZqiPo6OhjXoWVac6jByJkfK6JfRQItcD312rRr60VfhBj1ICqycuh3HdADHnzTMpm/7twflhPSBM+RdJ3DN4ls6bsWUIsJtOG3c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1616083927675206.3755115697237; Thu, 18 Mar 2021 09:12:07 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-205-5-TVzPSnNbSSRBsIA44VEg-1; Thu, 18 Mar 2021 12:12:03 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 13A43801817; Thu, 18 Mar 2021 16:11:57 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E26E35C1D1; Thu, 18 Mar 2021 16:11:56 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id A0DCD4BB7C; Thu, 18 Mar 2021 16:11:56 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12IGBrmv024198 for ; Thu, 18 Mar 2021 12:11:53 -0400 Received: by smtp.corp.redhat.com (Postfix) id E74CC60CEC; Thu, 18 Mar 2021 16:11:53 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id 427D760CD7; Thu, 18 Mar 2021 16:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616083926; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=7Qqx+IuMwpZEkuWCbzlmnEsb3NiVzGK2PLCaLgM1ySc=; b=Rd36qc+JH6GtC6Pytnuxo54IZYeKxw5fjFSPdswVZoWBqrtQzF3DcVtpHHe4tlFlet6/Lr qFt4f9ruhS+f1BYlYF79IBRG0v79O2jnoSXYtyGvBH9r/uIl96h2rkwfa/vJQFVIH+k5yD srPFVh1wywMIAANNadQhWHuvVXH9jAc= X-MC-Unique: 5-TVzPSnNbSSRBsIA44VEg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v4 3/6] ci: Introduce a util module Date: Thu, 18 Mar 2021 17:11:43 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" With the gradual rewrite of the Makefile to the 'helper' script will require helper functions that would better live in a separate util module. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/util.py | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 ci/util.py diff --git a/ci/util.py b/ci/util.py new file mode 100644 index 0000000000..f9f8320276 --- /dev/null +++ b/ci/util.py @@ -0,0 +1,40 @@ +import json +import urllib.request +import urllib.parse + +from typing import Dict, List + + +def get_registry_uri(namespace: str, + gitlab_uri: str =3D "https://gitlab.com") -> str: + """ + Construct a v4 API URI pointing the namespaced project's image registr= y. + + :param namespace: GitLab project namespace, e.g. "libvirt/libvirt" + :param gitlab_uri: GitLab base URI, can be a private deployment + :param api_version: GitLab REST API version number + :return: URI pointing to a namespaced project's image registry + """ + + # this converts something like "libvirt/libvirt" to "libvirt%2Flibvirt" + namespace_urlenc =3D urllib.parse.quote_plus(namespace) + + project_uri =3D f"{gitlab_uri}/api/v4/projects/{namespace_urlenc}" + + uri =3D project_uri + "/registry/repositories" + return uri + + +def get_registry_images(uri: str) -> List[Dict]: + """ + List all container images that are currently available in the given Gi= tLab + project. + + :param uri: URI pointing to a GitLab instance's image registry + :return: list of container image names + """ + + r =3D urllib.request.urlopen(uri + "?per_page=3D100") + + # read the HTTP response and load the JSON part of it + return json.loads(r.read().decode()) --=20 2.30.2 From nobody Thu May 9 13:40:29 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) client-ip=63.128.21.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616083945; cv=none; d=zohomail.com; s=zohoarc; b=jzgqXGFTFiEzijop86cwyo1omYSLmxd162TajWtUXfxgxWeIkO3E5MyhHjyrq962nC0TUg58nhQ97nCvxZVmGnG6M+rm1nDlbOohAHLPDTPNX4Rzbn1MPgxcKBaShQZfSZBzmHuW6qJ9viU+W70uDY8Urwigt8h6EUQQFX5lQKU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616083945; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=zecwz5wqw9mdGjV/g/rTzPo5R5xyepQKqXOVYHFtNmY=; b=guLYAETsK7DaySzCZtL+9e6v3i1cAC1Wtkv6hMLnlQ3qXhlmyGYF8gMYTXJXfTMTCh7r+bMSkQ95hsj0+x9h9LEx5Jk97Tnvqlu1QsA//ndIIII2XPYvaT5uZuMyooq98/RYMg6Ts959HlF3Gj42YqoRvV3a4Bhy0omPvZ+ETLo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mx.zohomail.com with SMTPS id 1616083945526937.6714249670263; Thu, 18 Mar 2021 09:12:25 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-336-xmMq4vv_MDqdQHDLVVSuVw-1; Thu, 18 Mar 2021 12:12:09 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 446FB84BA4D; Thu, 18 Mar 2021 16:12:03 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0B20119726; Thu, 18 Mar 2021 16:12:03 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B36E14A7CB; Thu, 18 Mar 2021 16:12:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12IGBsN0024208 for ; Thu, 18 Mar 2021 12:11:54 -0400 Received: by smtp.corp.redhat.com (Postfix) id E658E60938; Thu, 18 Mar 2021 16:11:54 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43AE66B8E5; Thu, 18 Mar 2021 16:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616083944; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=zecwz5wqw9mdGjV/g/rTzPo5R5xyepQKqXOVYHFtNmY=; b=e90F5/8hwVN8V6VNSJkKyIQ/oLiBAel3+aLrT5EsssC6O10313s2wkuT/BhhOVr+fwyZTI JLbtFb6OKIS3T9ub9rB+wV79+knWJxbsB3gg3sv7h1WLQGj2M4wBM4PVAL0n6ALb66w9sR Gw3hb8ADgZKDIjxc0T8AZUrXo1KWNW0= X-MC-Unique: xmMq4vv_MDqdQHDLVVSuVw-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v4 4/6] ci: helper: Rewrite image listing to Python Date: Thu, 18 Mar 2021 17:11:44 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" The corresponding Bash script is dropped. After this patch's rewrite, the Makefile's original image listing target remains intact only to notify the user to use the Python helper instead. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/Makefile | 12 ------------ ci/helper | 38 +++++++++++++++++++++++++++++++++++++- ci/list-images.sh | 14 -------------- 3 files changed, 37 insertions(+), 27 deletions(-) delete mode 100644 ci/list-images.sh diff --git a/ci/Makefile b/ci/Makefile index 72f5bda942..02ce0df7ee 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -211,17 +211,6 @@ ci-build@%: ci-test@%: $(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_NINJA_ARGS=3Dtest =20 -ci-list-images: - @echo - @echo "Available x86 container images:" - @echo - @sh list-images.sh "$(CI_IMAGE_PREFIX)" | grep -v cross - @echo - @echo "Available cross-compiler container images:" - @echo - @sh list-images.sh "$(CI_IMAGE_PREFIX)" | grep cross - @echo - ci-help: @echo @echo @@ -241,7 +230,6 @@ ci-help: @echo " ci-build@\$$IMAGE - run a default 'ninja' build" @echo " ci-test@\$$IMAGE - run a 'ninja test'" @echo " ci-shell@\$$IMAGE - run an interactive shell" - @echo " ci-list-images - list available images" @echo " ci-help - show this help message" @echo @echo "Available make variables:" diff --git a/ci/helper b/ci/helper index 53baef3cb1..31cf72fbdf 100755 --- a/ci/helper +++ b/ci/helper @@ -11,6 +11,8 @@ import shutil import subprocess import sys =20 +import util + =20 class Parser: def __init__(self): @@ -66,6 +68,20 @@ class Parser: help=3D"path to lcitool binary", ) =20 + # Options that are common to actions communicating with a GitLab + # instance + gitlabparser =3D argparse.ArgumentParser(add_help=3DFalse) + gitlabparser.add_argument( + "--namespace", + default=3D"libvirt/libvirt", + help=3D"GitLab project namespace" + ) + gitlabparser.add_argument( + "--gitlab-uri", + default=3D"https://gitlab.com", + help=3D"base GitLab URI" + ) + # Main parser self.parser =3D argparse.ArgumentParser() subparsers =3D self.parser.add_subparsers( @@ -105,6 +121,7 @@ class Parser: listimagesparser =3D subparsers.add_parser( "list-images", help=3D"list known container images", + parents=3D[gitlabparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) listimagesparser.set_defaults(func=3DApplication.action_list_image= s) @@ -249,7 +266,26 @@ class Application: self.make_run(f"ci-shell@{self.args.target}") =20 def action_list_images(self): - self.make_run(f"ci-list-images") + registry_uri =3D util.get_registry_uri(self.args.namespace, + self.args.gitlab_uri) + images =3D util.get_registry_images(registry_uri) + + # skip the "ci-" prefix each of our container images' name has + name_prefix =3D "ci-" + names =3D [i["name"][len(name_prefix):] for i in images] + names.sort() + + native =3D [name for name in names if "-cross-" not in name] + cross =3D [name for name in names if "-cross-" in name] + + spacing =3D 4 * " " + print("Available x86 container images:\n") + print(spacing + ("\n" + spacing).join(native)) + + if cross: + print() + print("Available cross-compiler container images:\n") + print(spacing + ("\n" + spacing).join(cross)) =20 def action_refresh(self): self.refresh_containers() diff --git a/ci/list-images.sh b/ci/list-images.sh deleted file mode 100644 index b85b132253..0000000000 --- a/ci/list-images.sh +++ /dev/null @@ -1,14 +0,0 @@ -#!/bin/sh - -prefix=3D"${1##registry.gitlab.com/}" - -PROJECT_ID=3D192693 - -all_repos() { - curl -s "https://gitlab.com/api/v4/projects/$PROJECT_ID/registry/reposit= ories?per_page=3D100" \ - | tr , '\n' | grep '"path":' | sed 's,"path":",,g;s,"$,,g' -} - -all_repos | grep "^$prefix" | sed "s,^$prefix,,g" | while read repo; do - echo " $repo" -done | sort -u --=20 2.30.2 From nobody Thu May 9 13:40:29 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) client-ip=63.128.21.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616083937; cv=none; d=zohomail.com; s=zohoarc; b=iSE17XLhnXGPFS9f+VJjjralz7SeOgulEVb/J8oLkxWqUA1iBmZ0viyMdlEYyyH/3SR0oO8IT3oqKTmTKtfPA9AF1e6t7dc5Il+rH3A1nwS6qbeyJvmSbwu1L+VV0YCVQ/ECwVsbn40L+L5Ec5aL50X4B3u7Kv+OyQ+XJDJx3ck= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616083937; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=L1j2yTxLBvG/HASoJPR3sGyATiwme4Td/Z1Rx7hC64E=; b=Xz3TdQDqsZVLzU9eYKnBq3faqut7n5bpT/LBUTmjau/3mDM71jb82er1UKT3TmjCmLAoTJuwCqKTKrCKXh1ATeF0lckkzen+4byeLtxPL1zYxn2Y0TNJQw7S6THwPZZN9cWgHnCNFSaQpwbjGhslui5ga/Wewm1yRQgH3HGU1AA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mx.zohomail.com with SMTPS id 1616083937425172.6548269730107; Thu, 18 Mar 2021 09:12:17 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-472-1tpnNhBqO6-Wbj0IBb4k8w-1; Thu, 18 Mar 2021 12:12:13 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 30890107B7C5; Thu, 18 Mar 2021 16:12:07 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 095C260CEC; Thu, 18 Mar 2021 16:12:07 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id BAD6E4A705; Thu, 18 Mar 2021 16:12:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12IGBtOD024219 for ; Thu, 18 Mar 2021 12:11:55 -0400 Received: by smtp.corp.redhat.com (Postfix) id E69A917DC3; Thu, 18 Mar 2021 16:11:55 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id 41EFA60CD7; Thu, 18 Mar 2021 16:11:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616083936; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=L1j2yTxLBvG/HASoJPR3sGyATiwme4Td/Z1Rx7hC64E=; b=Bk9D406PISMiDBNwm6hYV3iHXnRvcFSAzP10Ba0LKAOj7Kk6hs9RQE1IvO1ym3qjWkge71 adbdeSKasRPXEgQfthT6alXjUfCK/mTpUMrtG/fK3oqFu24MUEdxJOJ8eN2UUkvhw/e+yj MuzMDyXLs60jkf6V5qZS24A/BUZjeRo= X-MC-Unique: 1tpnNhBqO6-Wbj0IBb4k8w-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v4 5/6] ci: util: Add a registry checker for stale images Date: Thu, 18 Mar 2021 17:11:45 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This function checks whether there are any stale Docker images in the registry that can be purged. Since we're pulling available container images from our GitLab registry with the 'list-images' action, it could happen that we'd list old (already unsupported) images and make them available for the user to consume and run a build in them. Naturally, the build will most likely fail leaving the user confused. Signed-off-by: Erik Skultety --- ci/helper | 46 +++++++++++++++++++++++++++++++++++++++++++++- ci/util.py | 41 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+), 1 deletion(-) diff --git a/ci/helper b/ci/helper index 31cf72fbdf..0743f95e13 100755 --- a/ci/helper +++ b/ci/helper @@ -10,6 +10,7 @@ import pty import shutil import subprocess import sys +import textwrap =20 import util =20 @@ -130,7 +131,7 @@ class Parser: refreshparser =3D subparsers.add_parser( "refresh", help=3D"refresh data generated with lcitool", - parents=3D[lcitoolparser], + parents=3D[lcitoolparser, gitlabparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) refreshparser.add_argument( @@ -139,6 +140,13 @@ class Parser: default=3DFalse, help=3D"refresh data silently" ) + refreshparser.add_argument( + "--check-stale", + action=3D"store", + choices=3D["yes", "no"], + default=3D"yes", + help=3D"check for existence of stale images on the GitLab inst= ance" + ) refreshparser.set_defaults(func=3DApplication.action_refresh) =20 def parse(self): @@ -287,10 +295,46 @@ class Application: print("Available cross-compiler container images:\n") print(spacing + ("\n" + spacing).join(cross)) =20 + def check_stale_images(self): + namespace =3D self.args.namespace + gitlab_uri =3D self.args.gitlab_uri + registry_uri =3D util.get_registry_uri(namespace, gitlab_uri) + lcitool_hosts =3D self.lcitool_get_hosts() + + stale_images =3D util.get_registry_stale_images(registry_uri, + lcitool_hosts) + if stale_images: + spacing =3D "\n" + 4 * " " + stale_fmt =3D [f"{k} (ID: {v})" for k, v in stale_images.items= ()] + stale_details =3D spacing.join(stale_fmt) + stale_ids =3D ' '.join([str(id) for id in stale_images.values(= )]) + registry_uri =3D util.get_registry_uri(namespace, gitlab_uri) + + msg =3D textwrap.dedent(f""" + The following images are stale and can be purged from the = registry: + + STALE_DETAILS + + You can delete the images listed above using this shell sn= ippet: + + $ for image_id in {stale_ids}; do + curl --request DELETE --header "PRIVATE-TOKEN: " \\ + {registry_uri}/$image_id; + done + + You can generate a personal access token here: + + {gitlab_uri}/-/profile/personal_access_tokens + """) + print(msg.replace("STALE_DETAILS", stale_details)) + def action_refresh(self): self.refresh_containers() self.refresh_cirrus() =20 + if self.args.check_stale =3D=3D "yes" and not self.args.quiet: + self.check_stale_images() + def run(self): self.args.func(self) =20 diff --git a/ci/util.py b/ci/util.py index f9f8320276..90d58454be 100644 --- a/ci/util.py +++ b/ci/util.py @@ -38,3 +38,44 @@ def get_registry_images(uri: str) -> List[Dict]: =20 # read the HTTP response and load the JSON part of it return json.loads(r.read().decode()) + + +def get_image_distro(image_name: str) -> str: + """ + Extract the name of the distro in the GitLab image registry name, e.g. + ci-debian-9-cross-mipsel --> debian-9 + + :param image_name: name of the GitLab registry image + :return: distro name as a string + """ + name_prefix =3D "ci-" + name_suffix =3D "-cross-" + + distro =3D image_name[len(name_prefix):] + + index =3D distro.find(name_suffix) + if index > 0: + distro =3D distro[:index] + + return distro + + +def get_registry_stale_images(registry_uri: str, + supported_distros: List[str]) -> Dict[str, i= nt]: + """ + Check the GitLab image registry for images that we no longer support a= nd + which should be deleted. + + :param uri: URI pointing to a GitLab instance's image registry + :param supported_distros: list of hosts supported by lcitool + :return: dictionary formatted as: {: } + """ + + images =3D get_registry_images(registry_uri) + + stale_images =3D {} + for img in images: + if get_image_distro(img["name"]) not in supported_distros: + stale_images[img["name"]] =3D img["id"] + + return stale_images --=20 2.30.2 From nobody Thu May 9 13:40:29 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1616083933; cv=none; d=zohomail.com; s=zohoarc; b=BLSEEsmK/0/TAfK+NPjx7hhiu0jGP8GTO/ngWto5uIMWmdWsiqzzi1FZ5l9qLnHqDyA+B/LNk3ROFpf+MNfh08ow1Nbll4kNvLxdCG7l9ik6AwLFu9b2NKcOLRDdcWc70y9rtFS8DpQ/EaQzbnKvdCkEFW2RdnfzuKDRYoifux8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616083933; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fu4XcQPdwE9hWYZYGnAUBFEdu/3sA2Ph7DDfTv9klFk=; b=galYEGoxedegsggIr9uLCh1W0pT/Rn0G+/C/LFzhvpyQpyOCFSs53GD8SyObSalE0R9b2pUIZsvgoZ+S1mMi61w+HFKCdaZh7DeZecaSWpW2iYwlIEJTFwHNvTmnyJ7Jc7y62ut5Bxxniz+fg8QAWhX/KTVEGORZP7lS3zaenMg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1616083933740674.6089294576078; Thu, 18 Mar 2021 09:12:13 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-459-aOSHtUgiM82Y6e5zTqp7yQ-1; Thu, 18 Mar 2021 12:12:09 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3E047107B7C4; Thu, 18 Mar 2021 16:12:03 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 17B7B37DD; Thu, 18 Mar 2021 16:12:03 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B6380180B451; Thu, 18 Mar 2021 16:12:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12IGBuWj024234 for ; Thu, 18 Mar 2021 12:11:56 -0400 Received: by smtp.corp.redhat.com (Postfix) id E695560938; Thu, 18 Mar 2021 16:11:56 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id 426C660CDF; Thu, 18 Mar 2021 16:11:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616083932; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=fu4XcQPdwE9hWYZYGnAUBFEdu/3sA2Ph7DDfTv9klFk=; b=Jzy28WBG/59HTjI5Oqx4+affXwIQwkOYbSnpuhIrSO1mUqjBmex4G3DvjCvb1W5RsuJSst DQr/fh7yR+KpaabYqQRY+cFQ8+4gORq7evshC5Iw7VnOpLFW9Ih5t9vzHymj1SfawzYHMI 1BBoA6gOgiDSky7jww5E+Sxl+2FFFWs= X-MC-Unique: aOSHtUgiM82Y6e5zTqp7yQ-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v4 6/6] ci: helper: Apply Python naming practice to private methods/attributes Date: Thu, 18 Mar 2021 17:11:46 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As documented at [1], the common practice with respect to private attributes/methods naming is to prefix them with an underscore. [1] https://docs.python.org/3/tutorial/classes.html#private-variables Signed-off-by: Erik Skultety --- ci/helper | 122 +++++++++++++++++++++++++++--------------------------- 1 file changed, 61 insertions(+), 61 deletions(-) diff --git a/ci/helper b/ci/helper index 0743f95e13..70c9cb4f6b 100755 --- a/ci/helper +++ b/ci/helper @@ -84,8 +84,8 @@ class Parser: ) =20 # Main parser - self.parser =3D argparse.ArgumentParser() - subparsers =3D self.parser.add_subparsers( + self._parser =3D argparse.ArgumentParser() + subparsers =3D self._parser.add_subparsers( dest=3D"action", metavar=3D"ACTION", ) @@ -98,7 +98,7 @@ class Parser: parents=3D[containerparser, mesonparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) - buildparser.set_defaults(func=3DApplication.action_build) + buildparser.set_defaults(func=3DApplication._action_build) =20 # test action testparser =3D subparsers.add_parser( @@ -107,7 +107,7 @@ class Parser: parents=3D[containerparser, mesonparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) - testparser.set_defaults(func=3DApplication.action_test) + testparser.set_defaults(func=3DApplication._action_test) =20 # shell action shellparser =3D subparsers.add_parser( @@ -116,7 +116,7 @@ class Parser: parents=3D[containerparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) - shellparser.set_defaults(func=3DApplication.action_shell) + shellparser.set_defaults(func=3DApplication._action_shell) =20 # list-images action listimagesparser =3D subparsers.add_parser( @@ -125,7 +125,7 @@ class Parser: parents=3D[gitlabparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) - listimagesparser.set_defaults(func=3DApplication.action_list_image= s) + listimagesparser.set_defaults(func=3DApplication._action_list_imag= es) =20 # refresh action refreshparser =3D subparsers.add_parser( @@ -147,56 +147,56 @@ class Parser: default=3D"yes", help=3D"check for existence of stale images on the GitLab inst= ance" ) - refreshparser.set_defaults(func=3DApplication.action_refresh) + refreshparser.set_defaults(func=3DApplication._action_refresh) =20 def parse(self): - return self.parser.parse_args() + return self._parser.parse_args() =20 =20 class Application: def __init__(self): - self.basedir =3D pathlib.Path(__file__).resolve().parent - self.args =3D Parser().parse() + self._basedir =3D pathlib.Path(__file__).resolve().parent + self._args =3D Parser().parse() =20 - if self.args.action =3D=3D "refresh": - if not shutil.which(self.args.lcitool): + if self._args.action =3D=3D "refresh": + if not shutil.which(self._args.lcitool): sys.exit("error: 'lcitool' not installed") =20 - def make_run(self, target): + def _make_run(self, target): args =3D [ "-C", - self.basedir, + self._basedir, target, ] =20 - if self.args.action in ["build", "test", "shell"]: + if self._args.action in ["build", "test", "shell"]: args.extend([ - f"CI_ENGINE=3D{self.args.engine}", - f"CI_USER_LOGIN=3D{self.args.login}", - f"CI_IMAGE_PREFIX=3D{self.args.image_prefix}", - f"CI_IMAGE_TAG=3D{self.args.image_tag}", + f"CI_ENGINE=3D{self._args.engine}", + f"CI_USER_LOGIN=3D{self._args.login}", + f"CI_IMAGE_PREFIX=3D{self._args.image_prefix}", + f"CI_IMAGE_TAG=3D{self._args.image_tag}", ]) =20 - if self.args.action in ["build", "test"]: + if self._args.action in ["build", "test"]: args.extend([ - f"CI_MESON_ARGS=3D{self.args.meson_args}", - f"CI_NINJA_ARGS=3D{self.args.ninja_args}", + f"CI_MESON_ARGS=3D{self._args.meson_args}", + f"CI_NINJA_ARGS=3D{self._args.ninja_args}", ]) =20 if pty.spawn(["make"] + args) !=3D 0: sys.exit("error: 'make' failed") =20 - def lcitool_run(self, args): - output =3D subprocess.check_output([self.args.lcitool] + args) + def _lcitool_run(self, args): + output =3D subprocess.check_output([self._args.lcitool] + args) return output.decode("utf-8") =20 - def lcitool_get_hosts(self): - output =3D self.lcitool_run(["hosts"]) + def _lcitool_get_hosts(self): + output =3D self._lcitool_run(["hosts"]) return output.splitlines() =20 - def generate_dockerfile(self, host, cross=3DNone): + def _generate_dockerfile(self, host, cross=3DNone): args =3D ["dockerfile", host, "libvirt"] - outdir =3D self.basedir.joinpath("containers") + outdir =3D self._basedir.joinpath("containers") outfile =3D f"ci-{host}.Dockerfile" =20 if cross: @@ -204,27 +204,27 @@ class Application: outfile =3D f"ci-{host}-cross-{cross}.Dockerfile" =20 outpath =3D outdir.joinpath(outfile) - if not self.args.quiet: + if not self._args.quiet: print(outpath) =20 - output =3D self.lcitool_run(args) + output =3D self._lcitool_run(args) with open(outpath, "w") as f: f.write(output) =20 - def generate_vars(self, host): + def _generate_vars(self, host): args =3D ["variables", host, "libvirt"] - outdir =3D self.basedir.joinpath("cirrus") + outdir =3D self._basedir.joinpath("cirrus") outfile =3D f"{host}.vars" =20 outpath =3D outdir.joinpath(outfile) - if not self.args.quiet: + if not self._args.quiet: print(outpath) =20 - output =3D self.lcitool_run(args) + output =3D self._lcitool_run(args) with open(outpath, "w") as f: f.write(output) =20 - def refresh_containers(self): + def _refresh_containers(self): debian_cross =3D [ "aarch64", "armv6l", @@ -241,41 +241,41 @@ class Application: "mingw64", ] =20 - for host in self.lcitool_get_hosts(): + for host in self._lcitool_get_hosts(): if host.startswith("freebsd-") or host.startswith("macos-"): continue =20 - self.generate_dockerfile(host) + self._generate_dockerfile(host) =20 if host =3D=3D "fedora-rawhide": for cross in fedora_cross: - self.generate_dockerfile(host, cross) + self._generate_dockerfile(host, cross) =20 if host.startswith("debian-"): for cross in debian_cross: if host =3D=3D "debian-sid" and cross =3D=3D "mips": continue - self.generate_dockerfile(host, cross) + self._generate_dockerfile(host, cross) =20 - def refresh_cirrus(self): - for host in self.lcitool_get_hosts(): + def _refresh_cirrus(self): + for host in self._lcitool_get_hosts(): if not (host.startswith("freebsd-") or host.startswith("macos-= ")): continue =20 - self.generate_vars(host) + self._generate_vars(host) =20 - def action_build(self): - self.make_run(f"ci-build@{self.args.target}") + def _action_build(self): + self._make_run(f"ci-build@{self._args.target}") =20 - def action_test(self): - self.make_run(f"ci-test@{self.args.target}") + def _action_test(self): + self._make_run(f"ci-test@{self._args.target}") =20 - def action_shell(self): - self.make_run(f"ci-shell@{self.args.target}") + def _action_shell(self): + self._make_run(f"ci-shell@{self._args.target}") =20 - def action_list_images(self): - registry_uri =3D util.get_registry_uri(self.args.namespace, - self.args.gitlab_uri) + def _action_list_images(self): + registry_uri =3D util.get_registry_uri(self._args.namespace, + self._args.gitlab_uri) images =3D util.get_registry_images(registry_uri) =20 # skip the "ci-" prefix each of our container images' name has @@ -295,11 +295,11 @@ class Application: print("Available cross-compiler container images:\n") print(spacing + ("\n" + spacing).join(cross)) =20 - def check_stale_images(self): - namespace =3D self.args.namespace - gitlab_uri =3D self.args.gitlab_uri + def _check_stale_images(self): + namespace =3D self._args.namespace + gitlab_uri =3D self._args.gitlab_uri registry_uri =3D util.get_registry_uri(namespace, gitlab_uri) - lcitool_hosts =3D self.lcitool_get_hosts() + lcitool_hosts =3D self._lcitool_get_hosts() =20 stale_images =3D util.get_registry_stale_images(registry_uri, lcitool_hosts) @@ -328,15 +328,15 @@ class Application: """) print(msg.replace("STALE_DETAILS", stale_details)) =20 - def action_refresh(self): - self.refresh_containers() - self.refresh_cirrus() + def _action_refresh(self): + self._refresh_containers() + self._refresh_cirrus() =20 - if self.args.check_stale =3D=3D "yes" and not self.args.quiet: - self.check_stale_images() + if self._args.check_stale =3D=3D "yes" and not self._args.quiet: + self._check_stale_images() =20 def run(self): - self.args.func(self) + self._args.func(self) =20 =20 if __name__ =3D=3D "__main__": --=20 2.30.2