From nobody Mon May 6 17:54:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) client-ip=63.128.21.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615916000; cv=none; d=zohomail.com; s=zohoarc; b=PZ2KkelzPXo46s3Z3o3nrRfp+QECOlMN8s/j0naaStFAxtERtEddnBzdHbzozkIQHr4DldD1zwLq7bVG24zKgw1EGgXTcLAVMcbGstRlUp/7X5HtNnywp6ZZiC1HBy08+E31lGXr6tI9mVI3vQ/QF7KjJQKggZKvUfy1r6CFXlM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615916000; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FgiIJEp+iT4+yJn8+lJmGuQsKxYPr4oA0rr/Qa1hCuM=; b=nLI1PcU3BK9IWJwQUf2g3Yor0/hlg4Ut1pROTGHqr5FAqUk88Xcj98ep+QG5NAuSuh9z1B8MGtgDiN9PW9uwSokMVyAu1xhwURJOYCFmZQiZj+R3M+8zm1uBUXOMmtQdib10i4+2qDWbWo5n1ku9PE9n+OQvymRl7J/DIFsyjyY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mx.zohomail.com with SMTPS id 161591600017625.08267840794315; Tue, 16 Mar 2021 10:33:20 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-416-8U1puHMgMoKj0Ek66NRU2g-1; Tue, 16 Mar 2021 13:33:15 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5B1F91084D68; Tue, 16 Mar 2021 17:33:10 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0DE23189B6; Tue, 16 Mar 2021 17:33:10 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id E77AD85F5; Tue, 16 Mar 2021 17:33:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12GHX6u6011372 for ; Tue, 16 Mar 2021 13:33:06 -0400 Received: by smtp.corp.redhat.com (Postfix) id 2A0475C1C2; Tue, 16 Mar 2021 17:33:06 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.7]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4CE585C1A1; Tue, 16 Mar 2021 17:33:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615915998; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=FgiIJEp+iT4+yJn8+lJmGuQsKxYPr4oA0rr/Qa1hCuM=; b=YAwEHLfSpwywQicu+pBYZE+Kb7GO5KO+oQyOYAcOzk76y+9SdzsCA/cguWv8gFyXFYmmTx Cb2/5k2GdpFC19PQYxEtq5Gp3dHsejnroextz/32riRITTedpEZLF4m86kFo/rWKNxfy4o 0vn/6ZdExNLA9oVhVaxt8PJlL51mZ6w= X-MC-Unique: 8U1puHMgMoKj0Ek66NRU2g-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v2 1/6] ci: Makefile: Specify a help target to replace ci-help Date: Tue, 16 Mar 2021 18:32:55 +0100 Message-Id: <0ca826156a3c594083122c1afcd9a1db0d5b9cc9.1615915976.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" It's quite pointless to have a 'ci-help' target in the Makefile when one needs to actually open the Makefile to go through the list of targets to know what functionality the Makefile actually provides. It's much more intuitive to run "make help" in that case. Therefore, add a 'help' target and replace the old 'ci-help' target with it. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ci/Makefile b/ci/Makefile index 72f5bda942..f83ecac1e5 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -222,7 +222,7 @@ ci-list-images: @sh list-images.sh "$(CI_IMAGE_PREFIX)" | grep cross @echo =20 -ci-help: +help: @echo @echo @echo @@ -242,7 +242,7 @@ ci-help: @echo " ci-test@\$$IMAGE - run a 'ninja test'" @echo " ci-shell@\$$IMAGE - run an interactive shell" @echo " ci-list-images - list available images" - @echo " ci-help - show this help message" + @echo " help - show this help message" @echo @echo "Available make variables:" @echo --=20 2.29.2 From nobody Mon May 6 17:54:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615916000; cv=none; d=zohomail.com; s=zohoarc; b=ThDoPHgIIOC+qs3a2xu30YTL+NZi5LnTfBckPrdf8vdv3RBUhrehxkaD2yZZ+pWO32Jgg6YlAH9MpXcNCR/QWw6g7Z8f6iCetmVZuk6bLBMZKHEC6oBRqnWGwetrz/xni/U0w+LvZXu798QQYR1cMjwdgyKpKaMgdVTnCrcfaSs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615916000; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Gv/0jkmVL2YY/lrwbw0WN2RBpR+JnKCxmULzBdlhFRI=; b=lWdYGtDCyiGBCNY50vj7fcydCRjn58ETvwEvbDyghNrlewkZxIKZ50veaYw0dJMNGQRanT5IEa0z1lUG1G93nA9iXShyVbst7X6sZ+qtkaoKp5FiTp7Yh8GIYkr/nR/15odvSeVHT+g0aJ3qtZz47Y0pynxOT6fEYGNJN5xLiAg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1615916000639812.091971127569; Tue, 16 Mar 2021 10:33:20 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-379-oPMeXrsXNZKAHdTLx0cAOQ-1; Tue, 16 Mar 2021 13:33:16 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 18FD7800C78; Tue, 16 Mar 2021 17:33:10 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C9FB71001B2C; Tue, 16 Mar 2021 17:33:09 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 843861800657; Tue, 16 Mar 2021 17:33:09 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12GHX7r6011383 for ; Tue, 16 Mar 2021 13:33:07 -0400 Received: by smtp.corp.redhat.com (Postfix) id 2DEAA5C1C2; Tue, 16 Mar 2021 17:33:07 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.7]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7CD105C1A1; Tue, 16 Mar 2021 17:33:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615915999; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Gv/0jkmVL2YY/lrwbw0WN2RBpR+JnKCxmULzBdlhFRI=; b=GSZUKFnPXJElBUSXY94OIh8ghph8DaZA5sPDRBO3WVFFYuDKkgNcGXmlMxRsjJp+rN6U6F vctrscpEEpm6by/NRmo0go173u+Yt0ElDgA+hcvS9BDdKprKxvJBy9/bVwB32bApYA+1jP XxxFzfzpFFsXQaAZ88j4HAZL3zKgDsM= X-MC-Unique: oPMeXrsXNZKAHdTLx0cAOQ-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v2 2/6] ci: helper: Use the ArgumentDefaultsHelpFormatter help formatter Date: Tue, 16 Mar 2021 18:32:56 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This help formatter class reports the defaults we use for options taking an argument. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/helper | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/ci/helper b/ci/helper index 73a3f729da..8f34f4b59d 100755 --- a/ci/helper +++ b/ci/helper @@ -79,6 +79,7 @@ class Parser: "build", help=3D"run a build in a container", parents=3D[containerparser, mesonparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) buildparser.set_defaults(func=3DApplication.action_build) =20 @@ -87,6 +88,7 @@ class Parser: "test", help=3D"run a build in a container (including tests)", parents=3D[containerparser, mesonparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) testparser.set_defaults(func=3DApplication.action_test) =20 @@ -95,6 +97,7 @@ class Parser: "shell", help=3D"start a shell in a container", parents=3D[containerparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) shellparser.set_defaults(func=3DApplication.action_shell) =20 @@ -102,6 +105,7 @@ class Parser: listimagesparser =3D subparsers.add_parser( "list-images", help=3D"list known container images", + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) listimagesparser.set_defaults(func=3DApplication.action_list_image= s) =20 @@ -110,6 +114,7 @@ class Parser: "refresh", help=3D"refresh data generated with lcitool", parents=3D[lcitoolparser], + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) refreshparser.set_defaults(func=3DApplication.action_refresh) =20 --=20 2.29.2 From nobody Mon May 6 17:54:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615916012; cv=none; d=zohomail.com; s=zohoarc; b=d01V3Nq1pNlrWVrs9/c4BvWCKKfymgZUJM7hp+EUQeM5PlwaqeQ7Or8qpbtSF3DIWaJN24HmsOkRj18xKSkwG/M1nrslempknjwzKpOEBWIH4tDPtExuMKpg32j1oXk4fMYTA60RJjh/4jkHL/e4zFQXoSr85S1WbeObie6lWKE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615916012; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=joNHph4mv05g7MIInF2RqIapQPzGk4pz/RxOxZFRLbY=; b=BEBQBJPvJrmodguiTuYw6xVA6LOf5/hnCLM69csffOfM6cBkmSdmXg6NHxcvGknIGb6HOdDRKhFe/4IRqXGD3MSDsLsow3WC3AGdTwB6mh35RwanW1lzgifvnhKb3AhXEymBtk8KERdhimi2TDpltuQtMRhRnyRN8bb4rJIKHjE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1615916012675769.6202569682546; Tue, 16 Mar 2021 10:33:32 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-307-XG6neXjDPRmYgjwVuIahjA-1; Tue, 16 Mar 2021 13:33:29 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9DA7E801596; Tue, 16 Mar 2021 17:33:22 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 734DD1001B2C; Tue, 16 Mar 2021 17:33:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3636A1800B8B; Tue, 16 Mar 2021 17:33:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12GHX8uc011391 for ; Tue, 16 Mar 2021 13:33:08 -0400 Received: by smtp.corp.redhat.com (Postfix) id 313B55C1C2; Tue, 16 Mar 2021 17:33:08 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.7]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7EB525C1A1; Tue, 16 Mar 2021 17:33:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615916011; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=joNHph4mv05g7MIInF2RqIapQPzGk4pz/RxOxZFRLbY=; b=dcZQrQ3MGeObfZcqQwBAgzqommCSuW3mBknmvwygWQhysP0aYmbZeIssCs5/yFlPWfOkiO kHP5zUr04YxrU88JKEuA15W209ngrEBqmywpcZEfCYgnCWgVvI/XqWtySRf+2ZOiytClgU Xd8m5QpJ87om+I895KMrWbyCKvL+rso= X-MC-Unique: XG6neXjDPRmYgjwVuIahjA-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v2 3/6] ci: helper: Introduce --quiet Date: Tue, 16 Mar 2021 18:32:57 +0100 Message-Id: <82bd6ee76b879644a2266ec0e45b1bd34e623f3a.1615915976.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Offer an option to silence all output to stdout, e.g. when generating dockerfiles/varsfiles. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- ci/helper | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/ci/helper b/ci/helper index 8f34f4b59d..abac686d22 100755 --- a/ci/helper +++ b/ci/helper @@ -68,6 +68,12 @@ class Parser: =20 # Main parser self.parser =3D argparse.ArgumentParser() + self.parser.add_argument( + "--quiet", + action=3D"store_true", + default=3DFalse, + help=3D"silence all output" + ) subparsers =3D self.parser.add_subparsers( dest=3D"action", metavar=3D"ACTION", @@ -173,7 +179,8 @@ class Application: outfile =3D f"ci-{host}-cross-{cross}.Dockerfile" =20 outpath =3D outdir.joinpath(outfile) - print(outpath) + if not self.args.quiet: + print(outpath) =20 output =3D self.lcitool_run(args) with open(outpath, "w") as f: @@ -185,7 +192,8 @@ class Application: outfile =3D f"{host}.vars" =20 outpath =3D outdir.joinpath(outfile) - print(outpath) + if not self.args.quiet: + print(outpath) =20 output =3D self.lcitool_run(args) with open(outpath, "w") as f: --=20 2.29.2 From nobody Mon May 6 17:54:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) client-ip=63.128.21.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615916044; cv=none; d=zohomail.com; s=zohoarc; b=lT0CYElYjDp09BcjSSRPrkLSvT/BxL4xAbLvC1wA4+LNtYZIKqKyzcJdZM+XDqUyULf2tMr0mRAQlK/eAF4Xn5CVVRr22XWv3XixmcIjmJToL7LZqOkwFShn0i2tVeobCTCBYbWHfCQA+bdkOepTjhmwQQsHOSReEsfGLcsrI5c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615916044; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ffPiYFyiL3DFwW62Al7ig9kVtxmjQvpjL6ezFYjhhUs=; b=eI6Uvjqv/bRLUXs0VB0thpZVHbn2iAtzknA+R2lScfEqs//JjEdxikV+raZ3D0CaTWs96YlAoQ9OBoSxJMFk8Lu71s9Y7sPF+sMJvkE8bPO8AfHj7sL2cmJlfvnSK2gM9+hfWhfBuz0f8n7Owhrzy//PC08wVFSDVieYvWz2XCI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mx.zohomail.com with SMTPS id 1615916044144572.4950744853178; Tue, 16 Mar 2021 10:34:04 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-307--WFp6GeuMvOc5zrcqanoiw-1; Tue, 16 Mar 2021 13:33:29 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9CD771934101; Tue, 16 Mar 2021 17:33:22 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 755B3610AF; Tue, 16 Mar 2021 17:33:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 36D7157DC4; Tue, 16 Mar 2021 17:33:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12GHX95r011400 for ; Tue, 16 Mar 2021 13:33:09 -0400 Received: by smtp.corp.redhat.com (Postfix) id 34B1D5C1A3; Tue, 16 Mar 2021 17:33:09 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.7]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8201D5C1A1; Tue, 16 Mar 2021 17:33:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615916043; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ffPiYFyiL3DFwW62Al7ig9kVtxmjQvpjL6ezFYjhhUs=; b=ExBAasjfFB1cLB02SbDnI+2jhQO3yuiI5GSdb0Y3WrLuh0fuZWmyK9hvz804rsIgF8puq9 k68x74lq1wshxt0TmfiB9BJZLhxxkyivJHWvUMV6IQk2XELPdEhAcoWH2LNXbhKm5stDYQ cCLZGTi0HfaKw59XJTYiaGyimwEdlws= X-MC-Unique: -WFp6GeuMvOc5zrcqanoiw-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v2 4/6] ci: Introduce a util module Date: Tue, 16 Mar 2021 18:32:58 +0100 Message-Id: <95f5167da1c326f97c8409372ac31b30d1f53069.1615915976.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" With the gradual rewrite of the Makefile to the 'helper' script will require helper functions that would better live in a separate util module. Signed-off-by: Erik Skultety --- ci/util.py | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 ci/util.py diff --git a/ci/util.py b/ci/util.py new file mode 100644 index 0000000000..8a2d6d8f47 --- /dev/null +++ b/ci/util.py @@ -0,0 +1,39 @@ +import json +import urllib.request as urllibrequest +import urllib.parse as urllibparse + +from typing import Dict, List + + +def get_registry_uri(namespace: str, + gitlab_uri=3D"https://gitlab.com", + api_version=3D4) -> str: + """ + :param namespace: GitLab project namespace, e.g. "libvirt/libvirt" + :param gitlab_uri: GitLab base URI, can be a private deployment + :param api_version: GitLab REST API version number + :return: URI pointing to a namespaced project's image registry + """ + + # this converts something like "libvirt/libvirt" to "libvirt%2Flibvirt" + namespace_urlenc =3D urllibparse.quote_plus(namespace) + + apistr =3D str(api_version) + project_uri =3D f"{gitlab_uri}/api/v{apistr}/projects/{namespace_urlen= c}" + + uri =3D project_uri + "/registry/repositories" + return uri + + +def get_registry_images(uri: str) -> Dict[str, str]: + """ + Returns all container images as currently available for the given GitL= ab + project. + + :return: list of container image names + """ + + r =3D urllibrequest.urlopen(uri + "?per_page=3D100") + + # read the HTTP response and load the JSON part of it + return json.loads(r.read().decode()) --=20 2.29.2 From nobody Mon May 6 17:54:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615916013; cv=none; d=zohomail.com; s=zohoarc; b=IP+HDWUX68NhA33l7I2tru0UvPCtTDyLlBClK1sgF1JUv8uGiRiIlPQpfO0atjkcZxM8vrurVPiMdl/yPzL1xsfE7gYSaQED9/eu4t0ywkhYJlEcz5y/QpqPFHmuz4b5I3xbImZxYwrNa5Ow990Ydg1HBiw8ThDuGO8UqmPPHi0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615916013; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=E3LuFgeHOsED1BGxXrvs7+WRjJ/rKZSL6xu+8vaao9U=; b=Zv6wTc8vVJ7dO/pd1AA5RxVb1uXwYg+QcB8WnTAVdIt7TbZvxXs7Gsn01q6fxXeK231WU0uyP92Lkwsti2paMUvp3wK1q+qmHT54nT2J+ftdRRbEQ/4b/IOwYpZDZAu3vFkltlVxRnNazZvVakVs1GlI4WZW4jg3moS9OaPWnjs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1615916013758281.91493028531124; Tue, 16 Mar 2021 10:33:33 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-171-YPS-RmkgNvGGtnLwhacWTA-1; Tue, 16 Mar 2021 13:33:29 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 844B469726; Tue, 16 Mar 2021 17:33:22 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5F5135D719; Tue, 16 Mar 2021 17:33:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 16AC71800B89; Tue, 16 Mar 2021 17:33:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12GHXAMP011410 for ; Tue, 16 Mar 2021 13:33:10 -0400 Received: by smtp.corp.redhat.com (Postfix) id 3615E5C1C2; Tue, 16 Mar 2021 17:33:10 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.7]) by smtp.corp.redhat.com (Postfix) with ESMTP id 847A75C1A1; Tue, 16 Mar 2021 17:33:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615916012; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=E3LuFgeHOsED1BGxXrvs7+WRjJ/rKZSL6xu+8vaao9U=; b=ZviP1EZ1vwJjX+wUhIuzIRsTbqNJJvjkev2IYHAFHck59qzZDrMRqpWd4G4JdYS93ETK8U 3OAl4kvc3lyzgA/FbUW5DvbX5Bpsw48luflYCb7siFfyEu0bq19QsmrcAvJcR76Q2XFRH7 1DiK/M/3VkkGfBd1YNd3C7f7vNSsbCE= X-MC-Unique: YPS-RmkgNvGGtnLwhacWTA-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v2 5/6] ci: helper: Rewrite image listing to Python Date: Tue, 16 Mar 2021 18:32:59 +0100 Message-Id: <94021666b6706d2c91a008082fb444d93c10907c.1615915976.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" The corresponding Bash script is dropped. After this patch's rewrite, the Makefile's original image listing target remains intact only to notify the user to use the Python helper instead. Signed-off-by: Erik Skultety --- ci/Makefile | 12 +----------- ci/helper | 37 ++++++++++++++++++++++++++++++++++++- ci/list-images.sh | 14 -------------- 3 files changed, 37 insertions(+), 26 deletions(-) delete mode 100644 ci/list-images.sh diff --git a/ci/Makefile b/ci/Makefile index f83ecac1e5..7720dddfb9 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -211,16 +211,7 @@ ci-build@%: ci-test@%: $(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_NINJA_ARGS=3Dtest =20 -ci-list-images: - @echo - @echo "Available x86 container images:" - @echo - @sh list-images.sh "$(CI_IMAGE_PREFIX)" | grep -v cross - @echo - @echo "Available cross-compiler container images:" - @echo - @sh list-images.sh "$(CI_IMAGE_PREFIX)" | grep cross - @echo +ci-list-images: help =20 help: @echo @@ -241,7 +232,6 @@ help: @echo " ci-build@\$$IMAGE - run a default 'ninja' build" @echo " ci-test@\$$IMAGE - run a 'ninja test'" @echo " ci-shell@\$$IMAGE - run an interactive shell" - @echo " ci-list-images - list available images" @echo " help - show this help message" @echo @echo "Available make variables:" diff --git a/ci/helper b/ci/helper index abac686d22..1b7675c58a 100755 --- a/ci/helper +++ b/ci/helper @@ -11,6 +11,8 @@ import shutil import subprocess import sys =20 +import util + =20 class Parser: def __init__(self): @@ -66,6 +68,20 @@ class Parser: help=3D"path to lcitool binary", ) =20 + # Options that are common to actions communicating with a GitLab + # instance + gitlabparser =3D argparse.ArgumentParser(add_help=3DFalse) + gitlabparser.add_argument( + "--namespace", + default=3D"libvirt/libvirt", + help=3D"GitLab project namespace" + ) + gitlabparser.add_argument( + "--gitlab-uri", + default=3D"https://gitlab.com", + help=3D"base GitLab URI" + ) + # Main parser self.parser =3D argparse.ArgumentParser() self.parser.add_argument( @@ -111,6 +127,7 @@ class Parser: listimagesparser =3D subparsers.add_parser( "list-images", help=3D"list known container images", + parents=3D[gitlabparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) listimagesparser.set_defaults(func=3DApplication.action_list_image= s) @@ -249,7 +266,25 @@ class Application: self.make_run(f"ci-shell@{self.args.target}") =20 def action_list_images(self): - self.make_run(f"ci-list-images") + registry_uri =3D util.get_registry_uri(self.args.namespace, + self.args.gitlab_uri) + images =3D util.get_registry_images(registry_uri) + + # skip the "ci-" prefix each of our container images' name has + name_prefix =3D "ci-" + names =3D [i["name"][len(name_prefix):] for i in images] + names.sort() + + native =3D [name for name in names if "-cross-" not in name] + cross =3D [name for name in names if "-cross-" in name] + + print("Available x86 container images:\n") + print("\t" + "\n\t".join(native)) + + if cross: + print() + print("Available cross-compiler container images:\n") + print("\t" + "\n\t".join(cross)) =20 def action_refresh(self): self.refresh_containers() diff --git a/ci/list-images.sh b/ci/list-images.sh deleted file mode 100644 index b85b132253..0000000000 --- a/ci/list-images.sh +++ /dev/null @@ -1,14 +0,0 @@ -#!/bin/sh - -prefix=3D"${1##registry.gitlab.com/}" - -PROJECT_ID=3D192693 - -all_repos() { - curl -s "https://gitlab.com/api/v4/projects/$PROJECT_ID/registry/reposit= ories?per_page=3D100" \ - | tr , '\n' | grep '"path":' | sed 's,"path":",,g;s,"$,,g' -} - -all_repos | grep "^$prefix" | sed "s,^$prefix,,g" | while read repo; do - echo " $repo" -done | sort -u --=20 2.29.2 From nobody Mon May 6 17:54:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615916017; cv=none; d=zohomail.com; s=zohoarc; b=X18XDZ2FhE/5RBtfJNU0/6E6WQmcN996ExEximABBM7Hdg3SVj1Cvcao4F2E3cxeCzmbUrZM2w82XDbAeta8fpJLGzgmCVn3P1MTpBeRpS+HhXXX7X3UVxpsrI3K2UTlN9B0NWvSlB7YJRGPBTvD2x6u6iXQmlTpId/YUtfWPEw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615916017; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=koudpeVRjToxDjCxgXtLoz0gbbDSYabItJnQHWyYB00=; b=IXljtaUHmdW7fVS+tATbj1EAT+iNRmGex6RuuHuNUHOAxGot6QY6/SJEpNFbSbvyv9HL1jVKpcRzPgomHDJotqgZxF2Qnxc3LSFgdomzZ5rvvtTNgS+Q9fE06/rZMdtLBRbg7T7AE/OqDYNS+fvVOwjBexptatfNc5oXWRlVWtA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1615916017313185.4701306470513; Tue, 16 Mar 2021 10:33:37 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-291-6YqdVJmrP0ekbX6-29lD8g-1; Tue, 16 Mar 2021 13:33:33 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 942721934100; Tue, 16 Mar 2021 17:33:25 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 690931754A; Tue, 16 Mar 2021 17:33:25 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 292C61800BAE; Tue, 16 Mar 2021 17:33:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 12GHXBed011425 for ; Tue, 16 Mar 2021 13:33:11 -0400 Received: by smtp.corp.redhat.com (Postfix) id 3E0D35C1C2; Tue, 16 Mar 2021 17:33:11 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.192.7]) by smtp.corp.redhat.com (Postfix) with ESMTP id 878135C1A1; Tue, 16 Mar 2021 17:33:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615916016; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=koudpeVRjToxDjCxgXtLoz0gbbDSYabItJnQHWyYB00=; b=H+LHaeS7U04AAH4ILJkg0oXKskOjGEEguDTe6Kh37eD8mkQv6bDAsMa6jpOJ0mzBVLLnHl aDjS36qTZAR1P1+v2ZmwlLc14huXehzKmuLECEKn/e3xDcsQg4wSJuPEwWcXvmmKPp4RzD SlLAnBFThZR4dRdDFbDBLHWXFdGmie8= X-MC-Unique: 6YqdVJmrP0ekbX6-29lD8g-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v2 6/6] ci: util: Add a registry checker for stale images Date: Tue, 16 Mar 2021 18:33:00 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: eskultet@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This function checks whether there are any stale Docker images in the registry that can be purged. Since we're pulling available container images from our GitLab registry with the 'list-images' action, it could happen that we'd list old (already unsupported) images and make them available for the user to consume and run a build in them. Naturally, the build will most likely fail leaving the user confused. Signed-off-by: Erik Skultety --- ci/helper | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++++- ci/util.py | 13 +++++++++++++ 2 files changed, 66 insertions(+), 1 deletion(-) diff --git a/ci/helper b/ci/helper index 1b7675c58a..dbe3f80555 100755 --- a/ci/helper +++ b/ci/helper @@ -136,9 +136,16 @@ class Parser: refreshparser =3D subparsers.add_parser( "refresh", help=3D"refresh data generated with lcitool", - parents=3D[lcitoolparser], + parents=3D[lcitoolparser, gitlabparser], formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, ) + refreshparser.add_argument( + "--check-stale", + action=3D"store", + choices=3D["yes", "no"], + default=3D"yes", + help=3D"check for existence of stale images on the GitLab inst= ance" + ) refreshparser.set_defaults(func=3DApplication.action_refresh) =20 def parse(self): @@ -286,10 +293,55 @@ class Application: print("Available cross-compiler container images:\n") print("\t" + "\n\t".join(cross)) =20 + def _list_stale_images(self): + # get list of hosts supported by lcitool + lcitool_hosts =3D self.lcitool_get_hosts() + + # get images from gitlab registry + registry_uri =3D util.get_registry_uri(self.args.namespace, + self.args.gitlab_uri) + images =3D util.get_registry_images(registry_uri) + + # extract distro names from the list of registry images + registry_distros =3D [util.get_image_distro(i["name"]) for i in im= ages] + + # check for stale images in GitLab registry + stale =3D set(registry_distros) - set(lcitool_hosts) + if stale: + stale_images =3D {} + for item in stale: + for img in images: + if item in img["name"]: + stale_images[img["name"]] =3D img["id"] + + return stale_images + def action_refresh(self): + # refresh Dockerfiles and vars files self.refresh_containers() self.refresh_cirrus() =20 + # check for stale images + if self.args.check_stale =3D=3D "yes" and not self.args.quiet: + namespace =3D self.args.namespace + gitlab_uri =3D self.args.gitlab_uri + + stale_images =3D self._list_stale_images() + if stale_images: + spacing =3D "\n" + 4 * " " + stale_fmt =3D [f"{k}: {v}" for k, v in stale_images.items(= )] + + print(f""" +The following images are stale and can be purged from the registry: +{spacing + spacing.join(stale_fmt)} + +You can remove the above images over the API with the following code snipp= et: + + $ for image_id in {' '.join([str(id) for id in stale_images.values()])= }; do + curl --request DELETE --header "PRIVATE-TOKEN: " \\ + {util.get_registry_uri(namespace, gitlab_uri)}/$image_id + done""") + def run(self): self.args.func(self) =20 diff --git a/ci/util.py b/ci/util.py index 8a2d6d8f47..f50c0e6100 100644 --- a/ci/util.py +++ b/ci/util.py @@ -37,3 +37,16 @@ def get_registry_images(uri: str) -> Dict[str, str]: =20 # read the HTTP response and load the JSON part of it return json.loads(r.read().decode()) + + +def get_image_distro(image_name: str): + name_prefix =3D "ci-" + name_suffix =3D "-cross-" + + distro =3D image_name[len(name_prefix):] + + index =3D distro.find(name_suffix) + if index > 0: + distro =3D distro[:index] + + return distro --=20 2.29.2