[libvirt] [PATCH] ci: Fetch list of available container images dynamically

Andrea Bolognani posted 1 patch 4 years, 3 months ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20191211175619.189870-1-abologna@redhat.com
ci/Makefile       | 37 +++++++++++++------------------------
ci/list-images.sh | 26 ++++++++++++++++++++++++++
2 files changed, 39 insertions(+), 24 deletions(-)
create mode 100644 ci/list-images.sh
[libvirt] [PATCH] ci: Fetch list of available container images dynamically
Posted by Andrea Bolognani 4 years, 3 months ago
Any static list of images is destined to become outdated eventually,
so let's start generating it dynamically instead.

Unfortunately there doesn't seem to be a straightforward way to get
Podman/Docker to list all repositories under quay.io/libvirt, so we
have to resort to searching and filtering manually; and since the
two tools behave slightly differently in that regard, it's more
sane to have the logic in a separate shell script than it would be
to keep it inline in the Makefile with all the annoying escaping
that would entail.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
 ci/Makefile       | 37 +++++++++++++------------------------
 ci/list-images.sh | 26 ++++++++++++++++++++++++++
 2 files changed, 39 insertions(+), 24 deletions(-)
 create mode 100644 ci/list-images.sh

diff --git a/ci/Makefile b/ci/Makefile
index 27c1049b38..acb655941c 100644
--- a/ci/Makefile
+++ b/ci/Makefile
@@ -238,6 +238,17 @@ ci-build@%:
 ci-check@%:
 	$(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_MAKE_ARGS="check"
 
+ci-list-images:
+	@echo
+	@echo "Available x86 container images:"
+	@echo
+	@sh list-images.sh "$(CI_ENGINE)" "$(CI_IMAGE_PREFIX)" | grep -v cross
+	@echo
+	@echo "Available cross-compiler container images:"
+	@echo
+	@sh list-images.sh "$(CI_ENGINE)" "$(CI_IMAGE_PREFIX)" | grep cross
+	@echo
+
 ci-help:
 	@echo "Build libvirt inside containers used for CI"
 	@echo
@@ -246,30 +257,8 @@ ci-help:
 	@echo "    ci-build@\$$IMAGE - run a default 'make'"
 	@echo "    ci-check@\$$IMAGE - run a 'make check'"
 	@echo "    ci-shell@\$$IMAGE - run an interactive shell"
-	@echo
-	@echo "Available x86 container images:"
-	@echo
-	@echo "    centos-7"
-	@echo "    debian-9"
-	@echo "    debian-10"
-	@echo "    debian-sid"
-	@echo "    fedora-29"
-	@echo "    fedora-30"
-	@echo "    fedora-rawhide"
-	@echo "    ubuntu-16"
-	@echo "    ubuntu-18"
-	@echo
-	@echo "Available cross-compiler container images:"
-	@echo
-	@echo "    debian-{9,10,sid}-cross-aarch64"
-	@echo "    debian-{9,10,sid}-cross-armv6l"
-	@echo "    debian-{9,10,sid}-cross-armv7l"
-	@echo "    debian-{10,sid}-cross-i686"
-	@echo "    debian-{9,10,sid}-cross-mips64el"
-	@echo "    debian-{9,10,sid}-cross-mips"
-	@echo "    debian-{9,10,sid}-cross-mipsel"
-	@echo "    debian-{9,10,sid}-cross-ppc64le"
-	@echo "    debian-{9,10,sid}-cross-s390x"
+	@echo "    ci-list-images  - list available images"
+	@echo "    ci-help         - show this help message"
 	@echo
 	@echo "Available make variables:"
 	@echo
diff --git a/ci/list-images.sh b/ci/list-images.sh
new file mode 100644
index 0000000000..35efdb6982
--- /dev/null
+++ b/ci/list-images.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+engine="$1"
+prefix="$2"
+
+do_podman() {
+    # Podman freaks out if the search term ends with a dash, which ours
+    # by default does, so let's strip it. The repository name is the
+    # second field in the output, and it already starts with the registry
+    podman search --limit 100 "${prefix%-}" | while read _ repo _; do
+        echo "$repo"
+    done
+}
+
+do_docker() {
+    # Docker doesn't include the registry name in the output, so we have
+    # to add it. The repository name is the first field in the output
+    registry="${prefix%%/*}"
+    docker search --limit 100 "$prefix" | while read repo _; do
+        echo "$registry/$repo"
+    done
+}
+
+"do_$engine" | grep "^$prefix" | sed "s,^$prefix,,g" | while read repo; do
+    echo "    $repo"
+done | sort -u
-- 
2.23.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH] ci: Fetch list of available container images dynamically
Posted by Cole Robinson 4 years, 3 months ago
On 12/11/19 12:56 PM, Andrea Bolognani wrote:
> Any static list of images is destined to become outdated eventually,
> so let's start generating it dynamically instead.
> 
> Unfortunately there doesn't seem to be a straightforward way to get
> Podman/Docker to list all repositories under quay.io/libvirt, so we
> have to resort to searching and filtering manually; and since the
> two tools behave slightly differently in that regard, it's more
> sane to have the logic in a separate shell script than it would be
> to keep it inline in the Makefile with all the annoying escaping
> that would entail.
> 
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>

Reviewed-by: Cole Robinson <crobinso@redhat.com>

But the other ci/ scripts are listed in Makefile.am EXTRA_DIST, so
adjust that before pushing unless it was deliberate

- Cole

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH] ci: Fetch list of available container images dynamically
Posted by Andrea Bolognani 4 years, 2 months ago
On Tue, 2019-12-17 at 11:21 -0500, Cole Robinson wrote:
> But the other ci/ scripts are listed in Makefile.am EXTRA_DIST, so
> adjust that before pushing unless it was deliberate

Nah, it was just an oversight. Good catch!

I've fixed it before pushing.

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list