From nobody Sat Jul 25 17:34:02 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1784197573; cv=none; d=zohomail.com; s=zohoarc; b=VIP/8PzHBQ7B1fAdMAg0Tm7GpXqmgzLvfW4N+BG+az/8Ab3YvJpSxOEqpUfurGcyjPrEyRQtB5EVvq/VmeqF8TIwaIgAjFHrvj6SXFjYdNjCNPSdW8WxjipAyDutgaXqQVzADVL/szH42OzpXhoa5tKOrzxXWYFlZXtkR9SZlx0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1784197573; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=zMadrffyHPto0rUIdW/xxD0skGqij8qb5Ota3D+XKxk=; b=Pg8vFXPtr8l542F1+fqhy/f/3x6JYUsgj0vcp5cHlKmQRVEuTN9nul7/g+eQxOFRPLYtrRAV4NL5pL9VXBCTEKKpck/BTEIKy5saHU8Dx8YTywaeWWt5K00NUSht2bAI7HLS3Ed/H/3/fKmx8X3ONQ8ZWt1IoNPyh9NBrHOlmQE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1784197573676414.1228822837501; Thu, 16 Jul 2026 03:26:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wkJHd-0006P5-1Y; Thu, 16 Jul 2026 06:25:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wkJHY-0006OM-5r for qemu-devel@nongnu.org; Thu, 16 Jul 2026 06:25:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wkJHU-0005u1-5h for qemu-devel@nongnu.org; Thu, 16 Jul 2026 06:25:30 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-150-qRm4FQ-jMMKg_SuWCXRgCg-1; Thu, 16 Jul 2026 06:25:20 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1239D195604D; Thu, 16 Jul 2026 10:25:19 +0000 (UTC) Received: from berrange.com (unknown [10.44.34.221]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2FE0C1955F78; Thu, 16 Jul 2026 10:25:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1784197525; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zMadrffyHPto0rUIdW/xxD0skGqij8qb5Ota3D+XKxk=; b=EEk+pcTZGZ2UIxf3TlpDt1DvY8vmr/i/oSJgLXtxogQ7w0wJl6Kji7fy2GKovNtNylcllN NvyS+TmtXxwhKvIAsTMwiZKrTrVypAWdzLx1l7FGHSOj7dUhFBSpwBw9X2/NF95j0h3EG+ fG6qBmR0BssINgMt3lA4S1cmKbU6f+g= X-MC-Unique: qRm4FQ-jMMKg_SuWCXRgCg-1 X-Mimecast-MFC-AGG-ID: qRm4FQ-jMMKg_SuWCXRgCg_1784197519 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Pierrick Bouvier , Paolo Bonzini , =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v3] tests/docker: ensure container command is probed at most once Date: Thu, 16 Jul 2026 11:25:15 +0100 Message-ID: <20260716102515.1503404-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, T_SPF_PERMERROR=0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1784197574897158500 The '--engine' arg accepts either 'podman' or 'docker', which is not sufficiently granular to map directly to a command. This means that docker.py still has to then probe the exact command to use. Meanwhile the 'probe' command prints out the full command to use but this cannot be passed back to docker.py to avoid probing again, so the caching is only useful in the few case where we run a container directly bypassing docker.py. Address this by replacing --engine with --command for docker.py. This in turn requires the --container-engine configure arg to be replaced with --container-command. With these changes the container command is probed at most once during configure and never again, while running in an unconfigured tree will still probe on demand. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Pierrick Bouvier Tested-by: Pierrick Bouvier --- In v3: - Avoid bashism when testing probe result configure | 25 +++++++------ tests/docker/Makefile.include | 13 ++++--- tests/docker/docker.py | 67 ++++++++++++----------------------- 3 files changed, 40 insertions(+), 65 deletions(-) diff --git a/configure b/configure index d8bc10060e..6ddfe4b534 100755 --- a/configure +++ b/configure @@ -172,7 +172,7 @@ fi # some defaults, based on the host environment =20 # default parameters -container_engine=3D"auto" +container_command=3D"" cpu=3D"" cross_compile=3D"no" cross_prefix=3D"" @@ -734,7 +734,7 @@ for opt do ;; --disable-containers) use_containers=3D"no" ;; - --container-engine=3D*) container_engine=3D"$optarg" + --container-command=3D*) container_command=3D"$optarg" ;; --rust-target-triple=3D*) rust_target_triple=3D"$optarg" ;; @@ -869,7 +869,7 @@ Advanced options (experts only): --enable-debug enable common debug build options --cpu=3DCPU Build for host CPU [$cpu] --disable-containers don't use containers for cross-building - --container-engine=3DTYPE which container engine to use [$container_eng= ine] + --container-command=3DCMD which container command to use [autodetect] --gdb=3DGDB-path gdb to use for gdbstub tests [$gdb_bin] --wasm64-32bit-address-limit Restrict wasm64 address space to 32-bit (de= fault is to use the whole 64-bit range). @@ -1291,12 +1291,12 @@ fi ########################################## # functions to probe cross compilers =20 -runc=3D"no" -if test $use_containers =3D "yes" && (has "docker" || has "podman"); then - runc=3D$($python "$source_path"/tests/docker/docker.py --engine "$cont= ainer_engine" probe) - if test "$runc" !=3D "no"; then - docker_py=3D"$python $source_path/tests/docker/docker.py --engine = $container_engine" - fi +if test "$container_command" =3D ""; then + container_command=3D$($python "$source_path"/tests/docker/docker.py pr= obe) + test "$container_command" =3D "no" && container_command=3D"" +fi +if test $use_containers =3D "yes" && test "$container_command" !=3D ""; th= en + docker_py=3D"$python $source_path/tests/docker/docker.py --command $co= ntainer_command" fi =20 # cross compilers defaults, can be overridden with --cross-cc-ARCH @@ -1415,7 +1415,7 @@ probe_target_compiler() { esac =20 for host in $container_hosts; do - test "$runc" !=3D no || continue + test "$container_command" !=3D "" || continue test "$host" =3D "$cpu" || continue case $target_arch in # debian-all-test-cross architectures @@ -1736,9 +1736,8 @@ echo all: >> $config_host_mak echo "SRC_PATH=3D$source_path" >> $config_host_mak echo "TARGET_DIRS=3D$target_list" >> $config_host_mak echo "GDB=3D$gdb_bin" >> $config_host_mak -if test "$runc" !=3D no; then - echo "RUNC=3D$runc" >> $config_host_mak - echo "CONTAINER_ENGINE=3D$container_engine" >> $config_host_mak +if test "$container_command" !=3D ""; then + echo "CONTAINER_COMMAND=3D$container_command" >> $config_host_mak fi echo "SUBDIRS=3D$subdirs" >> $config_host_mak if test "$rust" !=3D disabled; then diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 4725c39807..0adddb6a5c 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -16,9 +16,8 @@ DOCKER_DEFAULT_REGISTRY :=3D registry.gitlab.com/qemu-pro= ject/qemu endif DOCKER_REGISTRY :=3D $(if $(REGISTRY),$(REGISTRY),$(DOCKER_DEFAULT_REGISTR= Y)) =20 -CONTAINER_ENGINE =3D auto -DOCKER_SCRIPT=3D$(SRC_PATH)/tests/docker/docker.py --engine $(CONTAINER_EN= GINE) -RUNC ?=3D $(shell $(DOCKER_SCRIPT) probe) +CONTAINER_COMMAND ?=3D $(shell $(SRC_PATH)/tests/docker/docker.py probe) +DOCKER_SCRIPT=3D$(SRC_PATH)/tests/docker/docker.py --command "$(CONTAINER_= COMMAND)" =20 CUR_TIME :=3D $(shell date +%Y-%m-%d-%H.%M.%S.$$$$) DOCKER_SRC_COPY :=3D $(BUILD_DIR)/docker-src.$(CUR_TIME) @@ -41,7 +40,7 @@ docker-qemu-src: $(DOCKER_SRC_COPY) # General rule for building docker images. docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(call quiet-command, \ - DOCKER_BUILDKIT=3D1 $(RUNC) build \ + DOCKER_BUILDKIT=3D1 $(CONTAINER_COMMAND) build \ $(if $(DOCKER_V),,--quiet) \ $(if $(NOCACHE),--no-cache, \ $(if $(DOCKER_REGISTRY),--cache-from $(DOCKER_REGISTRY)/qemu/$*)) \ @@ -152,7 +151,7 @@ $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOC= KER_IMAGES)), \ ) =20 docker: - @echo 'Build QEMU and run tests inside $(RUNC) containers' + @echo 'Build QEMU and run tests inside $(CONTAINER_COMMAND) containers' @echo @echo 'Available targets:' @echo @@ -219,10 +218,10 @@ docker-run: docker-qemu-src $(IMAGE) --executable $(EXECUTABLE), \ " COPYING $(EXECUTABLE) to $(IMAGE)")) $(call quiet-command, \ - $(RUNC) run \ + $(CONTAINER_COMMAND) run \ --rm \ $(if $(NOUSER),, \ - $(if $(filter docker,$(RUNC)), \ + $(if $(filter docker,$(CONTAINER_COMMAND)), \ -u $(UID), \ --userns keep-id \ ) \ diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 9e18b984f4..d2f39b5645 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -35,27 +35,6 @@ =20 DEVNULL =3D open(os.devnull, 'wb') =20 -class EngineEnum(enum.IntEnum): - AUTO =3D 1 - DOCKER =3D 2 - PODMAN =3D 3 - - def __str__(self): - return self.name.lower() - - def __repr__(self): - return str(self) - - @staticmethod - def argparse(s): - try: - return EngineEnum[s.upper()] - except KeyError: - return s - - -USE_ENGINE =3D EngineEnum.AUTO - def _bytes_checksum(bytes): """Calculate a digest string unique to the text content""" return hashlib.sha1(bytes).hexdigest() @@ -73,12 +52,11 @@ def _file_checksum(filename): =20 def _guess_engine_command(): """ Guess a working engine command or raise exception if not found""" - commands =3D [] - - if USE_ENGINE in [EngineEnum.AUTO, EngineEnum.PODMAN]: - commands +=3D [["podman"], ["podman-remote"], ["podman", "--remote= "]] - if USE_ENGINE in [EngineEnum.AUTO, EngineEnum.DOCKER]: - commands +=3D [["docker"], ["sudo", "-n", "docker"]] + commands =3D [["podman"], + ["podman-remote"], + ["podman", "--remote"], + ["docker"], + ["sudo", "-n", "docker"]] for cmd in commands: try: # 'version' is not sufficient to prove a working binary @@ -222,8 +200,11 @@ def _dockerfile_verify_flat(df): =20 class Docker(object): """ Running Docker commands """ - def __init__(self): - self._command =3D _guess_engine_command() + def __init__(self, commandstr=3DNone): + if commandstr is None: + self._command =3D _guess_engine_command() + else: + self._command =3D commandstr.split(" ") =20 if ("docker" in self._command and "TRAVIS" not in os.environ and @@ -411,8 +392,8 @@ def args(self, parser): help=3D"Run container using the current user's= uid") =20 def run(self, args, argv): - return Docker().run(argv, args.keep, quiet=3Dargs.quiet, - as_user=3Dargs.run_as_current_user) + return Docker(args.command).run(argv, args.keep, quiet=3Dargs.quie= t, + as_user=3Dargs.run_as_current_user) =20 =20 class BuildCommand(SubCommand): @@ -445,7 +426,7 @@ def run(self, args, argv): dockerfile =3D _read_dockerfile(args.dockerfile) tag =3D args.tag =20 - dkr =3D Docker() + dkr =3D Docker(args.command) if "--no-cache" not in argv and \ dkr.image_matches_dockerfile(tag, dockerfile): if not args.quiet: @@ -512,7 +493,7 @@ def args(self, parser): help=3D"Docker registry") =20 def run(self, args, argv): - dkr =3D Docker() + dkr =3D Docker(args.command) dkr.command(cmd=3D"pull", quiet=3Dargs.quiet, argv=3D["%s/%s" % (args.registry, args.tag)]) dkr.command(cmd=3D"tag", quiet=3Dargs.quiet, @@ -590,7 +571,7 @@ def run(self, args, argv): tmp.seek(0) =20 # Run the build with our tarball context - dkr =3D Docker() + dkr =3D Docker(args.command) dkr.update_image(args.tag, tmp, quiet=3Dargs.quiet) =20 return 0 @@ -601,7 +582,7 @@ class CleanCommand(SubCommand): name =3D "clean" =20 def run(self, args, argv): - Docker().clean() + Docker(args.command).clean() return 0 =20 =20 @@ -610,7 +591,7 @@ class ImagesCommand(SubCommand): name =3D "images" =20 def run(self, args, argv): - return Docker().command("images", argv, args.quiet) + return Docker(args.command).command("images", argv, args.quiet) =20 =20 class ProbeCommand(SubCommand): @@ -619,7 +600,7 @@ class ProbeCommand(SubCommand): =20 def run(self, args, argv): try: - docker =3D Docker() + docker =3D Docker(args.command) print(" ".join(docker._command)) except Exception: print("no") @@ -651,18 +632,16 @@ def run(self, args, argv): cmd +=3D ["-v", "%s:%s:ro,z" % (p, p)] cmd +=3D [args.image, args.cc] cmd +=3D argv - return Docker().run(cmd, False, quiet=3Dargs.quiet, - as_user=3DTrue) + return Docker(args.command).run(cmd, False, quiet=3Dargs.quiet, + as_user=3DTrue) =20 =20 def main(): - global USE_ENGINE - parser =3D argparse.ArgumentParser(description=3D"A Docker helper", usage=3D"%s ..." % os.path.basename(sys.argv[0])) - parser.add_argument("--engine", type=3DEngineEnum.argparse, choices=3D= list(EngineEnum), - help=3D"specify which container engine to use") + parser.add_argument("--command", + help=3D"specify which container engine command to = use") subparsers =3D parser.add_subparsers(title=3D"subcommands", help=3DNon= e) for cls in SubCommand.__subclasses__(): cmd =3D cls() @@ -671,8 +650,6 @@ def main(): cmd.args(subp) subp.set_defaults(cmdobj=3Dcmd) args, argv =3D parser.parse_known_args() - if args.engine: - USE_ENGINE =3D args.engine return args.cmdobj.run(args, argv) =20 =20 --=20 2.55.0