From nobody Tue Feb 10 15:45:14 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1579898892; cv=none; d=zohomail.com; s=zohoarc; b=R23I/OMKaZcC/Ll7fMo5fk0vEnWt0fOHK+5Nr4/lwfZqfKBg/Yfb26MrmmCVCrIWmA6/B2jXtGL3CZj7/DwOrvw983o4YsSlTlpj7QwprTeBFvHOVA1FuJ+RddPq9oUp+yXR0TOxsgo+9LULTGszbehiSkwWKbjei1w11+eHfn0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579898892; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CO6Bh5iCXjaTaQynC+oib+Hg/v44ekrp+1dnmcnUnXM=; b=IA8rEMEpfIOPh3Bp98qwax7lCjQdAfU0IIa76Ad6HINE0iafrsjfBsyW8Vxgr0CIHerz23rph1/bYDml6d1l1RHQtv1uL3xJZmo2wVjwwHeE0K/lb3qZL4KIKBMCv65Ly3cP8gzXdiz9yQdTuiLJir+NCsaXQmILHaqm0W3dlUg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1579898892262394.31912716302975; Fri, 24 Jan 2020 12:48:12 -0800 (PST) Received: from localhost ([::1]:47828 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iv5sB-0003Zr-7s for importer@patchew.org; Fri, 24 Jan 2020 15:48:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42779) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iv5kx-0008BA-L4 for qemu-devel@nongnu.org; Fri, 24 Jan 2020 15:40:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iv5ku-00065y-WA for qemu-devel@nongnu.org; Fri, 24 Jan 2020 15:40:43 -0500 Received: from mail-wr1-x42e.google.com ([2a00:1450:4864:20::42e]:35990) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iv5ku-00063w-Oi for qemu-devel@nongnu.org; Fri, 24 Jan 2020 15:40:40 -0500 Received: by mail-wr1-x42e.google.com with SMTP id z3so3602380wru.3 for ; Fri, 24 Jan 2020 12:40:40 -0800 (PST) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id e12sm8804760wrn.56.2020.01.24.12.40.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2020 12:40:33 -0800 (PST) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 319821FF99; Fri, 24 Jan 2020 20:40:27 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CO6Bh5iCXjaTaQynC+oib+Hg/v44ekrp+1dnmcnUnXM=; b=YXdYUiwC+kskyrxKRiG0WuNYg7xAiqdjeV9ItNT2q8p54GRtB9XcpDKbXMA+BnknGN IjFvspL9XBEB6Xefggz2xBlDRxqP0rRLmv17vR7UItX5EGAhq7/3vhombcyr2I7zyISk qGQDpYyrCzG4YgQjDuTytt8YyKozdfZOqoR1ifXoBRCLJZXOgUNjKqi0ZS7QQgbtKGwH RKdEmwSB6URxoU77HwdICnJL6DoQHVdSru+lVIUeBArgendSxLk+CsOhWcSG/7pbWWDG o6Yu+0Axjp0h6ZCtfEghSG77Jr8VQl/LqHafKFimh/y0nKz6J/tUyhla8RbQ4zBMcn1r uiaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CO6Bh5iCXjaTaQynC+oib+Hg/v44ekrp+1dnmcnUnXM=; b=SZer4ZL7Lzlx0orOuICg1zgviTqgB6rARBg2x6KawwPGW+1Zr1EvpDv+LM+heBfgqf oXqGtz6ipApKjMYVC9FaFNTtvM93Xycd21UxuEidCuNY0Nq6C0XRjpHqyE6FCrxaGYia 6r2I/vySjF8FTNrjsC6abAjCsOpmgVfUv98FE6ZxTo612jfXlG0ygrDPjJCpkYAXIS6M CKlqJA7deqmfsLsiMsWOi96Y65ZsvAZI1je3rXwGdWKJ0n79SJxsN6QZo3CBaUj+LlAu VJMARGmIi4b5n+sBsjQjv1ZB1kCuFYiA5pjFeExJzRaMQyLaPMTr/F3OQ3Ct1QrZjBOu iqRA== X-Gm-Message-State: APjAAAUV7/1iAfk0YtjvTtLAhCIfcH3hyGGoY7ytn8cnK2HOXorIW3ig rUgc4ajTgsLwrPuXlxR6WhntSQ== X-Google-Smtp-Source: APXvYqw6b5Y7RfoCS77JpioH/+3gAcR2znNFrr9ZLZ/ec+lLJlk2K8sWieYE+Nggd8juDSUABpLsYg== X-Received: by 2002:adf:f6c1:: with SMTP id y1mr6098434wrp.17.1579898439673; Fri, 24 Jan 2020 12:40:39 -0800 (PST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v1 09/13] tests/docker: switch to multiarch aware container support Date: Fri, 24 Jan 2020 20:40:22 +0000 Message-Id: <20200124204026.2107-10-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200124204026.2107-1-alex.bennee@linaro.org> References: <20200124204026.2107-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::42e X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, stefanb@linux.vnet.ibm.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, stefanha@redhat.com, marcandre.lureau@redhat.com, pbonzini@redhat.com, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) We drop support for DOCKER_PARTIAL_IMAGE and related hacks in favour of explicit building of lists of images by type. Currently we can't do any of the QEMU builds but the docker-image target will allow us to test that every docker image we declare can actually be built. Later commits will re-enable the builds for each host architecture we support. Signed-off-by: Alex Benn=C3=A9e --- tests/docker/Makefile.include | 124 +++++++++++++--------------------- 1 file changed, 46 insertions(+), 78 deletions(-) diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index c9e8bc40a1..b2b41c3e9b 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -5,12 +5,37 @@ HOST_ARCH =3D $(if $(ARCH),$(ARCH),$(shell uname -m)) =20 DOCKER_SUFFIX :=3D .docker -DOCKER_FILES_DIR :=3D $(SRC_PATH)/tests/docker/dockerfiles -# we don't run tests on intermediate images (used as base by another image) -DOCKER_PARTIAL_IMAGES :=3D debian9 debian10 -DOCKER_PARTIAL_IMAGES +=3D debian9-mxe debian-bootstrap -DOCKER_IMAGES :=3D $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DI= R)/*.docker)))) -DOCKER_TARGETS :=3D $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) +DOCKER_BASE :=3D $(SRC_PATH)/tests/docker + +# +# There are 3 classes of containers +# +# BASE_CONTAINERS - intermeadiate containers which are used to build ot= hers +# TEST_BUILD_CONTAINERS - contain enough compiler support to build tests +# QEMU_BUILD_CONTAINERS - contain build dependencies enough to build QE= MU +# + +BASE_CONTAINERS :=3D +TEST_BUILD_CONTAINERS :=3D +QEMU_BUILD_CONTAINERS :=3D + +# +# Helpers for sub-arch includes +# +# $1 =3D list of dockerfile paths +get-basenames =3D $(sort $(notdir $(basename $1))) + +# +# Dealing with different host architectures. +# +# For x86_64 this is easy enough as most cross compilers are built for +# x86. However for other build hosts we may have a limited choice. To +# fully support all potential build hosts we can use QEMU's own +# linux-user support to have a native version of the toolchain. +# +-include $(DOCKER_BASE)/dockerfiles.multiarch/Makefile.include +-include $(DOCKER_BASE)/dockerfiles.$(HOST_ARCH)/Makefile.include + # Use a global constant ccache directory to speed up repetitive builds DOCKER_CCACHE_DIR :=3D $$HOME/.cache/qemu-docker-ccache =20 @@ -42,18 +67,21 @@ $(DOCKER_SRC_COPY): =20 docker-qemu-src: $(DOCKER_SRC_COPY) =20 -docker-image: ${DOCKER_TARGETS} +# The global docker-image rule builds all docker images we are capable of = building +DOCKER_TARGETS :=3D $(patsubst %,docker-image-%,$(BASE_CONTAINERS) $(TEST_= BUILD_CONTAINERS) $(QEMU_BUILD_CONTAINERS)) + +docker-image: $(DOCKER_TARGETS) =20 # General rule for building docker images. If we are a sub-make # invoked with SKIP_DOCKER_BUILD we still check the image is up to date # though ifdef SKIP_DOCKER_BUILD -docker-image-%: $(DOCKER_FILES_DIR)/%.docker +docker-image-%: %.docker $(call quiet-command, \ $(DOCKER_SCRIPT) check --quiet qemu:$* $<, \ "CHECK", "$*") else -docker-image-%: $(DOCKER_FILES_DIR)/%.docker +docker-image-%: %.docker $(call quiet-command,\ $(DOCKER_SCRIPT) build qemu:$* $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ @@ -62,7 +90,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(if $(EXECUTABLE),--include-executable=3D$(EXECUTABLE)),\ "BUILD","$*") =20 -docker-binfmt-image-debian-%: $(MULTARCH_BASE)/debian-bootstrap.docker +docker-binfmt-image-debian-%: $(DOCKER_BASE)/dockerfiles/debian-bootstrap.= docker $(if $(EXECUTABLE),,\ $(error EXECUTABLE not set, debootstrap of debian-$* would fail)) $(if $(DEB_ARCH),,\ @@ -86,69 +114,8 @@ docker-binfmt-image-debian-%: $(MULTARCH_BASE)/debian-b= ootstrap.docker "CHECK", "debian-$* exists")) endif =20 -# Enforce dependencies for composite images -docker-image-debian9-mxe: docker-image-debian9 -ifeq ($(HOST_ARCH),x86_64) -docker-image-debian-amd64: docker-image-debian10 -DOCKER_PARTIAL_IMAGES +=3D debian-amd64-cross -else -docker-image-debian-amd64-cross: docker-image-debian10 -DOCKER_PARTIAL_IMAGES +=3D debian-amd64 -endif -docker-image-debian-win32-cross: docker-image-debian9-mxe -docker-image-debian-win64-cross: docker-image-debian9-mxe - -# For non-x86 hosts not all cross-compilers have been packaged -ifneq ($(HOST_ARCH),x86_64) -DOCKER_PARTIAL_IMAGES +=3D debian-mips-cross debian-mipsel-cross debian-mi= ps64el-cross -DOCKER_PARTIAL_IMAGES +=3D debian-ppc64el-cross -DOCKER_PARTIAL_IMAGES +=3D debian-s390x-cross -DOCKER_PARTIAL_IMAGES +=3D debian-win32-cross debian-win64-cross -DOCKER_PARTIAL_IMAGES +=3D fedora travis -endif - -docker-image-debian-alpha-cross: docker-image-debian10 -docker-image-debian-arm64-cross: docker-image-debian10 -docker-image-debian-armel-cross: docker-image-debian10 -docker-image-debian-armhf-cross: docker-image-debian10 -docker-image-debian-hppa-cross: docker-image-debian10 -docker-image-debian-m68k-cross: docker-image-debian10 -docker-image-debian-mips-cross: docker-image-debian10 -docker-image-debian-mips64-cross: docker-image-debian10 -docker-image-debian-mips64el-cross: docker-image-debian10 -docker-image-debian-mipsel-cross: docker-image-debian10 -docker-image-debian-powerpc-cross: docker-image-debian10 -docker-image-debian-ppc64-cross: docker-image-debian10 -docker-image-debian-ppc64el-cross: docker-image-debian10 -docker-image-debian-riscv64-cross: docker-image-debian10 -docker-image-debian-s390x-cross: docker-image-debian10 -docker-image-debian-sh4-cross: docker-image-debian10 -docker-image-debian-sparc64-cross: docker-image-debian10 - -docker-image-travis: NOUSER=3D1 - -# Specialist build images, sometimes very limited tools -docker-image-tricore-cross: docker-image-debian9 - -# These images may be good enough for building tests but not for test buil= ds -DOCKER_PARTIAL_IMAGES +=3D debian-alpha-cross -DOCKER_PARTIAL_IMAGES +=3D debian-hppa-cross -DOCKER_PARTIAL_IMAGES +=3D debian-m68k-cross debian-mips64-cross -DOCKER_PARTIAL_IMAGES +=3D debian-powerpc-cross debian-ppc64-cross -DOCKER_PARTIAL_IMAGES +=3D debian-riscv64-cross -DOCKER_PARTIAL_IMAGES +=3D debian-sh4-cross debian-sparc64-cross -DOCKER_PARTIAL_IMAGES +=3D debian-tricore-cross -DOCKER_PARTIAL_IMAGES +=3D debian-xtensa-cross -DOCKER_PARTIAL_IMAGES +=3D fedora-i386-cross fedora-cris-cross - -# Rules for building linux-user powered images -# -# These are slower than using native cross compiler setups but can -# work around issues with poorly working multi-arch systems and broken -# packages. - # Expand all the pre-requistes for each docker image and test combination -$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \ +$(foreach i,$(QEMU_BUILD_CONTAINERS), \ $(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \ $(eval .PHONY: docker-$t@$i) \ $(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \ @@ -174,15 +141,16 @@ docker: @echo ' "IMAGE" is one of the listed container na= me.' @echo ' docker-image: Build all images.' @echo ' docker-image-IMAGE: Build image "IMAGE".' + @echo ' docker-binfmt-image-debian-IMAGE:' + @echo ' Build a binfmt Debian bootstraped IMAGE' + @echo ' Specify DEB_ARCH, DEB_TYPE and EXECUTABLE= .' + @echo ' Optionally specify DEB_URL' @echo ' docker-run: For manually running a "TEST" with "IMAGE= ".' @echo @echo 'Available container images:' - @echo ' $(DOCKER_IMAGES)' -ifneq ($(DOCKER_USER_IMAGES),) - @echo - @echo 'Available linux-user images (docker-binfmt-image-debian-%):' - @echo ' $(DOCKER_USER_IMAGES)' -endif + @echo ' BASE: $(BASE_CONTAINERS)' + @echo ' BUILD TESTS: $(TEST_BUILD_CONTAINERS)' + @echo ' BUILD QEMU: $(QEMU_BUILD_CONTAINERS)' @echo @echo 'Available tests:' @echo ' $(DOCKER_TESTS)' --=20 2.20.1