From nobody Sun May 19 03:12:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1669904785936927.4557834887208; Thu, 1 Dec 2022 06:26:25 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.450779.708231 (Exim 4.92) (envelope-from ) id 1p0kVk-000231-N7; Thu, 01 Dec 2022 14:26:00 +0000 Received: by outflank-mailman (output) from mailman id 450779.708231; Thu, 01 Dec 2022 14:26:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p0kVk-00022u-KC; Thu, 01 Dec 2022 14:26:00 +0000 Received: by outflank-mailman (input) for mailman id 450779; Thu, 01 Dec 2022 14:25:59 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p0kVj-000228-GV for xen-devel@lists.xenproject.org; Thu, 01 Dec 2022 14:25:59 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 12bb8471-7184-11ed-91b6-6bf2151ebd3b; Thu, 01 Dec 2022 15:25:58 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 820DDD6E; Thu, 1 Dec 2022 06:26:04 -0800 (PST) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 747163F73D; Thu, 1 Dec 2022 06:25:56 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 12bb8471-7184-11ed-91b6-6bf2151ebd3b From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu , Doug Goldstein Subject: [PATCH v6 1/3] automation: Create Yocto docker images Date: Thu, 1 Dec 2022 14:25:30 +0000 Message-Id: <43aa7a440ee7af24c829dc6a70064e2f5fce2302.1669904508.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1669904788733100001 Content-Type: text/plain; charset="utf-8" Add containers suitable to run yocto kirkstone build based on ubuntu 22.04. It contains all packages required by Yocto and a checkout of the layers required to build Xen with Yocto. Add a generic docker image template to be used to automatically generate docker files for different configurations: - specific yocto version - different targets (qemu arm, arm64 and x86) - different host platforms (x86 or arm64) During a call to 'make all', only the images for the host platform we run it on in the CI is generated. If needed, images for an other host platform can be generated manually by calling the right make target (see make help). Add a build script to build and run xen on qemu using Yocto. The script supports arm32, arm64 and x86_64 and checks that dom0 is properly booting. At this stage this does not run any guest on top of dom0. The script is to be executed in one of the docker images to build and run a system using a Xen source tree. Add automation/build/yocto/*.dockerfile to gitignore as those files are generated. Signed-off-by: Bertrand Marquis Reviewed-by: Michal Orzel Reviewed-by: Stefano Stabellini Tested-by: Stefano Stabellini --- Changes in v6: - move include of yocto.inc to keep help the default make target - add yocto/*.dockerfile to gitignore - fix typo {} instead of () - fix clean rule to remove s to dockerfile - tell make to evaluate CONTAINER right away - add example command in comment in yocto.inc - fix compilation of qemux86 on arm by switching to AUTOREV to use the latest version of Xen which includes fixes required. Changes in v5: - typo fixes - add INTERMEDIATE to remove intermediate dockerfile - use container architecture - use full words in foreach loops - fix shellcheck findings in build-yocto.sh - rebase on sstabellini next branch Changes in v4: - Rework the system to have one dockerfile template from which make will generate the required dockerfiles for the wanted configuration - add support for different host architectures - Merge the generation system into one single dockerfile - Merge patches 1 and 2 in a single patch - Introduce CONTAINERS_EXTRA to have extra containers not built by default (for those not used by CI but useful to users) Changes in v3: - limit number of jobs in yocto by default to 8 and add --num-jobs option to the script to set a custom number of jobs - do not copy anymore the build-yocto.sh script inside the main image so that the current one in the repository is used when running Changes in v2: - add a --dump-log command line option to build-yocto.sh script to dump the logs if an error occurs. Changes in v1: - add --image command line argument to build-yocto.sh to allow building something different than xen-image-minimal. - modify dockerfile to have one layer per line and make it easier to add other. I kept the for loop to keep the number of docker steps lower - update commit message to warn that no guest are tested. - fix build-yocto script to properly return with an error if expect script ends up in timeout or EOF. --- .gitignore | 1 + automation/build/Makefile | 16 +- automation/build/yocto/build-yocto.sh | 356 +++++++++++++++++++++ automation/build/yocto/yocto.dockerfile.in | 114 +++++++ automation/build/yocto/yocto.inc | 44 +++ 5 files changed, 528 insertions(+), 3 deletions(-) create mode 100755 automation/build/yocto/build-yocto.sh create mode 100644 automation/build/yocto/yocto.dockerfile.in create mode 100644 automation/build/yocto/yocto.inc diff --git a/.gitignore b/.gitignore index ea3243af9dde..ce7caa56af6d 100644 --- a/.gitignore +++ b/.gitignore @@ -38,6 +38,7 @@ dist stubdom/*.tar.gz =20 autom4te.cache/ +automation/build/yocto/*.dockerfile build-*/ config.log config.status diff --git a/automation/build/Makefile b/automation/build/Makefile index a4b2b85178cf..c4aaa747ffdf 100644 --- a/automation/build/Makefile +++ b/automation/build/Makefile @@ -1,20 +1,30 @@ =20 # the base of where these containers will appear REGISTRY :=3D registry.gitlab.com/xen-project/xen -CONTAINERS =3D $(subst .dockerfile,,$(wildcard */*.dockerfile)) +CONTAINERS :=3D $(filter-out yocto/%,$(subst .dockerfile,,$(wildcard */*.d= ockerfile))) +CONTAINERS_EXTRA =3D DOCKER_CMD ?=3D docker =20 help: @echo "Builds containers for building Xen based on different distros" @echo "To build one run 'make DISTRO/VERSION'. Available containers:" - @$(foreach file,$(sort $(CONTAINERS)),echo ${file};) + @$(foreach file,$(sort $(CONTAINERS)),echo $(file);) + @echo "Extra containers (not built using make all):" + @$(foreach file,$(sort $(CONTAINERS_EXTRA)),echo $(file);) @echo "To push container builds, set the env var PUSH" =20 +include yocto/yocto.inc + %: %.dockerfile ## Builds containers $(DOCKER_CMD) build -t $(REGISTRY)/$(@D):$(@F) -f $< $( "${LOGDIR}/${task_log}.log" 2>&1 + else + "$@" 2>&1 | tee "${LOGDIR}/${task_log}.log" + fi + + if [ ${?} -ne 0 ]; then + echo "Error" + build_result=3D$((build_result+1)) + if [ "${do_dump}" =3D "y" ]; then + echo + echo "############ LOGS-START ############" + cat "${LOGDIR}/${task_log}.log" + echo "############ LOGS-END ############" + echo + fi + return 1 + else + echo "OK" + return 0 + fi +} + +function project_create() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + + ( + # init yocto project + source "${YOCTODIR}/poky/oe-init-build-env" "${destdir}" + + # add needed layers + for layer in ${build_layerlist}; do + bitbake-layers add-layer "${YOCTODIR}/${layer}" || exit 1 + done + ) || return 1 + + # Detect latest version available in Yocto and use it instead of defau= lt + # one. + XENVERS=3D$(grep -e "^XEN_REL" \ + "${YOCTODIR}"/meta-virtualization/recipes-extended/xen/xen_*.bb \ + 2> /dev/null | tr -d ' ' | tr -d '?' | tr -d '"' \ + | sed -e "s/.*=3D//" | sort -V | tail -n 1) + + # customize project configuration + cat <> "${destdir}/conf/local.conf" +# Yocto BSP +MACHINE =3D "${target}" + +# Use local cache to reuse previous builds results +SSTATE_DIR =3D "${CACHEDIR}/sstate-cache" +DL_DIR =3D "${CACHEDIR}/downloads" + +# Enable xen and virtualization +DISTRO_FEATURES =3D " virtualization xen ipv4" + +# Speed up run by not generating ssh host keys +IMAGE_INSTALL:append:pn-xen-image-minimal =3D " ssh-pregen-hostkeys" + +# Save some disk space +INHERIT +=3D "rm_work" + +# Reduce number of jobs +BB_NUMBER_THREADS=3D"${BUILDJOBS}" + +# Use latest Xen version +PREFERRED_VERSION:pn-xen =3D "${XENVERS}%" +PREFERRED_VERSION:pn-xen-tools =3D "${XENVERS}%" + +# Use autorev for now as Xen SHA used by latest yocto recipe for Xen does = not +# include fixes required to build x86 on arm +SRCREV:pn-xen =3D "\${AUTOREV}" +SRCREV:pn-xen-tools =3D "\${AUTOREV}" + +# Disable all QA errors as the recipe is not up to date with changes in Xen +# when we use local sources +ERROR_QA:pn-xen =3D "arch" +ERROR_QA:pn-xen-tools =3D "arch" + +EOF + + if [ "${do_localsrc}" =3D "y" ]; then + XENBASE=3D$(dirname "$(realpath -m "${XENDIR}")") + XENSUB=3D$(basename "$(realpath -m "${XENDIR}")") + + cat <> "${destdir}/conf/local.conf" +# Use local sources for xen and xen-tools +FILESEXTRAPATHS:prepend:pn-xen :=3D "${XENBASE}:" +FILESEXTRAPATHS:prepend:pn-xen-tools :=3D "${XENBASE}:" + +SRC_URI:pn-xen =3D "file://${XENSUB}/;subdir=3Dlocal-xen/" +SRC_URI:pn-xen-tools =3D "file://${XENSUB}/;subdir=3Dlocal-xen/" + +S:pn-xen =3D "\${WORKDIR}/local-xen/${XENSUB}" +S:pn-xen-tools =3D "\${WORKDIR}/local-xen/${XENSUB}" + +SRCPV:pn-xen =3D "1" +SRCPV:pn-xen-tools =3D "1" + +EOF + fi +} + +function project_build() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + + ( + source "${YOCTODIR}/poky/oe-init-build-env" "${destdir}" + + bitbake "${build_image}" || exit 1 + ) || return 1 +} + +function project_clean() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + + rm -rf "${destdir}" +} + +function project_run() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + ( + source "${YOCTODIR}/poky/oe-init-build-env" "${destdir}" > /dev/nu= ll 2>&1 + + /usr/bin/expect < $$@ + +endef + +$(eval $(foreach version,$(YOCTO_VERSION),\ + $(foreach target,$(YOCTO_TARGETS),\ + $(foreach arch,$(YOCTO_ARCHS),\ + $(call GEN_DOCKER,$(version),$(target),$(arch),$(if $(filter amd64,= $(arch)),,-$(arch))))))) --=20 2.25.1 From nobody Sun May 19 03:12:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1669904781208596.9656256241756; Thu, 1 Dec 2022 06:26:21 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.450780.708237 (Exim 4.92) (envelope-from ) id 1p0kVl-00026C-3R; Thu, 01 Dec 2022 14:26:01 +0000 Received: by outflank-mailman (output) from mailman id 450780.708237; Thu, 01 Dec 2022 14:26:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p0kVk-00024m-SC; Thu, 01 Dec 2022 14:26:00 +0000 Received: by outflank-mailman (input) for mailman id 450780; Thu, 01 Dec 2022 14:26:00 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p0kVk-0001kY-0X for xen-devel@lists.xenproject.org; Thu, 01 Dec 2022 14:26:00 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 134ff323-7184-11ed-8fd2-01056ac49cbb; Thu, 01 Dec 2022 15:25:59 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 646AFED1; Thu, 1 Dec 2022 06:26:05 -0800 (PST) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1CCC43F73D; Thu, 1 Dec 2022 06:25:58 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 134ff323-7184-11ed-8fd2-01056ac49cbb From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Doug Goldstein , Stefano Stabellini Subject: [PATCH v6 2/3] automation: Add a clean rule for containers Date: Thu, 1 Dec 2022 14:25:31 +0000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1669904782424100001 Content-Type: text/plain; charset="utf-8" Add make clean support to remove the containers from the local docker registry. make clean- must be called to remove an image: make clean-yocto/kirkstone-qemuarm: remove yocto kirkstone for qemuarm image Signed-off-by: Bertrand Marquis Acked-by: Stefano Stabellini Reviewed-by: Michal Orzel Tested-by: Stefano Stabellini --- Changes in v6: - Rework clean rule to prevent calling docker during make parsing Changes in v5: - remove cleaning of all images using make clean Changes in v4: - also generate clean rule for CONTAINERS_EXTRA Changes in v3: - none Changes in v2: - none Changes in v1: - patch added --- automation/build/Makefile | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/automation/build/Makefile b/automation/build/Makefile index c4aaa747ffdf..f1d56ce532a1 100644 --- a/automation/build/Makefile +++ b/automation/build/Makefile @@ -28,3 +28,13 @@ all: $(CONTAINERS) clean: rm -f yocto/*.dockerfile =20 +define CLEAN_RULE +.PHONY: clean-$(1) +clean-$(1): + if [ -n "$$$$(docker image ls -q $(REGISTRY)/$(subst /,:,$(1)))" ]; then \ + docker image rm $(REGISTRY)/$(subst /,:,$(1)); \ + fi + +endef + +$(eval $(foreach img,$(CONTAINERS) $(CONTAINERS_EXTRA),$(call CLEAN_RULE,$= (img)))) --=20 2.25.1 From nobody Sun May 19 03:12:54 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1669904782330836.3577488609195; Thu, 1 Dec 2022 06:26:22 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.450781.708253 (Exim 4.92) (envelope-from ) id 1p0kVm-0002Zv-H1; Thu, 01 Dec 2022 14:26:02 +0000 Received: by outflank-mailman (output) from mailman id 450781.708253; Thu, 01 Dec 2022 14:26:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p0kVm-0002Yd-D3; Thu, 01 Dec 2022 14:26:02 +0000 Received: by outflank-mailman (input) for mailman id 450781; Thu, 01 Dec 2022 14:26:01 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p0kVl-0001kY-0Y for xen-devel@lists.xenproject.org; Thu, 01 Dec 2022 14:26:01 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 13d470d9-7184-11ed-8fd2-01056ac49cbb; Thu, 01 Dec 2022 15:26:00 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4719C13D5; Thu, 1 Dec 2022 06:26:06 -0800 (PST) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F3CA63F73D; Thu, 1 Dec 2022 06:25:58 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 13d470d9-7184-11ed-8fd2-01056ac49cbb From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Doug Goldstein , Stefano Stabellini Subject: [PATCH v6 3/3] automation: Add CI test jobs for Yocto Date: Thu, 1 Dec 2022 14:25:32 +0000 Message-Id: <9a50c9bfad26f6828ab0f6325721286b2af15581.1669904508.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1669904784493100001 Content-Type: text/plain; charset="utf-8" From: Michal Orzel Populate test jobs for Yocto based tests using the provided containers. Due to the size restrictions, it is currently not possible to split the build and run tasks, therefore everything is done in a single step. Test jobs for the supported Yocto targets are generic to avoid the necessity to add new ones after each Yocto release. The only thing required to be changed after updating the containers is the variable YOCTO_VERSION stored in a .yocto-test template. Signed-off-by: Michal Orzel Reviewed-by: Stefano Stabellini Tested-by: Stefano Stabellini --- Changes in v6: - add comment explaining why the unused x86 yocto entry is there Changes in v5: - none Changes in v4: - add .yocto-test for arm64 and x86 - make yocto jobs use arm64 version Changes in v3: - patch added --- automation/gitlab-ci/test.yaml | 45 ++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml index c7e0078e04f1..2d57f4feb74a 100644 --- a/automation/gitlab-ci/test.yaml +++ b/automation/gitlab-ci/test.yaml @@ -54,6 +54,35 @@ tags: - x86_64 =20 +.yocto-test: + extends: .test-jobs-common + script: + - ./automation/build/yocto/build-yocto.sh -v --log-dir=3D./logs --xen-= dir=3D`pwd` ${YOCTO_BOARD} + variables: + YOCTO_VERSION: kirkstone + CONTAINER: yocto:${YOCTO_VERSION}-${YOCTO_BOARD}-${YOCTO_HOST} + artifacts: + paths: + - 'logs/*' + when: always + needs: [] + +.yocto-test-arm64: + extends: .yocto-test + variables: + YOCTO_HOST: arm64v8 + tags: + - arm64 + +# This is not used by any test job as we only run Yocto on arm based machi= nes. +# Keep it here so that someone having x86 hardware can easily add jobs. +.yocto-test-x86-64: + extends: .yocto-test + variables: + YOCTO_HOST: amd64 + tags: + - x86_64 + # Test jobs build-each-commit-gcc: extends: .test-jobs-common @@ -188,3 +217,19 @@ qemu-smoke-x86-64-clang-pvh: - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE} needs: - debian-unstable-clang-debug + +# Yocto test jobs +yocto-qemuarm64: + extends: .yocto-test-arm64 + variables: + YOCTO_BOARD: qemuarm64 + +yocto-qemuarm: + extends: .yocto-test-arm64 + variables: + YOCTO_BOARD: qemuarm + +yocto-qemux86-64: + extends: .yocto-test-arm64 + variables: + YOCTO_BOARD: qemux86-64 --=20 2.25.1