From nobody Fri May 17 05:26:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1667224932245818.3143658424561; Mon, 31 Oct 2022 07:02:12 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.432887.685581 (Exim 4.92) (envelope-from ) id 1opVM5-0004dU-2V; Mon, 31 Oct 2022 14:01:33 +0000 Received: by outflank-mailman (output) from mailman id 432887.685581; Mon, 31 Oct 2022 14:01:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1opVM4-0004cv-Ro; Mon, 31 Oct 2022 14:01:32 +0000 Received: by outflank-mailman (input) for mailman id 432887; Mon, 31 Oct 2022 14:01:31 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1opVM3-0004Zq-19 for xen-devel@lists.xenproject.org; Mon, 31 Oct 2022 14:01:31 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 84240b00-5924-11ed-91b5-6bf2151ebd3b; Mon, 31 Oct 2022 15:01:29 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C5D811FB; Mon, 31 Oct 2022 07:01:34 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B71723F5A1; Mon, 31 Oct 2022 07:01:27 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 84240b00-5924-11ed-91b5-6bf2151ebd3b From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Doug Goldstein , Stefano Stabellini Subject: [PATCH v3 1/4] automation: Add elements for Yocto test and run Date: Mon, 31 Oct 2022 14:00:53 +0000 Message-Id: <447141266843aeb30cf08507183e06815e957dfe.1667224593.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1667224934838100003 Content-Type: text/plain; charset="utf-8" Add a container suitable to run yocto kirkstone build based on ubuntu 22.04. It contains all packages required by Yocto and a checkout of the layers required to build Xen with Yocto. Add a build script to build and run xen on qemu using Yocto. The script supports arm32, arm64 and x86_64 and checks that dom0 is properly booting. At this stage this does not run any guest on top of dom0. The build script is currently in the same directory than the docker files so that it can be included directly in the image. This is required for the next step which creates images with an initial cache content to speed up testing. Signed-off-by: Bertrand Marquis --- Changes in v3: - limit number of jobs in yocto by default to 8 and add --num-jobs option to the script to set a custom number of jobs - do not copy anymore the build-yocto.sh script inside the main image so that the current one in the repository is used when running Changes in v2: - add a --dump-log command line option to build-yocto.sh script to dump the logs if an error occurs. Changes in v1: - add --image command line argument to build-yocto.sh to allow building something different than xen-image-minimal. - modify dockerfile to have one layer per line and make it easier to add other. I kept the for loop to keep the number of docker steps lower - update commit message to warn that no guest are tested. - fix build-yocto script to properly return with an error if expect script ends up in timeout or EOF. --- automation/build/yocto/build-yocto.sh | 349 ++++++++++++++++++++ automation/build/yocto/kirkstone.dockerfile | 97 ++++++ 2 files changed, 446 insertions(+) create mode 100755 automation/build/yocto/build-yocto.sh create mode 100644 automation/build/yocto/kirkstone.dockerfile diff --git a/automation/build/yocto/build-yocto.sh b/automation/build/yocto= /build-yocto.sh new file mode 100755 index 000000000000..d0c93dfaffe0 --- /dev/null +++ b/automation/build/yocto/build-yocto.sh @@ -0,0 +1,349 @@ +#!/bin/bash +# +# Yocto meta virtualization build and run script +# +# This script is building Yocto xen-image-minimal for qemu targets and run +# them using runqemu inside yocto to check that dom0 is booting properly +# The build is using a local xen source tree so that specific patches can = be +# tested. +# In order to optimize the build time, a build cache is used so that only = xen +# packages and its dependencies are rebuilt (qemu and final image mainly). +# + +# Directories +YOCTODIR=3D"$HOME/yocto-layers" +CACHEDIR=3D"$HOME/yocto-cache" +LOGDIR=3D"$HOME/logs" +XENDIR=3D"$HOME/xen" +BUILDDIR=3D"$HOME/build" + +# what yocto bsp we support +TARGET_SUPPORTED=3D"qemuarm qemuarm64 qemux86-64" +VERBOSE=3D"n" +TARGETLIST=3D"" +BUILDJOBS=3D"8" + +# actions to do +do_clean=3D"n" +do_build=3D"y" +do_run=3D"y" +do_localsrc=3D"n" +do_dump=3D"n" +build_result=3D0 + +# layers to include in the project +build_layerlist=3D"poky/meta poky/meta-poky poky/meta-yocto-bsp \ + meta-openembedded/meta-oe meta-openembedded/meta-python \ + meta-openembedded/meta-filesystems \ + meta-openembedded/meta-networking meta-virtualization" + +# yocto image to build +build_image=3D"xen-image-minimal" + +function print_progress() { + echo -n "$(date +%T) $*" +} + +function run_task() { + local task_name=3D"$1" + local task_target=3D"$2" + + task_log=3D"${task_name//project_}-${task_target}" + + mkdir -p "${LOGDIR}" + print_progress + echo -n "${task_name//project_} ${task_target}: " + if [ "${VERBOSE}" =3D "n" ]; then + $@ > "${LOGDIR}/${task_log}.log" 2>&1 + else + $@ 2>&1 | tee "${LOGDIR}/${task_log}.log" + fi + + if [ ${?} -ne 0 ]; then + echo "Error" + build_result=3D$((build_result+1)) + if [ "${do_dump}" =3D "y" ]; then + echo + echo "############ LOGS-START ############" + cat "${LOGDIR}/${task_log}.log" + echo "############ LOGS-END ############" + echo + fi + return 1 + else + echo "OK" + return 0 + fi +} + +function project_create() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + + ( + # init yocto project + source ${YOCTODIR}/poky/oe-init-build-env "${destdir}" + + # add needed layers + for layer in ${build_layerlist}; do + bitbake-layers add-layer ${YOCTODIR}/${layer} || exit 1 + done + ) || return 1 + + # customize project configuration + cat <> "${destdir}/conf/local.conf" +# Yocto BSP +MACHINE =3D "${target}" + +# Use local cache to reuse previous builds results +SSTATE_DIR =3D "${CACHEDIR}/sstate-cache" +DL_DIR =3D "${CACHEDIR}/downloads" + +# Enable xen and virtualization +DISTRO_FEATURES =3D " virtualization xen ipv4" + +# Speed up run by not generating ssh host keys +IMAGE_INSTALL:append:pn-xen-image-minimal =3D " ssh-pregen-hostkeys" + +# Save some disk space +INHERIT +=3D "rm_work" + +# Reduce number of jobs +BB_NUMBER_THREADS=3D"${BUILDJOBS}" + +EOF + + if [ "${do_localsrc}" =3D "y" ]; then + XENVERS=3D$(grep -e "^XEN_REL" \ + "${YOCTODIR}"/meta-virtualization/recipes-extended/xen/xen_*.b= b \ + 2> /dev/null | tr -d ' ' | tr -d '?' | tr -d '"' \ + | sed -e "s/.*=3D//" | sort -V | tail -n 1) + + XENBASE=3D$(dirname "$(realpath -m "${XENDIR}")") + XENSUB=3D$(basename "$(realpath -m "${XENDIR}")") + + cat <> "${destdir}/conf/local.conf" +# Use local sources for xen and xen-tools +FILESEXTRAPATHS:prepend:pn-xen :=3D "${XENBASE}:" +FILESEXTRAPATHS:prepend:pn-xen-tools :=3D "${XENBASE}:" + +SRC_URI:pn-xen =3D "file://${XENSUB}/;subdir=3Dlocal-xen/" +SRC_URI:pn-xen-tools =3D "file://${XENSUB}/;subdir=3Dlocal-xen/" + +PREFERRED_VERSION:pn-xen =3D "${XENVERS}%" +PREFERRED_VERSION:pn-xen-tools =3D "${XENVERS}%" + +S:pn-xen =3D "\${WORKDIR}/local-xen/${XENSUB}" +S:pn-xen-tools =3D "\${WORKDIR}/local-xen/${XENSUB}" + +SRCREV:pn-xen =3D "\${AUTOREV}" +SRCREV:pn-xen-tools =3D "\${AUTOREV}" + +SRCPV:pn-xen =3D "1" +SRCPV:pn-xen-tools =3D "1" + +# Disable all QA errors as the recipe is not up to date with changes in Xen +# when we use local sources +ERROR_QA:pn-xen =3D "arch" +ERROR_QA:pn-xen-tools =3D "arch" + +EOF + fi +} + +function project_build() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + + ( + source ${YOCTODIR}/poky/oe-init-build-env "${destdir}" + + bitbake "${build_image}" || exit 1 + ) || return 1 +} + +function project_clean() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + + rm -rf "${destdir}" +} + +function project_run() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + ( + source ${YOCTODIR}/poky/oe-init-build-env "${destdir}" > /dev/null= 2>&1 + + /usr/bin/expect < Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1667224933551195.99707149863082; Mon, 31 Oct 2022 07:02:13 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.432886.685577 (Exim 4.92) (envelope-from ) id 1opVM4-0004ad-ML; Mon, 31 Oct 2022 14:01:32 +0000 Received: by outflank-mailman (output) from mailman id 432886.685577; Mon, 31 Oct 2022 14:01:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1opVM4-0004aW-Im; Mon, 31 Oct 2022 14:01:32 +0000 Received: by outflank-mailman (input) for mailman id 432886; Mon, 31 Oct 2022 14:01:30 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1opVM2-0004Ib-OI for xen-devel@lists.xenproject.org; Mon, 31 Oct 2022 14:01:30 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 82f2a89d-5924-11ed-8fd0-01056ac49cbb; Mon, 31 Oct 2022 15:01:27 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A826523A; Mon, 31 Oct 2022 07:01:35 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B48333F5A1; Mon, 31 Oct 2022 07:01:28 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 82f2a89d-5924-11ed-8fd0-01056ac49cbb From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Doug Goldstein , Stefano Stabellini Subject: [PATCH v3 2/4] automation: Add yocto containers with cache Date: Mon, 31 Oct 2022 14:00:54 +0000 Message-Id: <6538dd8bc1fce8f6848567a0a7fbab00d49bd2ae.1667224593.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1667224934567100001 Content-Type: text/plain; charset="utf-8" Create yocto containers for kirkstone with a builder step to fill the yocto cache by running the build script once. Those containers depend on the kirkstone.dockerfile container to be created and use it as base image. To run a build test on one image the following command must be called inside the container: build-yocto.sh --xen-dir=3DXENSRC TARGET where - XENSRC is the directory where xen sources are available in the container - TARGET is the build target (qemuarm64, qemuarm or qemux86_64) In case of error, the script will return an error code and the logs will be in /home/docker-build/logs. !!! The build of each container can take several hours depending on the network bandwith (for downloading all sources) and the amount of computing power on the machine where the docker images are created. !!! Signed-off-by: Bertrand Marquis --- Changes in v3: - Copy build-yocto.sh inside the container before running it to generate the cache. Changes in v2: - pass --dump-log option to build-yocto.sh during the image generation so that we have more info when there is an error during the image creation. Changes in v1: - fix dependency in Makefile --- automation/build/Makefile | 2 ++ .../build/yocto/kirkstone-qemuarm.dockerfile | 29 +++++++++++++++++++ .../yocto/kirkstone-qemuarm64.dockerfile | 29 +++++++++++++++++++ .../yocto/kirkstone-qemux86-64.dockerfile | 29 +++++++++++++++++++ 4 files changed, 89 insertions(+) create mode 100644 automation/build/yocto/kirkstone-qemuarm.dockerfile create mode 100644 automation/build/yocto/kirkstone-qemuarm64.dockerfile create mode 100644 automation/build/yocto/kirkstone-qemux86-64.dockerfile diff --git a/automation/build/Makefile b/automation/build/Makefile index a4b2b85178cf..b0dc68f00b70 100644 --- a/automation/build/Makefile +++ b/automation/build/Makefile @@ -16,5 +16,7 @@ help: $(DOCKER_CMD) push $(REGISTRY)/$(@D):$(@F); \ fi =20 +$(subst .dockerfile,,$(wildcard yocto/kirkstone-*)): yocto/kirkstone + .PHONY: all all: $(CONTAINERS) diff --git a/automation/build/yocto/kirkstone-qemuarm.dockerfile b/automati= on/build/yocto/kirkstone-qemuarm.dockerfile new file mode 100644 index 000000000000..e4540021d0af --- /dev/null +++ b/automation/build/yocto/kirkstone-qemuarm.dockerfile @@ -0,0 +1,29 @@ +# Docker file to populate the initial sstate and download cache for yocto +# for a specific target +# +# This requires the build-yocto.sh script to be in the same directory as t= his +# dockerfile during the image creation +# +# Arguments that can be passed during image creation using --build-arg: +# "from_image=3Dimg" to select the docker image and tag to start from +# "target=3Dqemuxxx": to select which target to build in this image + +ARG from_image=3Dregistry.gitlab.com/xen-project/xen/yocto:kirkstone + +FROM $from_image AS builder + +ARG target=3Dqemuarm + +# This step can take one to several hours depending on your download bandw= ith +# and the speed of your computer +COPY ./build-yocto.sh / +RUN /build-yocto.sh --dump-log $target + +FROM $from_image + +# Only copy the cache status +COPY --from=3Dbuilder /home/$USER_NAME/yocto-cache /home/$USER_NAME/yocto-= cache/. + +LABEL maintainer.name=3D"The Xen Project " \ + maintainer.email=3D"xen-devel@lists.xenproject.org" + diff --git a/automation/build/yocto/kirkstone-qemuarm64.dockerfile b/automa= tion/build/yocto/kirkstone-qemuarm64.dockerfile new file mode 100644 index 000000000000..aea3fc1f3e5d --- /dev/null +++ b/automation/build/yocto/kirkstone-qemuarm64.dockerfile @@ -0,0 +1,29 @@ +# Docker file to populate the initial sstate and download cache for yocto +# for a specific target +# +# This requires the build-yocto.sh script to be in the same directory as t= his +# dockerfile during the image creation +# +# Arguments that can be passed during image creation using --build-arg: +# "from_image=3Dimg" to select the docker image and tag to start from +# "target=3Dqemuxxx": to select which target to build in this image + +ARG from_image=3Dregistry.gitlab.com/xen-project/xen/yocto:kirkstone + +FROM $from_image AS builder + +ARG target=3Dqemuarm64 + +# This step can take one to several hours depending on your download bandw= ith +# and the speed of your computer +COPY ./build-yocto.sh / +RUN /build-yocto.sh --dump-log $target + +FROM $from_image + +# Only copy the cache status +COPY --from=3Dbuilder /home/$USER_NAME/yocto-cache /home/$USER_NAME/yocto-= cache/. + +LABEL maintainer.name=3D"The Xen Project " \ + maintainer.email=3D"xen-devel@lists.xenproject.org" + diff --git a/automation/build/yocto/kirkstone-qemux86-64.dockerfile b/autom= ation/build/yocto/kirkstone-qemux86-64.dockerfile new file mode 100644 index 000000000000..9b2277a65e33 --- /dev/null +++ b/automation/build/yocto/kirkstone-qemux86-64.dockerfile @@ -0,0 +1,29 @@ +# Docker file to populate the initial sstate and download cache for yocto +# for a specific target +# +# This requires the build-yocto.sh script to be in the same directory as t= his +# dockerfile during the image creation +# +# Arguments that can be passed during image creation using --build-arg: +# "from_image=3Dimg" to select the docker image and tag to start from +# "target=3Dqemuxxx": to select which target to build in this image + +ARG from_image=3Dregistry.gitlab.com/xen-project/xen/yocto:kirkstone + +FROM $from_image AS builder + +ARG target=3Dqemux86-64 + +# This step can take one to several hours depending on your download bandw= ith +# and the speed of your computer +COPY ./build-yocto.sh / +RUN /build-yocto.sh --dump-log $target + +FROM $from_image + +# Only copy the cache status +COPY --from=3Dbuilder /home/$USER_NAME/yocto-cache /home/$USER_NAME/yocto-= cache/. + +LABEL maintainer.name=3D"The Xen Project " \ + maintainer.email=3D"xen-devel@lists.xenproject.org" + --=20 2.25.1 From nobody Fri May 17 05:26:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1667224921900886.9132079635456; Mon, 31 Oct 2022 07:02:01 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.432888.685588 (Exim 4.92) (envelope-from ) id 1opVM5-0004ji-F2; Mon, 31 Oct 2022 14:01:33 +0000 Received: by outflank-mailman (output) from mailman id 432888.685588; Mon, 31 Oct 2022 14:01:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1opVM5-0004hm-6V; Mon, 31 Oct 2022 14:01:33 +0000 Received: by outflank-mailman (input) for mailman id 432888; Mon, 31 Oct 2022 14:01:31 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1opVM3-0004Ib-OT for xen-devel@lists.xenproject.org; Mon, 31 Oct 2022 14:01:31 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 83773202-5924-11ed-8fd0-01056ac49cbb; Mon, 31 Oct 2022 15:01:28 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8ACE8113E; Mon, 31 Oct 2022 07:01:36 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 972413F5A1; Mon, 31 Oct 2022 07:01:29 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 83773202-5924-11ed-8fd0-01056ac49cbb From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Doug Goldstein , Stefano Stabellini Subject: [PATCH v3 3/4] automation: Add a clean rule for containers Date: Mon, 31 Oct 2022 14:00:55 +0000 Message-Id: <08c3c29a85d0857ec7652f86c66e801bf0bc3e89.1667224593.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1667224922427100001 Content-Type: text/plain; charset="utf-8" Add make clean support to remove the containers from the local docker registry: - make clean: remove all images - clean-yocto/kirkstone-qemuarm: remove yocto kirkstone for qemuarm image Signed-off-by: Bertrand Marquis --- Changes in v3: - none Changes in v2: - none Changes in v1: - patch added --- automation/build/Makefile | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/automation/build/Makefile b/automation/build/Makefile index b0dc68f00b70..9fbb49c8717e 100644 --- a/automation/build/Makefile +++ b/automation/build/Makefile @@ -4,6 +4,17 @@ REGISTRY :=3D registry.gitlab.com/xen-project/xen CONTAINERS =3D $(subst .dockerfile,,$(wildcard */*.dockerfile)) DOCKER_CMD ?=3D docker =20 +define CLEAN_RULE +.PHONY: clean-$(1) +clean-$(1): +ifneq ($$(shell docker image ls -q $(REGISTRY)/$(subst /,:,$(1))),) + docker image rm $(REGISTRY)/$(subst /,:,$(1)) +endif + +clean: clean-$(1) + +endef + help: @echo "Builds containers for building Xen based on different distros" @echo "To build one run 'make DISTRO/VERSION'. Available containers:" @@ -18,5 +29,9 @@ help: =20 $(subst .dockerfile,,$(wildcard yocto/kirkstone-*)): yocto/kirkstone =20 -.PHONY: all +.PHONY: all clean all: $(CONTAINERS) + +clean: + +$(eval $(foreach img,$(CONTAINERS),$(call CLEAN_RULE,$(img)))) --=20 2.25.1 From nobody Fri May 17 05:26:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1667224924370110.68437370808226; Mon, 31 Oct 2022 07:02:04 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.432889.685610 (Exim 4.92) (envelope-from ) id 1opVM6-0005MV-Tu; Mon, 31 Oct 2022 14:01:34 +0000 Received: by outflank-mailman (output) from mailman id 432889.685610; Mon, 31 Oct 2022 14:01:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1opVM6-0005LU-Qn; Mon, 31 Oct 2022 14:01:34 +0000 Received: by outflank-mailman (input) for mailman id 432889; Mon, 31 Oct 2022 14:01:32 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1opVM4-0004Ib-Oq for xen-devel@lists.xenproject.org; Mon, 31 Oct 2022 14:01:32 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 83ff3c8c-5924-11ed-8fd0-01056ac49cbb; Mon, 31 Oct 2022 15:01:28 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6DDE911FB; Mon, 31 Oct 2022 07:01:37 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7A3373F5A1; Mon, 31 Oct 2022 07:01:30 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 83ff3c8c-5924-11ed-8fd0-01056ac49cbb From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Doug Goldstein , Stefano Stabellini Subject: [PATCH v3 4/4] automation: Add CI test jobs for Yocto Date: Mon, 31 Oct 2022 14:00:56 +0000 Message-Id: <00335fd0abaee077c09a33ec6c3ae93ddce30f5b.1667224593.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1667224926581100001 Content-Type: text/plain; charset="utf-8" From: Michal Orzel Populate test jobs for Yocto based tests using the provided containers. Due to the size restrictions, it is currently not possible to split the build and run tasks, therefore everything is done in a single step. Test jobs for the supported Yocto targets are generic to avoid the necessity to add new ones after each Yocto release. The only thing required to be changed after updating the containers is the variable YOCTO_VERSION stored in a .yocto-test template. Signed-off-by: Michal Orzel --- Changes in v3: - patch added --- automation/gitlab-ci/test.yaml | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml index fc0884b12082..44febcf97262 100644 --- a/automation/gitlab-ci/test.yaml +++ b/automation/gitlab-ci/test.yaml @@ -52,6 +52,18 @@ tags: - x86_64 =20 +.yocto-test: + extends: .test-jobs-common + script: + - ./automation/build/yocto/build-yocto.sh -v --log-dir=3D./logs --xen-= dir=3D`pwd` ${YOCTO_BOARD} + variables: + YOCTO_VERSION: kirkstone + CONTAINER: yocto:${YOCTO_VERSION}-${YOCTO_BOARD} + artifacts: + paths: + - 'logs/*' + when: always + # Test jobs build-each-commit-gcc: extends: .test-jobs-common @@ -186,3 +198,25 @@ qemu-smoke-x86-64-clang-pvh: - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE} needs: - debian-unstable-clang-debug + +# Yocto test jobs +yocto-qemuarm64: + extends: .yocto-test + variables: + YOCTO_BOARD: qemuarm64 + tags: + - arm64 + +yocto-qemuarm: + extends: .yocto-test + variables: + YOCTO_BOARD: qemuarm + tags: + - arm32 + +yocto-qemux86-64: + extends: .yocto-test + variables: + YOCTO_BOARD: qemux86-64 + tags: + - x86_64 --=20 2.25.1