From nobody Mon Feb 9 21:39:58 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 166556180297686.4411009138089; Wed, 12 Oct 2022 01:03:22 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.420829.665885 (Exim 4.92) (envelope-from ) id 1oiWha-000595-Dr; Wed, 12 Oct 2022 08:02:54 +0000 Received: by outflank-mailman (output) from mailman id 420829.665885; Wed, 12 Oct 2022 08:02:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oiWha-00058x-AF; Wed, 12 Oct 2022 08:02:54 +0000 Received: by outflank-mailman (input) for mailman id 420829; Wed, 12 Oct 2022 08:02:53 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oiWhY-0004qO-OQ for xen-devel@lists.xenproject.org; Wed, 12 Oct 2022 08:02:52 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 451f85d9-4a04-11ed-8fd0-01056ac49cbb; Wed, 12 Oct 2022 10:02:52 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A620315A1; Wed, 12 Oct 2022 01:02:57 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C33733F67D; Wed, 12 Oct 2022 01:02:50 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 451f85d9-4a04-11ed-8fd0-01056ac49cbb From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: Doug Goldstein , Stefano Stabellini Subject: [PATCH v2 1/3] automation: Add elements for Yocto test and run Date: Wed, 12 Oct 2022 09:02:23 +0100 Message-Id: <5ee13af0b3928460e7cc18dfb363dd75679e66c8.1665561024.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1665561803865100004 Content-Type: text/plain; charset="utf-8" Add a container suitable to run yocto kirkstone build based on ubuntu 22.04. It contains all packages required by Yocto and a checkout of the layers required to build Xen with Yocto. Add a build script to build and run xen on qemu using Yocto. The script supports arm32, arm64 and x86_64 and checks that dom0 is properly booting. At this stage this does not run any guest on top of dom0. The build script is currently in the same directory than the docker files so that it can be included directly in the image. This is required for the next step which creates images with an initial cache content to speed up testing. Signed-off-by: Bertrand Marquis --- Changes in v2: - add a --dump-log command line option to build-yocto.sh script to dump the logs if an error occurs. Changes in v1: - add --image command line argument to build-yocto.sh to allow building something different than xen-image-minimal. - modify dockerfile to have one layer per line and make it easier to add other. I kept the for loop to keep the number of docker steps lower - update commit message to warn that no guest are tested. - fix build-yocto script to properly return with an error if expect script ends up in timeout or EOF. --- automation/build/yocto/build-yocto.sh | 340 ++++++++++++++++++++ automation/build/yocto/kirkstone.dockerfile | 100 ++++++ 2 files changed, 440 insertions(+) create mode 100755 automation/build/yocto/build-yocto.sh create mode 100644 automation/build/yocto/kirkstone.dockerfile diff --git a/automation/build/yocto/build-yocto.sh b/automation/build/yocto= /build-yocto.sh new file mode 100755 index 000000000000..0d31dad607fb --- /dev/null +++ b/automation/build/yocto/build-yocto.sh @@ -0,0 +1,340 @@ +#!/bin/bash +# +# Yocto meta virtualization build and run script +# +# This script is building Yocto xen-image-minimal for qemu targets and run +# them using runqemu inside yocto to check that dom0 is booting properly +# The build is using a local xen source tree so that specific patches can = be +# tested. +# In order to optimize the build time, a build cache is used so that only = xen +# packages and its dependencies are rebuilt (qemu and final image mainly). +# + +# Directories +YOCTODIR=3D"$HOME/yocto-layers" +CACHEDIR=3D"$HOME/yocto-cache" +LOGDIR=3D"$HOME/logs" +XENDIR=3D"$HOME/xen" +BUILDDIR=3D"$HOME/build" + +# what yocto bsp we support +TARGET_SUPPORTED=3D"qemuarm qemuarm64 qemux86-64" +VERBOSE=3D"n" +TARGETLIST=3D"" + +# actions to do +do_clean=3D"n" +do_build=3D"y" +do_run=3D"y" +do_localsrc=3D"n" +do_dump=3D"n" +build_result=3D0 + +# layers to include in the project +build_layerlist=3D"poky/meta poky/meta-poky poky/meta-yocto-bsp \ + meta-openembedded/meta-oe meta-openembedded/meta-python \ + meta-openembedded/meta-filesystems \ + meta-openembedded/meta-networking meta-virtualization" + +# yocto image to build +build_image=3D"xen-image-minimal" + +function print_progress() { + echo -n "$(date +%T) $*" +} + +function run_task() { + local task_name=3D"$1" + local task_target=3D"$2" + + task_log=3D"${task_name//project_}-${task_target}" + + mkdir -p "${LOGDIR}" + print_progress + echo -n "${task_name//project_} ${task_target}: " + if [ "${VERBOSE}" =3D "n" ]; then + $@ > "${LOGDIR}/${task_log}.log" 2>&1 + else + $@ 2>&1 | tee "${LOGDIR}/${task_log}.log" + fi + + if [ ${?} -ne 0 ]; then + echo "Error" + build_result=3D$((build_result+1)) + if [ "${do_dump}" =3D "y" ]; then + echo + echo "############ LOGS-START ############" + cat "${LOGDIR}/${task_log}.log" + echo "############ LOGS-END ############" + echo + fi + return 1 + else + echo "OK" + return 0 + fi +} + +function project_create() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + + ( + # init yocto project + source ${YOCTODIR}/poky/oe-init-build-env "${destdir}" + + # add needed layers + for layer in ${build_layerlist}; do + bitbake-layers add-layer ${YOCTODIR}/${layer} || exit 1 + done + ) || return 1 + + # customize project configuration + cat <> "${destdir}/conf/local.conf" +# Yocto BSP +MACHINE =3D "${target}" + +# Use local cache to reuse previous builds results +SSTATE_DIR =3D "${CACHEDIR}/sstate-cache" +DL_DIR =3D "${CACHEDIR}/downloads" + +# Enable xen and virtualization +DISTRO_FEATURES =3D " virtualization xen ipv4" + +# Speed up run by not generating ssh host keys +IMAGE_INSTALL:append:pn-xen-image-minimal =3D " ssh-pregen-hostkeys" + +# Save some disk space +INHERIT +=3D "rm_work" + +EOF + + if [ "${do_localsrc}" =3D "y" ]; then + XENVERS=3D$(grep -e "^XEN_REL" \ + "${YOCTODIR}"/meta-virtualization/recipes-extended/xen/xen_*.b= b \ + 2> /dev/null | tr -d ' ' | tr -d '?' | tr -d '"' \ + | sed -e "s/.*=3D//" | sort -V | tail -n 1) + + XENBASE=3D$(dirname "$(realpath -m "${XENDIR}")") + XENSUB=3D$(basename "$(realpath -m "${XENDIR}")") + + cat <> "${destdir}/conf/local.conf" +# Use local sources for xen and xen-tools +FILESEXTRAPATHS:prepend:pn-xen :=3D "${XENBASE}:" +FILESEXTRAPATHS:prepend:pn-xen-tools :=3D "${XENBASE}:" + +SRC_URI:pn-xen =3D "file://${XENSUB}/;subdir=3Dlocal-xen/" +SRC_URI:pn-xen-tools =3D "file://${XENSUB}/;subdir=3Dlocal-xen/" + +PREFERRED_VERSION:pn-xen =3D "${XENVERS}%" +PREFERRED_VERSION:pn-xen-tools =3D "${XENVERS}%" + +S:pn-xen =3D "\${WORKDIR}/local-xen/${XENSUB}" +S:pn-xen-tools =3D "\${WORKDIR}/local-xen/${XENSUB}" + +SRCREV:pn-xen =3D "\${AUTOREV}" +SRCREV:pn-xen-tools =3D "\${AUTOREV}" + +SRCPV:pn-xen =3D "1" +SRCPV:pn-xen-tools =3D "1" + +# Disable all QA errors as the recipe is not up to date with changes in Xen +# when we use local sources +ERROR_QA:pn-xen =3D "arch" +ERROR_QA:pn-xen-tools =3D "arch" + +EOF + fi +} + +function project_build() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + + ( + source ${YOCTODIR}/poky/oe-init-build-env "${destdir}" + + bitbake "${build_image}" || exit 1 + ) || return 1 +} + +function project_clean() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + + rm -rf "${destdir}" +} + +function project_run() { + target=3D"${1:?}" + destdir=3D"${BUILDDIR}/${target}" + ( + source ${YOCTODIR}/poky/oe-init-build-env "${destdir}" > /dev/null= 2>&1 + + /usr/bin/expect <