This patch series is a first attempt to check if we could use Yocto in
gitlab ci to build and run xen on qemu for arm, arm64 and x86.

The first patch is creating a container with all elements required to
build Yocto, a checkout of the yocto layers required and an helper
script to build and run xen on qemu with yocto.

The second patch is creating containers with a first build of yocto done
so that susbsequent build with those containers would only rebuild what
was changed and take the rest from the cache.

This is is mainly for discussion and sharing as there are still some
issues/problem to solve:
- building the qemu* containers can take several hours depending on the
  network bandwith and computing power of the machine where those are
  created
- produced containers containing the cache have a size between 8 and
  12GB depending on the architecture. We might need to store the build
  cache somewhere else to reduce the size. If we choose to have one
  single image, the needed size is around 20GB and we need up to 40GB
  during the build, which is why I splitted them.
- during the build and run, we use a bit more then 20GB of disk which is
  over the allowed size in gitlab

Once all problems passed, this can be used to build and run dom0 on qemu
with a modified Xen on the 3 archs in less than 10 minutes.

Bertrand Marquis (2):
  automation: Add elements for Yocto test and run
  automation: Add yocto containers with cache

 automation/build/Makefile                     |   2 +
 automation/build/yocto/build-yocto.sh         | 322 ++++++++++++++++++
 .../build/yocto/kirkstone-qemuarm.dockerfile  |  28 ++
 .../yocto/kirkstone-qemuarm64.dockerfile      |  28 ++
 .../yocto/kirkstone-qemux86-64.dockerfile     |  28 ++
 automation/build/yocto/kirkstone.dockerfile   |  98 ++++++
 6 files changed, 506 insertions(+)
 create mode 100755 automation/build/yocto/build-yocto.sh
 create mode 100644 automation/build/yocto/kirkstone-qemuarm.dockerfile
 create mode 100644 automation/build/yocto/kirkstone-qemuarm64.dockerfile
 create mode 100644 automation/build/yocto/kirkstone-qemux86-64.dockerfile
 create mode 100644 automation/build/yocto/kirkstone.dockerfile

-- 
2.25.1

On Thu, Jul 14, 2022 at 3:10 AM Bertrand Marquis
<bertrand.marquis@arm.com> wrote:
>
> This patch series is a first attempt to check if we could use Yocto in
> gitlab ci to build and run xen on qemu for arm, arm64 and x86.

Hi Bertrand, thanks for posting this.

I'm still making my way through it, and should be able to speak more
to the OE/Yocto aspects than the Xen automation integration but at
first pass, I think that this is work in the right direction.
A few quick early points:
- The build-yocto.sh script is clear to understand, which is helpful.
- The layers that you have selected to include in the build are good.
Might be worth considering using openembedded-core, which is poky's
upstream, but I think either is a valid choice.
    - listing the layers one-per-line in the script might make it
easier to patch in additional layers downstream, if needed
- The target image of 'xen-image-minimal' is the right start; it would
be nice to be able to pass that as an input from the dockerfile to
allow for using this with other images.
- Possibly worth mentioning somewhere in the series description that
this introduces coverage for x86-64 but not 32-bit x86 guests - it's
the right choice given that this is just booting to a dom0.

Christopher

> The first patch is creating a container with all elements required to
> build Yocto, a checkout of the yocto layers required and an helper
> script to build and run xen on qemu with yocto.
>
> The second patch is creating containers with a first build of yocto done
> so that susbsequent build with those containers would only rebuild what
> was changed and take the rest from the cache.
>
> This is is mainly for discussion and sharing as there are still some
> issues/problem to solve:
> - building the qemu* containers can take several hours depending on the
>   network bandwith and computing power of the machine where those are
>   created
> - produced containers containing the cache have a size between 8 and
>   12GB depending on the architecture. We might need to store the build
>   cache somewhere else to reduce the size. If we choose to have one
>   single image, the needed size is around 20GB and we need up to 40GB
>   during the build, which is why I splitted them.
> - during the build and run, we use a bit more then 20GB of disk which is
>   over the allowed size in gitlab
>
> Once all problems passed, this can be used to build and run dom0 on qemu
> with a modified Xen on the 3 archs in less than 10 minutes.
>
> Bertrand Marquis (2):
>   automation: Add elements for Yocto test and run
>   automation: Add yocto containers with cache
>
>  automation/build/Makefile                     |   2 +
>  automation/build/yocto/build-yocto.sh         | 322 ++++++++++++++++++
>  .../build/yocto/kirkstone-qemuarm.dockerfile  |  28 ++
>  .../yocto/kirkstone-qemuarm64.dockerfile      |  28 ++
>  .../yocto/kirkstone-qemux86-64.dockerfile     |  28 ++
>  automation/build/yocto/kirkstone.dockerfile   |  98 ++++++
>  6 files changed, 506 insertions(+)
>  create mode 100755 automation/build/yocto/build-yocto.sh
>  create mode 100644 automation/build/yocto/kirkstone-qemuarm.dockerfile
>  create mode 100644 automation/build/yocto/kirkstone-qemuarm64.dockerfile
>  create mode 100644 automation/build/yocto/kirkstone-qemux86-64.dockerfile
>  create mode 100644 automation/build/yocto/kirkstone.dockerfile
>
> --
> 2.25.1
>
>

Hi Christopher,

> On 19 Jul 2022, at 05:34, Christopher Clark <christopher.w.clark@gmail.com> wrote:
> 
> On Thu, Jul 14, 2022 at 3:10 AM Bertrand Marquis
> <bertrand.marquis@arm.com> wrote:
>> 
>> This patch series is a first attempt to check if we could use Yocto in
>> gitlab ci to build and run xen on qemu for arm, arm64 and x86.
> 
> Hi Bertrand, thanks for posting this.
> 
> I'm still making my way through it, and should be able to speak more
> to the OE/Yocto aspects than the Xen automation integration but at
> first pass, I think that this is work in the right direction.
> A few quick early points:
> - The build-yocto.sh script is clear to understand, which is helpful.

Thanks

> - The layers that you have selected to include in the build are good.
> Might be worth considering using openembedded-core, which is poky's
> upstream, but I think either is a valid choice.

That was how I did it first but packing them in one call is reducing the
number of intermediate images during the creation. If having more is ok
I can split this in v2

>    - listing the layers one-per-line in the script might make it
> easier to patch in additional layers downstream, if needed
> - The target image of 'xen-image-minimal' is the right start; it would
> be nice to be able to pass that as an input from the dockerfile to
> allow for using this with other images.

Using a different image might also trigger other changes (for example
if you want to build xen-guest-image-minimal then you do not need the
same features).
Anyway I will add a parameter to build-yocto.sh to do it.

> - Possibly worth mentioning somewhere in the series description that
> this introduces coverage for x86-64 but not 32-bit x86 guests - it's
> the right choice given that this is just booting to a dom0.

I will add something saying that it does not cover booting guests (yet !!)
but the 32bit guest issue is also true for arm64 so mentioning it for x86
would be weird.

Cheers
Bertrand

> 
> Christopher
> 
>> The first patch is creating a container with all elements required to
>> build Yocto, a checkout of the yocto layers required and an helper
>> script to build and run xen on qemu with yocto.
>> 
>> The second patch is creating containers with a first build of yocto done
>> so that susbsequent build with those containers would only rebuild what
>> was changed and take the rest from the cache.
>> 
>> This is is mainly for discussion and sharing as there are still some
>> issues/problem to solve:
>> - building the qemu* containers can take several hours depending on the
>>  network bandwith and computing power of the machine where those are
>>  created
>> - produced containers containing the cache have a size between 8 and
>>  12GB depending on the architecture. We might need to store the build
>>  cache somewhere else to reduce the size. If we choose to have one
>>  single image, the needed size is around 20GB and we need up to 40GB
>>  during the build, which is why I splitted them.
>> - during the build and run, we use a bit more then 20GB of disk which is
>>  over the allowed size in gitlab
>> 
>> Once all problems passed, this can be used to build and run dom0 on qemu
>> with a modified Xen on the 3 archs in less than 10 minutes.
>> 
>> Bertrand Marquis (2):
>>  automation: Add elements for Yocto test and run
>>  automation: Add yocto containers with cache
>> 
>> automation/build/Makefile                     |   2 +
>> automation/build/yocto/build-yocto.sh         | 322 ++++++++++++++++++
>> .../build/yocto/kirkstone-qemuarm.dockerfile  |  28 ++
>> .../yocto/kirkstone-qemuarm64.dockerfile      |  28 ++
>> .../yocto/kirkstone-qemux86-64.dockerfile     |  28 ++
>> automation/build/yocto/kirkstone.dockerfile   |  98 ++++++
>> 6 files changed, 506 insertions(+)
>> create mode 100755 automation/build/yocto/build-yocto.sh
>> create mode 100644 automation/build/yocto/kirkstone-qemuarm.dockerfile
>> create mode 100644 automation/build/yocto/kirkstone-qemuarm64.dockerfile
>> create mode 100644 automation/build/yocto/kirkstone-qemux86-64.dockerfile
>> create mode 100644 automation/build/yocto/kirkstone.dockerfile
>> 
>> --
>> 2.25.1
>> 
>>