1. Patchew
  2. Xen
  3. View series

[PATCH] CI: workaround broken selinux+docker interaction in yocto

Marek Marczykowski-Górecki posted 1 patch 1 month, 2 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://gitlab.com/xen-project/patchew/xen tags/patchew/20240720001508.1673856-1-marmarek@invisiblethingslab.com
automation/build/yocto/yocto.dockerfile.in | 4 ++++
1 file changed, 4 insertions(+)
[PATCH] CI: workaround broken selinux+docker interaction in yocto
Posted by Marek Marczykowski-Górecki 1 month, 2 weeks ago 
`cp --preserve=xattr` doesn't work in docker when SELinux is enabled. It
tries to set the "security.selinux" xattr, but SELinux (or overlay fs?)
denies it.
Workaround it by skipping selinux.selinux xattr copying.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Tested here:
https://gitlab.com/xen-project/people/marmarek/xen/-/jobs/7386198058

But since yocto container fails to build, it isn't exactly easy to apply
this patch...
"kirkstone" branch of meta-virtualization seems to target Xen 4.15 and
4.16, so it isn't exactly surprising it fails to build with 4.19.

I tried also bumping yocto version to scarthgap (which supposedly should
have updated pygrub patch), but that fails to build for me too, with a
different error:

    ERROR: Layer 'filesystems-layer' depends on layer 'networking-layer', but this layer is not enabled in your configuration
    ERROR: Parse failure with the specified layer added, exiting.
    ...
    ERROR: Nothing PROVIDES 'xen-image-minimal'. Close matches:
      core-image-minimal
      core-image-minimal-dev
    Parsing of 2472 .bb files complete (0 cached, 2472 parsed). 4309 targets, 101 skipped, 0 masked, 0 errors.
---
 automation/build/yocto/yocto.dockerfile.in | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/automation/build/yocto/yocto.dockerfile.in b/automation/build/yocto/yocto.dockerfile.in
index fbaa4e191caa..600db7bf4d19 100644
--- a/automation/build/yocto/yocto.dockerfile.in
+++ b/automation/build/yocto/yocto.dockerfile.in
@@ -68,6 +68,10 @@ RUN locale-gen en_US.UTF-8 && update-locale LC_ALL=en_US.UTF-8 \
 ENV LANG en_US.UTF-8
 ENV LC_ALL en_US.UTF-8
 
+# Workaround `cp --preserve=xattr` not working in docker when SELinux is
+# enabled
+RUN echo "security.selinux skip" >> /etc/xattr.conf
+
 # Create a user for the build (we don't want to build as root).
 ENV USER_NAME docker-build
 ARG host_uid=1000
-- 
2.45.2
Re: [PATCH] CI: workaround broken selinux+docker interaction in yocto
Posted by Stefano Stabellini 1 month, 1 week ago 
On Sat, 20 Jul 2024, Marek Marczykowski-Górecki wrote:
> `cp --preserve=xattr` doesn't work in docker when SELinux is enabled. It
> tries to set the "security.selinux" xattr, but SELinux (or overlay fs?)
> denies it.
> Workaround it by skipping selinux.selinux xattr copying.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

I tested this patch together with another patch to upgrade Yocto to
scarthgap, I'll send that separately. I have already built and pushed
the yocto scarthgap containers.

> ---
> Tested here:
> https://gitlab.com/xen-project/people/marmarek/xen/-/jobs/7386198058
> 
> But since yocto container fails to build, it isn't exactly easy to apply
> this patch...
> "kirkstone" branch of meta-virtualization seems to target Xen 4.15 and
> 4.16, so it isn't exactly surprising it fails to build with 4.19.
> 
> I tried also bumping yocto version to scarthgap (which supposedly should
> have updated pygrub patch), but that fails to build for me too, with a
> different error:
> 
>     ERROR: Layer 'filesystems-layer' depends on layer 'networking-layer', but this layer is not enabled in your configuration
>     ERROR: Parse failure with the specified layer added, exiting.
>     ...
>     ERROR: Nothing PROVIDES 'xen-image-minimal'. Close matches:
>       core-image-minimal
>       core-image-minimal-dev
>     Parsing of 2472 .bb files complete (0 cached, 2472 parsed). 4309 targets, 101 skipped, 0 masked, 0 errors.
> ---
>  automation/build/yocto/yocto.dockerfile.in | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/automation/build/yocto/yocto.dockerfile.in b/automation/build/yocto/yocto.dockerfile.in
~ index fbaa4e191caa..600db7bf4d19 100644
> --- a/automation/build/yocto/yocto.dockerfile.in
> +++ b/automation/build/yocto/yocto.dockerfile.in
> @@ -68,6 +68,10 @@ RUN locale-gen en_US.UTF-8 && update-locale LC_ALL=en_US.UTF-8 \
>  ENV LANG en_US.UTF-8
>  ENV LC_ALL en_US.UTF-8
>  
> +# Workaround `cp --preserve=xattr` not working in docker when SELinux is
> +# enabled
> +RUN echo "security.selinux skip" >> /etc/xattr.conf
> +
>  # Create a user for the build (we don't want to build as root).
>  ENV USER_NAME docker-build
>  ARG host_uid=1000
> -- 
> 2.45.2
>
Re: [PATCH] CI: workaround broken selinux+docker interaction in yocto
Posted by Stefano Stabellini 1 month, 2 weeks ago 
On Sat, 20 Jul 2024, Marek Marczykowski-Górecki wrote:

> `cp --preserve=xattr` doesn't work in docker when SELinux is enabled. It
> tries to set the "security.selinux" xattr, but SELinux (or overlay fs?)
> denies it.
> Workaround it by skipping selinux.selinux xattr copying.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
> Tested here:
> https://gitlab.com/xen-project/people/marmarek/xen/-/jobs/7386198058
> 
> But since yocto container fails to build, it isn't exactly easy to apply
> this patch...
> "kirkstone" branch of meta-virtualization seems to target Xen 4.15 and
> 4.16, so it isn't exactly surprising it fails to build with 4.19.
> 
> I tried also bumping yocto version to scarthgap (which supposedly should
> have updated pygrub patch), but that fails to build for me too, with a
> different error:
> 
>     ERROR: Layer 'filesystems-layer' depends on layer 'networking-layer', but this layer is not enabled in your configuration
>     ERROR: Parse failure with the specified layer added, exiting.
>     ...
>     ERROR: Nothing PROVIDES 'xen-image-minimal'. Close matches:
>       core-image-minimal
>       core-image-minimal-dev
>     Parsing of 2472 .bb files complete (0 cached, 2472 parsed). 4309 targets, 101 skipped, 0 masked, 0 errors.

This patch should upgrade succesfully to scarthgap, can you give it a
try? I could rebuild succefully the container, including a successful
Yocto build, but I did not push the container to the Hub yet. I only
tried x86, I haven't tried ARM yet.

---
automation: upgrade Yocto to scarthgap

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

diff --git a/automation/build/yocto/build-yocto.sh b/automation/build/yocto/build-yocto.sh
index 93ce81ce82..06efcce6ca 100755
--- a/automation/build/yocto/build-yocto.sh
+++ b/automation/build/yocto/build-yocto.sh
@@ -38,8 +38,9 @@ build_result=0
 # layers to include in the project
 build_layerlist="poky/meta poky/meta-poky poky/meta-yocto-bsp \
                  meta-openembedded/meta-oe meta-openembedded/meta-python \
+                 meta-openembedded/meta-networking \
                  meta-openembedded/meta-filesystems \
-                 meta-openembedded/meta-networking meta-virtualization"
+                 meta-virtualization"
 
 # yocto image to build
 build_image="xen-image-minimal"
diff --git a/automation/build/yocto/yocto.inc b/automation/build/yocto/yocto.inc
index 2f3b1a5b2a..209df7dde9 100644
--- a/automation/build/yocto/yocto.inc
+++ b/automation/build/yocto/yocto.inc
@@ -6,10 +6,10 @@
 # YOCTOVERSION-TARGET for x86_64 hosts
 # YOCTOVERSION-TARGET-arm64v8 for arm64 hosts
 # For example you can build an arm64 container with the following command:
-# make yocto/kirkstone-qemuarm64-arm64v8
+# make yocto/scarthgap-qemuarm64-arm64v8
 
 # Yocto versions we are currently using.
-YOCTO_VERSION = kirkstone
+YOCTO_VERSION = scarthgap
 
 # Yocto BSPs we want to build for.
 YOCTO_TARGETS = qemuarm64 qemuarm qemux86-64
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 7ce88d38e7..32045cef0c 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -212,7 +212,7 @@
   script:
     - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs --xen-dir=`pwd` ${YOCTO_BOARD} ${YOCTO_OUTPUT}
   variables:
-    YOCTO_VERSION: kirkstone
+    YOCTO_VERSION: scarthgap
     CONTAINER: yocto:${YOCTO_VERSION}-${YOCTO_BOARD}${YOCTO_HOST}
   artifacts:
     paths:
Re: [PATCH] CI: workaround broken selinux+docker interaction in yocto
Posted by Andrew Cooper 1 month, 2 weeks ago 
On 20/07/2024 1:15 am, Marek Marczykowski-Górecki wrote:
> `cp --preserve=xattr` doesn't work in docker when SELinux is enabled. It
> tries to set the "security.selinux" xattr, but SELinux (or overlay fs?)
> denies it.
> Workaround it by skipping selinux.selinux xattr copying.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
> Tested here:
> https://gitlab.com/xen-project/people/marmarek/xen/-/jobs/7386198058
>
> But since yocto container fails to build, it isn't exactly easy to apply
> this patch...
> "kirkstone" branch of meta-virtualization seems to target Xen 4.15 and
> 4.16, so it isn't exactly surprising it fails to build with 4.19.

Why is the external version of Xen relevant to rebuilding the container ?

Or is it that kirkstone has updated since the container was last built?

I'm not familiar with yocto, and a quick glance at the docs haven't
helped...

~Andrew

>
> I tried also bumping yocto version to scarthgap (which supposedly should
> have updated pygrub patch), but that fails to build for me too, with a
> different error:
>
>     ERROR: Layer 'filesystems-layer' depends on layer 'networking-layer', but this layer is not enabled in your configuration
>     ERROR: Parse failure with the specified layer added, exiting.
>     ...
>     ERROR: Nothing PROVIDES 'xen-image-minimal'. Close matches:
>       core-image-minimal
>       core-image-minimal-dev
>     Parsing of 2472 .bb files complete (0 cached, 2472 parsed). 4309 targets, 101 skipped, 0 masked, 0 errors.
> ---
>  automation/build/yocto/yocto.dockerfile.in | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/automation/build/yocto/yocto.dockerfile.in b/automation/build/yocto/yocto.dockerfile.in
> index fbaa4e191caa..600db7bf4d19 100644
> --- a/automation/build/yocto/yocto.dockerfile.in
> +++ b/automation/build/yocto/yocto.dockerfile.in
> @@ -68,6 +68,10 @@ RUN locale-gen en_US.UTF-8 && update-locale LC_ALL=en_US.UTF-8 \
>  ENV LANG en_US.UTF-8
>  ENV LC_ALL en_US.UTF-8
>  
> +# Workaround `cp --preserve=xattr` not working in docker when SELinux is
> +# enabled
> +RUN echo "security.selinux skip" >> /etc/xattr.conf
> +
>  # Create a user for the build (we don't want to build as root).
>  ENV USER_NAME docker-build
>  ARG host_uid=1000
Re: [PATCH] CI: workaround broken selinux+docker interaction in yocto
Posted by Marek Marczykowski-Górecki 1 month, 2 weeks ago 
On Mon, Jul 22, 2024 at 06:16:51PM +0100, Andrew Cooper wrote:
> On 20/07/2024 1:15 am, Marek Marczykowski-Górecki wrote:
> > `cp --preserve=xattr` doesn't work in docker when SELinux is enabled. It
> > tries to set the "security.selinux" xattr, but SELinux (or overlay fs?)
> > denies it.
> > Workaround it by skipping selinux.selinux xattr copying.
> >
> > Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> > ---
> > Tested here:
> > https://gitlab.com/xen-project/people/marmarek/xen/-/jobs/7386198058
> >
> > But since yocto container fails to build, it isn't exactly easy to apply
> > this patch...
> > "kirkstone" branch of meta-virtualization seems to target Xen 4.15 and
> > 4.16, so it isn't exactly surprising it fails to build with 4.19.
> 
> Why is the external version of Xen relevant to rebuilding the container ?

I think it tries to build xen_git.bb, which fetches "master" branch, and
this fails to build with its current state.

> Or is it that kirkstone has updated since the container was last built?
> 
> I'm not familiar with yocto, and a quick glance at the docs haven't
> helped...
> 
> ~Andrew
> 
> >
> > I tried also bumping yocto version to scarthgap (which supposedly should
> > have updated pygrub patch), but that fails to build for me too, with a
> > different error:
> >
> >     ERROR: Layer 'filesystems-layer' depends on layer 'networking-layer', but this layer is not enabled in your configuration
> >     ERROR: Parse failure with the specified layer added, exiting.
> >     ...
> >     ERROR: Nothing PROVIDES 'xen-image-minimal'. Close matches:
> >       core-image-minimal
> >       core-image-minimal-dev
> >     Parsing of 2472 .bb files complete (0 cached, 2472 parsed). 4309 targets, 101 skipped, 0 masked, 0 errors.

In the meantime I've solved this issue by reordering layers in
build-yocto.sh (meta-networking before meta-filesystems). But then, ran
out of disk space (40GB wasn't enough) and hasn't retried yet...

> > ---
> >  automation/build/yocto/yocto.dockerfile.in | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/automation/build/yocto/yocto.dockerfile.in b/automation/build/yocto/yocto.dockerfile.in
> > index fbaa4e191caa..600db7bf4d19 100644
> > --- a/automation/build/yocto/yocto.dockerfile.in
> > +++ b/automation/build/yocto/yocto.dockerfile.in
> > @@ -68,6 +68,10 @@ RUN locale-gen en_US.UTF-8 && update-locale LC_ALL=en_US.UTF-8 \
> >  ENV LANG en_US.UTF-8
> >  ENV LC_ALL en_US.UTF-8
> >  
> > +# Workaround `cp --preserve=xattr` not working in docker when SELinux is
> > +# enabled
> > +RUN echo "security.selinux skip" >> /etc/xattr.conf
> > +
> >  # Create a user for the build (we don't want to build as root).
> >  ENV USER_NAME docker-build
> >  ARG host_uid=1000
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Patchew 2.1-devel-b67e4c2

© 2016 - 2024 Red Hat, Inc.