automation: Add missing and drop obsoleted aliases from containerize

[PATCH] automation: Add missing and drop obsoleted aliases from containerize

Posted by Michal Orzel 1 year, 1 month ago

Add missing aliases for:
 - debian:unstable-cppcheck
 - debian:unstable-arm64v8-arm32-gcc
 - ubuntu:bionic

Remove aliases for no longer used containers:
 - centos:7.2
 - debian:unstable-arm32-gcc

Modify docs to refer to CentOS 7 instead of 7.2 not to create confusion.

Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
Open questions related to the CI cleanup (@Andrew, @Anthony):
 - Why do we keep suse:sles11sp4 dockerfile?
 - Why do we keep jessie dockefiles?
---
 automation/build/README.md      | 4 ++--
 automation/scripts/containerize | 5 +++--
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/automation/build/README.md b/automation/build/README.md
index 4cc1acb6b41c..2d07cafe0eaa 100644
--- a/automation/build/README.md
+++ b/automation/build/README.md
@@ -44,10 +44,10 @@ understands.
   DOCKER_CMD=podman ./automation/scripts/containerize make
   ```
 
-- CONTAINER: This overrides the container to use. For CentOS 7.2, use:
+- CONTAINER: This overrides the container to use. For CentOS 7, use:
 
   ```
-  CONTAINER=centos72 ./automation/scripts/containerize make
+  CONTAINER=centos7 ./automation/scripts/containerize make
   ```
 
 - CONTAINER_PATH: This overrides the path that will be available under the
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index 9b1a302d0565..5476ff0ea10d 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -29,7 +29,6 @@ case "_${CONTAINER}" in
     _archlinux|_arch) CONTAINER="${BASE}/archlinux:current" ;;
     _riscv64) CONTAINER="${BASE}/archlinux:current-riscv64" ;;
     _centos7) CONTAINER="${BASE}/centos:7" ;;
-    _centos72) CONTAINER="${BASE}/centos:7.2" ;;
     _fedora) CONTAINER="${BASE}/fedora:29";;
     _focal) CONTAINER="${BASE}/ubuntu:focal" ;;
     _jessie) CONTAINER="${BASE}/debian:jessie" ;;
@@ -39,8 +38,10 @@ case "_${CONTAINER}" in
     _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;;
     _unstable|_) CONTAINER="${BASE}/debian:unstable" ;;
     _unstable-i386) CONTAINER="${BASE}/debian:unstable-i386" ;;
-    _unstable-arm32-gcc) CONTAINER="${BASE}/debian:unstable-arm32-gcc" ;;
+    _unstable-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:unstable-arm64v8-arm32-gcc" ;;
     _unstable-arm64v8) CONTAINER="${BASE}/debian:unstable-arm64v8" ;;
+    _unstable-cppcheck) CONTAINER="${BASE}/debian:unstable-cppcheck" ;;
+    _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;;
     _trusty) CONTAINER="${BASE}/ubuntu:trusty" ;;
     _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;;
     _opensuse-leap|_leap) CONTAINER="${BASE}/suse:opensuse-leap" ;;
-- 
2.25.1

Re: [PATCH] automation: Add missing and drop obsoleted aliases from containerize

Posted by Andrew Cooper 1 year, 1 month ago

On 01/03/2023 9:06 am, Michal Orzel wrote:
> Add missing aliases for:
>  - debian:unstable-cppcheck
>  - debian:unstable-arm64v8-arm32-gcc
>  - ubuntu:bionic
>
> Remove aliases for no longer used containers:
>  - centos:7.2
>  - debian:unstable-arm32-gcc
>
> Modify docs to refer to CentOS 7 instead of 7.2 not to create confusion.
>
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> ---
> Open questions related to the CI cleanup (@Andrew, @Anthony):
>  - Why do we keep suse:sles11sp4 dockerfile?
>  - Why do we keep jessie dockefiles?

Because we don't yet have a process for retirement of such things.

Right now, all branches other than staging are using the jessie
containers.  While it's still in use on any branch, we need some way to
rebuild the container (potentially with modifications - see the recent
HTTPS fun), and standard practice is "patches into staging".

An alternative could be to patch into the most recent staging branch
containing the dockerfile.

sles11sp4 is more complicated.  We specifically tried to get a SLES
container working, but it was always in a weird state (only @suse people
could rebuild the container).  We did eventually replace it with
OpenSUSE containers, but I can't currently locate any evidence in the
gitlab yml that we wired sles11sp4 up.   Which is confusing because I
swear we did have it running at some point in the past...

~Andrew

Re: [PATCH] automation: Add missing and drop obsoleted aliases from containerize

Posted by Michal Orzel 1 year, 1 month ago


On 03/03/2023 12:59, Andrew Cooper wrote:
> 
> 
> On 01/03/2023 9:06 am, Michal Orzel wrote:
>> Add missing aliases for:
>>  - debian:unstable-cppcheck
>>  - debian:unstable-arm64v8-arm32-gcc
>>  - ubuntu:bionic
>>
>> Remove aliases for no longer used containers:
>>  - centos:7.2
>>  - debian:unstable-arm32-gcc
>>
>> Modify docs to refer to CentOS 7 instead of 7.2 not to create confusion.
>>
>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>> ---
>> Open questions related to the CI cleanup (@Andrew, @Anthony):
>>  - Why do we keep suse:sles11sp4 dockerfile?
>>  - Why do we keep jessie dockefiles?
> 
> Because we don't yet have a process for retirement of such things.
> 
> Right now, all branches other than staging are using the jessie
> containers.  While it's still in use on any branch, we need some way to
> rebuild the container (potentially with modifications - see the recent
> HTTPS fun), and standard practice is "patches into staging".
> 
> An alternative could be to patch into the most recent staging branch
> containing the dockerfile.
> 
> 
> sles11sp4 is more complicated.  We specifically tried to get a SLES
> container working, but it was always in a weird state (only @suse people
> could rebuild the container).  We did eventually replace it with
> OpenSUSE containers, but I can't currently locate any evidence in the
> gitlab yml that we wired sles11sp4 up.   Which is confusing because I
> swear we did have it running at some point in the past...

Ok, thanks for answering. Stefano wanted me to remove these unused dockefiles.
Are you ok with that or only to remove sles for now?

~Michal

Re: [PATCH] automation: Add missing and drop obsoleted aliases from containerize

Posted by Andrew Cooper 1 year, 1 month ago

On 03/03/2023 12:09 pm, Michal Orzel wrote:
>
> On 03/03/2023 12:59, Andrew Cooper wrote:
>>
>> On 01/03/2023 9:06 am, Michal Orzel wrote:
>>> Add missing aliases for:
>>>  - debian:unstable-cppcheck
>>>  - debian:unstable-arm64v8-arm32-gcc
>>>  - ubuntu:bionic
>>>
>>> Remove aliases for no longer used containers:
>>>  - centos:7.2
>>>  - debian:unstable-arm32-gcc
>>>
>>> Modify docs to refer to CentOS 7 instead of 7.2 not to create confusion.
>>>
>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>>> ---
>>> Open questions related to the CI cleanup (@Andrew, @Anthony):
>>>  - Why do we keep suse:sles11sp4 dockerfile?
>>>  - Why do we keep jessie dockefiles?
>> Because we don't yet have a process for retirement of such things.
>>
>> Right now, all branches other than staging are using the jessie
>> containers.  While it's still in use on any branch, we need some way to
>> rebuild the container (potentially with modifications - see the recent
>> HTTPS fun), and standard practice is "patches into staging".
>>
>> An alternative could be to patch into the most recent staging branch
>> containing the dockerfile.
>>
>>
>> sles11sp4 is more complicated.  We specifically tried to get a SLES
>> container working, but it was always in a weird state (only @suse people
>> could rebuild the container).  We did eventually replace it with
>> OpenSUSE containers, but I can't currently locate any evidence in the
>> gitlab yml that we wired sles11sp4 up.   Which is confusing because I
>> swear we did have it running at some point in the past...
> Ok, thanks for answering. Stefano wanted me to remove these unused dockefiles.
> Are you ok with that or only to remove sles for now?

So https://gitlab.com/xen-project/xen/-/pipelines/791687536 is the
latest pipeline on Xen 4.14  (yeah, that's a concerning amount of
red...) but I don't see any SLES runs and I do see the OpenSUSE runs.

So I think we can safely drop the sles dockerfile, and drop the tags.

But I'd like to keep jessie around until we have figured out what our
deprecation process is.

~Andrew

Re: [PATCH] automation: Add missing and drop obsoleted aliases from containerize

Posted by Michal Orzel 1 year, 1 month ago


On 03/03/2023 13:13, Andrew Cooper wrote:
> 
> 
> On 03/03/2023 12:09 pm, Michal Orzel wrote:
>>
>> On 03/03/2023 12:59, Andrew Cooper wrote:
>>>
>>> On 01/03/2023 9:06 am, Michal Orzel wrote:
>>>> Add missing aliases for:
>>>>  - debian:unstable-cppcheck
>>>>  - debian:unstable-arm64v8-arm32-gcc
>>>>  - ubuntu:bionic
>>>>
>>>> Remove aliases for no longer used containers:
>>>>  - centos:7.2
>>>>  - debian:unstable-arm32-gcc
>>>>
>>>> Modify docs to refer to CentOS 7 instead of 7.2 not to create confusion.
>>>>
>>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>>>> ---
>>>> Open questions related to the CI cleanup (@Andrew, @Anthony):
>>>>  - Why do we keep suse:sles11sp4 dockerfile?
>>>>  - Why do we keep jessie dockefiles?
>>> Because we don't yet have a process for retirement of such things.
>>>
>>> Right now, all branches other than staging are using the jessie
>>> containers.  While it's still in use on any branch, we need some way to
>>> rebuild the container (potentially with modifications - see the recent
>>> HTTPS fun), and standard practice is "patches into staging".
>>>
>>> An alternative could be to patch into the most recent staging branch
>>> containing the dockerfile.
>>>
>>>
>>> sles11sp4 is more complicated.  We specifically tried to get a SLES
>>> container working, but it was always in a weird state (only @suse people
>>> could rebuild the container).  We did eventually replace it with
>>> OpenSUSE containers, but I can't currently locate any evidence in the
>>> gitlab yml that we wired sles11sp4 up.   Which is confusing because I
>>> swear we did have it running at some point in the past...
>> Ok, thanks for answering. Stefano wanted me to remove these unused dockefiles.
>> Are you ok with that or only to remove sles for now?
> 
> So https://gitlab.com/xen-project/xen/-/pipelines/791687536 is the
> latest pipeline on Xen 4.14  (yeah, that's a concerning amount of
> red...) but I don't see any SLES runs and I do see the OpenSUSE runs.
> 
> So I think we can safely drop the sles dockerfile, and drop the tags.
Ok. FWICS, there is only a dockefile for sles. No jobs, no tags, so just
dockerfile to remove.

> 
> But I'd like to keep jessie around until we have figured out what our
> deprecation process is.
I'm ok with that and I think that your alternative approach is a good way to go.
That said, I think we could still remove jessie aliases from containerize, while
keeping the dockerfiles for now until deprecation process is ready.

~Michal

Re: [PATCH] automation: Add missing and drop obsoleted aliases from containerize

Posted by Michal Orzel 1 year, 1 month ago


On 03/03/2023 13:21, Michal Orzel wrote:
> 
> 
> On 03/03/2023 13:13, Andrew Cooper wrote:
>>
>>
>> On 03/03/2023 12:09 pm, Michal Orzel wrote:
>>>
>>> On 03/03/2023 12:59, Andrew Cooper wrote:
>>>>
>>>> On 01/03/2023 9:06 am, Michal Orzel wrote:
>>>>> Add missing aliases for:
>>>>>  - debian:unstable-cppcheck
>>>>>  - debian:unstable-arm64v8-arm32-gcc
>>>>>  - ubuntu:bionic
>>>>>
>>>>> Remove aliases for no longer used containers:
>>>>>  - centos:7.2
>>>>>  - debian:unstable-arm32-gcc
>>>>>
>>>>> Modify docs to refer to CentOS 7 instead of 7.2 not to create confusion.
>>>>>
>>>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>>>>> ---
>>>>> Open questions related to the CI cleanup (@Andrew, @Anthony):
>>>>>  - Why do we keep suse:sles11sp4 dockerfile?
>>>>>  - Why do we keep jessie dockefiles?
>>>> Because we don't yet have a process for retirement of such things.
>>>>
>>>> Right now, all branches other than staging are using the jessie
>>>> containers.  While it's still in use on any branch, we need some way to
>>>> rebuild the container (potentially with modifications - see the recent
>>>> HTTPS fun), and standard practice is "patches into staging".
>>>>
>>>> An alternative could be to patch into the most recent staging branch
>>>> containing the dockerfile.
>>>>
>>>>
>>>> sles11sp4 is more complicated.  We specifically tried to get a SLES
>>>> container working, but it was always in a weird state (only @suse people
>>>> could rebuild the container).  We did eventually replace it with
>>>> OpenSUSE containers, but I can't currently locate any evidence in the
>>>> gitlab yml that we wired sles11sp4 up.   Which is confusing because I
>>>> swear we did have it running at some point in the past...
>>> Ok, thanks for answering. Stefano wanted me to remove these unused dockefiles.
>>> Are you ok with that or only to remove sles for now?
>>
>> So https://gitlab.com/xen-project/xen/-/pipelines/791687536 is the
>> latest pipeline on Xen 4.14  (yeah, that's a concerning amount of
>> red...) but I don't see any SLES runs and I do see the OpenSUSE runs.
>>
>> So I think we can safely drop the sles dockerfile, and drop the tags.
> Ok. FWICS, there is only a dockefile for sles. No jobs, no tags, so just
> dockerfile to remove.
Now I realized that by tags you meant removing it from registry as well.

> 
>>
>> But I'd like to keep jessie around until we have figured out what our
>> deprecation process is.
> I'm ok with that and I think that your alternative approach is a good way to go.
> That said, I think we could still remove jessie aliases from containerize, while
> keeping the dockerfiles for now until deprecation process is ready.
+ I think we can remove jessie aliases together with dockerfiles once process is ready.

> 
> ~Michal
> 

~Michal

Re: [PATCH] automation: Add missing and drop obsoleted aliases from containerize

Posted by Stefano Stabellini 1 year, 1 month ago

On Wed, 1 Mar 2023, Michal Orzel wrote:
> Add missing aliases for:
>  - debian:unstable-cppcheck
>  - debian:unstable-arm64v8-arm32-gcc
>  - ubuntu:bionic
> 
> Remove aliases for no longer used containers:
>  - centos:7.2
>  - debian:unstable-arm32-gcc
> 
> Modify docs to refer to CentOS 7 instead of 7.2 not to create confusion.
> 
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Open questions related to the CI cleanup (@Andrew, @Anthony):
>  - Why do we keep suse:sles11sp4 dockerfile?

please remove, it EOLed in 2016


>  - Why do we keep jessie dockefiles?

please remove, it EOLed in 2020


> ---
>  automation/build/README.md      | 4 ++--
>  automation/scripts/containerize | 5 +++--
>  2 files changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/automation/build/README.md b/automation/build/README.md
> index 4cc1acb6b41c..2d07cafe0eaa 100644
> --- a/automation/build/README.md
> +++ b/automation/build/README.md
> @@ -44,10 +44,10 @@ understands.
>    DOCKER_CMD=podman ./automation/scripts/containerize make
>    ```
>  
> -- CONTAINER: This overrides the container to use. For CentOS 7.2, use:
> +- CONTAINER: This overrides the container to use. For CentOS 7, use:
>  
>    ```
> -  CONTAINER=centos72 ./automation/scripts/containerize make
> +  CONTAINER=centos7 ./automation/scripts/containerize make
>    ```
>  
>  - CONTAINER_PATH: This overrides the path that will be available under the
> diff --git a/automation/scripts/containerize b/automation/scripts/containerize
> index 9b1a302d0565..5476ff0ea10d 100755
> --- a/automation/scripts/containerize
> +++ b/automation/scripts/containerize
> @@ -29,7 +29,6 @@ case "_${CONTAINER}" in
>      _archlinux|_arch) CONTAINER="${BASE}/archlinux:current" ;;
>      _riscv64) CONTAINER="${BASE}/archlinux:current-riscv64" ;;
>      _centos7) CONTAINER="${BASE}/centos:7" ;;
> -    _centos72) CONTAINER="${BASE}/centos:7.2" ;;
>      _fedora) CONTAINER="${BASE}/fedora:29";;
>      _focal) CONTAINER="${BASE}/ubuntu:focal" ;;
>      _jessie) CONTAINER="${BASE}/debian:jessie" ;;
> @@ -39,8 +38,10 @@ case "_${CONTAINER}" in
>      _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;;
>      _unstable|_) CONTAINER="${BASE}/debian:unstable" ;;
>      _unstable-i386) CONTAINER="${BASE}/debian:unstable-i386" ;;
> -    _unstable-arm32-gcc) CONTAINER="${BASE}/debian:unstable-arm32-gcc" ;;
> +    _unstable-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:unstable-arm64v8-arm32-gcc" ;;
>      _unstable-arm64v8) CONTAINER="${BASE}/debian:unstable-arm64v8" ;;
> +    _unstable-cppcheck) CONTAINER="${BASE}/debian:unstable-cppcheck" ;;
> +    _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;;
>      _trusty) CONTAINER="${BASE}/ubuntu:trusty" ;;
>      _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;;
>      _opensuse-leap|_leap) CONTAINER="${BASE}/suse:opensuse-leap" ;;
> -- 
> 2.25.1
>

Re: [PATCH] automation: Add missing and drop obsoleted aliases from containerize

Posted by Michal Orzel 1 year, 1 month ago

On 02/03/2023 01:23, Stefano Stabellini wrote:
> 
> 
> On Wed, 1 Mar 2023, Michal Orzel wrote:
>> Add missing aliases for:
>>  - debian:unstable-cppcheck
>>  - debian:unstable-arm64v8-arm32-gcc
>>  - ubuntu:bionic
>>
>> Remove aliases for no longer used containers:
>>  - centos:7.2
>>  - debian:unstable-arm32-gcc
>>
>> Modify docs to refer to CentOS 7 instead of 7.2 not to create confusion.
>>
>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> 
> Acked-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> 
>> ---
>> Open questions related to the CI cleanup (@Andrew, @Anthony):
>>  - Why do we keep suse:sles11sp4 dockerfile?
> 
> please remove, it EOLed in 2016
ok will do

> 
> 
>>  - Why do we keep jessie dockefiles?
> 
> please remove, it EOLed in 2020
For this I would like to check with Anthony and Andrew \wrt recent patches
that first modified the Jessie containers to use EOL tag and then removed Jessie related jobs.
I am confused why we keep dockerfiles in git tree while not having any jobs for them.

Anthony, Andrew?

~Michal