The Argo work already moved this artefact into the new form.  Reuse that, and
drop one test job.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Victor Lira <victorm.lira@amd.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1522281815
---
 automation/gitlab-ci/build.yaml | 11 -----------
 automation/gitlab-ci/test.yaml  |  4 +++-
 2 files changed, 3 insertions(+), 12 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index af2b1ceba3a4..b5ca2c3f1833 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -304,17 +304,6 @@ qemu-system-aarch64-6.0.0-arm32-export:
 
 # x86_64 test artifacts
 
-alpine-3.18-rootfs-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.18
-  script:
-    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
-  artifacts:
-    paths:
-      - binaries/initrd.tar.gz
-  tags:
-    - x86_64
-
 kernel-6.1.19-export:
   extends: .test-jobs-artifact-common
   image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index e8f57e87bd19..fe023dae0d42 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -11,8 +11,10 @@
   - qemu-system-aarch64-6.0.0-arm32-export
 
 .x86-64-test-needs: &x86-64-test-needs
-  - alpine-3.18-rootfs-export
   - kernel-6.1.19-export
+  - project: xen-project/hardware/test-artifacts
+    job: x86_64-rootfs-alpine-3.18
+    ref: master
 
 .qemu-arm64:
   extends: .test-jobs-common
-- 
2.39.5

On Thu, 31 Oct 2024, Andrew Cooper wrote:
> The Argo work already moved this artefact into the new form.  Reuse that, and
> drop one test job.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

This is good but should we also remove
automation/tests-artifacts/alpine/3.18.dockerfile since we are at it?


> ---
> CC: Anthony PERARD <anthony.perard@vates.tech>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Doug Goldstein <cardoe@cardoe.com>
> CC: Victor Lira <victorm.lira@amd.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> 
> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1522281815
> ---
>  automation/gitlab-ci/build.yaml | 11 -----------
>  automation/gitlab-ci/test.yaml  |  4 +++-
>  2 files changed, 3 insertions(+), 12 deletions(-)
> 
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index af2b1ceba3a4..b5ca2c3f1833 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -304,17 +304,6 @@ qemu-system-aarch64-6.0.0-arm32-export:
>  
>  # x86_64 test artifacts
>  
> -alpine-3.18-rootfs-export:
> -  extends: .test-jobs-artifact-common
> -  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.18
> -  script:
> -    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
> -  artifacts:
> -    paths:
> -      - binaries/initrd.tar.gz
> -  tags:
> -    - x86_64
> -
>  kernel-6.1.19-export:
>    extends: .test-jobs-artifact-common
>    image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index e8f57e87bd19..fe023dae0d42 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -11,8 +11,10 @@
>    - qemu-system-aarch64-6.0.0-arm32-export
>  
>  .x86-64-test-needs: &x86-64-test-needs
> -  - alpine-3.18-rootfs-export
>    - kernel-6.1.19-export
> +  - project: xen-project/hardware/test-artifacts
> +    job: x86_64-rootfs-alpine-3.18
> +    ref: master
>  
>  .qemu-arm64:
>    extends: .test-jobs-common
> -- 
> 2.39.5
> 

On 31/10/2024 10:46 pm, Stefano Stabellini wrote:
> On Thu, 31 Oct 2024, Andrew Cooper wrote:
>> The Argo work already moved this artefact into the new form.  Reuse that, and
>> drop one test job.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> This is good but should we also remove
> automation/tests-artifacts/alpine/3.18.dockerfile since we are at it?

Well, that's another thing that needs careful consideration.

That dockerfile needs updating in tandem with the build container (or
lib$FOO.so's don't work when running the test), and it's hard enough to
keep track of things when they're all in one repo.

~Andrew

On Thu, 31 Oct 2024, Andrew Cooper wrote:
> On 31/10/2024 10:46 pm, Stefano Stabellini wrote:
> > On Thu, 31 Oct 2024, Andrew Cooper wrote:
> >> The Argo work already moved this artefact into the new form.  Reuse that, and
> >> drop one test job.
> >>
> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > This is good but should we also remove
> > automation/tests-artifacts/alpine/3.18.dockerfile since we are at it?
> 
> Well, that's another thing that needs careful consideration.
> 
> That dockerfile needs updating in tandem with the build container (or
> lib$FOO.so's don't work when running the test), and it's hard enough to
> keep track of things when they're all in one repo.

Uhm, you have a good point. Two things come to mind. First, for this
patch:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


Second, I think maybe it would be better for test-artifacts to use the
build containers from xen.git/automation/build ? So that at least all
build containers come from the same place?

For instance, we would have to add any missing dependencies to
automation/build/alpine/3.18.dockerfile, from the list currently in
images/alpine/x86_64-build.dockerfile. Only a couple of things are
missing. Then remove images/alpine/x86_64-build.dockerfile, and use
registry.gitlab.com/xen-project/xen/alpine:3.18 in the test-artifacts
build jobs?

On 31/10/2024 11:05 pm, Stefano Stabellini wrote:
> On Thu, 31 Oct 2024, Andrew Cooper wrote:
>> On 31/10/2024 10:46 pm, Stefano Stabellini wrote:
>>> On Thu, 31 Oct 2024, Andrew Cooper wrote:
>>>> The Argo work already moved this artefact into the new form.  Reuse that, and
>>>> drop one test job.
>>>>
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> This is good but should we also remove
>>> automation/tests-artifacts/alpine/3.18.dockerfile since we are at it?
>> Well, that's another thing that needs careful consideration.
>>
>> That dockerfile needs updating in tandem with the build container (or
>> lib$FOO.so's don't work when running the test), and it's hard enough to
>> keep track of things when they're all in one repo.
> Uhm, you have a good point. Two things come to mind. First, for this
> patch:
>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Thanks.

> Second, I think maybe it would be better for test-artifacts to use the
> build containers from xen.git/automation/build ? So that at least all
> build containers come from the same place?
>
> For instance, we would have to add any missing dependencies to
> automation/build/alpine/3.18.dockerfile, from the list currently in
> images/alpine/x86_64-build.dockerfile. Only a couple of things are
> missing. Then remove images/alpine/x86_64-build.dockerfile, and use
> registry.gitlab.com/xen-project/xen/alpine:3.18 in the test-artifacts
> build jobs?

First, I think it is important that hw/t-a uses main Xen's containers,
rather than its own.  That way, all the dockerfiles are in one place.


As for merging the build and test-artefact dockerfile, I'm not so sure. 
They're almost completely dissimilar, and (amongst other things) we
don't want to be including two full toolchains in the test initrd.

So, I think they do need to remain separate, but the build/test-artefact
split is weird already because build/ is both build and some test, but
not all.  I think things would benefit from relaying out automation/
somewhat.

~Andrew

On Thu, 31 Oct 2024, Andrew Cooper wrote:
> On 31/10/2024 10:46 pm, Stefano Stabellini wrote:
> > On Thu, 31 Oct 2024, Andrew Cooper wrote:
> >> The Argo work already moved this artefact into the new form.  Reuse that, and
> >> drop one test job.
> >>
> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > This is good but should we also remove
> > automation/tests-artifacts/alpine/3.18.dockerfile since we are at it?
> 
> Well, that's another thing that needs careful consideration.
> 
> That dockerfile needs updating in tandem with the build container (or
> lib$FOO.so's don't work when running the test), and it's hard enough to
> keep track of things when they're all in one repo.

Uhm, you have a good point. Two things come to mind. First, for this
patch:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


Second, I think maybe it would be better for test-artifacts to use the
build containers from xen.git/automation/build ? So that at least all
build containers come from the same place?

For instance, we would have to add any missing dependencies to
automation/build/alpine/3.18.dockerfile, from the list currently in
images/alpine/x86_64-build.dockerfile. Only a couple of things are
missing. Then remove images/alpine/x86_64-build.dockerfile, and use
registry.gitlab.com/xen-project/xen/alpine:3.18 in the test-artifacts
build jobs?