The current s3cp implementation does not work anymore after the
migration, and instead of fixing it and propagating the fix down to us,
it's simpler to directly use curl. Uprev mesa [1][2] to adapt these
changes. Also replace broken s3cp command with a curl wrapper call in
drm-ci.
[1] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34120
[2] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34244
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
---
drivers/gpu/drm/ci/build-igt.sh | 2 +-
drivers/gpu/drm/ci/build.sh | 6 +++---
drivers/gpu/drm/ci/gitlab-ci.yml | 6 ++++--
drivers/gpu/drm/ci/image-tags.yml | 2 +-
drivers/gpu/drm/ci/lava-submit.sh | 2 +-
5 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/ci/build-igt.sh b/drivers/gpu/drm/ci/build-igt.sh
index eddb5f782a5e..caa2f4804ed5 100644
--- a/drivers/gpu/drm/ci/build-igt.sh
+++ b/drivers/gpu/drm/ci/build-igt.sh
@@ -71,4 +71,4 @@ tar -cf artifacts/igt.tar /igt
# Pass needed files to the test stage
S3_ARTIFACT_NAME="igt.tar.gz"
gzip -c artifacts/igt.tar > ${S3_ARTIFACT_NAME}
-ci-fairy s3cp --token-file "${S3_JWT_FILE}" ${S3_ARTIFACT_NAME} https://${PIPELINE_ARTIFACTS_BASE}/${KERNEL_ARCH}/${S3_ARTIFACT_NAME}
+s3_upload ${S3_ARTIFACT_NAME} https://${PIPELINE_ARTIFACTS_BASE}/${KERNEL_ARCH}/
diff --git a/drivers/gpu/drm/ci/build.sh b/drivers/gpu/drm/ci/build.sh
index 284873e94d8d..6fb74c51abe2 100644
--- a/drivers/gpu/drm/ci/build.sh
+++ b/drivers/gpu/drm/ci/build.sh
@@ -148,13 +148,13 @@ if [[ "$UPLOAD_TO_MINIO" = "1" ]]; then
ls -l "${S3_JWT_FILE}"
for f in $FILES_TO_UPLOAD; do
- ci-fairy s3cp --token-file "${S3_JWT_FILE}" /kernel/$f \
- https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}/$f
+ s3_upload /kernel/$f \
+ https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}/
done
S3_ARTIFACT_NAME="kernel-files.tar.zst"
tar --zstd -cf $S3_ARTIFACT_NAME install
- ci-fairy s3cp --token-file "${S3_JWT_FILE}" ${S3_ARTIFACT_NAME} https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}/${S3_ARTIFACT_NAME}
+ s3_upload ${S3_ARTIFACT_NAME} https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}/
echo "Download vmlinux.xz from https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}/vmlinux.xz"
fi
diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
index 6850ce99a673..e54e382bf5fb 100644
--- a/drivers/gpu/drm/ci/gitlab-ci.yml
+++ b/drivers/gpu/drm/ci/gitlab-ci.yml
@@ -1,6 +1,6 @@
variables:
DRM_CI_PROJECT_PATH: &drm-ci-project-path mesa/mesa
- DRM_CI_COMMIT_SHA: &drm-ci-commit-sha 82ab58f6c6f94fa80ca7e1615146f08356e3ba69
+ DRM_CI_COMMIT_SHA: &drm-ci-commit-sha f73132f1215a37ce8ffc711a0136c90649aaf128
UPSTREAM_REPO: https://gitlab.freedesktop.org/drm/kernel.git
TARGET_BRANCH: drm-next
@@ -20,8 +20,10 @@ variables:
rm download-git-cache.sh
set +o xtrace
S3_JWT_FILE: /s3_jwt
+ S3_JWT_HEADER_FILE: /s3_jwt_header
S3_JWT_FILE_SCRIPT: |-
echo -n '${S3_JWT}' > '${S3_JWT_FILE}' &&
+ echo -n "Authorization: Bearer ${S3_JWT}" > '${S3_JWT_HEADER_FILE}' &&
unset CI_JOB_JWT S3_JWT # Unsetting vulnerable env variables
S3_HOST: s3.freedesktop.org
# This bucket is used to fetch the kernel image
@@ -253,7 +255,7 @@ make git archive:
- tar -cvzf ../$CI_PROJECT_NAME.tar.gz .
# Use id_tokens for JWT auth
- - ci-fairy s3cp --token-file "${S3_JWT_FILE}" ../$CI_PROJECT_NAME.tar.gz https://$S3_HOST/${S3_GITCACHE_BUCKET}/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/$CI_PROJECT_NAME.tar.gz
+ - s3_upload ../$CI_PROJECT_NAME.tar.gz https://$S3_HOST/${S3_GITCACHE_BUCKET}/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/
# Sanity checks of MR settings and commit logs
diff --git a/drivers/gpu/drm/ci/image-tags.yml b/drivers/gpu/drm/ci/image-tags.yml
index c04ba0e69935..53fe34b86578 100644
--- a/drivers/gpu/drm/ci/image-tags.yml
+++ b/drivers/gpu/drm/ci/image-tags.yml
@@ -1,5 +1,5 @@
variables:
- CONTAINER_TAG: "20250307-mesa-uprev"
+ CONTAINER_TAG: "20250328-mesa-uprev"
DEBIAN_X86_64_BUILD_BASE_IMAGE: "debian/x86_64_build-base"
DEBIAN_BASE_TAG: "${CONTAINER_TAG}"
diff --git a/drivers/gpu/drm/ci/lava-submit.sh b/drivers/gpu/drm/ci/lava-submit.sh
index f22720359b33..a1e8b34fb2d4 100755
--- a/drivers/gpu/drm/ci/lava-submit.sh
+++ b/drivers/gpu/drm/ci/lava-submit.sh
@@ -54,7 +54,7 @@ cp artifacts/ci-common/init-*.sh results/job-rootfs-overlay/
cp "$SCRIPTS_DIR"/setup-test-env.sh results/job-rootfs-overlay/
tar zcf job-rootfs-overlay.tar.gz -C results/job-rootfs-overlay/ .
-ci-fairy s3cp --token-file "${S3_JWT_FILE}" job-rootfs-overlay.tar.gz "https://${JOB_ROOTFS_OVERLAY_PATH}"
+s3_upload job-rootfs-overlay.tar.gz "https://${JOB_ARTIFACTS_BASE}"
# Prepare env vars for upload.
section_switch variables "Environment variables passed through to device:"
--
2.47.2Hi Vignesh, On Fri, 28 Mar 2025 at 11:03, Vignesh Raman <vignesh.raman@collabora.com> wrote: > The current s3cp implementation does not work anymore after the > migration, and instead of fixing it and propagating the fix down to us, > it's simpler to directly use curl. Uprev mesa [1][2] to adapt these > changes. Also replace broken s3cp command with a curl wrapper call in > drm-ci. Thanks a lot for fixing this. Sorry the fallout has been so bad. You can also upgrade ci-templates to get an s3cp which works again. Cheers, Daniel
Hi Daniel, On 28/03/25 17:06, Daniel Stone wrote: > Hi Vignesh, > > On Fri, 28 Mar 2025 at 11:03, Vignesh Raman <vignesh.raman@collabora.com> wrote: >> The current s3cp implementation does not work anymore after the >> migration, and instead of fixing it and propagating the fix down to us, >> it's simpler to directly use curl. Uprev mesa [1][2] to adapt these >> changes. Also replace broken s3cp command with a curl wrapper call in >> drm-ci. > > Thanks a lot for fixing this. Sorry the fallout has been so bad. > > You can also upgrade ci-templates to get an s3cp which works again. Thanks for fixing this. Will use the latest ci-templates and test it. Regards, Vignesh > > Cheers, > Daniel
Hi Daniel, On 28/03/25 17:29, Vignesh Raman wrote: > Hi Daniel, > > On 28/03/25 17:06, Daniel Stone wrote: >> Hi Vignesh, >> >> On Fri, 28 Mar 2025 at 11:03, Vignesh Raman >> <vignesh.raman@collabora.com> wrote: >>> The current s3cp implementation does not work anymore after the >>> migration, and instead of fixing it and propagating the fix down to us, >>> it's simpler to directly use curl. Uprev mesa [1][2] to adapt these >>> changes. Also replace broken s3cp command with a curl wrapper call in >>> drm-ci. >> >> Thanks a lot for fixing this. Sorry the fallout has been so bad. >> >> You can also upgrade ci-templates to get an s3cp which works again. > > Thanks for fixing this. Will use the latest ci-templates and test it. We need to update mesa to use the latest ci-templates and then uprev mesa in drm-ci. I will send this in a separate series after fixing it in mesa. Regards, Vignesh > > Regards, > Vignesh > >> >> Cheers, >> Daniel >
On Thu, Apr 03, 2025 at 11:29:55AM +0530, Vignesh Raman wrote: > Hi Daniel, > > On 28/03/25 17:29, Vignesh Raman wrote: > > Hi Daniel, > > > > On 28/03/25 17:06, Daniel Stone wrote: > > > Hi Vignesh, > > > > > > On Fri, 28 Mar 2025 at 11:03, Vignesh Raman > > > <vignesh.raman@collabora.com> wrote: > > > > The current s3cp implementation does not work anymore after the > > > > migration, and instead of fixing it and propagating the fix down to us, > > > > it's simpler to directly use curl. Uprev mesa [1][2] to adapt these > > > > changes. Also replace broken s3cp command with a curl wrapper call in > > > > drm-ci. > > > > > > Thanks a lot for fixing this. Sorry the fallout has been so bad. > > > > > > You can also upgrade ci-templates to get an s3cp which works again. > > > > Thanks for fixing this. Will use the latest ci-templates and test it. > > We need to update mesa to use the latest ci-templates and then uprev mesa in > drm-ci. I will send this in a separate series after fixing it in mesa. Vignesh, Daniel, any updates on this? Currently drm/ci is broken both in master and in drm-misc. -- With best wishes Dmitry
Hi Dmitry, On 29/04/25 23:51, Dmitry Baryshkov wrote: > On Thu, Apr 03, 2025 at 11:29:55AM +0530, Vignesh Raman wrote: >> Hi Daniel, >> >> On 28/03/25 17:29, Vignesh Raman wrote: >>> Hi Daniel, >>> >>> On 28/03/25 17:06, Daniel Stone wrote: >>>> Hi Vignesh, >>>> >>>> On Fri, 28 Mar 2025 at 11:03, Vignesh Raman >>>> <vignesh.raman@collabora.com> wrote: >>>>> The current s3cp implementation does not work anymore after the >>>>> migration, and instead of fixing it and propagating the fix down to us, >>>>> it's simpler to directly use curl. Uprev mesa [1][2] to adapt these >>>>> changes. Also replace broken s3cp command with a curl wrapper call in >>>>> drm-ci. >>>> >>>> Thanks a lot for fixing this. Sorry the fallout has been so bad. >>>> >>>> You can also upgrade ci-templates to get an s3cp which works again. >>> >>> Thanks for fixing this. Will use the latest ci-templates and test it. >> >> We need to update mesa to use the latest ci-templates and then uprev mesa in >> drm-ci. I will send this in a separate series after fixing it in mesa. > > Vignesh, Daniel, any updates on this? Currently drm/ci is broken both in > master and in drm-misc. > mesa-ci is updated to use s3cp and drop the s3_upload, which is a curl wrapper. These patches were merged this week and I will uprev mesa in drm-ci to get s3cp working. I will send update to this series. Regards, Vignesh
© 2016 - 2025 Red Hat, Inc.