[PATCH v2 0/3] fix build failures from incorrectly skipped container build jobs

Daniel P. Berrangé posted 3 patches 3 years, 2 months ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20210216132954.295906-1-berrange@redhat.com
.gitlab-ci.d/containers.yml  |  7 ----
.gitlab-ci.d/crossbuilds.yml | 46 ++++++++++++++++++++++
.gitlab-ci.yml               | 74 ++++++++++++++++++++++++++++++++----
3 files changed, 112 insertions(+), 15 deletions(-)
[PATCH v2 0/3] fix build failures from incorrectly skipped container build jobs
Posted by Daniel P. Berrangé 3 years, 2 months ago
This series fixes a problem with our gitlab CI rules that cause
container builds to be skipped. See the commit description in the
first patch for the details on this problem.

The overall result of this series though is a small increase in overall
pipeline time.

Previously

 - When container jobs are skipped: approx 1hr 5 mins
 - When container jobs are run, cached by docker: approx 1hr 15 mins
 - When container jobs are run, not cached by docker: approx 1hr 30 mins

With this series applied the first scenario no longer exists, so
all piplines are either 1hr 15 or 1hr 30 depending on whether the
container phase is skipped.

On the plus side the builds are more reliable as we're actually
building container images at correct times.

There is still a race condition though where build jobs can run
with the wrong containers. This happens if you push two different
branches to gitlab with different docker file content. If the
container jobs for the 2nd branch finish before the 1st
branch runs its build jobs, the 1st branch can end up using
containers fro the second branch.  The only fix to truely fix
that would be to stop using "latest" docker tag and always
use a tag based on the branch name. This would mean we build
up a growing set of docker images in the gitlab registry.

At least this series is much more correct that what exists in
git currently. I'm intending to continue to investigate a solution
for the remaining race condition, but don't want to block this
series from merging, since this already solves the problem for the
majority of contributors' usage.

Changed in v2:

 - Set needs rules for cross jobs too

Daniel P. Berrangé (3):
  gitlab: always build container images
  gitlab: add fine grained job deps for all build jobs
  gitlab: fix inconsistent indentation

 .gitlab-ci.d/containers.yml  |  7 ----
 .gitlab-ci.d/crossbuilds.yml | 46 ++++++++++++++++++++++
 .gitlab-ci.yml               | 74 ++++++++++++++++++++++++++++++++----
 3 files changed, 112 insertions(+), 15 deletions(-)

-- 
2.29.2