[PATCH 0/3] ci: use cirrus-run to utilize Cirrus CI from GitLab CI

Daniel P. Berrangé posted 3 patches 2 years, 10 months ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20210625172211.451010-1-berrange@redhat.com
.cirrus.yml                         |  55 ---------------
.gitlab-ci.d/cirrus.yml             | 103 ++++++++++++++++++++++++++++
.gitlab-ci.d/cirrus/README.rst      |  54 +++++++++++++++
.gitlab-ci.d/cirrus/build.yml       |  35 ++++++++++
.gitlab-ci.d/cirrus/freebsd-12.vars |  13 ++++
.gitlab-ci.d/cirrus/freebsd-13.vars |  13 ++++
.gitlab-ci.d/cirrus/macos-11.vars   |  15 ++++
.gitlab-ci.d/qemu-project.yml       |   1 +
meson.build                         |  13 ++++
9 files changed, 247 insertions(+), 55 deletions(-)
create mode 100644 .gitlab-ci.d/cirrus.yml
create mode 100644 .gitlab-ci.d/cirrus/README.rst
create mode 100644 .gitlab-ci.d/cirrus/build.yml
create mode 100644 .gitlab-ci.d/cirrus/freebsd-12.vars
create mode 100644 .gitlab-ci.d/cirrus/freebsd-13.vars
create mode 100644 .gitlab-ci.d/cirrus/macos-11.vars
[PATCH 0/3] ci: use cirrus-run to utilize Cirrus CI from GitLab CI
Posted by Daniel P. Berrangé 2 years, 10 months ago
Currently the Cirrus CI pipelines are completely separate from the
GitLab CI pipelines. This means contributors/maintainers have to
monitor two distinct places.

This series uses the 'cirrus-run' tool from within a GitLab CI job
to invoke a Cirrus CI job. Effectively Cirrus CI becomes a custom
runner for GitLab in this way.

The GitLab CI pipeline nows shows a complete picture for CI results
covering GitLab, Travis and Cirrus jobs.

The implementation used here is a direct copy of what is done in
libvirt with cirrus-run. In fact the jobs are using the container
image published by https://gitlab.com/libvirt/libvirt-ci for
getting the 'cirrus-run' tool.

Second, we use the 'lcitool' program to generate the
.gitlab-ci.d/cirrus/*.vars files which hold the package lists to
be installed on FreeBSD and macOS. As a result of the work done
with lcitool to generate the dockerfiles, this was easy to extend
to Cirrus CI, and as a result this new Cirrus config installs a
massively larger set of packages. Thus our testing coverage on
macOS and FreeBSD improves.

The MSys Windows job still remains in the .cirrus.yml file. This
can be addressed to, if we extend libvirt-ci to have package
mapping information for MSys.

Using this setup does require the contributor to do a one time
configuration task. They need to modify gitlab CI settings to
add two environment variables, providing the API token for the
Cirrus CI REST API.

I have a demo pipeline here showing the 4 cirrus CI jobs:

  https://gitlab.com/berrange/qemu/-/pipelines/327362404

Note in this pipeline above, I temporarily disabled all the
normal GitLab CI jobs to reduce burden when I was debugging.
An earlier pipeline shows the full set of jobs:

  https://gitlab.com/berrange/qemu/-/pipelines/327324780

This series adds cirrus as always on jobs, but we should probably
use 'allow_failure: true' for a few weeks to demonstrate that
they are stable enough to be part of the main gating set.

Daniel P. Berrangé (3):
  build: validate that system capstone works before using it
  gitlab: support for FreeBSD 12, 13 and macOS 11 via cirrus-run
  cirrus: delete FreeBSD and macOS jobs

 .cirrus.yml                         |  55 ---------------
 .gitlab-ci.d/cirrus.yml             | 103 ++++++++++++++++++++++++++++
 .gitlab-ci.d/cirrus/README.rst      |  54 +++++++++++++++
 .gitlab-ci.d/cirrus/build.yml       |  35 ++++++++++
 .gitlab-ci.d/cirrus/freebsd-12.vars |  13 ++++
 .gitlab-ci.d/cirrus/freebsd-13.vars |  13 ++++
 .gitlab-ci.d/cirrus/macos-11.vars   |  15 ++++
 .gitlab-ci.d/qemu-project.yml       |   1 +
 meson.build                         |  13 ++++
 9 files changed, 247 insertions(+), 55 deletions(-)
 create mode 100644 .gitlab-ci.d/cirrus.yml
 create mode 100644 .gitlab-ci.d/cirrus/README.rst
 create mode 100644 .gitlab-ci.d/cirrus/build.yml
 create mode 100644 .gitlab-ci.d/cirrus/freebsd-12.vars
 create mode 100644 .gitlab-ci.d/cirrus/freebsd-13.vars
 create mode 100644 .gitlab-ci.d/cirrus/macos-11.vars

-- 
2.31.1



Re: [PATCH 0/3] ci: use cirrus-run to utilize Cirrus CI from GitLab CI
Posted by Alex Bennée 2 years, 10 months ago
Daniel P. Berrangé <berrange@redhat.com> writes:

> Currently the Cirrus CI pipelines are completely separate from the
> GitLab CI pipelines. This means contributors/maintainers have to
> monitor two distinct places.

Queued to testing/next, thanks.

-- 
Alex Bennée

Re: [PATCH 0/3] ci: use cirrus-run to utilize Cirrus CI from GitLab CI
Posted by Thomas Huth 2 years, 10 months ago
On 25/06/2021 19.22, Daniel P. Berrangé wrote:
[...]
> The MSys Windows job still remains in the .cirrus.yml file. This
> can be addressed to, if we extend libvirt-ci to have package
> mapping information for MSys.

I think gitlab-CI offers shared Windows runners, too, see e.g.:

  https://about.gitlab.com/blog/2020/01/21/windows-shared-runner-beta/

So I think we likely should rather convert that job to a shared gitlab-CI 
Windows runner instead?

  Thomas