1. Patchew
  2. QEMU
  3. [PATCH v6 0/4] GitLab Custom Runners and Jobs (was: QEMU Gating CI)
  4. View patch

[PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder

Cleber Rosa posted 4 patches 4 years, 8 months ago
Maintainers: Willian Rampazzo <willianr@redhat.com>, "Philippe Mathieu-Daudé" <f4bug@amsat.org>, "Alex Bennée" <alex.bennee@linaro.org>, Wainer dos Santos Moschetta <wainersm@redhat.com>, Thomas Huth <thuth@redhat.com>
There is a newer version of this series
[PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder
Posted by Cleber Rosa 4 years, 8 months ago 
As described in the included documentation, the "custom runner" jobs
extend the GitLab CI jobs already in place.  One of their primary
goals of catching and preventing regressions on a wider number of host
systems than the ones provided by GitLab's shared runners.

This sets the stage in which other community members can add their own
machine configuration documentation/scripts, and accompanying job
definitions.  As a general rule, those newly added contributed jobs
should run as "non-gating", until their reliability is verified (AKA
"allow_failure: true").

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 .gitlab-ci.d/custom-runners.yml | 14 ++++++++++++++
 .gitlab-ci.d/qemu-project.yml   |  1 +
 docs/devel/ci.rst               | 28 ++++++++++++++++++++++++++++
 docs/devel/index.rst            |  1 +
 4 files changed, 44 insertions(+)
 create mode 100644 .gitlab-ci.d/custom-runners.yml
 create mode 100644 docs/devel/ci.rst

diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
new file mode 100644
index 0000000000..a07b27384c
--- /dev/null
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -0,0 +1,14 @@
+# The CI jobs defined here require GitLab runners installed and
+# registered on machines that match their operating system names,
+# versions and architectures.  This is in contrast to the other CI
+# jobs that are intended to run on GitLab's "shared" runners.
+
+# Different than the default approach on "shared" runners, based on
+# containers, the custom runners have no such *requirement*, as those
+# jobs should be capable of running on operating systems with no
+# compatible container implementation, or no support from
+# gitlab-runner.  To avoid problems that gitlab-runner can cause while
+# reusing the GIT repository, let's enable the clone strategy, which
+# guarantees a fresh repository on each job run.
+variables:
+  GIT_STRATEGY: clone
diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
index 64cb2ba1da..dde8270301 100644
--- a/.gitlab-ci.d/qemu-project.yml
+++ b/.gitlab-ci.d/qemu-project.yml
@@ -9,3 +9,4 @@ include:
   - local: '/.gitlab-ci.d/crossbuilds.yml'
   - local: '/.gitlab-ci.d/buildtest.yml'
   - local: '/.gitlab-ci.d/static_checks.yml'
+  - local: '/.gitlab-ci.d/custom-runners.yml'
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
new file mode 100644
index 0000000000..585b7bf4b8
--- /dev/null
+++ b/docs/devel/ci.rst
@@ -0,0 +1,28 @@
+==
+CI
+==
+
+QEMU has configurations enabled for a number of different CI services.
+The most up to date information about them and their status can be
+found at::
+
+   https://wiki.qemu.org/Testing/CI
+
+Jobs on Custom Runners
+======================
+
+Besides the jobs run under the various CI systems listed before, there
+are a number additional jobs that will run before an actual merge.
+These use the same GitLab CI's service/framework already used for all
+other GitLab based CI jobs, but rely on additional systems, not the
+ones provided by GitLab as "shared runners".
+
+The architecture of GitLab's CI service allows different machines to
+be set up with GitLab's "agent", called gitlab-runner, which will take
+care of running jobs created by events such as a push to a branch.
+Here, the combination of a machine, properly configured with GitLab's
+gitlab-runner, is called a "custom runner".
+
+The GitLab CI jobs definition for the custom runners are located under::
+
+  .gitlab-ci.d/custom-runners.yml
diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index 791925dcda..c9a02e786e 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -27,6 +27,7 @@ Contents:
    migration
    atomics
    stable-process
+   ci
    qtest
    decodetree
    secure-coding-practices
-- 
2.25.4
Re: [PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder
Posted by Wainer dos Santos Moschetta 4 years, 8 months ago 
On 6/8/21 12:14 AM, Cleber Rosa wrote:
> As described in the included documentation, the "custom runner" jobs
> extend the GitLab CI jobs already in place.  One of their primary
> goals of catching and preventing regressions on a wider number of host
> systems than the ones provided by GitLab's shared runners.
>
> This sets the stage in which other community members can add their own
> machine configuration documentation/scripts, and accompanying job
> definitions.  As a general rule, those newly added contributed jobs
> should run as "non-gating", until their reliability is verified (AKA
> "allow_failure: true").
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   .gitlab-ci.d/custom-runners.yml | 14 ++++++++++++++
>   .gitlab-ci.d/qemu-project.yml   |  1 +
>   docs/devel/ci.rst               | 28 ++++++++++++++++++++++++++++
>   docs/devel/index.rst            |  1 +
>   4 files changed, 44 insertions(+)
>   create mode 100644 .gitlab-ci.d/custom-runners.yml
>   create mode 100644 docs/devel/ci.rst


Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


>
> diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> new file mode 100644
> index 0000000000..a07b27384c
> --- /dev/null
> +++ b/.gitlab-ci.d/custom-runners.yml
> @@ -0,0 +1,14 @@
> +# The CI jobs defined here require GitLab runners installed and
> +# registered on machines that match their operating system names,
> +# versions and architectures.  This is in contrast to the other CI
> +# jobs that are intended to run on GitLab's "shared" runners.
> +
> +# Different than the default approach on "shared" runners, based on
> +# containers, the custom runners have no such *requirement*, as those
> +# jobs should be capable of running on operating systems with no
> +# compatible container implementation, or no support from
> +# gitlab-runner.  To avoid problems that gitlab-runner can cause while
> +# reusing the GIT repository, let's enable the clone strategy, which
> +# guarantees a fresh repository on each job run.
> +variables:
> +  GIT_STRATEGY: clone
> diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
> index 64cb2ba1da..dde8270301 100644
> --- a/.gitlab-ci.d/qemu-project.yml
> +++ b/.gitlab-ci.d/qemu-project.yml
> @@ -9,3 +9,4 @@ include:
>     - local: '/.gitlab-ci.d/crossbuilds.yml'
>     - local: '/.gitlab-ci.d/buildtest.yml'
>     - local: '/.gitlab-ci.d/static_checks.yml'
> +  - local: '/.gitlab-ci.d/custom-runners.yml'
> diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
> new file mode 100644
> index 0000000000..585b7bf4b8
> --- /dev/null
> +++ b/docs/devel/ci.rst
> @@ -0,0 +1,28 @@
> +==
> +CI
> +==
> +
> +QEMU has configurations enabled for a number of different CI services.
> +The most up to date information about them and their status can be
> +found at::
> +
> +   https://wiki.qemu.org/Testing/CI
> +
> +Jobs on Custom Runners
> +======================
> +
> +Besides the jobs run under the various CI systems listed before, there
> +are a number additional jobs that will run before an actual merge.
> +These use the same GitLab CI's service/framework already used for all
> +other GitLab based CI jobs, but rely on additional systems, not the
> +ones provided by GitLab as "shared runners".
> +
> +The architecture of GitLab's CI service allows different machines to
> +be set up with GitLab's "agent", called gitlab-runner, which will take
> +care of running jobs created by events such as a push to a branch.
> +Here, the combination of a machine, properly configured with GitLab's
> +gitlab-runner, is called a "custom runner".
> +
> +The GitLab CI jobs definition for the custom runners are located under::
> +
> +  .gitlab-ci.d/custom-runners.yml
> diff --git a/docs/devel/index.rst b/docs/devel/index.rst
> index 791925dcda..c9a02e786e 100644
> --- a/docs/devel/index.rst
> +++ b/docs/devel/index.rst
> @@ -27,6 +27,7 @@ Contents:
>      migration
>      atomics
>      stable-process
> +   ci
>      qtest
>      decodetree
>      secure-coding-practices
Re: [PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder
Posted by Alex Bennée 4 years, 8 months ago 
Cleber Rosa <crosa@redhat.com> writes:

> As described in the included documentation, the "custom runner" jobs
> extend the GitLab CI jobs already in place.  One of their primary
> goals of catching and preventing regressions on a wider number of host
> systems than the ones provided by GitLab's shared runners.
>
> This sets the stage in which other community members can add their own
> machine configuration documentation/scripts, and accompanying job
> definitions.  As a general rule, those newly added contributed jobs
> should run as "non-gating", until their reliability is verified (AKA
> "allow_failure: true").
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée
Re: [PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder
Posted by Thomas Huth 4 years, 8 months ago 
On 08/06/2021 05.14, Cleber Rosa wrote:
> As described in the included documentation, the "custom runner" jobs
> extend the GitLab CI jobs already in place.  One of their primary
> goals of catching and preventing regressions on a wider number of host
> systems than the ones provided by GitLab's shared runners.
> 
> This sets the stage in which other community members can add their own
> machine configuration documentation/scripts, and accompanying job
> definitions.  As a general rule, those newly added contributed jobs
> should run as "non-gating", until their reliability is verified (AKA
> "allow_failure: true").
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   .gitlab-ci.d/custom-runners.yml | 14 ++++++++++++++
>   .gitlab-ci.d/qemu-project.yml   |  1 +
>   docs/devel/ci.rst               | 28 ++++++++++++++++++++++++++++
>   docs/devel/index.rst            |  1 +
>   4 files changed, 44 insertions(+)
>   create mode 100644 .gitlab-ci.d/custom-runners.yml
>   create mode 100644 docs/devel/ci.rst
> 
> diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> new file mode 100644
> index 0000000000..a07b27384c
> --- /dev/null
> +++ b/.gitlab-ci.d/custom-runners.yml
> @@ -0,0 +1,14 @@
> +# The CI jobs defined here require GitLab runners installed and
> +# registered on machines that match their operating system names,
> +# versions and architectures.  This is in contrast to the other CI
> +# jobs that are intended to run on GitLab's "shared" runners.
> +
> +# Different than the default approach on "shared" runners, based on
> +# containers, the custom runners have no such *requirement*, as those
> +# jobs should be capable of running on operating systems with no
> +# compatible container implementation, or no support from
> +# gitlab-runner.  To avoid problems that gitlab-runner can cause while
> +# reusing the GIT repository, let's enable the clone strategy, which
> +# guarantees a fresh repository on each job run.
> +variables:
> +  GIT_STRATEGY: clone

I assume this now does not influence the shared runners anymore, now that 
the file is handled as last entry in the list below? Or is this a global 
setting that will also be used for the shared runners? Anyway, let's keep it 
for now, we can still try to improve later...

> diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
> index 64cb2ba1da..dde8270301 100644
> --- a/.gitlab-ci.d/qemu-project.yml
> +++ b/.gitlab-ci.d/qemu-project.yml
> @@ -9,3 +9,4 @@ include:
>     - local: '/.gitlab-ci.d/crossbuilds.yml'
>     - local: '/.gitlab-ci.d/buildtest.yml'
>     - local: '/.gitlab-ci.d/static_checks.yml'
> +  - local: '/.gitlab-ci.d/custom-runners.yml'

Reviewed-by: Thomas Huth <thuth@redhat.com>
Re: [PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder
Posted by Willian Rampazzo 4 years, 8 months ago 
On Tue, Jun 8, 2021 at 12:14 AM Cleber Rosa <crosa@redhat.com> wrote:
>
> As described in the included documentation, the "custom runner" jobs
> extend the GitLab CI jobs already in place.  One of their primary
> goals of catching and preventing regressions on a wider number of host
> systems than the ones provided by GitLab's shared runners.
>
> This sets the stage in which other community members can add their own
> machine configuration documentation/scripts, and accompanying job
> definitions.  As a general rule, those newly added contributed jobs
> should run as "non-gating", until their reliability is verified (AKA
> "allow_failure: true").
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  .gitlab-ci.d/custom-runners.yml | 14 ++++++++++++++
>  .gitlab-ci.d/qemu-project.yml   |  1 +
>  docs/devel/ci.rst               | 28 ++++++++++++++++++++++++++++
>  docs/devel/index.rst            |  1 +
>  4 files changed, 44 insertions(+)
>  create mode 100644 .gitlab-ci.d/custom-runners.yml
>  create mode 100644 docs/devel/ci.rst
>

Reviewed-by: Willian Rampazzo <willianr@redhat.com>

Patchew 2.1-devel-b67e4c2

© 2016 - 2026 Red Hat, Inc.