[PULL for 5.2-rc3 0/7] various CI cleanups (scripts, avocado, gitlab)

Alex Bennée posted 7 patches 3 years, 5 months ago
Only 3 patches received!
.cirrus.yml                                |  2 ++
.gitlab-ci.yml                             | 35 +++++++++++++++++++++++
.travis.yml                                | 45 ------------------------------
python/qemu/machine.py                     |  3 +-
scripts/ci/gitlab-pipeline-status          | 24 ++++++++--------
tests/acceptance/avocado_qemu/__init__.py  |  4 ++-
tests/docker/dockerfiles/ubuntu2004.docker |  1 +
7 files changed, 56 insertions(+), 58 deletions(-)
[PULL for 5.2-rc3 0/7] various CI cleanups (scripts, avocado, gitlab)
Posted by Alex Bennée 3 years, 5 months ago
The following changes since commit 8cc30eb1400fc01f2b139cdd3dc524f8b84dbe07:

  Merge remote-tracking branch 'remotes/mcayland/tags/qemu-sparc-20201122' into staging (2020-11-22 15:02:52 +0000)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-for-5.2-rc3-231120-1

for you to fetch changes up to 534f80e1dffbf520bed9bf5fd5ae98de6662e126:

  .cirrus.yml: bump timeout period for MacOS builds (2020-11-23 09:55:25 +0000)

----------------------------------------------------------------
Misc CI fixes:

  - more helpful logic for git-pipeline-status
  - fix tempdir leak in avocado
  - move remaining x86 check-tcg to gitlab
  - add tracing headers to ubuntu2004 docker
  - move tracing backend tests to gitlab
  - bump up timeouts on cirrus MacOS

----------------------------------------------------------------
Alex Bennée (5):
      scripts/ci: clean up default args logic a little
      tests: add prefixes to the bare mkdtemp calls
      tests/avocado: clean-up socket directory after run
      gitlab: move remaining x86 check-tcg targets to gitlab
      .cirrus.yml: bump timeout period for MacOS builds

Philippe Mathieu-Daudé (2):
      tests/docker: Install liblttng-ust-dev package in Ubuntu 20.04 image
      gitlab-ci: Move trace backend tests across to gitlab

 .cirrus.yml                                |  2 ++
 .gitlab-ci.yml                             | 35 +++++++++++++++++++++++
 .travis.yml                                | 45 ------------------------------
 python/qemu/machine.py                     |  3 +-
 scripts/ci/gitlab-pipeline-status          | 24 ++++++++--------
 tests/acceptance/avocado_qemu/__init__.py  |  4 ++-
 tests/docker/dockerfiles/ubuntu2004.docker |  1 +
 7 files changed, 56 insertions(+), 58 deletions(-)

-- 
2.20.1


Re: [PULL for 5.2-rc3 0/7] various CI cleanups (scripts, avocado, gitlab)
Posted by Peter Maydell 3 years, 5 months ago
On Mon, 23 Nov 2020 at 11:25, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 8cc30eb1400fc01f2b139cdd3dc524f8b84dbe07:
>
>   Merge remote-tracking branch 'remotes/mcayland/tags/qemu-sparc-20201122' into staging (2020-11-22 15:02:52 +0000)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-for-5.2-rc3-231120-1
>
> for you to fetch changes up to 534f80e1dffbf520bed9bf5fd5ae98de6662e126:
>
>   .cirrus.yml: bump timeout period for MacOS builds (2020-11-23 09:55:25 +0000)
>
> ----------------------------------------------------------------
> Misc CI fixes:
>
>   - more helpful logic for git-pipeline-status
>   - fix tempdir leak in avocado
>   - move remaining x86 check-tcg to gitlab
>   - add tracing headers to ubuntu2004 docker
>   - move tracing backend tests to gitlab
>   - bump up timeouts on cirrus MacOS
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.2
for any user-visible changes.

-- PMM

[PULL 1/7] scripts/ci: clean up default args logic a little
Posted by Alex Bennée 3 years, 5 months ago
This allows us to do:

  ./scripts/ci/gitlab-pipeline-status -w -b HEAD -p 2961854

to check out own pipeline status of a recently pushed branch.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201117173635.29101-2-alex.bennee@linaro.org>

diff --git a/scripts/ci/gitlab-pipeline-status b/scripts/ci/gitlab-pipeline-status
index bac8233079..78e72f6008 100755
--- a/scripts/ci/gitlab-pipeline-status
+++ b/scripts/ci/gitlab-pipeline-status
@@ -31,7 +31,7 @@ class NoPipelineFound(Exception):
     """Communication is successfull but pipeline is not found."""
 
 
-def get_local_branch_commit(branch='staging'):
+def get_local_branch_commit(branch):
     """
     Returns the commit sha1 for the *local* branch named "staging"
     """
@@ -126,18 +126,16 @@ def create_parser():
                         help=('The GitLab project ID. Defaults to the project '
                               'for https://gitlab.com/qemu-project/qemu, that '
                               'is, "%(default)s"'))
-    try:
-        default_commit = get_local_branch_commit()
-        commit_required = False
-    except ValueError:
-        default_commit = ''
-        commit_required = True
-    parser.add_argument('-c', '--commit', required=commit_required,
-                        default=default_commit,
+    parser.add_argument('-b', '--branch', type=str, default="staging",
+                        help=('Specify the branch to check. '
+                              'Use HEAD for your current branch. '
+                              'Otherwise looks at "%(default)s"'))
+    parser.add_argument('-c', '--commit',
+                        default=None,
                         help=('Look for a pipeline associated with the given '
                               'commit.  If one is not explicitly given, the '
-                              'commit associated with the local branch named '
-                              '"staging" is used.  Default: %(default)s'))
+                              'commit associated with the default branch '
+                              'is used.'))
     parser.add_argument('--verbose', action='store_true', default=False,
                         help=('A minimal verbosity level that prints the '
                               'overall result of the check/wait'))
@@ -149,6 +147,10 @@ def main():
     """
     parser = create_parser()
     args = parser.parse_args()
+
+    if not args.commit:
+        args.commit = get_local_branch_commit(args.branch)
+
     success = False
     try:
         if args.wait:
-- 
2.20.1


[PULL 3/7] tests/avocado: clean-up socket directory after run
Posted by Alex Bennée 3 years, 5 months ago
Previously we were leaving temporary directories behind. While the
QEMUMachine does make efforts to clean up after itself the directory
belongs to the calling function. We use TemporaryDirectory to wrap
this although we explicitly clear the reference in tearDown() as it
doesn't get cleaned up otherwise.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20201117173635.29101-4-alex.bennee@linaro.org>

diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index 307020be45..bf54e419da 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -171,8 +171,8 @@ class Test(avocado.Test):
             self.cancel("No QEMU binary defined or found in the build tree")
 
     def _new_vm(self, *args):
-        sd = tempfile.mkdtemp(prefix="avo_qemu_sock_")
-        vm = QEMUMachine(self.qemu_bin, sock_dir=sd)
+        self._sd = tempfile.TemporaryDirectory(prefix="avo_qemu_sock_")
+        vm = QEMUMachine(self.qemu_bin, sock_dir=self._sd.name)
         if args:
             vm.add_args(*args)
         return vm
@@ -193,6 +193,7 @@ class Test(avocado.Test):
     def tearDown(self):
         for vm in self._vms.values():
             vm.shutdown()
+        self._sd = None
 
     def fetch_asset(self, name,
                     asset_hash=None, algorithm=None,
-- 
2.20.1


[PULL 5/7] tests/docker: Install liblttng-ust-dev package in Ubuntu 20.04 image
Posted by Alex Bennée 3 years, 5 months ago
From: Philippe Mathieu-Daudé <philmd@redhat.com>

Install the liblttng-ust-dev package to be able to
build QEMU using the User-Space Tracer trace backend
(configure --enable-trace-backends=ust).

Suggested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20201111121234.3246812-2-philmd@redhat.com>
Message-Id: <20201117173635.29101-6-alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index 355bbb3c63..ae889d8482 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -23,6 +23,7 @@ ENV PACKAGES flex bison \
     libiscsi-dev \
     libjemalloc-dev \
     libjpeg-turbo8-dev \
+    liblttng-ust-dev \
     liblzo2-dev \
     libncurses5-dev \
     libncursesw5-dev \
-- 
2.20.1


[PULL 7/7] .cirrus.yml: bump timeout period for MacOS builds
Posted by Alex Bennée 3 years, 5 months ago
These seem to trigger timeouts with some regularity.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201118140739.18377-1-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.cirrus.yml b/.cirrus.yml
index f0209b7a3e..08db7c419f 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -18,6 +18,7 @@ freebsd_12_task:
     - gmake -j$(sysctl -n hw.ncpu) check V=1
 
 macos_task:
+  timeout_in: 90m
   osx_instance:
     image: catalina-base
   install_script:
@@ -32,6 +33,7 @@ macos_task:
     - gmake check V=1
 
 macos_xcode_task:
+  timeout_in: 90m
   osx_instance:
     # this is an alias for the latest Xcode
     image: catalina-xcode
-- 
2.20.1