[PULL 00/14] testing updates (python, plugins)

Alex Bennée posted 14 patches 3 years, 7 months ago
Only 6 patches received!
docs/conf.py                                       |  4 +-
configure                                          |  4 +-
migration/global_state.c                           |  4 +-
.gitlab-ci.d/containers.yml                        | 38 ++++---------
.gitlab-ci.d/crossbuilds.yml                       |  5 +-
.gitlab-ci.yml                                     | 29 +++++++++-
.shippable.yml                                     |  4 --
.travis.yml                                        | 66 +++++-----------------
tests/docker/Makefile.include                      |  2 +-
.../docker/dockerfiles/debian-tricore-cross.docker |  2 +-
tests/docker/dockerfiles/debian-win32-cross.docker | 38 -------------
tests/docker/dockerfiles/debian-win64-cross.docker | 45 ---------------
tests/docker/dockerfiles/debian9-mxe.docker        | 21 -------
tests/docker/dockerfiles/debian9.docker            | 32 -----------
tests/docker/dockerfiles/fedora-win32-cross.docker | 42 ++++++++++++++
tests/docker/dockerfiles/fedora-win64-cross.docker | 38 +++++++++++++
tests/qemu-iotests/iotests.py                      |  2 -
17 files changed, 142 insertions(+), 234 deletions(-)
delete mode 100644 tests/docker/dockerfiles/debian-win32-cross.docker
delete mode 100644 tests/docker/dockerfiles/debian-win64-cross.docker
delete mode 100644 tests/docker/dockerfiles/debian9-mxe.docker
delete mode 100644 tests/docker/dockerfiles/debian9.docker
create mode 100644 tests/docker/dockerfiles/fedora-win32-cross.docker
create mode 100644 tests/docker/dockerfiles/fedora-win64-cross.docker
[PULL 00/14] testing updates (python, plugins)
Posted by Alex Bennée 3 years, 7 months ago
The following changes since commit b5ce42f5d138d7546f9faa2decbd6ee8702243a3:

  Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/ide-pull-request' into staging (2020-10-01 19:55:10 +0100)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-testing-and-python-021020-1

for you to fetch changes up to 2614670b7585ce4ec503546bc3023844d392f270:

  gitlab: split deprecated job into build/check stages (2020-10-02 12:31:34 +0100)

----------------------------------------------------------------
Python testing updates:

  - drop python 3.5 test from travis
  - replace Debian 9 containers with 10
  - increase cross build timeout
  - bump minimum python version in configure
  - move user plugins tests to gitlab
  - split deprecated builds into build and test

----------------------------------------------------------------
Alex Bennée (2):
      gitlab: move linux-user plugins test across to gitlab
      gitlab: split deprecated job into build/check stages

Thomas Huth (12):
      migration: Silence compiler warning in global_state_store_running()
      travis.yml: Drop the default softmmu builds
      travis.yml: Update Travis to use Bionic and Focal instead of Xenial
      travis.yml: Drop the superfluous Python 3.6 build
      travis.yml: Drop the Python 3.5 build
      tests/docker: Use Fedora containers for MinGW cross-builds in the gitlab-CI
      gitlab-ci: Remove the Debian9-based containers and containers-layer3
      tests/docker: Update the tricore container to debian 10
      shippable.yml: Remove the Debian9-based MinGW cross-compiler tests
      tests/docker: Remove old Debian 9 containers
      gitlab-ci: Increase the timeout for the cross-compiler builds
      configure: Bump the minimum required Python version to 3.6

 docs/conf.py                                       |  4 +-
 configure                                          |  4 +-
 migration/global_state.c                           |  4 +-
 .gitlab-ci.d/containers.yml                        | 38 ++++---------
 .gitlab-ci.d/crossbuilds.yml                       |  5 +-
 .gitlab-ci.yml                                     | 29 +++++++++-
 .shippable.yml                                     |  4 --
 .travis.yml                                        | 66 +++++-----------------
 tests/docker/Makefile.include                      |  2 +-
 .../docker/dockerfiles/debian-tricore-cross.docker |  2 +-
 tests/docker/dockerfiles/debian-win32-cross.docker | 38 -------------
 tests/docker/dockerfiles/debian-win64-cross.docker | 45 ---------------
 tests/docker/dockerfiles/debian9-mxe.docker        | 21 -------
 tests/docker/dockerfiles/debian9.docker            | 32 -----------
 tests/docker/dockerfiles/fedora-win32-cross.docker | 42 ++++++++++++++
 tests/docker/dockerfiles/fedora-win64-cross.docker | 38 +++++++++++++
 tests/qemu-iotests/iotests.py                      |  2 -
 17 files changed, 142 insertions(+), 234 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian-win32-cross.docker
 delete mode 100644 tests/docker/dockerfiles/debian-win64-cross.docker
 delete mode 100644 tests/docker/dockerfiles/debian9-mxe.docker
 delete mode 100644 tests/docker/dockerfiles/debian9.docker
 create mode 100644 tests/docker/dockerfiles/fedora-win32-cross.docker
 create mode 100644 tests/docker/dockerfiles/fedora-win64-cross.docker

-- 
2.20.1


Re: [PULL 00/14] testing updates (python, plugins)
Posted by Peter Maydell 3 years, 7 months ago
On Fri, 2 Oct 2020 at 12:36, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit b5ce42f5d138d7546f9faa2decbd6ee8702243a3:
>
>   Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/ide-pull-request' into staging (2020-10-01 19:55:10 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-testing-and-python-021020-1
>
> for you to fetch changes up to 2614670b7585ce4ec503546bc3023844d392f270:
>
>   gitlab: split deprecated job into build/check stages (2020-10-02 12:31:34 +0100)
>
> ----------------------------------------------------------------
> Python testing updates:
>
>   - drop python 3.5 test from travis
>   - replace Debian 9 containers with 10
>   - increase cross build timeout
>   - bump minimum python version in configure
>   - move user plugins tests to gitlab
>   - split deprecated builds into build and test


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.2
for any user-visible changes.

-- PMM

[PULL 01/14] migration: Silence compiler warning in global_state_store_running()
Posted by Alex Bennée 3 years, 7 months ago
From: Thomas Huth <thuth@redhat.com>

GCC 9.3.0 on Ubuntu complains:

In file included from /usr/include/string.h:495,
                 from /home/travis/build/huth/qemu/include/qemu/osdep.h:87,
                 from ../migration/global_state.c:13:
In function ‘strncpy’,
    inlined from ‘global_state_store_running’ at ../migration/global_state.c:47:5:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: error:
 ‘__builtin_strncpy’ specified bound 100 equals destination size [-Werror=stringop-truncation]
  106 |   return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest));
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

... but we apparently really want to do a strncpy here - the size is already
checked with the assert() statement right in front of it. To silence the
warning, simply replace it with our strpadcpy() function.

Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com> (two years ago)
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200918103430.297167-4-thuth@redhat.com>
Message-Id: <20200925154027.12672-5-alex.bennee@linaro.org>

diff --git a/migration/global_state.c b/migration/global_state.c
index 25311479a4..a33947ca32 100644
--- a/migration/global_state.c
+++ b/migration/global_state.c
@@ -44,8 +44,8 @@ void global_state_store_running(void)
 {
     const char *state = RunState_str(RUN_STATE_RUNNING);
     assert(strlen(state) < sizeof(global_state.runstate));
-    strncpy((char *)global_state.runstate,
-           state, sizeof(global_state.runstate));
+    strpadcpy((char *)global_state.runstate, sizeof(global_state.runstate),
+              state, '\0');
 }
 
 bool global_state_received(void)
-- 
2.20.1


[PULL 02/14] travis.yml: Drop the default softmmu builds
Posted by Alex Bennée 3 years, 7 months ago
From: Thomas Huth <thuth@redhat.com>

The total runtime of all Travis jobs is very long and we are testing
all softmmu targets in the gitlab-CI already - so we can speed up the
Travis testing a little bit by not testing the softmmu targets here
anymore.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200918103430.297167-5-thuth@redhat.com>
Message-Id: <20200925154027.12672-6-alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index bd9a6fc06c..b2d492f8c6 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -123,20 +123,6 @@ jobs:
         - CONFIG="--disable-system --static"
         - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
 
-
-    # we split the system builds as it takes a while to build them all
-    - name: "GCC (main-softmmu)"
-      env:
-        - CONFIG="--disable-user --target-list=${MAIN_SOFTMMU_TARGETS}"
-        - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
-
-
-    - name: "GCC (other-softmmu)"
-      env:
-       - CONFIG="--disable-user --target-list-exclude=${MAIN_SOFTMMU_TARGETS}"
-        - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
-
-
     # Just build tools and run minimal unit and softfloat checks
     - name: "GCC check-unit and check-softfloat"
       env:
-- 
2.20.1


[PULL 03/14] travis.yml: Update Travis to use Bionic and Focal instead of Xenial
Posted by Alex Bennée 3 years, 7 months ago
From: Thomas Huth <thuth@redhat.com>

According to our support policy, we do not support Xenial anymore.
Time to switch the bigger parts of the builds to Focal instead.
Some few jobs have to be updated to Bionic instead, since they are
currently still failing on Focal otherwise. Also "--disable-pie" is
causing linker problems with newer versions of Ubuntu ... so remove
that switch from the jobs now (we still test it in a gitlab CI job,
so we don't lose much test coverage here).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200918103430.297167-6-thuth@redhat.com>
Message-Id: <20200925154027.12672-7-alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index b2d492f8c6..65b825ff64 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,7 +2,7 @@
 # Additional builds with specific requirements for a full VM need to
 # be added as additional matrix: entries later on
 os: linux
-dist: xenial
+dist: focal
 language: c
 compiler:
   - gcc
@@ -10,7 +10,7 @@ cache:
   # There is one cache per branch and compiler version.
   # characteristics of each job are used to identify the cache:
   # - OS name (currently only linux)
-  # - OS distribution (for Linux, xenial, trusty, or precise)
+  # - OS distribution (for Linux, bionic or focal)
   # - Names and values of visible environment variables set in .travis.yml or Settings panel
   timeout: 1200
   ccache: true
@@ -27,7 +27,7 @@ addons:
       - libattr1-dev
       - libbrlapi-dev
       - libcap-ng-dev
-      - libgcc-4.8-dev
+      - libgcc-7-dev
       - libgnutls28-dev
       - libgtk-3-dev
       - libiscsi-dev
@@ -210,8 +210,10 @@ jobs:
 
     # gprof/gcov are GCC features
     - name: "GCC gprof/gcov"
+      dist: bionic
       env:
-        - CONFIG="--enable-gprof --enable-gcov --disable-pie --target-list=${MAIN_SOFTMMU_TARGETS}"
+        - CONFIG="--enable-gprof --enable-gcov --disable-libssh
+                  --target-list=${MAIN_SOFTMMU_TARGETS}"
       after_success:
         - ${SRC_DIR}/scripts/travis/coverage-summary.sh
 
@@ -270,6 +272,7 @@ jobs:
 
     # Using newer GCC with sanitizers
     - name: "GCC9 with sanitizers (softmmu)"
+      dist: bionic
       addons:
         apt:
           update: true
@@ -285,7 +288,7 @@ jobs:
             - libattr1-dev
             - libbrlapi-dev
             - libcap-ng-dev
-            - libgnutls-dev
+            - libgnutls28-dev
             - libgtk-3-dev
             - libiscsi-dev
             - liblttng-ust-dev
@@ -293,14 +296,13 @@ jobs:
             - libncurses5-dev
             - libnss3-dev
             - libpixman-1-dev
-            - libpng12-dev
+            - libpng-dev
             - librados-dev
             - libsdl2-dev
             - libsdl2-image-dev
             - libseccomp-dev
             - libspice-protocol-dev
             - libspice-server-dev
-            - libssh-dev
             - liburcu-dev
             - libusb-1.0-0-dev
             - libvte-2.91-dev
@@ -310,11 +312,11 @@ jobs:
       compiler: none
       env:
         - COMPILER_NAME=gcc CXX=g++-9 CC=gcc-9
-        - CONFIG="--cc=gcc-9 --cxx=g++-9 --disable-pie --disable-linux-user"
+        - CONFIG="--cc=gcc-9 --cxx=g++-9 --disable-linux-user"
         - TEST_CMD=""
       before_script:
         - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
-        - ${SRC_DIR}/configure ${CONFIG} --extra-cflags="-g3 -O0 -Wno-error=stringop-truncation -fsanitize=thread" --extra-ldflags="-fuse-ld=gold" || { cat config.log && exit 1; }
+        - ${SRC_DIR}/configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread" || { cat config.log && exit 1; }
 
 
     # Run check-tcg against linux-user
@@ -356,7 +358,7 @@ jobs:
 
     - name: "[aarch64] GCC check-tcg"
       arch: arm64
-      dist: xenial
+      dist: focal
       addons:
         apt_packages:
           - libaio-dev
@@ -389,7 +391,7 @@ jobs:
 
     - name: "[ppc64] GCC check-tcg"
       arch: ppc64le
-      dist: xenial
+      dist: focal
       addons:
         apt_packages:
           - libaio-dev
-- 
2.20.1


[PULL 04/14] travis.yml: Drop the superfluous Python 3.6 build
Posted by Alex Bennée 3 years, 7 months ago
From: Thomas Huth <thuth@redhat.com>

Python 3.6 is already the default Python in the jobs that are based
on Ubuntu Bionic, so it does not make much sense to test this again
separately.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200918103430.297167-7-thuth@redhat.com>
Message-Id: <20200925154027.12672-8-alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 65b825ff64..990dd11e6f 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -262,14 +262,6 @@ jobs:
       python: 3.5
 
 
-    - name: "GCC Python 3.6 (x86_64-softmmu)"
-      env:
-        - CONFIG="--target-list=x86_64-softmmu"
-        - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
-      language: python
-      python: 3.6
-
-
     # Using newer GCC with sanitizers
     - name: "GCC9 with sanitizers (softmmu)"
       dist: bionic
-- 
2.20.1


[PULL 05/14] travis.yml: Drop the Python 3.5 build
Posted by Alex Bennée 3 years, 7 months ago
From: Thomas Huth <thuth@redhat.com>

We are soon going to remove the support for Python 3.5. So remove
the CI job now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200922070441.48844-1-thuth@redhat.com>
Message-Id: <20200925154027.12672-9-alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 990dd11e6f..c255c331a7 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -253,15 +253,6 @@ jobs:
         - TEST_CMD=""
 
 
-    # Python builds
-    - name: "GCC Python 3.5 (x86_64-softmmu)"
-      env:
-        - CONFIG="--target-list=x86_64-softmmu"
-        - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
-      language: python
-      python: 3.5
-
-
     # Using newer GCC with sanitizers
     - name: "GCC9 with sanitizers (softmmu)"
       dist: bionic
-- 
2.20.1


[PULL 09/14] shippable.yml: Remove the Debian9-based MinGW cross-compiler tests
Posted by Alex Bennée 3 years, 7 months ago
From: Thomas Huth <thuth@redhat.com>

We're not supporting Debian 9 anymore, and we are now testing
MinGW cross-compiler builds in the gitlab-CI, too, so we do not
really need these jobs in the shippable.yml anymore.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200921174320.46062-5-thuth@redhat.com>
Message-Id: <20200925154027.12672-13-alex.bennee@linaro.org>

diff --git a/.shippable.yml b/.shippable.yml
index 0b4fd6df1d..14350e6de8 100644
--- a/.shippable.yml
+++ b/.shippable.yml
@@ -7,10 +7,6 @@ env:
   matrix:
     - IMAGE=debian-amd64
       TARGET_LIST=x86_64-softmmu,x86_64-linux-user
-    - IMAGE=debian-win32-cross
-      TARGET_LIST=arm-softmmu,i386-softmmu
-    - IMAGE=debian-win64-cross
-      TARGET_LIST=aarch64-softmmu,sparc64-softmmu,x86_64-softmmu
     - IMAGE=debian-armel-cross
       TARGET_LIST=arm-softmmu,arm-linux-user,armeb-linux-user
     - IMAGE=debian-armhf-cross
-- 
2.20.1


[PULL 11/14] gitlab-ci: Increase the timeout for the cross-compiler builds
Posted by Alex Bennée 3 years, 7 months ago
From: Thomas Huth <thuth@redhat.com>

Some of the cross-compiler builds (the mips build and the win64 build
for example) are quite slow and sometimes hit the 1h time limit.
Increase the limit a little bit to make sure that we do not get failures
in the CI runs just because of some few minutes.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200921174320.46062-7-thuth@redhat.com>
Message-Id: <20200925154027.12672-15-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 510cfec03b..03ebfabb3f 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -2,6 +2,7 @@
 .cross_system_build_job_template: &cross_system_build_job_definition
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+  timeout: 80m
   script:
     - mkdir build
     - cd build
-- 
2.20.1


[PULL 14/14] gitlab: split deprecated job into build/check stages
Posted by Alex Bennée 3 years, 7 months ago
While the job is pretty fast for only a few targets we still want to
catch breakage of the build. By splitting the test step we can
allow_failures for that while still ensuring we don't miss the build
breaking.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201002091538.3017-1-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 346f23acf7..a51c89554f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -270,9 +270,24 @@ build-deprecated:
   variables:
     IMAGE: debian-all-test-cross
     CONFIGURE_ARGS: --disable-docs --disable-tools
-    MAKE_CHECK_ARGS: check-tcg
+    MAKE_CHECK_ARGS: build-tcg
     TARGETS: ppc64abi32-linux-user tilegx-linux-user lm32-softmmu
       unicore32-softmmu
+  artifacts:
+    expire_in: 2 days
+    paths:
+      - build
+
+# We split the check-tcg step as test failures are expected but we still
+# want to catch the build breaking.
+check-deprecated:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-deprecated
+      artifacts: true
+  variables:
+    IMAGE: debian-all-test-cross
+    MAKE_CHECK_ARGS: check-tcg
   allow_failure: true
 
 build-oss-fuzz:
-- 
2.20.1