[libvirt] [PATCH] travis: switch to using Ubuntu 16.04 and 18.04

Daniel P. Berrangé posted 1 patch 5 years, 10 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20180605174714.29068-1-berrange@redhat.com
Test syntax-check passed
There is a newer version of this series
.travis.yml | 212 ++++++++++++++++++++++++++++------------------------
1 file changed, 116 insertions(+), 96 deletions(-)
[libvirt] [PATCH] travis: switch to using Ubuntu 16.04 and 18.04
Posted by Daniel P. Berrangé 5 years, 10 months ago
The container images provided by Travis only support Ubuntu 14.04,
however, Travis has ability to run docker, which allows the build
script to use arbitrary OS images. This takes advantage of that to
convert the build over to Ubuntu 16.04 and 18.04

This is using the official Ubuntu provided images and installing
extra build deps required, as we previously did with Travis container
images. With the switch to Docker though, this can be improved, by
building custom Docker images with all the deps pre-installed which
will cut down build time. This can be driven from the package lists
in libvirt-jenkins-ci repo, to remove the duplication. This work
for future improvement though, this just does the minimal conversion
to match what we already do, but with newer distro.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .travis.yml | 212 ++++++++++++++++++++++++++++------------------------
 1 file changed, 116 insertions(+), 96 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index c71411dcae..15b5e933e9 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,6 +1,5 @@
-sudo: false
-language: c
-cache: ccache
+sudo: required
+language: generic
 
 branches:
   except:
@@ -8,17 +7,38 @@ branches:
 
 matrix:
   include:
-    - compiler: gcc
-      dist: trusty
+    - services:
+        - docker
       env:
-        - PYTHON=$(which python2)
-        - DISTCHECK_CONFIGURE_FLAGS="--with-init-script=upstart"
-    - compiler: clang
-      dist: trusty
+        - DISTCHECK_FLAGS="--with-init-script=upstart"
+      before_install:
+        - docker pull berrange/test
+        - docker run
+            --privileged
+            -v `pwd`:/build
+            -w /build
+            -e VIR_TEST_DEBUG="$VIR_TEST_DEBUG"
+            -e PACKAGES="$PACKAGES"
+            -e DISTCHECK_FLAGS="$DISTCHECK_FLAGS"
+            ubuntu:18.04
+            /bin/sh -c "$DOCKER_CMD"
+    - services:
+        - docker
       env:
-        - PYTHON=$(which python3)
-        - DISTCHECK_CONFIGURE_FLAGS="--with-init-script=systemd"
+        - DISTCHECK_FLAGS="--with-init-script=upstart"
+      before_install:
+        - docker pull berrange/test
+        - docker run
+            --privileged
+            -v `pwd`:/build
+            -w /build
+            -e VIR_TEST_DEBUG="$VIR_TEST_DEBUG"
+            -e PACKAGES="$PACKAGES"
+            -e DISTCHECK_FLAGS="$DISTCHECK_FLAGS"
+            ubuntu:16.04
+            /bin/sh -c "$DOCKER_CMD"
     - compiler: clang
+      language: c
       os: osx
       env:
         - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/ccache/libexec:/usr/local/opt/rpcgen/bin:$PATH"
@@ -27,80 +47,18 @@ matrix:
         - brew update
         - brew upgrade
         - brew install python ccache rpcgen yajl
+      before_script:
+        - ./autogen.sh --prefix=$(pwd)/install-root
       script:
         # We can't run make distcheck/syntax-check because they
         # fail on macOS, but doing 'install' and 'dist' gives us
         # some useful coverage
         - make -j3 && make -j3 install && make -j3 dist
+      after_failure:
+        - echo '=== LOG FILE(S) START ==='
+        - find -name test-suite.log | xargs cat
+        - echo '=== LOG FILE(S) END ==='
 
-addons:
-  apt:
-    # Please keep this list sorted alphabetically
-    packages:
-      - augeas-tools
-      - autoconf
-      - automake
-      - autopoint
-      - bash-completion
-      - ccache
-      - dnsmasq-base
-      - dwarves
-      - ebtables
-      - gcc
-      - gettext
-      - glusterfs-client
-      - libacl1-dev
-      - libapparmor-dev
-      - libattr1-dev
-      - libaudit-dev
-      - libavahi-client-dev
-      - libblkid-dev
-      - libc6-dev
-      - libcap-ng-dev
-      - libc-dev-bin
-      - libcurl4-gnutls-dev
-      - libdbus-1-dev
-      - libdevmapper-dev
-      - libfuse-dev
-      - libgnutls-dev
-      - libnetcf-dev
-      - libnl-3-dev
-      - libnl-route-3-dev
-      - libnuma-dev
-      - libopenwsman-dev
-      - libparted-dev
-      - libpcap-dev
-      - libpciaccess-dev
-      - librbd-dev
-      - libreadline-dev
-      - libsanlock-dev
-      - libsasl2-dev
-      - libselinux1-dev
-      - libssh2-1-dev
-      - libssh-dev
-      - libtirpc-dev
-      - libtool
-      - libudev-dev
-      - libxen-dev
-      - libxml2-dev
-      - libxml2-utils
-      - libyajl-dev
-      - lvm2
-      - make
-#     - nfs-common    # broken on trusty since build #807 (2018-01-09)
-      - open-iscsi
-      - parted
-      - patch
-      - perl
-      - pkgconf
-      - policykit-1
-      - qemu-utils
-      - radvd
-      - scrub
-      - sheepdog
-      - systemtap-sdt-dev
-      - xsltproc
-      - zfs-fuse
 
 git:
   submodules: true
@@ -108,23 +66,85 @@ git:
 env:
   global:
     - VIR_TEST_DEBUG=1
-
-before_script:
-  - ./autogen.sh --prefix=$(pwd)/install-root
-
-script:
-  - make -j3 && make -j3 syntax-check && make -j3 distcheck DISTCHECK_CONFIGURE_FLAGS=$DISTCHECK_CONFIGURE_FLAGS
-
-after_failure:
-  - echo '============================================================================'
-  - 'ver=$(sed -n "s/AC_INIT(\[libvirt\], \[\([0-9.]*\)\],.*/\1/p" configure.ac);
-    if [ -f $(pwd)/tests/test-suite.log ]; then
-        cat $(pwd)/tests/test-suite.log;
-    elif [ -f libvirt-$ver/_build/tests/test-suite.log ]; then
-        cat libvirt-$ver/_build/tests/test-suite.log;
-    else
-        echo "=== NO LOG FILE FOUND ===";
-    fi'
+    - DOCKER_CMD="
+        apt-get update &&
+        apt-get install -y \$PACKAGES &&
+        ./autogen.sh --prefix=/build/install-root &&
+        make -j 3 &&
+        make -j 3 syntax-check &&
+        make -j 3 distcheck DISTCHECK_CONFIGURE_FLAGS=\$DISTCHECK_FLAGS ||
+        (
+          echo '=== LOG FILE(S) START ==='
+          find -name test-suite.log | xargs cat
+          echo '=== LOG FILE(S) END ==='
+          exit 1
+        )
+      "
+    - PACKAGES="
+        augeas-tools
+        autoconf
+        automake
+        autopoint
+        bash-completion
+        ccache
+        dnsmasq-base
+        dwarves
+        ebtables
+        gcc
+        gettext
+        git
+        glusterfs-client
+        libacl1-dev
+        libapparmor-dev
+        libattr1-dev
+        libaudit-dev
+        libavahi-client-dev
+        libblkid-dev
+        libc6-dev
+        libcap-ng-dev
+        libc-dev-bin
+        libdbus-1-dev
+        libdevmapper-dev
+        libfuse-dev
+        libgnutls28-dev
+        libnetcf-dev
+        libnl-3-dev
+        libnl-route-3-dev
+        libnuma-dev
+        libopenwsman-dev
+        libparted-dev
+        libpcap-dev
+        libpciaccess-dev
+        librbd-dev
+        libreadline-dev
+        libsanlock-dev
+        libsasl2-dev
+        libselinux1-dev
+        libssh2-1-dev
+        libssh-dev
+        libtirpc-dev
+        libtool
+        libudev-dev
+        libxen-dev
+        libxml2-dev
+        libxml2-utils
+        libyajl-dev
+        lvm2
+        make
+        nfs-common
+        open-iscsi
+        parted
+        patch
+        perl
+        pkgconf
+        policykit-1
+        qemu-utils
+        radvd
+        scrub
+        sheepdog
+        systemtap-sdt-dev
+        xsltproc
+        zfs-fuse"
 
 notifications:
   irc:
-- 
2.17.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH] travis: switch to using Ubuntu 16.04 and 18.04
Posted by Andrea Bolognani 5 years, 10 months ago
On Tue, 2018-06-05 at 18:47 +0100, Daniel P. Berrangé wrote:
> The container images provided by Travis only support Ubuntu 14.04,
> however, Travis has ability to run docker, which allows the build
> script to use arbitrary OS images. This takes advantage of that to
> convert the build over to Ubuntu 16.04 and 18.04

Neat! I was not aware that was possible. TIL :)

> This is using the official Ubuntu provided images and installing
> extra build deps required, as we previously did with Travis container
> images.

Everything after this doesn't belong in the commit message IMHO.

> With the switch to Docker though, this can be improved, by
> building custom Docker images with all the deps pre-installed which
> will cut down build time. This can be driven from the package lists
> in libvirt-jenkins-ci repo, to remove the duplication. This work
> for future improvement though, this just does the minimal conversion
> to match what we already do, but with newer distro.

We will probably want to do that, since installing packages takes
quite a bit of time and using Docker like this apparently causes
jobs to serialize, which makes shaving down time all the more
important.

I'm wondering if using Ubuntu 16.04 and 18.04 as base image is the
best option, however. Pavel is trying to get more hardware assigned
to us in the CentOS CI environment, and assuming that's successful
we will get those distributions added there as well for full-stack
testing; Travis CI is more useful for developers to smoke test
their patch series before posting, however, and with that in mind
I think it would be much better to run the build on the latest
CentOS, Fedora, Debian and Ubuntu to give reasonable coverage,
leaving up to CentOS CI to catch issues related to interactions
with other components or compatibility with older distros after
merge.

[...]
> +      before_install:

Not sure 'before_install' is the best choice here, I would probably
have gone with 'script' instead.

On the other hand, it seems like the distinction between job phases
is fairly blurry, so it probably doesn't matter after all.

> +        - docker pull berrange/test

I'm pretty sure we don't want this :)

> +        - docker run
> +            --privileged

Is '--privileged' really needed here? I'm not familiar enough
with Docker, but it feels like we shouldn't need it. Feel free to
point out exactly why I'm wrong :)

> +            -v `pwd`:/build

Please use $(pwd) instead of `pwd`.

> +            -w /build
> +            -e VIR_TEST_DEBUG="$VIR_TEST_DEBUG"
> +            -e PACKAGES="$PACKAGES"
> +            -e DISTCHECK_FLAGS="$DISTCHECK_FLAGS"
> +            ubuntu:18.04

It would be great to have the base image name in the environment
as well, so that in case the build fails on only one of the
Ubuntu versions you could easily tell from a quick look at the
build summary which one that is.

[...]
>        env:
> -        - PYTHON=$(which python3)

You got rid of this variable entirely. I think we won't lose any
coverage because configure will naturally pick python2 on 16.04
and python3 on 18.04 (I can't check because Travis CI decided to
only keep around a truncated log of my test run), but it was kinda
nice to have that explicitly in the summary. As long as you can
confirm the above is correct and no coverage is lost, though, I'm
okay with the removal.

> -        - DISTCHECK_CONFIGURE_FLAGS="--with-init-script=systemd"
> +        - DISTCHECK_FLAGS="--with-init-script=upstart"

You copied the wrong line here, we're going to test upstart twice.

I don't see a good reason to change the variable name, however:
let's just stick with matching the name expected by distcheck
itself.

[...]
>      - compiler: clang
> +      language: c
>        os: osx

You lost the ccache setting. That's probably okay because we
install the package and set up PATH explicitly anyway.

[...]
> +      after_failure:
> +        - echo '=== LOG FILE(S) START ==='
> +        - find -name test-suite.log | xargs cat
> +        - echo '=== LOG FILE(S) END ==='

This is simpler than the previous approach. I like it :)

[...]
> +    - DOCKER_CMD="
> +        apt-get update &&
> +        apt-get install -y \$PACKAGES &&
> +        ./autogen.sh --prefix=/build/install-root &&

Use

  --prefix=\$(pwd)/install-root

here for consistency with the macOS part.

> +        make -j 3 &&

No space between '-j' and '3'.

> +        make -j 3 syntax-check &&
> +        make -j 3 distcheck DISTCHECK_CONFIGURE_FLAGS=\$DISTCHECK_FLAGS ||

This won't work if we have more than a single flag, will it?

> +        (
> +          echo '=== LOG FILE(S) START ==='
> +          find -name test-suite.log | xargs cat
> +          echo '=== LOG FILE(S) END ==='
> +          exit 1
> +        )
> +      "
> +    - PACKAGES="

You lost the comment about this list being sorted alphabetically.

> +        augeas-tools
[...]
> +        zfs-fuse"

Put the closing quote on a separate line for neater future diffs.

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH] travis: switch to using Ubuntu 16.04 and 18.04
Posted by Daniel P. Berrangé 5 years, 10 months ago
On Wed, Jun 06, 2018 at 11:27:38AM +0200, Andrea Bolognani wrote:
> On Tue, 2018-06-05 at 18:47 +0100, Daniel P. Berrangé wrote:
> > The container images provided by Travis only support Ubuntu 14.04,
> > however, Travis has ability to run docker, which allows the build
> > script to use arbitrary OS images. This takes advantage of that to
> > convert the build over to Ubuntu 16.04 and 18.04
> 
> Neat! I was not aware that was possible. TIL :)

Yeah, nor was I until yesterday when I found someone saying this is
the way to get off the ancient Ubuntu versions in Travis :-)

> > This is using the official Ubuntu provided images and installing
> > extra build deps required, as we previously did with Travis container
> > images.
> 
> Everything after this doesn't belong in the commit message IMHO.
> 
> > With the switch to Docker though, this can be improved, by
> > building custom Docker images with all the deps pre-installed which
> > will cut down build time. This can be driven from the package lists
> > in libvirt-jenkins-ci repo, to remove the duplication. This work
> > for future improvement though, this just does the minimal conversion
> > to match what we already do, but with newer distro.
> 
> We will probably want to do that, since installing packages takes
> quite a bit of time and using Docker like this apparently causes
> jobs to serialize, which makes shaving down time all the more
> important.

When I tested it, the jobs still ran in parallel. The key difference
is that it causes the jobs to be scheduled on the VM based infra
in Travis instead of the container based infra. There's a slightly
longer startup time for jobs but otherwise nothing changes. The
jobs still took approx 25-30 minutes to complete, which is the same
as before this change. Pre-built docker images will definitely cut
it down though, at least 5 minutes off perhaps more.

> I'm wondering if using Ubuntu 16.04 and 18.04 as base image is the
> best option, however. Pavel is trying to get more hardware assigned
> to us in the CentOS CI environment, and assuming that's successful
> we will get those distributions added there as well for full-stack
> testing; Travis CI is more useful for developers to smoke test
> their patch series before posting, however, and with that in mind
> I think it would be much better to run the build on the latest
> CentOS, Fedora, Debian and Ubuntu to give reasonable coverage,
> leaving up to CentOS CI to catch issues related to interactions
> with other components or compatibility with older distros after
> merge.

Yeah, I just picked ubuntu as an initial target since it is close
to what we have. I would like to evolve it to have similar options
for coverage as the main CI. Ideally at least one of centos,
ubuntu, fedora and mingw, so 5 jobs in total including macOS.

> [...]
> > +      before_install:
> 
> Not sure 'before_install' is the best choice here, I would probably
> have gone with 'script' instead.

Yes, cut+paste mistake.

> On the other hand, it seems like the distinction between job phases
> is fairly blurry, so it probably doesn't matter after all.
> 
> > +        - docker pull berrange/test
> 
> I'm pretty sure we don't want this :)

Opps, yes, thats cruft from my testing custom images

> 
> > +        - docker run
> > +            --privileged
> 
> Is '--privileged' really needed here? I'm not familiar enough
> with Docker, but it feels like we shouldn't need it. Feel free to
> point out exactly why I'm wrong :)

We need privileged because we have to apt-get install extra
packages. Once we switch to pre-built images we can drop
the --privileged arg.

> 
> > +            -v `pwd`:/build
> 
> Please use $(pwd) instead of `pwd`.
> 
> > +            -w /build
> > +            -e VIR_TEST_DEBUG="$VIR_TEST_DEBUG"
> > +            -e PACKAGES="$PACKAGES"
> > +            -e DISTCHECK_FLAGS="$DISTCHECK_FLAGS"
> > +            ubuntu:18.04
> 
> It would be great to have the base image name in the environment
> as well, so that in case the build fails on only one of the
> Ubuntu versions you could easily tell from a quick look at the
> build summary which one that is.

Yeah, I'll see how to get that visible

> 
> [...]
> >        env:
> > -        - PYTHON=$(which python3)
> 
> You got rid of this variable entirely. I think we won't lose any
> coverage because configure will naturally pick python2 on 16.04
> and python3 on 18.04 (I can't check because Travis CI decided to
> only keep around a truncated log of my test run), but it was kinda
> nice to have that explicitly in the summary. As long as you can
> confirm the above is correct and no coverage is lost, though, I'm
> okay with the removal.

Yeah, I'll check

> 
> > -        - DISTCHECK_CONFIGURE_FLAGS="--with-init-script=systemd"
> > +        - DISTCHECK_FLAGS="--with-init-script=upstart"
> 
> You copied the wrong line here, we're going to test upstart twice.
> 
> I don't see a good reason to change the variable name, however:
> let's just stick with matching the name expected by distcheck
> itself.

ok

> 
> [...]
> >      - compiler: clang
> > +      language: c
> >        os: osx
> 
> You lost the ccache setting. That's probably okay because we
> install the package and set up PATH explicitly anyway.

The ccache setting applies to the "host", and so wouldn't
have an effect in side the docker container. I guess it
would still have affected ox-s though so I can re-add.

> 
> [...]
> > +      after_failure:
> > +        - echo '=== LOG FILE(S) START ==='
> > +        - find -name test-suite.log | xargs cat
> > +        - echo '=== LOG FILE(S) END ==='
> 
> This is simpler than the previous approach. I like it :)

Mostly because I couldn't figure out the right damn path to
use. It does have the benefit though that if gnulib tests
fail, we'll be catting the gnulib log file too.

> 
> [...]
> > +    - DOCKER_CMD="
> > +        apt-get update &&
> > +        apt-get install -y \$PACKAGES &&
> > +        ./autogen.sh --prefix=/build/install-root &&
> 
> Use
> 
>   --prefix=\$(pwd)/install-root
> 
> here for consistency with the macOS part.

ok

> 
> > +        make -j 3 &&
> 
> No space between '-j' and '3'.
> 
> > +        make -j 3 syntax-check &&
> > +        make -j 3 distcheck DISTCHECK_CONFIGURE_FLAGS=\$DISTCHECK_FLAGS ||
> 
> This won't work if we have more than a single flag, will it?

Good point, i'll quote it

> 
> > +        (
> > +          echo '=== LOG FILE(S) START ==='
> > +          find -name test-suite.log | xargs cat
> > +          echo '=== LOG FILE(S) END ==='
> > +          exit 1
> > +        )
> > +      "
> > +    - PACKAGES="
> 
> You lost the comment about this list being sorted alphabetically.

ok

> 
> > +        augeas-tools
> [...]
> > +        zfs-fuse"
> 
> Put the closing quote on a separate line for neater future diffs.
> 
> -- 
> Andrea Bolognani / Red Hat / Virtualization

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH] travis: switch to using Ubuntu 16.04 and 18.04
Posted by Andrea Bolognani 5 years, 10 months ago
On Wed, 2018-06-06 at 11:28 +0100, Daniel P. Berrangé wrote:
> On Wed, Jun 06, 2018 at 11:27:38AM +0200, Andrea Bolognani wrote:
> > We will probably want to do that, since installing packages takes
> > quite a bit of time and using Docker like this apparently causes
> > jobs to serialize, which makes shaving down time all the more
> > important.
> 
> When I tested it, the jobs still ran in parallel. The key difference
> is that it causes the jobs to be scheduled on the VM based infra
> in Travis instead of the container based infra. There's a slightly
> longer startup time for jobs but otherwise nothing changes. The
> jobs still took approx 25-30 minutes to complete, which is the same
> as before this change.

Doesn't match my experience: Ubuntu jobs took 30 minutes each and
the second one only started once the first one was done, meaning
the whole build took about one hour to finish. The macOS job was
running at the same time as the first Ubuntu job.

Might have been a coincidence, though.

> > I'm wondering if using Ubuntu 16.04 and 18.04 as base image is the
> > best option, however. Pavel is trying to get more hardware assigned
> > to us in the CentOS CI environment, and assuming that's successful
> > we will get those distributions added there as well for full-stack
> > testing; Travis CI is more useful for developers to smoke test
> > their patch series before posting, however, and with that in mind
> > I think it would be much better to run the build on the latest
> > CentOS, Fedora, Debian and Ubuntu to give reasonable coverage,
> > leaving up to CentOS CI to catch issues related to interactions
> > with other components or compatibility with older distros after
> > merge.
> 
> Yeah, I just picked ubuntu as an initial target since it is close
> to what we have. I would like to evolve it to have similar options
> for coverage as the main CI. Ideally at least one of centos,
> ubuntu, fedora and mingw, so 5 jobs in total including macOS.

I don't like the fact that you didn't include Debian :)

Ideally we'd also throw openSUSE into the mix, after adding it to
the CentOS CI, but as with Ubuntu that depends on getting acces to
extra capacity.

> > > +        - docker run
> > > +            --privileged
> > 
> > Is '--privileged' really needed here? I'm not familiar enough
> > with Docker, but it feels like we shouldn't need it. Feel free to
> > point out exactly why I'm wrong :)
> 
> We need privileged because we have to apt-get install extra
> packages. Once we switch to pre-built images we can drop
> the --privileged arg.

Makes sense.

> > >      - compiler: clang
> > > +      language: c
> > >        os: osx
> > 
> > You lost the ccache setting. That's probably okay because we
> > install the package and set up PATH explicitly anyway.
> 
> The ccache setting applies to the "host", and so wouldn't
> have an effect in side the docker container. I guess it
> would still have affected ox-s though so I can re-add.

As I said it's probably not strictly necessary, but it certainly
won't hurt to keep it around. Your call.

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH] travis: switch to using Ubuntu 16.04 and 18.04
Posted by Daniel P. Berrangé 5 years, 10 months ago
On Wed, Jun 06, 2018 at 01:01:05PM +0200, Andrea Bolognani wrote:
> On Wed, 2018-06-06 at 11:28 +0100, Daniel P. Berrangé wrote:
> > On Wed, Jun 06, 2018 at 11:27:38AM +0200, Andrea Bolognani wrote:
> > > We will probably want to do that, since installing packages takes
> > > quite a bit of time and using Docker like this apparently causes
> > > jobs to serialize, which makes shaving down time all the more
> > > important.
> > 
> > When I tested it, the jobs still ran in parallel. The key difference
> > is that it causes the jobs to be scheduled on the VM based infra
> > in Travis instead of the container based infra. There's a slightly
> > longer startup time for jobs but otherwise nothing changes. The
> > jobs still took approx 25-30 minutes to complete, which is the same
> > as before this change.
> 
> Doesn't match my experience: Ubuntu jobs took 30 minutes each and
> the second one only started once the first one was done, meaning
> the whole build took about one hour to finish. The macOS job was
> running at the same time as the first Ubuntu job.
> 
> Might have been a coincidence, though.

Yeah, I think that is unrelated, because I'm definitely seeing both
ubuntu jobs running at the same time. Probably travis just didn't
have spare workers to spin up another job at the time you tested.

> > Yeah, I just picked ubuntu as an initial target since it is close
> > to what we have. I would like to evolve it to have similar options
> > for coverage as the main CI. Ideally at least one of centos,
> > ubuntu, fedora and mingw, so 5 jobs in total including macOS.
> 
> I don't like the fact that you didn't include Debian :)
> 
> Ideally we'd also throw openSUSE into the mix, after adding it to
> the CentOS CI, but as with Ubuntu that depends on getting acces to
> extra capacity.

Sure I don't mind, though note that travis limits you to 5
concurrent jobs, so if we go over that, we effectively
double the time to complete the test run as later jobs will
have to wait.

In terms of core project post-commit CI, the main value is the
macOS job, as that's not overlapping with centos OS.

The other jobs are largely there for letting developers test
changes pre-posting for review/merge. So I don't think we need
to go over the top and add every platform into the travis
setup, just a reasonable set that will catch common problems
before review. Let the centos CI handle the full matrix.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH] travis: switch to using Ubuntu 16.04 and 18.04
Posted by Martin Kletzander 5 years, 10 months ago
On Wed, Jun 06, 2018 at 11:28:27AM +0100, Daniel P. Berrangé wrote:
>On Wed, Jun 06, 2018 at 11:27:38AM +0200, Andrea Bolognani wrote:
>> On Tue, 2018-06-05 at 18:47 +0100, Daniel P. Berrangé wrote:
>> > The container images provided by Travis only support Ubuntu 14.04,
>> > however, Travis has ability to run docker, which allows the build
>> > script to use arbitrary OS images. This takes advantage of that to
>> > convert the build over to Ubuntu 16.04 and 18.04
>>
>> Neat! I was not aware that was possible. TIL :)
>
>Yeah, nor was I until yesterday when I found someone saying this is
>the way to get off the ancient Ubuntu versions in Travis :-)
>

Oh, maybe you just forgot, we had some improvement suggestions like a year ago
but we missed some e-mails, I guess.

Have a nice day,
Martin--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH] travis: switch to using Ubuntu 16.04 and 18.04
Posted by Andrea Bolognani 5 years, 10 months ago
On Thu, 2018-06-07 at 09:52 +0200, Martin Kletzander wrote:
> On Wed, Jun 06, 2018 at 11:28:27AM +0100, Daniel P. Berrangé wrote:
> > On Wed, Jun 06, 2018 at 11:27:38AM +0200, Andrea Bolognani wrote:
> > > On Tue, 2018-06-05 at 18:47 +0100, Daniel P. Berrangé wrote:
> > > > The container images provided by Travis only support Ubuntu 14.04,
> > > > however, Travis has ability to run docker, which allows the build
> > > > script to use arbitrary OS images. This takes advantage of that to
> > > > convert the build over to Ubuntu 16.04 and 18.04
> > > 
> > > Neat! I was not aware that was possible. TIL :)
> > 
> > Yeah, nor was I until yesterday when I found someone saying this is
> > the way to get off the ancient Ubuntu versions in Travis :-)
> 
> Oh, maybe you just forgot, we had some improvement suggestions like a year ago
> but we missed some e-mails, I guess.

Sometimes stuff slips through the cracks ¯\_(ツ)_/¯

If you feel like fishing that out and checking whether any of the
suggested improvements would still apply, that would be great; that
said, outside of the minor issues pointed out during review, Dan's
approach looks pretty solid to me.

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list