.gitignore | 1 + .travis.yml | 14 +++- MAINTAINERS | 1 + Makefile | 24 +++++++ docs/devel/testing.rst | 21 ++++-- linux-user/Makefile.objs | 2 +- linux-user/exit.c | 35 ++++++++++ linux-user/qemu.h | 8 +++ linux-user/syscall.c | 10 +-- scripts/travis/coverage-summary.sh | 27 ++++++++ tests/Makefile.include | 10 --- tests/docker/Makefile.include | 74 ++++++++++++++++++++-- tests/docker/docker.py | 4 ++ tests/docker/dockerfiles/debian-bootstrap.docker | 2 - tests/docker/dockerfiles/debian-bootstrap.pre | 11 +++- .../dockerfiles/debian-powerpc-user-cross.docker | 15 +++++ tests/docker/dockerfiles/debian8-mxe.docker | 2 +- tests/docker/dockerfiles/travis.docker | 2 +- tests/docker/dockerfiles/ubuntu.docker | 8 +-- 19 files changed, 228 insertions(+), 43 deletions(-) create mode 100644 linux-user/exit.c create mode 100755 scripts/travis/coverage-summary.sh create mode 100644 tests/docker/dockerfiles/debian-powerpc-user-cross.docker
The following changes since commit 46d0885adff9b99622d72f23a8b04c298a8bf91d:
Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2018-07-03 09:49:20 +0100)
are available in the Git repository at:
https://github.com/stsquad/qemu.git tags/pull-code-coverage-and-build-tweaks-030718-1
for you to fetch changes up to e84a4f16f0fd67e5e503ceb50a9d069f7b76fcb3:
docker: add linux-user powered cross builder for QEMU (2018-07-03 11:09:56 +0100)
----------------------------------------------------------------
Code coverage and other build tweaks
- some travis speed-ups
- modernise code coverage support
- docker image cleanups
- clean-up binfmt_misc docker infrastructure
- add debian-powerpc-user-cross image for ppc32 build
----------------------------------------------------------------
Alex Bennée (14):
build-system: remove per-test GCOV reporting
.gitignore: add .gcov files
docker: add gcovr to travis image
travis: add gcovr summary for GCOV build
build-system: add clean-coverage target
build-system: add coverage-report target
linux-user: introduce preexit_cleanup
linux-user: add gcov support to preexit_cleanup
docker: filter out linux-user builds for mingw
docker: drop QEMU build-dep from bootstrap
docker: debian-bootstrap.pre allow customising of variant/url
docker: add special handling for FROM:debian-%-user targets
docker: add special rule for deboostrapped images
docker: add linux-user powered cross builder for QEMU
Philippe Mathieu-Daudé (6):
travis: do not waste time cloning unused submodules
travis: test out-of-tree builds
docker: ubuntu: Update the package list before installing new ones
docker: ubuntu: Use SDL2
docker: Clean the MXE base image
docker: Do not run tests in 'intermediate' images
.gitignore | 1 +
.travis.yml | 14 +++-
MAINTAINERS | 1 +
Makefile | 24 +++++++
docs/devel/testing.rst | 21 ++++--
linux-user/Makefile.objs | 2 +-
linux-user/exit.c | 35 ++++++++++
linux-user/qemu.h | 8 +++
linux-user/syscall.c | 10 +--
scripts/travis/coverage-summary.sh | 27 ++++++++
tests/Makefile.include | 10 ---
tests/docker/Makefile.include | 74 ++++++++++++++++++++--
tests/docker/docker.py | 4 ++
tests/docker/dockerfiles/debian-bootstrap.docker | 2 -
tests/docker/dockerfiles/debian-bootstrap.pre | 11 +++-
.../dockerfiles/debian-powerpc-user-cross.docker | 15 +++++
tests/docker/dockerfiles/debian8-mxe.docker | 2 +-
tests/docker/dockerfiles/travis.docker | 2 +-
tests/docker/dockerfiles/ubuntu.docker | 8 +--
19 files changed, 228 insertions(+), 43 deletions(-)
create mode 100644 linux-user/exit.c
create mode 100755 scripts/travis/coverage-summary.sh
create mode 100644 tests/docker/dockerfiles/debian-powerpc-user-cross.docker
--
2.17.1
On 03/07/2018 12:14, Alex Bennée wrote: > The following changes since commit 46d0885adff9b99622d72f23a8b04c298a8bf91d: > > Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2018-07-03 09:49:20 +0100) > > are available in the Git repository at: > > https://github.com/stsquad/qemu.git tags/pull-code-coverage-and-build-tweaks-030718-1 > > for you to fetch changes up to e84a4f16f0fd67e5e503ceb50a9d069f7b76fcb3: > > docker: add linux-user powered cross builder for QEMU (2018-07-03 11:09:56 +0100) > > ---------------------------------------------------------------- > Code coverage and other build tweaks > > - some travis speed-ups > - modernise code coverage support > - docker image cleanups > - clean-up binfmt_misc docker infrastructure > - add debian-powerpc-user-cross image for ppc32 build > > ---------------------------------------------------------------- > Alex Bennée (14): > build-system: remove per-test GCOV reporting > .gitignore: add .gcov files > docker: add gcovr to travis image > travis: add gcovr summary for GCOV build > build-system: add clean-coverage target > build-system: add coverage-report target > linux-user: introduce preexit_cleanup > linux-user: add gcov support to preexit_cleanup > docker: filter out linux-user builds for mingw > docker: drop QEMU build-dep from bootstrap > docker: debian-bootstrap.pre allow customising of variant/url > docker: add special handling for FROM:debian-%-user targets > docker: add special rule for deboostrapped images > docker: add linux-user powered cross builder for QEMU > > Philippe Mathieu-Daudé (6): > travis: do not waste time cloning unused submodules > travis: test out-of-tree builds > docker: ubuntu: Update the package list before installing new ones > docker: ubuntu: Use SDL2 > docker: Clean the MXE base image > docker: Do not run tests in 'intermediate' images > > .gitignore | 1 + > .travis.yml | 14 +++- > MAINTAINERS | 1 + > Makefile | 24 +++++++ > docs/devel/testing.rst | 21 ++++-- > linux-user/Makefile.objs | 2 +- > linux-user/exit.c | 35 ++++++++++ > linux-user/qemu.h | 8 +++ > linux-user/syscall.c | 10 +-- > scripts/travis/coverage-summary.sh | 27 ++++++++ > tests/Makefile.include | 10 --- > tests/docker/Makefile.include | 74 ++++++++++++++++++++-- > tests/docker/docker.py | 4 ++ > tests/docker/dockerfiles/debian-bootstrap.docker | 2 - > tests/docker/dockerfiles/debian-bootstrap.pre | 11 +++- > .../dockerfiles/debian-powerpc-user-cross.docker | 15 +++++ > tests/docker/dockerfiles/debian8-mxe.docker | 2 +- > tests/docker/dockerfiles/travis.docker | 2 +- > tests/docker/dockerfiles/ubuntu.docker | 8 +-- > 19 files changed, 228 insertions(+), 43 deletions(-) > create mode 100644 linux-user/exit.c > create mode 100755 scripts/travis/coverage-summary.sh > create mode 100644 tests/docker/dockerfiles/debian-powerpc-user-cross.docker > > -- > 2.17.1 > > A bit too fast perhaps... Can we just revert the offending patch instead of patch 11? Thanks, Paolo
Paolo Bonzini <pbonzini@redhat.com> writes: > On 03/07/2018 12:14, Alex Bennée wrote: >> The following changes since commit 46d0885adff9b99622d72f23a8b04c298a8bf91d: >> >> Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2018-07-03 09:49:20 +0100) >> >> are available in the Git repository at: >> >> https://github.com/stsquad/qemu.git tags/pull-code-coverage-and-build-tweaks-030718-1 >> >> for you to fetch changes up to e84a4f16f0fd67e5e503ceb50a9d069f7b76fcb3: >> >> docker: add linux-user powered cross builder for QEMU (2018-07-03 11:09:56 +0100) >> >> ---------------------------------------------------------------- >> Code coverage and other build tweaks >> >> - some travis speed-ups >> - modernise code coverage support >> - docker image cleanups >> - clean-up binfmt_misc docker infrastructure >> - add debian-powerpc-user-cross image for ppc32 build >> >> ---------------------------------------------------------------- >> Alex Bennée (14): >> build-system: remove per-test GCOV reporting >> .gitignore: add .gcov files >> docker: add gcovr to travis image >> travis: add gcovr summary for GCOV build >> build-system: add clean-coverage target >> build-system: add coverage-report target >> linux-user: introduce preexit_cleanup >> linux-user: add gcov support to preexit_cleanup >> docker: filter out linux-user builds for mingw >> docker: drop QEMU build-dep from bootstrap >> docker: debian-bootstrap.pre allow customising of variant/url >> docker: add special handling for FROM:debian-%-user targets >> docker: add special rule for deboostrapped images >> docker: add linux-user powered cross builder for QEMU >> >> Philippe Mathieu-Daudé (6): >> travis: do not waste time cloning unused submodules >> travis: test out-of-tree builds >> docker: ubuntu: Update the package list before installing new ones >> docker: ubuntu: Use SDL2 >> docker: Clean the MXE base image >> docker: Do not run tests in 'intermediate' images >> >> .gitignore | 1 + >> .travis.yml | 14 +++- >> MAINTAINERS | 1 + >> Makefile | 24 +++++++ >> docs/devel/testing.rst | 21 ++++-- >> linux-user/Makefile.objs | 2 +- >> linux-user/exit.c | 35 ++++++++++ >> linux-user/qemu.h | 8 +++ >> linux-user/syscall.c | 10 +-- >> scripts/travis/coverage-summary.sh | 27 ++++++++ >> tests/Makefile.include | 10 --- >> tests/docker/Makefile.include | 74 ++++++++++++++++++++-- >> tests/docker/docker.py | 4 ++ >> tests/docker/dockerfiles/debian-bootstrap.docker | 2 - >> tests/docker/dockerfiles/debian-bootstrap.pre | 11 +++- >> .../dockerfiles/debian-powerpc-user-cross.docker | 15 +++++ >> tests/docker/dockerfiles/debian8-mxe.docker | 2 +- >> tests/docker/dockerfiles/travis.docker | 2 +- >> tests/docker/dockerfiles/ubuntu.docker | 8 +-- >> 19 files changed, 228 insertions(+), 43 deletions(-) >> create mode 100644 linux-user/exit.c >> create mode 100755 scripts/travis/coverage-summary.sh >> create mode 100644 tests/docker/dockerfiles/debian-powerpc-user-cross.docker >> >> -- >> 2.17.1 >> >> > > A bit too fast perhaps... Can we just revert the offending patch > instead of patch 11? OK, I'll re-base and regen the PULL REQ. -- Alex Bennée
Alex Bennée <alex.bennee@linaro.org> writes: > The following changes since commit 46d0885adff9b99622d72f23a8b04c298a8bf91d: > > Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2018-07-03 09:49:20 +0100) > > are available in the Git repository at: > > https://github.com/stsquad/qemu.git > tags/pull-code-coverage-and-build-tweaks-030718-1 v2 sent: Subject: [PULL v2 00/20] Travis, Code Coverage and Cross Build updates Date: Wed, 4 Jul 2018 10:06:42 +0100 Message-Id: <20180704090642.3469-1-alex.bennee@linaro.org> -- Alex Bennée
From: Philippe Mathieu-Daudé <f4bug@amsat.org>
Builds only require:
- dtc
- keycodemapdb
- capstone
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: drop wget cache]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
diff --git a/.travis.yml b/.travis.yml
index bd66c18fed..134d5331fe 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -66,8 +66,7 @@ git:
before_install:
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
- - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
- - git submodule update --init --recursive
+ - git submodule update --init --recursive capstone dtc ui/keycodemapdb
before_script:
- ./configure ${CONFIG} || { cat config.log && exit 1; }
script:
--
2.17.1
From: Philippe Mathieu-Daudé <f4bug@amsat.org>
Force one config to build 'out-of-tree' (object files and executables
are created in a tree outside the project source code).
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
diff --git a/.travis.yml b/.travis.yml
index 134d5331fe..32188d51f1 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -50,6 +50,8 @@ notifications:
on_failure: always
env:
global:
+ - SRC_DIR="."
+ - BUILD_DIR="."
- TEST_CMD="make check"
- MAKEFLAGS="-j3"
matrix:
@@ -68,11 +70,15 @@ before_install:
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
- git submodule update --init --recursive capstone dtc ui/keycodemapdb
before_script:
- - ./configure ${CONFIG} || { cat config.log && exit 1; }
+ - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
+ - ${SRC_DIR}/configure ${CONFIG} || { cat config.log && exit 1; }
script:
- make ${MAKEFLAGS} && ${TEST_CMD}
matrix:
include:
+ # Test out-of-tree builds
+ - env: CONFIG="--enable-debug --enable-debug-tcg"
+ BUILD_DIR="out-of-tree/build/dir" SRC_DIR="../../.."
# Test with Clang for compile portability (Travis uses clang-5.0)
- env: CONFIG="--disable-system"
compiler: clang
--
2.17.1
These are temporary files generated on gcov runs and shouldn't be included in the source tree. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> diff --git a/.gitignore b/.gitignore index 9da3b3e626..5668d02782 100644 --- a/.gitignore +++ b/.gitignore @@ -155,6 +155,7 @@ .sdk *.gcda *.gcno +*.gcov /pc-bios/bios-pq/status /pc-bios/vgabios-pq/status /pc-bios/optionrom/linuxboot.asm -- 2.17.1
Useful for debugging if nothing else as the gcovr on the Travis images are a little old. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker index c5ad39b533..03ebfb0ef2 100644 --- a/tests/docker/dockerfiles/travis.docker +++ b/tests/docker/dockerfiles/travis.docker @@ -5,7 +5,7 @@ ENV LC_ALL en_US.UTF-8 RUN cat /etc/apt/sources.list | sed "s/# deb-src/deb-src/" >> /etc/apt/sources.list RUN apt-get update RUN apt-get -y build-dep qemu -RUN apt-get -y install device-tree-compiler python2.7 python-yaml dh-autoreconf gdb strace lsof net-tools +RUN apt-get -y install device-tree-compiler python2.7 python-yaml dh-autoreconf gdb strace lsof net-tools gcovr # Travis tools require PhantomJS / Neo4j / Maven accessible # in their PATH (QEMU build won't access them). ENV PATH /usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin -- 2.17.1
As we don't always take the normal exit path when running a guest we
can skip the normal exit destructors where gcov normally dumps it's
info. The GCC manual suggests long running programs use __gcov_dump()
to flush out the coverage state periodically so we use that here.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
diff --git a/linux-user/exit.c b/linux-user/exit.c
index aed8713fae..14e94e28fa 100644
--- a/linux-user/exit.c
+++ b/linux-user/exit.c
@@ -19,10 +19,17 @@
#include "qemu/osdep.h"
#include "qemu.h"
+#ifdef CONFIG_GCOV
+extern void __gcov_dump(void);
+#endif
+
void preexit_cleanup(CPUArchState *env, int code)
{
#ifdef TARGET_GPROF
_mcleanup();
+#endif
+#ifdef CONFIG_GCOV
+ __gcov_dump();
#endif
gdb_exit(env, code);
}
--
2.17.1
The recent change from TARGET_DIRS to TARGET_LIST (208ecb3e1) had the effect of defaulting all docker builds to the current configured set of targets. This is actually reasonable behaviour but does run into problems if you have linux-user builds configured and you want to test the windows cross builds. This commit fixes that by adding a DOCKER_FILTER_TARGETS variable which is special-cased for mingw builds so we don't pass the whole set down. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Cc: Paolo Bonzini <pbonzini@redhat.com> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 91d9665517..1813ec0781 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -20,6 +20,9 @@ DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py TESTS ?= % IMAGES ?= % +# This is used to filter targets from some docker builds +DOCKER_FILTER_TARGETS ?= + CUR_TIME := $(shell date +%Y-%m-%d-%H.%M.%S.$$$$) DOCKER_SRC_COPY := $(BUILD_DIR)/docker-src.$(CUR_TIME) @@ -108,6 +111,12 @@ $(foreach i,$(DOCKER_IMAGES) $(DOCKER_DEPRECATED_IMAGES), \ ) \ ) +# Special cases +# mingw/windows builds cannot build linux-user +docker-%-win32-cross: DOCKER_FILTER_TARGETS = %-linux-user +docker-%-win64-cross: DOCKER_FILTER_TARGETS = %-linux-user +docker-test-mingw@%: DOCKER_FILTER_TARGETS = %-linux-user + docker: @echo 'Build QEMU and run tests inside Docker containers' @echo @@ -174,7 +183,7 @@ docker-run: docker-qemu-src $(if $V,,--rm) \ $(if $(DEBUG),-ti,) \ $(if $(NETWORK),$(if $(subst $(NETWORK),,1),--net=$(NETWORK)),--net=none) \ - -e TARGET_LIST=$(subst $(SPACE),$(COMMA),$(TARGET_LIST)) \ + -e TARGET_LIST=$(subst $(SPACE),$(COMMA),$(filter-out $(DOCKER_FILTER_TARGETS),$(TARGET_LIST))) \ -e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \ -e V=$V -e J=$J -e DEBUG=$(DEBUG) \ -e SHOW_ENV=$(SHOW_ENV) \ @@ -195,7 +204,8 @@ docker-run: docker-qemu-src docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/') docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/') docker-run-%: - @$(MAKE) docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE) + @$(MAKE) docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE) DOCKER_FILTER_TARGETS=$(DOCKER_FILTER_TARGETS) + docker-clean: $(call quiet-command, $(DOCKER_SCRIPT) clean) -- 2.17.1
On 03/07/2018 12:14, Alex Bennée wrote: > The recent change from TARGET_DIRS to TARGET_LIST (208ecb3e1) had the > effect of defaulting all docker builds to the current configured set > of targets. This is actually reasonable behaviour but does run into > problems if you have linux-user builds configured and you want to test > the windows cross builds. This commit fixes that by adding a > DOCKER_FILTER_TARGETS variable which is special-cased for mingw builds > so we don't pass the whole set down. > +# Special cases > +# mingw/windows builds cannot build linux-user > +docker-%-win32-cross: DOCKER_FILTER_TARGETS = %-linux-user > +docker-%-win64-cross: DOCKER_FILTER_TARGETS = %-linux-user > +docker-test-mingw@%: DOCKER_FILTER_TARGETS = %-linux-user Some questions: 1) Any reason to keep all three? 2) Can we do it in the test script instead? 3) DEF_TARGET_LIST in the test script becomes meaningless. 4) You now have to override TARGET_LIST with "make TARGET_LIST=foo-softmmu docker-test-mingw@fedora". Does this interact badly with other uses of TARGET_LIST in the Makefile? Thanks, Paolo
Paolo Bonzini <pbonzini@redhat.com> writes: > On 03/07/2018 12:14, Alex Bennée wrote: >> The recent change from TARGET_DIRS to TARGET_LIST (208ecb3e1) had the >> effect of defaulting all docker builds to the current configured set >> of targets. This is actually reasonable behaviour but does run into >> problems if you have linux-user builds configured and you want to test >> the windows cross builds. This commit fixes that by adding a >> DOCKER_FILTER_TARGETS variable which is special-cased for mingw builds >> so we don't pass the whole set down. > >> +# Special cases >> +# mingw/windows builds cannot build linux-user >> +docker-%-win32-cross: DOCKER_FILTER_TARGETS = %-linux-user >> +docker-%-win64-cross: DOCKER_FILTER_TARGETS = %-linux-user >> +docker-test-mingw@%: DOCKER_FILTER_TARGETS = %-linux-user > > Some questions: > > 1) Any reason to keep all three? We could prune but it doesn't hurt. > > 2) Can we do it in the test script instead? > > 3) DEF_TARGET_LIST in the test script becomes meaningless. Good point. > > 4) You now have to override TARGET_LIST with "make > TARGET_LIST=foo-softmmu docker-test-mingw@fedora". Does this interact > badly with other uses of TARGET_LIST in the Makefile? Yeah I originally thought this made sense from the "docker builds the same as my configured setup" sense but your right that's overkill. > > Thanks, > > Paolo -- Alex Bennée
From: Philippe Mathieu-Daudé <f4bug@amsat.org>
Since docker caches the different layers, updating the package
list does not invalidate the previous "apt-get update" layer,
and it is likely "apt-get install" hits an outdated repository.
See https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get
This fixes:
$ make docker-image-ubuntu V=1
./tests/docker/docker.py build qemu:ubuntu tests/docker/dockerfiles/ubuntu.docker --add-current-user
Sending build context to Docker daemon 3.072kB
[...]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa_17.0.7-0ubuntu0.16.04.2_amd64.deb 404 Not Found
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa-dev_17.0.7-0ubuntu0.16.04.2_amd64.deb 404 Not Found
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get -y install $PACKAGES' returned a non-zero code: 100
tests/docker/Makefile.include:40: recipe for target 'docker-image-ubuntu' failed
make: *** [docker-image-ubuntu] Error 1
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index dabbf2a8a4..c03520ce3f 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -1,7 +1,6 @@
FROM ubuntu:16.04
RUN echo "deb http://archive.ubuntu.com/ubuntu/ trusty universe multiverse" >> \
/etc/apt/sources.list
-RUN apt-get update
ENV PACKAGES flex bison \
libusb-1.0-0-dev libiscsi-dev librados-dev libncurses5-dev libncursesw5-dev \
libseccomp-dev libgnutls-dev libssh2-1-dev libspice-server-dev \
@@ -13,6 +12,7 @@ ENV PACKAGES flex bison \
libjemalloc-dev libcacard-dev libusbredirhost-dev libnfs-dev libcap-dev libattr1-dev \
texinfo \
gettext git make ccache python-yaml gcc clang sparse
-RUN apt-get -y install $PACKAGES
+RUN apt-get update && \
+ apt-get -y install $PACKAGES
RUN dpkg -l $PACKAGES | sort > /packages.txt
ENV FEATURES clang pyyaml
--
2.17.1
From: Philippe Mathieu-Daudé <f4bug@amsat.org>
Do not test the deprecated API versions (see cabd35840749d).
Debian MXE MinGW cross images are already using SDL2.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index c03520ce3f..7d724e7f53 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -5,7 +5,7 @@ ENV PACKAGES flex bison \
libusb-1.0-0-dev libiscsi-dev librados-dev libncurses5-dev libncursesw5-dev \
libseccomp-dev libgnutls-dev libssh2-1-dev libspice-server-dev \
libspice-protocol-dev libnss3-dev libfdt-dev \
- libgtk-3-dev libvte-2.91-dev libsdl1.2-dev libpng12-dev libpixman-1-dev \
+ libgtk-3-dev libvte-2.91-dev libsdl2-dev libpng12-dev libpixman-1-dev \
libvdeplug-dev liblzo2-dev libsnappy-dev libbz2-dev libxen-dev librdmacm-dev libibverbs-dev \
libsasl2-dev libjpeg-turbo8-dev xfslibs-dev libcap-ng-dev libbrlapi-dev libcurl4-gnutls-dev \
libbluetooth-dev librbd-dev libaio-dev glusterfs-common libnuma-dev libepoxy-dev libdrm-dev libgbm-dev \
@@ -15,4 +15,4 @@ ENV PACKAGES flex bison \
RUN apt-get update && \
apt-get -y install $PACKAGES
RUN dpkg -l $PACKAGES | sort > /packages.txt
-ENV FEATURES clang pyyaml
+ENV FEATURES clang pyyaml sdl2
--
2.17.1
From: Philippe Mathieu-Daudé <f4bug@amsat.org>
Using the duplicated same package is confusing.
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
diff --git a/tests/docker/dockerfiles/debian8-mxe.docker b/tests/docker/dockerfiles/debian8-mxe.docker
index 9b8e577b03..2df4cc8c5c 100644
--- a/tests/docker/dockerfiles/debian8-mxe.docker
+++ b/tests/docker/dockerfiles/debian8-mxe.docker
@@ -14,6 +14,6 @@ RUN apt-get update
RUN DEBIAN_FRONTEND=noninteractive eatmydata \
apt-get install -y --no-install-recommends \
libpython2.7-stdlib \
- $(apt-get -s install -y --no-install-recommends gw32.shared-mingw-w64 gw32.shared-mingw-w64 | egrep "^Inst mxe-x86-64-unknown-" | cut -d\ -f2)
+ $(apt-get -s install -y --no-install-recommends gw32.shared-mingw-w64 | egrep "^Inst mxe-x86-64-unknown-" | cut -d\ -f2)
ENV PATH $PATH:/usr/lib/mxe/usr/bin/
--
2.17.1
From: Philippe Mathieu-Daudé <f4bug@amsat.org> We can still build the DOCKER_INTERMEDIATE_IMAGES images, but they won't appear in 'make test*@$IMAGE'. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 1813ec0781..8641f5da2c 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -5,6 +5,8 @@ DOCKER_SUFFIX := .docker DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles DOCKER_DEPRECATED_IMAGES := debian +# we don't run tests on intermediate images (used as base by another image) +DOCKER_INTERMEDIATE_IMAGES := debian8 debian9 debian8-mxe debian-ports debian-sid DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))) DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive builds @@ -101,7 +103,7 @@ docker-image-travis: NOUSER=1 docker-image-tricore-cross: docker-image-debian9 # Expand all the pre-requistes for each docker image and test combination -$(foreach i,$(DOCKER_IMAGES) $(DOCKER_DEPRECATED_IMAGES), \ +$(foreach i,$(filter-out $(DOCKER_INTERMEDIATE_IMAGES),$(DOCKER_IMAGES) $(DOCKER_DEPRECATED_IMAGES)), \ $(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \ $(eval .PHONY: docker-$t@$i) \ $(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \ -- 2.17.1
This is best done with any child images that actually need it. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> diff --git a/tests/docker/dockerfiles/debian-bootstrap.docker b/tests/docker/dockerfiles/debian-bootstrap.docker index 3a9125e497..14212b9cf4 100644 --- a/tests/docker/dockerfiles/debian-bootstrap.docker +++ b/tests/docker/dockerfiles/debian-bootstrap.docker @@ -17,5 +17,3 @@ RUN /debootstrap/debootstrap --second-stage # At this point we can install additional packages if we want # Duplicate deb line as deb-src RUN cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list -RUN apt-get update -RUN apt-get -y build-dep qemu -- 2.17.1
We default to the buildd variant as most of our images are for
building. However lets give the user the ability to specify "minbase"
if they want to create a simple base image for experimentation.
Allowing the tweaking of DEB_URL means we can also bootstrap other
Debian based OS's. For example:
make docker-binfmt-image-debian-ubuntu-bionic-arm64 \
DEB_ARCH=arm64 DEB_TYPE=bionic \
DEB_VARIANT=minbase DEB_URL=http://ports.ubuntu.com/ \
EXECUTABLE=./aarch64-linux-user/qemu-aarch64
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
diff --git a/tests/docker/dockerfiles/debian-bootstrap.pre b/tests/docker/dockerfiles/debian-bootstrap.pre
index 7c76dce663..56e1aa7a21 100755
--- a/tests/docker/dockerfiles/debian-bootstrap.pre
+++ b/tests/docker/dockerfiles/debian-bootstrap.pre
@@ -32,6 +32,15 @@ if [ -z "${DEB_TYPE}" ]; then
fi
+# The following allow finer grain control over the defaults
+if [ -z "${DEB_VARIANT}" ]; then
+ DEB_VARIANT=buildd
+fi
+
+if [ -z "${DEB_URL}" ]; then
+ DEB_URL="http://httpredir.debian.org/debian"
+fi
+
# We check in order for
#
# - DEBOOTSTRAP_DIR pointing at a development checkout
@@ -107,5 +116,5 @@ fi
echo "Building a rootfs using ${FAKEROOT} and ${DEBOOTSTRAP} ${DEB_ARCH}/${DEB_TYPE}"
-${FAKEROOT} ${DEBOOTSTRAP} --variant=buildd --foreign --arch=$DEB_ARCH $DEB_TYPE . http://httpredir.debian.org/debian || exit 1
+${FAKEROOT} ${DEBOOTSTRAP} --variant=$DEB_VARIANT --foreign --arch=$DEB_ARCH $DEB_TYPE . $DEB_URL || exit 1
exit 0
--
2.17.1
These will have been build with debootstrap so we need to check
against the debian-bootstrap dockerfile. This does mean sticking to
debian-FOO-user as the naming conventions for boot-strapped images.
The actual cross image is built on top.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index b279836154..69e7130db7 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -113,6 +113,10 @@ def _copy_binary_with_libs(src, dest_dir):
_copy_with_mkdir(l , dest_dir, so_path)
def _read_qemu_dockerfile(img_name):
+ # special case for Debian linux-user images
+ if img_name.startswith("debian") and img_name.endswith("user"):
+ img_name = "debian-bootstrap"
+
df = os.path.join(os.path.dirname(__file__), "dockerfiles",
img_name + ".docker")
return open(df, "r").read()
--
2.17.1
We might as well have a custom rule for this. For one thing the dependencies are different. As the primary dependency for docker-image-% could never be docker-image-debian-bootstrap we can drop that test in the main rule as well. Missing EXECUTABLE, DEB_ARCH and DEB_TYPE are treated as hard faults now. We also error out if the EXECUTABLE file isn't there. We should really do this with a dependency on any source rules but currently subdir-FOO-linux-user isn't enough on a clean build. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 8641f5da2c..d43af0db32 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -49,9 +49,6 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker "CHECK", "$*") else docker-image-%: $(DOCKER_FILES_DIR)/%.docker - @if test "$@" = docker-image-debian-bootstrap -a -z "$(EXECUTABLE)"; then \ - echo WARNING: EXECUTABLE is not set, debootstrap may fail. 2>&1 ; \ - fi $(call quiet-command,\ $(DOCKER_SCRIPT) build qemu:$* $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ @@ -59,6 +56,28 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\ $(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\ "BUILD","$*") + +# Special rule for debootstraped binfmt linux-user images +docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker + $(if $(EXECUTABLE),,\ + $(error EXECUTABLE not set, debootstrap of debian-$* would fail)) + @if ! test -f "$(EXECUTABLE)"; \ + $(error Please build $(EXECUTABLE) first) \ + fi + $(if $(DEB_ARCH),,\ + $(error DEB_ARCH not set, debootstrap of debian-$* would fail)) + $(if $(DEB_TYPE),,\ + $(error DEB_TYPE not set, debootstrap of debian-$* would fail)) + $(call quiet-command, \ + DEB_ARCH=$(DEB_ARCH) \ + DEB_TYPE=$(DEB_TYPE) \ + $(DOCKER_SCRIPT) build qemu:debian-$* $< \ + $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ + $(if $(NOUSER),,--add-current-user) \ + $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES)) \ + $(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)), \ + "BUILD","binfmt debian-$* (debootstrapped)") + endif # Enforce dependencies for composite images -- 2.17.1
© 2016 - 2025 Red Hat, Inc.