.gitlab-ci.d/crossbuild-template.yml | 3 +- .gitlab-ci.d/crossbuilds.yml | 4 + docs/about/removed-features.rst | 183 ++++++++++++++++++++++++++++++++++- scripts/oss-fuzz/build.sh | 24 ++--- storage-daemon/meson.build | 8 +- tests/qtest/meson.build | 7 +- tests/qtest/vhost-user-blk-test.c | 8 ++ 7 files changed, 215 insertions(+), 22 deletions(-)
Hi Peter,
in case we're going to have an -rc4, here's a pull request that contains
the fixes for getting the gitlab-CI green again. I also added some doc
updates since they should be completely riskless. But if we won't have an
rc4 due to other reasons, this pull request here certainly also does not
justify another RC, so please ignore this PR in that case.
The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71:
Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100)
are available in the Git repository at:
https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11
for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee:
docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200)
CI run can be seen here:
https://gitlab.com/thuth/qemu/-/pipelines/351602605
----------------------------------------------------------------
* Fixes for the gitlab-CI (fix the hanging build-oss-fuzz pipeline)
* Add documentation about features that have been removed in older versions
----------------------------------------------------------------
Alexander Bulekov (1):
fuzz: avoid building twice, when running on gitlab
Daniel P. Berrangé (2):
gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds
gitlab: skip many more targets in windows cross builds
Thomas Huth (8):
storage-daemon: Add missing build dependency to the vhost-user-blk-test
tests/qtest/vhost-user-blk-test: Check whether qemu-storage-daemon is available
docs/about/removed-features: Document removed CLI options from QEMU v2.12
docs/about/removed-features: Document removed CLI options from QEMU v3.0
docs/about/removed-features: Document removed CLI options from QEMU v3.1
docs/about/removed-features: Document removed HMP commands from QEMU v2.12
docs/about/removed-features: Document removed devices from older QEMU versions
docs/about/removed-features: Document removed machines from older QEMU versions
.gitlab-ci.d/crossbuild-template.yml | 3 +-
.gitlab-ci.d/crossbuilds.yml | 4 +
docs/about/removed-features.rst | 183 ++++++++++++++++++++++++++++++++++-
scripts/oss-fuzz/build.sh | 24 ++---
storage-daemon/meson.build | 8 +-
tests/qtest/meson.build | 7 +-
tests/qtest/vhost-user-blk-test.c | 8 ++
7 files changed, 215 insertions(+), 22 deletions(-)
On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote: > > Hi Peter, > > in case we're going to have an -rc4, here's a pull request that contains > the fixes for getting the gitlab-CI green again. I also added some doc > updates since they should be completely riskless. But if we won't have an > rc4 due to other reasons, this pull request here certainly also does not > justify another RC, so please ignore this PR in that case. > > The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71: > > Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100) > > are available in the Git repository at: > > https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11 > > for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee: > > docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200) > > CI run can be seen here: > > https://gitlab.com/thuth/qemu/-/pipelines/351602605 > > ---------------------------------------------------------------- > * Fixes for the gitlab-CI (fix the hanging build-oss-fuzz pipeline) > * Add documentation about features that have been removed in older versions > Applied, thanks. Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1 for any user-visible changes. -- PMM
On 210816 1001, Peter Maydell wrote: > On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote: > > > > Hi Peter, > > > > in case we're going to have an -rc4, here's a pull request that contains > > the fixes for getting the gitlab-CI green again. I also added some doc > > updates since they should be completely riskless. But if we won't have an > > rc4 due to other reasons, this pull request here certainly also does not > > justify another RC, so please ignore this PR in that case. > > > > The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71: > > > > Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100) > > > > are available in the Git repository at: > > > > https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11 > > > > for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee: > > > > docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200) > > > > CI run can be seen here: > > > > https://gitlab.com/thuth/qemu/-/pipelines/351602605 > > > > ---------------------------------------------------------------- > > * Fixes for the gitlab-CI (fix the hanging build-oss-fuzz pipeline) > > * Add documentation about features that have been removed in older versions > > > > > Applied, thanks. > > Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1 > for any user-visible changes. https://gitlab.com/qemu-project/qemu/-/jobs/1505950978 Looks like build-oss-fuzz is still timing out, even without the issue in the vhost-usr-blk test. At this point the job should essentially just build + test qemu-system-i386 with some extra time spent on linking the fuzzer and briefly running through all the fuzzer configs. Maybe the only way to make this work is to split the job into a build + test stage? > > -- PMM >
On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote: > On 210816 1001, Peter Maydell wrote: > > On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote: > > > > > > Hi Peter, > > > > > > in case we're going to have an -rc4, here's a pull request that contains > > > the fixes for getting the gitlab-CI green again. I also added some doc > > > updates since they should be completely riskless. But if we won't have an > > > rc4 due to other reasons, this pull request here certainly also does not > > > justify another RC, so please ignore this PR in that case. > > > > > > The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71: > > > > > > Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100) > > > > > > are available in the Git repository at: > > > > > > https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11 > > > > > > for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee: > > > > > > docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200) > > > > > > CI run can be seen here: > > > > > > https://gitlab.com/thuth/qemu/-/pipelines/351602605 > > > > > > ---------------------------------------------------------------- > > > * Fixes for the gitlab-CI (fix the hanging build-oss-fuzz pipeline) > > > * Add documentation about features that have been removed in older versions > > > > > > > > > Applied, thanks. > > > > Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1 > > for any user-visible changes. > > https://gitlab.com/qemu-project/qemu/-/jobs/1505950978 > Looks like build-oss-fuzz is still timing out, even without the issue > in the vhost-usr-blk test. At this point the job should essentially just > build + test qemu-system-i386 with some extra time spent on linking > the fuzzer and briefly running through all the fuzzer configs. Maybe the > only way to make this work is to split the job into a build + test > stage? At this point I think we should just disable the job in gitlab entirely. We've spent too long debugging this, while leaving CI red for everyone. Whomever is interested in this can then work to find a way to make it reliable and request it be re-enabled once confident that it will work. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 210816 1211, Daniel P. Berrangé wrote: > On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote: > > On 210816 1001, Peter Maydell wrote: > > > On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote: > > > > > > > > Hi Peter, > > > > > > > > in case we're going to have an -rc4, here's a pull request that contains > > > > the fixes for getting the gitlab-CI green again. I also added some doc > > > > updates since they should be completely riskless. But if we won't have an > > > > rc4 due to other reasons, this pull request here certainly also does not > > > > justify another RC, so please ignore this PR in that case. > > > > > > > > The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71: > > > > > > > > Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100) > > > > > > > > are available in the Git repository at: > > > > > > > > https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11 > > > > > > > > for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee: > > > > > > > > docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200) > > > > > > > > CI run can be seen here: > > > > > > > > https://gitlab.com/thuth/qemu/-/pipelines/351602605 > > > > > > > > ---------------------------------------------------------------- > > > > * Fixes for the gitlab-CI (fix the hanging build-oss-fuzz pipeline) > > > > * Add documentation about features that have been removed in older versions > > > > > > > > > > > > > Applied, thanks. > > > > > > Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1 > > > for any user-visible changes. > > > > https://gitlab.com/qemu-project/qemu/-/jobs/1505950978 > > Looks like build-oss-fuzz is still timing out, even without the issue > > in the vhost-usr-blk test. At this point the job should essentially just > > build + test qemu-system-i386 with some extra time spent on linking > > the fuzzer and briefly running through all the fuzzer configs. Maybe the > > only way to make this work is to split the job into a build + test > > stage? > > At this point I think we should just disable the job in gitlab entirely. > We've spent too long debugging this, while leaving CI red for everyone. > > Whomever is interested in this can then work to find a way to make it > reliable and request it be re-enabled once confident that it will work. > On my mirror the job succeeded in 41 minutes... I guess you have to get lucky with scheduling/ambient load. https://gitlab.com/a1xndr/qemu/-/jobs/1506197531 If all we want is to check that oss-fuzz builds work while keeping green CI, the only thing that needs to be done is to remove the "make check check-qtest-i386 check-unit" from the job. The problem is that QEMU has a ton of nice tests, and by simply building with --enable-sanitizers, these tests become even more useful because we can catch all sorts of memory-corruptions. In fact, some of the regression tests require --enable-sanitizers to catch issues. When I looked at this originally, build-oss-fuzz was the only job that was running with any sanitizers. Now, it looks like at least ubuntu-18.04-s390x-clang and ubuntu-20.04-aarch64-clang also build with ASAN. Is there some other i386 build (gcc or clang) where we could add --enable-sanitizers? -Alex
On 8/16/21 1:30 PM, Alexander Bulekov wrote: > On 210816 1211, Daniel P. Berrangé wrote: >> On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote: >>> https://gitlab.com/qemu-project/qemu/-/jobs/1505950978 >>> Looks like build-oss-fuzz is still timing out, even without the issue >>> in the vhost-usr-blk test. At this point the job should essentially just >>> build + test qemu-system-i386 with some extra time spent on linking >>> the fuzzer and briefly running through all the fuzzer configs. Maybe the >>> only way to make this work is to split the job into a build + test >>> stage? >> >> At this point I think we should just disable the job in gitlab entirely. >> We've spent too long debugging this, while leaving CI red for everyone. >> >> Whomever is interested in this can then work to find a way to make it >> reliable and request it be re-enabled once confident that it will work. >> > > On my mirror the job succeeded in 41 minutes... I guess you have to get > lucky with scheduling/ambient load. > https://gitlab.com/a1xndr/qemu/-/jobs/1506197531 TBH I stopped looking at this job console out because it fails too often in my pipelines :(
On 210816 1340, Philippe Mathieu-Daudé wrote: > On 8/16/21 1:30 PM, Alexander Bulekov wrote: > > On 210816 1211, Daniel P. Berrangé wrote: > >> On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote: > > >>> https://gitlab.com/qemu-project/qemu/-/jobs/1505950978 > >>> Looks like build-oss-fuzz is still timing out, even without the issue > >>> in the vhost-usr-blk test. At this point the job should essentially just > >>> build + test qemu-system-i386 with some extra time spent on linking > >>> the fuzzer and briefly running through all the fuzzer configs. Maybe the > >>> only way to make this work is to split the job into a build + test > >>> stage? > >> > >> At this point I think we should just disable the job in gitlab entirely. > >> We've spent too long debugging this, while leaving CI red for everyone. > >> > >> Whomever is interested in this can then work to find a way to make it > >> reliable and request it be re-enabled once confident that it will work. > >> > > > > On my mirror the job succeeded in 41 minutes... I guess you have to get > > lucky with scheduling/ambient load. > > https://gitlab.com/a1xndr/qemu/-/jobs/1506197531 > > TBH I stopped looking at this job console out because it fails too often > in my pipelines :( > Of course if the job times out 50% of the time, nobody will want to look at it. And as a result, issues like the one with the vhost-user-blk test are unnoticed. My hope was that removing the redundant build would take care of timeouts. Maybe if we find that this new sporadic timeout issue is also due to some test-failure, the job will again be useful for someone. -Alex
On Mon, Aug 16, 2021 at 07:30:59AM -0400, Alexander Bulekov wrote: > On 210816 1211, Daniel P. Berrangé wrote: > > On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote: > > > On 210816 1001, Peter Maydell wrote: > > > > On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote: > > > > > > > > > > Hi Peter, > > > > > > > > > > in case we're going to have an -rc4, here's a pull request that contains > > > > > the fixes for getting the gitlab-CI green again. I also added some doc > > > > > updates since they should be completely riskless. But if we won't have an > > > > > rc4 due to other reasons, this pull request here certainly also does not > > > > > justify another RC, so please ignore this PR in that case. > > > > > > > > > > The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71: > > > > > > > > > > Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100) > > > > > > > > > > are available in the Git repository at: > > > > > > > > > > https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11 > > > > > > > > > > for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee: > > > > > > > > > > docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200) > > > > > > > > > > CI run can be seen here: > > > > > > > > > > https://gitlab.com/thuth/qemu/-/pipelines/351602605 > > > > > > > > > > ---------------------------------------------------------------- > > > > > * Fixes for the gitlab-CI (fix the hanging build-oss-fuzz pipeline) > > > > > * Add documentation about features that have been removed in older versions > > > > > > > > > > > > > > > > > Applied, thanks. > > > > > > > > Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1 > > > > for any user-visible changes. > > > > > > https://gitlab.com/qemu-project/qemu/-/jobs/1505950978 > > > Looks like build-oss-fuzz is still timing out, even without the issue > > > in the vhost-usr-blk test. At this point the job should essentially just > > > build + test qemu-system-i386 with some extra time spent on linking > > > the fuzzer and briefly running through all the fuzzer configs. Maybe the > > > only way to make this work is to split the job into a build + test > > > stage? > > > > At this point I think we should just disable the job in gitlab entirely. > > We've spent too long debugging this, while leaving CI red for everyone. > > > > Whomever is interested in this can then work to find a way to make it > > reliable and request it be re-enabled once confident that it will work. > > > > On my mirror the job succeeded in 41 minutes... I guess you have to get > lucky with scheduling/ambient load. > https://gitlab.com/a1xndr/qemu/-/jobs/1506197531 I don't think load would make that much of a difference. It smells like there is still some non-deterministic bug in there that is causing a hang of the job. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 16/08/2021 13.11, Daniel P. Berrangé wrote: > On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote: >> On 210816 1001, Peter Maydell wrote: >>> On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote: >>>> >>>> Hi Peter, >>>> >>>> in case we're going to have an -rc4, here's a pull request that contains >>>> the fixes for getting the gitlab-CI green again. I also added some doc >>>> updates since they should be completely riskless. But if we won't have an >>>> rc4 due to other reasons, this pull request here certainly also does not >>>> justify another RC, so please ignore this PR in that case. >>>> >>>> The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71: >>>> >>>> Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100) >>>> >>>> are available in the Git repository at: >>>> >>>> https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11 >>>> >>>> for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee: >>>> >>>> docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200) >>>> >>>> CI run can be seen here: >>>> >>>> https://gitlab.com/thuth/qemu/-/pipelines/351602605 >>>> >>>> ---------------------------------------------------------------- >>>> * Fixes for the gitlab-CI (fix the hanging build-oss-fuzz pipeline) >>>> * Add documentation about features that have been removed in older versions >>>> >>> >>> >>> Applied, thanks. >>> >>> Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1 >>> for any user-visible changes. >> >> https://gitlab.com/qemu-project/qemu/-/jobs/1505950978 >> Looks like build-oss-fuzz is still timing out, even without the issue >> in the vhost-usr-blk test. It worked fine in the staging branch, finished within 40 minutes: https://gitlab.com/qemu-project/qemu/-/jobs/1504868929 >> At this point the job should essentially just >> build + test qemu-system-i386 with some extra time spent on linking >> the fuzzer and briefly running through all the fuzzer configs. Maybe the >> only way to make this work is to split the job into a build + test >> stage? > > At this point I think we should just disable the job in gitlab entirely. > We've spent too long debugging this, while leaving CI red for everyone. I don't think so. That seems to be a new, unrelated problem: Running test qtest-i386/usb-hcd-ehci-test socket_accept failed: Resource temporarily unavailable ** ERROR:../tests/qtest/libqtest.c:319:qtest_init_without_qmp_handshake: assertion failed: (s->fd >= 0 && s->qmp_fd >= 0) ERROR qtest-i386/usb-hcd-ehci-test - Bail out! ERROR:../tests/qtest/libqtest.c:319:qtest_init_without_qmp_handshake: assertion failed: (s->fd >= 0 && s->qmp_fd >= 0) That "socket_accept failed: Resource temporarily unavailable" sounds like the build machine was maybe temporarily overloaded? Question is why the test framework was running into a timeout afterwards here instead of bailing out immediately? Thomas
© 2016 - 2024 Red Hat, Inc.