.gitlab-ci.d/cirrus/build.yml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
The meson log files can get very big, especially if running the tests in
verbose mode. So dumping those logs to the console was a bad idea, since
gitlab truncates the output if it is getting too big. Let's publish the
logs as artifacts instead. This has the disadvantage that you have to
look up the logs on cirrus-ci.com now instead, but that's still better
than not having the important part of the log at all since it got
truncated.
Fixes: 998f334722 ("gitlab: show testlog.txt contents ...")
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
Note: I also tried to publish the junit xml files as artifacts
instead, but while the cirrus-ci docs claim to support it, I only
got unreadable XML output in my browser that way, so the .txt
files look like the better option to me.
.gitlab-ci.d/cirrus/build.yml | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/.gitlab-ci.d/cirrus/build.yml b/.gitlab-ci.d/cirrus/build.yml
index 7ef6af8d33..a9444902ec 100644
--- a/.gitlab-ci.d/cirrus/build.yml
+++ b/.gitlab-ci.d/cirrus/build.yml
@@ -32,6 +32,9 @@ build_task:
- $MAKE -j$(sysctl -n hw.ncpu)
- for TARGET in $TEST_TARGETS ;
do
- $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1
- || { cat meson-logs/testlog.txt; exit 1; } ;
+ $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 ;
done
+ always:
+ build_result_artifacts:
+ path: build/meson-logs/*log.txt
+ type: text/plain
--
2.31.1
Thomas Huth <thuth@redhat.com> writes: > The meson log files can get very big, especially if running the tests in > verbose mode. So dumping those logs to the console was a bad idea, since > gitlab truncates the output if it is getting too big. Let's publish the > logs as artifacts instead. This has the disadvantage that you have to > look up the logs on cirrus-ci.com now instead, but that's still better > than not having the important part of the log at all since it got > truncated. > > Fixes: 998f334722 ("gitlab: show testlog.txt contents ...") > Signed-off-by: Thomas Huth <thuth@redhat.com> Queued to testing/next, thanks. -- Alex Bennée Virtualisation Tech Lead @ Linaro
On Wed, Feb 15, 2023 at 03:25:03PM +0100, Thomas Huth wrote: > The meson log files can get very big, especially if running the tests in > verbose mode. So dumping those logs to the console was a bad idea, since > gitlab truncates the output if it is getting too big. Let's publish the > logs as artifacts instead. This has the disadvantage that you have to > look up the logs on cirrus-ci.com now instead, but that's still better > than not having the important part of the log at all since it got > truncated. Having to go over to cirrus-ci.com is pretty awful user experiance, especially as there's no direct link. > Fixes: 998f334722 ("gitlab: show testlog.txt contents ...") > Signed-off-by: Thomas Huth <thuth@redhat.com> > --- > Note: I also tried to publish the junit xml files as artifacts > instead, but while the cirrus-ci docs claim to support it, I only > got unreadable XML output in my browser that way, so the .txt > files look like the better option to me. > > .gitlab-ci.d/cirrus/build.yml | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/.gitlab-ci.d/cirrus/build.yml b/.gitlab-ci.d/cirrus/build.yml > index 7ef6af8d33..a9444902ec 100644 > --- a/.gitlab-ci.d/cirrus/build.yml > +++ b/.gitlab-ci.d/cirrus/build.yml > @@ -32,6 +32,9 @@ build_task: > - $MAKE -j$(sysctl -n hw.ncpu) > - for TARGET in $TEST_TARGETS ; > do > - $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 > - || { cat meson-logs/testlog.txt; exit 1; } ; > + $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 ; > done > + always: > + build_result_artifacts: > + path: build/meson-logs/*log.txt > + type: text/plain Does it have to be either/or, can't we do both ? 95% of the time the truncated testlog.txt is sufficient on its own. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 15/02/2023 15.58, Daniel P. Berrangé wrote: > On Wed, Feb 15, 2023 at 03:25:03PM +0100, Thomas Huth wrote: >> The meson log files can get very big, especially if running the tests in >> verbose mode. So dumping those logs to the console was a bad idea, since >> gitlab truncates the output if it is getting too big. Let's publish the >> logs as artifacts instead. This has the disadvantage that you have to >> look up the logs on cirrus-ci.com now instead, but that's still better >> than not having the important part of the log at all since it got >> truncated. > > Having to go over to cirrus-ci.com is pretty awful user experiance, > especially as there's no direct link. It's not that bad, see e.g.: https://gitlab.com/thuth/qemu/-/jobs/3775523498 The log shows a link to cirrus-ci.com at the end: Build failed: https://cirrus-ci.com/build/4811273133621248 If you click on that URL, you've just got to go into the "build" and click through the build artifacts to get to the log that you want. >> + type: text/plain > > Does it have to be either/or, can't we do both ? > > 95% of the time the truncated testlog.txt is sufficient on its own. I'd say 95% of the time the normal console output (without the "cat .../testlog.txt") is already enough since meson prints the stdout and stderr of the failing tests to the console already. FWIW, this is the test run with the truncated output that Peter complained about earlier today in IRC: https://gitlab.com/qemu-project/qemu/-/jobs/3768540680 Even if you click on the "complete raw" link there, you'll have a hard time finding the information that you are interested in. So I'd really prefer to not dump testlog.txt by default and only have it via the artifacts instead. Thomas
On Wed, 15 Feb 2023 at 15:33, Thomas Huth <thuth@redhat.com> wrote: > > On 15/02/2023 15.58, Daniel P. Berrangé wrote: > > On Wed, Feb 15, 2023 at 03:25:03PM +0100, Thomas Huth wrote: > >> The meson log files can get very big, especially if running the tests in > >> verbose mode. So dumping those logs to the console was a bad idea, since > >> gitlab truncates the output if it is getting too big. Let's publish the > >> logs as artifacts instead. This has the disadvantage that you have to > >> look up the logs on cirrus-ci.com now instead, but that's still better > >> than not having the important part of the log at all since it got > >> truncated. > > > > Having to go over to cirrus-ci.com is pretty awful user experiance, > > especially as there's no direct link. > > It's not that bad, see e.g.: > > https://gitlab.com/thuth/qemu/-/jobs/3775523498 > > The log shows a link to cirrus-ci.com at the end: > > Build failed: https://cirrus-ci.com/build/4811273133621248 > > If you click on that URL, you've just got to go into the "build" and click > through the build artifacts to get to the log that you want. Could somebody write up some documentation for how to get from "top level gitlab CI pipeline page" to "detailed logs for a job failure", please? I'm finding that the UI for this (both Cirrus and gitlab) is very non-obvious, so a writeup like the above of "follow this link from the log, click this button, go look at build artefact X" would be very helpful. Somewhere in https://wiki.qemu.org/Merges is probably a convenient place for thsi. thanks -- PMM
On Wed, Feb 15, 2023 at 04:33:42PM +0100, Thomas Huth wrote: > On 15/02/2023 15.58, Daniel P. Berrangé wrote: > > On Wed, Feb 15, 2023 at 03:25:03PM +0100, Thomas Huth wrote: > > > The meson log files can get very big, especially if running the tests in > > > verbose mode. So dumping those logs to the console was a bad idea, since > > > gitlab truncates the output if it is getting too big. Let's publish the > > > logs as artifacts instead. This has the disadvantage that you have to > > > look up the logs on cirrus-ci.com now instead, but that's still better > > > than not having the important part of the log at all since it got > > > truncated. > > > > Having to go over to cirrus-ci.com is pretty awful user experiance, > > especially as there's no direct link. > > It's not that bad, see e.g.: > > https://gitlab.com/thuth/qemu/-/jobs/3775523498 > > The log shows a link to cirrus-ci.com at the end: > > Build failed: https://cirrus-ci.com/build/4811273133621248 > > If you click on that URL, you've just got to go into the "build" and click > through the build artifacts to get to the log that you want. > > > > + type: text/plain > > > > Does it have to be either/or, can't we do both ? > > > > 95% of the time the truncated testlog.txt is sufficient on its own. > > I'd say 95% of the time the normal console output (without the "cat > .../testlog.txt") is already enough since meson prints the stdout and stderr > of the failing tests to the console already. > > FWIW, this is the test run with the truncated output that Peter complained > about earlier today in IRC: > > https://gitlab.com/qemu-project/qemu/-/jobs/3768540680 > > Even if you click on the "complete raw" link there, you'll have a hard time > finding the information that you are interested in. So I'd really prefer to > not dump testlog.txt by default and only have it via the artifacts instead. Ok, you convinced me. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Hi Thomas, On 15/2/23 15:25, Thomas Huth wrote: > The meson log files can get very big, especially if running the tests in > verbose mode. So dumping those logs to the console was a bad idea, since > gitlab truncates the output if it is getting too big. Let's publish the > logs as artifacts instead. This has the disadvantage that you have to > look up the logs on cirrus-ci.com now instead, but that's still better > than not having the important part of the log at all since it got > truncated. > > Fixes: 998f334722 ("gitlab: show testlog.txt contents ...") > Signed-off-by: Thomas Huth <thuth@redhat.com> > --- > Note: I also tried to publish the junit xml files as artifacts > instead, but while the cirrus-ci docs claim to support it, I only > got unreadable XML output in my browser that way, so the .txt > files look like the better option to me. > > .gitlab-ci.d/cirrus/build.yml | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/.gitlab-ci.d/cirrus/build.yml b/.gitlab-ci.d/cirrus/build.yml > index 7ef6af8d33..a9444902ec 100644 > --- a/.gitlab-ci.d/cirrus/build.yml > +++ b/.gitlab-ci.d/cirrus/build.yml > @@ -32,6 +32,9 @@ build_task: > - $MAKE -j$(sysctl -n hw.ncpu) > - for TARGET in $TEST_TARGETS ; > do > - $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 > - || { cat meson-logs/testlog.txt; exit 1; } ; > + $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 ; > done > + always: > + build_result_artifacts: > + path: build/meson-logs/*log.txt > + type: text/plain How long is that log kept available?
On 15/02/2023 15.57, Philippe Mathieu-Daudé wrote: > Hi Thomas, > > On 15/2/23 15:25, Thomas Huth wrote: >> The meson log files can get very big, especially if running the tests in >> verbose mode. So dumping those logs to the console was a bad idea, since >> gitlab truncates the output if it is getting too big. Let's publish the >> logs as artifacts instead. This has the disadvantage that you have to >> look up the logs on cirrus-ci.com now instead, but that's still better >> than not having the important part of the log at all since it got >> truncated. >> >> Fixes: 998f334722 ("gitlab: show testlog.txt contents ...") >> Signed-off-by: Thomas Huth <thuth@redhat.com> >> --- >> Note: I also tried to publish the junit xml files as artifacts >> instead, but while the cirrus-ci docs claim to support it, I only >> got unreadable XML output in my browser that way, so the .txt >> files look like the better option to me. >> >> .gitlab-ci.d/cirrus/build.yml | 7 +++++-- >> 1 file changed, 5 insertions(+), 2 deletions(-) >> >> diff --git a/.gitlab-ci.d/cirrus/build.yml b/.gitlab-ci.d/cirrus/build.yml >> index 7ef6af8d33..a9444902ec 100644 >> --- a/.gitlab-ci.d/cirrus/build.yml >> +++ b/.gitlab-ci.d/cirrus/build.yml >> @@ -32,6 +32,9 @@ build_task: >> - $MAKE -j$(sysctl -n hw.ncpu) >> - for TARGET in $TEST_TARGETS ; >> do >> - $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 >> - || { cat meson-logs/testlog.txt; exit 1; } ; >> + $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 ; >> done >> + always: >> + build_result_artifacts: >> + path: build/meson-logs/*log.txt >> + type: text/plain > > How long is that log kept available? No clue, I haven't seen anything related in the docs: https://cirrus-ci.org/guide/writing-tasks/#artifacts-instruction ... but I assume they will at least be available for a couple of days so you've got enough time to look at them in case a job failed. Thomas
On 15/2/23 16:45, Thomas Huth wrote: > On 15/02/2023 15.57, Philippe Mathieu-Daudé wrote: >> Hi Thomas, >> >> On 15/2/23 15:25, Thomas Huth wrote: >>> The meson log files can get very big, especially if running the tests in >>> verbose mode. So dumping those logs to the console was a bad idea, since >>> gitlab truncates the output if it is getting too big. Let's publish the >>> logs as artifacts instead. This has the disadvantage that you have to >>> look up the logs on cirrus-ci.com now instead, but that's still better >>> than not having the important part of the log at all since it got >>> truncated. >>> >>> Fixes: 998f334722 ("gitlab: show testlog.txt contents ...") >>> Signed-off-by: Thomas Huth <thuth@redhat.com> >>> --- >>> Note: I also tried to publish the junit xml files as artifacts >>> instead, but while the cirrus-ci docs claim to support it, I only >>> got unreadable XML output in my browser that way, so the .txt >>> files look like the better option to me. >>> >>> .gitlab-ci.d/cirrus/build.yml | 7 +++++-- >>> 1 file changed, 5 insertions(+), 2 deletions(-) >>> >>> diff --git a/.gitlab-ci.d/cirrus/build.yml >>> b/.gitlab-ci.d/cirrus/build.yml >>> index 7ef6af8d33..a9444902ec 100644 >>> --- a/.gitlab-ci.d/cirrus/build.yml >>> +++ b/.gitlab-ci.d/cirrus/build.yml >>> @@ -32,6 +32,9 @@ build_task: >>> - $MAKE -j$(sysctl -n hw.ncpu) >>> - for TARGET in $TEST_TARGETS ; >>> do >>> - $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 >>> - || { cat meson-logs/testlog.txt; exit 1; } ; >>> + $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 ; >>> done >>> + always: >>> + build_result_artifacts: >>> + path: build/meson-logs/*log.txt >>> + type: text/plain >> >> How long is that log kept available? > > No clue, I haven't seen anything related in the docs: > > https://cirrus-ci.org/guide/writing-tasks/#artifacts-instruction > > ... but I assume they will at least be available for a couple of days so > you've got enough time to look at them in case a job failed. OK thanks.
© 2016 - 2024 Red Hat, Inc.