In order to convert the existing Makefile target from a manual
invocation of mkvenv to one that uses the meson dependency system, we
need to not suppress ninja here.
I'm not sure if this creates problems I am not aware of; but invoking
ninja here is no longer spurious but will become necessary.
Signed-off-by: John Snow <jsnow@redhat.com>
---
.gitlab-ci.d/buildtest.yml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 18d72d3058b..2cb2cf25b4a 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -101,7 +101,7 @@ crash-test-debian:
IMAGE: debian
script:
- cd build
- - make NINJA=":" check-venv
+ - make check-venv
- pyvenv/bin/python3 scripts/device-crash-test -q --tcg-only ./qemu-system-i386
build-system-fedora:
@@ -158,7 +158,7 @@ crash-test-fedora:
IMAGE: fedora
script:
- cd build
- - make NINJA=":" check-venv
+ - make check-venv
- pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc
- pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32
--
2.51.1On Mon, Nov 17, 2025 at 01:51:25PM -0500, John Snow wrote:
> In order to convert the existing Makefile target from a manual
> invocation of mkvenv to one that uses the meson dependency system, we
> need to not suppress ninja here.
>
> I'm not sure if this creates problems I am not aware of; but invoking
> ninja here is no longer spurious but will become necessary.
Yes, this will likely create problems. From the commit message that
introduced NINJA=":"...
commit 4d3bd91b26a69b39a178744d3d6e5f23050afb23
Author: Thomas Huth <thuth@redhat.com>
Date: Mon Apr 24 10:22:35 2023 +0100
gitlab-ci: Avoid to re-run "configure" in the device-crash-test jobs
After "make check-venv" had been added to these jobs, they started
to re-run "configure" each time since our logic in the makefile
thinks that some files are out of date here. Avoid it with the same
trick that we are using in buildtest-template.yml already by disabling
the up-to-date check via NINJA=":".
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> .gitlab-ci.d/buildtest.yml | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> index 18d72d3058b..2cb2cf25b4a 100644
> --- a/.gitlab-ci.d/buildtest.yml
> +++ b/.gitlab-ci.d/buildtest.yml
> @@ -101,7 +101,7 @@ crash-test-debian:
> IMAGE: debian
> script:
> - cd build
> - - make NINJA=":" check-venv
> + - make check-venv
> - pyvenv/bin/python3 scripts/device-crash-test -q --tcg-only ./qemu-system-i386
>
> build-system-fedora:
> @@ -158,7 +158,7 @@ crash-test-fedora:
> IMAGE: fedora
> script:
> - cd build
> - - make NINJA=":" check-venv
> + - make check-venv
> - pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc
> - pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32
>
> --
> 2.51.1
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Il lun 17 nov 2025, 20:07 Daniel P. Berrangé <berrange@redhat.com> ha scritto: > On Mon, Nov 17, 2025 at 01:51:25PM -0500, John Snow wrote: > > In order to convert the existing Makefile target from a manual > > invocation of mkvenv to one that uses the meson dependency system, we > > need to not suppress ninja here. > > > > I'm not sure if this creates problems I am not aware of; but invoking > > ninja here is no longer spurious but will become necessary. > > Yes, this will likely create problems. From the commit message that > introduced NINJA=":"... > > Avoid it with the same > trick that we are using in buildtest-template.yml already by disabling > the up-to-date check via NINJA=":". > Move the check-venv call to buildtest-template.yml, right after configure? It's cheap enough. Paolo > > > > > Signed-off-by: John Snow <jsnow@redhat.com> > > --- > > .gitlab-ci.d/buildtest.yml | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml > > index 18d72d3058b..2cb2cf25b4a 100644 > > --- a/.gitlab-ci.d/buildtest.yml > > +++ b/.gitlab-ci.d/buildtest.yml > > @@ -101,7 +101,7 @@ crash-test-debian: > > IMAGE: debian > > script: > > - cd build > > - - make NINJA=":" check-venv > > + - make check-venv > > - pyvenv/bin/python3 scripts/device-crash-test -q --tcg-only > ./qemu-system-i386 > > > > build-system-fedora: > > @@ -158,7 +158,7 @@ crash-test-fedora: > > IMAGE: fedora > > script: > > - cd build > > - - make NINJA=":" check-venv > > + - make check-venv > > - pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc > > - pyvenv/bin/python3 scripts/device-crash-test -q > ./qemu-system-riscv32 > > > > -- > > 2.51.1 > > > > With regards, > Daniel > -- > |: https://berrange.com -o- > https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- > https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- > https://www.instagram.com/dberrange :| > >
On Mon, Nov 17, 2025, 3:47 PM Paolo Bonzini <pbonzini@redhat.com> wrote: > > > Il lun 17 nov 2025, 20:07 Daniel P. Berrangé <berrange@redhat.com> ha > scritto: > >> On Mon, Nov 17, 2025 at 01:51:25PM -0500, John Snow wrote: >> > In order to convert the existing Makefile target from a manual >> > invocation of mkvenv to one that uses the meson dependency system, we >> > need to not suppress ninja here. >> > >> > I'm not sure if this creates problems I am not aware of; but invoking >> > ninja here is no longer spurious but will become necessary. >> >> Yes, this will likely create problems. From the commit message that >> introduced NINJA=":"... >> >> Avoid it with the same >> trick that we are using in buildtest-template.yml already by disabling >> the up-to-date check via NINJA=":". >> > > Move the check-venv call to buildtest-template.yml, right after configure? > It's cheap enough. > > Paolo > What's the root issue here? That there's enough of a time delay between the actual configure and the test running that it re-runs configure? (And presumably this is bad mostly for wasted CI time...?) > >> >> > >> > Signed-off-by: John Snow <jsnow@redhat.com> >> > --- >> > .gitlab-ci.d/buildtest.yml | 4 ++-- >> > 1 file changed, 2 insertions(+), 2 deletions(-) >> > >> > diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml >> > index 18d72d3058b..2cb2cf25b4a 100644 >> > --- a/.gitlab-ci.d/buildtest.yml >> > +++ b/.gitlab-ci.d/buildtest.yml >> > @@ -101,7 +101,7 @@ crash-test-debian: >> > IMAGE: debian >> > script: >> > - cd build >> > - - make NINJA=":" check-venv >> > + - make check-venv >> > - pyvenv/bin/python3 scripts/device-crash-test -q --tcg-only >> ./qemu-system-i386 >> > >> > build-system-fedora: >> > @@ -158,7 +158,7 @@ crash-test-fedora: >> > IMAGE: fedora >> > script: >> > - cd build >> > - - make NINJA=":" check-venv >> > + - make check-venv >> > - pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc >> > - pyvenv/bin/python3 scripts/device-crash-test -q >> ./qemu-system-riscv32 >> > >> > -- >> > 2.51.1 >> > >> >> With regards, >> Daniel >> -- >> |: https://berrange.com -o- >> https://www.flickr.com/photos/dberrange :| >> |: https://libvirt.org -o- >> https://fstop138.berrange.com :| >> |: https://entangle-photo.org -o- >> https://www.instagram.com/dberrange :| >> >>
Il mar 18 nov 2025, 22:50 John Snow <jsnow@redhat.com> ha scritto: > > > On Mon, Nov 17, 2025, 3:47 PM Paolo Bonzini <pbonzini@redhat.com> wrote: > >> >> >> Il lun 17 nov 2025, 20:07 Daniel P. Berrangé <berrange@redhat.com> ha >> scritto: >> >>> On Mon, Nov 17, 2025 at 01:51:25PM -0500, John Snow wrote: >>> > In order to convert the existing Makefile target from a manual >>> > invocation of mkvenv to one that uses the meson dependency system, we >>> > need to not suppress ninja here. >>> > >>> > I'm not sure if this creates problems I am not aware of; but invoking >>> > ninja here is no longer spurious but will become necessary. >>> >>> Yes, this will likely create problems. From the commit message that >>> introduced NINJA=":"... >>> >>> Avoid it with the same >>> trick that we are using in buildtest-template.yml already by >>> disabling >>> the up-to-date check via NINJA=":". >>> >> >> Move the check-venv call to buildtest-template.yml, right after >> configure? It's cheap enough. >> >> Paolo >> > > What's the root issue here? That there's enough of a time delay between > the actual configure and the test running that it re-runs configure? > That the git repo is cloned again and is therefore newer than the artifacts. That triggers a full rebuild. > (And presumably this is bad mostly for wasted CI time...?) > > > >> >>> >>> > >>> > Signed-off-by: John Snow <jsnow@redhat.com> >>> > --- >>> > .gitlab-ci.d/buildtest.yml | 4 ++-- >>> > 1 file changed, 2 insertions(+), 2 deletions(-) >>> > >>> > diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml >>> > index 18d72d3058b..2cb2cf25b4a 100644 >>> > --- a/.gitlab-ci.d/buildtest.yml >>> > +++ b/.gitlab-ci.d/buildtest.yml >>> > @@ -101,7 +101,7 @@ crash-test-debian: >>> > IMAGE: debian >>> > script: >>> > - cd build >>> > - - make NINJA=":" check-venv >>> > + - make check-venv >>> > - pyvenv/bin/python3 scripts/device-crash-test -q --tcg-only >>> ./qemu-system-i386 >>> > >>> > build-system-fedora: >>> > @@ -158,7 +158,7 @@ crash-test-fedora: >>> > IMAGE: fedora >>> > script: >>> > - cd build >>> > - - make NINJA=":" check-venv >>> > + - make check-venv >>> > - pyvenv/bin/python3 scripts/device-crash-test -q >>> ./qemu-system-ppc >>> > - pyvenv/bin/python3 scripts/device-crash-test -q >>> ./qemu-system-riscv32 >>> > >>> > -- >>> > 2.51.1 >>> > >>> >>> With regards, >>> Daniel >>> -- >>> |: https://berrange.com -o- >>> https://www.flickr.com/photos/dberrange :| >>> |: https://libvirt.org -o- >>> https://fstop138.berrange.com :| >>> |: https://entangle-photo.org -o- >>> https://www.instagram.com/dberrange :| >>> >>>
On 19/11/2025 09.05, Paolo Bonzini wrote: > > > Il mar 18 nov 2025, 22:50 John Snow <jsnow@redhat.com > <mailto:jsnow@redhat.com>> ha scritto: > > > > On Mon, Nov 17, 2025, 3:47 PM Paolo Bonzini <pbonzini@redhat.com > <mailto:pbonzini@redhat.com>> wrote: > > > > Il lun 17 nov 2025, 20:07 Daniel P. Berrangé <berrange@redhat.com > <mailto:berrange@redhat.com>> ha scritto: > > On Mon, Nov 17, 2025 at 01:51:25PM -0500, John Snow wrote: > > In order to convert the existing Makefile target from a manual > > invocation of mkvenv to one that uses the meson dependency > system, we > > need to not suppress ninja here. > > > > I'm not sure if this creates problems I am not aware of; but > invoking > > ninja here is no longer spurious but will become necessary. > > Yes, this will likely create problems. From the commit message that > introduced NINJA=":"... > > Avoid it with the same > trick that we are using in buildtest-template.yml already > by disabling > the up-to-date check via NINJA=":". > > > Move the check-venv call to buildtest-template.yml, right after > configure? It's cheap enough. > > Paolo > > > What's the root issue here? That there's enough of a time delay between > the actual configure and the test running that it re-runs configure? > > > That the git repo is cloned again and is therefore newer than the artifacts. > That triggers a full rebuild. Right. At least for the other "functional" and "check-system" jobs in our CI, this was a big issue, calling "make check-functional/qtest/iotest" there triggered a rebuild of all binaries, increasing our CI runtime significantly. For the "crash test" jobs, it was mainly about avoiding to rerun the "configure" step for cosmetical reasons each time the job runs. The rebuild of the qemu binaries is triggered by "make check-iotests/qtests/function" in the other jobs, and not by "make check-venv" alone, so it's not as bad as in the other jobs here: Without your patch, the jobs take about 16 minutes (crash-test-debian) and 10 minutes (crash-test-fedora): https://gitlab.com/thuth/qemu/-/jobs/12138792667 https://gitlab.com/thuth/qemu/-/jobs/12138792674 With your patch applied: https://gitlab.com/thuth/qemu/-/jobs/12139438583 https://gitlab.com/thuth/qemu/-/jobs/12139438599 the jobs take approx. the same amount of time, so I think it's OK here to drop the NINJA=":" again. But I think we should give Paolo's idea a try and move the "make check-venv" to buildtest-template.yml instead. In fact, that might even help to slightly decrease the total runtime of the jobs since all the other "check" jobs don't have to do this again (for example the check-functional jobs are running it, too, see: https://gitlab.com/qemu-project/qemu/-/jobs/12139584312#L1051). Thomas
On Wed, Nov 19, 2025, 9:15 AM Thomas Huth <thuth@redhat.com> wrote: > On 19/11/2025 09.05, Paolo Bonzini wrote: > > > > > > Il mar 18 nov 2025, 22:50 John Snow <jsnow@redhat.com > > <mailto:jsnow@redhat.com>> ha scritto: > > > > > > > > On Mon, Nov 17, 2025, 3:47 PM Paolo Bonzini <pbonzini@redhat.com > > <mailto:pbonzini@redhat.com>> wrote: > > > > > > > > Il lun 17 nov 2025, 20:07 Daniel P. Berrangé < > berrange@redhat.com > > <mailto:berrange@redhat.com>> ha scritto: > > > > On Mon, Nov 17, 2025 at 01:51:25PM -0500, John Snow wrote: > > > In order to convert the existing Makefile target from a > manual > > > invocation of mkvenv to one that uses the meson dependency > > system, we > > > need to not suppress ninja here. > > > > > > I'm not sure if this creates problems I am not aware of; > but > > invoking > > > ninja here is no longer spurious but will become > necessary. > > > > Yes, this will likely create problems. From the commit > message that > > introduced NINJA=":"... > > > > Avoid it with the same > > trick that we are using in buildtest-template.yml > already > > by disabling > > the up-to-date check via NINJA=":". > > > > > > Move the check-venv call to buildtest-template.yml, right after > > configure? It's cheap enough. > > > > Paolo > > > > > > What's the root issue here? That there's enough of a time delay > between > > the actual configure and the test running that it re-runs configure? > > > > > > That the git repo is cloned again and is therefore newer than the > artifacts. > > That triggers a full rebuild. > > Right. At least for the other "functional" and "check-system" jobs in our > CI, this was a big issue, calling "make check-functional/qtest/iotest" > there > triggered a rebuild of all binaries, increasing our CI runtime > significantly. > > For the "crash test" jobs, it was mainly about avoiding to rerun the > "configure" step for cosmetical reasons each time the job runs. The > rebuild > of the qemu binaries is triggered by "make check-iotests/qtests/function" > in > the other jobs, and not by "make check-venv" alone, so it's not as bad as > in > the other jobs here: > Without your patch, the jobs take about 16 minutes (crash-test-debian) and > 10 minutes (crash-test-fedora): > > https://gitlab.com/thuth/qemu/-/jobs/12138792667 > https://gitlab.com/thuth/qemu/-/jobs/12138792674 > > With your patch applied: > > https://gitlab.com/thuth/qemu/-/jobs/12139438583 > https://gitlab.com/thuth/qemu/-/jobs/12139438599 > > the jobs take approx. the same amount of time, so I think it's OK here to > drop the NINJA=":" again. > > But I think we should give Paolo's idea a try and move the "make > check-venv" > to buildtest-template.yml instead. In fact, that might even help to > slightly > decrease the total runtime of the jobs since all the other "check" jobs > don't have to do this again (for example the check-functional jobs are > running it, too, see: > https://gitlab.com/qemu-project/qemu/-/jobs/12139584312#L1051). > > Thomas > Ah, I see now, thanks! I can think of some remedies for this. No problem.
© 2016 - 2025 Red Hat, Inc.