[PATCH v2 10/16] tests: conditionally run "make check-venv" during build phase

John Snow posted 16 patches 2 months, 2 weeks ago
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Thomas Huth <thuth@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, "Marc-André Lureau" <marcandre.lureau@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, John Snow <jsnow@redhat.com>, Cleber Rosa <crosa@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>
There is a newer version of this series
[PATCH v2 10/16] tests: conditionally run "make check-venv" during build phase
Posted by John Snow 2 months, 2 weeks ago
Some tests need test dependencies, some tests don't. Instead of running
"make check" manually, use a CI variable for the template that allows us
to front-load the testing dependencies without needing to incur another
re-configure command.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index d866cb12bb1..cfa123d3a10 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -32,6 +32,10 @@
       then
         pyvenv/bin/meson configure . -Dbackend_max_links="$LD_JOBS" ;
       fi || exit 1;
+    - if test -n "$SETUP_CHECK_VENV";
+      then
+        make check-venv;
+      fi;
     - section_end configure
     - section_start build "Building QEMU"
     - $MAKE -j"$JOBS"
-- 
2.51.1
Re: [PATCH v2 10/16] tests: conditionally run "make check-venv" during build phase
Posted by Thomas Huth 2 months, 2 weeks ago
On 25/11/2025 05.00, John Snow wrote:
> Some tests need test dependencies, some tests don't. Instead of running
> "make check" manually, use a CI variable for the template that allows us
> to front-load the testing dependencies without needing to incur another
> re-configure command.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>   .gitlab-ci.d/buildtest-template.yml | 4 ++++
>   1 file changed, 4 insertions(+)
> 
> diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
> index d866cb12bb1..cfa123d3a10 100644
> --- a/.gitlab-ci.d/buildtest-template.yml
> +++ b/.gitlab-ci.d/buildtest-template.yml
> @@ -32,6 +32,10 @@
>         then
>           pyvenv/bin/meson configure . -Dbackend_max_links="$LD_JOBS" ;
>         fi || exit 1;
> +    - if test -n "$SETUP_CHECK_VENV";
> +      then
> +        make check-venv;
> +      fi;
I'm not sure, but I think this is likely not quite working as you intended 
it. The above code hunk is added to native_build_job_template, i.e. it's 
executed for the build-* jobs, but in the next patch, you only set the 
environment variable on the crash-test-* jobs. I don't think that the 
environment variables propagate backward from a later job to an earlier one.

Looking at the output of another build job, e.g. build-system-alpine:

  https://gitlab.com/thuth/qemu/-/jobs/12211712932#L2156

... it looks like pygdbmi is now also always installed there, i.e. something 
else triggers "check-venv" on all build jobs now, and that's why you were 
able to drop the "check-venv" in the crash-test-* jobs in the next patch 
now. No clue what's causing this now, but IMHO it should be fine if we just 
unconditionally do "check-venv" in all build jobs anyway (we also need the 
venv in a bunch of other test jobs anyway), so I'd rather do the "make 
check-venv" in this patch unconditionally here, and drop the next patch that 
sets SETUP_CHECK_VENV in the crash-test jobs. WDYT?

  Thomas
Re: [PATCH v2 10/16] tests: conditionally run "make check-venv" during build phase
Posted by John Snow 2 months, 1 week ago
On Wed, Nov 26, 2025 at 3:16 AM Thomas Huth <thuth@redhat.com> wrote:
>
> On 25/11/2025 05.00, John Snow wrote:
> > Some tests need test dependencies, some tests don't. Instead of running
> > "make check" manually, use a CI variable for the template that allows us
> > to front-load the testing dependencies without needing to incur another
> > re-configure command.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> >   .gitlab-ci.d/buildtest-template.yml | 4 ++++
> >   1 file changed, 4 insertions(+)
> >
> > diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
> > index d866cb12bb1..cfa123d3a10 100644
> > --- a/.gitlab-ci.d/buildtest-template.yml
> > +++ b/.gitlab-ci.d/buildtest-template.yml
> > @@ -32,6 +32,10 @@
> >         then
> >           pyvenv/bin/meson configure . -Dbackend_max_links="$LD_JOBS" ;
> >         fi || exit 1;
> > +    - if test -n "$SETUP_CHECK_VENV";
> > +      then
> > +        make check-venv;
> > +      fi;
> I'm not sure, but I think this is likely not quite working as you intended
> it. The above code hunk is added to native_build_job_template, i.e. it's
> executed for the build-* jobs, but in the next patch, you only set the
> environment variable on the crash-test-* jobs. I don't think that the
> environment variables propagate backward from a later job to an earlier one.
>
> Looking at the output of another build job, e.g. build-system-alpine:
>
>   https://gitlab.com/thuth/qemu/-/jobs/12211712932#L2156
>
> ... it looks like pygdbmi is now also always installed there, i.e. something
> else triggers "check-venv" on all build jobs now, and that's why you were
> able to drop the "check-venv" in the crash-test-* jobs in the next patch
> now. No clue what's causing this now, but IMHO it should be fine if we just
> unconditionally do "check-venv" in all build jobs anyway (we also need the
> venv in a bunch of other test jobs anyway), so I'd rather do the "make
> check-venv" in this patch unconditionally here, and drop the next patch that
> sets SETUP_CHECK_VENV in the crash-test jobs. WDYT?
>
>   Thomas
>

Yeah, it wasn't working. Oops, and good catch.

I moved "make check-venv" into the build-system-debian/fedora phases
instead and that works like I expect.

However, the pygdbmi bit is still confusing. Apparently both of the
new ensuregroup targets get installed with "make check-build", even
though they aren't used for building anything. I'm trying to sift
through the make system interplay to figure out why it's part of that
target...

--js
Re: [PATCH v2 10/16] tests: conditionally run "make check-venv" during build phase
Posted by John Snow 2 months, 1 week ago
On Wed, Dec 3, 2025 at 8:30 PM John Snow <jsnow@redhat.com> wrote:
>
> On Wed, Nov 26, 2025 at 3:16 AM Thomas Huth <thuth@redhat.com> wrote:
> >
> > On 25/11/2025 05.00, John Snow wrote:
> > > Some tests need test dependencies, some tests don't. Instead of running
> > > "make check" manually, use a CI variable for the template that allows us
> > > to front-load the testing dependencies without needing to incur another
> > > re-configure command.
> > >
> > > Signed-off-by: John Snow <jsnow@redhat.com>
> > > ---
> > >   .gitlab-ci.d/buildtest-template.yml | 4 ++++
> > >   1 file changed, 4 insertions(+)
> > >
> > > diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
> > > index d866cb12bb1..cfa123d3a10 100644
> > > --- a/.gitlab-ci.d/buildtest-template.yml
> > > +++ b/.gitlab-ci.d/buildtest-template.yml
> > > @@ -32,6 +32,10 @@
> > >         then
> > >           pyvenv/bin/meson configure . -Dbackend_max_links="$LD_JOBS" ;
> > >         fi || exit 1;
> > > +    - if test -n "$SETUP_CHECK_VENV";
> > > +      then
> > > +        make check-venv;
> > > +      fi;
> > I'm not sure, but I think this is likely not quite working as you intended
> > it. The above code hunk is added to native_build_job_template, i.e. it's
> > executed for the build-* jobs, but in the next patch, you only set the
> > environment variable on the crash-test-* jobs. I don't think that the
> > environment variables propagate backward from a later job to an earlier one.
> >
> > Looking at the output of another build job, e.g. build-system-alpine:
> >
> >   https://gitlab.com/thuth/qemu/-/jobs/12211712932#L2156
> >
> > ... it looks like pygdbmi is now also always installed there, i.e. something
> > else triggers "check-venv" on all build jobs now, and that's why you were
> > able to drop the "check-venv" in the crash-test-* jobs in the next patch
> > now. No clue what's causing this now, but IMHO it should be fine if we just
> > unconditionally do "check-venv" in all build jobs anyway (we also need the
> > venv in a bunch of other test jobs anyway), so I'd rather do the "make
> > check-venv" in this patch unconditionally here, and drop the next patch that
> > sets SETUP_CHECK_VENV in the crash-test jobs. WDYT?
> >
> >   Thomas
> >
>
> Yeah, it wasn't working. Oops, and good catch.
>
> I moved "make check-venv" into the build-system-debian/fedora phases
> instead and that works like I expect.
>
> However, the pygdbmi bit is still confusing. Apparently both of the
> new ensuregroup targets get installed with "make check-build", even
> though they aren't used for building anything. I'm trying to sift
> through the make system interplay to figure out why it's part of that
> target...

It's something to do with mtest2make, I think, adding the func test /
precache deps to the "check-build" target in Makefile.mtest - but I'm
not exactly sure where to make the incision here or what consequences
it might have.

I notice that the python test dependencies seem to get added here
regardless of if they are defined for the tests themselves, or for the
precache targets. Odd.

I think I need to consult with Paolo.
Re: [PATCH v2 10/16] tests: conditionally run "make check-venv" during build phase
Posted by John Snow 2 months, 2 weeks ago
On Wed, Nov 26, 2025, 3:16 AM Thomas Huth <thuth@redhat.com> wrote:

> On 25/11/2025 05.00, John Snow wrote:
> > Some tests need test dependencies, some tests don't. Instead of running
> > "make check" manually, use a CI variable for the template that allows us
> > to front-load the testing dependencies without needing to incur another
> > re-configure command.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> >   .gitlab-ci.d/buildtest-template.yml | 4 ++++
> >   1 file changed, 4 insertions(+)
> >
> > diff --git a/.gitlab-ci.d/buildtest-template.yml
> b/.gitlab-ci.d/buildtest-template.yml
> > index d866cb12bb1..cfa123d3a10 100644
> > --- a/.gitlab-ci.d/buildtest-template.yml
> > +++ b/.gitlab-ci.d/buildtest-template.yml
> > @@ -32,6 +32,10 @@
> >         then
> >           pyvenv/bin/meson configure . -Dbackend_max_links="$LD_JOBS" ;
> >         fi || exit 1;
> > +    - if test -n "$SETUP_CHECK_VENV";
> > +      then
> > +        make check-venv;
> > +      fi;
> I'm not sure, but I think this is likely not quite working as you intended
> it. The above code hunk is added to native_build_job_template, i.e. it's
> executed for the build-* jobs, but in the next patch, you only set the
> environment variable on the crash-test-* jobs. I don't think that the
> environment variables propagate backward from a later job to an earlier
> one.
>
> Looking at the output of another build job, e.g. build-system-alpine:
>
>   https://gitlab.com/thuth/qemu/-/jobs/12211712932#L2156
>
> ... it looks like pygdbmi is now also always installed there, i.e.
> something
> else triggers "check-venv" on all build jobs now, and that's why you were
> able to drop the "check-venv" in the crash-test-* jobs in the next patch
> now. No clue what's causing this now, but IMHO it should be fine if we
> just
> unconditionally do "check-venv" in all build jobs anyway (we also need the
> venv in a bunch of other test jobs anyway), so I'd rather do the "make
> check-venv" in this patch unconditionally here, and drop the next patch
> that
> sets SETUP_CHECK_VENV in the crash-test jobs. WDYT?
>
>   Thomas
>

... Huh. Well, I think it's probably fine to do it unconditionally for now,
but in the future we might have even more deps that might cause more
delays, so it'd be nice to have this mechanism working.

Shame on me for not noticing. The CI runs just take so long that I'm
delighted when they pass 🫠

Lemme look into it, but you might be right that the unconditional approach
might be best...

>