[PATCH v4 05/17] meson, mkvenv: add functests custom target

John Snow posted 17 patches 2 weeks, 6 days ago
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Thomas Huth <thuth@redhat.com>, Ed Maste <emaste@freebsd.org>, Li-Wen Hsu <lwhsu@freebsd.org>, Yonggang Luo <luoyonggang@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>, "Marc-André Lureau" <marcandre.lureau@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, John Snow <jsnow@redhat.com>, Cleber Rosa <crosa@redhat.com>, Maksim Davydov <davydov-max@yandex-team.ru>, Markus Armbruster <armbru@redhat.com>, Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>, Peter Xu <peterx@redhat.com>, Fabiano Rosas <farosas@suse.de>
There is a newer version of this series
[PATCH v4 05/17] meson, mkvenv: add functests custom target
Posted by John Snow 2 weeks, 6 days ago
add the "pyvenv_functests_group" target to meson. This target will
invoke mkvenv.py to install the associated dependency group to the
build/pyvenv directory.

A "pyvenv_tooling_group" is not included here as it is the plan to
always install this group by default, so it will not need an on-demand
trigger.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 meson.build        |  1 +
 pyvenv/meson.build | 28 ++++++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 pyvenv/meson.build

diff --git a/meson.build b/meson.build
index 3108f01e887..245f1bc2ec9 100644
--- a/meson.build
+++ b/meson.build
@@ -4551,6 +4551,7 @@ subdir('scripts')
 subdir('tools')
 subdir('pc-bios')
 subdir('docs')
+subdir('pyvenv')
 # Tests are disabled on emscripten because they rely on host features that aren't
 # supported by emscripten (e.g. fork and unix socket).
 if host_os != 'emscripten'
diff --git a/pyvenv/meson.build b/pyvenv/meson.build
new file mode 100644
index 00000000000..2bfddeb4a52
--- /dev/null
+++ b/pyvenv/meson.build
@@ -0,0 +1,28 @@
+# Note that this file only controls "optional" dependency groups; groups
+# *required* for the build are handled directly in configure instead:
+# namely, meson and sphinx.
+
+# NB: This command may or may not include the "--online" flag, depending
+# on the results of configure.
+ensuregroup_cmd = config_host['MKVENV_ENSUREGROUP'].split()
+
+pyvenv_common_deps = files(
+    meson.project_source_root() + '/pythondeps.toml',
+    meson.project_source_root() + '/python/scripts/mkvenv.py'
+)
+pyvenv_wheel_dir = meson.project_source_root() + '/python/wheels'
+
+
+# This group is allowed to rely on internet, to fetch from PyPI.
+# If --online was not passed to configure, this could fail.
+pyvenv_functests_group = custom_target(
+    'pyvenv_functests_group',
+    output: 'functests.group',
+    input: pyvenv_common_deps,
+    command: ensuregroup_cmd + [
+        '--online',
+        '--dir', pyvenv_wheel_dir,
+        '@INPUT0@',
+        'functests',
+    ],
+)
-- 
2.52.0
Re: [PATCH v4 05/17] meson, mkvenv: add functests custom target
Posted by Daniel P. Berrangé 2 weeks, 6 days ago
On Mon, Jan 19, 2026 at 04:27:31PM -0500, John Snow wrote:
> add the "pyvenv_functests_group" target to meson. This target will
> invoke mkvenv.py to install the associated dependency group to the
> build/pyvenv directory.
> 
> A "pyvenv_tooling_group" is not included here as it is the plan to
> always install this group by default, so it will not need an on-demand
> trigger.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  meson.build        |  1 +
>  pyvenv/meson.build | 28 ++++++++++++++++++++++++++++
>  2 files changed, 29 insertions(+)
>  create mode 100644 pyvenv/meson.build
> 
> diff --git a/meson.build b/meson.build
> index 3108f01e887..245f1bc2ec9 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -4551,6 +4551,7 @@ subdir('scripts')
>  subdir('tools')
>  subdir('pc-bios')
>  subdir('docs')
> +subdir('pyvenv')
>  # Tests are disabled on emscripten because they rely on host features that aren't
>  # supported by emscripten (e.g. fork and unix socket).
>  if host_os != 'emscripten'
> diff --git a/pyvenv/meson.build b/pyvenv/meson.build
> new file mode 100644
> index 00000000000..2bfddeb4a52
> --- /dev/null
> +++ b/pyvenv/meson.build
> @@ -0,0 +1,28 @@
> +# Note that this file only controls "optional" dependency groups; groups
> +# *required* for the build are handled directly in configure instead:
> +# namely, meson and sphinx.
> +
> +# NB: This command may or may not include the "--online" flag, depending
> +# on the results of configure.
> +ensuregroup_cmd = config_host['MKVENV_ENSUREGROUP'].split()

I'm confused because this implies that "ensuregroup_cmd" use of the
--online flag is supposed to be controlled by configure


> +
> +pyvenv_common_deps = files(
> +    meson.project_source_root() + '/pythondeps.toml',
> +    meson.project_source_root() + '/python/scripts/mkvenv.py'
> +)
> +pyvenv_wheel_dir = meson.project_source_root() + '/python/wheels'
> +
> +
> +# This group is allowed to rely on internet, to fetch from PyPI.
> +# If --online was not passed to configure, this could fail.

...and this also suggests --online is supposed to be controlled
by configure, but...

> +pyvenv_functests_group = custom_target(
> +    'pyvenv_functests_group',
> +    output: 'functests.group',
> +    input: pyvenv_common_deps,
> +    command: ensuregroup_cmd + [
> +        '--online',

now we just go ahead and hardcode use of --online regardless.

> +        '--dir', pyvenv_wheel_dir,
> +        '@INPUT0@',
> +        'functests',
> +    ],
> +)
> -- 
> 2.52.0
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [PATCH v4 05/17] meson, mkvenv: add functests custom target
Posted by John Snow 2 weeks, 5 days ago
On Tue, Jan 20, 2026 at 3:53 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Mon, Jan 19, 2026 at 04:27:31PM -0500, John Snow wrote:
> > add the "pyvenv_functests_group" target to meson. This target will
> > invoke mkvenv.py to install the associated dependency group to the
> > build/pyvenv directory.
> >
> > A "pyvenv_tooling_group" is not included here as it is the plan to
> > always install this group by default, so it will not need an on-demand
> > trigger.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> >  meson.build        |  1 +
> >  pyvenv/meson.build | 28 ++++++++++++++++++++++++++++
> >  2 files changed, 29 insertions(+)
> >  create mode 100644 pyvenv/meson.build
> >
> > diff --git a/meson.build b/meson.build
> > index 3108f01e887..245f1bc2ec9 100644
> > --- a/meson.build
> > +++ b/meson.build
> > @@ -4551,6 +4551,7 @@ subdir('scripts')
> >  subdir('tools')
> >  subdir('pc-bios')
> >  subdir('docs')
> > +subdir('pyvenv')
> >  # Tests are disabled on emscripten because they rely on host features that aren't
> >  # supported by emscripten (e.g. fork and unix socket).
> >  if host_os != 'emscripten'
> > diff --git a/pyvenv/meson.build b/pyvenv/meson.build
> > new file mode 100644
> > index 00000000000..2bfddeb4a52
> > --- /dev/null
> > +++ b/pyvenv/meson.build
> > @@ -0,0 +1,28 @@
> > +# Note that this file only controls "optional" dependency groups; groups
> > +# *required* for the build are handled directly in configure instead:
> > +# namely, meson and sphinx.
> > +
> > +# NB: This command may or may not include the "--online" flag, depending
> > +# on the results of configure.
> > +ensuregroup_cmd = config_host['MKVENV_ENSUREGROUP'].split()
>
> I'm confused because this implies that "ensuregroup_cmd" use of the
> --online flag is supposed to be controlled by configure
>
>
> > +
> > +pyvenv_common_deps = files(
> > +    meson.project_source_root() + '/pythondeps.toml',
> > +    meson.project_source_root() + '/python/scripts/mkvenv.py'
> > +)
> > +pyvenv_wheel_dir = meson.project_source_root() + '/python/wheels'
> > +
> > +
> > +# This group is allowed to rely on internet, to fetch from PyPI.
> > +# If --online was not passed to configure, this could fail.
>
> ...and this also suggests --online is supposed to be controlled
> by configure, but...
>
> > +pyvenv_functests_group = custom_target(
> > +    'pyvenv_functests_group',
> > +    output: 'functests.group',
> > +    input: pyvenv_common_deps,
> > +    command: ensuregroup_cmd + [
> > +        '--online',
>
> now we just go ahead and hardcode use of --online regardless.

Whoops, leftover from when I had both groups. What is now the tooling
group used to obey the flags, and I garbled the comments. My gut tells
me that we should force --online here, and if it fails because we're
actually in an isolated environment, then so be it.

On the other hand, if we were configured offline, the error message
you get from mkvenv might be better.

Do you have an opinion?


>
> > +        '--dir', pyvenv_wheel_dir,
> > +        '@INPUT0@',
> > +        'functests',
> > +    ],
> > +)
> > --
> > 2.52.0
> >
>
> With regards,
> Daniel
> --
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>
Re: [PATCH v4 05/17] meson, mkvenv: add functests custom target
Posted by Daniel P. Berrangé 2 weeks, 5 days ago
On Tue, Jan 20, 2026 at 12:27:01PM -0500, John Snow wrote:
> On Tue, Jan 20, 2026 at 3:53 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > On Mon, Jan 19, 2026 at 04:27:31PM -0500, John Snow wrote:
> > > add the "pyvenv_functests_group" target to meson. This target will
> > > invoke mkvenv.py to install the associated dependency group to the
> > > build/pyvenv directory.
> > >
> > > A "pyvenv_tooling_group" is not included here as it is the plan to
> > > always install this group by default, so it will not need an on-demand
> > > trigger.
> > >
> > > Signed-off-by: John Snow <jsnow@redhat.com>
> > > ---
> > >  meson.build        |  1 +
> > >  pyvenv/meson.build | 28 ++++++++++++++++++++++++++++
> > >  2 files changed, 29 insertions(+)
> > >  create mode 100644 pyvenv/meson.build
> > >
> > > diff --git a/meson.build b/meson.build
> > > index 3108f01e887..245f1bc2ec9 100644
> > > --- a/meson.build
> > > +++ b/meson.build
> > > @@ -4551,6 +4551,7 @@ subdir('scripts')
> > >  subdir('tools')
> > >  subdir('pc-bios')
> > >  subdir('docs')
> > > +subdir('pyvenv')
> > >  # Tests are disabled on emscripten because they rely on host features that aren't
> > >  # supported by emscripten (e.g. fork and unix socket).
> > >  if host_os != 'emscripten'
> > > diff --git a/pyvenv/meson.build b/pyvenv/meson.build
> > > new file mode 100644
> > > index 00000000000..2bfddeb4a52
> > > --- /dev/null
> > > +++ b/pyvenv/meson.build
> > > @@ -0,0 +1,28 @@
> > > +# Note that this file only controls "optional" dependency groups; groups
> > > +# *required* for the build are handled directly in configure instead:
> > > +# namely, meson and sphinx.
> > > +
> > > +# NB: This command may or may not include the "--online" flag, depending
> > > +# on the results of configure.
> > > +ensuregroup_cmd = config_host['MKVENV_ENSUREGROUP'].split()
> >
> > I'm confused because this implies that "ensuregroup_cmd" use of the
> > --online flag is supposed to be controlled by configure
> >
> >
> > > +
> > > +pyvenv_common_deps = files(
> > > +    meson.project_source_root() + '/pythondeps.toml',
> > > +    meson.project_source_root() + '/python/scripts/mkvenv.py'
> > > +)
> > > +pyvenv_wheel_dir = meson.project_source_root() + '/python/wheels'
> > > +
> > > +
> > > +# This group is allowed to rely on internet, to fetch from PyPI.
> > > +# If --online was not passed to configure, this could fail.
> >
> > ...and this also suggests --online is supposed to be controlled
> > by configure, but...
> >
> > > +pyvenv_functests_group = custom_target(
> > > +    'pyvenv_functests_group',
> > > +    output: 'functests.group',
> > > +    input: pyvenv_common_deps,
> > > +    command: ensuregroup_cmd + [
> > > +        '--online',
> >
> > now we just go ahead and hardcode use of --online regardless.
> 
> Whoops, leftover from when I had both groups. What is now the tooling
> group used to obey the flags, and I garbled the comments. My gut tells
> me that we should force --online here, and if it fails because we're
> actually in an isolated environment, then so be it.
> 
> On the other hand, if we were configured offline, the error message
> you get from mkvenv might be better.
> 
> Do you have an opinion?

IIUC, the later patch only adds pyvenv_functests_group to the thorough
functional tests, NOT the quick tests.  Given that the thorough tests
mandated online access to download assets, it feels reasonable to also
hardcode --online here for their code prereqs.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|


Re: [PATCH v4 05/17] meson, mkvenv: add functests custom target
Posted by John Snow 2 weeks, 5 days ago
On Tue, Jan 20, 2026 at 12:33 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Tue, Jan 20, 2026 at 12:27:01PM -0500, John Snow wrote:
> > On Tue, Jan 20, 2026 at 3:53 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
> > >
> > > On Mon, Jan 19, 2026 at 04:27:31PM -0500, John Snow wrote:
> > > > add the "pyvenv_functests_group" target to meson. This target will
> > > > invoke mkvenv.py to install the associated dependency group to the
> > > > build/pyvenv directory.
> > > >
> > > > A "pyvenv_tooling_group" is not included here as it is the plan to
> > > > always install this group by default, so it will not need an on-demand
> > > > trigger.
> > > >
> > > > Signed-off-by: John Snow <jsnow@redhat.com>
> > > > ---
> > > >  meson.build        |  1 +
> > > >  pyvenv/meson.build | 28 ++++++++++++++++++++++++++++
> > > >  2 files changed, 29 insertions(+)
> > > >  create mode 100644 pyvenv/meson.build
> > > >
> > > > diff --git a/meson.build b/meson.build
> > > > index 3108f01e887..245f1bc2ec9 100644
> > > > --- a/meson.build
> > > > +++ b/meson.build
> > > > @@ -4551,6 +4551,7 @@ subdir('scripts')
> > > >  subdir('tools')
> > > >  subdir('pc-bios')
> > > >  subdir('docs')
> > > > +subdir('pyvenv')
> > > >  # Tests are disabled on emscripten because they rely on host features that aren't
> > > >  # supported by emscripten (e.g. fork and unix socket).
> > > >  if host_os != 'emscripten'
> > > > diff --git a/pyvenv/meson.build b/pyvenv/meson.build
> > > > new file mode 100644
> > > > index 00000000000..2bfddeb4a52
> > > > --- /dev/null
> > > > +++ b/pyvenv/meson.build
> > > > @@ -0,0 +1,28 @@
> > > > +# Note that this file only controls "optional" dependency groups; groups
> > > > +# *required* for the build are handled directly in configure instead:
> > > > +# namely, meson and sphinx.
> > > > +
> > > > +# NB: This command may or may not include the "--online" flag, depending
> > > > +# on the results of configure.
> > > > +ensuregroup_cmd = config_host['MKVENV_ENSUREGROUP'].split()
> > >
> > > I'm confused because this implies that "ensuregroup_cmd" use of the
> > > --online flag is supposed to be controlled by configure
> > >
> > >
> > > > +
> > > > +pyvenv_common_deps = files(
> > > > +    meson.project_source_root() + '/pythondeps.toml',
> > > > +    meson.project_source_root() + '/python/scripts/mkvenv.py'
> > > > +)
> > > > +pyvenv_wheel_dir = meson.project_source_root() + '/python/wheels'
> > > > +
> > > > +
> > > > +# This group is allowed to rely on internet, to fetch from PyPI.
> > > > +# If --online was not passed to configure, this could fail.
> > >
> > > ...and this also suggests --online is supposed to be controlled
> > > by configure, but...
> > >
> > > > +pyvenv_functests_group = custom_target(
> > > > +    'pyvenv_functests_group',
> > > > +    output: 'functests.group',
> > > > +    input: pyvenv_common_deps,
> > > > +    command: ensuregroup_cmd + [
> > > > +        '--online',
> > >
> > > now we just go ahead and hardcode use of --online regardless.
> >
> > Whoops, leftover from when I had both groups. What is now the tooling
> > group used to obey the flags, and I garbled the comments. My gut tells
> > me that we should force --online here, and if it fails because we're
> > actually in an isolated environment, then so be it.
> >
> > On the other hand, if we were configured offline, the error message
> > you get from mkvenv might be better.
> >
> > Do you have an opinion?
>
> IIUC, the later patch only adds pyvenv_functests_group to the thorough
> functional tests, NOT the quick tests.  Given that the thorough tests
> mandated online access to download assets, it feels reasonable to also
> hardcode --online here for their code prereqs.

Works for me; will fix my mumbled comments.

>
> With regards,
> Daniel
> --
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>