configure | 25 +++------------- meson.build | 56 +++++------------------------------ .gitignore | 3 ++ .gitmodules | 3 -- dtc | 1 - meson_options.txt | 5 ++-- scripts/meson-buildoptions.sh | 4 +-- scripts/mtest2make.py | 9 +++--- subprojects/dtc.wrap | 6 ++++ subprojects/slirp.wrap | 6 ++++ 10 files changed, 35 insertions(+), 83 deletions(-) delete mode 160000 dtc create mode 100644 subprojects/dtc.wrap create mode 100644 subprojects/slirp.wrap
From: Marc-André Lureau <marcandre.lureau@redhat.com> Hi, Meson "wrap" is a mechanism to build dependencies that doesn't rely on git submodules and integrate external dependencies as subproject()s. This offers developpers a simpler way to build QEMU with missing system dependencies (ex, libslirp in my case), but also simplify the fallback build definition of dtc/libfdt. In constrast with QEMU configure submodule handling, the subprojects are not downloaded automatically, and the user has to call "meson subprojects download" himself prior to running configure/meson. Marc-André Lureau (4): mtest2make.py: teach suite name that are just "PROJECT" build-sys: prevent meson from downloading wrapped subprojects build-sys: add slirp.wrap build-sys: replace dtc submodule with dtc.wrap configure | 25 +++------------- meson.build | 56 +++++------------------------------ .gitignore | 3 ++ .gitmodules | 3 -- dtc | 1 - meson_options.txt | 5 ++-- scripts/meson-buildoptions.sh | 4 +-- scripts/mtest2make.py | 9 +++--- subprojects/dtc.wrap | 6 ++++ subprojects/slirp.wrap | 6 ++++ 10 files changed, 35 insertions(+), 83 deletions(-) delete mode 160000 dtc create mode 100644 subprojects/dtc.wrap create mode 100644 subprojects/slirp.wrap -- 2.39.2
On Thu, Mar 02, 2023 at 05:18:44PM +0400, marcandre.lureau@redhat.com wrote: > From: Marc-André Lureau <marcandre.lureau@redhat.com> > > Hi, > > Meson "wrap" is a mechanism to build dependencies that doesn't rely on git > submodules and integrate external dependencies as subproject()s. > > This offers developpers a simpler way to build QEMU with missing system > dependencies (ex, libslirp in my case), but also simplify the fallback build > definition of dtc/libfdt. Do we actually need this facility though ? We've already determined that every platform we need has libslirp now, and IIUC Thomas determined recently that dtc is also available everywhere we need it to be. So why would we want to continue to special case these two libraries, out of all the many many many other libraries we also have deps on. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 3/6/23 11:06, Daniel P. Berrangé wrote: >> This offers developpers a simpler way to build QEMU with missing system >> dependencies (ex, libslirp in my case), but also simplify the fallback build >> definition of dtc/libfdt. > Do we actually need this facility though ? We've already determined > that every platform we need has libslirp now, and IIUC Thomas determined > recently that dtc is also available everywhere we need it to be. libvfio-user can use Meson subprojects instead of submodules, too; and with Pip support probably coming in 8.1, we can remove the meson submodule. Then, the only mostly-mandatory submodule would be keycodemapdb; SLOF requires a cross-compiler and the pre-built is binary is shipped binary, while softfloat/testfloat are test only and maybe could even be embedded. So there is a path towards getting rid of submodules at least for the main QEMU build process. Also, Windows installer builds could benefit from having wrapdb support for the mandatory dependencies in Windows (pixman, zlib, glib, possibly SDL). Since they are unintrusive and easy to revert, I think we could include patches 1-3 as experimental in 8.0, though I'm happy to oblige if people disagree (and only include patch 1). Paolo
On Mon, Mar 06, 2023 at 12:36:25PM +0100, Paolo Bonzini wrote: > On 3/6/23 11:06, Daniel P. Berrangé wrote: > > > This offers developpers a simpler way to build QEMU with missing system > > > dependencies (ex, libslirp in my case), but also simplify the fallback build > > > definition of dtc/libfdt. > > Do we actually need this facility though ? We've already determined > > that every platform we need has libslirp now, and IIUC Thomas determined > > recently that dtc is also available everywhere we need it to be. > > libvfio-user can use Meson subprojects instead of submodules, too; and with > Pip support probably coming in 8.1, we can remove the meson submodule. > Then, the only mostly-mandatory submodule would be keycodemapdb; On the topic of keycodemapdb, we designed it as a copylib thing because it seemed like the simplest approach, but I have wondered whether we should change tack and promote it as installable data package. Basically the keycodemap-gen tool ino /usr/bin, and CSV data file into /usr/share, and perhaps add a pkg-config file to enable apps to query its existance / path to keycodemap-gen ? The main downside is we would need to promise back compat of the CLI tool. The upside is that consumers of it would eliminate a git submodule and in theory be more up2date, though that depends on the distros actually updating the datafile periodically. I'm really on the fence about whether its worth it vs carrying on as a submodule (or meson subproject if that's nicer long term). With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Hi On Mon, Mar 6, 2023 at 2:06 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > On Thu, Mar 02, 2023 at 05:18:44PM +0400, marcandre.lureau@redhat.com > wrote: > > From: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > Hi, > > > > Meson "wrap" is a mechanism to build dependencies that doesn't rely on > git > > submodules and integrate external dependencies as subproject()s. > > > > This offers developpers a simpler way to build QEMU with missing system > > dependencies (ex, libslirp in my case), but also simplify the fallback > build > > definition of dtc/libfdt. > > Do we actually need this facility though ? We've already determined > that every platform we need has libslirp now, and IIUC Thomas determined > recently that dtc is also available everywhere we need it to be. > The main benefit is for developers: you have the source code of QEMU-related projects with the source tree. Code navigation, debugging, or various build tests are easier (compilation flags, static build etc). You don't have to "pollute" your system with (what could be) QEMU-specific libraries. > So why would we want to continue to special case these two libraries, > out of all the many many many other libraries we also have deps on. > They are more often updated, or developped with QEMU? For the reasons I listed, I would welcome more wrapped subprojects.
On Mon, Mar 06, 2023 at 02:19:25PM +0400, Marc-André Lureau wrote: > Hi > > On Mon, Mar 6, 2023 at 2:06 PM Daniel P. Berrangé <berrange@redhat.com> > wrote: > > > On Thu, Mar 02, 2023 at 05:18:44PM +0400, marcandre.lureau@redhat.com > > wrote: > > > From: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > > > Hi, > > > > > > Meson "wrap" is a mechanism to build dependencies that doesn't rely on > > git > > > submodules and integrate external dependencies as subproject()s. > > > > > > This offers developpers a simpler way to build QEMU with missing system > > > dependencies (ex, libslirp in my case), but also simplify the fallback > > build > > > definition of dtc/libfdt. > > > > Do we actually need this facility though ? We've already determined > > that every platform we need has libslirp now, and IIUC Thomas determined > > recently that dtc is also available everywhere we need it to be. > > > > The main benefit is for developers: you have the source code of > QEMU-related projects with the source tree. Code navigation, debugging, or > various build tests are easier (compilation flags, static build etc). You > don't have to "pollute" your system with (what could be) QEMU-specific > libraries. That's pushing developers to use builds of 3rd party libararies that don't actually match what our users are going to end up deploying with though. > > So why would we want to continue to special case these two libraries, > > out of all the many many many other libraries we also have deps on. > > > > They are more often updated, or developped with QEMU? For the reasons I > listed, I would welcome more wrapped subprojects. I don't think that they actually have more frequent updates that other libraries. In any case from QEMU's POV it doesn't matter how often upstream does releases. We're targetting the existing versions available in the OS and so don't want to use bleeding edge upstream releases. This also significantly expands our testing matrix. For each library we have the possiblity that users have the distro version vs the wrapped version. That is many new combinations users are exposed to, that we are realistically never going to have the bandwidth to test in CI. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Hi On Mon, Mar 6, 2023 at 2:33 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Mon, Mar 06, 2023 at 02:19:25PM +0400, Marc-André Lureau wrote: > > Hi > > > > On Mon, Mar 6, 2023 at 2:06 PM Daniel P. Berrangé <berrange@redhat.com> > > wrote: > > > > > On Thu, Mar 02, 2023 at 05:18:44PM +0400, marcandre.lureau@redhat.com > > > wrote: > > > > From: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > > > > > Hi, > > > > > > > > Meson "wrap" is a mechanism to build dependencies that doesn't rely on > > > git > > > > submodules and integrate external dependencies as subproject()s. > > > > > > > > This offers developpers a simpler way to build QEMU with missing system > > > > dependencies (ex, libslirp in my case), but also simplify the fallback > > > build > > > > definition of dtc/libfdt. > > > > > > Do we actually need this facility though ? We've already determined > > > that every platform we need has libslirp now, and IIUC Thomas determined > > > recently that dtc is also available everywhere we need it to be. > > > > > > > The main benefit is for developers: you have the source code of > > QEMU-related projects with the source tree. Code navigation, debugging, or > > various build tests are easier (compilation flags, static build etc). You > > don't have to "pollute" your system with (what could be) QEMU-specific > > libraries. > > That's pushing developers to use builds of 3rd party libararies > that don't actually match what our users are going to end up > deploying with though. The combinations of versions used by developers is already virtually limitless. I don't think adding a specific upstream version/revision to the mix is making the situation worse. It could be even the opposite, if we manage to build an "official" static version of qemu for example (with specific dependencies). > > > > So why would we want to continue to special case these two libraries, > > > out of all the many many many other libraries we also have deps on. > > > > > > > They are more often updated, or developped with QEMU? For the reasons I > > listed, I would welcome more wrapped subprojects. > > I don't think that they actually have more frequent updates that other > libraries. In any case from QEMU's POV it doesn't matter how often upstream > does releases. We're targetting the existing versions available in the OS > and so don't want to use bleeding edge upstream releases. That doesn't change that policy. > > This also significantly expands our testing matrix. For each library > we have the possiblity that users have the distro version vs the wrapped > version. That is many new combinations users are exposed to, that we are > realistically never going to have the bandwidth to test in CI. yes, I would consider this a tier 2 support, since it aims at developers, and not having to cover it in CI. -- Marc-André Lureau
On Mon, 6 Mar 2023 at 10:06, Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Thu, Mar 02, 2023 at 05:18:44PM +0400, marcandre.lureau@redhat.com wrote: > > From: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > Hi, > > > > Meson "wrap" is a mechanism to build dependencies that doesn't rely on git > > submodules and integrate external dependencies as subproject()s. > > > > This offers developpers a simpler way to build QEMU with missing system > > dependencies (ex, libslirp in my case), but also simplify the fallback build > > definition of dtc/libfdt. > > Do we actually need this facility though ? We've already determined > that every platform we need has libslirp now, and IIUC Thomas determined > recently that dtc is also available everywhere we need it to be. > > So why would we want to continue to special case these two libraries, > out of all the many many many other libraries we also have deps on. Also, it looks like the wrap mechanism is still basically "we have a file that indicates what the external git URL of the dependency is and specifies a commit hash to use", it's just changing the mechanism we use to get the source from git submodules to this new thing. Maybe the new thing really is better -- certainly git submodules are absolutely awful -- but we should have one mechanism, not two. thanks -- PMM
Hi On Mon, Mar 6, 2023 at 2:18 PM Peter Maydell <peter.maydell@linaro.org> wrote: > On Mon, 6 Mar 2023 at 10:06, Daniel P. Berrangé <berrange@redhat.com> > wrote: > > > > On Thu, Mar 02, 2023 at 05:18:44PM +0400, marcandre.lureau@redhat.com > wrote: > > > From: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > > > Hi, > > > > > > Meson "wrap" is a mechanism to build dependencies that doesn't rely on > git > > > submodules and integrate external dependencies as subproject()s. > > > > > > This offers developpers a simpler way to build QEMU with missing system > > > dependencies (ex, libslirp in my case), but also simplify the fallback > build > > > definition of dtc/libfdt. > > > > Do we actually need this facility though ? We've already determined > > that every platform we need has libslirp now, and IIUC Thomas determined > > recently that dtc is also available everywhere we need it to be. > > > > So why would we want to continue to special case these two libraries, > > out of all the many many many other libraries we also have deps on. > > Also, it looks like the wrap mechanism is still basically "we have > a file that indicates what the external git URL of the dependency > is and specifies a commit hash to use", it's just changing the > mechanism we use to get the source from git submodules to this > new thing. Maybe the new thing really is better -- certainly > git submodules are absolutely awful -- but we should have one > mechanism, not two. > I think we all experienced git submodules are more intrusive and may break various git workflows. wrap files may indeed point to a git URL with a revision, but they are more versatile than that and can download from released tarball instead, for example. Some projects, like glib, use both submodules and wrap files ( https://gitlab.gnome.org/GNOME/glib/-/tree/main/subprojects), there is no incompatibility with both solutions. As often, use what fits best your needs.
© 2016 - 2024 Red Hat, Inc.