From: Giuseppe Lettieri <giuseppe.lettieri@unipi.it>
With this patch, netmap support can be enabled with
the following options to the configure script:
--enable-netmap[=system]
Use the host system netmap installation.
Fail if not found.
--enable-netmap=git
clone the official netmap repository on
github (mostly useful for CI)
Signed-off-by: Giuseppe Lettieri <giuseppe.lettieri@unipi.it>
---
.gitmodules | 3 +++
configure | 64 +++++++++++++++++++++++++++++++++++++++++++++--------
2 files changed, 58 insertions(+), 9 deletions(-)
diff --git a/.gitmodules b/.gitmodules
index c5c474169d..bf75dbc5e3 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -58,3 +58,6 @@
[submodule "roms/opensbi"]
path = roms/opensbi
url = https://git.qemu.org/git/opensbi.git
+[submodule "netmap"]
+ path = netmap
+ url = https://github.com/luigirizzo/netmap.git
diff --git a/configure b/configure
index 8f8446f52b..cb2c6c70d6 100755
--- a/configure
+++ b/configure
@@ -1132,6 +1132,10 @@ for opt do
;;
--enable-netmap) netmap="yes"
;;
+ --enable-netmap=git) netmap="git"
+ ;;
+ --enable-netmap=system) netmap="system"
+ ;;
--disable-xen) xen="no"
;;
--enable-xen) xen="yes"
@@ -3318,8 +3322,9 @@ fi
# a minor/major version number. Minor new features will be marked with values up
# to 15, and if something happens that requires a change to the backend we will
# move above 15, submit the backend fixes and modify this two bounds.
-if test "$netmap" != "no" ; then
- cat > $TMPC << EOF
+case "$netmap" in
+ "" | yes | system)
+ cat > $TMPC << EOF
#include <inttypes.h>
#include <net/if.h>
#include <net/netmap.h>
@@ -3330,14 +3335,55 @@ if test "$netmap" != "no" ; then
int main(void) { return 0; }
EOF
if compile_prog "" "" ; then
- netmap=yes
+ netmap_system=yes
else
- if test "$netmap" = "yes" ; then
- feature_not_found "netmap"
- fi
- netmap=no
+ netmap_system=no
fi
-fi
+ ;;
+esac
+
+case "$netmap" in
+ "" | yes)
+ if test "$netmap_system" = "yes"; then
+ netmap=system
+ elif test -e "${source_path}/.git" && test $git_update = 'yes' ; then
+ netmap=git
+ elif test -e "${source_path}/netmap/configure" ; then
+ netmap=internal
+ elif test -z "$netmap" ; then
+ netmap=no
+ else
+ feature_not_found "netmap" "Install netmap or git submodule"
+ fi
+ ;;
+
+ system)
+ if test "$netmap_system" = "no"; then
+ feature_not_found "netmap" "Install netmap"
+ fi
+ ;;
+esac
+
+case "$netmap" in
+ git | internal)
+ if test "$netmap" = git; then
+ git_submodules="${git_submodules} netmap"
+ fi
+ mkdir -p netmap
+ QEMU_CFLAGS="$QEMU_CFLAGS -I\$(SRC_PATH)/netmap/sys"
+ ;;
+
+ system)
+ ;;
+
+ no)
+ ;;
+ *)
+ error_exit "Unknown state for netmap: $netmap"
+ ;;
+esac
+
+##########################################
##########################################
# libcap-ng library probe
@@ -6585,7 +6631,7 @@ if test "$vde" = "yes" ; then
echo "CONFIG_VDE=y" >> $config_host_mak
echo "VDE_LIBS=$vde_libs" >> $config_host_mak
fi
-if test "$netmap" = "yes" ; then
+if test "$netmap" != "no" ; then
echo "CONFIG_NETMAP=y" >> $config_host_mak
fi
if test "$l2tpv3" = "yes" ; then
--
2.21.0
On Fri, 4 Oct 2019 at 14:03, Giuseppe Lettieri <g.lettieri@iet.unipi.it> wrote: > > From: Giuseppe Lettieri <giuseppe.lettieri@unipi.it> > > With this patch, netmap support can be enabled with > the following options to the configure script: > > --enable-netmap[=system] > > Use the host system netmap installation. > Fail if not found. > > --enable-netmap=git > > clone the official netmap repository on > github (mostly useful for CI) > > Signed-off-by: Giuseppe Lettieri <giuseppe.lettieri@unipi.it> > --- > .gitmodules | 3 +++ > configure | 64 +++++++++++++++++++++++++++++++++++++++++++++-------- > 2 files changed, 58 insertions(+), 9 deletions(-) > > diff --git a/.gitmodules b/.gitmodules > index c5c474169d..bf75dbc5e3 100644 > --- a/.gitmodules > +++ b/.gitmodules > @@ -58,3 +58,6 @@ > [submodule "roms/opensbi"] > path = roms/opensbi > url = https://git.qemu.org/git/opensbi.git > +[submodule "netmap"] > + path = netmap > + url = https://github.com/luigirizzo/netmap.git Hi; this patch seems to be missing the rationale for why we want to do this. New submodules: * should always be on git.qemu.org (we need to mirror them in case the original upstream vanishes) * need a strong justification for why they're required (ie why we can't just use whatever the system-provided version of the library is, or fall back to not providing the feature if the library isn't present) Basically new submodules are a pain so we seek to minimize the use of them. thanks -- PMM
Peter Maydell <peter.maydell@linaro.org> writes: D> On Fri, 4 Oct 2019 at 14:03, Giuseppe Lettieri <g.lettieri@iet.unipi.it> wrote: >> >> From: Giuseppe Lettieri <giuseppe.lettieri@unipi.it> >> >> With this patch, netmap support can be enabled with >> the following options to the configure script: >> >> --enable-netmap[=system] >> >> Use the host system netmap installation. >> Fail if not found. >> >> --enable-netmap=git >> >> clone the official netmap repository on >> github (mostly useful for CI) >> >> Signed-off-by: Giuseppe Lettieri <giuseppe.lettieri@unipi.it> >> --- >> .gitmodules | 3 +++ >> configure | 64 +++++++++++++++++++++++++++++++++++++++++++++-------- >> 2 files changed, 58 insertions(+), 9 deletions(-) >> >> diff --git a/.gitmodules b/.gitmodules >> index c5c474169d..bf75dbc5e3 100644 >> --- a/.gitmodules >> +++ b/.gitmodules >> @@ -58,3 +58,6 @@ >> [submodule "roms/opensbi"] >> path = roms/opensbi >> url = https://git.qemu.org/git/opensbi.git >> +[submodule "netmap"] >> + path = netmap >> + url = https://github.com/luigirizzo/netmap.git > > Hi; this patch seems to be missing the rationale for why > we want to do this. New submodules: > * should always be on git.qemu.org (we need to mirror them > in case the original upstream vanishes) > * need a strong justification for why they're required > (ie why we can't just use whatever the system-provided > version of the library is, or fall back to not providing > the feature if the library isn't present) > > Basically new submodules are a pain so we seek to minimize > the use of them. I suggested making it a submodule upthread[*]. Let me try to distill the conversation into a rationale. Giuseppe, please correct mistakes. To make use of QEMU's netmap backend (CONFIG_NETMAP), you have to build and install netmap software from sources[**]. Which pretty much ensures developers compile with CONFIG_NETMAP off, and the code rots. For other dependencies that aren't readily available on common development hosts (slirp, capstone), we use submodules to avoid such rot. If the system provides, we use that, and if it doesn't, we fall back to the submodule. This has served us well. For netmap, falling back to the submodule when the host doesn't provide tends not to be useful beyond compile-testing. Because of that, we fall back only when the user explicitly asks for it by passing --enable-netmap=git to configure. CI should do that. [*] Message-ID: <87blwzho1y.fsf@dusky.pond.sub.org> [**] FreeBSD hosts may be an exception; I'm not sure.
On Mon, 7 Oct 2019 at 11:50, Markus Armbruster <armbru@redhat.com> wrote: > Peter Maydell <peter.maydell@linaro.org> writes: > > Basically new submodules are a pain so we seek to minimize > > the use of them. > > I suggested making it a submodule upthread[*]. Let me try to distill > the conversation into a rationale. Giuseppe, please correct mistakes. > > To make use of QEMU's netmap backend (CONFIG_NETMAP), you have to build > and install netmap software from sources[**]. Which pretty much ensures > developers compile with CONFIG_NETMAP off, and the code rots. > > For other dependencies that aren't readily available on common > development hosts (slirp, capstone), we use submodules to avoid such > rot. If the system provides, we use that, and if it doesn't, we fall > back to the submodule. This has served us well. I would put this differently. We don't use submodules to avoid code-rot. We use submodules where a dependency is needed for us to provide QEMU features that are sufficiently important that we want to provide them to users even if those users don't have the dependency available to them as a system library. There are lots of features of QEMU that only compile with sufficiently recent versions of dependencies, and we don't try to submodule-ize them because the features aren't really that important for the bulk of our users. For instance, we provided pixman as a submodule for a while because the features that require it (our graphics layer code) are important to almost all users. But we didn't provide spice as a module even when you pretty much needed to be running bleeding-edge redhat to satisfy the version dependency we had, because most users don't care about spice support. Shipping our dependencies as submodules imposes real costs on the project (for instance we then need to track the upstream to see when we should be updating, including checking whether we need to update to fix security issues). Submodules should be the exception, not the rule. > For netmap, falling back to the submodule when the host doesn't provide > tends not to be useful beyond compile-testing. Because of that, we fall > back only when the user explicitly asks for it by passing > --enable-netmap=git to configure. CI should do that. This sounds like netmap is in the same position as most of our dependencies: OK to compile if the system provides the library, but if the system doesn't then almost all users won't care that the feature isn't present. If CI of the QEMU code is useful, get the library supported by and shipped in distros. If you can't get anybody in a distro (Linux or BSD) to care enough to ship the library, this is a really niche feature, and up for consideration for deprecate-and-drop from QEMU, I think. thanks -- PMM
Peter Maydell <peter.maydell@linaro.org> writes: > On Mon, 7 Oct 2019 at 11:50, Markus Armbruster <armbru@redhat.com> wrote: >> Peter Maydell <peter.maydell@linaro.org> writes: >> > Basically new submodules are a pain so we seek to minimize >> > the use of them. >> >> I suggested making it a submodule upthread[*]. Let me try to distill >> the conversation into a rationale. Giuseppe, please correct mistakes. >> >> To make use of QEMU's netmap backend (CONFIG_NETMAP), you have to build >> and install netmap software from sources[**]. Which pretty much ensures >> developers compile with CONFIG_NETMAP off, and the code rots. >> >> For other dependencies that aren't readily available on common >> development hosts (slirp, capstone), we use submodules to avoid such >> rot. If the system provides, we use that, and if it doesn't, we fall >> back to the submodule. This has served us well. > > I would put this differently. We don't use submodules to avoid > code-rot. We use submodules where a dependency is needed for us > to provide QEMU features that are sufficiently important that we > want to provide them to users even if those users don't have the > dependency available to them as a system library. > > There are lots of features of QEMU that only compile with sufficiently > recent versions of dependencies, and we don't try to submodule-ize > them because the features aren't really that important for the bulk > of our users. For instance, we provided pixman as a submodule for > a while because the features that require it (our graphics layer > code) are important to almost all users. But we didn't provide > spice as a module even when you pretty much needed to be > running bleeding-edge redhat to satisfy the version dependency > we had, because most users don't care about spice support. The ability to compile close to everything in a single build tree is nevertheless convenient for developers. Submodules aren't necessary for that as long as "bleeding-edge redhat" can do the job. And it pretty much can: in my "everything" tree, config.status shows less than two dozen "no", and most of them are uninteresting for compile-testing (e.g. tcmalloc and jemalloc), or simply impossible to have (e.g. KVM is for Linux, HAX is not for Linux, can't have both). netmap's "no" is one of the few that is due to missing dependencies. Another one is vde. > Shipping our dependencies as submodules imposes real costs > on the project (for instance we then need to track the upstream > to see when we should be updating, including checking whether > we need to update to fix security issues). Submodules should be > the exception, not the rule. Point taken. [...]
Peter Maydell <peter.maydell@linaro.org> writes: > On Mon, 7 Oct 2019 at 11:50, Markus Armbruster <armbru@redhat.com> wrote: >> Peter Maydell <peter.maydell@linaro.org> writes: >> > Basically new submodules are a pain so we seek to minimize >> > the use of them. >> >> I suggested making it a submodule upthread[*]. Let me try to distill >> the conversation into a rationale. Giuseppe, please correct mistakes. >> >> To make use of QEMU's netmap backend (CONFIG_NETMAP), you have to build >> and install netmap software from sources[**]. Which pretty much ensures >> developers compile with CONFIG_NETMAP off, and the code rots. >> >> For other dependencies that aren't readily available on common >> development hosts (slirp, capstone), we use submodules to avoid such >> rot. If the system provides, we use that, and if it doesn't, we fall >> back to the submodule. This has served us well. > > I would put this differently. We don't use submodules to avoid > code-rot. We use submodules where a dependency is needed for us > to provide QEMU features that are sufficiently important that we > want to provide them to users even if those users don't have the > dependency available to them as a system library. > > There are lots of features of QEMU that only compile with sufficiently > recent versions of dependencies, and we don't try to submodule-ize > them because the features aren't really that important for the bulk > of our users. For instance, we provided pixman as a submodule for > a while because the features that require it (our graphics layer > code) are important to almost all users. But we didn't provide > spice as a module even when you pretty much needed to be > running bleeding-edge redhat to satisfy the version dependency > we had, because most users don't care about spice support. > Shipping our dependencies as submodules imposes real costs > on the project (for instance we then need to track the upstream > to see when we should be updating, including checking whether > we need to update to fix security issues). Submodules should be > the exception, not the rule. > >> For netmap, falling back to the submodule when the host doesn't provide >> tends not to be useful beyond compile-testing. Because of that, we fall >> back only when the user explicitly asks for it by passing >> --enable-netmap=git to configure. CI should do that. > > This sounds like netmap is in the same position as most of our > dependencies: OK to compile if the system provides the library, > but if the system doesn't then almost all users won't care > that the feature isn't present. If CI of the QEMU code is useful, If CI of QEMU code isn't useful, then I suspect the QEMU code isn't useful, period. Giuseppe assures us the netmap QEMU code *is* useful. It followe we better make sure our CI covers it. A submodule would make sure, but it looks like it won't fly. So let's try another tack: > get the library supported by and shipped in distros. If you can't > get anybody in a distro (Linux or BSD) to care enough to ship the > library, this is a really niche feature, and up for consideration > for deprecate-and-drop from QEMU, I think. Giuseppe, you mentioned netmap is in FreeBSD, and getting it into Linux is unlikely, so let's focus on FreeBSD. We have a FreeBSD section in .patchew.yml, which makes me guess Patchew CI tests FreeBSD. Does it test with CONFIG_NETMAP out of the box? If not, how do we have to tweak its configuration to get CONFIG_NETMAP enabled? Who could help with this?
On Mon, 7 Oct 2019 at 13:36, Markus Armbruster <armbru@redhat.com> wrote: > If CI of QEMU code isn't useful, then I suspect the QEMU code isn't > useful, period. Giuseppe assures us the netmap QEMU code *is* useful. > It followe we better make sure our CI covers it. It would be an interesting idea to have a requirement that any new library dependency can't be introduced into QEMU unless one of the systems we do builds on can be set up so the new code is compiled... thanks -- PMM
On Mon, Oct 07, 2019 at 01:39:30PM +0100, Peter Maydell wrote:
> On Mon, 7 Oct 2019 at 13:36, Markus Armbruster <armbru@redhat.com> wrote:
> > If CI of QEMU code isn't useful, then I suspect the QEMU code isn't
> > useful, period. Giuseppe assures us the netmap QEMU code *is* useful.
> > It followe we better make sure our CI covers it.
>
> It would be an interesting idea to have a requirement that
> any new library dependency can't be introduced into QEMU
> unless one of the systems we do builds on can be set up
> so the new code is compiled...
Being able to at least compile code successfully is a pretty low bar
to cross in terms of CI, so I think that's reasonable in general.
I would not stop in terms of libraries though. We should document our
broader expectations for CI
Compilation
- All new code must be compiled in one of[1] our CI systems.
This implies
- Libraries must be available in some OS we compile for
- New host architectures must have cross compilers available
- New OS distro targets must have VM test image support
This is not far off what we unofficially expect already, so
it shouldn't be too distruptive if we formally adopt it as a
mandatory rule.
Testing
- All significant new features should have either unit or
functional or integration test coverage
... something something something ...
We've not really set any expectations around CI beyond compile
testing thus far. We've got a mix of unit testing & functional
testing in the tests/ dir. We're increasingly getting stuff
added there when major features are added. Making this mandatory
is probably too large a step, but it is likely helpful if we
at least set some recommendations / guidelines to push people
in the direction we want to go longer term.
Regards,
Daniel
[1] Having to say "one of" is not especially great. It would be very nice
to get to the point where we have either container images or VM images
and no matter what CI harness(es) we use, they always run with either
a container or VM image[2]. Even if we have bare metal machines available
we can still execute actual builds inside containers or VM images so
everyone uses a consistent environment for everything related to CI.
[2] macOS is a problem/exception here given that we can't legally distribute
VM images, nor can we provide a cross compiler toolchain. For everything
else we can make VM/container images though.
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Daniel P. Berrangé <berrange@redhat.com> writes: > On Mon, Oct 07, 2019 at 01:39:30PM +0100, Peter Maydell wrote: >> On Mon, 7 Oct 2019 at 13:36, Markus Armbruster <armbru@redhat.com> wrote: >> > If CI of QEMU code isn't useful, then I suspect the QEMU code isn't >> > useful, period. Giuseppe assures us the netmap QEMU code *is* useful. >> > It followe we better make sure our CI covers it. >> >> It would be an interesting idea to have a requirement that >> any new library dependency can't be introduced into QEMU >> unless one of the systems we do builds on can be set up >> so the new code is compiled... > > Being able to at least compile code successfully is a pretty low bar > to cross in terms of CI, so I think that's reasonable in general. > > I would not stop in terms of libraries though. We should document our > broader expectations for CI > > Compilation > > - All new code must be compiled in one of[1] our CI systems. I think we should hold old code to the same standard. Anything that's not compiled now either gets fixed or deprecated. If it's still unfixed at the end of the deprecation grace period, it goes. > This implies > > - Libraries must be available in some OS we compile for How do we track compliance? It's not obvious to (ignorant) me what features exactly each CI compile enables. Without that, it's not obvious whether any CI run enables use of a certain library. > - New host architectures must have cross compilers available Native tool chain in CI also satisfies "must be compiled in CI". > - New OS distro targets must have VM test image support Non-virtual host in CI also satisfies "must be compiled in CI". > This is not far off what we unofficially expect already, so > it shouldn't be too distruptive if we formally adopt it as a > mandatory rule. Feels like a no-brainer, to be honest. > Testing > > - All significant new features should have either unit or > functional or integration test coverage > > ... something something something ... Spot the weasel words! ;-P > We've not really set any expectations around CI beyond compile > testing thus far. We've got a mix of unit testing & functional > testing in the tests/ dir. We're increasingly getting stuff > added there when major features are added. Making this mandatory > is probably too large a step, but it is likely helpful if we > at least set some recommendations / guidelines to push people > in the direction we want to go longer term. We've been pushing, but not evenly. It's basically up to maintainers, and their expectations on testing vary. > Regards, > Daniel > > [1] Having to say "one of" is not especially great. It would be very nice > to get to the point where we have either container images or VM images > and no matter what CI harness(es) we use, they always run with either > a container or VM image[2]. Even if we have bare metal machines available > we can still execute actual builds inside containers or VM images so > everyone uses a consistent environment for everything related to CI. > > [2] macOS is a problem/exception here given that we can't legally distribute > VM images, nor can we provide a cross compiler toolchain. For everything > else we can make VM/container images though. That makes Macs a secondary host at best. If it breaks, it breaks. If somebody fixes it, nice, if not, *shrug*. Don't expect git-bisect to work.
On 07/10/2019 14.35, Markus Armbruster wrote:
> Peter Maydell <peter.maydell@linaro.org> writes:
>
>> On Mon, 7 Oct 2019 at 11:50, Markus Armbruster <armbru@redhat.com> wrote:
>>> Peter Maydell <peter.maydell@linaro.org> writes:
>>>> Basically new submodules are a pain so we seek to minimize
>>>> the use of them.
>>>
>>> I suggested making it a submodule upthread[*]. Let me try to distill
>>> the conversation into a rationale. Giuseppe, please correct mistakes.
>>>
>>> To make use of QEMU's netmap backend (CONFIG_NETMAP), you have to build
>>> and install netmap software from sources[**]. Which pretty much ensures
>>> developers compile with CONFIG_NETMAP off, and the code rots.
>>>
>>> For other dependencies that aren't readily available on common
>>> development hosts (slirp, capstone), we use submodules to avoid such
>>> rot. If the system provides, we use that, and if it doesn't, we fall
>>> back to the submodule. This has served us well.
>>
>> I would put this differently. We don't use submodules to avoid
>> code-rot. We use submodules where a dependency is needed for us
>> to provide QEMU features that are sufficiently important that we
>> want to provide them to users even if those users don't have the
>> dependency available to them as a system library.
>>
>> There are lots of features of QEMU that only compile with sufficiently
>> recent versions of dependencies, and we don't try to submodule-ize
>> them because the features aren't really that important for the bulk
>> of our users. For instance, we provided pixman as a submodule for
>> a while because the features that require it (our graphics layer
>> code) are important to almost all users. But we didn't provide
>> spice as a module even when you pretty much needed to be
>> running bleeding-edge redhat to satisfy the version dependency
>> we had, because most users don't care about spice support.
>> Shipping our dependencies as submodules imposes real costs
>> on the project (for instance we then need to track the upstream
>> to see when we should be updating, including checking whether
>> we need to update to fix security issues). Submodules should be
>> the exception, not the rule.
>>
>>> For netmap, falling back to the submodule when the host doesn't provide
>>> tends not to be useful beyond compile-testing. Because of that, we fall
>>> back only when the user explicitly asks for it by passing
>>> --enable-netmap=git to configure. CI should do that.
>>
>> This sounds like netmap is in the same position as most of our
>> dependencies: OK to compile if the system provides the library,
>> but if the system doesn't then almost all users won't care
>> that the feature isn't present. If CI of the QEMU code is useful,
>
> If CI of QEMU code isn't useful, then I suspect the QEMU code isn't
> useful, period. Giuseppe assures us the netmap QEMU code *is* useful.
> It followe we better make sure our CI covers it.
>
> A submodule would make sure, but it looks like it won't fly. So let's
> try another tack:
>
>> get the library supported by and shipped in distros. If you can't
>> get anybody in a distro (Linux or BSD) to care enough to ship the
>> library, this is a really niche feature, and up for consideration
>> for deprecate-and-drop from QEMU, I think.
>
> Giuseppe, you mentioned netmap is in FreeBSD, and getting it into Linux
> is unlikely, so let's focus on FreeBSD.
>
> We have a FreeBSD section in .patchew.yml, which makes me guess Patchew
> CI tests FreeBSD. Does it test with CONFIG_NETMAP out of the box? If
> not, how do we have to tweak its configuration to get CONFIG_NETMAP
> enabled? Who could help with this?
I just tried this patch here:
diff --git a/.cirrus.yml b/.cirrus.yml
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -8,7 +8,7 @@ freebsd_12_task:
memory: 8G
install_script: pkg install -y
bash bison curl cyrus-sasl git glib gmake gnutls gsed
- nettle perl5 pixman pkgconf png usbredir
+ nettle perl5 pixman pkgconf png usbredir netmap
script:
- mkdir build
- cd build
... and looks like net/netmap.c now gets successfully compiled on
FreeBSD in the Cirrus-CI:
https://api.cirrus-ci.com/v1/task/5669479475838976/logs/main.log
We can also add it to the vm-freebsd test:
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -54,6 +54,9 @@ class FreeBSDVM(basevm.BaseVM):
# libs: opengl
"libepoxy",
"mesa-libs",
+
+ # libs: network
+ "netmap",
]
BUILD_SCRIPT = """
... then it gets compiled succesfully during "make vm-build-freebsd".
So does that sound like a good way to keep netmap.c from bitrotting? If
so, I can send the above two diffs as a proper patch, if you like.
Thomas
Thomas Huth <thuth@redhat.com> writes: > On 07/10/2019 14.35, Markus Armbruster wrote: >> Peter Maydell <peter.maydell@linaro.org> writes: >> >>> On Mon, 7 Oct 2019 at 11:50, Markus Armbruster <armbru@redhat.com> wrote: >>>> Peter Maydell <peter.maydell@linaro.org> writes: >>>>> Basically new submodules are a pain so we seek to minimize >>>>> the use of them. >>>> >>>> I suggested making it a submodule upthread[*]. Let me try to distill >>>> the conversation into a rationale. Giuseppe, please correct mistakes. >>>> >>>> To make use of QEMU's netmap backend (CONFIG_NETMAP), you have to build >>>> and install netmap software from sources[**]. Which pretty much ensures >>>> developers compile with CONFIG_NETMAP off, and the code rots. >>>> >>>> For other dependencies that aren't readily available on common >>>> development hosts (slirp, capstone), we use submodules to avoid such >>>> rot. If the system provides, we use that, and if it doesn't, we fall >>>> back to the submodule. This has served us well. >>> >>> I would put this differently. We don't use submodules to avoid >>> code-rot. We use submodules where a dependency is needed for us >>> to provide QEMU features that are sufficiently important that we >>> want to provide them to users even if those users don't have the >>> dependency available to them as a system library. >>> >>> There are lots of features of QEMU that only compile with sufficiently >>> recent versions of dependencies, and we don't try to submodule-ize >>> them because the features aren't really that important for the bulk >>> of our users. For instance, we provided pixman as a submodule for >>> a while because the features that require it (our graphics layer >>> code) are important to almost all users. But we didn't provide >>> spice as a module even when you pretty much needed to be >>> running bleeding-edge redhat to satisfy the version dependency >>> we had, because most users don't care about spice support. >>> Shipping our dependencies as submodules imposes real costs >>> on the project (for instance we then need to track the upstream >>> to see when we should be updating, including checking whether >>> we need to update to fix security issues). Submodules should be >>> the exception, not the rule. >>> >>>> For netmap, falling back to the submodule when the host doesn't provide >>>> tends not to be useful beyond compile-testing. Because of that, we fall >>>> back only when the user explicitly asks for it by passing >>>> --enable-netmap=git to configure. CI should do that. >>> >>> This sounds like netmap is in the same position as most of our >>> dependencies: OK to compile if the system provides the library, >>> but if the system doesn't then almost all users won't care >>> that the feature isn't present. If CI of the QEMU code is useful, >> >> If CI of QEMU code isn't useful, then I suspect the QEMU code isn't >> useful, period. Giuseppe assures us the netmap QEMU code *is* useful. >> It followe we better make sure our CI covers it. >> >> A submodule would make sure, but it looks like it won't fly. So let's >> try another tack: >> >>> get the library supported by and shipped in distros. If you can't >>> get anybody in a distro (Linux or BSD) to care enough to ship the >>> library, this is a really niche feature, and up for consideration >>> for deprecate-and-drop from QEMU, I think. >> >> Giuseppe, you mentioned netmap is in FreeBSD, and getting it into Linux >> is unlikely, so let's focus on FreeBSD. >> >> We have a FreeBSD section in .patchew.yml, which makes me guess Patchew >> CI tests FreeBSD. Does it test with CONFIG_NETMAP out of the box? If >> not, how do we have to tweak its configuration to get CONFIG_NETMAP >> enabled? Who could help with this? > > I just tried this patch here: > > diff --git a/.cirrus.yml b/.cirrus.yml > --- a/.cirrus.yml > +++ b/.cirrus.yml > @@ -8,7 +8,7 @@ freebsd_12_task: > memory: 8G > install_script: pkg install -y > bash bison curl cyrus-sasl git glib gmake gnutls gsed > - nettle perl5 pixman pkgconf png usbredir > + nettle perl5 pixman pkgconf png usbredir netmap > script: > - mkdir build > - cd build > > ... and looks like net/netmap.c now gets successfully compiled on > FreeBSD in the Cirrus-CI: > > https://api.cirrus-ci.com/v1/task/5669479475838976/logs/main.log Awesome :) > We can also add it to the vm-freebsd test: > > diff --git a/tests/vm/freebsd b/tests/vm/freebsd > --- a/tests/vm/freebsd > +++ b/tests/vm/freebsd > @@ -54,6 +54,9 @@ class FreeBSDVM(basevm.BaseVM): > # libs: opengl > "libepoxy", > "mesa-libs", > + > + # libs: network > + "netmap", > ] > > BUILD_SCRIPT = """ > > ... then it gets compiled succesfully during "make vm-build-freebsd". > > So does that sound like a good way to keep netmap.c from bitrotting? If > so, I can send the above two diffs as a proper patch, if you like. Yes, please!
© 2016 - 2025 Red Hat, Inc.