If the limit for max opened files is way bigger than the default (1024),
say 3 orders bigger then spawning a child through virCommand can be
expensive because we iterate over ALL FDs within the limit and close
them. There's no need to that since we can learn the list of opened FDs
from /proc/self/fd/.

Michal Prívozník (3):
  virNetDevOpenvswitchInterfaceStats: Optimize for speed
  vircommand: Separate mass FD closing into a function
  virCommand: use procfs to learn opened FDs

 src/util/vircommand.c           | 114 +++++++++++++++++++++++++++-----
 src/util/virnetdevopenvswitch.c | 111 ++++++++++++++++++++-----------
 2 files changed, 170 insertions(+), 55 deletions(-)

-- 
2.21.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On Wed, Jul 03, 2019 at 09:19:17AM +0200, Michal Privoznik wrote:
> If the limit for max opened files is way bigger than the default (1024),
> say 3 orders bigger then spawning a child through virCommand can be
> expensive because we iterate over ALL FDs within the limit and close
> them. There's no need to that since we can learn the list of opened FDs
> from /proc/self/fd/.

Yeah I've seen this in docker containers in particular where the
nfiles limit is something like 2 million.

Hopefully Linux will finally get something equiv to BSD's closefrom()
syscall we can use in the future:

  https://lwn.net/Articles/593778/

meanwhile relying on /proc is reasonable I guess. The extra syscalls
to read the dir should be balanced by fewer close syscalls quite
easily.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On Wed, Jul 03, 2019 at 09:44:41AM +0100, Daniel P. Berrangé wrote:
> On Wed, Jul 03, 2019 at 09:19:17AM +0200, Michal Privoznik wrote:
> > If the limit for max opened files is way bigger than the default (1024),
> > say 3 orders bigger then spawning a child through virCommand can be
> > expensive because we iterate over ALL FDs within the limit and close
> > them. There's no need to that since we can learn the list of opened FDs
> > from /proc/self/fd/.
> 
> Yeah I've seen this in docker containers in particular where the
> nfiles limit is something like 2 million.
> 
> Hopefully Linux will finally get something equiv to BSD's closefrom()
> syscall we can use in the future:
> 
>   https://lwn.net/Articles/593778/

Opps, wrong LWN link

  https://lwn.net/Articles/789023/


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list