The following changes since commit ec7a8bf0b8f7dc7288fe8745464ee8217528cc6c:

  Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2017-11-06 10:04:16 +0000)

are available in the git repository at:

  git://github.com/berrange/qemu tags/pull-qio-2017-11-06-1

for you to fetch changes up to 74c0035408d5b327eac5985d976bca3009e0c5d6:

  tests: Add test-listen - a stress test for QEMU socket listen (2017-11-06 11:49:15 +0000)

----------------------------------------------------------------
Merge qio 2017-11-06 v1

----------------------------------------------------------------

Daniel P. Berrange (1):
  sockets: avoid leak of listen file descriptor

Knut Omang (1):
  tests: Add test-listen - a stress test for QEMU socket listen

 tests/Makefile.include |   2 +
 tests/test-listen.c    | 253 +++++++++++++++++++++++++++++++++++++++++++++++++
 util/qemu-sockets.c    |  52 +++++-----
 3 files changed, 284 insertions(+), 23 deletions(-)
 create mode 100644 tests/test-listen.c

-- 
2.13.6

On 6 November 2017 at 15:33, Daniel P. Berrange <berrange@redhat.com> wrote:
> The following changes since commit ec7a8bf0b8f7dc7288fe8745464ee8217528cc6c:
>
>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2017-11-06 10:04:16 +0000)
>
> are available in the git repository at:
>
>   git://github.com/berrange/qemu tags/pull-qio-2017-11-06-1
>
> for you to fetch changes up to 74c0035408d5b327eac5985d976bca3009e0c5d6:
>
>   tests: Add test-listen - a stress test for QEMU socket listen (2017-11-06 11:49:15 +0000)
>
> ----------------------------------------------------------------
> Merge qio 2017-11-06 v1
>
> ----------------------------------------------------------------

Hi. I'm afraid this fails on various hosts.

arm 32 bit (in a chroot on a 64 bit box):
TEST: tests/test-listen... (pid=319)
  /socket/listen-serial/ipv4:                                          OK
  /socket/listen-serial/ipv6:
** Failed to assign a port to
thread 0 (errno = 98)
** Failed to assign a port to thread 1 (errno = 98)
** Failed to assign a port to thread 2 (errno = 98)
** Failed to assign a port to thread 3 (errno = 98)
** Failed to assign a port to thread 4 (errno = 98)
[...repeats all the way up to...]
** Failed to assign a port to thread 197 (errno = 98)
** Failed to assign a port to thread 198 (errno = 98)
** Failed to assign a port to thread 199 (errno = 98)
**
ERROR:/home/peter.maydell/qemu/tests/test-listen.c:195:listen_compete_nthr:
assertion failed (failed_listens == 0): (200 == 0)
FAIL
GTester: last random seed: R02Sbad2a742dbd146e85cb412ce8f8f3750
(pid=321)
  /socket/listen-serial/generic:                                       OK
  /socket/listen-compete/ipv4:                                         OK
  /socket/listen-compete/ipv6:                                         OK
  /socket/listen-compete/generic:                                      OK
FAIL: tests/test-listen

Compile failure for the test on FreeBSD:

/root/qemu/tests/test-listen.c:53:19: error: use of undeclared
identifier 'EAI_ADDRFAMILY'
        if (rc == EAI_ADDRFAMILY ||
                  ^


Test failure on OpenBSD:

TEST: tests/test-listen... (pid=9208)
  /socket/listen-serial/ipv4:
** Failed to assign a port to
thread 124 (errno = 48)
** Failed to assign a port to thread 125 (errno = 48)
** Failed to assign a port to thread 126 (errno = 48)
** Failed to assign a port to thread 127 (errno = 48)
[repeats up to]
** Failed to assign a port to thread 197 (errno = 48)
** Failed to assign a port to thread 198 (errno = 48)
** Failed to assign a port to thread 199 (errno = 48)
**
ERROR:/home/qemu/tests/test-listen.c:195:listen_compete_nthr:
assertion failed (failed_listens == 0): (76 == 0)
FAIL
GTester: last random seed: R02S17d50ab00329e514ffbe630191e76e83
(pid=80652)

& similarly for the other subtests.

Linux x86-64:

make: Entering directory '/home/petmay01/linaro/qemu-for-merges/build/all'
  GTESTER tests/test-listen
** Failed to assign a port to thread 0 (errno = 98)
** Failed to assign a port to thread 1 (errno = 98)
** Failed to assign a port to thread 2 (errno = 98)
** Failed to assign a port to thread 3 (errno = 98)
** Failed to assign a port to thread 4 (errno = 98)
** Failed to assign a port to thread 5 (errno = 98)
** Failed to assign a port to thread 6 (errno = 98)
[up to]
** Failed to assign a port to thread 199 (errno = 98)
**
ERROR:/home/petmay01/linaro/qemu-for-merges/tests/test-listen.c:195:listen_compete_nthr:
assertion failed (failed_listens == 0): (200 == 0)
GTester: last random seed: R02S041a056d256c9ae190cf026f5adfd979
/home/petmay01/linaro/qemu-for-merges/tests/Makefile.include:863:
recipe for target 'check-tests/test-listen' failed

thanks
-- PMM

On Mon, Nov 06, 2017 at 04:11:56PM +0000, Peter Maydell wrote:
> On 6 November 2017 at 15:33, Daniel P. Berrange <berrange@redhat.com> wrote:
> > The following changes since commit ec7a8bf0b8f7dc7288fe8745464ee8217528cc6c:
> >
> >   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2017-11-06 10:04:16 +0000)
> >
> > are available in the git repository at:
> >
> >   git://github.com/berrange/qemu tags/pull-qio-2017-11-06-1
> >
> > for you to fetch changes up to 74c0035408d5b327eac5985d976bca3009e0c5d6:
> >
> >   tests: Add test-listen - a stress test for QEMU socket listen (2017-11-06 11:49:15 +0000)
> >
> > ----------------------------------------------------------------
> > Merge qio 2017-11-06 v1
> >
> > ----------------------------------------------------------------
> 
> Hi. I'm afraid this fails on various hosts.

Sigh, anything network related in unit tests is soo hard to get right :-(
I was hoping the previous fail you saw with this test was all due to the
file descriptor leak the first patch fixed, but I guess not....

> 
> arm 32 bit (in a chroot on a 64 bit box):
> TEST: tests/test-listen... (pid=319)
>   /socket/listen-serial/ipv4:                                          OK
>   /socket/listen-serial/ipv6:
> ** Failed to assign a port to
> thread 0 (errno = 98)
> ** Failed to assign a port to thread 1 (errno = 98)
> ** Failed to assign a port to thread 2 (errno = 98)
> ** Failed to assign a port to thread 3 (errno = 98)
> ** Failed to assign a port to thread 4 (errno = 98)
> [...repeats all the way up to...]
> ** Failed to assign a port to thread 197 (errno = 98)
> ** Failed to assign a port to thread 198 (errno = 98)
> ** Failed to assign a port to thread 199 (errno = 98)
> **
> ERROR:/home/peter.maydell/qemu/tests/test-listen.c:195:listen_compete_nthr:
> assertion failed (failed_listens == 0): (200 == 0)

Hmm, every single thread failed, which suggests some fundamental config in
this environment that prevents any use of networking at all from tests.

Anything interesting about this build env related to network availability
that would prevent use of even localhost ?

> FAIL
> GTester: last random seed: R02Sbad2a742dbd146e85cb412ce8f8f3750
> (pid=321)
>   /socket/listen-serial/generic:                                       OK
>   /socket/listen-compete/ipv4:                                         OK
>   /socket/listen-compete/ipv6:                                         OK
>   /socket/listen-compete/generic:                                      OK
> FAIL: tests/test-listen
> 
> Compile failure for the test on FreeBSD:
> 
> /root/qemu/tests/test-listen.c:53:19: error: use of undeclared
> identifier 'EAI_ADDRFAMILY'
>         if (rc == EAI_ADDRFAMILY ||
>                   ^

Ok, that's at least an easy one to fix.

> 
> 
> Test failure on OpenBSD:
> 
> TEST: tests/test-listen... (pid=9208)
>   /socket/listen-serial/ipv4:
> ** Failed to assign a port to
> thread 124 (errno = 48)
> ** Failed to assign a port to thread 125 (errno = 48)
> ** Failed to assign a port to thread 126 (errno = 48)
> ** Failed to assign a port to thread 127 (errno = 48)
> [repeats up to]
> ** Failed to assign a port to thread 197 (errno = 48)
> ** Failed to assign a port to thread 198 (errno = 48)
> ** Failed to assign a port to thread 199 (errno = 48)
> **
> ERROR:/home/qemu/tests/test-listen.c:195:listen_compete_nthr:
> assertion failed (failed_listens == 0): (76 == 0)
> FAIL
> GTester: last random seed: R02S17d50ab00329e514ffbe630191e76e83
> (pid=80652)

Only a subset failed, which is more interesting - this is definitely
more like the problem seen when FD leaks. Could you mention what the
ulimits are for your OpenBSD host - I wondering if it hits max FDs
due to having a parallel builds / test run.


> & similarly for the other subtests.
> 
> Linux x86-64:
> 
> make: Entering directory '/home/petmay01/linaro/qemu-for-merges/build/all'
>   GTESTER tests/test-listen
> ** Failed to assign a port to thread 0 (errno = 98)
> ** Failed to assign a port to thread 1 (errno = 98)
> ** Failed to assign a port to thread 2 (errno = 98)
> ** Failed to assign a port to thread 3 (errno = 98)
> ** Failed to assign a port to thread 4 (errno = 98)
> ** Failed to assign a port to thread 5 (errno = 98)
> ** Failed to assign a port to thread 6 (errno = 98)
> [up to]
> ** Failed to assign a port to thread 199 (errno = 98)
> **
> ERROR:/home/petmay01/linaro/qemu-for-merges/tests/test-listen.c:195:listen_compete_nthr:
> assertion failed (failed_listens == 0): (200 == 0)
> GTester: last random seed: R02S041a056d256c9ae190cf026f5adfd979
> /home/petmay01/linaro/qemu-for-merges/tests/Makefile.include:863:
> recipe for target 'check-tests/test-listen' failed

Looks same as your arm env, which suggests no network access at all.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On 6 November 2017 at 17:43, Daniel P. Berrange <berrange@redhat.com> wrote:
> On Mon, Nov 06, 2017 at 04:11:56PM +0000, Peter Maydell wrote:
>> On 6 November 2017 at 15:33, Daniel P. Berrange <berrange@redhat.com> wrote:
>> > The following changes since commit ec7a8bf0b8f7dc7288fe8745464ee8217528cc6c:
>> >
>> >   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2017-11-06 10:04:16 +0000)
>> >
>> > are available in the git repository at:
>> >
>> >   git://github.com/berrange/qemu tags/pull-qio-2017-11-06-1
>> >
>> > for you to fetch changes up to 74c0035408d5b327eac5985d976bca3009e0c5d6:
>> >
>> >   tests: Add test-listen - a stress test for QEMU socket listen (2017-11-06 11:49:15 +0000)
>> >
>> > ----------------------------------------------------------------
>> > Merge qio 2017-11-06 v1
>> >
>> > ----------------------------------------------------------------
>>
>> Hi. I'm afraid this fails on various hosts.
>
> Sigh, anything network related in unit tests is soo hard to get right :-(
> I was hoping the previous fail you saw with this test was all due to the
> file descriptor leak the first patch fixed, but I guess not....
>
>>
>> arm 32 bit (in a chroot on a 64 bit box):
>> TEST: tests/test-listen... (pid=319)
>>   /socket/listen-serial/ipv4:                                          OK
>>   /socket/listen-serial/ipv6:
>> ** Failed to assign a port to
>> thread 0 (errno = 98)
>> ** Failed to assign a port to thread 1 (errno = 98)
>> ** Failed to assign a port to thread 2 (errno = 98)
>> ** Failed to assign a port to thread 3 (errno = 98)
>> ** Failed to assign a port to thread 4 (errno = 98)
>> [...repeats all the way up to...]
>> ** Failed to assign a port to thread 197 (errno = 98)
>> ** Failed to assign a port to thread 198 (errno = 98)
>> ** Failed to assign a port to thread 199 (errno = 98)
>> **
>> ERROR:/home/peter.maydell/qemu/tests/test-listen.c:195:listen_compete_nthr:
>> assertion failed (failed_listens == 0): (200 == 0)
>
> Hmm, every single thread failed, which suggests some fundamental config in
> this environment that prevents any use of networking at all from tests.
>
> Anything interesting about this build env related to network availability
> that would prevent use of even localhost ?

You'll notice that the IPV4 test worked fine. I assume this is an
IPV6 specific problem.

>> Test failure on OpenBSD:
>>
>> TEST: tests/test-listen... (pid=9208)
>>   /socket/listen-serial/ipv4:
>> ** Failed to assign a port to
>> thread 124 (errno = 48)
>> ** Failed to assign a port to thread 125 (errno = 48)
>> ** Failed to assign a port to thread 126 (errno = 48)
>> ** Failed to assign a port to thread 127 (errno = 48)
>> [repeats up to]
>> ** Failed to assign a port to thread 197 (errno = 48)
>> ** Failed to assign a port to thread 198 (errno = 48)
>> ** Failed to assign a port to thread 199 (errno = 48)
>> **
>> ERROR:/home/qemu/tests/test-listen.c:195:listen_compete_nthr:
>> assertion failed (failed_listens == 0): (76 == 0)
>> FAIL
>> GTester: last random seed: R02S17d50ab00329e514ffbe630191e76e83
>> (pid=80652)
>
> Only a subset failed, which is more interesting - this is definitely
> more like the problem seen when FD leaks. Could you mention what the
> ulimits are for your OpenBSD host - I wondering if it hits max FDs
> due to having a parallel builds / test run.

# ulimit -a
time(cpu-seconds)    unlimited
file(blocks)         unlimited
coredump(blocks)     unlimited
data(kbytes)         33554432
stack(kbytes)        8192
lockedmem(kbytes)    670784
memory(kbytes)       2010588
nofiles(descriptors) 128
processes            1310

That nofiles limit looks quite close to the number where we
start failing...

>> Linux x86-64:
>>
>> make: Entering directory '/home/petmay01/linaro/qemu-for-merges/build/all'
>>   GTESTER tests/test-listen
>> ** Failed to assign a port to thread 0 (errno = 98)
>> ** Failed to assign a port to thread 1 (errno = 98)
>> ** Failed to assign a port to thread 2 (errno = 98)
>> ** Failed to assign a port to thread 3 (errno = 98)
>> ** Failed to assign a port to thread 4 (errno = 98)
>> ** Failed to assign a port to thread 5 (errno = 98)
>> ** Failed to assign a port to thread 6 (errno = 98)
>> [up to]
>> ** Failed to assign a port to thread 199 (errno = 98)
>> **
>> ERROR:/home/petmay01/linaro/qemu-for-merges/tests/test-listen.c:195:listen_compete_nthr:
>> assertion failed (failed_listens == 0): (200 == 0)
>> GTester: last random seed: R02S041a056d256c9ae190cf026f5adfd979
>> /home/petmay01/linaro/qemu-for-merges/tests/Makefile.include:863:
>> recipe for target 'check-tests/test-listen' failed
>
> Looks same as your arm env, which suggests no network access at all.

This is running native on my primary development machine, so its
network access is fine :-)

thanks
-- PMM

On Mon, Nov 06, 2017 at 05:58:41PM +0000, Peter Maydell wrote:
> On 6 November 2017 at 17:43, Daniel P. Berrange <berrange@redhat.com> wrote:
> > On Mon, Nov 06, 2017 at 04:11:56PM +0000, Peter Maydell wrote:
> >> On 6 November 2017 at 15:33, Daniel P. Berrange <berrange@redhat.com> wrote:
> >> > The following changes since commit ec7a8bf0b8f7dc7288fe8745464ee8217528cc6c:
> >> >
> >> >   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2017-11-06 10:04:16 +0000)
> >> >
> >> > are available in the git repository at:
> >> >
> >> >   git://github.com/berrange/qemu tags/pull-qio-2017-11-06-1
> >> >
> >> > for you to fetch changes up to 74c0035408d5b327eac5985d976bca3009e0c5d6:
> >> >
> >> >   tests: Add test-listen - a stress test for QEMU socket listen (2017-11-06 11:49:15 +0000)
> >> >
> >> > ----------------------------------------------------------------
> >> > Merge qio 2017-11-06 v1
> >> >
> >> > ----------------------------------------------------------------
> >>
> >> Hi. I'm afraid this fails on various hosts.
> >
> > Sigh, anything network related in unit tests is soo hard to get right :-(
> > I was hoping the previous fail you saw with this test was all due to the
> > file descriptor leak the first patch fixed, but I guess not....
> >
> >>
> >> arm 32 bit (in a chroot on a 64 bit box):
> >> TEST: tests/test-listen... (pid=319)
> >>   /socket/listen-serial/ipv4:                                          OK
> >>   /socket/listen-serial/ipv6:
> >> ** Failed to assign a port to
> >> thread 0 (errno = 98)
> >> ** Failed to assign a port to thread 1 (errno = 98)
> >> ** Failed to assign a port to thread 2 (errno = 98)
> >> ** Failed to assign a port to thread 3 (errno = 98)
> >> ** Failed to assign a port to thread 4 (errno = 98)
> >> [...repeats all the way up to...]
> >> ** Failed to assign a port to thread 197 (errno = 98)
> >> ** Failed to assign a port to thread 198 (errno = 98)
> >> ** Failed to assign a port to thread 199 (errno = 98)
> >> **
> >> ERROR:/home/peter.maydell/qemu/tests/test-listen.c:195:listen_compete_nthr:
> >> assertion failed (failed_listens == 0): (200 == 0)
> >
> > Hmm, every single thread failed, which suggests some fundamental config in
> > this environment that prevents any use of networking at all from tests.
> >
> > Anything interesting about this build env related to network availability
> > that would prevent use of even localhost ?
> 
> You'll notice that the IPV4 test worked fine. I assume this is an
> IPV6 specific problem.

Oh yes, so it is.  The strange thing is that the test does check if we have
a protocol available by doing a test bind() at start.

> >> Test failure on OpenBSD:
> >>
> >> TEST: tests/test-listen... (pid=9208)
> >>   /socket/listen-serial/ipv4:
> >> ** Failed to assign a port to
> >> thread 124 (errno = 48)
> >> ** Failed to assign a port to thread 125 (errno = 48)
> >> ** Failed to assign a port to thread 126 (errno = 48)
> >> ** Failed to assign a port to thread 127 (errno = 48)
> >> [repeats up to]
> >> ** Failed to assign a port to thread 197 (errno = 48)
> >> ** Failed to assign a port to thread 198 (errno = 48)
> >> ** Failed to assign a port to thread 199 (errno = 48)
> >> **
> >> ERROR:/home/qemu/tests/test-listen.c:195:listen_compete_nthr:
> >> assertion failed (failed_listens == 0): (76 == 0)
> >> FAIL
> >> GTester: last random seed: R02S17d50ab00329e514ffbe630191e76e83
> >> (pid=80652)
> >
> > Only a subset failed, which is more interesting - this is definitely
> > more like the problem seen when FD leaks. Could you mention what the
> > ulimits are for your OpenBSD host - I wondering if it hits max FDs
> > due to having a parallel builds / test run.
> 
> # ulimit -a
> time(cpu-seconds)    unlimited
> file(blocks)         unlimited
> coredump(blocks)     unlimited
> data(kbytes)         33554432
> stack(kbytes)        8192
> lockedmem(kbytes)    670784
> memory(kbytes)       2010588
> nofiles(descriptors) 128
> processes            1310
> 
> That nofiles limit looks quite close to the number where we
> start failing...

Ok, I think I'll just make the test skip unless nofiles >= 1024
so we have a good margin of safety when things run very parallel


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Tue, Nov 07, 2017 at 09:19:45AM +0000, Daniel P. Berrange wrote:
> On Mon, Nov 06, 2017 at 05:58:41PM +0000, Peter Maydell wrote:
> > On 6 November 2017 at 17:43, Daniel P. Berrange <berrange@redhat.com> wrote:
> > > On Mon, Nov 06, 2017 at 04:11:56PM +0000, Peter Maydell wrote:
> > >> On 6 November 2017 at 15:33, Daniel P. Berrange <berrange@redhat.com> wrote:
> > >> > The following changes since commit ec7a8bf0b8f7dc7288fe8745464ee8217528cc6c:
> > >> >
> > >> >   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2017-11-06 10:04:16 +0000)
> > >> >
> > >> > are available in the git repository at:
> > >> >
> > >> >   git://github.com/berrange/qemu tags/pull-qio-2017-11-06-1
> > >> >
> > >> > for you to fetch changes up to 74c0035408d5b327eac5985d976bca3009e0c5d6:
> > >> >
> > >> >   tests: Add test-listen - a stress test for QEMU socket listen (2017-11-06 11:49:15 +0000)
> > >> >
> > >> > ----------------------------------------------------------------
> > >> > Merge qio 2017-11-06 v1
> > >> >
> > >> > ----------------------------------------------------------------
> > >>
> > >> Hi. I'm afraid this fails on various hosts.
> > >
> > > Sigh, anything network related in unit tests is soo hard to get right :-(
> > > I was hoping the previous fail you saw with this test was all due to the
> > > file descriptor leak the first patch fixed, but I guess not....
> > >
> > >>
> > >> arm 32 bit (in a chroot on a 64 bit box):
> > >> TEST: tests/test-listen... (pid=319)
> > >>   /socket/listen-serial/ipv4:                                          OK
> > >>   /socket/listen-serial/ipv6:
> > >> ** Failed to assign a port to
> > >> thread 0 (errno = 98)
> > >> ** Failed to assign a port to thread 1 (errno = 98)
> > >> ** Failed to assign a port to thread 2 (errno = 98)
> > >> ** Failed to assign a port to thread 3 (errno = 98)
> > >> ** Failed to assign a port to thread 4 (errno = 98)
> > >> [...repeats all the way up to...]
> > >> ** Failed to assign a port to thread 197 (errno = 98)
> > >> ** Failed to assign a port to thread 198 (errno = 98)
> > >> ** Failed to assign a port to thread 199 (errno = 98)
> > >> **
> > >> ERROR:/home/peter.maydell/qemu/tests/test-listen.c:195:listen_compete_nthr:
> > >> assertion failed (failed_listens == 0): (200 == 0)
> > >
> > > Hmm, every single thread failed, which suggests some fundamental config in
> > > this environment that prevents any use of networking at all from tests.
> > >
> > > Anything interesting about this build env related to network availability
> > > that would prevent use of even localhost ?
> > 
> > You'll notice that the IPV4 test worked fine. I assume this is an
> > IPV6 specific problem.
> 
> Oh yes, so it is.  The strange thing is that the test does check if we have
> a protocol available by doing a test bind() at start.
> 
> > >> Test failure on OpenBSD:
> > >>
> > >> TEST: tests/test-listen... (pid=9208)
> > >>   /socket/listen-serial/ipv4:
> > >> ** Failed to assign a port to
> > >> thread 124 (errno = 48)
> > >> ** Failed to assign a port to thread 125 (errno = 48)
> > >> ** Failed to assign a port to thread 126 (errno = 48)
> > >> ** Failed to assign a port to thread 127 (errno = 48)
> > >> [repeats up to]
> > >> ** Failed to assign a port to thread 197 (errno = 48)
> > >> ** Failed to assign a port to thread 198 (errno = 48)
> > >> ** Failed to assign a port to thread 199 (errno = 48)
> > >> **
> > >> ERROR:/home/qemu/tests/test-listen.c:195:listen_compete_nthr:
> > >> assertion failed (failed_listens == 0): (76 == 0)
> > >> FAIL
> > >> GTester: last random seed: R02S17d50ab00329e514ffbe630191e76e83
> > >> (pid=80652)
> > >
> > > Only a subset failed, which is more interesting - this is definitely
> > > more like the problem seen when FD leaks. Could you mention what the
> > > ulimits are for your OpenBSD host - I wondering if it hits max FDs
> > > due to having a parallel builds / test run.
> > 
> > # ulimit -a
> > time(cpu-seconds)    unlimited
> > file(blocks)         unlimited
> > coredump(blocks)     unlimited
> > data(kbytes)         33554432
> > stack(kbytes)        8192
> > lockedmem(kbytes)    670784
> > memory(kbytes)       2010588
> > nofiles(descriptors) 128
> > processes            1310
> > 
> > That nofiles limit looks quite close to the number where we
> > start failing...
> 
> Ok, I think I'll just make the test skip unless nofiles >= 1024
> so we have a good margin of safety when things run very parallel

FYI, I've dropped the test suite addition in my 2nd PULL request, as I'm
not confident I can solve all the portability problems quickly, and don't
want to waste your time going back + forth testing new versions during
freeze.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|