Hi!

This patch series attempts to reduce the number of failing tests on macOS.

The fixes involve some funk with macOS dynamic and static linkers, dyld and
ld64, respectively.

As result, instead of 15 failing tests we get only 5.
The tests have been fixed:
  qemublocktest
  qemumonitorjsontest
  viriscsitest
  virmacmaptest
  virnetserverclienttest
  vircryptotest
  qemufirmwaretest
  domaincapstest
  commandtest
  sockettest

The tests are still failing:
  qemumemlocktest
  storagepoolxml2argvtest
  qemuxml2xmltest
  qemusecuritytest
  qemuxml2argvtest

qemucapsprobe doesn't yet works but I started working on the fix.

The failing tests depend on virpcimock that is guarded by ifdefs so no
functions are injected and the mock is no-op on macOS. How can we fix
the tests that rely on the mock? Should we select only specific tests to
run on macOS or we should make virpci mock cross-platform?  Skipping
them entirely is not an option IMO as I think qemu driver can be used on
macOS with qemu/hvf/haxm domains and the tests are helpful for the
domains.

And as soon as we get working tests and qemucapsprobe I'd want to resend hvf
patchset.

Best regards,
Roman

Roman Bolshakov (11):
  tests: Don't test octal localhost IP in sockettest on macOS
  tests: Avoid IPv4-translated IPv6 address in sockettest
  tests: Preload mocks with DYLD_INSERT_LIBRARIES on macOS
  tests: Add lib- prefix to all mocks
  tests: Remove -module flag for mocks
  tests: Drop /private CWD prefix in commandhelper
  build: Use flat namespace for libvirt on macOS
  tests: Lookup extended stat/lstat in mocks
  tests: Use flat namespace on macOS
  tests: Avoid gnulib replacements in mocks
  tests: Make references to global symbols indirect in test drivers

 configure.ac                   |   1 +
 src/Makefile.am                |   9 +-
 tests/Makefile.am              | 199 +++++++++++++++++----------------
 tests/bhyveargv2xmltest.c      |   2 +-
 tests/bhyvexml2argvtest.c      |   2 +-
 tests/bhyvexml2xmltest.c       |   2 +-
 tests/commandhelper.c          |   9 ++
 tests/domaincapstest.c         |   6 +-
 tests/fchosttest.c             |   2 +-
 tests/libxlxml2domconfigtest.c |   2 +-
 tests/nsstest.c                |   2 +-
 tests/qemucaps2xmltest.c       |   2 +-
 tests/qemucapsprobe.c          |   2 +-
 tests/qemumemlocktest.c        |   3 +-
 tests/qemumonitorjsontest.c    |   2 +-
 tests/qemuxml2argvtest.c       |   8 +-
 tests/qemuxml2xmltest.c        |   6 +-
 tests/sockettest.c             |   6 +-
 tests/testutils.c              |   4 +-
 tests/testutils.h              |  18 ++-
 tests/vircaps2xmltest.c        |   2 +-
 tests/vircgrouptest.c          |   2 +-
 tests/vircryptotest.c          |   2 +-
 tests/virfilecachetest.c       |   2 +-
 tests/virfiletest.c            |   2 +-
 tests/virfilewrapper.c         |   5 +
 tests/virfirewalltest.c        |   2 +-
 tests/virhostcputest.c         |   2 +-
 tests/virhostdevtest.c         |   2 +-
 tests/viriscsitest.c           |   3 +-
 tests/virmacmaptest.c          |   2 +-
 tests/virmock.h                |  10 ++
 tests/virmockstathelpers.c     |  18 +++
 tests/virnetdaemontest.c       |   2 +-
 tests/virnetdevbandwidthtest.c |   2 +-
 tests/virnetdevtest.c          |   2 +-
 tests/virnetserverclienttest.c |   2 +-
 tests/virnettlscontexttest.c   |   2 +-
 tests/virnettlssessiontest.c   |   2 +-
 tests/virpcitest.c             |   2 +-
 tests/virpolkittest.c          |   2 +-
 tests/virportallocatortest.c   |   2 +-
 tests/virsystemdtest.c         |   2 +-
 tests/virusbtest.c             |   2 +-
 44 files changed, 214 insertions(+), 149 deletions(-)

-- 
2.22.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On Wed, Aug 21, 2019 at 07:13:12PM +0300, Roman Bolshakov wrote:
> Hi!
> 
> This patch series attempts to reduce the number of failing tests on macOS.
> 
> The fixes involve some funk with macOS dynamic and static linkers, dyld and
> ld64, respectively.
> 
> As result, instead of 15 failing tests we get only 5.
> The tests have been fixed:
>   qemublocktest
>   qemumonitorjsontest
>   viriscsitest
>   virmacmaptest
>   virnetserverclienttest
>   vircryptotest
>   qemufirmwaretest
>   domaincapstest
>   commandtest
>   sockettest
> 
> The tests are still failing:
>   qemumemlocktest
>   storagepoolxml2argvtest
>   qemuxml2xmltest
>   qemusecuritytest
>   qemuxml2argvtest
> 
> qemucapsprobe doesn't yet works but I started working on the fix.
> 
> The failing tests depend on virpcimock that is guarded by ifdefs so no
> functions are injected and the mock is no-op on macOS. How can we fix
> the tests that rely on the mock? Should we select only specific tests to
> run on macOS or we should make virpci mock cross-platform?  Skipping
> them entirely is not an option IMO as I think qemu driver can be used on
> macOS with qemu/hvf/haxm domains and the tests are helpful for the
> domains.

Realistically the PCI code will only ever execute on Linux, since
it uses VFIO linux kernel features, so if we don't test PCI bits
on macOS that's fine.

That said though, I'd be fine if you wanted to make the virpcimock
cross-platform too.

I'd pick whichever strategy results in the nicest code to maintain


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On Wed, Aug 21, 2019 at 06:18:11PM +0100, Daniel P. Berrangé wrote:
> On Wed, Aug 21, 2019 at 07:13:12PM +0300, Roman Bolshakov wrote:
> > Hi!
> > 
> > This patch series attempts to reduce the number of failing tests on macOS.
> > 
> > The fixes involve some funk with macOS dynamic and static linkers, dyld and
> > ld64, respectively.
> > 
> > As result, instead of 15 failing tests we get only 5.
> > The tests have been fixed:
> >   qemublocktest
> >   qemumonitorjsontest
> >   viriscsitest
> >   virmacmaptest
> >   virnetserverclienttest
> >   vircryptotest
> >   qemufirmwaretest
> >   domaincapstest
> >   commandtest
> >   sockettest
> > 
> > The tests are still failing:
> >   qemumemlocktest
> >   storagepoolxml2argvtest
> >   qemuxml2xmltest
> >   qemusecuritytest
> >   qemuxml2argvtest
> > 
> > qemucapsprobe doesn't yet works but I started working on the fix.
> > 
> > The failing tests depend on virpcimock that is guarded by ifdefs so no
> > functions are injected and the mock is no-op on macOS. How can we fix
> > the tests that rely on the mock? Should we select only specific tests to
> > run on macOS or we should make virpci mock cross-platform?  Skipping
> > them entirely is not an option IMO as I think qemu driver can be used on
> > macOS with qemu/hvf/haxm domains and the tests are helpful for the
> > domains.
> 
> Realistically the PCI code will only ever execute on Linux, since
> it uses VFIO linux kernel features, so if we don't test PCI bits
> on macOS that's fine.
> 
> That said though, I'd be fine if you wanted to make the virpcimock
> cross-platform too.
> 
> I'd pick whichever strategy results in the nicest code to maintain

BTW, I've acked all patches, but will give a little more time
in case anyone else has comments, before pushing. 


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list