https://bugzilla.redhat.com/1693587

'QEMU 4.2.0 and later, combined with a sufficiently recent guest
virtio-net driver, supports setting up a simple network bond device
comprised of one virtio emulated NIC and one hostdev NIC (which must
be an SRIOV VF). The allure of this setup is that the bond will always
favor the hostdev device, providing better performance, until the
guest is migrated - at that time QEMU will automatically unplug the
hostdev NIC and the bond will send all traffic via the virtio NIC
until migration is completed, then QEMU on the destination side will
hotplug a new hostdev NIC and the bond will switch back to using the
hostdev for network traffic. The result is that guests desiring the
extra performance of a hostdev NIC are now migratable without network
downtime (performance is just degraded during migration) and without
requiring a complicated bonding configuration in the guest OS network
config and complicated unplug/replug logic in the management
application on the host - it can instead all be accomplished in
libvirt with the interface <driver> subelement "failover" and
"backupAlias" attributes.

Patches 1-4 are just simple refactoring with no functional change,
5-10 are the new functionality, patch 11 is documentation, and Patch
12 is an RFC for a method to solve a problem that oVirt has when
trying to use this feature - while the virtio guest driver requires
the pair of interfaces to have matching MAC addresses, oVirt requires
every network interface to have a unique MAC. I'm not sure that I like
having this hackishness in libvirt (see the commit log message), but
it does solve oVirt's problem, and also makes direct config with
libvirt XML simpler (since it removes the need to manually specify any
MAC addresses in order to arrive at a working config, which is
something that has always been the case before now). I'll leave it up to the jury to decide whether or not it's acceptable :-)

Laine Stump (12):
  conf: refactor hostdev driver subelement format for simpler additions
  conf: change virDomainVirtioNet*Format() to return void
  conf: rename two static functions
  conf: refactor <interface>'s <driver> subelement parse/format
  qemu: add capabilities flag for failover feature
  conf: add failover attribute to <driver> subelement of <interface>
  qemu: add backupAlias attribute to <driver> subelement of hostdev
    devices
  conf: add backupAlias attribute to <interface> driver subelement
  qemu: allow migration with assigned PCI hostdev if backupAlias is set
  qemu: add wait-unplug to qemu migration status enum
  docs: document virtio failover / QEMU auto-plug of hostdev during
    migration
  conf/qemu: new <driver> attribute "useBackupMAC"

 docs/formatdomain.html.in                     |  74 +++
 docs/news.xml                                 |  27 +
 docs/schemas/domaincommon.rng                 |  15 +
 src/conf/domain_conf.c                        | 559 ++++++++++--------
 src/conf/domain_conf.h                        |  57 +-
 src/libxl/libxl_driver.c                      |   2 +-
 src/qemu/qemu_capabilities.c                  |   2 +
 src/qemu/qemu_capabilities.h                  |   1 +
 src/qemu/qemu_command.c                       |   5 +
 src/qemu/qemu_domain.c                        |  21 +-
 src/qemu/qemu_hostdev.c                       |   5 +-
 src/qemu/qemu_hostdev.h                       |   1 +
 src/qemu/qemu_hotplug.c                       |   2 +-
 src/qemu/qemu_migration.c                     |  49 +-
 src/qemu/qemu_monitor.c                       |   1 +
 src/qemu/qemu_monitor.h                       |   1 +
 src/qemu/qemu_monitor_json.c                  |   1 +
 src/util/virhostdev.c                         |  47 +-
 src/util/virhostdev.h                         |   1 +
 .../caps_4.2.0.aarch64.xml                    |   1 +
 .../caps_4.2.0.x86_64.xml                     |   1 +
 .../net-virtio-failover-network.xml           |  37 ++
 .../qemuxml2argvdata/net-virtio-failover.args |  40 ++
 .../qemuxml2argvdata/net-virtio-failover.xml  |  50 ++
 tests/qemuxml2argvtest.c                      |   4 +
 .../net-virtio-failover-network.xml           |  51 ++
 .../net-virtio-failover.xml                   |  66 +++
 tests/qemuxml2xmltest.c                       |   6 +
 tests/virhostdevtest.c                        |  18 +-
 29 files changed, 856 insertions(+), 289 deletions(-)
 create mode 100644 tests/qemuxml2argvdata/net-virtio-failover-network.xml
 create mode 100644 tests/qemuxml2argvdata/net-virtio-failover.args
 create mode 100644 tests/qemuxml2argvdata/net-virtio-failover.xml
 create mode 100644 tests/qemuxml2xmloutdata/net-virtio-failover-network.xml
 create mode 100644 tests/qemuxml2xmloutdata/net-virtio-failover.xml

-- 
2.24.1

I forgot to mention in the cover letter - there is a bug in the QEMU 
4.2.0 release that causes the qemu process to crash whenever the vfio 
device is unplugged, and for some reason when booting a Fedora 31 guest 
with a failover pair, the guest kernel attempts to unplug the vfio 
device during its initial boot! Fortunately that bug is already fixed, 
by this qemu commit:


  https://git.qemu.org/?p=qemu.git;a=commitdiff;h=0446f8121723b134ca1d1ed0b73e96d4a0a8689d;hp=48008198270e3ebcc9394401d676c54ed5ac139c


We've also noticed that if the vfio device of the pair is manually 
unplugged from the guest, the DEVICE_DELETED event never makes it back 
up to libvirt, and so the device isn't removed from the libvirt config 
for the guest, is marked as still in-use, and can't be re-plugged into 
the same or any other guest until that particular guest is shut down. We 
still haven't found the root cause of that problem, but it seems likely 
it's not in these patches (more likely an issue in qemu or in the 
guest), so I posted them :-)