include/hw/virtio/virtio-net.h | 1 + hw/net/virtio-net.c | 49 +++++++++++++++++++++++++++++----- 2 files changed, 44 insertions(+), 6 deletions(-)
v2: use validate_features() to disable the guest driver rather
than setting vring.num to 0.
With failover, when the guest virtio-net driver doesn't support the
STANDBY feature, the primary device is not plugged and only the virtio-net
device is kept. Doing like that we can migrate the machine and
keep the network connection.
But in some cases, when performance is more important than availability
we would prefer to keep the VFIO device rather than the virtio-net one,
even if it means to lose the network connection during the migration of
the machine.
To do that we can't simply unplug the virtio-net device and plug the
VFIO one because for the migration the initial state must be kept
(virtio-net plugged, VFIO unplugged) but we can try to disable the
virtio-net driver and plug the VFIO card, so the initial state is
correct (the virtio-net card is plugged, but disabled in guest, and
the VFIO card is unplugged before migration).
This change doesn't impact the case when guest and host support
the STANDBY feature.
I've introduced the "failover-default" property to virtio-net device
to set which device to keep (failover-default=true keeps the virtio-net
device, =off the other one).
For example, with a guest driver that doesn't support STANDBY:
...
-device virtio-net-pci,id=virtio0,failover=on,failover-default=on \
-device vfio-pci,host=$PCI,id=hostdev0,failover_pair_id=virtio0 \
...
[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP q0
link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff
inet 192.168.20.2/24 brd 192.168.20.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::2428:c5ff:fe7f:1424/64 scope link
valid_lft forever preferred_lft forever
# ethtool -i eth0
driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: 0000:04:00.0
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
...
-device virtio-net-pci,id=virtio0,failover=on,failover-default=off \
-device vfio-pci,host=$PCI,id=hostdev0,failover_pair_id=virtio0 \
...
[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 100
link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff
inet 192.168.20.2/24 brd 192.168.20.255 scope global enp2s0
valid_lft forever preferred_lft forever
inet6 fe80::2428:c5ff:fe7f:1424/64 scope link
valid_lft forever preferred_lft forever
[root@localhost ~]# ethtool -i enp2s0
driver: i40evf
version: 1.6.27-k
firmware-version: N/A
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
With guest driver that supports STANDBY, we would always have:
[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group defau0
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP gr0
link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff
inet 192.168.20.2/24 brd 192.168.20.255 scope global noprefixroute enp4s0
valid_lft forever preferred_lft forever
inet6 fe80::2428:c5ff:fe7f:1424/64 scope link
valid_lft forever preferred_lft forever
3: enp4s0nsby: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master0
link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff
4: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master enp4s0 st0
link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff
[root@localhost ~]# ethtool -i enp4s0
driver: net_failover
version: 0.1
firmware-version:
expansion-rom-version:
bus-info:
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
[root@localhost ~]# ethtool -i enp4s0nsby
driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: 0000:04:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
[root@localhost ~]# ethtool -i enp2s0
driver: iavf
version: 4.18.0-310.el8.x86_64
firmware-version: N/A
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes
Laurent Vivier (1):
virtio: failover: define the default device to use in case of error
include/hw/virtio/virtio-net.h | 1 +
hw/net/virtio-net.c | 49 +++++++++++++++++++++++++++++-----
2 files changed, 44 insertions(+), 6 deletions(-)
--
2.31.1
On Tue, Aug 10, 2021 at 1:13 AM Laurent Vivier <lvivier@redhat.com> wrote: > > v2: use validate_features() to disable the guest driver rather > than setting vring.num to 0. > > With failover, when the guest virtio-net driver doesn't support the > STANDBY feature, the primary device is not plugged and only the virtio-net > device is kept. Doing like that we can migrate the machine and > keep the network connection. > > But in some cases, when performance is more important than availability > we would prefer to keep the VFIO device rather than the virtio-net one, > even if it means to lose the network connection during the migration of > the machine. I think we still need to seek a way to recover the network after migration. Thanks > > To do that we can't simply unplug the virtio-net device and plug the > VFIO one because for the migration the initial state must be kept > (virtio-net plugged, VFIO unplugged) but we can try to disable the > virtio-net driver and plug the VFIO card, so the initial state is > correct (the virtio-net card is plugged, but disabled in guest, and > the VFIO card is unplugged before migration). > > This change doesn't impact the case when guest and host support > the STANDBY feature. > > I've introduced the "failover-default" property to virtio-net device > to set which device to keep (failover-default=true keeps the virtio-net > device, =off the other one). > > For example, with a guest driver that doesn't support STANDBY: > > ... > -device virtio-net-pci,id=virtio0,failover=on,failover-default=on \ > -device vfio-pci,host=$PCI,id=hostdev0,failover_pair_id=virtio0 \ > ... > > [root@localhost ~]# ip a > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > valid_lft forever preferred_lft forever > inet6 ::1/128 scope host > valid_lft forever preferred_lft forever > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP q0 > link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff > inet 192.168.20.2/24 brd 192.168.20.255 scope global eth0 > valid_lft forever preferred_lft forever > inet6 fe80::2428:c5ff:fe7f:1424/64 scope link > valid_lft forever preferred_lft forever > # ethtool -i eth0 > driver: virtio_net > version: 1.0.0 > firmware-version: > expansion-rom-version: > bus-info: 0000:04:00.0 > supports-statistics: no > supports-test: no > supports-eeprom-access: no > supports-register-dump: no > supports-priv-flags: no > > ... > -device virtio-net-pci,id=virtio0,failover=on,failover-default=off \ > -device vfio-pci,host=$PCI,id=hostdev0,failover_pair_id=virtio0 \ > ... > > [root@localhost ~]# ip a > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > valid_lft forever preferred_lft forever > inet6 ::1/128 scope host > valid_lft forever preferred_lft forever > 2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 100 > link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff > inet 192.168.20.2/24 brd 192.168.20.255 scope global enp2s0 > valid_lft forever preferred_lft forever > inet6 fe80::2428:c5ff:fe7f:1424/64 scope link > valid_lft forever preferred_lft forever > [root@localhost ~]# ethtool -i enp2s0 > driver: i40evf > version: 1.6.27-k > firmware-version: N/A > expansion-rom-version: > bus-info: 0000:02:00.0 > supports-statistics: yes > supports-test: no > supports-eeprom-access: no > supports-register-dump: no > supports-priv-flags: no > > With guest driver that supports STANDBY, we would always have: > > [root@localhost ~]# ip a > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group defau0 > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > valid_lft forever preferred_lft forever > inet6 ::1/128 scope host > valid_lft forever preferred_lft forever > 2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP gr0 > link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff > inet 192.168.20.2/24 brd 192.168.20.255 scope global noprefixroute enp4s0 > valid_lft forever preferred_lft forever > inet6 fe80::2428:c5ff:fe7f:1424/64 scope link > valid_lft forever preferred_lft forever > 3: enp4s0nsby: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master0 > link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff > 4: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master enp4s0 st0 > link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff > [root@localhost ~]# ethtool -i enp4s0 > driver: net_failover > version: 0.1 > firmware-version: > expansion-rom-version: > bus-info: > supports-statistics: no > supports-test: no > supports-eeprom-access: no > supports-register-dump: no > supports-priv-flags: no > [root@localhost ~]# ethtool -i enp4s0nsby > driver: virtio_net > version: 1.0.0 > firmware-version: > expansion-rom-version: > bus-info: 0000:04:00.0 > supports-statistics: yes > supports-test: no > supports-eeprom-access: no > supports-register-dump: no > supports-priv-flags: no > [root@localhost ~]# ethtool -i enp2s0 > driver: iavf > version: 4.18.0-310.el8.x86_64 > firmware-version: N/A > expansion-rom-version: > bus-info: 0000:02:00.0 > supports-statistics: yes > supports-test: no > supports-eeprom-access: no > supports-register-dump: no > supports-priv-flags: yes > > Laurent Vivier (1): > virtio: failover: define the default device to use in case of error > > include/hw/virtio/virtio-net.h | 1 + > hw/net/virtio-net.c | 49 +++++++++++++++++++++++++++++----- > 2 files changed, 44 insertions(+), 6 deletions(-) > > -- > 2.31.1 > >
On 11/08/2021 06:17, Jason Wang wrote: > On Tue, Aug 10, 2021 at 1:13 AM Laurent Vivier <lvivier@redhat.com> wrote: >> >> v2: use validate_features() to disable the guest driver rather >> than setting vring.num to 0. >> >> With failover, when the guest virtio-net driver doesn't support the >> STANDBY feature, the primary device is not plugged and only the virtio-net >> device is kept. Doing like that we can migrate the machine and >> keep the network connection. >> >> But in some cases, when performance is more important than availability >> we would prefer to keep the VFIO device rather than the virtio-net one, >> even if it means to lose the network connection during the migration of >> the machine. > > I think we still need to seek a way to recover the network after migration. If the network device is configured to have an IP address at boot, the IP will be restored when the card is hotplugged. The only difference with virtio-net is _during_ the migration. Witht virtio-net we keep the networking while the VM is migrated, without it we lose the networking when the VFIO card is unplugged and restored when it is plugged back. Thanks, Laurent > > Thanks > >> >> To do that we can't simply unplug the virtio-net device and plug the >> VFIO one because for the migration the initial state must be kept >> (virtio-net plugged, VFIO unplugged) but we can try to disable the >> virtio-net driver and plug the VFIO card, so the initial state is >> correct (the virtio-net card is plugged, but disabled in guest, and >> the VFIO card is unplugged before migration). >> >> This change doesn't impact the case when guest and host support >> the STANDBY feature. >> >> I've introduced the "failover-default" property to virtio-net device >> to set which device to keep (failover-default=true keeps the virtio-net >> device, =off the other one). >> >> For example, with a guest driver that doesn't support STANDBY: >> >> ... >> -device virtio-net-pci,id=virtio0,failover=on,failover-default=on \ >> -device vfio-pci,host=$PCI,id=hostdev0,failover_pair_id=virtio0 \ >> ... >> >> [root@localhost ~]# ip a >> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 >> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 >> inet 127.0.0.1/8 scope host lo >> valid_lft forever preferred_lft forever >> inet6 ::1/128 scope host >> valid_lft forever preferred_lft forever >> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP q0 >> link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff >> inet 192.168.20.2/24 brd 192.168.20.255 scope global eth0 >> valid_lft forever preferred_lft forever >> inet6 fe80::2428:c5ff:fe7f:1424/64 scope link >> valid_lft forever preferred_lft forever >> # ethtool -i eth0 >> driver: virtio_net >> version: 1.0.0 >> firmware-version: >> expansion-rom-version: >> bus-info: 0000:04:00.0 >> supports-statistics: no >> supports-test: no >> supports-eeprom-access: no >> supports-register-dump: no >> supports-priv-flags: no >> >> ... >> -device virtio-net-pci,id=virtio0,failover=on,failover-default=off \ >> -device vfio-pci,host=$PCI,id=hostdev0,failover_pair_id=virtio0 \ >> ... >> >> [root@localhost ~]# ip a >> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 >> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 >> inet 127.0.0.1/8 scope host lo >> valid_lft forever preferred_lft forever >> inet6 ::1/128 scope host >> valid_lft forever preferred_lft forever >> 2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 100 >> link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff >> inet 192.168.20.2/24 brd 192.168.20.255 scope global enp2s0 >> valid_lft forever preferred_lft forever >> inet6 fe80::2428:c5ff:fe7f:1424/64 scope link >> valid_lft forever preferred_lft forever >> [root@localhost ~]# ethtool -i enp2s0 >> driver: i40evf >> version: 1.6.27-k >> firmware-version: N/A >> expansion-rom-version: >> bus-info: 0000:02:00.0 >> supports-statistics: yes >> supports-test: no >> supports-eeprom-access: no >> supports-register-dump: no >> supports-priv-flags: no >> >> With guest driver that supports STANDBY, we would always have: >> >> [root@localhost ~]# ip a >> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group defau0 >> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 >> inet 127.0.0.1/8 scope host lo >> valid_lft forever preferred_lft forever >> inet6 ::1/128 scope host >> valid_lft forever preferred_lft forever >> 2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP gr0 >> link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff >> inet 192.168.20.2/24 brd 192.168.20.255 scope global noprefixroute enp4s0 >> valid_lft forever preferred_lft forever >> inet6 fe80::2428:c5ff:fe7f:1424/64 scope link >> valid_lft forever preferred_lft forever >> 3: enp4s0nsby: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master0 >> link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff >> 4: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master enp4s0 st0 >> link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff >> [root@localhost ~]# ethtool -i enp4s0 >> driver: net_failover >> version: 0.1 >> firmware-version: >> expansion-rom-version: >> bus-info: >> supports-statistics: no >> supports-test: no >> supports-eeprom-access: no >> supports-register-dump: no >> supports-priv-flags: no >> [root@localhost ~]# ethtool -i enp4s0nsby >> driver: virtio_net >> version: 1.0.0 >> firmware-version: >> expansion-rom-version: >> bus-info: 0000:04:00.0 >> supports-statistics: yes >> supports-test: no >> supports-eeprom-access: no >> supports-register-dump: no >> supports-priv-flags: no >> [root@localhost ~]# ethtool -i enp2s0 >> driver: iavf >> version: 4.18.0-310.el8.x86_64 >> firmware-version: N/A >> expansion-rom-version: >> bus-info: 0000:02:00.0 >> supports-statistics: yes >> supports-test: no >> supports-eeprom-access: no >> supports-register-dump: no >> supports-priv-flags: yes >> >> Laurent Vivier (1): >> virtio: failover: define the default device to use in case of error >> >> include/hw/virtio/virtio-net.h | 1 + >> hw/net/virtio-net.c | 49 +++++++++++++++++++++++++++++----- >> 2 files changed, 44 insertions(+), 6 deletions(-) >> >> -- >> 2.31.1 >> >> >
© 2016 - 2024 Red Hat, Inc.