[PATCH 00/20] handle AWOL SR-IOV VF hostdevs during domain lifetime

Daniel Henrique Barboza posted 20 patches 3 years, 3 months ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20210104125444.200536-1-danielhb413@gmail.com
include/libvirt/libvirt-nodedev.h  |  2 +-
src/conf/domain_audit.c            |  6 +-
src/datatypes.h                    |  2 +
src/hypervisor/domain_driver.c     | 30 ++++++++++
src/hypervisor/domain_driver.h     |  4 ++
src/hypervisor/virhostdev.c        | 88 ++++++++++++++++++++++------
src/hypervisor/virhostdev.h        |  2 +
src/libvirt-nodedev.c              | 15 +++--
src/libvirt_private.syms           |  3 +
src/libxl/libxl_driver.c           | 87 ++++++++--------------------
src/node_device/node_device_udev.c | 11 ++--
src/qemu/qemu_cgroup.c             | 10 ++++
src/qemu/qemu_domain_address.c     |  5 +-
src/qemu/qemu_driver.c             | 81 +++++++-------------------
src/security/security_apparmor.c   |  3 +-
src/security/security_dac.c        | 61 ++++++++------------
src/security/security_selinux.c    | 66 +++++++++------------
src/security/virt-aa-helper.c      |  6 +-
src/util/virnetdev.c               |  3 +-
src/util/virnvme.c                 |  5 +-
src/util/virpci.c                  | 93 ++++++++++++++----------------
src/util/virpci.h                  | 14 ++---
tests/virhostdevtest.c             |  3 +-
tests/virpcitest.c                 | 35 ++++++++---
tools/virsh-domain.c               | 18 ++----
25 files changed, 325 insertions(+), 328 deletions(-)
[PATCH 00/20] handle AWOL SR-IOV VF hostdevs during domain lifetime
Posted by Daniel Henrique Barboza 3 years, 3 months ago
Hi,

This series is an attempt to fix https://gitlab.com/libvirt/libvirt/-/issues/72.

The root issue is that, for obvious reasons,  virPCIDeviceNew() requires the
device to exist in the host (i.e. the config file must be present in sysfs).
The unexpected absence of the device (e.g. removing the VFs from the host while
a domain was using them) will make virPCIDeviceNew() return -1 in several
code paths, and the effects in Libvirt can be bothersome. Patch 20 will
explain it in greater detail.

Changing virPCIDeviceNew() internals to not fail in this scenario would
be a herculean refactor of innocent code that doesn't have anything to
do with the problem. My approach here is to check if the device exists
in the sysfs in key places, keeping the default virPCIDeviceNew()
behavior intact.

I mentioned this is patch 20 but to emphasize: this is NOT an attempt
to implement surprise hostdev hotunplug support in Libvirt. Removing
SR-IOVs VFs used by any domain is still not cool and should be avoided.
But after this series, Libvirt can at least fall on its feet if this
situation happens.

The series was tested using a Mellanox MT28800 card in a Power 9 server.

Daniel Henrique Barboza (20):
  virpci, domain_audit: use virPCIDeviceAddressAsString()
  qemu, lxc: move NodeDeviceGetPCIInfo() function to domain_driver.c
  domain_driver.c: use PCI address with
    virDomainDriverNodeDeviceGetPCIInfo()
  virpci.c: simplify virPCIDeviceNew() signature
  virpci: introduce virPCIDeviceExists()
  virhostdev.c: virHostdevGetPCIHostDevice() now reports missing device
  security_selinux.c: modernize set/restore hostdev subsys label
    functions
  security_dac.c: modernize hostdev label set/restore functions
  dac, selinux: skip setting/restoring label for absent PCI devices
  libvirt-nodedev.c: remove return value from virNodeDeviceFree()
  qemu_driver.c: modernize qemuNodeDeviceReAttach()
  libxl_driver.c: modernize libxlNodeDeviceReAttach()
  virsh-domain.c: modernize cmdDetachDevice()
  virhostdev.c: add virHostdevIsPCIDevice() helper
  qemu_cgroup.c: skip absent PCI devices in qemuTeardownHostdevCgroup()
  virpci.c: use virPCIDeviceAddressPtr in virPCIDeviceListFindIndex()
  virpci.c: use virPCIDeviceAddressPtr in virPCIDeviceListFind()
  virpci.c: use virPCIDeviceAddressPtr in virPCIDeviceListSteal()
  virpci.c: use virPCIDeviceAddressPtr in virPCIDeviceListDel()
  virhostdev.c: remove missing PCI devs from hostdev manager

 include/libvirt/libvirt-nodedev.h  |  2 +-
 src/conf/domain_audit.c            |  6 +-
 src/datatypes.h                    |  2 +
 src/hypervisor/domain_driver.c     | 30 ++++++++++
 src/hypervisor/domain_driver.h     |  4 ++
 src/hypervisor/virhostdev.c        | 88 ++++++++++++++++++++++------
 src/hypervisor/virhostdev.h        |  2 +
 src/libvirt-nodedev.c              | 15 +++--
 src/libvirt_private.syms           |  3 +
 src/libxl/libxl_driver.c           | 87 ++++++++--------------------
 src/node_device/node_device_udev.c | 11 ++--
 src/qemu/qemu_cgroup.c             | 10 ++++
 src/qemu/qemu_domain_address.c     |  5 +-
 src/qemu/qemu_driver.c             | 81 +++++++-------------------
 src/security/security_apparmor.c   |  3 +-
 src/security/security_dac.c        | 61 ++++++++------------
 src/security/security_selinux.c    | 66 +++++++++------------
 src/security/virt-aa-helper.c      |  6 +-
 src/util/virnetdev.c               |  3 +-
 src/util/virnvme.c                 |  5 +-
 src/util/virpci.c                  | 93 ++++++++++++++----------------
 src/util/virpci.h                  | 14 ++---
 tests/virhostdevtest.c             |  3 +-
 tests/virpcitest.c                 | 35 ++++++++---
 tools/virsh-domain.c               | 18 ++----
 25 files changed, 325 insertions(+), 328 deletions(-)

-- 
2.26.2

Re: [PATCH 00/20] handle AWOL SR-IOV VF hostdevs during domain lifetime
Posted by Daniel Henrique Barboza 3 years, 3 months ago
Ping

On 1/4/21 9:54 AM, Daniel Henrique Barboza wrote:
> Hi,
> 
> This series is an attempt to fix https://gitlab.com/libvirt/libvirt/-/issues/72.
> 
> The root issue is that, for obvious reasons,  virPCIDeviceNew() requires the
> device to exist in the host (i.e. the config file must be present in sysfs).
> The unexpected absence of the device (e.g. removing the VFs from the host while
> a domain was using them) will make virPCIDeviceNew() return -1 in several
> code paths, and the effects in Libvirt can be bothersome. Patch 20 will
> explain it in greater detail.
> 
> Changing virPCIDeviceNew() internals to not fail in this scenario would
> be a herculean refactor of innocent code that doesn't have anything to
> do with the problem. My approach here is to check if the device exists
> in the sysfs in key places, keeping the default virPCIDeviceNew()
> behavior intact.
> 
> I mentioned this is patch 20 but to emphasize: this is NOT an attempt
> to implement surprise hostdev hotunplug support in Libvirt. Removing
> SR-IOVs VFs used by any domain is still not cool and should be avoided.
> But after this series, Libvirt can at least fall on its feet if this
> situation happens.
> 
> The series was tested using a Mellanox MT28800 card in a Power 9 server.
> 
> Daniel Henrique Barboza (20):
>    virpci, domain_audit: use virPCIDeviceAddressAsString()
>    qemu, lxc: move NodeDeviceGetPCIInfo() function to domain_driver.c
>    domain_driver.c: use PCI address with
>      virDomainDriverNodeDeviceGetPCIInfo()
>    virpci.c: simplify virPCIDeviceNew() signature
>    virpci: introduce virPCIDeviceExists()
>    virhostdev.c: virHostdevGetPCIHostDevice() now reports missing device
>    security_selinux.c: modernize set/restore hostdev subsys label
>      functions
>    security_dac.c: modernize hostdev label set/restore functions
>    dac, selinux: skip setting/restoring label for absent PCI devices
>    libvirt-nodedev.c: remove return value from virNodeDeviceFree()
>    qemu_driver.c: modernize qemuNodeDeviceReAttach()
>    libxl_driver.c: modernize libxlNodeDeviceReAttach()
>    virsh-domain.c: modernize cmdDetachDevice()
>    virhostdev.c: add virHostdevIsPCIDevice() helper
>    qemu_cgroup.c: skip absent PCI devices in qemuTeardownHostdevCgroup()
>    virpci.c: use virPCIDeviceAddressPtr in virPCIDeviceListFindIndex()
>    virpci.c: use virPCIDeviceAddressPtr in virPCIDeviceListFind()
>    virpci.c: use virPCIDeviceAddressPtr in virPCIDeviceListSteal()
>    virpci.c: use virPCIDeviceAddressPtr in virPCIDeviceListDel()
>    virhostdev.c: remove missing PCI devs from hostdev manager
> 
>   include/libvirt/libvirt-nodedev.h  |  2 +-
>   src/conf/domain_audit.c            |  6 +-
>   src/datatypes.h                    |  2 +
>   src/hypervisor/domain_driver.c     | 30 ++++++++++
>   src/hypervisor/domain_driver.h     |  4 ++
>   src/hypervisor/virhostdev.c        | 88 ++++++++++++++++++++++------
>   src/hypervisor/virhostdev.h        |  2 +
>   src/libvirt-nodedev.c              | 15 +++--
>   src/libvirt_private.syms           |  3 +
>   src/libxl/libxl_driver.c           | 87 ++++++++--------------------
>   src/node_device/node_device_udev.c | 11 ++--
>   src/qemu/qemu_cgroup.c             | 10 ++++
>   src/qemu/qemu_domain_address.c     |  5 +-
>   src/qemu/qemu_driver.c             | 81 +++++++-------------------
>   src/security/security_apparmor.c   |  3 +-
>   src/security/security_dac.c        | 61 ++++++++------------
>   src/security/security_selinux.c    | 66 +++++++++------------
>   src/security/virt-aa-helper.c      |  6 +-
>   src/util/virnetdev.c               |  3 +-
>   src/util/virnvme.c                 |  5 +-
>   src/util/virpci.c                  | 93 ++++++++++++++----------------
>   src/util/virpci.h                  | 14 ++---
>   tests/virhostdevtest.c             |  3 +-
>   tests/virpcitest.c                 | 35 ++++++++---
>   tools/virsh-domain.c               | 18 ++----
>   25 files changed, 325 insertions(+), 328 deletions(-)
>