[PATCH v2 00/21] PCI Multifunction hotplug/hotunplug support

Daniel Henrique Barboza posted 21 patches 4 years, 2 months ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20200130164433.2694119-1-danielhb413@gmail.com
src/conf/device_conf.h                        |   7 +
src/conf/domain_addr.c                        | 129 ++++-
src/conf/domain_addr.h                        |  43 +-
src/conf/domain_conf.c                        | 198 +++++++-
src/conf/domain_conf.h                        |  35 ++
src/libvirt_private.syms                      |  10 +
src/qemu/qemu_domain.c                        |  73 +++
src/qemu/qemu_domain.h                        |  21 +-
src/qemu/qemu_domain_address.c                | 366 ++++++++++++++-
src/qemu/qemu_domain_address.h                |  16 +
src/qemu/qemu_driver.c                        | 242 +++++++---
src/qemu/qemu_hotplug.c                       | 440 +++++++++++++++---
src/qemu/qemu_hotplug.h                       |  14 +
src/util/virhostdev.c                         |  54 +++
src/util/virhostdev.h                         |   5 +
src/util/virpci.c                             |  17 +
src/util/virpci.h                             |   4 +
tests/qemuhotplugtest.c                       |  68 ++-
...emuhotplug-multifunction-hostdev-pci-2.xml |  14 +
...plug-multifunction-hostdev-pci-partial.xml |  27 ++
.../qemuhotplug-multifunction-hostdev-pci.xml |  26 ++
...live+multifunction-hostdev-pci-partial.xml |  82 ++++
...ug-base-live+multifunction-hostdev-pci.xml |  82 ++++
...-base-live+multifunction-hostdev-pci-2.xml |  59 +++
...es-base-live+multifunction-hostdev-pci.xml |  69 +++
.../hostdev-pci-address-unassigned.args       |   9 +-
.../hostdev-pci-multifunction.args            |  18 +-
.../hostdev-pci-multifunction.xml             |   8 +-
.../hostdev-pci-no-primary-function.xml       |  23 +
.../hostdev-pci-validate.args                 |  30 ++
.../qemuxml2argvdata/hostdev-pci-validate.xml |  29 ++
.../qemuxml2argvdata/pseries-hostdevs-1.args  |   5 +-
.../qemuxml2argvdata/pseries-hostdevs-3.args  |   5 +-
tests/qemuxml2argvtest.c                      |  14 +-
.../hostdev-pci-address-unassigned.xml        |   8 +-
.../hostdev-pci-multifunction.xml             |  24 +-
.../qemuxml2xmloutdata/pseries-hostdevs-1.xml |   4 +-
.../qemuxml2xmloutdata/pseries-hostdevs-3.xml |   4 +-
tests/virpcitestdata/0005-90-01.1.config      | Bin 256 -> 256 bytes
tests/virpcitestdata/0005-90-01.2.config      | Bin 256 -> 256 bytes
tests/virpcitestdata/0005-90-01.3.config      | Bin 0 -> 256 bytes
41 files changed, 2023 insertions(+), 259 deletions(-)
create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-multifunction-hostdev-pci-2.xml
create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-multifunction-hostdev-pci-partial.xml
create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-multifunction-hostdev-pci.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+multifunction-hostdev-pci-partial.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+multifunction-hostdev-pci.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+multifunction-hostdev-pci-2.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+multifunction-hostdev-pci.xml
create mode 100644 tests/qemuxml2argvdata/hostdev-pci-no-primary-function.xml
create mode 100644 tests/qemuxml2argvdata/hostdev-pci-validate.args
create mode 100644 tests/qemuxml2argvdata/hostdev-pci-validate.xml
create mode 100644 tests/virpcitestdata/0005-90-01.3.config
[PATCH v2 00/21] PCI Multifunction hotplug/hotunplug support
Posted by Daniel Henrique Barboza 4 years, 2 months ago
This series adds PCI multifunction hotplug/unplug capabilities
for Libvirt. Some of these patches were sent last year in a
shorter prep series in [1]. The patches then got a bit of
rework to keep up with Libvirt changes in master. This
work follows the considerations made for the unplug design
in [2]. A first version of this series were sent back in
2018 [3], so this is the official version 2 of that work.

The design guideline for the patches can be summed up as:

- attach/detach functions were changed to handle a list of
devices instead of a single device definition. The regular
device attach/detach is represented with a list with
size = 1;
 
- common code between single device and multifunction device
mechanics were moved to 'internal' versions of the functions;

- for the 'Live' operations, both attach and detach were
handled in specialized functions for the multifunction case.
The regular case is still being handled by the same
functions.

This allowed us to add the multifunction support without
changing existing regular attach/detach device support.

Attaching/detaching a multifunction device works by supplying
the <devices> XML to the same attach/detach commands we
already use. It is expected to supply the same XML when
detaching the device, as discussed in [2].

Despite the changes and additions I've made, this is still
adherent to the original 2018 series from Shivaprasad G Bhat.


[1] https://www.redhat.com/archives/libvir-list/2019-August/msg01382.html
[2] https://www.redhat.com/archives/libvir-list/2020-January/msg00865.html
[3] https://www.redhat.com/archives/libvir-list/2018-March/msg00729.html


Daniel Henrique Barboza (4):
  utils: PCI multifunction detection helpers
  qemu_hotplug.c: tune unplugTimeout for multifunction detach
  qemu_hotplug: do not hotplug/hotunplug 'unassigned' hostdevs
  qemu_hotplug.c: use enhanced multifunction unplug if available

Shivaprasad G Bhat (17):
  qemu: address: Separate the slots into multiple aggregates
  virhostdev: Introduce virHostdevPCIDevicesBelongToSameSlot
  qemu: address: Enable auto addressing multifunction cards
  conf: qemu: validate multifunction hostdevice domain configs
  conf: Add helper to get active functions of a slot of domain
  qemu: hostdev: Move the hostdev preparation to a separate function
  qemu: hotplug: Move the detach of PCI device to the beginning of live
    hotplug
  qemu: hotplug: move assignment outside qemuDomainAttachHostPCIDevice
  Introduce virDomainDeviceDefParseXMLMany
  Introduce qemuDomainDeviceParseXMLMany
  qemu: refactor qemuDomain[Attach|Detach]DeviceConfig
  qemu: refactor qemuDomain[Attach|Detach]DeviceLive
  qemu: hotplug: Queue and wait for multiple devices
  domain: addr: Introduce virDomainPCIAddressEnsureMultifunctionAddress
  qemu: hotplug: Implement multifunction device hotplug
  qemu: hotplug: Prevent updates to multifunction device
  qemu: hotplug: Implement multifunction device unplug

 src/conf/device_conf.h                        |   7 +
 src/conf/domain_addr.c                        | 129 ++++-
 src/conf/domain_addr.h                        |  43 +-
 src/conf/domain_conf.c                        | 198 +++++++-
 src/conf/domain_conf.h                        |  35 ++
 src/libvirt_private.syms                      |  10 +
 src/qemu/qemu_domain.c                        |  73 +++
 src/qemu/qemu_domain.h                        |  21 +-
 src/qemu/qemu_domain_address.c                | 366 ++++++++++++++-
 src/qemu/qemu_domain_address.h                |  16 +
 src/qemu/qemu_driver.c                        | 242 +++++++---
 src/qemu/qemu_hotplug.c                       | 440 +++++++++++++++---
 src/qemu/qemu_hotplug.h                       |  14 +
 src/util/virhostdev.c                         |  54 +++
 src/util/virhostdev.h                         |   5 +
 src/util/virpci.c                             |  17 +
 src/util/virpci.h                             |   4 +
 tests/qemuhotplugtest.c                       |  68 ++-
 ...emuhotplug-multifunction-hostdev-pci-2.xml |  14 +
 ...plug-multifunction-hostdev-pci-partial.xml |  27 ++
 .../qemuhotplug-multifunction-hostdev-pci.xml |  26 ++
 ...live+multifunction-hostdev-pci-partial.xml |  82 ++++
 ...ug-base-live+multifunction-hostdev-pci.xml |  82 ++++
 ...-base-live+multifunction-hostdev-pci-2.xml |  59 +++
 ...es-base-live+multifunction-hostdev-pci.xml |  69 +++
 .../hostdev-pci-address-unassigned.args       |   9 +-
 .../hostdev-pci-multifunction.args            |  18 +-
 .../hostdev-pci-multifunction.xml             |   8 +-
 .../hostdev-pci-no-primary-function.xml       |  23 +
 .../hostdev-pci-validate.args                 |  30 ++
 .../qemuxml2argvdata/hostdev-pci-validate.xml |  29 ++
 .../qemuxml2argvdata/pseries-hostdevs-1.args  |   5 +-
 .../qemuxml2argvdata/pseries-hostdevs-3.args  |   5 +-
 tests/qemuxml2argvtest.c                      |  14 +-
 .../hostdev-pci-address-unassigned.xml        |   8 +-
 .../hostdev-pci-multifunction.xml             |  24 +-
 .../qemuxml2xmloutdata/pseries-hostdevs-1.xml |   4 +-
 .../qemuxml2xmloutdata/pseries-hostdevs-3.xml |   4 +-
 tests/virpcitestdata/0005-90-01.1.config      | Bin 256 -> 256 bytes
 tests/virpcitestdata/0005-90-01.2.config      | Bin 256 -> 256 bytes
 tests/virpcitestdata/0005-90-01.3.config      | Bin 0 -> 256 bytes
 41 files changed, 2023 insertions(+), 259 deletions(-)
 create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-multifunction-hostdev-pci-2.xml
 create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-multifunction-hostdev-pci-partial.xml
 create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-multifunction-hostdev-pci.xml
 create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+multifunction-hostdev-pci-partial.xml
 create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+multifunction-hostdev-pci.xml
 create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+multifunction-hostdev-pci-2.xml
 create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+multifunction-hostdev-pci.xml
 create mode 100644 tests/qemuxml2argvdata/hostdev-pci-no-primary-function.xml
 create mode 100644 tests/qemuxml2argvdata/hostdev-pci-validate.args
 create mode 100644 tests/qemuxml2argvdata/hostdev-pci-validate.xml
 create mode 100644 tests/virpcitestdata/0005-90-01.3.config

-- 
2.24.1


Re: [PATCH v2 00/21] PCI Multifunction hotplug/hotunplug support
Posted by Daniel Henrique Barboza 4 years, 2 months ago
I forgot to add in the cover:

- you can find the feature in this github branch, applied on top of master at
commit 49882b3337:

https://github.com/danielhb/libvirt/tree/multif_v2

- scenarios tested with the feature, all of them using a BCM5719 multifunction
network card on a Power8 host:

* managed='yes', all functions attached/detached
* managed='yes', only a few functions attached/detached (using address='unassigned')
* managed='no' all functions attached/detached
* managed='no', only a few functions attached/detached
* managed='no', only a few functions attached/detached (using address='unassigned')


Thanks,


DHB


On 1/30/20 1:44 PM, Daniel Henrique Barboza wrote:
> This series adds PCI multifunction hotplug/unplug capabilities
> for Libvirt. Some of these patches were sent last year in a
> shorter prep series in [1]. The patches then got a bit of
> rework to keep up with Libvirt changes in master. This
> work follows the considerations made for the unplug design
> in [2]. A first version of this series were sent back in
> 2018 [3], so this is the official version 2 of that work.
> 
> The design guideline for the patches can be summed up as:
> 
> - attach/detach functions were changed to handle a list of
> devices instead of a single device definition. The regular
> device attach/detach is represented with a list with
> size = 1;
>   
> - common code between single device and multifunction device
> mechanics were moved to 'internal' versions of the functions;
> 
> - for the 'Live' operations, both attach and detach were
> handled in specialized functions for the multifunction case.
> The regular case is still being handled by the same
> functions.
> 
> This allowed us to add the multifunction support without
> changing existing regular attach/detach device support.
> 
> Attaching/detaching a multifunction device works by supplying
> the <devices> XML to the same attach/detach commands we
> already use. It is expected to supply the same XML when
> detaching the device, as discussed in [2].
> 
> Despite the changes and additions I've made, this is still
> adherent to the original 2018 series from Shivaprasad G Bhat.
> 
> 
> [1] https://www.redhat.com/archives/libvir-list/2019-August/msg01382.html
> [2] https://www.redhat.com/archives/libvir-list/2020-January/msg00865.html
> [3] https://www.redhat.com/archives/libvir-list/2018-March/msg00729.html
> 
> 
> Daniel Henrique Barboza (4):
>    utils: PCI multifunction detection helpers
>    qemu_hotplug.c: tune unplugTimeout for multifunction detach
>    qemu_hotplug: do not hotplug/hotunplug 'unassigned' hostdevs
>    qemu_hotplug.c: use enhanced multifunction unplug if available
> 
> Shivaprasad G Bhat (17):
>    qemu: address: Separate the slots into multiple aggregates
>    virhostdev: Introduce virHostdevPCIDevicesBelongToSameSlot
>    qemu: address: Enable auto addressing multifunction cards
>    conf: qemu: validate multifunction hostdevice domain configs
>    conf: Add helper to get active functions of a slot of domain
>    qemu: hostdev: Move the hostdev preparation to a separate function
>    qemu: hotplug: Move the detach of PCI device to the beginning of live
>      hotplug
>    qemu: hotplug: move assignment outside qemuDomainAttachHostPCIDevice
>    Introduce virDomainDeviceDefParseXMLMany
>    Introduce qemuDomainDeviceParseXMLMany
>    qemu: refactor qemuDomain[Attach|Detach]DeviceConfig
>    qemu: refactor qemuDomain[Attach|Detach]DeviceLive
>    qemu: hotplug: Queue and wait for multiple devices
>    domain: addr: Introduce virDomainPCIAddressEnsureMultifunctionAddress
>    qemu: hotplug: Implement multifunction device hotplug
>    qemu: hotplug: Prevent updates to multifunction device
>    qemu: hotplug: Implement multifunction device unplug
> 
>   src/conf/device_conf.h                        |   7 +
>   src/conf/domain_addr.c                        | 129 ++++-
>   src/conf/domain_addr.h                        |  43 +-
>   src/conf/domain_conf.c                        | 198 +++++++-
>   src/conf/domain_conf.h                        |  35 ++
>   src/libvirt_private.syms                      |  10 +
>   src/qemu/qemu_domain.c                        |  73 +++
>   src/qemu/qemu_domain.h                        |  21 +-
>   src/qemu/qemu_domain_address.c                | 366 ++++++++++++++-
>   src/qemu/qemu_domain_address.h                |  16 +
>   src/qemu/qemu_driver.c                        | 242 +++++++---
>   src/qemu/qemu_hotplug.c                       | 440 +++++++++++++++---
>   src/qemu/qemu_hotplug.h                       |  14 +
>   src/util/virhostdev.c                         |  54 +++
>   src/util/virhostdev.h                         |   5 +
>   src/util/virpci.c                             |  17 +
>   src/util/virpci.h                             |   4 +
>   tests/qemuhotplugtest.c                       |  68 ++-
>   ...emuhotplug-multifunction-hostdev-pci-2.xml |  14 +
>   ...plug-multifunction-hostdev-pci-partial.xml |  27 ++
>   .../qemuhotplug-multifunction-hostdev-pci.xml |  26 ++
>   ...live+multifunction-hostdev-pci-partial.xml |  82 ++++
>   ...ug-base-live+multifunction-hostdev-pci.xml |  82 ++++
>   ...-base-live+multifunction-hostdev-pci-2.xml |  59 +++
>   ...es-base-live+multifunction-hostdev-pci.xml |  69 +++
>   .../hostdev-pci-address-unassigned.args       |   9 +-
>   .../hostdev-pci-multifunction.args            |  18 +-
>   .../hostdev-pci-multifunction.xml             |   8 +-
>   .../hostdev-pci-no-primary-function.xml       |  23 +
>   .../hostdev-pci-validate.args                 |  30 ++
>   .../qemuxml2argvdata/hostdev-pci-validate.xml |  29 ++
>   .../qemuxml2argvdata/pseries-hostdevs-1.args  |   5 +-
>   .../qemuxml2argvdata/pseries-hostdevs-3.args  |   5 +-
>   tests/qemuxml2argvtest.c                      |  14 +-
>   .../hostdev-pci-address-unassigned.xml        |   8 +-
>   .../hostdev-pci-multifunction.xml             |  24 +-
>   .../qemuxml2xmloutdata/pseries-hostdevs-1.xml |   4 +-
>   .../qemuxml2xmloutdata/pseries-hostdevs-3.xml |   4 +-
>   tests/virpcitestdata/0005-90-01.1.config      | Bin 256 -> 256 bytes
>   tests/virpcitestdata/0005-90-01.2.config      | Bin 256 -> 256 bytes
>   tests/virpcitestdata/0005-90-01.3.config      | Bin 0 -> 256 bytes
>   41 files changed, 2023 insertions(+), 259 deletions(-)
>   create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-multifunction-hostdev-pci-2.xml
>   create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-multifunction-hostdev-pci-partial.xml
>   create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-multifunction-hostdev-pci.xml
>   create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+multifunction-hostdev-pci-partial.xml
>   create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+multifunction-hostdev-pci.xml
>   create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+multifunction-hostdev-pci-2.xml
>   create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+multifunction-hostdev-pci.xml
>   create mode 100644 tests/qemuxml2argvdata/hostdev-pci-no-primary-function.xml
>   create mode 100644 tests/qemuxml2argvdata/hostdev-pci-validate.args
>   create mode 100644 tests/qemuxml2argvdata/hostdev-pci-validate.xml
>   create mode 100644 tests/virpcitestdata/0005-90-01.3.config
>