During migration restoring, vfio_enable_vectors() is called to restore
enabling MSI-X interrupts for assigned devices. It sets the range from
0 to nr_vectors to kernel to enable MSI-X and the vectors unmasked in
guest. During the MSI-X enabling, all the vectors within the range are
allocated according to the VFIO_DEVICE_SET_IRQS ioctl.
When dynamic MSI-X allocation is supported, we only want the guest
unmasked vectors being allocated and enabled. Use vector 0 with an
invalid fd to get MSI-X enabled, after that, all the vectors can be
allocated in need.
Signed-off-by: Jing Liu <jing2.liu@intel.com>
---
Changes since v1:
- No change.
Changes since RFC v1:
- Revise the comments. (Alex)
- Call the new helper function in previous patch to enable MSI-X. (Alex)
---
hw/vfio/pci.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 0117f230e934..f5f891dc0792 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -402,6 +402,23 @@ static int vfio_enable_vectors(VFIOPCIDevice *vdev, bool msix)
int ret = 0, i, argsz;
int32_t *fds;
+ /*
+ * If dynamic MSI-X allocation is supported, the vectors to be allocated
+ * and enabled can be scattered. Before kernel enabling MSI-X, setting
+ * nr_vectors causes all these vectors to be allocated on host.
+ *
+ * To keep allocation as needed, use vector 0 with an invalid fd to get
+ * MSI-X enabled first, then set vectors with a potentially sparse set of
+ * eventfds to enable interrupts only when enabled in guest.
+ */
+ if (msix && !vdev->msix->noresize) {
+ ret = vfio_enable_msix_no_vec(vdev);
+
+ if (ret) {
+ return ret;
+ }
+ }
+
argsz = sizeof(*irq_set) + (vdev->nr_vectors * sizeof(*fds));
irq_set = g_malloc0(argsz);
--
2.27.0
On 9/18/23 11:45, Jing Liu wrote: > During migration restoring, vfio_enable_vectors() is called to restore > enabling MSI-X interrupts for assigned devices. It sets the range from > 0 to nr_vectors to kernel to enable MSI-X and the vectors unmasked in > guest. During the MSI-X enabling, all the vectors within the range are > allocated according to the VFIO_DEVICE_SET_IRQS ioctl. > > When dynamic MSI-X allocation is supported, we only want the guest > unmasked vectors being allocated and enabled. Use vector 0 with an > invalid fd to get MSI-X enabled, after that, all the vectors can be > allocated in need. > > Signed-off-by: Jing Liu <jing2.liu@intel.com> Reviewed-by: Cédric Le Goater <clg@redhat.com> Thanks, C. > --- > Changes since v1: > - No change. > > Changes since RFC v1: > - Revise the comments. (Alex) > - Call the new helper function in previous patch to enable MSI-X. (Alex) > --- > hw/vfio/pci.c | 17 +++++++++++++++++ > 1 file changed, 17 insertions(+) > > diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c > index 0117f230e934..f5f891dc0792 100644 > --- a/hw/vfio/pci.c > +++ b/hw/vfio/pci.c > @@ -402,6 +402,23 @@ static int vfio_enable_vectors(VFIOPCIDevice *vdev, bool msix) > int ret = 0, i, argsz; > int32_t *fds; > > + /* > + * If dynamic MSI-X allocation is supported, the vectors to be allocated > + * and enabled can be scattered. Before kernel enabling MSI-X, setting > + * nr_vectors causes all these vectors to be allocated on host. > + * > + * To keep allocation as needed, use vector 0 with an invalid fd to get > + * MSI-X enabled first, then set vectors with a potentially sparse set of > + * eventfds to enable interrupts only when enabled in guest. > + */ > + if (msix && !vdev->msix->noresize) { > + ret = vfio_enable_msix_no_vec(vdev); > + > + if (ret) { > + return ret; > + } > + } > + > argsz = sizeof(*irq_set) + (vdev->nr_vectors * sizeof(*fds)); > > irq_set = g_malloc0(argsz);
Hi Cédric, > On 9/19/2023 11:21 PM, Cédric Le Goater wrote: > > On 9/18/23 11:45, Jing Liu wrote: > > During migration restoring, vfio_enable_vectors() is called to restore > > enabling MSI-X interrupts for assigned devices. It sets the range from > > 0 to nr_vectors to kernel to enable MSI-X and the vectors unmasked in > > guest. During the MSI-X enabling, all the vectors within the range are > > allocated according to the VFIO_DEVICE_SET_IRQS ioctl. > > > > When dynamic MSI-X allocation is supported, we only want the guest > > unmasked vectors being allocated and enabled. Use vector 0 with an > > invalid fd to get MSI-X enabled, after that, all the vectors can be > > allocated in need. > > > > Signed-off-by: Jing Liu <jing2.liu@intel.com> > > > Reviewed-by: Cédric Le Goater <clg@redhat.com> Thanks very much for your feedback. Jing > > Thanks, > > C. > > > > --- > > Changes since v1: > > - No change. > > > > Changes since RFC v1: > > - Revise the comments. (Alex) > > - Call the new helper function in previous patch to enable MSI-X. > > (Alex) > > --- > > hw/vfio/pci.c | 17 +++++++++++++++++ > > 1 file changed, 17 insertions(+) > > > > diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index > > 0117f230e934..f5f891dc0792 100644 > > --- a/hw/vfio/pci.c > > +++ b/hw/vfio/pci.c > > @@ -402,6 +402,23 @@ static int vfio_enable_vectors(VFIOPCIDevice *vdev, > bool msix) > > int ret = 0, i, argsz; > > int32_t *fds; > > > > + /* > > + * If dynamic MSI-X allocation is supported, the vectors to be allocated > > + * and enabled can be scattered. Before kernel enabling MSI-X, setting > > + * nr_vectors causes all these vectors to be allocated on host. > > + * > > + * To keep allocation as needed, use vector 0 with an invalid fd to get > > + * MSI-X enabled first, then set vectors with a potentially sparse set of > > + * eventfds to enable interrupts only when enabled in guest. > > + */ > > + if (msix && !vdev->msix->noresize) { > > + ret = vfio_enable_msix_no_vec(vdev); > > + > > + if (ret) { > > + return ret; > > + } > > + } > > + > > argsz = sizeof(*irq_set) + (vdev->nr_vectors * sizeof(*fds)); > > > > irq_set = g_malloc0(argsz);
© 2016 - 2024 Red Hat, Inc.