[v2] virtio-balloon-pci: Allow setting nvectors, so we can use MSI-X

[PATCH v2] virtio-balloon-pci: Allow setting nvectors, so we can use MSI-X

Posted by Reza Arbab 2 months, 2 weeks ago

Most virtio-pci devices allow MSI-X. Add it to virtio-balloon-pci, but
only enable it in new machine types, so we don't break migration of
existing machine types between different qemu versions.

This copies what was done for virtio-rng-pci in:
9ea02e8f1306 ("virtio-rng-pci: Allow setting nvectors, so we can use MSI-X")
bad9c5a5166f ("virtio-rng-pci: fix migration compat for vectors")
62bdb8871512 ("virtio-rng-pci: fix transitional migration compat for vectors")

Acked-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
---
v2:
- Remove DEFINE_PROP_END_OF_LIST()
- Declare virtio_balloon_properties as const

 hw/core/machine.c              |  3 +++
 hw/virtio/virtio-balloon-pci.c | 12 ++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index c23b39949649..8f396ef8035a 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -38,6 +38,9 @@
 
 GlobalProperty hw_compat_9_2[] = {
     {"arm-cpu", "backcompat-pauth-default-use-qarma5", "true"},
+    { "virtio-balloon-pci", "vectors", "0" },
+    { "virtio-balloon-pci-transitional", "vectors", "0" },
+    { "virtio-balloon-pci-non-transitional", "vectors", "0" },
 };
 const size_t hw_compat_9_2_len = G_N_ELEMENTS(hw_compat_9_2);
 
diff --git a/hw/virtio/virtio-balloon-pci.c b/hw/virtio/virtio-balloon-pci.c
index ce2645ba7187..db7e1cb47537 100644
--- a/hw/virtio/virtio-balloon-pci.c
+++ b/hw/virtio/virtio-balloon-pci.c
@@ -35,11 +35,22 @@ struct VirtIOBalloonPCI {
     VirtIOBalloon vdev;
 };
 
+static const Property virtio_balloon_properties[] = {
+    DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags,
+                    VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true),
+    DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors,
+                       DEV_NVECTORS_UNSPECIFIED),
+};
+
 static void virtio_balloon_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
 {
     VirtIOBalloonPCI *dev = VIRTIO_BALLOON_PCI(vpci_dev);
     DeviceState *vdev = DEVICE(&dev->vdev);
 
+    if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) {
+        vpci_dev->nvectors = 2;
+    }
+
     vpci_dev->class_code = PCI_CLASS_OTHERS;
     qdev_realize(vdev, BUS(&vpci_dev->bus), errp);
 }
@@ -55,6 +66,7 @@ static void virtio_balloon_pci_class_init(ObjectClass *klass, void *data)
     pcidev_k->device_id = PCI_DEVICE_ID_VIRTIO_BALLOON;
     pcidev_k->revision = VIRTIO_PCI_ABI_VERSION;
     pcidev_k->class_id = PCI_CLASS_OTHERS;
+    device_class_set_props(dc, virtio_balloon_properties);
 }
 
 static void virtio_balloon_pci_instance_init(Object *obj)

base-commit: 7433709a147706ad7d1956b15669279933d0f82b
-- 
2.47.1

Re: [PATCH v2] virtio-balloon-pci: Allow setting nvectors, so we can use MSI-X

Posted by Mario Casquero 2 months ago

This patch v2 has been successfully tested in s390x. Here the detailed
test steps:

Boot up a VM already containing a memory object
/home/qemu/build/qemu-system-s390x \
...
-m 4G,maxmem=20G \
-object memory-backend-ram,id=mem0,size=16G,reserve=off \
...

Check the virtio-balloon
(qemu) info balloon
Error: No balloon device has been activated

Check the PCI devices in the guest
[root@localhost ~]# lspci -v
[root@localhost ~]#

Hotplug a virtio-balloon-pci device and query it again.
(qemu) device_add virtio-balloon-pci,id=balloon0
(qemu) info balloon
balloon: actual=4096

Deflate the balloon verify the new size
(qemu) balloon 2048
(qemu) info balloon
balloon: actual=2048

Check again the PCI devices in the guest
[root@localhost ~]# lspci -v
0001:00:00.0 Unclassified device [00ff]: Red Hat, Inc. Virtio memory balloon
Subsystem: Red Hat, Inc. Device 0005
Physical Slot: 00000000
Flags: bus master, fast devsel, latency 0, IOMMU group 0
Memory at 4001000000000000 (32-bit, non-prefetchable) [virtual] [size=4K]
Memory at 4002000000000000 (64-bit, prefetchable) [size=16K]
Capabilities: [98] MSI-X: Enable+ Count=2 Masked-
Capabilities: [84] Vendor Specific Information: VirtIO: <unknown>
Capabilities: [70] Vendor Specific Information: VirtIO: Notify
Capabilities: [60] Vendor Specific Information: VirtIO: DeviceCfg
Capabilities: [50] Vendor Specific Information: VirtIO: ISR
Capabilities: [40] Vendor Specific Information: VirtIO: CommonCfg
Kernel driver in use: virtio-pci

Tested-by: Mario Casquero <mcasquer@redhat.com>


On Wed, Jan 15, 2025 at 5:16 PM Reza Arbab <arbab@linux.ibm.com> wrote:
>
> Most virtio-pci devices allow MSI-X. Add it to virtio-balloon-pci, but
> only enable it in new machine types, so we don't break migration of
> existing machine types between different qemu versions.
>
> This copies what was done for virtio-rng-pci in:
> 9ea02e8f1306 ("virtio-rng-pci: Allow setting nvectors, so we can use MSI-X")
> bad9c5a5166f ("virtio-rng-pci: fix migration compat for vectors")
> 62bdb8871512 ("virtio-rng-pci: fix transitional migration compat for vectors")
>
> Acked-by: David Hildenbrand <david@redhat.com>
> Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
> ---
> v2:
> - Remove DEFINE_PROP_END_OF_LIST()
> - Declare virtio_balloon_properties as const
>
>  hw/core/machine.c              |  3 +++
>  hw/virtio/virtio-balloon-pci.c | 12 ++++++++++++
>  2 files changed, 15 insertions(+)
>
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index c23b39949649..8f396ef8035a 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -38,6 +38,9 @@
>
>  GlobalProperty hw_compat_9_2[] = {
>      {"arm-cpu", "backcompat-pauth-default-use-qarma5", "true"},
> +    { "virtio-balloon-pci", "vectors", "0" },
> +    { "virtio-balloon-pci-transitional", "vectors", "0" },
> +    { "virtio-balloon-pci-non-transitional", "vectors", "0" },
>  };
>  const size_t hw_compat_9_2_len = G_N_ELEMENTS(hw_compat_9_2);
>
> diff --git a/hw/virtio/virtio-balloon-pci.c b/hw/virtio/virtio-balloon-pci.c
> index ce2645ba7187..db7e1cb47537 100644
> --- a/hw/virtio/virtio-balloon-pci.c
> +++ b/hw/virtio/virtio-balloon-pci.c
> @@ -35,11 +35,22 @@ struct VirtIOBalloonPCI {
>      VirtIOBalloon vdev;
>  };
>
> +static const Property virtio_balloon_properties[] = {
> +    DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags,
> +                    VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true),
> +    DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors,
> +                       DEV_NVECTORS_UNSPECIFIED),
> +};
> +
>  static void virtio_balloon_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
>  {
>      VirtIOBalloonPCI *dev = VIRTIO_BALLOON_PCI(vpci_dev);
>      DeviceState *vdev = DEVICE(&dev->vdev);
>
> +    if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) {
> +        vpci_dev->nvectors = 2;
> +    }
> +
>      vpci_dev->class_code = PCI_CLASS_OTHERS;
>      qdev_realize(vdev, BUS(&vpci_dev->bus), errp);
>  }
> @@ -55,6 +66,7 @@ static void virtio_balloon_pci_class_init(ObjectClass *klass, void *data)
>      pcidev_k->device_id = PCI_DEVICE_ID_VIRTIO_BALLOON;
>      pcidev_k->revision = VIRTIO_PCI_ABI_VERSION;
>      pcidev_k->class_id = PCI_CLASS_OTHERS;
> +    device_class_set_props(dc, virtio_balloon_properties);
>  }
>
>  static void virtio_balloon_pci_instance_init(Object *obj)
>
> base-commit: 7433709a147706ad7d1956b15669279933d0f82b
> --
> 2.47.1
>
>

Re: [PATCH v2] virtio-balloon-pci: Allow setting nvectors, so we can use MSI-X

Posted by David Hildenbrand 2 months, 1 week ago

On 15.01.25 17:14, Reza Arbab wrote:
> Most virtio-pci devices allow MSI-X. Add it to virtio-balloon-pci, but
> only enable it in new machine types, so we don't break migration of
> existing machine types between different qemu versions.
> 
> This copies what was done for virtio-rng-pci in:
> 9ea02e8f1306 ("virtio-rng-pci: Allow setting nvectors, so we can use MSI-X")
> bad9c5a5166f ("virtio-rng-pci: fix migration compat for vectors")
> 62bdb8871512 ("virtio-rng-pci: fix transitional migration compat for vectors")
> 
> Acked-by: David Hildenbrand <david@redhat.com>
> Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
> ---

This patch enables the virtio-balloon-pci device to be plugged on s390x.
Before this change, plugging that device would fail with "MSI-X support is
mandatory on s390".

However, the device does not seem to be initialized by the Linux guest:

# lspci -v
0001:00:00.0 Unclassified device [00ff]: Red Hat, Inc. Virtio memory balloon
         Subsystem: Red Hat, Inc. Device 0005
         Physical Slot: 00000000
         Flags: fast devsel, IOMMU group 0
         Memory at 4001000000000000 (32-bit, non-prefetchable) [virtual] [size=4K]
         Memory at 4002000000000000 (64-bit, prefetchable) [size=16K]
         Capabilities: [98] MSI-X: Enable- Count=2 Masked-
         Capabilities: [84] Vendor Specific Information: VirtIO: <unknown>
         Capabilities: [70] Vendor Specific Information: VirtIO: Notify
         Capabilities: [60] Vendor Specific Information: VirtIO: DeviceCfg
         Capabilities: [50] Vendor Specific Information: VirtIO: ISR
         Capabilities: [40] Vendor Specific Information: VirtIO: CommonCfg


I tried enabling it:

[  343.718529] pci 0001:00:00.0: enabling device (0000 -> 0002)

Without any change. @s390x folks, is there something missing to make it fly,
or are there additional steps to initialize+use such devices from inside the VM?

-- 
Cheers,

David / dhildenb

Re: [PATCH v2] virtio-balloon-pci: Allow setting nvectors, so we can use MSI-X

Posted by David Hildenbrand 2 months, 1 week ago

On 27.01.25 14:11, David Hildenbrand wrote:
> On 15.01.25 17:14, Reza Arbab wrote:
>> Most virtio-pci devices allow MSI-X. Add it to virtio-balloon-pci, but
>> only enable it in new machine types, so we don't break migration of
>> existing machine types between different qemu versions.
>>
>> This copies what was done for virtio-rng-pci in:
>> 9ea02e8f1306 ("virtio-rng-pci: Allow setting nvectors, so we can use MSI-X")
>> bad9c5a5166f ("virtio-rng-pci: fix migration compat for vectors")
>> 62bdb8871512 ("virtio-rng-pci: fix transitional migration compat for vectors")
>>
>> Acked-by: David Hildenbrand <david@redhat.com>
>> Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
>> ---
> 
> This patch enables the virtio-balloon-pci device to be plugged on s390x.
> Before this change, plugging that device would fail with "MSI-X support is
> mandatory on s390".
> 
> However, the device does not seem to be initialized by the Linux guest:
> 
> # lspci -v
> 0001:00:00.0 Unclassified device [00ff]: Red Hat, Inc. Virtio memory balloon
>           Subsystem: Red Hat, Inc. Device 0005
>           Physical Slot: 00000000
>           Flags: fast devsel, IOMMU group 0
>           Memory at 4001000000000000 (32-bit, non-prefetchable) [virtual] [size=4K]
>           Memory at 4002000000000000 (64-bit, prefetchable) [size=16K]
>           Capabilities: [98] MSI-X: Enable- Count=2 Masked-
>           Capabilities: [84] Vendor Specific Information: VirtIO: <unknown>
>           Capabilities: [70] Vendor Specific Information: VirtIO: Notify
>           Capabilities: [60] Vendor Specific Information: VirtIO: DeviceCfg
>           Capabilities: [50] Vendor Specific Information: VirtIO: ISR
>           Capabilities: [40] Vendor Specific Information: VirtIO: CommonCfg
> 
> 
> I tried enabling it:
> 
> [  343.718529] pci 0001:00:00.0: enabling device (0000 -> 0002)
> 
> Without any change. @s390x folks, is there something missing to make it fly,
> or are there additional steps to initialize+use such devices from inside the VM?
> 

Ah! might just be

# CONFIG_VIRTIO_PCI is not set

in my Fedora config,

With that enabled, I think it's working.

-- 
Cheers,

David / dhildenb