[PATCH v7 resend 0/4] add generic vDPA device support

Longpeng(Mike) via posted 4 patches 1 year, 6 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20221105083629.1058-1-longpeng2@huawei.com
Maintainers: "Michael S. Tsirkin" <mst@redhat.com>
There is a newer version of this series
docs/system/devices/vhost-vdpa-device.rst |  43 +++
hw/virtio/Kconfig                         |   5 +
hw/virtio/meson.build                     |   2 +
hw/virtio/vdpa-dev-pci.c                  | 102 ++++++
hw/virtio/vdpa-dev.c                      | 377 ++++++++++++++++++++++
hw/virtio/virtio-pci.c                    |  88 +++++
include/hw/virtio/vdpa-dev.h              |  43 +++
include/hw/virtio/virtio-pci.h            |   5 +
8 files changed, 665 insertions(+)
create mode 100644 docs/system/devices/vhost-vdpa-device.rst
create mode 100644 hw/virtio/vdpa-dev-pci.c
create mode 100644 hw/virtio/vdpa-dev.c
create mode 100644 include/hw/virtio/vdpa-dev.h
[PATCH v7 resend 0/4] add generic vDPA device support
Posted by Longpeng(Mike) via 1 year, 6 months ago
From: Longpeng <longpeng2@huawei.com>

Hi guys,

With the generic vDPA device, QEMU won't need to touch the device
types any more, such like vfio.

We can use the generic vDPA device as follow:
  -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
  Or
  -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
  vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x

Changes v6 -> v7:
    (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html)
    - rebase. [Jason]
    - add documentation . [Stefan]

Changes v5 -> v6:
  Patch 2:
    - Turn to the original approach in the RFC to initialize the
      virtio_pci_id_info array. [Michael]
	  https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/
  Patch 3:
    - Fix logical error of exception handler around the post_init.
      [Stefano]
    - Fix some coding style warnings. [Stefano]
  Patch 4:
    - Fix some coding style warnings. [Stefano]

Changes v4 -> v5:
  Patch 3:
    - remove vhostfd [Jason]
    - support virtio-mmio [Jason]

Changes v3 -> v4:
  v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html
  - reorganize the series [Stefano]
  - fix some typos [Stefano]
  - fix logical error in vhost_vdpa_device_realize [Stefano]

Changes v2 -> v3
  Patch 4 & 5:
    - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng]
    - s/VQS_NUM/VQS_COUNT  [Stefano]
    - check both vdpa_dev_fd and vdpa_dev [Stefano]
  Patch 6:
    - move all steps into vhost_vdpa_device_unrealize. [Stefano]

Changes RFC -> v2
  Patch 1:
    - rename 'pdev_id' to 'trans_devid'  [Michael]
    - only use transitional device id for the devices
      listed in the spec  [Michael]
    - use macros to make the id_info table clearer  [Longpeng]
    - add some modern devices in the id_info table  [Longpeng]
  Patch 2:
    - remove the GET_VECTORS_NUM command  [Jason]
  Patch 4:
    - expose vdpa_dev_fd as a QOM preperty  [Stefan]
    - introduce vhost_vdpa_device_get_u32 as a common
      function to make the code clearer  [Stefan]
    - fix the misleading description of 'dc->desc'  [Stefano]
  Patch 5:
    - check returned number of virtqueues  [Stefan]
  Patch 6:
    - init s->num_queues  [Stefano]
    - free s->dev.vqs  [Stefano]


Longpeng (Mike) (4):
  virtio: get class_id and pci device id by the virtio id
  vdpa: add vdpa-dev support
  vdpa: add vdpa-dev-pci support
  docs: Add generic vhost-vdpa device documentation

 docs/system/devices/vhost-vdpa-device.rst |  43 +++
 hw/virtio/Kconfig                         |   5 +
 hw/virtio/meson.build                     |   2 +
 hw/virtio/vdpa-dev-pci.c                  | 102 ++++++
 hw/virtio/vdpa-dev.c                      | 377 ++++++++++++++++++++++
 hw/virtio/virtio-pci.c                    |  88 +++++
 include/hw/virtio/vdpa-dev.h              |  43 +++
 include/hw/virtio/virtio-pci.h            |   5 +
 8 files changed, 665 insertions(+)
 create mode 100644 docs/system/devices/vhost-vdpa-device.rst
 create mode 100644 hw/virtio/vdpa-dev-pci.c
 create mode 100644 hw/virtio/vdpa-dev.c
 create mode 100644 include/hw/virtio/vdpa-dev.h

-- 
2.23.0
Re: [PATCH v7 resend 0/4] add generic vDPA device support
Posted by Michael S. Tsirkin 1 year, 6 months ago
On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote:
> From: Longpeng <longpeng2@huawei.com>
> 
> Hi guys,
> 
> With the generic vDPA device, QEMU won't need to touch the device
> types any more, such like vfio.

With this kind of passthrough migration is completely MIA right?
Better add a blocker...
And given this is there an advantage over VFIO?

> We can use the generic vDPA device as follow:
>   -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
>   Or
>   -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
>   vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x

> Changes v6 -> v7:
>     (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html)
>     - rebase. [Jason]
>     - add documentation . [Stefan]
> 
> Changes v5 -> v6:
>   Patch 2:
>     - Turn to the original approach in the RFC to initialize the
>       virtio_pci_id_info array. [Michael]
> 	  https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/
>   Patch 3:
>     - Fix logical error of exception handler around the post_init.
>       [Stefano]
>     - Fix some coding style warnings. [Stefano]
>   Patch 4:
>     - Fix some coding style warnings. [Stefano]
> 
> Changes v4 -> v5:
>   Patch 3:
>     - remove vhostfd [Jason]
>     - support virtio-mmio [Jason]
> 
> Changes v3 -> v4:
>   v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html
>   - reorganize the series [Stefano]
>   - fix some typos [Stefano]
>   - fix logical error in vhost_vdpa_device_realize [Stefano]
> 
> Changes v2 -> v3
>   Patch 4 & 5:
>     - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng]
>     - s/VQS_NUM/VQS_COUNT  [Stefano]
>     - check both vdpa_dev_fd and vdpa_dev [Stefano]
>   Patch 6:
>     - move all steps into vhost_vdpa_device_unrealize. [Stefano]
> 
> Changes RFC -> v2
>   Patch 1:
>     - rename 'pdev_id' to 'trans_devid'  [Michael]
>     - only use transitional device id for the devices
>       listed in the spec  [Michael]
>     - use macros to make the id_info table clearer  [Longpeng]
>     - add some modern devices in the id_info table  [Longpeng]
>   Patch 2:
>     - remove the GET_VECTORS_NUM command  [Jason]
>   Patch 4:
>     - expose vdpa_dev_fd as a QOM preperty  [Stefan]
>     - introduce vhost_vdpa_device_get_u32 as a common
>       function to make the code clearer  [Stefan]
>     - fix the misleading description of 'dc->desc'  [Stefano]
>   Patch 5:
>     - check returned number of virtqueues  [Stefan]
>   Patch 6:
>     - init s->num_queues  [Stefano]
>     - free s->dev.vqs  [Stefano]
> 
> 
> Longpeng (Mike) (4):
>   virtio: get class_id and pci device id by the virtio id
>   vdpa: add vdpa-dev support
>   vdpa: add vdpa-dev-pci support
>   docs: Add generic vhost-vdpa device documentation
> 
>  docs/system/devices/vhost-vdpa-device.rst |  43 +++
>  hw/virtio/Kconfig                         |   5 +
>  hw/virtio/meson.build                     |   2 +
>  hw/virtio/vdpa-dev-pci.c                  | 102 ++++++
>  hw/virtio/vdpa-dev.c                      | 377 ++++++++++++++++++++++
>  hw/virtio/virtio-pci.c                    |  88 +++++
>  include/hw/virtio/vdpa-dev.h              |  43 +++
>  include/hw/virtio/virtio-pci.h            |   5 +
>  8 files changed, 665 insertions(+)
>  create mode 100644 docs/system/devices/vhost-vdpa-device.rst
>  create mode 100644 hw/virtio/vdpa-dev-pci.c
>  create mode 100644 hw/virtio/vdpa-dev.c
>  create mode 100644 include/hw/virtio/vdpa-dev.h
> 
> -- 
> 2.23.0
Re: [PATCH v7 resend 0/4] add generic vDPA device support
Posted by longpeng2--- via 1 year, 6 months ago

在 2022/11/6 0:43, Michael S. Tsirkin 写道:
> On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote:
>> From: Longpeng <longpeng2@huawei.com>
>>
>> Hi guys,
>>
>> With the generic vDPA device, QEMU won't need to touch the device
>> types any more, such like vfio.
> 
> With this kind of passthrough migration is completely MIA right?
> Better add a blocker...

Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 
and I'll add it in the next version.

We'll support passthrough migration in the next step. We have already 
written a demo that can migrate between some offloading cards.

> And given this is there an advantage over VFIO?

I think the answer is the same as "why we need vDPA" if we compare it 
with VFIO.

> 
>> We can use the generic vDPA device as follow:
>>    -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
>>    Or
>>    -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
>>    vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x
> 
>> Changes v6 -> v7:
>>      (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html)
>>      - rebase. [Jason]
>>      - add documentation . [Stefan]
>>
>> Changes v5 -> v6:
>>    Patch 2:
>>      - Turn to the original approach in the RFC to initialize the
>>        virtio_pci_id_info array. [Michael]
>> 	  https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/
>>    Patch 3:
>>      - Fix logical error of exception handler around the post_init.
>>        [Stefano]
>>      - Fix some coding style warnings. [Stefano]
>>    Patch 4:
>>      - Fix some coding style warnings. [Stefano]
>>
>> Changes v4 -> v5:
>>    Patch 3:
>>      - remove vhostfd [Jason]
>>      - support virtio-mmio [Jason]
>>
>> Changes v3 -> v4:
>>    v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html
>>    - reorganize the series [Stefano]
>>    - fix some typos [Stefano]
>>    - fix logical error in vhost_vdpa_device_realize [Stefano]
>>
>> Changes v2 -> v3
>>    Patch 4 & 5:
>>      - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng]
>>      - s/VQS_NUM/VQS_COUNT  [Stefano]
>>      - check both vdpa_dev_fd and vdpa_dev [Stefano]
>>    Patch 6:
>>      - move all steps into vhost_vdpa_device_unrealize. [Stefano]
>>
>> Changes RFC -> v2
>>    Patch 1:
>>      - rename 'pdev_id' to 'trans_devid'  [Michael]
>>      - only use transitional device id for the devices
>>        listed in the spec  [Michael]
>>      - use macros to make the id_info table clearer  [Longpeng]
>>      - add some modern devices in the id_info table  [Longpeng]
>>    Patch 2:
>>      - remove the GET_VECTORS_NUM command  [Jason]
>>    Patch 4:
>>      - expose vdpa_dev_fd as a QOM preperty  [Stefan]
>>      - introduce vhost_vdpa_device_get_u32 as a common
>>        function to make the code clearer  [Stefan]
>>      - fix the misleading description of 'dc->desc'  [Stefano]
>>    Patch 5:
>>      - check returned number of virtqueues  [Stefan]
>>    Patch 6:
>>      - init s->num_queues  [Stefano]
>>      - free s->dev.vqs  [Stefano]
>>
>>
>> Longpeng (Mike) (4):
>>    virtio: get class_id and pci device id by the virtio id
>>    vdpa: add vdpa-dev support
>>    vdpa: add vdpa-dev-pci support
>>    docs: Add generic vhost-vdpa device documentation
>>
>>   docs/system/devices/vhost-vdpa-device.rst |  43 +++
>>   hw/virtio/Kconfig                         |   5 +
>>   hw/virtio/meson.build                     |   2 +
>>   hw/virtio/vdpa-dev-pci.c                  | 102 ++++++
>>   hw/virtio/vdpa-dev.c                      | 377 ++++++++++++++++++++++
>>   hw/virtio/virtio-pci.c                    |  88 +++++
>>   include/hw/virtio/vdpa-dev.h              |  43 +++
>>   include/hw/virtio/virtio-pci.h            |   5 +
>>   8 files changed, 665 insertions(+)
>>   create mode 100644 docs/system/devices/vhost-vdpa-device.rst
>>   create mode 100644 hw/virtio/vdpa-dev-pci.c
>>   create mode 100644 hw/virtio/vdpa-dev.c
>>   create mode 100644 include/hw/virtio/vdpa-dev.h
>>
>> -- 
>> 2.23.0
> 
> .

Re: [PATCH v7 resend 0/4] add generic vDPA device support
Posted by Michael S. Tsirkin 1 year, 6 months ago
On Sun, Nov 06, 2022 at 08:17:07AM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote:
> 
> 
> 在 2022/11/6 0:43, Michael S. Tsirkin 写道:
> > On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote:
> > > From: Longpeng <longpeng2@huawei.com>
> > > 
> > > Hi guys,
> > > 
> > > With the generic vDPA device, QEMU won't need to touch the device
> > > types any more, such like vfio.
> > 
> > With this kind of passthrough migration is completely MIA right?
> > Better add a blocker...
> 
> Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 and
> I'll add it in the next version.
> 
> We'll support passthrough migration in the next step. We have already
> written a demo that can migrate between some offloading cards.

Hmm ok. Backend disconnect can't work though, can it? State
is by necessity lost when backend crashes.

> > And given this is there an advantage over VFIO?
> 
> I think the answer is the same as "why we need vDPA" if we compare it with
> VFIO.

The answer is mostly because you can migrate and support backend
disconnect, no?

> > 
> > > We can use the generic vDPA device as follow:
> > >    -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
> > >    Or
> > >    -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
> > >    vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x
> > 
> > > Changes v6 -> v7:
> > >      (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html)
> > >      - rebase. [Jason]
> > >      - add documentation . [Stefan]
> > > 
> > > Changes v5 -> v6:
> > >    Patch 2:
> > >      - Turn to the original approach in the RFC to initialize the
> > >        virtio_pci_id_info array. [Michael]
> > > 	  https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/
> > >    Patch 3:
> > >      - Fix logical error of exception handler around the post_init.
> > >        [Stefano]
> > >      - Fix some coding style warnings. [Stefano]
> > >    Patch 4:
> > >      - Fix some coding style warnings. [Stefano]
> > > 
> > > Changes v4 -> v5:
> > >    Patch 3:
> > >      - remove vhostfd [Jason]
> > >      - support virtio-mmio [Jason]
> > > 
> > > Changes v3 -> v4:
> > >    v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html
> > >    - reorganize the series [Stefano]
> > >    - fix some typos [Stefano]
> > >    - fix logical error in vhost_vdpa_device_realize [Stefano]
> > > 
> > > Changes v2 -> v3
> > >    Patch 4 & 5:
> > >      - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng]
> > >      - s/VQS_NUM/VQS_COUNT  [Stefano]
> > >      - check both vdpa_dev_fd and vdpa_dev [Stefano]
> > >    Patch 6:
> > >      - move all steps into vhost_vdpa_device_unrealize. [Stefano]
> > > 
> > > Changes RFC -> v2
> > >    Patch 1:
> > >      - rename 'pdev_id' to 'trans_devid'  [Michael]
> > >      - only use transitional device id for the devices
> > >        listed in the spec  [Michael]
> > >      - use macros to make the id_info table clearer  [Longpeng]
> > >      - add some modern devices in the id_info table  [Longpeng]
> > >    Patch 2:
> > >      - remove the GET_VECTORS_NUM command  [Jason]
> > >    Patch 4:
> > >      - expose vdpa_dev_fd as a QOM preperty  [Stefan]
> > >      - introduce vhost_vdpa_device_get_u32 as a common
> > >        function to make the code clearer  [Stefan]
> > >      - fix the misleading description of 'dc->desc'  [Stefano]
> > >    Patch 5:
> > >      - check returned number of virtqueues  [Stefan]
> > >    Patch 6:
> > >      - init s->num_queues  [Stefano]
> > >      - free s->dev.vqs  [Stefano]
> > > 
> > > 
> > > Longpeng (Mike) (4):
> > >    virtio: get class_id and pci device id by the virtio id
> > >    vdpa: add vdpa-dev support
> > >    vdpa: add vdpa-dev-pci support
> > >    docs: Add generic vhost-vdpa device documentation
> > > 
> > >   docs/system/devices/vhost-vdpa-device.rst |  43 +++
> > >   hw/virtio/Kconfig                         |   5 +
> > >   hw/virtio/meson.build                     |   2 +
> > >   hw/virtio/vdpa-dev-pci.c                  | 102 ++++++
> > >   hw/virtio/vdpa-dev.c                      | 377 ++++++++++++++++++++++
> > >   hw/virtio/virtio-pci.c                    |  88 +++++
> > >   include/hw/virtio/vdpa-dev.h              |  43 +++
> > >   include/hw/virtio/virtio-pci.h            |   5 +
> > >   8 files changed, 665 insertions(+)
> > >   create mode 100644 docs/system/devices/vhost-vdpa-device.rst
> > >   create mode 100644 hw/virtio/vdpa-dev-pci.c
> > >   create mode 100644 hw/virtio/vdpa-dev.c
> > >   create mode 100644 include/hw/virtio/vdpa-dev.h
> > > 
> > > -- 
> > > 2.23.0
> > 
> > .


Re: [PATCH v7 resend 0/4] add generic vDPA device support
Posted by longpeng2--- via 1 year, 6 months ago

在 2022/11/6 13:22, Michael S. Tsirkin 写道:
> On Sun, Nov 06, 2022 at 08:17:07AM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote:
>>
>>
>> 在 2022/11/6 0:43, Michael S. Tsirkin 写道:
>>> On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote:
>>>> From: Longpeng <longpeng2@huawei.com>
>>>>
>>>> Hi guys,
>>>>
>>>> With the generic vDPA device, QEMU won't need to touch the device
>>>> types any more, such like vfio.
>>>
>>> With this kind of passthrough migration is completely MIA right?
>>> Better add a blocker...
>>
>> Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 and
>> I'll add it in the next version.
>>
>> We'll support passthrough migration in the next step. We have already
>> written a demo that can migrate between some offloading cards.
> 
> Hmm ok. Backend disconnect can't work though, can it? State
> is by necessity lost when backend crashes.
>Yes, it can't.

>>> And given this is there an advantage over VFIO?
>>
>> I think the answer is the same as "why we need vDPA" if we compare it with
>> VFIO.
> 
> The answer is mostly because you can migrate and support backend
> disconnect, no?
>
Migrating between different hardware is the first consideration in our 
requirement, supporting backend disconnect is a low priority.

>>>
>>>> We can use the generic vDPA device as follow:
>>>>     -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
>>>>     Or
>>>>     -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
>>>>     vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x
>>>
>>>> Changes v6 -> v7:
>>>>       (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html)
>>>>       - rebase. [Jason]
>>>>       - add documentation . [Stefan]
>>>>
>>>> Changes v5 -> v6:
>>>>     Patch 2:
>>>>       - Turn to the original approach in the RFC to initialize the
>>>>         virtio_pci_id_info array. [Michael]
>>>> 	  https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/
>>>>     Patch 3:
>>>>       - Fix logical error of exception handler around the post_init.
>>>>         [Stefano]
>>>>       - Fix some coding style warnings. [Stefano]
>>>>     Patch 4:
>>>>       - Fix some coding style warnings. [Stefano]
>>>>
>>>> Changes v4 -> v5:
>>>>     Patch 3:
>>>>       - remove vhostfd [Jason]
>>>>       - support virtio-mmio [Jason]
>>>>
>>>> Changes v3 -> v4:
>>>>     v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html
>>>>     - reorganize the series [Stefano]
>>>>     - fix some typos [Stefano]
>>>>     - fix logical error in vhost_vdpa_device_realize [Stefano]
>>>>
>>>> Changes v2 -> v3
>>>>     Patch 4 & 5:
>>>>       - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng]
>>>>       - s/VQS_NUM/VQS_COUNT  [Stefano]
>>>>       - check both vdpa_dev_fd and vdpa_dev [Stefano]
>>>>     Patch 6:
>>>>       - move all steps into vhost_vdpa_device_unrealize. [Stefano]
>>>>
>>>> Changes RFC -> v2
>>>>     Patch 1:
>>>>       - rename 'pdev_id' to 'trans_devid'  [Michael]
>>>>       - only use transitional device id for the devices
>>>>         listed in the spec  [Michael]
>>>>       - use macros to make the id_info table clearer  [Longpeng]
>>>>       - add some modern devices in the id_info table  [Longpeng]
>>>>     Patch 2:
>>>>       - remove the GET_VECTORS_NUM command  [Jason]
>>>>     Patch 4:
>>>>       - expose vdpa_dev_fd as a QOM preperty  [Stefan]
>>>>       - introduce vhost_vdpa_device_get_u32 as a common
>>>>         function to make the code clearer  [Stefan]
>>>>       - fix the misleading description of 'dc->desc'  [Stefano]
>>>>     Patch 5:
>>>>       - check returned number of virtqueues  [Stefan]
>>>>     Patch 6:
>>>>       - init s->num_queues  [Stefano]
>>>>       - free s->dev.vqs  [Stefano]
>>>>
>>>>
>>>> Longpeng (Mike) (4):
>>>>     virtio: get class_id and pci device id by the virtio id
>>>>     vdpa: add vdpa-dev support
>>>>     vdpa: add vdpa-dev-pci support
>>>>     docs: Add generic vhost-vdpa device documentation
>>>>
>>>>    docs/system/devices/vhost-vdpa-device.rst |  43 +++
>>>>    hw/virtio/Kconfig                         |   5 +
>>>>    hw/virtio/meson.build                     |   2 +
>>>>    hw/virtio/vdpa-dev-pci.c                  | 102 ++++++
>>>>    hw/virtio/vdpa-dev.c                      | 377 ++++++++++++++++++++++
>>>>    hw/virtio/virtio-pci.c                    |  88 +++++
>>>>    include/hw/virtio/vdpa-dev.h              |  43 +++
>>>>    include/hw/virtio/virtio-pci.h            |   5 +
>>>>    8 files changed, 665 insertions(+)
>>>>    create mode 100644 docs/system/devices/vhost-vdpa-device.rst
>>>>    create mode 100644 hw/virtio/vdpa-dev-pci.c
>>>>    create mode 100644 hw/virtio/vdpa-dev.c
>>>>    create mode 100644 include/hw/virtio/vdpa-dev.h
>>>>
>>>> -- 
>>>> 2.23.0
>>>
>>> .
> 
> 
> .

Re: [PATCH v7 resend 0/4] add generic vDPA device support
Posted by Michael S. Tsirkin 1 year, 6 months ago
On Sun, Nov 06, 2022 at 09:11:39PM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote:
> 
> 
> 在 2022/11/6 13:22, Michael S. Tsirkin 写道:
> > On Sun, Nov 06, 2022 at 08:17:07AM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote:
> > > 
> > > 
> > > 在 2022/11/6 0:43, Michael S. Tsirkin 写道:
> > > > On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote:
> > > > > From: Longpeng <longpeng2@huawei.com>
> > > > > 
> > > > > Hi guys,
> > > > > 
> > > > > With the generic vDPA device, QEMU won't need to touch the device
> > > > > types any more, such like vfio.
> > > > 
> > > > With this kind of passthrough migration is completely MIA right?
> > > > Better add a blocker...
> > > 
> > > Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 and
> > > I'll add it in the next version.
> > > 
> > > We'll support passthrough migration in the next step. We have already
> > > written a demo that can migrate between some offloading cards.
> > 
> > Hmm ok. Backend disconnect can't work though, can it? State
> > is by necessity lost when backend crashes.
> > Yes, it can't.
> 
> > > > And given this is there an advantage over VFIO?
> > > 
> > > I think the answer is the same as "why we need vDPA" if we compare it with
> > > VFIO.
> > 
> > The answer is mostly because you can migrate and support backend
> > disconnect, no?
> > 
> Migrating between different hardware is the first consideration in our
> requirement, supporting backend disconnect is a low priority.

I dislike non-orthogonal features though ...
And the advantage of keeping it out of process with qemu is
I presume security?

> > > > 
> > > > > We can use the generic vDPA device as follow:
> > > > >     -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
> > > > >     Or
> > > > >     -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
> > > > >     vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x
> > > > 
> > > > > Changes v6 -> v7:
> > > > >       (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html)
> > > > >       - rebase. [Jason]
> > > > >       - add documentation . [Stefan]
> > > > > 
> > > > > Changes v5 -> v6:
> > > > >     Patch 2:
> > > > >       - Turn to the original approach in the RFC to initialize the
> > > > >         virtio_pci_id_info array. [Michael]
> > > > > 	  https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/
> > > > >     Patch 3:
> > > > >       - Fix logical error of exception handler around the post_init.
> > > > >         [Stefano]
> > > > >       - Fix some coding style warnings. [Stefano]
> > > > >     Patch 4:
> > > > >       - Fix some coding style warnings. [Stefano]
> > > > > 
> > > > > Changes v4 -> v5:
> > > > >     Patch 3:
> > > > >       - remove vhostfd [Jason]
> > > > >       - support virtio-mmio [Jason]
> > > > > 
> > > > > Changes v3 -> v4:
> > > > >     v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html
> > > > >     - reorganize the series [Stefano]
> > > > >     - fix some typos [Stefano]
> > > > >     - fix logical error in vhost_vdpa_device_realize [Stefano]
> > > > > 
> > > > > Changes v2 -> v3
> > > > >     Patch 4 & 5:
> > > > >       - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng]
> > > > >       - s/VQS_NUM/VQS_COUNT  [Stefano]
> > > > >       - check both vdpa_dev_fd and vdpa_dev [Stefano]
> > > > >     Patch 6:
> > > > >       - move all steps into vhost_vdpa_device_unrealize. [Stefano]
> > > > > 
> > > > > Changes RFC -> v2
> > > > >     Patch 1:
> > > > >       - rename 'pdev_id' to 'trans_devid'  [Michael]
> > > > >       - only use transitional device id for the devices
> > > > >         listed in the spec  [Michael]
> > > > >       - use macros to make the id_info table clearer  [Longpeng]
> > > > >       - add some modern devices in the id_info table  [Longpeng]
> > > > >     Patch 2:
> > > > >       - remove the GET_VECTORS_NUM command  [Jason]
> > > > >     Patch 4:
> > > > >       - expose vdpa_dev_fd as a QOM preperty  [Stefan]
> > > > >       - introduce vhost_vdpa_device_get_u32 as a common
> > > > >         function to make the code clearer  [Stefan]
> > > > >       - fix the misleading description of 'dc->desc'  [Stefano]
> > > > >     Patch 5:
> > > > >       - check returned number of virtqueues  [Stefan]
> > > > >     Patch 6:
> > > > >       - init s->num_queues  [Stefano]
> > > > >       - free s->dev.vqs  [Stefano]
> > > > > 
> > > > > 
> > > > > Longpeng (Mike) (4):
> > > > >     virtio: get class_id and pci device id by the virtio id
> > > > >     vdpa: add vdpa-dev support
> > > > >     vdpa: add vdpa-dev-pci support
> > > > >     docs: Add generic vhost-vdpa device documentation
> > > > > 
> > > > >    docs/system/devices/vhost-vdpa-device.rst |  43 +++
> > > > >    hw/virtio/Kconfig                         |   5 +
> > > > >    hw/virtio/meson.build                     |   2 +
> > > > >    hw/virtio/vdpa-dev-pci.c                  | 102 ++++++
> > > > >    hw/virtio/vdpa-dev.c                      | 377 ++++++++++++++++++++++
> > > > >    hw/virtio/virtio-pci.c                    |  88 +++++
> > > > >    include/hw/virtio/vdpa-dev.h              |  43 +++
> > > > >    include/hw/virtio/virtio-pci.h            |   5 +
> > > > >    8 files changed, 665 insertions(+)
> > > > >    create mode 100644 docs/system/devices/vhost-vdpa-device.rst
> > > > >    create mode 100644 hw/virtio/vdpa-dev-pci.c
> > > > >    create mode 100644 hw/virtio/vdpa-dev.c
> > > > >    create mode 100644 include/hw/virtio/vdpa-dev.h
> > > > > 
> > > > > -- 
> > > > > 2.23.0
> > > > 
> > > > .
> > 
> > 
> > .


Re: [PATCH v7 resend 0/4] add generic vDPA device support
Posted by longpeng2--- via 1 year, 6 months ago

在 2022/11/6 21:47, Michael S. Tsirkin 写道:
> On Sun, Nov 06, 2022 at 09:11:39PM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote:
>>
>>
>> 在 2022/11/6 13:22, Michael S. Tsirkin 写道:
>>> On Sun, Nov 06, 2022 at 08:17:07AM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote:
>>>>
>>>>
>>>> 在 2022/11/6 0:43, Michael S. Tsirkin 写道:
>>>>> On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote:
>>>>>> From: Longpeng <longpeng2@huawei.com>
>>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>> With the generic vDPA device, QEMU won't need to touch the device
>>>>>> types any more, such like vfio.
>>>>>
>>>>> With this kind of passthrough migration is completely MIA right?
>>>>> Better add a blocker...
>>>>
>>>> Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 and
>>>> I'll add it in the next version.
>>>>
>>>> We'll support passthrough migration in the next step. We have already
>>>> written a demo that can migrate between some offloading cards.
>>>
>>> Hmm ok. Backend disconnect can't work though, can it? State
>>> is by necessity lost when backend crashes.
>>> Yes, it can't.
>>
>>>>> And given this is there an advantage over VFIO?
>>>>
>>>> I think the answer is the same as "why we need vDPA" if we compare it with
>>>> VFIO.
>>>
>>> The answer is mostly because you can migrate and support backend
>>> disconnect, no?
>>>
>> Migrating between different hardware is the first consideration in our
>> requirement, supporting backend disconnect is a low priority.
> 
> I dislike non-orthogonal features though ...
> And the advantage of keeping it out of process with qemu is
> I presume security?
> 

Yes, this is one of the reasons. The TCB of the generic vdpa device is 
smaller than the existing vdpa device (needs to use the 
virtio-net/blk/scsi emulation codes).

Besides, the generic vdpa device can support any virtio device, but the 
existing vdpa device only supports virtio-net yet.

Though the existing vdpa device is more powerful and the generic vdpa 
device would miss some features, it can be an alternative for some users.


>>>>>
>>>>>> We can use the generic vDPA device as follow:
>>>>>>      -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
>>>>>>      Or
>>>>>>      -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
>>>>>>      vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x
>>>>>
>>>>>> Changes v6 -> v7:
>>>>>>        (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html)
>>>>>>        - rebase. [Jason]
>>>>>>        - add documentation . [Stefan]
>>>>>>
>>>>>> Changes v5 -> v6:
>>>>>>      Patch 2:
>>>>>>        - Turn to the original approach in the RFC to initialize the
>>>>>>          virtio_pci_id_info array. [Michael]
>>>>>> 	  https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/
>>>>>>      Patch 3:
>>>>>>        - Fix logical error of exception handler around the post_init.
>>>>>>          [Stefano]
>>>>>>        - Fix some coding style warnings. [Stefano]
>>>>>>      Patch 4:
>>>>>>        - Fix some coding style warnings. [Stefano]
>>>>>>
>>>>>> Changes v4 -> v5:
>>>>>>      Patch 3:
>>>>>>        - remove vhostfd [Jason]
>>>>>>        - support virtio-mmio [Jason]
>>>>>>
>>>>>> Changes v3 -> v4:
>>>>>>      v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html
>>>>>>      - reorganize the series [Stefano]
>>>>>>      - fix some typos [Stefano]
>>>>>>      - fix logical error in vhost_vdpa_device_realize [Stefano]
>>>>>>
>>>>>> Changes v2 -> v3
>>>>>>      Patch 4 & 5:
>>>>>>        - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng]
>>>>>>        - s/VQS_NUM/VQS_COUNT  [Stefano]
>>>>>>        - check both vdpa_dev_fd and vdpa_dev [Stefano]
>>>>>>      Patch 6:
>>>>>>        - move all steps into vhost_vdpa_device_unrealize. [Stefano]
>>>>>>
>>>>>> Changes RFC -> v2
>>>>>>      Patch 1:
>>>>>>        - rename 'pdev_id' to 'trans_devid'  [Michael]
>>>>>>        - only use transitional device id for the devices
>>>>>>          listed in the spec  [Michael]
>>>>>>        - use macros to make the id_info table clearer  [Longpeng]
>>>>>>        - add some modern devices in the id_info table  [Longpeng]
>>>>>>      Patch 2:
>>>>>>        - remove the GET_VECTORS_NUM command  [Jason]
>>>>>>      Patch 4:
>>>>>>        - expose vdpa_dev_fd as a QOM preperty  [Stefan]
>>>>>>        - introduce vhost_vdpa_device_get_u32 as a common
>>>>>>          function to make the code clearer  [Stefan]
>>>>>>        - fix the misleading description of 'dc->desc'  [Stefano]
>>>>>>      Patch 5:
>>>>>>        - check returned number of virtqueues  [Stefan]
>>>>>>      Patch 6:
>>>>>>        - init s->num_queues  [Stefano]
>>>>>>        - free s->dev.vqs  [Stefano]
>>>>>>
>>>>>>
>>>>>> Longpeng (Mike) (4):
>>>>>>      virtio: get class_id and pci device id by the virtio id
>>>>>>      vdpa: add vdpa-dev support
>>>>>>      vdpa: add vdpa-dev-pci support
>>>>>>      docs: Add generic vhost-vdpa device documentation
>>>>>>
>>>>>>     docs/system/devices/vhost-vdpa-device.rst |  43 +++
>>>>>>     hw/virtio/Kconfig                         |   5 +
>>>>>>     hw/virtio/meson.build                     |   2 +
>>>>>>     hw/virtio/vdpa-dev-pci.c                  | 102 ++++++
>>>>>>     hw/virtio/vdpa-dev.c                      | 377 ++++++++++++++++++++++
>>>>>>     hw/virtio/virtio-pci.c                    |  88 +++++
>>>>>>     include/hw/virtio/vdpa-dev.h              |  43 +++
>>>>>>     include/hw/virtio/virtio-pci.h            |   5 +
>>>>>>     8 files changed, 665 insertions(+)
>>>>>>     create mode 100644 docs/system/devices/vhost-vdpa-device.rst
>>>>>>     create mode 100644 hw/virtio/vdpa-dev-pci.c
>>>>>>     create mode 100644 hw/virtio/vdpa-dev.c
>>>>>>     create mode 100644 include/hw/virtio/vdpa-dev.h
>>>>>>
>>>>>> -- 
>>>>>> 2.23.0
>>>>>
>>>>> .
>>>
>>>
>>> .
> 
> 
> .