[PATCH v4 00/19] vfio-user client

John Levon posted 19 patches 4 months, 4 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20250619133154.264786-1-john.levon@nutanix.com
Maintainers: John Levon <john.levon@nutanix.com>, Thanos Makatos <thanos.makatos@nutanix.com>, Paolo Bonzini <pbonzini@redhat.com>, Alex Williamson <alex.williamson@redhat.com>, "Cédric Le Goater" <clg@redhat.com>, "Marc-André Lureau" <marcandre.lureau@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>
There is a newer version of this series
MAINTAINERS                           |   11 +-
docs/interop/index.rst                |    1 +
docs/interop/vfio-user.rst            | 1518 +++++++++++++++++++++++++
docs/system/device-emulation.rst      |    1 +
docs/system/devices/vfio-user.rst     |   24 +
meson.build                           |    1 +
hw/vfio-user/container.h              |   23 +
hw/vfio-user/device.h                 |   24 +
hw/vfio-user/protocol.h               |  242 ++++
hw/vfio-user/proxy.h                  |  134 +++
hw/vfio-user/trace.h                  |    1 +
hw/vfio/pci.h                         |    1 +
include/hw/vfio/vfio-container-base.h |    1 +
include/hw/vfio/vfio-device.h         |    2 +
hw/vfio-user/container.c              |  345 ++++++
hw/vfio-user/device.c                 |  387 +++++++
hw/vfio-user/pci.c                    |  475 ++++++++
hw/vfio-user/proxy.c                  | 1311 +++++++++++++++++++++
hw/Kconfig                            |    1 +
hw/meson.build                        |    1 +
hw/vfio-user/Kconfig                  |    5 +
hw/vfio-user/meson.build              |    9 +
hw/vfio-user/trace-events             |   18 +
23 files changed, 4535 insertions(+), 1 deletion(-)
create mode 100644 docs/interop/vfio-user.rst
create mode 100644 docs/system/devices/vfio-user.rst
create mode 100644 hw/vfio-user/container.h
create mode 100644 hw/vfio-user/device.h
create mode 100644 hw/vfio-user/protocol.h
create mode 100644 hw/vfio-user/proxy.h
create mode 100644 hw/vfio-user/trace.h
create mode 100644 hw/vfio-user/container.c
create mode 100644 hw/vfio-user/device.c
create mode 100644 hw/vfio-user/pci.c
create mode 100644 hw/vfio-user/proxy.c
create mode 100644 hw/vfio-user/Kconfig
create mode 100644 hw/vfio-user/meson.build
create mode 100644 hw/vfio-user/trace-events
[PATCH v4 00/19] vfio-user client
Posted by John Levon 4 months, 4 weeks ago
The series contains an implement of a vfio-user client in QEMU, along with a few
more preparatory patches.

The vfio-user protocol allows for implementing (PCI) devices in another
userspace process; SPDK is one example, which includes a virtual NVMe
implementation.

The vfio-user framework consists of 3 parts:
 1) The VFIO user protocol specification.
 2) A client - the VFIO device in QEMU that encapsulates VFIO messages
    and sends them to the server.
 3) A server - a remote process that emulates a device.

This patchset implements parts 1 and 2.

It has been tested against libvfio-user test servers as well as SPDK.
A functional test is still pending.

A previous version of this series can be found at
https://lore.kernel.org/qemu-devel/20250607001056.335310-1-john.levon@nutanix.com/

Changes since last series:

 - the vfio-user client is now enabled by default, the configure option has been
   removed
 - the documentation has been relocated next to vhost-user's
 - SocketAddress is now used for the command line parameter
 - small bits of code review feedback

John Levon (18):
  vfio-user: add vfio-user class and container
  vfio-user: connect vfio proxy to remote server
  vfio-user: implement message receive infrastructure
  vfio-user: implement message send infrastructure
  vfio-user: implement VFIO_USER_DEVICE_GET_INFO
  vfio-user: implement VFIO_USER_DEVICE_GET_REGION_INFO
  vfio-user: implement VFIO_USER_REGION_READ/WRITE
  vfio-user: set up PCI in vfio_user_pci_realize()
  vfio-user: implement VFIO_USER_DEVICE_GET/SET_IRQ*
  vfio-user: forward MSI-X PBA BAR accesses to server
  vfio-user: set up container access to the proxy
  vfio-user: implement VFIO_USER_DEVICE_RESET
  vfio-user: implement VFIO_USER_DMA_MAP/UNMAP
  vfio-user: implement VFIO_USER_DMA_READ/WRITE
  vfio-user: add 'x-msg-timeout' option
  vfio-user: support posted writes
  vfio-user: add coalesced posted writes
  docs: add vfio-user documentation

Thanos Makatos (1):
  vfio-user: introduce vfio-user protocol specification

 MAINTAINERS                           |   11 +-
 docs/interop/index.rst                |    1 +
 docs/interop/vfio-user.rst            | 1518 +++++++++++++++++++++++++
 docs/system/device-emulation.rst      |    1 +
 docs/system/devices/vfio-user.rst     |   24 +
 meson.build                           |    1 +
 hw/vfio-user/container.h              |   23 +
 hw/vfio-user/device.h                 |   24 +
 hw/vfio-user/protocol.h               |  242 ++++
 hw/vfio-user/proxy.h                  |  134 +++
 hw/vfio-user/trace.h                  |    1 +
 hw/vfio/pci.h                         |    1 +
 include/hw/vfio/vfio-container-base.h |    1 +
 include/hw/vfio/vfio-device.h         |    2 +
 hw/vfio-user/container.c              |  345 ++++++
 hw/vfio-user/device.c                 |  387 +++++++
 hw/vfio-user/pci.c                    |  475 ++++++++
 hw/vfio-user/proxy.c                  | 1311 +++++++++++++++++++++
 hw/Kconfig                            |    1 +
 hw/meson.build                        |    1 +
 hw/vfio-user/Kconfig                  |    5 +
 hw/vfio-user/meson.build              |    9 +
 hw/vfio-user/trace-events             |   18 +
 23 files changed, 4535 insertions(+), 1 deletion(-)
 create mode 100644 docs/interop/vfio-user.rst
 create mode 100644 docs/system/devices/vfio-user.rst
 create mode 100644 hw/vfio-user/container.h
 create mode 100644 hw/vfio-user/device.h
 create mode 100644 hw/vfio-user/protocol.h
 create mode 100644 hw/vfio-user/proxy.h
 create mode 100644 hw/vfio-user/trace.h
 create mode 100644 hw/vfio-user/container.c
 create mode 100644 hw/vfio-user/device.c
 create mode 100644 hw/vfio-user/pci.c
 create mode 100644 hw/vfio-user/proxy.c
 create mode 100644 hw/vfio-user/Kconfig
 create mode 100644 hw/vfio-user/meson.build
 create mode 100644 hw/vfio-user/trace-events

-- 
2.43.0
Re: [PATCH v4 00/19] vfio-user client
Posted by Cédric Le Goater 4 months, 4 weeks ago
On 6/19/25 15:31, John Levon wrote:
> The series contains an implement of a vfio-user client in QEMU, along with a few
> more preparatory patches.
> 
> The vfio-user protocol allows for implementing (PCI) devices in another
> userspace process; SPDK is one example, which includes a virtual NVMe
> implementation.
> 
> The vfio-user framework consists of 3 parts:
>   1) The VFIO user protocol specification.
>   2) A client - the VFIO device in QEMU that encapsulates VFIO messages
>      and sends them to the server.
>   3) A server - a remote process that emulates a device.
> 
> This patchset implements parts 1 and 2.
> 
> It has been tested against libvfio-user test servers as well as SPDK.
> A functional test is still pending.
> 
> A previous version of this series can be found at
> https://lore.kernel.org/qemu-devel/20250607001056.335310-1-john.levon@nutanix.com/
> 
> Changes since last series:
> 
>   - the vfio-user client is now enabled by default, the configure option has been
>     removed
>   - the documentation has been relocated next to vhost-user's
>   - SocketAddress is now used for the command line parameter
>   - small bits of code review feedback
> 
> John Levon (18):
>    vfio-user: add vfio-user class and container
>    vfio-user: connect vfio proxy to remote server
>    vfio-user: implement message receive infrastructure
>    vfio-user: implement message send infrastructure
>    vfio-user: implement VFIO_USER_DEVICE_GET_INFO
>    vfio-user: implement VFIO_USER_DEVICE_GET_REGION_INFO
>    vfio-user: implement VFIO_USER_REGION_READ/WRITE
>    vfio-user: set up PCI in vfio_user_pci_realize()
>    vfio-user: implement VFIO_USER_DEVICE_GET/SET_IRQ*
>    vfio-user: forward MSI-X PBA BAR accesses to server
>    vfio-user: set up container access to the proxy
>    vfio-user: implement VFIO_USER_DEVICE_RESET
>    vfio-user: implement VFIO_USER_DMA_MAP/UNMAP
>    vfio-user: implement VFIO_USER_DMA_READ/WRITE
>    vfio-user: add 'x-msg-timeout' option
>    vfio-user: support posted writes
>    vfio-user: add coalesced posted writes
>    docs: add vfio-user documentation
> 
> Thanos Makatos (1):
>    vfio-user: introduce vfio-user protocol specification
> 
>   MAINTAINERS                           |   11 +-
>   docs/interop/index.rst                |    1 +
>   docs/interop/vfio-user.rst            | 1518 +++++++++++++++++++++++++
>   docs/system/device-emulation.rst      |    1 +
>   docs/system/devices/vfio-user.rst     |   24 +
>   meson.build                           |    1 +
>   hw/vfio-user/container.h              |   23 +
>   hw/vfio-user/device.h                 |   24 +
>   hw/vfio-user/protocol.h               |  242 ++++
>   hw/vfio-user/proxy.h                  |  134 +++
>   hw/vfio-user/trace.h                  |    1 +
>   hw/vfio/pci.h                         |    1 +
>   include/hw/vfio/vfio-container-base.h |    1 +
>   include/hw/vfio/vfio-device.h         |    2 +
>   hw/vfio-user/container.c              |  345 ++++++
>   hw/vfio-user/device.c                 |  387 +++++++
>   hw/vfio-user/pci.c                    |  475 ++++++++
>   hw/vfio-user/proxy.c                  | 1311 +++++++++++++++++++++
>   hw/Kconfig                            |    1 +
>   hw/meson.build                        |    1 +
>   hw/vfio-user/Kconfig                  |    5 +
>   hw/vfio-user/meson.build              |    9 +
>   hw/vfio-user/trace-events             |   18 +
>   23 files changed, 4535 insertions(+), 1 deletion(-)
>   create mode 100644 docs/interop/vfio-user.rst
>   create mode 100644 docs/system/devices/vfio-user.rst
>   create mode 100644 hw/vfio-user/container.h
>   create mode 100644 hw/vfio-user/device.h
>   create mode 100644 hw/vfio-user/protocol.h
>   create mode 100644 hw/vfio-user/proxy.h
>   create mode 100644 hw/vfio-user/trace.h
>   create mode 100644 hw/vfio-user/container.c
>   create mode 100644 hw/vfio-user/device.c
>   create mode 100644 hw/vfio-user/pci.c
>   create mode 100644 hw/vfio-user/proxy.c
>   create mode 100644 hw/vfio-user/Kconfig
>   create mode 100644 hw/vfio-user/meson.build
>   create mode 100644 hw/vfio-user/trace-events
> 

John,

Please run ./scripts/checkpatch.pl and resend later on, as we might get
reviews. I still hope to have a test too.


Thanks,

C.
Re: [PATCH v4 00/19] vfio-user client
Posted by John Levon 4 months, 3 weeks ago
On Fri, Jun 20, 2025 at 11:11:10AM +0200, Cédric Le Goater wrote:

> > The series contains an implement of a vfio-user client in QEMU, along with a few
> > more preparatory patches.
> 
> Please run ./scripts/checkpatch.pl and resend later on, as we might get
> reviews.

Output is below, MAINTAINERS looks correct to me, and the other complaints are
for files that are the same as in hw/vfio/

So please let me know what if anything needs to be fixed (and if I should make
the same retrospective fix to hw/vfio/ equivalent).

regards
john


1/19 Checking commit cfffef23476c (vfio-user: add vfio-user class and container)
WARNING: Does new file 'hw/vfio-user/Kconfig' need 'SPDX-License-Identifier'?
WARNING: Does new file 'hw/vfio-user/meson.build' need 'SPDX-License-Identifier'?
total: 0 errors, 2 warnings, 461 lines checked

Patch 1/19 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/19 Checking commit 132ba4d3a176 (vfio-user: connect vfio proxy to remote server)
WARNING: added, moved or deleted file(s):

  hw/vfio-user/proxy.h
  hw/vfio-user/proxy.c

Does MAINTAINERS need updating?

total: 0 errors, 1 warnings, 316 lines checked

Patch 2/19 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/19 Checking commit 459bf245dbde (vfio-user: implement message receive infrastructure)
ERROR: New file 'hw/vfio-user/trace.h' requires 'SPDX-License-Identifier'
WARNING: Does new file 'hw/vfio-user/trace-events' need 'SPDX-License-Identifier'?
WARNING: added, moved or deleted file(s):

  hw/vfio-user/protocol.h
  hw/vfio-user/trace.h
  hw/vfio-user/trace-events

Does MAINTAINERS need updating?

total: 1 errors, 2 warnings, 584 lines checked

Patch 3/19 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/19 Checking commit 8bc73aa81e1a (vfio-user: implement message send infrastructure)
total: 0 errors, 0 warnings, 698 lines checked

Patch 4/19 has no obvious style problems and is ready for submission.
5/19 Checking commit 0197d0323b7d (vfio-user: implement VFIO_USER_DEVICE_GET_INFO)
WARNING: added, moved or deleted file(s):

  hw/vfio-user/device.h
  hw/vfio-user/device.c

Does MAINTAINERS need updating?

total: 0 errors, 1 warnings, 165 lines checked

Patch 5/19 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/19 Checking commit 0d4bf7c4847d (vfio-user: implement VFIO_USER_DEVICE_GET_REGION_INFO)
total: 0 errors, 0 warnings, 134 lines checked

Patch 6/19 has no obvious style problems and is ready for submission.
7/19 Checking commit ab4156d34c6a (vfio-user: implement VFIO_USER_REGION_READ/WRITE)
total: 0 errors, 0 warnings, 96 lines checked

Patch 7/19 has no obvious style problems and is ready for submission.
8/19 Checking commit 880367f1b72c (vfio-user: set up PCI in vfio_user_pci_realize())
total: 0 errors, 0 warnings, 39 lines checked

Patch 8/19 has no obvious style problems and is ready for submission.
9/19 Checking commit 98aeea76f227 (vfio-user: implement VFIO_USER_DEVICE_GET/SET_IRQ*)
total: 0 errors, 0 warnings, 169 lines checked

Patch 9/19 has no obvious style problems and is ready for submission.
10/19 Checking commit 98dcb17f4a2f (vfio-user: forward MSI-X PBA BAR accesses to server)
total: 0 errors, 0 warnings, 89 lines checked

Patch 10/19 has no obvious style problems and is ready for submission.
11/19 Checking commit 90f17d9f77bc (vfio-user: set up container access to the proxy)
total: 0 errors, 0 warnings, 115 lines checked

Patch 11/19 has no obvious style problems and is ready for submission.
12/19 Checking commit 1cc6c7a2318a (vfio-user: implement VFIO_USER_DEVICE_RESET)
total: 0 errors, 0 warnings, 53 lines checked

Patch 12/19 has no obvious style problems and is ready for submission.
13/19 Checking commit 44a2750529ec (vfio-user: implement VFIO_USER_DMA_MAP/UNMAP)
total: 0 errors, 0 warnings, 306 lines checked

Patch 13/19 has no obvious style problems and is ready for submission.
14/19 Checking commit bd778e2c6e65 (vfio-user: implement VFIO_USER_DMA_READ/WRITE)
total: 0 errors, 0 warnings, 265 lines checked

Patch 14/19 has no obvious style problems and is ready for submission.
15/19 Checking commit 55b7d98a7919 (vfio-user: add 'x-msg-timeout' option)
total: 0 errors, 0 warnings, 55 lines checked

Patch 15/19 has no obvious style problems and is ready for submission.
16/19 Checking commit 39ed25427ff9 (vfio-user: support posted writes)
total: 0 errors, 0 warnings, 142 lines checked

Patch 16/19 has no obvious style problems and is ready for submission.
17/19 Checking commit 7d146a1fc463 (vfio-user: add coalesced posted writes)
total: 0 errors, 0 warnings, 276 lines checked

Patch 17/19 has no obvious style problems and is ready for submission.
18/19 Checking commit 1df337f26b29 (docs: add vfio-user documentation)
WARNING: Does new file 'docs/system/devices/vfio-user.rst' need 'SPDX-License-Identifier'?
WARNING: added, moved or deleted file(s):

  docs/system/devices/vfio-user.rst

Does MAINTAINERS need updating?

total: 0 errors, 2 warnings, 31 lines checked

Patch 18/19 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
19/19 Checking commit 472c86cd1066 (vfio-user: introduce vfio-user protocol specification)
WARNING: Does new file 'docs/interop/vfio-user.rst' need 'SPDX-License-Identifier'?
total: 0 errors, 1 warnings, 1540 lines checked

Patch 19/19 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Re: [PATCH v4 00/19] vfio-user client
Posted by Cédric Le Goater 4 months, 3 weeks ago
On 6/21/25 13:45, John Levon wrote:
> On Fri, Jun 20, 2025 at 11:11:10AM +0200, Cédric Le Goater wrote:
> 
>>> The series contains an implement of a vfio-user client in QEMU, along with a few
>>> more preparatory patches.
>>
>> Please run ./scripts/checkpatch.pl and resend later on, as we might get
>> reviews.
> 
> Output is below, MAINTAINERS looks correct to me, and the other complaints are
> for files that are the same as in hw/vfio/

Please fix the SPDX-License-Identifier warnings.

> So please let me know what if anything needs to be fixed (and if I should make
> the same retrospective fix to hw/vfio/ equivalent).

Did you update the QEMU tree ?

   $ git describe
   v10.0.0-1657-g6e1571533fd9


Thanks,

C.




Re: [PATCH v4 00/19] vfio-user client
Posted by John Levon 4 months, 3 weeks ago
On Sat, Jun 21, 2025 at 04:25:24PM +0200, Cédric Le Goater wrote:

> > So please let me know what if anything needs to be fixed (and if I should make
> > the same retrospective fix to hw/vfio/ equivalent).
> 
> Did you update the QEMU tree ?
> 
>   $ git describe
>   v10.0.0-1657-g6e1571533fd9

My series is against vfio-next, but regardless, none of

hw/vfio/{Kconfig,meson.build,trace.h,trace-events}

have SPDX identifiers there. I will add it to all of them in a prepatory patch.

regards
john
Re: [PATCH v4 00/19] vfio-user client
Posted by Cédric Le Goater 4 months, 3 weeks ago
On 6/21/25 16:50, John Levon wrote:
> On Sat, Jun 21, 2025 at 04:25:24PM +0200, Cédric Le Goater wrote:
> 
>>> So please let me know what if anything needs to be fixed (and if I should make
>>> the same retrospective fix to hw/vfio/ equivalent).
>>
>> Did you update the QEMU tree ?
>>
>>    $ git describe
>>    v10.0.0-1657-g6e1571533fd9
> 
> My series is against vfio-next, but regardless, none of
> 
> hw/vfio/{Kconfig,meson.build,trace.h,trace-events}
> 
> have SPDX identifiers there. I will add it to all of them in a prepatory patch.
> 
> regards
> john
> 

John,

Could you please send v5 this week ? I plan to merge it first as other
proposals are knocking at the VFIO door and I am OOO on week 28.

I will have a few days in week 29 to take care of the rest (live update),
which is currently lacking reviews.

Thanks,

C.


Re: [PATCH v4 00/19] vfio-user client
Posted by John Levon 4 months, 3 weeks ago
On Wed, Jun 25, 2025 at 10:56:44AM +0200, Cédric Le Goater wrote:

> Could you please send v5 this week ? I plan to merge it first as other
> proposals are knocking at the VFIO door and I am OOO on week 28.

Will take care of your Error comments then send out

thanks
john