From: Marc-André Lureau <marcandre.lureau@redhat.com>

Hi,

This series of patches adds support for running virtio GPUs in
seperate processes, thanks to vhost-user backend.

The QEMU support landed for 4.1. There are several benefits of running
the GPU/virgl in an external process, since Mesa is rather heavy on
the qemu main loop, and may block for a while, or crash.

The external GPU process is started with one end of a socket pair, the
other end is given to a QEMU chardev attached to a device. The
external process is also added to the cgroup to limit resources usage.

Thanks

Since RFC:
- discover helpers following the vhost-user spec
- change vhost-user <video> model for a vhostuser attribute
- add a patch to specify the rendernode on <accel>
- change the way command line is built, following qemu series changes
- socket labeling
- a few cleanup patches
- rebased

Marc-André Lureau (19):
  device-conf: removed unneeded virDomainDeviceInfoCopy()
  tpm: minor argument comment fix
  qemu_firmware: only set nfeatures on success
  qemu: extract out qemuFetchConfigs from firmware
  domain: add "vhostuser" attribute to virtio video model
  domain: add rendernode attribute on <accel>
  qemu-cgroup: allow accel rendernode access
  qemu: add vhost-user-gpu capabilities checks
  qemu: check that qemu is vhost-user-vga capable
  qemu: validate virtio-gpu with vhost-user
  qemu: restrict 'virgl=' option to non-vhostuser video type
  qemu: add vhost-user helpers
  qemu: add qemuSecurityStartVhostUserGPU helper
  qemu: add vhost-user-gpu helper unit
  qemu: prepare domain for vhost-user GPU
  qemu: start/stop the vhost-user-gpu external device
  qemu: build vhost-user GPU devices
  tests: mock execv/execve
  tests: add vhost-user-gpu xml2argv tests

 docs/formatdomain.html.in                     |  11 +
 docs/schemas/domaincommon.rng                 |  16 +-
 src/conf/device_conf.c                        |  22 +-
 src/conf/device_conf.h                        |   4 +-
 src/conf/domain_conf.c                        |  33 +-
 src/conf/domain_conf.h                        |   2 +
 src/libvirt_private.syms                      |   1 -
 src/qemu/Makefile.inc.am                      |   6 +
 src/qemu/qemu_capabilities.c                  |   6 +
 src/qemu/qemu_capabilities.h                  |   4 +
 src/qemu/qemu_cgroup.c                        |  24 ++
 src/qemu/qemu_command.c                       |  54 ++-
 src/qemu/qemu_configs.c                       | 183 ++++++++
 src/qemu/qemu_configs.h                       |  31 ++
 src/qemu/qemu_domain.c                        |  10 +-
 src/qemu/qemu_extdevice.c                     |  79 +++-
 src/qemu/qemu_extdevice.h                     |   5 +
 src/qemu/qemu_firmware.c                      | 146 +------
 src/qemu/qemu_process.c                       |  18 +-
 src/qemu/qemu_security.c                      |  48 +++
 src/qemu/qemu_security.h                      |   6 +
 src/qemu/qemu_tpm.c                           |   2 +-
 src/qemu/qemu_vhost_user.c                    | 395 ++++++++++++++++++
 src/qemu/qemu_vhost_user.h                    |  51 +++
 src/qemu/qemu_vhost_user_gpu.c                | 307 ++++++++++++++
 src/qemu/qemu_vhost_user_gpu.h                |  54 +++
 tests/Makefile.am                             |   9 +
 .../etc/qemu/vhost-user/40-gpu.json           |   1 +
 .../etc/qemu/vhost-user/50-gpu.json           |   0
 .../qemu/vhost-user/test-vhost-user-gpu       |  11 +
 .../usr/share/qemu/vhost-user/30-gpu.json     |   1 +
 .../usr/share/qemu/vhost-user/50-gpu.json     |   8 +
 .../usr/share/qemu/vhost-user/60-gpu.json     |   1 +
 tests/qemuvhostusertest.c                     | 132 ++++++
 .../vhost-user-gpu-secondary.args             |  35 ++
 .../vhost-user-gpu-secondary.xml              |  44 ++
 tests/qemuxml2argvdata/vhost-user-vga.args    |  32 ++
 tests/qemuxml2argvdata/vhost-user-vga.xml     |  41 ++
 tests/qemuxml2argvtest.c                      |  21 +
 tests/virfilewrapper.c                        |  22 +
 40 files changed, 1690 insertions(+), 186 deletions(-)
 create mode 100644 src/qemu/qemu_configs.c
 create mode 100644 src/qemu/qemu_configs.h
 create mode 100644 src/qemu/qemu_vhost_user.c
 create mode 100644 src/qemu/qemu_vhost_user.h
 create mode 100644 src/qemu/qemu_vhost_user_gpu.c
 create mode 100644 src/qemu/qemu_vhost_user_gpu.h
 create mode 120000 tests/qemuvhostuserdata/etc/qemu/vhost-user/40-gpu.json
 create mode 100644 tests/qemuvhostuserdata/etc/qemu/vhost-user/50-gpu.json
 create mode 100755 tests/qemuvhostuserdata/usr/libexec/qemu/vhost-user/test-vhost-user-gpu
 create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/30-gpu.json
 create mode 100644 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/50-gpu.json
 create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/60-gpu.json
 create mode 100644 tests/qemuvhostusertest.c
 create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.args
 create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.xml
 create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.args
 create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.xml

-- 
2.22.0.rc2.384.g1a9a72ea1d

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Hi!

ping

On Wed, Jun 5, 2019 at 2:32 PM <marcandre.lureau@redhat.com> wrote:
>
> From: Marc-André Lureau <marcandre.lureau@redhat.com>
>
> Hi,
>
> This series of patches adds support for running virtio GPUs in
> seperate processes, thanks to vhost-user backend.
>
> The QEMU support landed for 4.1. There are several benefits of running
> the GPU/virgl in an external process, since Mesa is rather heavy on
> the qemu main loop, and may block for a while, or crash.
>
> The external GPU process is started with one end of a socket pair, the
> other end is given to a QEMU chardev attached to a device. The
> external process is also added to the cgroup to limit resources usage.
>
> Thanks
>
> Since RFC:
> - discover helpers following the vhost-user spec
> - change vhost-user <video> model for a vhostuser attribute
> - add a patch to specify the rendernode on <accel>
> - change the way command line is built, following qemu series changes
> - socket labeling
> - a few cleanup patches
> - rebased
>
> Marc-André Lureau (19):
>   device-conf: removed unneeded virDomainDeviceInfoCopy()
>   tpm: minor argument comment fix
>   qemu_firmware: only set nfeatures on success
>   qemu: extract out qemuFetchConfigs from firmware
>   domain: add "vhostuser" attribute to virtio video model
>   domain: add rendernode attribute on <accel>
>   qemu-cgroup: allow accel rendernode access
>   qemu: add vhost-user-gpu capabilities checks
>   qemu: check that qemu is vhost-user-vga capable
>   qemu: validate virtio-gpu with vhost-user
>   qemu: restrict 'virgl=' option to non-vhostuser video type
>   qemu: add vhost-user helpers
>   qemu: add qemuSecurityStartVhostUserGPU helper
>   qemu: add vhost-user-gpu helper unit
>   qemu: prepare domain for vhost-user GPU
>   qemu: start/stop the vhost-user-gpu external device
>   qemu: build vhost-user GPU devices
>   tests: mock execv/execve
>   tests: add vhost-user-gpu xml2argv tests
>
>  docs/formatdomain.html.in                     |  11 +
>  docs/schemas/domaincommon.rng                 |  16 +-
>  src/conf/device_conf.c                        |  22 +-
>  src/conf/device_conf.h                        |   4 +-
>  src/conf/domain_conf.c                        |  33 +-
>  src/conf/domain_conf.h                        |   2 +
>  src/libvirt_private.syms                      |   1 -
>  src/qemu/Makefile.inc.am                      |   6 +
>  src/qemu/qemu_capabilities.c                  |   6 +
>  src/qemu/qemu_capabilities.h                  |   4 +
>  src/qemu/qemu_cgroup.c                        |  24 ++
>  src/qemu/qemu_command.c                       |  54 ++-
>  src/qemu/qemu_configs.c                       | 183 ++++++++
>  src/qemu/qemu_configs.h                       |  31 ++
>  src/qemu/qemu_domain.c                        |  10 +-
>  src/qemu/qemu_extdevice.c                     |  79 +++-
>  src/qemu/qemu_extdevice.h                     |   5 +
>  src/qemu/qemu_firmware.c                      | 146 +------
>  src/qemu/qemu_process.c                       |  18 +-
>  src/qemu/qemu_security.c                      |  48 +++
>  src/qemu/qemu_security.h                      |   6 +
>  src/qemu/qemu_tpm.c                           |   2 +-
>  src/qemu/qemu_vhost_user.c                    | 395 ++++++++++++++++++
>  src/qemu/qemu_vhost_user.h                    |  51 +++
>  src/qemu/qemu_vhost_user_gpu.c                | 307 ++++++++++++++
>  src/qemu/qemu_vhost_user_gpu.h                |  54 +++
>  tests/Makefile.am                             |   9 +
>  .../etc/qemu/vhost-user/40-gpu.json           |   1 +
>  .../etc/qemu/vhost-user/50-gpu.json           |   0
>  .../qemu/vhost-user/test-vhost-user-gpu       |  11 +
>  .../usr/share/qemu/vhost-user/30-gpu.json     |   1 +
>  .../usr/share/qemu/vhost-user/50-gpu.json     |   8 +
>  .../usr/share/qemu/vhost-user/60-gpu.json     |   1 +
>  tests/qemuvhostusertest.c                     | 132 ++++++
>  .../vhost-user-gpu-secondary.args             |  35 ++
>  .../vhost-user-gpu-secondary.xml              |  44 ++
>  tests/qemuxml2argvdata/vhost-user-vga.args    |  32 ++
>  tests/qemuxml2argvdata/vhost-user-vga.xml     |  41 ++
>  tests/qemuxml2argvtest.c                      |  21 +
>  tests/virfilewrapper.c                        |  22 +
>  40 files changed, 1690 insertions(+), 186 deletions(-)
>  create mode 100644 src/qemu/qemu_configs.c
>  create mode 100644 src/qemu/qemu_configs.h
>  create mode 100644 src/qemu/qemu_vhost_user.c
>  create mode 100644 src/qemu/qemu_vhost_user.h
>  create mode 100644 src/qemu/qemu_vhost_user_gpu.c
>  create mode 100644 src/qemu/qemu_vhost_user_gpu.h
>  create mode 120000 tests/qemuvhostuserdata/etc/qemu/vhost-user/40-gpu.json
>  create mode 100644 tests/qemuvhostuserdata/etc/qemu/vhost-user/50-gpu.json
>  create mode 100755 tests/qemuvhostuserdata/usr/libexec/qemu/vhost-user/test-vhost-user-gpu
>  create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/30-gpu.json
>  create mode 100644 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/50-gpu.json
>  create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/60-gpu.json
>  create mode 100644 tests/qemuvhostusertest.c
>  create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.args
>  create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.xml
>  create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.args
>  create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.xml
>
> --
> 2.22.0.rc2.384.g1a9a72ea1d
>
> --
> libvir-list mailing list
> libvir-list@redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list



-- 
Marc-André Lureau

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Hi

On Wed, Jun 5, 2019 at 2:32 PM <marcandre.lureau@redhat.com> wrote:
>
> From: Marc-André Lureau <marcandre.lureau@redhat.com>
>
> Hi,
>
> This series of patches adds support for running virtio GPUs in
> seperate processes, thanks to vhost-user backend.
>
> The QEMU support landed for 4.1. There are several benefits of running
> the GPU/virgl in an external process, since Mesa is rather heavy on
> the qemu main loop, and may block for a while, or crash.
>
> The external GPU process is started with one end of a socket pair, the
> other end is given to a QEMU chardev attached to a device. The
> external process is also added to the cgroup to limit resources usage.
>
> Thanks
>
> Since RFC:
> - discover helpers following the vhost-user spec
> - change vhost-user <video> model for a vhostuser attribute
> - add a patch to specify the rendernode on <accel>
> - change the way command line is built, following qemu series changes
> - socket labeling
> - a few cleanup patches
> - rebased

Can I get some reviews before I respin the series?

thanks

>
> Marc-André Lureau (19):
>   device-conf: removed unneeded virDomainDeviceInfoCopy()
>   tpm: minor argument comment fix
>   qemu_firmware: only set nfeatures on success
>   qemu: extract out qemuFetchConfigs from firmware
>   domain: add "vhostuser" attribute to virtio video model
>   domain: add rendernode attribute on <accel>
>   qemu-cgroup: allow accel rendernode access
>   qemu: add vhost-user-gpu capabilities checks
>   qemu: check that qemu is vhost-user-vga capable
>   qemu: validate virtio-gpu with vhost-user
>   qemu: restrict 'virgl=' option to non-vhostuser video type
>   qemu: add vhost-user helpers
>   qemu: add qemuSecurityStartVhostUserGPU helper
>   qemu: add vhost-user-gpu helper unit
>   qemu: prepare domain for vhost-user GPU
>   qemu: start/stop the vhost-user-gpu external device
>   qemu: build vhost-user GPU devices
>   tests: mock execv/execve
>   tests: add vhost-user-gpu xml2argv tests
>
>  docs/formatdomain.html.in                     |  11 +
>  docs/schemas/domaincommon.rng                 |  16 +-
>  src/conf/device_conf.c                        |  22 +-
>  src/conf/device_conf.h                        |   4 +-
>  src/conf/domain_conf.c                        |  33 +-
>  src/conf/domain_conf.h                        |   2 +
>  src/libvirt_private.syms                      |   1 -
>  src/qemu/Makefile.inc.am                      |   6 +
>  src/qemu/qemu_capabilities.c                  |   6 +
>  src/qemu/qemu_capabilities.h                  |   4 +
>  src/qemu/qemu_cgroup.c                        |  24 ++
>  src/qemu/qemu_command.c                       |  54 ++-
>  src/qemu/qemu_configs.c                       | 183 ++++++++
>  src/qemu/qemu_configs.h                       |  31 ++
>  src/qemu/qemu_domain.c                        |  10 +-
>  src/qemu/qemu_extdevice.c                     |  79 +++-
>  src/qemu/qemu_extdevice.h                     |   5 +
>  src/qemu/qemu_firmware.c                      | 146 +------
>  src/qemu/qemu_process.c                       |  18 +-
>  src/qemu/qemu_security.c                      |  48 +++
>  src/qemu/qemu_security.h                      |   6 +
>  src/qemu/qemu_tpm.c                           |   2 +-
>  src/qemu/qemu_vhost_user.c                    | 395 ++++++++++++++++++
>  src/qemu/qemu_vhost_user.h                    |  51 +++
>  src/qemu/qemu_vhost_user_gpu.c                | 307 ++++++++++++++
>  src/qemu/qemu_vhost_user_gpu.h                |  54 +++
>  tests/Makefile.am                             |   9 +
>  .../etc/qemu/vhost-user/40-gpu.json           |   1 +
>  .../etc/qemu/vhost-user/50-gpu.json           |   0
>  .../qemu/vhost-user/test-vhost-user-gpu       |  11 +
>  .../usr/share/qemu/vhost-user/30-gpu.json     |   1 +
>  .../usr/share/qemu/vhost-user/50-gpu.json     |   8 +
>  .../usr/share/qemu/vhost-user/60-gpu.json     |   1 +
>  tests/qemuvhostusertest.c                     | 132 ++++++
>  .../vhost-user-gpu-secondary.args             |  35 ++
>  .../vhost-user-gpu-secondary.xml              |  44 ++
>  tests/qemuxml2argvdata/vhost-user-vga.args    |  32 ++
>  tests/qemuxml2argvdata/vhost-user-vga.xml     |  41 ++
>  tests/qemuxml2argvtest.c                      |  21 +
>  tests/virfilewrapper.c                        |  22 +
>  40 files changed, 1690 insertions(+), 186 deletions(-)
>  create mode 100644 src/qemu/qemu_configs.c
>  create mode 100644 src/qemu/qemu_configs.h
>  create mode 100644 src/qemu/qemu_vhost_user.c
>  create mode 100644 src/qemu/qemu_vhost_user.h
>  create mode 100644 src/qemu/qemu_vhost_user_gpu.c
>  create mode 100644 src/qemu/qemu_vhost_user_gpu.h
>  create mode 120000 tests/qemuvhostuserdata/etc/qemu/vhost-user/40-gpu.json
>  create mode 100644 tests/qemuvhostuserdata/etc/qemu/vhost-user/50-gpu.json
>  create mode 100755 tests/qemuvhostuserdata/usr/libexec/qemu/vhost-user/test-vhost-user-gpu
>  create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/30-gpu.json
>  create mode 100644 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/50-gpu.json
>  create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/60-gpu.json
>  create mode 100644 tests/qemuvhostusertest.c
>  create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.args
>  create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.xml
>  create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.args
>  create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.xml
>
> --
> 2.22.0.rc2.384.g1a9a72ea1d
>
> --
> libvir-list mailing list
> libvir-list@redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list



-- 
Marc-André Lureau

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On 8/1/19 5:28 AM, Marc-André Lureau wrote:
> Hi
> 
> On Wed, Jun 5, 2019 at 2:32 PM <marcandre.lureau@redhat.com> wrote:
>>
>> From: Marc-André Lureau <marcandre.lureau@redhat.com>
>>
>> Hi,
>>
>> This series of patches adds support for running virtio GPUs in
>> seperate processes, thanks to vhost-user backend.
>>
>> The QEMU support landed for 4.1. There are several benefits of running
>> the GPU/virgl in an external process, since Mesa is rather heavy on
>> the qemu main loop, and may block for a while, or crash.
>>
>> The external GPU process is started with one end of a socket pair, the
>> other end is given to a QEMU chardev attached to a device. The
>> external process is also added to the cgroup to limit resources usage.
>>
>> Thanks
>>
>> Since RFC:
>> - discover helpers following the vhost-user spec
>> - change vhost-user <video> model for a vhostuser attribute
>> - add a patch to specify the rendernode on <accel>
>> - change the way command line is built, following qemu series changes
>> - socket labeling
>> - a few cleanup patches
>> - rebased
> 
> Can I get some reviews before I respin the series?

I rebased the series and fixed all the 'make syntax-check' errors,
patches are here:

https://github.com/crobinso/libvirt/tree/vhost-user-gpu

I'll do a review tomorrow

Thanks,
Cole

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On Thu, Aug 22, 2019 at 07:50:14PM -0400, Cole Robinson wrote:
>On 8/1/19 5:28 AM, Marc-André Lureau wrote:
>> Hi
>>
>> On Wed, Jun 5, 2019 at 2:32 PM <marcandre.lureau@redhat.com> wrote:
>>>
>>> From: Marc-André Lureau <marcandre.lureau@redhat.com>
>>>
>>> Hi,
>>>
>>> This series of patches adds support for running virtio GPUs in
>>> seperate processes, thanks to vhost-user backend.
>>>
>>> The QEMU support landed for 4.1. There are several benefits of running
>>> the GPU/virgl in an external process, since Mesa is rather heavy on
>>> the qemu main loop, and may block for a while, or crash.
>>>
>>> The external GPU process is started with one end of a socket pair, the
>>> other end is given to a QEMU chardev attached to a device. The
>>> external process is also added to the cgroup to limit resources usage.
>>>
>>> Thanks
>>>
>>> Since RFC:
>>> - discover helpers following the vhost-user spec
>>> - change vhost-user <video> model for a vhostuser attribute
>>> - add a patch to specify the rendernode on <accel>
>>> - change the way command line is built, following qemu series changes
>>> - socket labeling
>>> - a few cleanup patches
>>> - rebased
>>
>> Can I get some reviews before I respin the series?
>
>I rebased the series and fixed all the 'make syntax-check' errors,
>patches are here:
>
>https://github.com/crobinso/libvirt/tree/vhost-user-gpu
>
>I'll do a review tomorrow
>

If you plan to review your modified version, please send it to the list
first

Jano
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list