[PATCH v8 00/11] Support blob memory and venus on qemu

Dmitry Osipenko posted 11 patches 2 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20240418190040.1110210-1-dmitry.osipenko@collabora.com
Maintainers: "Michael S. Tsirkin" <mst@redhat.com>, Gerd Hoffmann <kraxel@redhat.com>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>, Richard Henderson <richard.henderson@linaro.org>, Eduardo Habkost <eduardo@habkost.net>, Cornelia Huck <cohuck@redhat.com>, "Marc-André Lureau" <marcandre.lureau@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, Thomas Huth <thuth@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>
There is a newer version of this series
hw/display/trace-events                       |   1 +
hw/display/virtio-gpu-base.c                  |   1 +
hw/display/virtio-gpu-gl.c                    |   5 +
hw/display/virtio-gpu-rutabaga.c              |   1 +
hw/display/virtio-gpu-virgl.c                 | 519 ++++++++++++-
hw/display/virtio-gpu.c                       |  42 +-
hw/i386/x86.c                                 |   8 -
include/hw/virtio/virtio-gpu.h                |  21 +
include/standard-headers/asm-x86/bootparam.h  |  17 +-
include/standard-headers/asm-x86/kvm_para.h   |   3 +-
include/standard-headers/asm-x86/setup_data.h |  83 +++
include/standard-headers/linux/ethtool.h      |  48 ++
include/standard-headers/linux/fuse.h         |  39 +-
.../linux/input-event-codes.h                 |   1 +
include/standard-headers/linux/virtio_gpu.h   |   2 +
include/standard-headers/linux/virtio_pci.h   |  10 +-
include/standard-headers/linux/virtio_snd.h   | 154 ++++
linux-headers/asm-arm64/kvm.h                 |  15 +-
linux-headers/asm-arm64/sve_context.h         |  11 +
linux-headers/asm-generic/bitsperlong.h       |   4 +
linux-headers/asm-loongarch/kvm.h             |   2 -
linux-headers/asm-mips/kvm.h                  |   2 -
linux-headers/asm-powerpc/kvm.h               |  45 +-
linux-headers/asm-riscv/kvm.h                 |   3 +-
linux-headers/asm-s390/kvm.h                  | 315 +++++++-
linux-headers/asm-x86/kvm.h                   | 308 +++++++-
linux-headers/linux/bits.h                    |  15 +
linux-headers/linux/kvm.h                     | 689 +-----------------
linux-headers/linux/psp-sev.h                 |  59 ++
linux-headers/linux/vhost.h                   |   7 +
meson.build                                   |  10 +-
scripts/update-linux-headers.sh               |   5 +-
32 files changed, 1679 insertions(+), 766 deletions(-)
create mode 100644 include/standard-headers/asm-x86/setup_data.h
create mode 100644 linux-headers/linux/bits.h
[PATCH v8 00/11] Support blob memory and venus on qemu
Posted by Dmitry Osipenko 2 weeks ago
Hello,

This series enables Vulkan Venus context support on virtio-gpu.

All virglrender and almost all Linux kernel prerequisite changes
needed by Venus are already in upstream. For kernel there is a pending
KVM patchset that fixes mapping of compound pages needed for DRM drivers
using TTM [1], othewrwise hostmem blob mapping will fail with a KVM error
from Qemu.

[1] https://lore.kernel.org/kvm/20240229025759.1187910-1-stevensd@google.com/

Example Qemu cmdline that enables Venus for virtio-gpu:

  qemu-system-x86_64 -device virtio-vga-gl,hostmem=4G,blob=true,vulkan=true


Changes from V7 to V8

- Supported suspension of virtio-gpu commands processing and made
  unmapping of hostmem region asynchronous by blocking/suspending
  cmd processing until region is unmapped. Suggested by Akihiko Odaki.

- Fixed arm64 building of x86 targets using updated linux-headers.
  Corrected the update script. Thanks to Rob Clark for reporting
  the issue.

- Added new patch that makes registration of virgl capsets dynamic.
  Requested by Antonio Caggiano and Pierre-Eric Pelloux-Prayer.

- Venus capset now isn't advertised if Vulkan is disabled with vulkan=false

Changes from V6 to V7

- Used scripts/update-linux-headers.sh to update Qemu headers based
  on Linux v6.8-rc3 that adds Venus capset definition to virtio-gpu
  protocol, was requested by Peter Maydel

- Added r-bs that were given to v6 patches. Corrected missing s-o-bs

- Dropped context_init Qemu's virtio-gpu device configuration flag,
  was suggested by Marc-André Lureau

- Added missing error condition checks spotted by Marc-André Lureau
  and Akihiko Odaki, and few more

- Returned back res->mr referencing to memory_region_init_ram_ptr() like
  was suggested by Akihiko Odaki. Incorporated fix suggested by Pierre-Eric
  to specify the MR name

- Dropped the virgl_gpu_resource wrapper, cleaned up and simplified
  patch that adds blob-cmd support

- Fixed improper blob resource removal from resource list on resource_unref
  that was spotted by Akihiko Odaki

- Change order of the blob patches, was suggested by Akihiko Odaki.
  The cmd_set_scanout_blob support is enabled first

- Factored out patch that adds resource management support to virtio-gpu-gl,
  was requested by Marc-André Lureau

- Simplified and improved the UUID support patch, dropped the hash table
  as we don't need it for now. Moved QemuUUID to virtio_gpu_simple_resource.
  This all was suggested by Akihiko Odaki and Marc-André Lureau

- Dropped console_has_gl() check, suggested by Akihiko Odaki

- Reworked Meson cheking of libvirglrender features, made new features
  available based on virglrender pkgconfig version instead of checking
  symbols in header. This should fix build error using older virglrender
  version, reported by Alex Bennée

- Made enabling of Venus context configrable via new virtio-gpu device
  "vulkan=true" flag, suggested by Marc-André Lureau. The flag is disabled
  by default because it requires blob and hostmem options to be enabled
  and configured

Changes from V5 to V6

- Move macros configurations under virgl.found() and rename
  HAVE_VIRGL_CONTEXT_CREATE_WITH_FLAGS.

- Handle the case while context_init is disabled.

- Enable context_init by default.

- Move virtio_gpu_virgl_resource_unmap() into
  virgl_cmd_resource_unmap_blob().

- Introduce new struct virgl_gpu_resource to store virgl specific members.

- Remove erro handling of g_new0, because glib will abort() on OOM.

- Set resource uuid as option.

- Implement optional subsection of vmstate_virtio_gpu_resource_uuid_state
  for virtio live migration.

- Use g_int_hash/g_int_equal instead of the default

- Add scanout_blob function for virtio-gpu-virgl

- Resolve the memory leak on virtio-gpu-virgl

- Remove the unstable API flags check because virglrenderer is already 1.0

- Squash the render server flag support into "Initialize Venus"

Changes from V4 (virtio gpu V4) to V5

- Inverted patch 5 and 6 because we should configure
  HAVE_VIRGL_CONTEXT_INIT firstly.

- Validate owner of memory region to avoid slowing down DMA.

- Use memory_region_init_ram_ptr() instead of
  memory_region_init_ram_device_ptr().

- Adjust sequence to allocate gpu resource before virglrender resource
  creation

- Add virtio migration handling for uuid.

- Send kernel patch to define VIRTIO_GPU_CAPSET_VENUS.
  https://lore.kernel.org/lkml/20230915105918.3763061-1-ray.huang@amd.com/

- Add meson check to make sure unstable APIs defined from 0.9.0.

Changes from V1 to V2 (virtio gpu V4)

- Remove unused #include "hw/virtio/virtio-iommu.h"

- Add a local function, called virgl_resource_destroy(), that is used
  to release a vgpu resource on error paths and in resource_unref.

- Remove virtio_gpu_virgl_resource_unmap from
  virtio_gpu_cleanup_mapping(),
  since this function won't be called on blob resources and also because
  blob resources are unmapped via virgl_cmd_resource_unmap_blob().

- In virgl_cmd_resource_create_blob(), do proper cleanup in error paths
  and move QTAILQ_INSERT_HEAD(&g->reslist, res, next) after the resource
  has been fully initialized.

- Memory region has a different life-cycle from virtio gpu resources
  i.e. cannot be released synchronously along with the vgpu resource.
  So, here the field "region" was changed to a pointer and is allocated
  dynamically when the blob is mapped.
  Also, since the pointer can be used to indicate whether the blob
  is mapped, the explicite field "mapped" was removed.

- In virgl_cmd_resource_map_blob(), add check on the value of
  res->region, to prevent beeing called twice on the same resource.

- Add a patch to enable automatic deallocation of memory regions to resolve
  use-after-free memory corruption with a reference.

Antonio Caggiano (3):
  virtio-gpu: Handle resource blob commands
  virtio-gpu: Resource UUID
  virtio-gpu: Support Venus context

Dmitry Osipenko (4):
  linux-headers: Update to Linux v6.9-rc3
  virtio-gpu: Use pkgconfig version to decide which virgl features are
    available
  virtio-gpu: Don't require udmabuf when blobs and virgl are enabled
  virtio-gpu: Support suspension of commands processing

Huang Rui (2):
  virtio-gpu: Support context-init feature with virglrenderer
  virtio-gpu: Add virgl resource management

Pierre-Eric Pelloux-Prayer (1):
  virtio-gpu: Register capsets dynamically

Robert Beckett (1):
  virtio-gpu: Support blob scanout using dmabuf fd

 hw/display/trace-events                       |   1 +
 hw/display/virtio-gpu-base.c                  |   1 +
 hw/display/virtio-gpu-gl.c                    |   5 +
 hw/display/virtio-gpu-rutabaga.c              |   1 +
 hw/display/virtio-gpu-virgl.c                 | 519 ++++++++++++-
 hw/display/virtio-gpu.c                       |  42 +-
 hw/i386/x86.c                                 |   8 -
 include/hw/virtio/virtio-gpu.h                |  21 +
 include/standard-headers/asm-x86/bootparam.h  |  17 +-
 include/standard-headers/asm-x86/kvm_para.h   |   3 +-
 include/standard-headers/asm-x86/setup_data.h |  83 +++
 include/standard-headers/linux/ethtool.h      |  48 ++
 include/standard-headers/linux/fuse.h         |  39 +-
 .../linux/input-event-codes.h                 |   1 +
 include/standard-headers/linux/virtio_gpu.h   |   2 +
 include/standard-headers/linux/virtio_pci.h   |  10 +-
 include/standard-headers/linux/virtio_snd.h   | 154 ++++
 linux-headers/asm-arm64/kvm.h                 |  15 +-
 linux-headers/asm-arm64/sve_context.h         |  11 +
 linux-headers/asm-generic/bitsperlong.h       |   4 +
 linux-headers/asm-loongarch/kvm.h             |   2 -
 linux-headers/asm-mips/kvm.h                  |   2 -
 linux-headers/asm-powerpc/kvm.h               |  45 +-
 linux-headers/asm-riscv/kvm.h                 |   3 +-
 linux-headers/asm-s390/kvm.h                  | 315 +++++++-
 linux-headers/asm-x86/kvm.h                   | 308 +++++++-
 linux-headers/linux/bits.h                    |  15 +
 linux-headers/linux/kvm.h                     | 689 +-----------------
 linux-headers/linux/psp-sev.h                 |  59 ++
 linux-headers/linux/vhost.h                   |   7 +
 meson.build                                   |  10 +-
 scripts/update-linux-headers.sh               |   5 +-
 32 files changed, 1679 insertions(+), 766 deletions(-)
 create mode 100644 include/standard-headers/asm-x86/setup_data.h
 create mode 100644 linux-headers/linux/bits.h

-- 
2.44.0


Re: [PATCH v8 00/11] Support blob memory and venus on qemu
Posted by Alex Bennée 1 week, 2 days ago
Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:

> Hello,
>
> This series enables Vulkan Venus context support on virtio-gpu.
>
> All virglrender and almost all Linux kernel prerequisite changes
> needed by Venus are already in upstream. For kernel there is a pending
> KVM patchset that fixes mapping of compound pages needed for DRM drivers
> using TTM [1], othewrwise hostmem blob mapping will fail with a KVM error
> from Qemu.
>
> [1]
> https://lore.kernel.org/kvm/20240229025759.1187910-1-stevensd@google.com/

Following the link for the TTM/KVM patches on the kernel side points at
changes for AMD cards getting NAK'ed so I'm a little confused as to what
parts are needed.

Is this only relevant for ensuring the virtual mappings to the
underlying hardware aren't moved around when KVM is exporting those
pages to the guest?

Our interest is in Xen which obviously mediates everything through stage
2 mappings to from the real PA to the IPA the domains see. However AIUI
all the blob allocation is managed by the GEM/TTM layer of whichever
kernel is responsible for driving the GPU. Does this layer work with
kernel vaddr or the underlying IPA of the resources? We shouldn't
expect the IPA to change between allocations should we?

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro
Re: [PATCH v8 00/11] Support blob memory and venus on qemu
Posted by Dmitry Osipenko 1 week, 2 days ago
On 4/23/24 11:30, Alex Bennée wrote:
> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
> 
>> Hello,
>>
>> This series enables Vulkan Venus context support on virtio-gpu.
>>
>> All virglrender and almost all Linux kernel prerequisite changes
>> needed by Venus are already in upstream. For kernel there is a pending
>> KVM patchset that fixes mapping of compound pages needed for DRM drivers
>> using TTM [1], othewrwise hostmem blob mapping will fail with a KVM error
>> from Qemu.
>>
>> [1]
>> https://lore.kernel.org/kvm/20240229025759.1187910-1-stevensd@google.com/
> 
> Following the link for the TTM/KVM patches on the kernel side points at
> changes for AMD cards getting NAK'ed so I'm a little confused as to what
> parts are needed.

I wouldn't say that patches are NAK'ed, they more having a problem with
getting a review. Without KMV patches host blobs don't work depending on
a host GPU driver and kernel configuration.

It's actually not only TTM drivers that are requiring the KVM changes,
but a non-TTM GPU drivers that use huge pages may also need them too.
You may need a patched KVM for i915 driver that doesn't use TTM,
depending on whether transparent huge pages are enabled in the kernel
config.

> Is this only relevant for ensuring the virtual mappings to the
> underlying hardware aren't moved around when KVM is exporting those
> pages to the guest?

Yes, host GPU driver needs to handle guest access page fault to keep
pages in place.

> Our interest is in Xen which obviously mediates everything through stage
> 2 mappings to from the real PA to the IPA the domains see. However AIUI
> all the blob allocation is managed by the GEM/TTM layer of whichever
> kernel is responsible for driving the GPU. Does this layer work with
> kernel vaddr or the underlying IPA of the resources? We shouldn't
> expect the IPA to change between allocations should we?

TTM works with memory pages and it moves pages around. It may swap out
pages and then relies on a working page faulting notification to swap-in
pages back.

Whether PA stays fixed, I don't know for sure. Robert Beckett or
somebody from AMD should know better how it works for Xen and may
comment on it.
-- 
Best regards,
Dmitry