[PATCH 0/3] virtio-gpu: Enable virglrenderer backend for rutabaga

Weifeng Liu posted 3 patches 5 months, 3 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20240605152832.11618-1-weifeng.liu.z@gmail.com
Maintainers: Gerd Hoffmann <kraxel@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>
hw/display/virtio-gpu-rutabaga.c | 104 ++++++++++++++++++++++++++++++-
include/hw/virtio/virtio-gpu.h   |   1 +
2 files changed, 104 insertions(+), 1 deletion(-)
[PATCH 0/3] virtio-gpu: Enable virglrenderer backend for rutabaga
Posted by Weifeng Liu 5 months, 3 weeks ago
Greetings,

I'd like to introduce you my attempt to enable virglrenderer backend for
rutabaga empowered virtio-gpu device.  I am aware that there have been
effort in supporting venus in virtio-gpu-virgl.c [1], but there is no
reason to prevent us from leveraging the virglrenderer component in
rutabaga_gfx, especially it being not very hard to add this
functionality.

Generally, the gap is the polling capability, i.e., virglrenderer
requires the main thread (namely the GPU command handling thread) to
poll virglrenderer at proper moments, which is not yet supported in
virtio-gpu-rutabaga device. This patch set try to add this so that
virglrenderer backend (including virgl and venus) can work as expected.

Slight change to rutabaga_gfx_ffi is also a requirement, which is
included in [2].

Further effort is required to tune the performance, since copying is
present before the rendered images get displayed. But I still think this
patch set could be a good starting point for the pending work.

For those interested in setting up environment and playing around with
this patch set, here is guideline in brief:

1. Clone the master/main branch of virglrenderer, compile and install it.

	git clone https://gitlab.freedesktop.org/virgl/virglrenderer
	cd virglrenderer
	meson setup builddir \
	  --prefix=$INSTALL_DIR/virglrenderer \
	  -Dvenus=true
	ninja -C builddir install

2. Clone the patched CrosVM, build and install rutabaga_gfx_ffi.

	git clone -b rutabaga_ffi_virgl https://github.com/phreer/crosvm.git
	cd crosvm/rutabaga_gfx/ffi
	export PKG_CONFIG_PATH=$INSTALL_DIR/virglrenderer/lib64/pkgconfig/
	meson setup builddir/ \
	  --prefix $HOME/install/rutabaga_gfx/rutabaga_gfx_ffi/ \
	  -Dvirglrenderer=true
	ninja -C builddir install

3. Applied this patch set to QEMU, build and install it:

	cd qemu	
	# Apply this patch set atop main branch ...
	mkdir builddir; cd builddir
	../configure --prefix=$INSTALL_DIR/qemu \
	  --target-list=x86_64-softmmu \
	  --disable-virglrenderer \
	  --enable-rutabaga_gfx
	ninja -C builddir install

4. If you are lucky and everything goes fine, you are prepared to launch
   VM with virglrenderer backed virtio-gpu-rutabaga device:

	export LD_LIBRARY_PATH=$INSTALL_DIR/virglrenderer/lib64/:$LD_LIBRARY_PATH
	export LD_LIBRARY_PATH=$INSTALL_DIR/rutabaga_gfx_ffi/lib64/:$LD_LIBRARY_PATH
	$INSTALL_DIR/qemu/bin/qemu-system-x86_64
	$QEMU -d guest_errors -enable-kvm -M q35 -smp 4 -m $MEM \
	  -object memory-backend-memfd,id=mem1,size=$MEM \
	  -machine memory-backend=mem1 \
	  -device virtio-vga-rutabaga,venus=on,virgl2=on,wsi=surfaceless,hostmem=$MEM \

Note:

- You might need this patch set [3] to avoid KVM bad address error when
  you are running on a GPU using TTM for memory management.

[1] https://lore.kernel.org/all/dba6eb97-e1d1-4694-bfb6-e72db95715dd@daynix.com/T/
[2] https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5599645/1
[3] https://lore.kernel.org/kvm/20240229025759.1187910-1-stevensd@google.com/

Weifeng Liu (3):
  virtio-gpu: rutabaga: Properly set stride when copying resources
  virtio-gpu: rutabaga: Poll rutabaga upon events
  virtio-gpu: rutabaga: Add options to enable virgl and venus contexts

 hw/display/virtio-gpu-rutabaga.c | 104 ++++++++++++++++++++++++++++++-
 include/hw/virtio/virtio-gpu.h   |   1 +
 2 files changed, 104 insertions(+), 1 deletion(-)

-- 
2.45.0
Re: [PATCH 0/3] virtio-gpu: Enable virglrenderer backend for rutabaga
Posted by Alex Bennée 5 months, 3 weeks ago
Weifeng Liu <weifeng.liu.z@gmail.com> writes:

> Greetings,
>
> I'd like to introduce you my attempt to enable virglrenderer backend for
> rutabaga empowered virtio-gpu device.  I am aware that there have been
> effort in supporting venus in virtio-gpu-virgl.c [1], but there is no
> reason to prevent us from leveraging the virglrenderer component in
> rutabaga_gfx, especially it being not very hard to add this
> functionality.
>
> Generally, the gap is the polling capability, i.e., virglrenderer
> requires the main thread (namely the GPU command handling thread) to
> poll virglrenderer at proper moments, which is not yet supported in
> virtio-gpu-rutabaga device. This patch set try to add this so that
> virglrenderer backend (including virgl and venus) can work as expected.
>
> Slight change to rutabaga_gfx_ffi is also a requirement, which is
> included in [2].
>
> Further effort is required to tune the performance, since copying is
> present before the rendered images get displayed. But I still think this
> patch set could be a good starting point for the pending work.
>
> For those interested in setting up environment and playing around with
> this patch set, here is guideline in brief:
>
> 1. Clone the master/main branch of virglrenderer, compile and install it.
>
> 	git clone https://gitlab.freedesktop.org/virgl/virglrenderer
> 	cd virglrenderer
> 	meson setup builddir \
> 	  --prefix=$INSTALL_DIR/virglrenderer \
> 	  -Dvenus=true
> 	ninja -C builddir install
>
> 2. Clone the patched CrosVM, build and install rutabaga_gfx_ffi.
>
> 	git clone -b rutabaga_ffi_virgl https://github.com/phreer/crosvm.git
> 	cd crosvm/rutabaga_gfx/ffi
> 	export PKG_CONFIG_PATH=$INSTALL_DIR/virglrenderer/lib64/pkgconfig/
> 	meson setup builddir/ \
> 	  --prefix $HOME/install/rutabaga_gfx/rutabaga_gfx_ffi/ \
> 	  -Dvirglrenderer=true
> 	ninja -C builddir install

Is there a PR going in for this? The moving parts for rutabaga are
complex enough I think we need support upstream before merging this.

Is this branch where I should be getting the poll helpers from?

  cc -m64 @qemu-system-arm.rsp
  /usr/bin/ld: libcommon.fa.p/hw_display_virtio-gpu-rutabaga.c.o: in function `virtio_gpu_fence_poll':
  /home/alex/lsrc/qemu.git/builds/vulkan/../../hw/display/virtio-gpu-rutabaga.c:909: undefined reference to `rutabaga_poll'
  /usr/bin/ld: libcommon.fa.p/hw_display_virtio-gpu-rutabaga.c.o: in function `virtio_gpu_rutabaga_init':
  /home/alex/lsrc/qemu.git/builds/vulkan/../../hw/display/virtio-gpu-rutabaga.c:1122: undefined reference to `rutabaga_poll_descriptor'
  collect2: error: ld returned 1 exit status
  ninja: build stopped: subcommand failed.


> 3. Applied this patch set to QEMU, build and install it:
>
> 	cd qemu	
> 	# Apply this patch set atop main branch ...
> 	mkdir builddir; cd builddir
> 	../configure --prefix=$INSTALL_DIR/qemu \
> 	  --target-list=x86_64-softmmu \
> 	  --disable-virglrenderer \
> 	  --enable-rutabaga_gfx
> 	ninja -C builddir install
>
> 4. If you are lucky and everything goes fine, you are prepared to launch
>    VM with virglrenderer backed virtio-gpu-rutabaga device:
>
> 	export LD_LIBRARY_PATH=$INSTALL_DIR/virglrenderer/lib64/:$LD_LIBRARY_PATH
> 	export LD_LIBRARY_PATH=$INSTALL_DIR/rutabaga_gfx_ffi/lib64/:$LD_LIBRARY_PATH
> 	$INSTALL_DIR/qemu/bin/qemu-system-x86_64
> 	$QEMU -d guest_errors -enable-kvm -M q35 -smp 4 -m $MEM \
> 	  -object memory-backend-memfd,id=mem1,size=$MEM \
> 	  -machine memory-backend=mem1 \
> 	  -device virtio-vga-rutabaga,venus=on,virgl2=on,wsi=surfaceless,hostmem=$MEM \
>

This should go into docs/system/devices/virtio-gpu.rst with some
explanation. Is there anything we need on the guest side or does this
skip the encapsulating requirements of wayland?

> Note:
>
> - You might need this patch set [3] to avoid KVM bad address error when
>   you are running on a GPU using TTM for memory management.
>
> [1] https://lore.kernel.org/all/dba6eb97-e1d1-4694-bfb6-e72db95715dd@daynix.com/T/
> [2] https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5599645/1
> [3] https://lore.kernel.org/kvm/20240229025759.1187910-1-stevensd@google.com/
>
> Weifeng Liu (3):
>   virtio-gpu: rutabaga: Properly set stride when copying resources
>   virtio-gpu: rutabaga: Poll rutabaga upon events
>   virtio-gpu: rutabaga: Add options to enable virgl and venus contexts
>
>  hw/display/virtio-gpu-rutabaga.c | 104 ++++++++++++++++++++++++++++++-
>  include/hw/virtio/virtio-gpu.h   |   1 +
>  2 files changed, 104 insertions(+), 1 deletion(-)

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro
Re: [PATCH 0/3] virtio-gpu: Enable virglrenderer backend for rutabaga
Posted by Weifeng Liu 5 months, 2 weeks ago
Hi Alex,

On Thu, 2024-06-06 at 11:43 +0100, Alex Bennée wrote:
> Weifeng Liu <weifeng.liu.z@gmail.com> writes:
> 
> > Greetings,
> > 
> > I'd like to introduce you my attempt to enable virglrenderer backend for
> > rutabaga empowered virtio-gpu device.  I am aware that there have been
> > effort in supporting venus in virtio-gpu-virgl.c [1], but there is no
> > reason to prevent us from leveraging the virglrenderer component in
> > rutabaga_gfx, especially it being not very hard to add this
> > functionality.
> > 
> > Generally, the gap is the polling capability, i.e., virglrenderer
> > requires the main thread (namely the GPU command handling thread) to
> > poll virglrenderer at proper moments, which is not yet supported in
> > virtio-gpu-rutabaga device. This patch set try to add this so that
> > virglrenderer backend (including virgl and venus) can work as expected.
> > 
> > Slight change to rutabaga_gfx_ffi is also a requirement, which is
> > included in [2].
> > 
> > Further effort is required to tune the performance, since copying is
> > present before the rendered images get displayed. But I still think this
> > patch set could be a good starting point for the pending work.
> > 
> > For those interested in setting up environment and playing around with
> > this patch set, here is guideline in brief:
> > 
> > 1. Clone the master/main branch of virglrenderer, compile and install it.
> > 
> > 	git clone https://gitlab.freedesktop.org/virgl/virglrenderer
> > 	cd virglrenderer
> > 	meson setup builddir \
> > 	  --prefix=$INSTALL_DIR/virglrenderer \
> > 	  -Dvenus=true
> > 	ninja -C builddir install
> > 
> > 2. Clone the patched CrosVM, build and install rutabaga_gfx_ffi.
> > 
> > 	git clone -b rutabaga_ffi_virgl https://github.com/phreer/crosvm.git
> > 	cd crosvm/rutabaga_gfx/ffi
> > 	export PKG_CONFIG_PATH=$INSTALL_DIR/virglrenderer/lib64/pkgconfig/
> > 	meson setup builddir/ \
> > 	  --prefix $HOME/install/rutabaga_gfx/rutabaga_gfx_ffi/ \
> > 	  -Dvirglrenderer=true
> > 	ninja -C builddir install
> 
> Is there a PR going in for this? The moving parts for rutabaga are
> complex enough I think we need support upstream before merging this.
> 

It's true that this patch set depends on the change of
rutabaga_gfx_ffi. I am trying get the modifications of
crosvm/rubataga_gfx_ffi merged in upstream, please refer to this link:
https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5599645

> Is this branch where I should be getting the poll helpers from?
> 
>   cc -m64 @qemu-system-arm.rsp
>   /usr/bin/ld: libcommon.fa.p/hw_display_virtio-gpu-rutabaga.c.o: in function `virtio_gpu_fence_poll':
>   /home/alex/lsrc/qemu.git/builds/vulkan/../../hw/display/virtio-gpu-rutabaga.c:909: undefined reference to `rutabaga_poll'
>   /usr/bin/ld: libcommon.fa.p/hw_display_virtio-gpu-rutabaga.c.o: in function `virtio_gpu_rutabaga_init':
>   /home/alex/lsrc/qemu.git/builds/vulkan/../../hw/display/virtio-gpu-rutabaga.c:1122: undefined reference to `rutabaga_poll_descriptor'
>   collect2: error: ld returned 1 exit status
>   ninja: build stopped: subcommand failed.
> 

The required patches are applied to the rutabaga_ffi_virgl branch of my
clone of crosvm already, so please check out to that branch.

> 
> > 3. Applied this patch set to QEMU, build and install it:
> > 
> > 	cd qemu	
> > 	# Apply this patch set atop main branch ...
> > 	mkdir builddir; cd builddir
> > 	../configure --prefix=$INSTALL_DIR/qemu \
> > 	  --target-list=x86_64-softmmu \
> > 	  --disable-virglrenderer \
> > 	  --enable-rutabaga_gfx
> > 	ninja -C builddir install
> > 
> > 4. If you are lucky and everything goes fine, you are prepared to launch
> >    VM with virglrenderer backed virtio-gpu-rutabaga device:
> > 
> > 	export LD_LIBRARY_PATH=$INSTALL_DIR/virglrenderer/lib64/:$LD_LIBRARY_PATH
> > 	export LD_LIBRARY_PATH=$INSTALL_DIR/rutabaga_gfx_ffi/lib64/:$LD_LIBRARY_PATH
> > 	$INSTALL_DIR/qemu/bin/qemu-system-x86_64
> > 	$QEMU -d guest_errors -enable-kvm -M q35 -smp 4 -m $MEM \
> > 	  -object memory-backend-memfd,id=mem1,size=$MEM \
> > 	  -machine memory-backend=mem1 \
> > 	  -device virtio-vga-rutabaga,venus=on,virgl2=on,wsi=surfaceless,hostmem=$MEM \
> > 
> 
> This should go into docs/system/devices/virtio-gpu.rst with some
> explanation. Is there anything we need on the guest side or does this
> skip the encapsulating requirements of wayland?
> 

Yeah, it's a good idea to add doc to explain the usage, thanks!

Best regards,
Weifeng

> > Note:
> > 
> > - You might need this patch set [3] to avoid KVM bad address error when
> >   you are running on a GPU using TTM for memory management.
> > 
> > [1] https://lore.kernel.org/all/dba6eb97-e1d1-4694-bfb6-e72db95715dd@daynix.com/T/
> > [2] https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5599645/1
> > [3] https://lore.kernel.org/kvm/20240229025759.1187910-1-stevensd@google.com/
> > 
> > Weifeng Liu (3):
> >   virtio-gpu: rutabaga: Properly set stride when copying resources
> >   virtio-gpu: rutabaga: Poll rutabaga upon events
> >   virtio-gpu: rutabaga: Add options to enable virgl and venus contexts
> > 
> >  hw/display/virtio-gpu-rutabaga.c | 104 ++++++++++++++++++++++++++++++-
> >  include/hw/virtio/virtio-gpu.h   |   1 +
> >  2 files changed, 104 insertions(+), 1 deletion(-)
> 
Re: [PATCH 0/3] virtio-gpu: Enable virglrenderer backend for rutabaga
Posted by Alex Bennée 5 months, 2 weeks ago
Weifeng Liu <weifeng.liu.z@gmail.com> writes:

> Hi Alex,
>
> On Thu, 2024-06-06 at 11:43 +0100, Alex Bennée wrote:
>> Weifeng Liu <weifeng.liu.z@gmail.com> writes:
>> 
>> > Greetings,
>> > 
>> > I'd like to introduce you my attempt to enable virglrenderer backend for
>> > rutabaga empowered virtio-gpu device.  I am aware that there have been
>> > effort in supporting venus in virtio-gpu-virgl.c [1], but there is no
>> > reason to prevent us from leveraging the virglrenderer component in
>> > rutabaga_gfx, especially it being not very hard to add this
>> > functionality.
>> > 
>> > Generally, the gap is the polling capability, i.e., virglrenderer
>> > requires the main thread (namely the GPU command handling thread) to
>> > poll virglrenderer at proper moments, which is not yet supported in
>> > virtio-gpu-rutabaga device. This patch set try to add this so that
>> > virglrenderer backend (including virgl and venus) can work as expected.
>> > 
>> > Slight change to rutabaga_gfx_ffi is also a requirement, which is
>> > included in [2].
>> > 
>> > Further effort is required to tune the performance, since copying is
>> > present before the rendered images get displayed. But I still think this
>> > patch set could be a good starting point for the pending work.
>> > 
>> > For those interested in setting up environment and playing around with
>> > this patch set, here is guideline in brief:
>> > 
>> > 1. Clone the master/main branch of virglrenderer, compile and install it.
>> > 
>> > 	git clone https://gitlab.freedesktop.org/virgl/virglrenderer
>> > 	cd virglrenderer
>> > 	meson setup builddir \
>> > 	  --prefix=$INSTALL_DIR/virglrenderer \
>> > 	  -Dvenus=true
>> > 	ninja -C builddir install
>> > 
>> > 2. Clone the patched CrosVM, build and install rutabaga_gfx_ffi.
>> > 
>> > 	git clone -b rutabaga_ffi_virgl https://github.com/phreer/crosvm.git
>> > 	cd crosvm/rutabaga_gfx/ffi
>> > 	export PKG_CONFIG_PATH=$INSTALL_DIR/virglrenderer/lib64/pkgconfig/
>> > 	meson setup builddir/ \
>> > 	  --prefix $HOME/install/rutabaga_gfx/rutabaga_gfx_ffi/ \
>> > 	  -Dvirglrenderer=true
>> > 	ninja -C builddir install
>> 
>> Is there a PR going in for this? The moving parts for rutabaga are
>> complex enough I think we need support upstream before merging this.
>> 
>
> It's true that this patch set depends on the change of
> rutabaga_gfx_ffi. I am trying get the modifications of
> crosvm/rubataga_gfx_ffi merged in upstream, please refer to this link:
> https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5599645
>
>> Is this branch where I should be getting the poll helpers from?
>> 
>>   cc -m64 @qemu-system-arm.rsp
>>   /usr/bin/ld: libcommon.fa.p/hw_display_virtio-gpu-rutabaga.c.o: in function `virtio_gpu_fence_poll':
>>   /home/alex/lsrc/qemu.git/builds/vulkan/../../hw/display/virtio-gpu-rutabaga.c:909: undefined reference to `rutabaga_poll'
>>   /usr/bin/ld: libcommon.fa.p/hw_display_virtio-gpu-rutabaga.c.o: in function `virtio_gpu_rutabaga_init':
>>   /home/alex/lsrc/qemu.git/builds/vulkan/../../hw/display/virtio-gpu-rutabaga.c:1122: undefined reference to `rutabaga_poll_descriptor'
>>   collect2: error: ld returned 1 exit status
>>   ninja: build stopped: subcommand failed.
>> 
>
> The required patches are applied to the rutabaga_ffi_virgl branch of my
> clone of crosvm already, so please check out to that branch.

I thought I had - maybe my install got polluted. I shall try again.

>
>> 
>> > 3. Applied this patch set to QEMU, build and install it:
>> > 
>> > 	cd qemu	
>> > 	# Apply this patch set atop main branch ...
>> > 	mkdir builddir; cd builddir
>> > 	../configure --prefix=$INSTALL_DIR/qemu \
>> > 	  --target-list=x86_64-softmmu \
>> > 	  --disable-virglrenderer \
>> > 	  --enable-rutabaga_gfx
>> > 	ninja -C builddir install
>> > 
>> > 4. If you are lucky and everything goes fine, you are prepared to launch
>> >    VM with virglrenderer backed virtio-gpu-rutabaga device:
>> > 
>> > 	export LD_LIBRARY_PATH=$INSTALL_DIR/virglrenderer/lib64/:$LD_LIBRARY_PATH
>> > 	export LD_LIBRARY_PATH=$INSTALL_DIR/rutabaga_gfx_ffi/lib64/:$LD_LIBRARY_PATH
>> > 	$INSTALL_DIR/qemu/bin/qemu-system-x86_64
>> > 	$QEMU -d guest_errors -enable-kvm -M q35 -smp 4 -m $MEM \
>> > 	  -object memory-backend-memfd,id=mem1,size=$MEM \
>> > 	  -machine memory-backend=mem1 \
>> > 	  -device virtio-vga-rutabaga,venus=on,virgl2=on,wsi=surfaceless,hostmem=$MEM \
>> > 
>> 
>> This should go into docs/system/devices/virtio-gpu.rst with some
>> explanation. Is there anything we need on the guest side or does this
>> skip the encapsulating requirements of wayland?
>> 
>
> Yeah, it's a good idea to add doc to explain the usage, thanks!
>
> Best regards,
> Weifeng
>
>> > Note:
>> > 
>> > - You might need this patch set [3] to avoid KVM bad address error when
>> >   you are running on a GPU using TTM for memory management.
>> > 
>> > [1] https://lore.kernel.org/all/dba6eb97-e1d1-4694-bfb6-e72db95715dd@daynix.com/T/
>> > [2] https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5599645/1
>> > [3] https://lore.kernel.org/kvm/20240229025759.1187910-1-stevensd@google.com/
>> > 
>> > Weifeng Liu (3):
>> >   virtio-gpu: rutabaga: Properly set stride when copying resources
>> >   virtio-gpu: rutabaga: Poll rutabaga upon events
>> >   virtio-gpu: rutabaga: Add options to enable virgl and venus contexts
>> > 
>> >  hw/display/virtio-gpu-rutabaga.c | 104 ++++++++++++++++++++++++++++++-
>> >  include/hw/virtio/virtio-gpu.h   |   1 +
>> >  2 files changed, 104 insertions(+), 1 deletion(-)
>> 

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro