Hi all,

In graphic world, the 3D applications/games are runing based on open
graphic libraries such as OpenGL and Vulkan. Mesa is the Linux
implemenatation of OpenGL and Vulkan for multiple hardware platforms.
Because the graphic libraries would like to have the GPU hardware
acceleration. In virtualization world, virtio-gpu and passthrough-gpu are
two of gpu virtualization technologies.

Current Xen only supports OpenGL (virgl:
https://docs.mesa3d.org/drivers/virgl.html) for virtio-gpu and passthrough
gpu based on PV dom0 for x86 platform. Today, we would like to introduce
Vulkan (venus: https://docs.mesa3d.org/drivers/venus.html) and another
OpenGL on Vulkan (zink: https://docs.mesa3d.org/drivers/zink.html) support
for VirtIO GPU on Xen. These functions are supported on KVM at this moment,
but so far, they are not supported on Xen. And we also introduce the PCIe
passthrough (GPU) function based on PVH dom0 for AMD x86 platform.

These supports required multiple repositories changes on kernel, xen, qemu,
mesa, and virglrenderer. Please check below branches:

Kernel: https://git.kernel.org/pub/scm/linux/kernel/git/rui/linux.git/log/?h=upstream-fox-xen
Xen: https://gitlab.com/huangrui123/xen/-/commits/upstream-for-xen
QEMU: https://gitlab.com/huangrui123/qemu/-/commits/upstream-for-xen
Mesa: https://gitlab.freedesktop.org/rui/mesa/-/commits/upstream-for-xen
Virglrenderer: https://gitlab.freedesktop.org/rui/virglrenderer/-/commits/upstream-for-xen

In xen part, we mainly add the PCIe passthrough support on PVH dom0. It's
using the QEMU to passthrough the GPU device into guest HVM domU. And
mainly work is to transfer the interrupt by using gsi, vector, and pirq.

Below are the screenshot of these functions, please take a look.

Venus:
https://drive.google.com/file/d/1_lPq6DMwHu1JQv7LUUVRx31dBj0HJYcL/view?usp=share_link

Zink:
https://drive.google.com/file/d/1FxLmKu6X7uJOxx1ZzwOm1yA6IL5WMGzd/view?usp=share_link

Passthrough GPU:
https://drive.google.com/file/d/17onr5gvDK8KM_LniHTSQEI2hGJZlI09L/view?usp=share_link

We are working to write the documentation that describe how to verify these
functions in the xen wiki page. And will update it in the future version.

Thanks,
Ray

Chen Jiqian (5):
  vpci: accept BAR writes if dom0 is PVH
  x86/pvh: shouldn't check pirq flag when map pirq in PVH
  x86/pvh: PVH dom0 also need PHYSDEVOP_setup_gsi call
  tools/libs/call: add linux os call to get gsi from irq
  tools/libs/light: pci: translate irq to gsi

Roger Pau Monne (1):
  x86/pvh: report ACPI VFCT table to dom0 if present

 tools/include/xen-sys/Linux/privcmd.h |  7 +++++++
 tools/include/xencall.h               |  2 ++
 tools/include/xenctrl.h               |  2 ++
 tools/libs/call/core.c                |  5 +++++
 tools/libs/call/libxencall.map        |  2 ++
 tools/libs/call/linux.c               | 14 ++++++++++++++
 tools/libs/call/private.h             |  9 +++++++++
 tools/libs/ctrl/xc_physdev.c          |  4 ++++
 tools/libs/light/libxl_pci.c          |  1 +
 xen/arch/x86/hvm/dom0_build.c         |  1 +
 xen/arch/x86/hvm/hypercall.c          |  3 +--
 xen/drivers/vpci/header.c             |  2 +-
 xen/include/acpi/actbl3.h             |  1 +
 13 files changed, 50 insertions(+), 3 deletions(-)

-- 
2.25.1

Hi Jan, Roger, Stefano, Andrew,

Sorry to late response, I was fully occupied by another problem last week.
And I will give the reply one by one in the mail tomorrow. Thanks for your
patience. :-)

Thanks,
Ray

On Sun, Mar 12, 2023 at 03:54:49PM +0800, Huang, Ray wrote:
> Hi all,
> 
> In graphic world, the 3D applications/games are runing based on open
> graphic libraries such as OpenGL and Vulkan. Mesa is the Linux
> implemenatation of OpenGL and Vulkan for multiple hardware platforms.
> Because the graphic libraries would like to have the GPU hardware
> acceleration. In virtualization world, virtio-gpu and passthrough-gpu are
> two of gpu virtualization technologies.
> 
> Current Xen only supports OpenGL (virgl:
> https://docs.mesa3d.org/drivers/virgl.html) for virtio-gpu and passthrough
> gpu based on PV dom0 for x86 platform. Today, we would like to introduce
> Vulkan (venus: https://docs.mesa3d.org/drivers/venus.html) and another
> OpenGL on Vulkan (zink: https://docs.mesa3d.org/drivers/zink.html) support
> for VirtIO GPU on Xen. These functions are supported on KVM at this moment,
> but so far, they are not supported on Xen. And we also introduce the PCIe
> passthrough (GPU) function based on PVH dom0 for AMD x86 platform.
> 
> These supports required multiple repositories changes on kernel, xen, qemu,
> mesa, and virglrenderer. Please check below branches:
> 
> Kernel: https://git.kernel.org/pub/scm/linux/kernel/git/rui/linux.git/log/?h=upstream-fox-xen
> Xen: https://gitlab.com/huangrui123/xen/-/commits/upstream-for-xen
> QEMU: https://gitlab.com/huangrui123/qemu/-/commits/upstream-for-xen
> Mesa: https://gitlab.freedesktop.org/rui/mesa/-/commits/upstream-for-xen
> Virglrenderer: https://gitlab.freedesktop.org/rui/virglrenderer/-/commits/upstream-for-xen
> 
> In xen part, we mainly add the PCIe passthrough support on PVH dom0. It's
> using the QEMU to passthrough the GPU device into guest HVM domU. And
> mainly work is to transfer the interrupt by using gsi, vector, and pirq.
> 
> Below are the screenshot of these functions, please take a look.
> 
> Venus:
> https://drive.google.com/file/d/1_lPq6DMwHu1JQv7LUUVRx31dBj0HJYcL/view?usp=share_link
> 
> Zink:
> https://drive.google.com/file/d/1FxLmKu6X7uJOxx1ZzwOm1yA6IL5WMGzd/view?usp=share_link
> 
> Passthrough GPU:
> https://drive.google.com/file/d/17onr5gvDK8KM_LniHTSQEI2hGJZlI09L/view?usp=share_link
> 
> We are working to write the documentation that describe how to verify these
> functions in the xen wiki page. And will update it in the future version.
> 
> Thanks,
> Ray
> 
> Chen Jiqian (5):
>   vpci: accept BAR writes if dom0 is PVH
>   x86/pvh: shouldn't check pirq flag when map pirq in PVH
>   x86/pvh: PVH dom0 also need PHYSDEVOP_setup_gsi call
>   tools/libs/call: add linux os call to get gsi from irq
>   tools/libs/light: pci: translate irq to gsi
> 
> Roger Pau Monne (1):
>   x86/pvh: report ACPI VFCT table to dom0 if present
> 
>  tools/include/xen-sys/Linux/privcmd.h |  7 +++++++
>  tools/include/xencall.h               |  2 ++
>  tools/include/xenctrl.h               |  2 ++
>  tools/libs/call/core.c                |  5 +++++
>  tools/libs/call/libxencall.map        |  2 ++
>  tools/libs/call/linux.c               | 14 ++++++++++++++
>  tools/libs/call/private.h             |  9 +++++++++
>  tools/libs/ctrl/xc_physdev.c          |  4 ++++
>  tools/libs/light/libxl_pci.c          |  1 +
>  xen/arch/x86/hvm/dom0_build.c         |  1 +
>  xen/arch/x86/hvm/hypercall.c          |  3 +--
>  xen/drivers/vpci/header.c             |  2 +-
>  xen/include/acpi/actbl3.h             |  1 +
>  13 files changed, 50 insertions(+), 3 deletions(-)
> 
> -- 
> 2.25.1
>

Hi Ray,

one nit comment on the style, apart from that looks technical correct.

But I'm *really* not and expert on all that stuff.

Regards,
Christian.

Am 12.03.23 um 08:54 schrieb Huang Rui:
> Hi all,
>
> In graphic world, the 3D applications/games are runing based on open
> graphic libraries such as OpenGL and Vulkan. Mesa is the Linux
> implemenatation of OpenGL and Vulkan for multiple hardware platforms.
> Because the graphic libraries would like to have the GPU hardware
> acceleration. In virtualization world, virtio-gpu and passthrough-gpu are
> two of gpu virtualization technologies.
>
> Current Xen only supports OpenGL (virgl:
> https://docs.mesa3d.org/drivers/virgl.html) for virtio-gpu and passthrough
> gpu based on PV dom0 for x86 platform. Today, we would like to introduce
> Vulkan (venus: https://docs.mesa3d.org/drivers/venus.html) and another
> OpenGL on Vulkan (zink: https://docs.mesa3d.org/drivers/zink.html) support
> for VirtIO GPU on Xen. These functions are supported on KVM at this moment,
> but so far, they are not supported on Xen. And we also introduce the PCIe
> passthrough (GPU) function based on PVH dom0 for AMD x86 platform.
>
> These supports required multiple repositories changes on kernel, xen, qemu,
> mesa, and virglrenderer. Please check below branches:
>
> Kernel: https://git.kernel.org/pub/scm/linux/kernel/git/rui/linux.git/log/?h=upstream-fox-xen
> Xen: https://gitlab.com/huangrui123/xen/-/commits/upstream-for-xen
> QEMU: https://gitlab.com/huangrui123/qemu/-/commits/upstream-for-xen
> Mesa: https://gitlab.freedesktop.org/rui/mesa/-/commits/upstream-for-xen
> Virglrenderer: https://gitlab.freedesktop.org/rui/virglrenderer/-/commits/upstream-for-xen
>
> In xen part, we mainly add the PCIe passthrough support on PVH dom0. It's
> using the QEMU to passthrough the GPU device into guest HVM domU. And
> mainly work is to transfer the interrupt by using gsi, vector, and pirq.
>
> Below are the screenshot of these functions, please take a look.
>
> Venus:
> https://drive.google.com/file/d/1_lPq6DMwHu1JQv7LUUVRx31dBj0HJYcL/view?usp=share_link
>
> Zink:
> https://drive.google.com/file/d/1FxLmKu6X7uJOxx1ZzwOm1yA6IL5WMGzd/view?usp=share_link
>
> Passthrough GPU:
> https://drive.google.com/file/d/17onr5gvDK8KM_LniHTSQEI2hGJZlI09L/view?usp=share_link
>
> We are working to write the documentation that describe how to verify these
> functions in the xen wiki page. And will update it in the future version.
>
> Thanks,
> Ray
>
> Chen Jiqian (5):
>    vpci: accept BAR writes if dom0 is PVH
>    x86/pvh: shouldn't check pirq flag when map pirq in PVH
>    x86/pvh: PVH dom0 also need PHYSDEVOP_setup_gsi call
>    tools/libs/call: add linux os call to get gsi from irq
>    tools/libs/light: pci: translate irq to gsi
>
> Roger Pau Monne (1):
>    x86/pvh: report ACPI VFCT table to dom0 if present
>
>   tools/include/xen-sys/Linux/privcmd.h |  7 +++++++
>   tools/include/xencall.h               |  2 ++
>   tools/include/xenctrl.h               |  2 ++
>   tools/libs/call/core.c                |  5 +++++
>   tools/libs/call/libxencall.map        |  2 ++
>   tools/libs/call/linux.c               | 14 ++++++++++++++
>   tools/libs/call/private.h             |  9 +++++++++
>   tools/libs/ctrl/xc_physdev.c          |  4 ++++
>   tools/libs/light/libxl_pci.c          |  1 +
>   xen/arch/x86/hvm/dom0_build.c         |  1 +
>   xen/arch/x86/hvm/hypercall.c          |  3 +--
>   xen/drivers/vpci/header.c             |  2 +-
>   xen/include/acpi/actbl3.h             |  1 +
>   13 files changed, 50 insertions(+), 3 deletions(-)
>

On Mon, Mar 13, 2023 at 03:24:55PM +0800, Koenig, Christian wrote:
> Hi Ray,
> 
> one nit comment on the style, apart from that looks technical correct.
> 
> But I'm *really* not and expert on all that stuff.

Christian, thanks anyway. :-)

Thanks,
Ray

> 
> Regards,
> Christian.
> 
> Am 12.03.23 um 08:54 schrieb Huang Rui:
> > Hi all,
> >
> > In graphic world, the 3D applications/games are runing based on open
> > graphic libraries such as OpenGL and Vulkan. Mesa is the Linux
> > implemenatation of OpenGL and Vulkan for multiple hardware platforms.
> > Because the graphic libraries would like to have the GPU hardware
> > acceleration. In virtualization world, virtio-gpu and passthrough-gpu are
> > two of gpu virtualization technologies.
> >
> > Current Xen only supports OpenGL (virgl:
> > https://docs.mesa3d.org/drivers/virgl.html) for virtio-gpu and passthrough
> > gpu based on PV dom0 for x86 platform. Today, we would like to introduce
> > Vulkan (venus: https://docs.mesa3d.org/drivers/venus.html) and another
> > OpenGL on Vulkan (zink: https://docs.mesa3d.org/drivers/zink.html) support
> > for VirtIO GPU on Xen. These functions are supported on KVM at this moment,
> > but so far, they are not supported on Xen. And we also introduce the PCIe
> > passthrough (GPU) function based on PVH dom0 for AMD x86 platform.
> >
> > These supports required multiple repositories changes on kernel, xen, qemu,
> > mesa, and virglrenderer. Please check below branches:
> >
> > Kernel: https://git.kernel.org/pub/scm/linux/kernel/git/rui/linux.git/log/?h=upstream-fox-xen
> > Xen: https://gitlab.com/huangrui123/xen/-/commits/upstream-for-xen
> > QEMU: https://gitlab.com/huangrui123/qemu/-/commits/upstream-for-xen
> > Mesa: https://gitlab.freedesktop.org/rui/mesa/-/commits/upstream-for-xen
> > Virglrenderer: https://gitlab.freedesktop.org/rui/virglrenderer/-/commits/upstream-for-xen
> >
> > In xen part, we mainly add the PCIe passthrough support on PVH dom0. It's
> > using the QEMU to passthrough the GPU device into guest HVM domU. And
> > mainly work is to transfer the interrupt by using gsi, vector, and pirq.
> >
> > Below are the screenshot of these functions, please take a look.
> >
> > Venus:
> > https://drive.google.com/file/d/1_lPq6DMwHu1JQv7LUUVRx31dBj0HJYcL/view?usp=share_link
> >
> > Zink:
> > https://drive.google.com/file/d/1FxLmKu6X7uJOxx1ZzwOm1yA6IL5WMGzd/view?usp=share_link
> >
> > Passthrough GPU:
> > https://drive.google.com/file/d/17onr5gvDK8KM_LniHTSQEI2hGJZlI09L/view?usp=share_link
> >
> > We are working to write the documentation that describe how to verify these
> > functions in the xen wiki page. And will update it in the future version.
> >
> > Thanks,
> > Ray
> >
> > Chen Jiqian (5):
> >    vpci: accept BAR writes if dom0 is PVH
> >    x86/pvh: shouldn't check pirq flag when map pirq in PVH
> >    x86/pvh: PVH dom0 also need PHYSDEVOP_setup_gsi call
> >    tools/libs/call: add linux os call to get gsi from irq
> >    tools/libs/light: pci: translate irq to gsi
> >
> > Roger Pau Monne (1):
> >    x86/pvh: report ACPI VFCT table to dom0 if present
> >
> >   tools/include/xen-sys/Linux/privcmd.h |  7 +++++++
> >   tools/include/xencall.h               |  2 ++
> >   tools/include/xenctrl.h               |  2 ++
> >   tools/libs/call/core.c                |  5 +++++
> >   tools/libs/call/libxencall.map        |  2 ++
> >   tools/libs/call/linux.c               | 14 ++++++++++++++
> >   tools/libs/call/private.h             |  9 +++++++++
> >   tools/libs/ctrl/xc_physdev.c          |  4 ++++
> >   tools/libs/light/libxl_pci.c          |  1 +
> >   xen/arch/x86/hvm/dom0_build.c         |  1 +
> >   xen/arch/x86/hvm/hypercall.c          |  3 +--
> >   xen/drivers/vpci/header.c             |  2 +-
> >   xen/include/acpi/actbl3.h             |  1 +
> >   13 files changed, 50 insertions(+), 3 deletions(-)
> >
>