Hey all,

This is the third revision of a patch set implementing an accelerator
for the MSHV kernel driver, exposing HyperV to Linux "Dom0" hosts in various
scenarios. Thank you for the feedback so far. Since the last revision we
incorporated feedback from the last review and identified further areas for
performance fixes, notably in the irqfd handling. I'm looking forward to your
comments.

Best regards,

magnus

Changelog:

v2 => v3

- Addressed code review comments (style)
- Reserve GSI 01-23 for IO-APIC pins (this resolved a problem in which MSI
  routes would be overwritten with interrupts from legacy devices, breaking
  irqfd notification for virtio-blk queues)
- Guard memory slot table with mutex and RCU mechanism (multiple threads
  might access the memory slot table, and in the event of an UNMAPPED_GPA
  exit we need to query the table for an unmapped region covering that GPA)
- Include memory slot manager in MshvState
- Produce mshv.h kernel header with ./scripts/update-linux-headers.sh from
  linux 6.16 (not all UAPI definitions are defined in the upstream kernel,
  hence we ship hw/hyper/hvgdk*.h and hw/hyperv/hvhdk*.h headers)
- Added a QMP command query-mshv (a requirement for integration into
  higher-level tooling)
- Removed handling of HALT vm exit, since this is not a supported HV
  message any more.
- Added 2 maintainers from Microsoft for the respective file hierarchy
- Added mshv as accelerator option in the documentation

RFC (v1) => v2

- Addressed code review comments (style, consilidation).
- Rewrote the logic that handles overlap-in-userspace mappings to use
  a static list of slots, inspired by the HVF accelerator code.
- Fixed a bug that wrote corrupt payload in a MSHV_SET_MSI_ROUTING
  call, preventing vhost=on to work on tap network devices.
- Removed an erronous truncation of guest addresses to 32bit when
  registering ioeventfd's using MSHV_IOEVENTFD. This resulted in
  shadowing of low memory when ioevents were registered with
  addresses beyond the 4gb barrier and thus unexpected "unmapped gpa"
  vm exits in lower mem regions (impacting io performance).
- Fixed problem in which the MSI routing table was committed for KVM
  KVM instead of MSHV in virtio-pci bus initialization.
- Added handler for HLT vm exits.
- The above fixes removed a few limitation present in the previous
  revision:
  - Guest with machine type "pc" are booting (testing is still mostly
    performed with q35)
  - Tap network devices can be used with vhost=on option.
  - Seabios can be used with >2.75G memory and multiple virtio-pci
    devices
  - I/O performance improvement as extranous MMIO vm exits are avoided
    by registering ioevents with a correct address.

Notes:

- A discrete kernel ioctl "set_immediate_exit" (to avoid a race condition
  when handling terminiation signals like ctrl-a x) has been tested and
  proven to mitigate the problem. Since other consumers of /dev/mshv have
  simular requirements as QEMU, we opted to iterate a bit more on the
  respective kernel interface.

Magnus Kulke (25):
  accel: Add Meson and config support for MSHV accelerator
  target/i386/emulate: Allow instruction decoding from stream
  target/i386/mshv: Add x86 decoder/emu implementation
  hw/intc: Generalize APIC helper names from kvm_* to accel_*
  include/hw/hyperv: Add MSHV ABI header definitions
  linux-headers/linux: Add mshv.h headers
  accel/mshv: Add accelerator skeleton
  accel/mshv: Register memory region listeners
  accel/mshv: Initialize VM partition
  accel/mshv: Add vCPU creation and execution loop
  accel/mshv: Add vCPU signal handling
  target/i386/mshv: Add CPU create and remove logic
  target/i386/mshv: Implement mshv_store_regs()
  target/i386/mshv: Implement mshv_get_standard_regs()
  target/i386/mshv: Implement mshv_get_special_regs()
  target/i386/mshv: Implement mshv_arch_put_registers()
  target/i386/mshv: Set local interrupt controller state
  target/i386/mshv: Register CPUID entries with MSHV
  target/i386/mshv: Register MSRs with MSHV
  target/i386/mshv: Integrate x86 instruction decoder/emulator
  target/i386/mshv: Write MSRs to the hypervisor
  target/i386/mshv: Implement mshv_vcpu_run()
  accel/mshv: Handle overlapping mem mappings
  docs: Add mshv to documentation
  MAINTAINERS: Add maintainers for mshv accelerator

Praveen K Paladugu (1):
  qapi/accel: Allow to query mshv capabilities

 MAINTAINERS                      |   15 +
 accel/Kconfig                    |    3 +
 accel/accel-irq.c                |  106 ++
 accel/meson.build                |    3 +-
 accel/mshv/irq.c                 |  397 +++++++
 accel/mshv/mem.c                 |  562 ++++++++++
 accel/mshv/meson.build           |    9 +
 accel/mshv/mshv-all.c            |  726 +++++++++++++
 accel/mshv/msr.c                 |  373 +++++++
 accel/mshv/trace-events          |   30 +
 accel/mshv/trace.h               |   14 +
 docs/devel/codebase.rst          |    2 +-
 hw/core/machine-qmp-cmds.c       |   14 +
 hw/intc/apic.c                   |    8 +
 hw/intc/ioapic.c                 |   20 +-
 hw/virtio/virtio-pci.c           |   21 +-
 include/hw/hyperv/hvgdk.h        |   19 +
 include/hw/hyperv/hvgdk_mini.h   |  864 +++++++++++++++
 include/hw/hyperv/hvhdk.h        |  164 +++
 include/hw/hyperv/hvhdk_mini.h   |  105 ++
 include/system/accel-irq.h       |   37 +
 include/system/mshv.h            |  195 ++++
 linux-headers/linux/mshv.h       |  291 ++++++
 meson.build                      |   11 +
 meson_options.txt                |    2 +
 qapi/accelerator.json            |   29 +
 qemu-options.hx                  |   16 +-
 scripts/meson-buildoptions.sh    |    3 +
 scripts/update-linux-headers.sh  |    2 +-
 target/i386/cpu.h                |    4 +-
 target/i386/emulate/meson.build  |    7 +-
 target/i386/emulate/x86_decode.c |   27 +-
 target/i386/emulate/x86_decode.h |    9 +
 target/i386/emulate/x86_emu.c    |    3 +-
 target/i386/emulate/x86_emu.h    |    2 +
 target/i386/meson.build          |    2 +
 target/i386/mshv/meson.build     |    8 +
 target/i386/mshv/mshv-cpu.c      | 1674 ++++++++++++++++++++++++++++++
 target/i386/mshv/x86.c           |  297 ++++++
 39 files changed, 6038 insertions(+), 36 deletions(-)
 create mode 100644 accel/accel-irq.c
 create mode 100644 accel/mshv/irq.c
 create mode 100644 accel/mshv/mem.c
 create mode 100644 accel/mshv/meson.build
 create mode 100644 accel/mshv/mshv-all.c
 create mode 100644 accel/mshv/msr.c
 create mode 100644 accel/mshv/trace-events
 create mode 100644 accel/mshv/trace.h
 create mode 100644 include/hw/hyperv/hvgdk.h
 create mode 100644 include/hw/hyperv/hvgdk_mini.h
 create mode 100644 include/hw/hyperv/hvhdk.h
 create mode 100644 include/hw/hyperv/hvhdk_mini.h
 create mode 100644 include/system/accel-irq.h
 create mode 100644 include/system/mshv.h
 create mode 100644 linux-headers/linux/mshv.h
 create mode 100644 target/i386/mshv/meson.build
 create mode 100644 target/i386/mshv/mshv-cpu.c
 create mode 100644 target/i386/mshv/x86.c

-- 
2.34.1

On Thu, Aug 07, 2025 at 04:39:25PM +0200, Magnus Kulke wrote:
> Hey all,
> 
[...]
> 
> - A discrete kernel ioctl "set_immediate_exit" (to avoid a race condition
>   when handling terminiation signals like ctrl-a x) has been tested and
>   proven to mitigate the problem. Since other consumers of /dev/mshv have
>   simular requirements as QEMU, we opted to iterate a bit more on the
>   respective kernel interface.
> 
> Magnus Kulke (25):
>   accel: Add Meson and config support for MSHV accelerator
>   target/i386/emulate: Allow instruction decoding from stream
>   target/i386/mshv: Add x86 decoder/emu implementation
>   hw/intc: Generalize APIC helper names from kvm_* to accel_*

This needs acks from KVM maintainers.

>   include/hw/hyperv: Add MSHV ABI header definitions
>   linux-headers/linux: Add mshv.h headers
>   accel/mshv: Add accelerator skeleton
>   accel/mshv: Register memory region listeners
>   accel/mshv: Initialize VM partition
>   accel/mshv: Add vCPU creation and execution loop
>   accel/mshv: Add vCPU signal handling
>   target/i386/mshv: Add CPU create and remove logic
>   target/i386/mshv: Implement mshv_store_regs()
>   target/i386/mshv: Implement mshv_get_standard_regs()
>   target/i386/mshv: Implement mshv_get_special_regs()
>   target/i386/mshv: Implement mshv_arch_put_registers()
>   target/i386/mshv: Set local interrupt controller state
>   target/i386/mshv: Register CPUID entries with MSHV
>   target/i386/mshv: Register MSRs with MSHV
>   target/i386/mshv: Integrate x86 instruction decoder/emulator
>   target/i386/mshv: Write MSRs to the hypervisor
>   target/i386/mshv: Implement mshv_vcpu_run()
>   accel/mshv: Handle overlapping mem mappings

I only had a cursory look at this. I'm definitively not an expert on
RCU, so the more reviews we can get the better. To the best of my
(limited) knowledge, the code looks reasonable.

>   docs: Add mshv to documentation
>   MAINTAINERS: Add maintainers for mshv accelerator
> 

The rest looks okay.

> Praveen K Paladugu (1):
>   qapi/accel: Allow to query mshv capabilities
> 

This looks problematic and probably needs to be changed.

I really hope that we can commit as many patches as possible to QEMU
tree, so that we don't need to keep rebasing.

Thanks,
Wei

On Thu, Aug 07, 2025 at 04:39:25PM +0200, Magnus Kulke wrote:
> Hey all,
> 
> This is the third revision of a patch set implementing an accelerator
> for the MSHV kernel driver, exposing HyperV to Linux "Dom0" hosts in various
> scenarios. Thank you for the feedback so far. Since the last revision we
> incorporated feedback from the last review and identified further areas for
> performance fixes, notably in the irqfd handling. I'm looking forward to your
> comments.
> 
> Best regards,
> 
> magnus


So regarding merging plans. Did you guys get in touch
with Sunil? That's the easiest smoothest path, through
an existing maintainer.


> Changelog:
> 
> v2 => v3
> 
> - Addressed code review comments (style)
> - Reserve GSI 01-23 for IO-APIC pins (this resolved a problem in which MSI
>   routes would be overwritten with interrupts from legacy devices, breaking
>   irqfd notification for virtio-blk queues)
> - Guard memory slot table with mutex and RCU mechanism (multiple threads
>   might access the memory slot table, and in the event of an UNMAPPED_GPA
>   exit we need to query the table for an unmapped region covering that GPA)
> - Include memory slot manager in MshvState
> - Produce mshv.h kernel header with ./scripts/update-linux-headers.sh from
>   linux 6.16 (not all UAPI definitions are defined in the upstream kernel,
>   hence we ship hw/hyper/hvgdk*.h and hw/hyperv/hvhdk*.h headers)
> - Added a QMP command query-mshv (a requirement for integration into
>   higher-level tooling)
> - Removed handling of HALT vm exit, since this is not a supported HV
>   message any more.
> - Added 2 maintainers from Microsoft for the respective file hierarchy
> - Added mshv as accelerator option in the documentation
> 
> RFC (v1) => v2
> 
> - Addressed code review comments (style, consilidation).
> - Rewrote the logic that handles overlap-in-userspace mappings to use
>   a static list of slots, inspired by the HVF accelerator code.
> - Fixed a bug that wrote corrupt payload in a MSHV_SET_MSI_ROUTING
>   call, preventing vhost=on to work on tap network devices.
> - Removed an erronous truncation of guest addresses to 32bit when
>   registering ioeventfd's using MSHV_IOEVENTFD. This resulted in
>   shadowing of low memory when ioevents were registered with
>   addresses beyond the 4gb barrier and thus unexpected "unmapped gpa"
>   vm exits in lower mem regions (impacting io performance).
> - Fixed problem in which the MSI routing table was committed for KVM
>   KVM instead of MSHV in virtio-pci bus initialization.
> - Added handler for HLT vm exits.
> - The above fixes removed a few limitation present in the previous
>   revision:
>   - Guest with machine type "pc" are booting (testing is still mostly
>     performed with q35)
>   - Tap network devices can be used with vhost=on option.
>   - Seabios can be used with >2.75G memory and multiple virtio-pci
>     devices
>   - I/O performance improvement as extranous MMIO vm exits are avoided
>     by registering ioevents with a correct address.
> 
> Notes:
> 
> - A discrete kernel ioctl "set_immediate_exit" (to avoid a race condition
>   when handling terminiation signals like ctrl-a x) has been tested and
>   proven to mitigate the problem. Since other consumers of /dev/mshv have
>   simular requirements as QEMU, we opted to iterate a bit more on the
>   respective kernel interface.
> 
> Magnus Kulke (25):
>   accel: Add Meson and config support for MSHV accelerator
>   target/i386/emulate: Allow instruction decoding from stream
>   target/i386/mshv: Add x86 decoder/emu implementation
>   hw/intc: Generalize APIC helper names from kvm_* to accel_*
>   include/hw/hyperv: Add MSHV ABI header definitions
>   linux-headers/linux: Add mshv.h headers
>   accel/mshv: Add accelerator skeleton
>   accel/mshv: Register memory region listeners
>   accel/mshv: Initialize VM partition
>   accel/mshv: Add vCPU creation and execution loop
>   accel/mshv: Add vCPU signal handling
>   target/i386/mshv: Add CPU create and remove logic
>   target/i386/mshv: Implement mshv_store_regs()
>   target/i386/mshv: Implement mshv_get_standard_regs()
>   target/i386/mshv: Implement mshv_get_special_regs()
>   target/i386/mshv: Implement mshv_arch_put_registers()
>   target/i386/mshv: Set local interrupt controller state
>   target/i386/mshv: Register CPUID entries with MSHV
>   target/i386/mshv: Register MSRs with MSHV
>   target/i386/mshv: Integrate x86 instruction decoder/emulator
>   target/i386/mshv: Write MSRs to the hypervisor
>   target/i386/mshv: Implement mshv_vcpu_run()
>   accel/mshv: Handle overlapping mem mappings
>   docs: Add mshv to documentation
>   MAINTAINERS: Add maintainers for mshv accelerator
> 
> Praveen K Paladugu (1):
>   qapi/accel: Allow to query mshv capabilities
> 
>  MAINTAINERS                      |   15 +
>  accel/Kconfig                    |    3 +
>  accel/accel-irq.c                |  106 ++
>  accel/meson.build                |    3 +-
>  accel/mshv/irq.c                 |  397 +++++++
>  accel/mshv/mem.c                 |  562 ++++++++++
>  accel/mshv/meson.build           |    9 +
>  accel/mshv/mshv-all.c            |  726 +++++++++++++
>  accel/mshv/msr.c                 |  373 +++++++
>  accel/mshv/trace-events          |   30 +
>  accel/mshv/trace.h               |   14 +
>  docs/devel/codebase.rst          |    2 +-
>  hw/core/machine-qmp-cmds.c       |   14 +
>  hw/intc/apic.c                   |    8 +
>  hw/intc/ioapic.c                 |   20 +-
>  hw/virtio/virtio-pci.c           |   21 +-
>  include/hw/hyperv/hvgdk.h        |   19 +
>  include/hw/hyperv/hvgdk_mini.h   |  864 +++++++++++++++
>  include/hw/hyperv/hvhdk.h        |  164 +++
>  include/hw/hyperv/hvhdk_mini.h   |  105 ++
>  include/system/accel-irq.h       |   37 +
>  include/system/mshv.h            |  195 ++++
>  linux-headers/linux/mshv.h       |  291 ++++++
>  meson.build                      |   11 +
>  meson_options.txt                |    2 +
>  qapi/accelerator.json            |   29 +
>  qemu-options.hx                  |   16 +-
>  scripts/meson-buildoptions.sh    |    3 +
>  scripts/update-linux-headers.sh  |    2 +-
>  target/i386/cpu.h                |    4 +-
>  target/i386/emulate/meson.build  |    7 +-
>  target/i386/emulate/x86_decode.c |   27 +-
>  target/i386/emulate/x86_decode.h |    9 +
>  target/i386/emulate/x86_emu.c    |    3 +-
>  target/i386/emulate/x86_emu.h    |    2 +
>  target/i386/meson.build          |    2 +
>  target/i386/mshv/meson.build     |    8 +
>  target/i386/mshv/mshv-cpu.c      | 1674 ++++++++++++++++++++++++++++++
>  target/i386/mshv/x86.c           |  297 ++++++
>  39 files changed, 6038 insertions(+), 36 deletions(-)
>  create mode 100644 accel/accel-irq.c
>  create mode 100644 accel/mshv/irq.c
>  create mode 100644 accel/mshv/mem.c
>  create mode 100644 accel/mshv/meson.build
>  create mode 100644 accel/mshv/mshv-all.c
>  create mode 100644 accel/mshv/msr.c
>  create mode 100644 accel/mshv/trace-events
>  create mode 100644 accel/mshv/trace.h
>  create mode 100644 include/hw/hyperv/hvgdk.h
>  create mode 100644 include/hw/hyperv/hvgdk_mini.h
>  create mode 100644 include/hw/hyperv/hvhdk.h
>  create mode 100644 include/hw/hyperv/hvhdk_mini.h
>  create mode 100644 include/system/accel-irq.h
>  create mode 100644 include/system/mshv.h
>  create mode 100644 linux-headers/linux/mshv.h
>  create mode 100644 target/i386/mshv/meson.build
>  create mode 100644 target/i386/mshv/mshv-cpu.c
>  create mode 100644 target/i386/mshv/x86.c
> 
> -- 
> 2.34.1

> On 11. Sep 2025, at 08:59, Michael S. Tsirkin <mst@redhat.com> wrote:
> 
> So regarding merging plans. Did you guys get in touch
> with Sunil? That's the easiest smoothest path, through
> an existing maintainer.


On that bit:

Didn’t submit this yet but this is coming in the next WHPX arm64 patch revision:

    MAINTAINERS: update maintainers for WHPX
    
    And add arm64 files.
    
    From Pedro Barbuda (on Teams):
    
    > we meant to have that switched a while back. you can add me as the maintainer. Pedro Barbuda (pbarbuda@microsoft.com)
    
    Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>

diff --git a/MAINTAINERS b/MAINTAINERS
index ece8624d01..6b1764ccf0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -544,11 +544,14 @@ F: accel/stubs/hvf-stub.c
 F: include/system/hvf.h
 F: include/system/hvf_int.h
 
-WHPX CPUs
-M: Sunil Muthuswamy <sunilmut@microsoft.com>
+WHPX
+M: Pedro Barbuda <pbarbuda@microsoft.com>
+M: Mohamed Mediouni <mohamed@unpredictable.fr>
 S: Supported
 F: accel/whpx/
 F: target/i386/whpx/
+F: target/arm/whpx_arm.h
+F: target/arm/whpx/
 F: accel/stubs/whpx-stub.c
 F: include/system/whpx.h
 F: include/system/whpx-accel-ops.h

On 9/11/25 08:59, Michael S. Tsirkin wrote:
> On Thu, Aug 07, 2025 at 04:39:25PM +0200, Magnus Kulke wrote:
>> Hey all,
>>
>> This is the third revision of a patch set implementing an accelerator
>> for the MSHV kernel driver, exposing HyperV to Linux "Dom0" hosts in various
>> scenarios. Thank you for the feedback so far. Since the last revision we
>> incorporated feedback from the last review and identified further areas for
>> performance fixes, notably in the irqfd handling. I'm looking forward to your
>> comments.
>>
>> Best regards,
>>
>> magnus
> 
> 
> So regarding merging plans. Did you guys get in touch
> with Sunil? That's the easiest smoothest path, through
> an existing maintainer.

There's hardly any code shared with WHPX; I am on vacation this week but 
I'll do a final review and merge it soon.

Paolo

On Thu, Sep 11, 2025 at 05:21:44PM +0200, Paolo Bonzini wrote:
> On 9/11/25 08:59, Michael S. Tsirkin wrote:
> > On Thu, Aug 07, 2025 at 04:39:25PM +0200, Magnus Kulke wrote:
> > > Hey all,
> > > 
> > > This is the third revision of a patch set implementing an accelerator
> > > for the MSHV kernel driver, exposing HyperV to Linux "Dom0" hosts in various
> > > scenarios. Thank you for the feedback so far. Since the last revision we
> > > incorporated feedback from the last review and identified further areas for
> > > performance fixes, notably in the irqfd handling. I'm looking forward to your
> > > comments.
> > > 
> > > Best regards,
> > > 
> > > magnus
> > 
> > 
> > So regarding merging plans. Did you guys get in touch
> > with Sunil? That's the easiest smoothest path, through
> > an existing maintainer.
> 
> There's hardly any code shared with WHPX; I am on vacation this week but
> I'll do a final review and merge it soon.
> 
> Paolo
> 

Hello Paolo, we intend to post a V4 soon-ish (next week probably) which
will include a necessary update to some IOCTLs calls. that and some minor
changes due to review feedback and a rebase, since this is a couple of
weeks old already. It probably makes sense to to wait for that.

best,

magnus

On Thu, Sep 11, 2025 at 05:21:44PM +0200, Paolo Bonzini wrote:
> On 9/11/25 08:59, Michael S. Tsirkin wrote:
> > On Thu, Aug 07, 2025 at 04:39:25PM +0200, Magnus Kulke wrote:
> > > Hey all,
> > > 
> > > This is the third revision of a patch set implementing an accelerator
> > > for the MSHV kernel driver, exposing HyperV to Linux "Dom0" hosts in various
> > > scenarios. Thank you for the feedback so far. Since the last revision we
> > > incorporated feedback from the last review and identified further areas for
> > > performance fixes, notably in the irqfd handling. I'm looking forward to your
> > > comments.
> > > 
> > > Best regards,
> > > 
> > > magnus
> > 
> > 
> > So regarding merging plans. Did you guys get in touch
> > with Sunil? That's the easiest smoothest path, through
> > an existing maintainer.
> 
> There's hardly any code shared with WHPX; I am on vacation this week but
> I'll do a final review and merge it soon.

+1 on this. Nothing's shared with WHPX.

Mangus, can you confirm this is ready to be merged? Is there another
version brewing?

Thanks,
Wei

> 
> Paolo
>