[RFC PATCH 00/19] GICv4 Support for Xen

Mykyta Poturai posted 19 patches 4 days, 11 hours ago
Patches applied successfully (tree, apply log)
git fetch https://gitlab.com/xen-project/patchew/xen tags/patchew/cover.1770046465.git.mykyta._5Fpoturai@epam.com
xen/arch/arm/Kconfig                   |    6 +
xen/arch/arm/Makefile                  |    1 +
xen/arch/arm/dom0less-build.c          |    1 +
xen/arch/arm/domain.c                  |   16 +
xen/arch/arm/gic-v2.c                  |    2 +-
xen/arch/arm/gic-v3-its.c              |  339 +++++--
xen/arch/arm/gic-v3-lpi.c              |  169 +++-
xen/arch/arm/gic-v3.c                  |  215 ++++-
xen/arch/arm/gic-v4-its.c              | 1136 ++++++++++++++++++++++++
xen/arch/arm/gic-vgic.c                |    6 +
xen/arch/arm/include/asm/gic.h         |    4 +-
xen/arch/arm/include/asm/gic_v3_defs.h |   22 +
xen/arch/arm/include/asm/gic_v3_its.h  |  139 ++-
xen/arch/arm/include/asm/gic_v4_its.h  |  114 +++
xen/arch/arm/include/asm/vgic.h        |   79 +-
xen/arch/arm/vgic-v3-its.c             |   60 +-
xen/arch/arm/vgic.c                    |   37 +-
xen/common/domain.c                    |   14 +-
xen/include/public/arch-arm.h          |    2 +
19 files changed, 2174 insertions(+), 188 deletions(-)
create mode 100644 xen/arch/arm/gic-v4-its.c
create mode 100644 xen/arch/arm/include/asm/gic_v4_its.h
[RFC PATCH 00/19] GICv4 Support for Xen
Posted by Mykyta Poturai 4 days, 11 hours ago
This series introduces GICv4 direct LPI injection for Xen.

Direct LPI injection relies on the GIC tracking the mapping between physical and
virtual CPUs. Each VCPU requires a VPE that is created and registered with the
GIC via the `VMAPP` ITS command. The GIC is then informed of the current
VPE-to-PCPU placement by programming `VPENDBASER` and `VPROPBASER` in the
appropriate redistributor. LPIs are associated with VPEs through the `VMAPTI`
ITS command, after which the GIC handles delivery without trapping into the
hypervisor for each interrupt.

When a VPE is not scheduled but has pending interrupts, the GIC raises a per-VPE
doorbell LPI. Doorbells are owned by the hypervisor and prompt rescheduling so
the VPE can drain its pending LPIs.

Because GICv4 lacks a native doorbell invalidation mechanism, this series
includes a helper that invalidates doorbell LPIs via synthetic “proxy” devices,
following the approach used until GICv4.1.

All of this work is mostly based on the work of Penny Zheng
<penny.zheng@arm.com> and Luca Fancellu <luca.fancellu@arm.com>. And also from
Linux patches by Mark Zyngier.

Some patches are still a little rough and need some styling fixes and more
testing, as all of them needed to be carved line by line from a giant ~4000 line
patch. This RFC is directed mostly to get a general idea if the proposed
approach is suitable and OK with everyone. And there is still an open question
of how to handle Signed-off-by lines for Penny and Luca, since they have not
indicated their preference yet.

Mykyta Poturai (19):
  arm/gicv4 add management structure definitions
  arm/gicv4-its: Add GICv4 ITS command definitions
  arm/its: Export struct its_device
  arm/its: Add vlpi configuration
  arm/irq: Add hw flag to pending_irq
  arm/gicv4-its: Add VLPI map/unmap operations
  xen/domain: Alloc enough pages for VCPU struct
  arm/gic: Keep track of GIC features
  arm/its: Implement LPI invalidation
  arm/its: Keep track of BASER regs
  arm/its: Add ITS VM and VPE allocation/teardown
  arm/gic: Add VPENDBASER/VPROPBASER accessors
  arm/gic: VPE scheduling
  arm/its: VPE affinity changes
  arm: Add gicv4 to domain creation
  arm/gic: Fix LR group handling for GICv4
  arm/gicv4: Handle doorbells
  arm/gic: Add VPE proxy support
  arm/gicv4: Add GICv4 to the build system

 xen/arch/arm/Kconfig                   |    6 +
 xen/arch/arm/Makefile                  |    1 +
 xen/arch/arm/dom0less-build.c          |    1 +
 xen/arch/arm/domain.c                  |   16 +
 xen/arch/arm/gic-v2.c                  |    2 +-
 xen/arch/arm/gic-v3-its.c              |  339 +++++--
 xen/arch/arm/gic-v3-lpi.c              |  169 +++-
 xen/arch/arm/gic-v3.c                  |  215 ++++-
 xen/arch/arm/gic-v4-its.c              | 1136 ++++++++++++++++++++++++
 xen/arch/arm/gic-vgic.c                |    6 +
 xen/arch/arm/include/asm/gic.h         |    4 +-
 xen/arch/arm/include/asm/gic_v3_defs.h |   22 +
 xen/arch/arm/include/asm/gic_v3_its.h  |  139 ++-
 xen/arch/arm/include/asm/gic_v4_its.h  |  114 +++
 xen/arch/arm/include/asm/vgic.h        |   79 +-
 xen/arch/arm/vgic-v3-its.c             |   60 +-
 xen/arch/arm/vgic.c                    |   37 +-
 xen/common/domain.c                    |   14 +-
 xen/include/public/arch-arm.h          |    2 +
 19 files changed, 2174 insertions(+), 188 deletions(-)
 create mode 100644 xen/arch/arm/gic-v4-its.c
 create mode 100644 xen/arch/arm/include/asm/gic_v4_its.h

-- 
2.51.2
Re: [RFC PATCH 00/19] GICv4 Support for Xen
Posted by Bertrand Marquis 3 days, 17 hours ago
Hi Mykyta,

We have a number of series from you which have not been merged yet and
reviewing them all in parallel might be challenging.

Would you mind giving us a status and maybe priorities on them.

I could list the following series:
- GICv4
- CPU Hotplug on arm
- PCI enumeration on arm
- IPMMU for pci on arm
- dom0less for pci passthrough on arm
- SR-IOV for pvh
- SMMU for pci on arm
- MSI injection on arm
- suspend to ram on arm

There might be others feel free to complete the list.

On GICv4...

> On 2 Feb 2026, at 17:14, Mykyta Poturai <Mykyta_Poturai@epam.com> wrote:
> 
> This series introduces GICv4 direct LPI injection for Xen.
> 
> Direct LPI injection relies on the GIC tracking the mapping between physical and
> virtual CPUs. Each VCPU requires a VPE that is created and registered with the
> GIC via the `VMAPP` ITS command. The GIC is then informed of the current
> VPE-to-PCPU placement by programming `VPENDBASER` and `VPROPBASER` in the
> appropriate redistributor. LPIs are associated with VPEs through the `VMAPTI`
> ITS command, after which the GIC handles delivery without trapping into the
> hypervisor for each interrupt.
> 
> When a VPE is not scheduled but has pending interrupts, the GIC raises a per-VPE
> doorbell LPI. Doorbells are owned by the hypervisor and prompt rescheduling so
> the VPE can drain its pending LPIs.
> 
> Because GICv4 lacks a native doorbell invalidation mechanism, this series
> includes a helper that invalidates doorbell LPIs via synthetic “proxy” devices,
> following the approach used until GICv4.1.
> 
> All of this work is mostly based on the work of Penny Zheng
> <penny.zheng@arm.com> and Luca Fancellu <luca.fancellu@arm.com>. And also from
> Linux patches by Mark Zyngier.
> 
> Some patches are still a little rough and need some styling fixes and more
> testing, as all of them needed to be carved line by line from a giant ~4000 line
> patch. This RFC is directed mostly to get a general idea if the proposed
> approach is suitable and OK with everyone. And there is still an open question
> of how to handle Signed-off-by lines for Penny and Luca, since they have not
> indicated their preference yet.

I would like to ask how much performance benefits you could
have with this.
Adding GICv4 support is adding a lot of code which will have to be maintained
and tested and there should be a good improvement to justify this.

Did you do some benchmarks ? what are the results ?

At the time where we started to work on that at Arm, we ended up in the conclusion
that the complexity in Xen compared to the benefit was not justifying it hence why
this work was stopped in favor of other features that we thought would be more
beneficial to Xen (like PCI passthrough or SMMUv3).

Cheers
Bertrand

Re: [RFC PATCH 00/19] GICv4 Support for Xen
Posted by Mykyta Poturai 3 days, 15 hours ago
On 03.02.26 12:01, Bertrand Marquis wrote:
> Hi Mykyta,
> 
> We have a number of series from you which have not been merged yet and
> reviewing them all in parallel might be challenging.
> 
> Would you mind giving us a status and maybe priorities on them.
> 
> I could list the following series:
> - GICv4
> - CPU Hotplug on arm
> - PCI enumeration on arm
> - IPMMU for pci on arm
> - dom0less for pci passthrough on arm
> - SR-IOV for pvh
> - SMMU for pci on arm
> - MSI injection on arm
> - suspend to ram on arm
> 
> There might be others feel free to complete the list.
> 
> On GICv4...
> 
>> On 2 Feb 2026, at 17:14, Mykyta Poturai <Mykyta_Poturai@epam.com> wrote:
>>
>> This series introduces GICv4 direct LPI injection for Xen.
>>
>> Direct LPI injection relies on the GIC tracking the mapping between physical and
>> virtual CPUs. Each VCPU requires a VPE that is created and registered with the
>> GIC via the `VMAPP` ITS command. The GIC is then informed of the current
>> VPE-to-PCPU placement by programming `VPENDBASER` and `VPROPBASER` in the
>> appropriate redistributor. LPIs are associated with VPEs through the `VMAPTI`
>> ITS command, after which the GIC handles delivery without trapping into the
>> hypervisor for each interrupt.
>>
>> When a VPE is not scheduled but has pending interrupts, the GIC raises a per-VPE
>> doorbell LPI. Doorbells are owned by the hypervisor and prompt rescheduling so
>> the VPE can drain its pending LPIs.
>>
>> Because GICv4 lacks a native doorbell invalidation mechanism, this series
>> includes a helper that invalidates doorbell LPIs via synthetic “proxy” devices,
>> following the approach used until GICv4.1.
>>
>> All of this work is mostly based on the work of Penny Zheng
>> <penny.zheng@arm.com> and Luca Fancellu <luca.fancellu@arm.com>. And also from
>> Linux patches by Mark Zyngier.
>>
>> Some patches are still a little rough and need some styling fixes and more
>> testing, as all of them needed to be carved line by line from a giant ~4000 line
>> patch. This RFC is directed mostly to get a general idea if the proposed
>> approach is suitable and OK with everyone. And there is still an open question
>> of how to handle Signed-off-by lines for Penny and Luca, since they have not
>> indicated their preference yet.
> 
> I would like to ask how much performance benefits you could
> have with this.
> Adding GICv4 support is adding a lot of code which will have to be maintained
> and tested and there should be a good improvement to justify this.
> 
> Did you do some benchmarks ? what are the results ?
> 
> At the time where we started to work on that at Arm, we ended up in the conclusion
> that the complexity in Xen compared to the benefit was not justifying it hence why
> this work was stopped in favor of other features that we thought would be more
> beneficial to Xen (like PCI passthrough or SMMUv3).
> 
> Cheers
> Bertrand
> 

Hi Bertrand

Current priorities are:

- CPU hotplug
- Suspend to RAM
- GICv4 (we will follow up with benchmarks)
- SR-IOV


MSI injection, dom0less for pci and PCI enumeration are low priority for now

Suspend to RAM is handled by Mykola Kvach

SMMU and IPMMU support are merged already AFAIU

-- 
Mykyta
Re: [RFC PATCH 00/19] GICv4 Support for Xen
Posted by Bertrand Marquis 3 days, 11 hours ago
Hi Mykyta,

> On 3 Feb 2026, at 13:24, Mykyta Poturai <Mykyta_Poturai@epam.com> wrote:
> 
> On 03.02.26 12:01, Bertrand Marquis wrote:
>> Hi Mykyta,
>> 
>> We have a number of series from you which have not been merged yet and
>> reviewing them all in parallel might be challenging.
>> 
>> Would you mind giving us a status and maybe priorities on them.
>> 
>> I could list the following series:
>> - GICv4
>> - CPU Hotplug on arm
>> - PCI enumeration on arm
>> - IPMMU for pci on arm
>> - dom0less for pci passthrough on arm
>> - SR-IOV for pvh
>> - SMMU for pci on arm
>> - MSI injection on arm
>> - suspend to ram on arm
>> 
>> There might be others feel free to complete the list.
>> 
>> On GICv4...
>> 
>>> On 2 Feb 2026, at 17:14, Mykyta Poturai <Mykyta_Poturai@epam.com> wrote:
>>> 
>>> This series introduces GICv4 direct LPI injection for Xen.
>>> 
>>> Direct LPI injection relies on the GIC tracking the mapping between physical and
>>> virtual CPUs. Each VCPU requires a VPE that is created and registered with the
>>> GIC via the `VMAPP` ITS command. The GIC is then informed of the current
>>> VPE-to-PCPU placement by programming `VPENDBASER` and `VPROPBASER` in the
>>> appropriate redistributor. LPIs are associated with VPEs through the `VMAPTI`
>>> ITS command, after which the GIC handles delivery without trapping into the
>>> hypervisor for each interrupt.
>>> 
>>> When a VPE is not scheduled but has pending interrupts, the GIC raises a per-VPE
>>> doorbell LPI. Doorbells are owned by the hypervisor and prompt rescheduling so
>>> the VPE can drain its pending LPIs.
>>> 
>>> Because GICv4 lacks a native doorbell invalidation mechanism, this series
>>> includes a helper that invalidates doorbell LPIs via synthetic “proxy” devices,
>>> following the approach used until GICv4.1.
>>> 
>>> All of this work is mostly based on the work of Penny Zheng
>>> <penny.zheng@arm.com> and Luca Fancellu <luca.fancellu@arm.com>. And also from
>>> Linux patches by Mark Zyngier.
>>> 
>>> Some patches are still a little rough and need some styling fixes and more
>>> testing, as all of them needed to be carved line by line from a giant ~4000 line
>>> patch. This RFC is directed mostly to get a general idea if the proposed
>>> approach is suitable and OK with everyone. And there is still an open question
>>> of how to handle Signed-off-by lines for Penny and Luca, since they have not
>>> indicated their preference yet.
>> 
>> I would like to ask how much performance benefits you could
>> have with this.
>> Adding GICv4 support is adding a lot of code which will have to be maintained
>> and tested and there should be a good improvement to justify this.
>> 
>> Did you do some benchmarks ? what are the results ?
>> 
>> At the time where we started to work on that at Arm, we ended up in the conclusion
>> that the complexity in Xen compared to the benefit was not justifying it hence why
>> this work was stopped in favor of other features that we thought would be more
>> beneficial to Xen (like PCI passthrough or SMMUv3).
>> 
>> Cheers
>> Bertrand
>> 
> 
> Hi Bertrand
> 
> Current priorities are:
> 
> - CPU hotplug
> - Suspend to RAM
> - GICv4 (we will follow up with benchmarks)
> - SR-IOV
> 

Ok Let's focus on what is already there and being reviewed before GICv4.

I will follow up and your CPU hotplug review and suspend to RAM is already advanced so
we should focus on finishing those first.

Cheers
Bertrand

> 
> MSI injection, dom0less for pci and PCI enumeration are low priority for now
> 
> Suspend to RAM is handled by Mykola Kvach
> 
> SMMU and IPMMU support are merged already AFAIU
> 
> -- 
> Mykyta