Hi all,
This series adds the initial virtual interrupt controller (vINTC) support
for RISC-V guests in Xen, based on the Advanced Interrupt Architecture
(AIA): a virtual APLIC (vAPLIC) in MSI mode backed by a virtual IMSIC
(vIMSIC) using hardware guest interrupt files.
Rather than emulating APLIC in direct-delivery mode (which requires
trap-and-emulate for every interrupt and is costly), the series targets
IMSIC from the start. AIA lets a hart implement several "guest interrupt
files" (up to GEILEN), so external interrupts can be delivered to a vCPU
directly by hardware via the VGEIN field of hstatus, without a hypervisor
round-trip. Xen only has to emulate the APLIC MMIO programming interface
and route the guest's intent onto the physical MSI topology; interrupt
delivery itself stays in hardware.
The work breaks down into a few logical blocks:
Preparatory fixes and cleanups (patches 1-6)
- Drop the unused pregs field of struct cpu_user_regs and bug.h's
duplicate instruction length helpers.
- Program hstatus.VSXL explicitly, as decoding a trapped instruction
depends on the effective XLEN of the guest.
- csr_read64() as the counterpart of csr_write64(), used for CSR_TIME so
that get_cycles() no longer truncates the time counter on RV32, and
UINT64_MAX rather than ULONG_MAX to disable the VS-timer.
- Request a G-stage flush on vmenter where VMIDs are unavailable, so a
domain cannot run on the translations left behind by the one which ran
on that hart before it.
APLIC groundwork and vAPLIC MMIO emulation (patches 7-11)
- Add the missing APLIC register offsets/masks needed by both the
physical and virtual APLIC code, rearranging asm/aplic.h in the style
of x86's asm/msr-index.h (no functional change).
- A per-domain MMIO handler table modelled on Arm's framework, so
emulated devices self-register their GPA ranges and the fault path
stays agnostic via a single try_handle_mmio() entry point.
- vAPLIC MMIO read/write emulation. Writes are gated by the domain's
authorised-IRQ bitmap so a guest cannot touch interrupts it does not
own, and TARGET writes are translated from virtual to physical
hart/guest-file indices. Delegation (SOURCECFG.D) is not yet
supported.
- Build the physical APLIC's target hart index with aplic_hart_field()
as well, dropping the last in-tree duplicate of the AIA hart index
formula, and add a helper to test for APLIC MSI mode.
vCPU context switching (patches 12-15)
- context_switch() and the helpers it needs: save/restore of H/VS CSRs,
the virtual timer and P2M context, and __context_switch() in assembly.
The VMID is claimed in p2m_ctxt_switch_to() rather than at the next
guest entry, which leaves p2m_handle_vmenter() with nothing to do.
- Save and restore the AIA CSRs a guest can change (vsiselect and
hviprio{1,2}), gated by hstateen0 where Smstateen is implemented.
- vintc_ctxt_switch_{from,to}() wrappers over new ctxt_switch_{from,to}
hooks in struct vintc_ops, called from the context switch path, plus
the IMSIC implementation of those hooks: it records which pCPU owns a
vCPU's guest interrupt file, as the pCPU id is part of the MSI
address.
Trap and instruction emulation infrastructure (patches 16-26)
- Extend the exception-table format with type/data fields and add
EX_TYPE_TRAP_INFO so fixups can capture sepc/scause/stval, and look
the table up for any trap taken in Xen context rather than for illegal
instructions only, so that the hlv/hlvx sequences reach their fixup.
- A guest page-fault handler, and trap_redirect() to forward a
synchronous trap back into the guest's VS-mode handler for the faults
which can never become an emulated access.
- Resolve the faulting guest physical address from htval and stval,
which first needs Shtvala to be detected, and define all four
INSN_PSEUDO_VS_* values independently of the hypervisor's XLEN, so
that a fault taken on an implicit VS-stage access is recognized as one
rather than mistaken for an MMIO trap.
- riscv_read_guest() (HLV/HLVX) to read guest memory and instructions
safely, the decoding helpers shared by both access types, and the load
and store emulation which dispatches the access through
try_handle_mmio().
vCPU migration between pCPUs (patches 27-34) (introduced here for better context
of VGEIN fumctions usage)
- arch_move_irqs(), dispatching through a new move_irqs hook in
struct vintc_ops down to imsic_migrate_vcpu(), and the case where a
vCPU has no guest interrupt file to move yet.
- The move of a vCPU's IMSIC guest interrupt file itself, following the
sequence the AIA spec prescribes: quiesce and save eidelivery/
eithreshold of the old file, zero the new one, G-stage remap it,
retarget the domain's APLIC interrupts at it
(aplic_reconfigure_target()) and fence off straggler MSIs with a
genmsi barrier, dump the old file's eip/eie arrays to memory, then
restore that state into the new file and update the vCPU's
hstatus.VGEIN.
VGEIN allocation and vCPU bring-up (patches 35-39)
- Per-pCPU VGEIN (guest interrupt file) allocator: a bitmap of the files
a hart implements (up to GEILEN) with helpers to assign and release
one, and an owners[] map so a file reported pending in HGEIP can be
traced back to the vCPU it belongs to.
- Watch a descheduled vCPU's guest interrupt file through HGEIE, so that
a guest blocked on an external interrupt is woken up instead of
waiting for an unrelated event to schedule it again.
- Stage-2 map a vCPU's physical guest interrupt file to the fixed
per-vCPU GPA page the guest expects at offset 0.
- continue_new_vcpu(): switch to the idle vCPU's own stack for the idle
vCPU, and enter the guest through the new return_to_new_vcpu() path in
entry.S for a guest one.
- imsic_vsfile_attach(), called once the pCPU a vCPU will run on is
known: it assigns a VGEIN, maps the guest interrupt file and records
the IMSIC state as a consistent unit.
CI tests: https://gitlab.com/xen-project/people/olkur/xen/-/pipelines/2796779115
The series depends on [1].
[1] https://lore.kernel.org/xen-devel/cover.1787836900.git.oleksii.kurochko@gmail.com/T/#t
---
Changes in v2:
- The series has grown from 17 to 39 patches. vCPU context switching, vCPU
migration between pCPUs and the vCPU bring-up path (continue_new_vcpu(),
attaching an IMSIC h/w interrupt file) are now part of it to have better
context of how things are using, together with the trap-side pieces the
MMIO emulation depends on (faulting GPA resolution, Shtvala detection,
instruction decoding).
- vintc_state_{save,restore}() became vintc_ctxt_switch_{from,to}() and
vcpu_aia_init() became imsic_vsfile_attach(); "xen/riscv: manage
IRQ_DISABLED flag in APLIC irq enable/disable callbacks" is no longer part
of this series. The remaining changes are described in the per-patch
changelogs.
---
Oleksii Kurochko (39):
xen/riscv: drop pregs from struct cpu_user_regs
xen/riscv: drop bug.h's duplicate instruction length helpers
xen/riscv: set the guest's XLEN explicitly in hstatus.VSXL
xen/riscv: introduce csr_read64()
xen/riscv: request a G-stage flush on vmenter when VMIDs are disabled
xen/riscv: use UINT64_MAX to disable the VS-timer
xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h
xen/riscv: introduce device-agnostic MMIO emulation dispatch
xen/riscv: implement virtual APLIC MMIO emulation
xen/riscv: build the target hart index via aplic_hart_field()
xen/riscv: add helper to check APLIC MSI mode
xen/riscv: implement vCPU context switching
xen/riscv: save and restore AIA state on vCPU context switch
xen/riscv: introduce vintc_ctxt_switch_{from,to}()
xen/riscv: add IMSIC vCPU context switch handlers
xen/riscv: extend exception tables with type and data fields
xen/riscv: decouple INSN_PSEUDO_VS_* from the hypervisor's XLEN
xen/riscv: add guest page fault handling stub
xen/riscv: implement trap redirection to a guest
xen/riscv: detect Shtvala
xen/riscv: resolve the faulting guest physical address
xen/riscv: add guest memory read helper
xen/riscv: look up the exception table for any trap taken in Xen
context
xen/riscv: add helpers for decoding a trapped load or store
xen/riscv: add guest load emulation for trapped MMIO accesses
xen/riscv: add guest store emulation for trapped MMIO accesses
xen/riscv: introduce arch_move_irqs()
xen/riscv: handle the case when no vCPU migration is needed
xen/riscv: introduce aplic_reconfigure_target()
xen/riscv: prepare new IMSIC VS-file
xen/riscv: implement APLIC-hart sync barrier for vCPU migration
xen/riscv: remap interrupts to new IMSIC VS-file
xen/riscv: dump old interrupt file to memory
xen/riscv: restore register state in the new IMSIC VS-file
xen/riscv: add basic VGEIN management for AIA guests
xen/riscv: wake up a descheduled vCPU on a guest external interrupt
xen/riscv: map IMSIC interrupt file for vCPUs
xen/riscv: implement continue_new_vcpu()
xen/riscv: introduce IMSIC h/w interrupt file attaching to vcpu
xen/arch/riscv/Makefile | 2 +
xen/arch/riscv/aia.c | 193 ++++++
xen/arch/riscv/aplic-priv.h | 3 +
xen/arch/riscv/aplic.c | 235 ++++++-
xen/arch/riscv/cpufeature.c | 1 +
xen/arch/riscv/domain.c | 271 +++++++-
xen/arch/riscv/emulate.c | 578 ++++++++++++++++
xen/arch/riscv/entry.S | 67 ++
xen/arch/riscv/extable.c | 70 +-
xen/arch/riscv/guestcopy.c | 87 +++
xen/arch/riscv/imsic.c | 691 ++++++++++++++++++++
xen/arch/riscv/include/asm/aia.h | 7 +
xen/arch/riscv/include/asm/aplic.h | 130 +++-
xen/arch/riscv/include/asm/bug.h | 19 -
xen/arch/riscv/include/asm/cpufeature.h | 1 +
xen/arch/riscv/include/asm/csr.h | 24 +
xen/arch/riscv/include/asm/current.h | 4 +
xen/arch/riscv/include/asm/domain.h | 23 +-
xen/arch/riscv/include/asm/emulate.h | 10 +
xen/arch/riscv/include/asm/extable.h | 64 +-
xen/arch/riscv/include/asm/gpr-num.h | 37 ++
xen/arch/riscv/include/asm/guest_access.h | 4 +
xen/arch/riscv/include/asm/imsic.h | 31 +
xen/arch/riscv/include/asm/intc.h | 12 +
xen/arch/riscv/include/asm/irq.h | 5 +-
xen/arch/riscv/include/asm/mmio.h | 63 ++
xen/arch/riscv/include/asm/p2m.h | 1 -
xen/arch/riscv/include/asm/processor.h | 16 +-
xen/arch/riscv/include/asm/riscv_encoding.h | 34 +-
xen/arch/riscv/include/asm/system.h | 4 +
xen/arch/riscv/include/asm/time.h | 4 +-
xen/arch/riscv/include/asm/traps.h | 9 +
xen/arch/riscv/include/asm/vaplic.h | 5 +
xen/arch/riscv/intc.c | 22 +
xen/arch/riscv/mmio.c | 176 +++++
xen/arch/riscv/p2m.c | 55 +-
xen/arch/riscv/riscv64/asm-offsets.c | 20 +-
xen/arch/riscv/stubs.c | 5 -
xen/arch/riscv/time.c | 4 +-
xen/arch/riscv/traps.c | 110 +++-
xen/arch/riscv/vaplic.c | 358 +++++++++-
xen/arch/riscv/vmid.c | 4 +-
xen/include/xen/config.h | 1 +
43 files changed, 3281 insertions(+), 179 deletions(-)
create mode 100644 xen/arch/riscv/emulate.c
create mode 100644 xen/arch/riscv/include/asm/emulate.h
create mode 100644 xen/arch/riscv/include/asm/gpr-num.h
create mode 100644 xen/arch/riscv/include/asm/mmio.h
create mode 100644 xen/arch/riscv/mmio.c
--
2.55.0