From nobody Wed Feb 11 04:19:44 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1770656005; cv=none; d=zohomail.com; s=zohoarc; b=a3Djn7QNJip6Xdw2rzS4hQjb++VyT3NTan+rhyum1PaIkPgJp1tB3RWGYLODq7OTqfLquks2M9598UBTrBQd9pmllWRxLOCj6fTjSZbkVDynoORUS85CRoRhq6NeuV/dx1QrJwWg4aXoXjPd7ZE7AJR0D2vEm5d6S+XJcgLUfEs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1770656005; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SzY4hdexyuBIJK3FnsuLFLVzhScz5N5GsSrSydDkBvI=; b=ZX8B9Qnull9eUxM0sDY4Goz6y6nIrHiSRYswToS2ISigaZID21AM7zXmf6nMkkknwxPWThA8vxgiQ1mieH8nRMdTg8XEFrMj3y7DFX4H/z1I33obT3PidoTan7O1EczRzZuI8dcLTfrwwwyGBLmIGgXNoCN7qg06To9R724vpm8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1770656005639357.61804993697615; Mon, 9 Feb 2026 08:53:25 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1225790.1532411 (Exim 4.92) (envelope-from ) id 1vpUVE-0000Lg-0r; Mon, 09 Feb 2026 16:52:48 +0000 Received: by outflank-mailman (output) from mailman id 1225790.1532411; Mon, 09 Feb 2026 16:52:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vpUVD-0000LF-Oo; Mon, 09 Feb 2026 16:52:47 +0000 Received: by outflank-mailman (input) for mailman id 1225790; Mon, 09 Feb 2026 16:52:46 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vpUVB-0007Zd-UZ for xen-devel@lists.xenproject.org; Mon, 09 Feb 2026 16:52:45 +0000 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [2a00:1450:4864:20::629]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c109f96a-05d7-11f1-9ccf-f158ae23cfc8; Mon, 09 Feb 2026 17:52:44 +0100 (CET) Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-b8871718b05so750093466b.2 for ; Mon, 09 Feb 2026 08:52:44 -0800 (PST) Received: from fedora (user-109-243-67-101.play-internet.pl. [109.243.67.101]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b8edacf1564sm415776866b.52.2026.02.09.08.52.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Feb 2026 08:52:42 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c109f96a-05d7-11f1-9ccf-f158ae23cfc8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1770655963; x=1771260763; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SzY4hdexyuBIJK3FnsuLFLVzhScz5N5GsSrSydDkBvI=; b=SbZrHIzIey+5pW5ALuvACrTCue2E67SCBmgQefYcjFBFlO3S2n89bkdRAoFD/vdRiM 1LU1UiYTh1kCwoMVm4j5kV7Dj1uKv1zVY9h183nRwlSWIfYOSaxyT2RMwcwZB0nGeHro Ymir2PJD0yHO8InUrkiNXk9Thq7PvcP2Hu1ZiyF3gr+jkaSGaEBo6quMtONSDvDI5eCZ Y6sfOzSNZLHLhItfdPzjRIRMRFZaKCBwLF26POOrVy0LVsXinHDOft+gYo2eqkNigoB5 NV5Jc8u6DbUIc/pu5ZCAwruBoYsk3kbMsf4EzRXWVeqiriS4DBdKRo5SmPUpVLHd3sN/ kRyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770655963; x=1771260763; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=SzY4hdexyuBIJK3FnsuLFLVzhScz5N5GsSrSydDkBvI=; b=VNdYFacGm3kqkV3sabQJeSEIE1NmxFLVNo16k5tNgaebF/oCblyNhkXmPqiJtiL0jD Utdq77yYVDqyDppDyLKpayKamnVJ4ELKg2kvHliN1hjIbJc2jjjnwBTustCs/XT2Rp20 5QY3ZGnhqZN/paXjazCYbAUoC3E//l3rebIxMVAG7pJFO2H2Pq1PE+4iQKE6CbjdclIY h2KJz8VZdhUsX2q9uniE31Bsa4vkl5o9PNn4L6PWbuKb9N6aKSr8V/IVWw0f5iCasw69 XyoUd4ihPZ6uIYRgG0k5vFQjzA9Hv3TCEtpdcm6+egHtDSlIz9uE4CF6DJqqyl8vft3U IT5A== X-Gm-Message-State: AOJu0YyqGVcJtEr7EL3VRbnqqpEx6KxepkyguKIhjzoB0e5XqVYoEOVI m8t4qw52fSnllAWAXTspJbA/uxRAnSr413uAjYfWOJbtzqthi9lhe04W/FJO93LO X-Gm-Gg: AZuq6aLbthQF4Z6K74gcG+mFzpXjO5iymuL2Hh2cSA+PBSmvd+60UDv602bFB6sxriN OxMzasQQf3WckcE81RefGJcRnLmhVBZqVJATmMBz5AnhS68qVJYH2h4y55gcsDvexFvnFYbKbZn X9it3HUjRODu+76m8xjGjw9BfQDxMjch79F5gSgJV8jB9/98x1B+D90Mz3h0gOov0iLSNe3s169 SmmN7TKaSVbdzNuALCiw7RIQOyfJcgDcWH275/wdcciv8lkUBxeRCzkZ/myDUjcOJH0tjUWf4pM /R6i5TywPjBCDyQg3hz6Stxh3x7COZNemEi4p6Y9pZ1zOzBv3dInENIpo5bUU4uu08WndD/Ge4K 2TTjV5JLWGTQLmQ4whYEURasXlzm7RQ0V1H5kIJMIPYu3sB/yA5Fe/wEo4R9KrLKYXFV1yFyokd mnitNlU3ziTyd9TeTY/RdsUY4qdRB9+JgtNXLWIXSwGfwFTbrjXbaLaQ== X-Received: by 2002:a17:906:c154:b0:b87:6953:9d5e with SMTP id a640c23a62f3a-b8edf3411bdmr674611966b.33.1770655962985; Mon, 09 Feb 2026 08:52:42 -0800 (PST) From: Oleksii Kurochko To: xen-devel@lists.xenproject.org Cc: Romain Caritey , Oleksii Kurochko , Alistair Francis , Connor Davis , Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Stefano Stabellini Subject: [PATCH v3 05/16] xen/riscv: introduce tracking of pending vCPU interrupts, part 1 Date: Mon, 9 Feb 2026 17:52:18 +0100 Message-ID: X-Mailer: git-send-email 2.52.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1770656006899158500 Based on Linux kernel v6.16.0. Note that smp_wmb() is used instead of smp_mb__before_atomic() as what we want to guarantee that if a bit in irqs_pending_mask is obversable that the correspondent bit in irqs_pending is observable too. Add lockless tracking of pending vCPU interrupts using atomic bitops. Two bitmaps are introduced: - irqs_pending =E2=80=94 interrupts currently pending for the vCPU - irqs_pending_mask =E2=80=94 bits that have changed in irqs_pending The design follows a multi-producer, single-consumer model, where the consumer is the vCPU itself. Producers may set bits in irqs_pending_mask without a lock. Clearing bits in irqs_pending_mask is performed only by the consumer via xchg_acquire(). The consumer must not write to irqs_pending and must not act on bits that are not set in the mask. Otherwise, extra synchronization should be provided. On RISC-V interrupts are not injected via guest registers, so pending interrupts must be recorded in irqs_pending (using the new vcpu_{un}set_interrupt() helpers) and flushed to the guest by updating HVIP before returning control to the guest. The consumer side is implemented in a follow-up patch. A barrier between updating irqs_pending and setting the corresponding mask bit in vcpu_set_interrupt()/vcpu_unset_interrupt() guarantees that if the consumer observes a mask bit set, the corresponding pending bit is also visible. This prevents missed interrupts during the flush. It is possible a guest could have pending bit not result in the hardware register without to be marked pending in irq_pending bitmap as: According to the RISC-V ISA specification: Bits hip.VSSIP and hie.VSSIE are the interrupt-pending and interrupt-enable bits for VS-level software interrupts. VSSIP in hip is an alias (writable) of the same bit in hvip. Additionally: When bit 2 of hideleg is zero, vsip.SSIP and vsie.SSIE are read-only zeros. Else, vsip.SSIP and vsie.SSIE are aliases of hip.VSSIP and hie.VSSIE. This means the guest may modify vsip.SSIP, which implicitly updates hip.VSSIP and the bit being writable with 1 would also trigger an interrupt as according to the RISC-V spec: These conditions for an interrupt trap to occur must be evaluated in a bounded amount of time from when an interrupt becomes, or ceases to be, pending in sip, and must also be evaluated immediately following the execution of an SRET instruction or an explicit write to a CSR on which these interrupt trap conditions expressly depend (including sip, sie and sstatus). What means that IRQ_VS_SOFT must be synchronized separately, what is done in vcpu_sync_interrupts(). Note, also, that IRQ_PMU_OVF would want to be synced for the similar reason as IRQ_VS_SOFT, but isn't sync-ed now as PMU isn't supported now. For the remaining VS-level interrupt types (IRQ_VS_TIMER and IRQ_VS_EXT), the specification states they cannot be modified by the guest and are read-only: Bits hip.VSEIP and hie.VSEIE are the interrupt-pending and interrupt-enab= le bits for VS-level external interrupts. VSEIP is read-only in hip, and is the logical-OR of these interrupt sources: =E2=80=A2 bit VSEIP of hvip; =E2=80=A2 the bit of hgeip selected by hstatus.VGEIN; and =E2=80=A2 any other platform-specific external interrupt signal directe= d to VS-level. Bits hip.VSTIP and hie.VSTIE are the interrupt-pending and interrupt-enab= le bits for VS-level timer interrupts. VSTIP is read-only in hip, and is the logical-OR of hvip.VSTIP and any other platform-specific timer interrupt signal directed to VS-level. Thus, for these interrupt types, it is sufficient to use vcpu_set_interrupt= () and vcpu_unset_interrupt(), and flush them during the call of vcpu_flush_interrupts() (which is introduced in follow up patch). vcpu_sync_interrupts(), which is called just before entering the VM, slightly bends the rule that the irqs_pending bit must be written first, followed by updating the corresponding bit in irqs_pending_mask. However, it still respects the core guarantee that the producer never clears the mask and only writes to irqs_pending if it is the one that flipped the corresponding mask bit from 0 to 1. Moreover, since the consumer won't run concurrently because vcpu_sync_interrupts() and the consumer path are going to be invoked equentially immediately before VM entry, it is safe to slightly relax this ordering rule in vcpu_sync_interrupts(). Signed-off-by: Oleksii Kurochko --- Changes in v3: - Use smp_wb() instead of smp_mb__before_atomic(). - Add explanation of the change above in the commit message. - Move vcpu_sync_interrupts() here to producers side. - Introduce check_for_pcpu_work() to be clear from where vcpu_sync_interru= pts() is called. --- Changes in V2: - Move the patch before an introduction of vtimer. - Drop bitmap_zero() of irqs_pending and irqs_pending_mask bitmaps as vcpu structure starts out all zeros. - Drop const for irq argument of vcpu_{un}set_interrupt(). - Drop check "irq < IRQ_LOCAL_MAX" in vcpu_{un}set_interrupt() as it could lead to overrun of irqs_pending and irqs_pending_mask bitmaps. - Drop IRQ_LOCAL_MAX as there is no usage for it now. --- xen/arch/riscv/domain.c | 70 +++++++++++++++++++++++++++++ xen/arch/riscv/include/asm/domain.h | 24 ++++++++++ xen/arch/riscv/traps.c | 8 ++++ 3 files changed, 102 insertions(+) diff --git a/xen/arch/riscv/domain.c b/xen/arch/riscv/domain.c index af9586a4eb0d..4513f778cdc4 100644 --- a/xen/arch/riscv/domain.c +++ b/xen/arch/riscv/domain.c @@ -5,6 +5,7 @@ #include #include =20 +#include #include #include #include @@ -124,3 +125,72 @@ void arch_vcpu_destroy(struct vcpu *v) { vfree((char *)v->arch.cpu_info + sizeof(struct cpu_info) - STACK_SIZE); } + +int vcpu_set_interrupt(struct vcpu *v, unsigned int irq) +{ + /* + * We only allow VS-mode software, timer, and external + * interrupts when irq is one of the local interrupts + * defined by RISC-V privilege specification. + */ + if ( irq !=3D IRQ_VS_SOFT && + irq !=3D IRQ_VS_TIMER && + irq !=3D IRQ_VS_EXT ) + return -EINVAL; + + set_bit(irq, v->arch.irqs_pending); + smp_wmb(); + set_bit(irq, v->arch.irqs_pending_mask); + + vcpu_kick(v); + + return 0; +} + +int vcpu_unset_interrupt(struct vcpu *v, unsigned int irq) +{ + /* + * We only allow VS-mode software, timer, external + * interrupts when irq is one of the local interrupts + * defined by RISC-V privilege specification. + */ + if ( irq !=3D IRQ_VS_SOFT && + irq !=3D IRQ_VS_TIMER && + irq !=3D IRQ_VS_EXT ) + return -EINVAL; + + clear_bit(irq, v->arch.irqs_pending); + smp_wmb(); + set_bit(irq, v->arch.irqs_pending_mask); + + return 0; +} + +void vcpu_sync_interrupts(struct vcpu *v) +{ + unsigned long hvip; + + /* Read current HVIP and VSIE CSRs */ + v->arch.vsie =3D csr_read(CSR_VSIE); + + /* Sync-up HVIP.VSSIP bit changes done by Guest */ + hvip =3D csr_read(CSR_HVIP); + if ( (v->arch.hvip ^ hvip) & BIT(IRQ_VS_SOFT, UL) && + !test_and_set_bit(IRQ_VS_SOFT, &v->arch.irqs_pending_mask) ) + { + if ( hvip & BIT(IRQ_VS_SOFT, UL) ) + set_bit(IRQ_VS_SOFT, &v->arch.irqs_pending); + else + clear_bit(IRQ_VS_SOFT, &v->arch.irqs_pending); + } + + /* + * Sync-up AIA high interrupts. + * + * It is necessary to do only for CONFIG_RISCV_32 which isn't supported + * now. + */ +#ifdef CONFIG_RISCV_32 +# error "Update v->arch.vsieh" +#endif +} diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/a= sm/domain.h index 6bb06a50c6ab..8d9432ec5a8b 100644 --- a/xen/arch/riscv/include/asm/domain.h +++ b/xen/arch/riscv/include/asm/domain.h @@ -54,6 +54,25 @@ struct arch_vcpu { register_t henvcfg; register_t hideleg; register_t hstateen0; + register_t hvip; + + register_t vsie; + + /* + * VCPU interrupts + * + * We have a lockless approach for tracking pending VCPU interrupts + * implemented using atomic bitops. The irqs_pending bitmap represent + * pending interrupts whereas irqs_pending_mask represent bits changed + * in irqs_pending. Our approach is modeled around multiple producer + * and single consumer problem where the consumer is the VCPU itself. + * + * DECLARE_BITMAP() is needed here to support 64 vCPU local interrupts + * on RV32 host. + */ +#define RISCV_VCPU_NR_IRQS 64 + DECLARE_BITMAP(irqs_pending, RISCV_VCPU_NR_IRQS); + DECLARE_BITMAP(irqs_pending_mask, RISCV_VCPU_NR_IRQS); }; =20 struct paging_domain { @@ -92,6 +111,11 @@ static inline void update_guest_memory_policy(struct vc= pu *v, =20 static inline void arch_vcpu_block(struct vcpu *v) {} =20 +int vcpu_set_interrupt(struct vcpu *v, unsigned int irq); +int vcpu_unset_interrupt(struct vcpu *v, unsigned int irq); + +void vcpu_sync_interrupts(struct vcpu *v); + #endif /* ASM__RISCV__DOMAIN_H */ =20 /* diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c index c81a4f79a0d2..82e1dc59cdea 100644 --- a/xen/arch/riscv/traps.c +++ b/xen/arch/riscv/traps.c @@ -169,6 +169,11 @@ static void do_unexpected_trap(const struct cpu_user_r= egs *regs) die(); } =20 +static void check_for_pcpu_work(void) +{ + vcpu_sync_interrupts(current); +} + void do_trap(struct cpu_user_regs *cpu_regs) { register_t pc =3D cpu_regs->sepc; @@ -222,6 +227,9 @@ void do_trap(struct cpu_user_regs *cpu_regs) do_unexpected_trap(cpu_regs); break; } + + if ( cpu_regs->hstatus & HSTATUS_SPV ) + check_for_pcpu_work(); } =20 void vcpu_show_execution_state(struct vcpu *v) --=20 2.52.0