From nobody Thu Aug 27 01:43:43 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=arm.com ARC-Seal: i=1; a=rsa-sha256; t=1776433320; cv=none; d=zohomail.com; s=zohoarc; b=c7G/TYqPD8A57OtOBga1vbzl2/j6tazgiiTAywh5/tAeeTCuTKEEyvhgUT9+/KZH0cq5sBygsqwSY5sG6V5XfHMzW4dnNljXIAJfrRpLwInojiV9mAzxneSumaBL+NbLUFfh9x+PUE/wdYpr5KPfmldU1DefAPNObdO9AQ9XzfI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1776433320; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6DmxDjavy012+r4PHPlnGQhDkz68JtI2TOFLrty0LVs=; b=ePJ492tcgtM/fyaw5L46+v48gNturc6vVuUNdb+Op+7pJrZQ1KZ1M15iVKvPnziXfxY00oBy71qQqGRuCLTwYo8L57yvPsIu0gjf2N2RyNGowncJ1QkxqZxAabLWatooHYAgBWqG4I4nVVVTNCVPCLeFxiwXXeFV4wYq9Ymp3wI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1776433320062191.11831504740712; Fri, 17 Apr 2026 06:42:00 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1284310.1566162 (Exim 4.92) (envelope-from ) id 1wDjRz-0006cF-Rn; Fri, 17 Apr 2026 13:41:39 +0000 Received: by outflank-mailman (output) from mailman id 1284310.1566162; Fri, 17 Apr 2026 13:41:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wDjRz-0006b5-Lc; Fri, 17 Apr 2026 13:41:39 +0000 Received: by outflank-mailman (input) for mailman id 1284310; Fri, 17 Apr 2026 13:41:39 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wDjRy-0006Xe-SW for xen-devel@lists.xenproject.org; Fri, 17 Apr 2026 13:41:38 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wDjRw-000EHD-Ne for xen-devel@lists.xenproject.org; Fri, 17 Apr 2026 15:41:38 +0200 Received: from [10.42.69.3] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69e2388e-bab6-0a2a0a5309dd-0a2a4503ed30-10 for ; Fri, 17 Apr 2026 15:41:38 +0200 Received: from [217.140.110.172] (helo=foss.arm.com) by tlsNG-33051d.mxtls.expurgate.net with ESMTP (eXpurgate 4.56.1) (envelope-from ) id 69e23891-672d-0a2a45030019-d98c6eacd12a-1 for ; Fri, 17 Apr 2026 15:41:37 +0200 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 529231516; Fri, 17 Apr 2026 06:41:31 -0700 (PDT) Received: from C3HXLD123V.arm.com (unknown [10.57.89.170]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D6D373F7D8; Fri, 17 Apr 2026 06:41:35 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=foss header.d=arm.com header.i="@arm.com" header.h="From:To:Cc:Subject:Date:In-Reply-To:References" DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1776433297; bh=cZZGPI6vYtUzakFAvu0YBSmNdIu2H7Rzaign3ej2VlA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y1XalVlal1hWmaJVNRfQYNZEX63Jd6p5Jf9DVO7ivurdKHvPxODE93GmrQBpPyrXJ OjFOR1PtxYL6A0kjZi4T+J7W8qdXR8ejrNDRuD3yPJcJiPxo/NdQw8yIZ1tM/sLTsI bzCi6GUXv+iF0ls9DWhFLcmdMLQFkRYjyYCyo+jY= From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: Volodymyr Babchuk , Jens Wiklander , Stefano Stabellini , Julien Grall , Michal Orzel Subject: [PATCH 6/6] xen/arm: ffa: Deliver VM-to-VM notifications locally Date: Fri, 17 Apr 2026 15:40:54 +0200 Message-ID: X-Mailer: git-send-email 2.53.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-33051d/1776433298-2BB6C938-83D67431/0/0 X-purgate-type: clean X-purgate-size: 12215 X-ZohoMail-DKIM: pass (identity @arm.com) X-ZM-MESSAGEID: 1776433321559158500 Content-Type: text/plain; charset="utf-8" VM notification binding and pending tracking exist for non-secure endpoints, but FFA_NOTIFICATION_SET still only forwards secure destinations to the SPMC. Non-secure VMs therefore cannot receive notifications from other VMs. Local NPI delivery also needs explicit re-arm tracking so repeated raises are not lost while the interrupt is already pending. Add a local VM notification delivery path for non-secure destinations. notification_set_vm() resolves the destination endpoint, verifies that every requested bit is bound to the sender, sets the receiver's vm_pending bitmap under notif_lock, and raises an NPI only when the receiver transitions from no local pending notifications to some. Track whether a local NPI is already armed with notif_irq_raised, clear that state once both VM and hypervisor pending bitmaps are drained, and roll back newly-added VM pending bits if no destination vCPU is online. Also expose firmware notification availability so FFA_FEATURES only advertises notification support when it is actually provided by the firmware or by CONFIG_FFA_VM_TO_VM. Functional impact: when CONFIG_FFA_VM_TO_VM is enabled, non-secure FFA_NOTIFICATION_SET delivers VM-to-VM notifications locally and keeps NPI delivery reliable across repeated raises. Signed-off-by: Bertrand Marquis --- xen/arch/arm/tee/ffa.c | 24 +++++-- xen/arch/arm/tee/ffa_notif.c | 126 +++++++++++++++++++++++++++++++-- xen/arch/arm/tee/ffa_private.h | 11 ++- 3 files changed, 147 insertions(+), 14 deletions(-) diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c index 1fe33f26454a..7fe021049cba 100644 --- a/xen/arch/arm/tee/ffa.c +++ b/xen/arch/arm/tee/ffa.c @@ -39,8 +39,13 @@ * o FFA_MSG_SEND_DIRECT_REQ: * - only supported from a VM to an SP * o FFA_NOTIFICATION_*: + * - only supported when firmware notifications are enabled or VM-to-VM + * support is built in * - only supports global notifications, that is, per vCPU notifications - * are not supported + * are not supported and secure per-vCPU notification information is + * not forwarded + * - the source endpoint ID reported for a notification may no longer + * exist by the time the receiver consumes it * - doesn't support signalling the secondary scheduler of pending * notification for secure partitions * - doesn't support notifications for Xen itself @@ -245,6 +250,8 @@ static void handle_features(struct cpu_user_regs *regs) uint32_t a1 =3D get_user_reg(regs, 1); struct domain *d =3D current->domain; struct ffa_ctx *ctx =3D d->arch.tee; + bool notif_supported =3D IS_ENABLED(CONFIG_FFA_VM_TO_VM) || + ffa_notif_fw_enabled(); =20 /* * FFA_FEATURES defines w2 as input properties only for specific @@ -343,10 +350,16 @@ static void handle_features(struct cpu_user_regs *reg= s) =20 break; case FFA_FEATURE_NOTIF_PEND_INTR: - ffa_set_regs_success(regs, GUEST_FFA_NOTIF_PEND_INTR_ID, 0); + if ( notif_supported ) + ffa_set_regs_success(regs, GUEST_FFA_NOTIF_PEND_INTR_ID, 0); + else + ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED); break; case FFA_FEATURE_SCHEDULE_RECV_INTR: - ffa_set_regs_success(regs, GUEST_FFA_SCHEDULE_RECV_INTR_ID, 0); + if ( notif_supported ) + ffa_set_regs_success(regs, GUEST_FFA_SCHEDULE_RECV_INTR_ID, 0); + else + ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED); break; case FFA_PARTITION_INFO_GET_REGS: if ( ACCESS_ONCE(ctx->guest_vers) >=3D FFA_VERSION_1_2 ) @@ -361,7 +374,10 @@ static void handle_features(struct cpu_user_regs *regs) case FFA_NOTIFICATION_SET: case FFA_NOTIFICATION_INFO_GET_32: case FFA_NOTIFICATION_INFO_GET_64: - ffa_set_regs_success(regs, 0, 0); + if ( notif_supported ) + ffa_set_regs_success(regs, 0, 0); + else + ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED); break; default: ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED); diff --git a/xen/arch/arm/tee/ffa_notif.c b/xen/arch/arm/tee/ffa_notif.c index 4def701f0130..e77321244926 100644 --- a/xen/arch/arm/tee/ffa_notif.c +++ b/xen/arch/arm/tee/ffa_notif.c @@ -20,7 +20,12 @@ static bool __ro_after_init fw_notif_enabled; static unsigned int __ro_after_init notif_sri_irq; static DEFINE_SPINLOCK(notif_info_lock); =20 -static void inject_notif_pending(struct domain *d) +bool ffa_notif_fw_enabled(void) +{ + return fw_notif_enabled; +} + +static bool inject_notif_pending(struct domain *d) { struct vcpu *v; =20 @@ -34,13 +39,15 @@ static void inject_notif_pending(struct domain *d) if ( is_vcpu_online(v) ) { vgic_inject_irq(d, v, GUEST_FFA_NOTIF_PEND_INTR_ID, true); - return; + return true; } } =20 if ( printk_ratelimit() ) printk(XENLOG_G_DEBUG "%pd: ffa: can't inject NPI, all vCPUs offli= ne\n", d); + + return false; } =20 static int32_t ffa_notif_parse_params(uint16_t dom_id, uint16_t caller_id, @@ -104,6 +111,73 @@ out_unlock: return ret; } =20 +/* + * Deliver a VM-to-VM notification. ctx->notif.notif_lock protects + * vm_bind/vm_pending so callers must not hold it already. + */ +static int32_t notification_set_vm(uint16_t dst_id, uint16_t src_id, + uint32_t flags, uint64_t bitmap) +{ + struct domain *dst_d; + struct ffa_ctx *dst_ctx; + unsigned int id; + int32_t ret; + uint64_t prev_bitmap =3D 0; + uint64_t new_bitmap; + bool inject =3D false; + + if ( flags ) + return FFA_RET_INVALID_PARAMETERS; + + ret =3D ffa_endpoint_domain_lookup(dst_id, &dst_d, &dst_ctx); + if ( ret ) + return ret; + + ret =3D FFA_RET_OK; + + spin_lock(&dst_ctx->notif.notif_lock); + + for ( id =3D 0; id < FFA_NUM_VM_NOTIF; id++ ) + { + if ( !(bitmap & BIT(id, ULL)) ) + continue; + + if ( dst_ctx->notif.vm_bind[id] !=3D src_id ) + { + ret =3D FFA_RET_DENIED; + goto out_unlock; + } + } + + prev_bitmap =3D dst_ctx->notif.vm_pending; + dst_ctx->notif.vm_pending |=3D bitmap; + if ( !dst_ctx->notif.notif_irq_raised && + (dst_ctx->notif.vm_pending || dst_ctx->notif.hyp_pending) ) + { + dst_ctx->notif.notif_irq_raised =3D true; + inject =3D true; + } + +out_unlock: + spin_unlock(&dst_ctx->notif.notif_lock); + + new_bitmap =3D bitmap & ~prev_bitmap; + if ( ret =3D=3D FFA_RET_OK && inject && new_bitmap && + !inject_notif_pending(dst_d) ) + { + spin_lock(&dst_ctx->notif.notif_lock); + dst_ctx->notif.vm_pending &=3D ~new_bitmap; + if ( !(dst_ctx->notif.vm_pending || dst_ctx->notif.hyp_pending) ) + dst_ctx->notif.notif_irq_raised =3D false; + spin_unlock(&dst_ctx->notif.notif_lock); + ret =3D FFA_RET_DENIED; + } + + rcu_unlock_domain(dst_d); + + return ret; +} + int32_t ffa_handle_notification_bind(struct cpu_user_regs *regs) { struct domain *d =3D current->domain; @@ -285,6 +359,8 @@ void ffa_handle_notification_get(struct cpu_user_regs *= regs) =20 if ( IS_ENABLED(CONFIG_FFA_VM_TO_VM) ) { + bool pending; + spin_lock(&ctx->notif.notif_lock); =20 if ( (flags & FFA_NOTIF_FLAG_BITMAP_HYP) && ctx->notif.hyp_pending= ) @@ -293,6 +369,18 @@ void ffa_handle_notification_get(struct cpu_user_regs = *regs) ctx->notif.hyp_pending =3D 0; } =20 + if ( (flags & FFA_NOTIF_FLAG_BITMAP_VM) && ctx->notif.vm_pending ) + { + w4 =3D (uint32_t)(ctx->notif.vm_pending & GENMASK(31, 0)); + w5 =3D (uint32_t)((ctx->notif.vm_pending >> 32) & GENMASK(31, = 0)); + ctx->notif.vm_pending =3D 0; + } + + pending =3D (ctx->notif.hyp_pending !=3D 0) || + (ctx->notif.vm_pending !=3D 0); + if ( !pending ) + ctx->notif.notif_irq_raised =3D false; + spin_unlock(&ctx->notif.notif_lock); } =20 @@ -318,9 +406,17 @@ int32_t ffa_handle_notification_set(struct cpu_user_re= gs *regs) if ( flags ) return FFA_RET_INVALID_PARAMETERS; =20 - if ( FFA_ID_IS_SECURE(dest_id) && fw_notif_enabled ) - return ffa_simple_call(FFA_NOTIFICATION_SET, src_dst, flags, bitma= p_lo, - bitmap_hi); + if ( FFA_ID_IS_SECURE(dest_id) ) + { + if ( fw_notif_enabled ) + return ffa_simple_call(FFA_NOTIFICATION_SET, src_dst, flags, + bitmap_lo, bitmap_hi); + } + else if ( IS_ENABLED(CONFIG_FFA_VM_TO_VM) ) + { + return notification_set_vm(dest_id, caller_id, flags, + ((uint64_t)bitmap_hi << 32) | bitmap_lo= ); + } =20 return FFA_RET_NOT_SUPPORTED; } @@ -330,6 +426,7 @@ void ffa_raise_rx_buffer_full(struct domain *d) { struct ffa_ctx *ctx =3D d->arch.tee; uint32_t prev_bitmap; + bool inject =3D false; =20 if ( !ctx ) return; @@ -337,10 +434,23 @@ void ffa_raise_rx_buffer_full(struct domain *d) spin_lock(&ctx->notif.notif_lock); prev_bitmap =3D ctx->notif.hyp_pending; ctx->notif.hyp_pending |=3D FFA_NOTIF_RX_BUFFER_FULL; + if ( !ctx->notif.notif_irq_raised && + (ctx->notif.vm_pending || ctx->notif.hyp_pending) ) + { + ctx->notif.notif_irq_raised =3D true; + inject =3D true; + } spin_unlock(&ctx->notif.notif_lock); =20 - if ( !(prev_bitmap & FFA_NOTIF_RX_BUFFER_FULL) ) - inject_notif_pending(d); + if ( inject && !(prev_bitmap & FFA_NOTIF_RX_BUFFER_FULL) && + !inject_notif_pending(d) ) + { + spin_lock(&ctx->notif.notif_lock); + ctx->notif.hyp_pending &=3D ~FFA_NOTIF_RX_BUFFER_FULL; + if ( !(ctx->notif.vm_pending || ctx->notif.hyp_pending) ) + ctx->notif.notif_irq_raised =3D false; + spin_unlock(&ctx->notif.notif_lock); + } } #endif =20 @@ -572,6 +682,7 @@ int ffa_notif_domain_init(struct domain *d) spin_lock_init(&ctx->notif.notif_lock); ctx->notif.secure_pending =3D false; ctx->notif.vm_pending =3D 0; + ctx->notif.notif_irq_raised =3D false; for ( i =3D 0; i < FFA_NUM_VM_NOTIF; i++ ) ctx->notif.vm_bind[i] =3D 0; ctx->notif.hyp_pending =3D 0; @@ -594,6 +705,7 @@ void ffa_notif_domain_destroy(struct domain *d) spin_lock(&ctx->notif.notif_lock); ctx->notif.secure_pending =3D false; ctx->notif.vm_pending =3D 0; + ctx->notif.notif_irq_raised =3D false; for ( i =3D 0; i < FFA_NUM_VM_NOTIF; i++ ) ctx->notif.vm_bind[i] =3D 0; ctx->notif.hyp_pending =3D 0; diff --git a/xen/arch/arm/tee/ffa_private.h b/xen/arch/arm/tee/ffa_private.h index 6d83afb3d00a..5bb19bd11dd0 100644 --- a/xen/arch/arm/tee/ffa_private.h +++ b/xen/arch/arm/tee/ffa_private.h @@ -344,13 +344,17 @@ struct ffa_ctx_notif { uint64_t vm_pending; =20 /* - * Source endpoint bound to each VM notification ID (0 means unbound). + * Tracks whether an NPI has been raised for local pending notificatio= ns. + * Protected by notif_lock. */ - uint16_t vm_bind[FFA_NUM_VM_NOTIF]; + bool notif_irq_raised; =20 /* - * Lock protecting the hypervisor-managed notification state. + * Source endpoint bound to each VM notification ID (0 means unbound). */ + uint16_t vm_bind[FFA_NUM_VM_NOTIF]; + + /* Lock protecting local notification state. */ spinlock_t notif_lock; =20 /* @@ -493,6 +497,7 @@ void ffa_notif_init(void); void ffa_notif_init_interrupt(void); int ffa_notif_domain_init(struct domain *d); void ffa_notif_domain_destroy(struct domain *d); +bool ffa_notif_fw_enabled(void); =20 int32_t ffa_handle_notification_bind(struct cpu_user_regs *regs); int32_t ffa_handle_notification_unbind(struct cpu_user_regs *regs); --=20 2.53.0