From nobody Tue Apr 16 15:56:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1573220140; cv=none; d=zoho.com; s=zohoarc; b=VNRNvmy3cz5DOKWJDRGrvdLZYvPp47BxXlXfv/ChoH3rpJIm9LZlgTEj9MrdkaxmP7FGSHjuVxJQJTfApa2zb7KKEAE7MfRXNs34XlQ/YWEsJUPC2AOlWOQeG4oOE+vi3uozKCwTlAfCMXLaVWC+7TvWHCDdcXRzQ+PpnIp3/CM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573220140; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=MiDFH9WTRLsbPB0Lm9MYI5XeAHAk9+zZVPtf4J8p/NA=; b=a3LZxnaLkucF4/FnejWQCNrQyAzLYaPh1hIxnFECNkcwRngcRxY9Ng1jDP3UQ4HPoqQsdq9USc82r3VNJqYXDbWDPG2onS2TdUWpM0fm7ygJbwdNH0aQZzAUNWej42v3ng/8lfhcUIV7iIP4aX9zKanNGo4WTVQjF7B9EVC8k64= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1573220140585321.34533363422065; Fri, 8 Nov 2019 05:35:40 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iT4PA-0004yF-R2; Fri, 08 Nov 2019 13:34:24 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iT4P9-0004y9-CB for xen-devel@lists.xenproject.org; Fri, 08 Nov 2019 13:34:23 +0000 Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78da6adc-022c-11ea-b678-bc764e2007e4; Fri, 08 Nov 2019 13:34:22 +0000 (UTC) X-Inumbo-ID: 78da6adc-022c-11ea-b678-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1573220062; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=UDbtv6wKaAC+VWFlETktGepmIbPLuOwmMrXAYKyyXRk=; b=HIP5I1GvTJ8e18CYPoTcXh4yEwId972pcOcufsqKm7+Xr1jNeCCnQRkh CrxB3Tze9k/gHhk7K30sOP88f0tQQzwPXALwg6hyQASmQg4Ek4abIEB7l c1IkgCwW88PjARDh/nzOdM9KvfaDQLlHMrF4LeAwFJPa0i3hlq2tvOkGi 4=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa1.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: z7QkHblNNZ2euVJSQgNBev1L7eqWfJjddGtVM8OCnCmBgKH7zFPumBxQw9dKaop5pIfYRiTL0O /U1nQrv2hpnuJfxFs7NlERKCtbJMMrOxfVvkKuaiohjnyVgQWsuh+ZWplJ+jGX1E8HZy+uqKhR 9EKw2eOTFt/KqcVr4tJzbXS4PO4iJX6xo1EAbOuvmm62UT20CmGqO8IqhWmCQ7x4R306s2nwgz b4421S+Lxppa47qQ4zR+pB5tzGYAsazEDN0Zkb3aMKLbQMEaAkz/N9aHxQ/XmLagQfESvMjw+B DFM= X-SBRS: 2.7 X-MesageID: 8168072 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.68,281,1569297600"; d="scan'208";a="8168072" From: Roger Pau Monne To: Date: Fri, 8 Nov 2019 14:34:14 +0100 Message-ID: <20191108133414.96381-1-roger.pau@citrix.com> X-Mailer: git-send-email 2.23.0 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH for-4.13 v3] x86/passthrough: fix migration of MSI when using posted interrupts X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Kevin Tian , Wei Liu , Andrew Cooper , Joe Jin , Jan Beulich , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) When using posted interrupts and the guest migrates MSI from vCPUs Xen needs to flush any pending PIRR vectors on the previous vCPU, or else those vectors could get wrongly injected at a later point when the MSI fields are already updated. Rename sync_pir_to_irr to vlapic_sync_pir_to_irr and export it so it can be called when updating the binding of physical interrupts to guests. Note that PIRR is synced to IRR both in pt_irq_destroy_bind and pt_irq_create_bind when the interrupt delivery data is being updated. Also store the vCPU ID in multi-destination mode when using posted interrupts so that the interrupt is always injected to a known vCPU in order to be able to flush the PIRR when modifying the binding. Reported-by: Joe Jin Signed-off-by: Roger Pau Monn=C3=A9 Tested-by: Joe Jin --- Cc: Joe Jin Cc: Juergen Gross --- I would like to see a bug fix for this issue in 4.13. The fix here only affects posted interrupts, hence I think the risk of breaking anything else is low. --- Changes since v2: - Also sync PIRR with IRR when using CPU posted interrupts. - Force the selection of a specific vCPU when using posted interrupts for multi-dest. - Change vmsi_deliver_pirq to honor dest_vcpu_id. Changes since v1: - Store the vcpu id also in multi-dest mode if the interrupt is bound to a vcpu for posted delivery. - s/#if/#ifdef/. --- xen/arch/x86/hvm/vlapic.c | 6 +++--- xen/arch/x86/hvm/vmsi.c | 11 ++++++++++- xen/drivers/passthrough/io.c | 29 +++++++++++++++++++++++------ xen/include/asm-x86/hvm/vlapic.h | 2 ++ 4 files changed, 38 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c index 9466258d6f..d255ad8db7 100644 --- a/xen/arch/x86/hvm/vlapic.c +++ b/xen/arch/x86/hvm/vlapic.c @@ -106,7 +106,7 @@ static void vlapic_clear_irr(int vector, struct vlapic = *vlapic) vlapic_clear_vector(vector, &vlapic->regs->data[APIC_IRR]); } =20 -static void sync_pir_to_irr(struct vcpu *v) +void vlapic_sync_pir_to_irr(struct vcpu *v) { if ( hvm_funcs.sync_pir_to_irr ) alternative_vcall(hvm_funcs.sync_pir_to_irr, v); @@ -114,7 +114,7 @@ static void sync_pir_to_irr(struct vcpu *v) =20 static int vlapic_find_highest_irr(struct vlapic *vlapic) { - sync_pir_to_irr(vlapic_vcpu(vlapic)); + vlapic_sync_pir_to_irr(vlapic_vcpu(vlapic)); =20 return vlapic_find_highest_vector(&vlapic->regs->data[APIC_IRR]); } @@ -1493,7 +1493,7 @@ static int lapic_save_regs(struct vcpu *v, hvm_domain= _context_t *h) if ( !has_vlapic(v->domain) ) return 0; =20 - sync_pir_to_irr(v); + vlapic_sync_pir_to_irr(v); =20 return hvm_save_entry(LAPIC_REGS, v->vcpu_id, h, vcpu_vlapic(v)->regs); } diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c index 6597d9f719..fe488ccc7d 100644 --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -118,7 +118,16 @@ void vmsi_deliver_pirq(struct domain *d, const struct = hvm_pirq_dpci *pirq_dpci) =20 ASSERT(pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI); =20 - vmsi_deliver(d, vector, dest, dest_mode, delivery_mode, trig_mode); + if ( hvm_funcs.deliver_posted_intr && pirq_dpci->gmsi.dest_vcpu_id != =3D -1 ) + /* + * When using posted interrupts multi-destination delivery mode is + * forced to select a specific vCPU so that the PIRR can be synced= into + * IRR when the interrupt is destroyed or moved. + */ + vmsi_inj_irq(vcpu_vlapic(d->vcpu[pirq_dpci->gmsi.dest_vcpu_id]), + vector, trig_mode, delivery_mode); + else + vmsi_deliver(d, vector, dest, dest_mode, delivery_mode, trig_mode); } =20 /* Return value, -1 : multi-dests, non-negative value: dest_vcpu_id */ diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c index b292e79382..d3f1ae5c39 100644 --- a/xen/drivers/passthrough/io.c +++ b/xen/drivers/passthrough/io.c @@ -341,7 +341,7 @@ int pt_irq_create_bind( { uint8_t dest, delivery_mode; bool dest_mode; - int dest_vcpu_id; + int dest_vcpu_id, prev_vcpu_id =3D -1; const struct vcpu *vcpu; uint32_t gflags =3D pt_irq_bind->u.msi.gflags & ~XEN_DOMCTL_VMSI_X86_UNMASKED; @@ -411,6 +411,7 @@ int pt_irq_create_bind( =20 pirq_dpci->gmsi.gvec =3D pt_irq_bind->u.msi.gvec; pirq_dpci->gmsi.gflags =3D gflags; + prev_vcpu_id =3D pirq_dpci->gmsi.dest_vcpu_id; } } /* Calculate dest_vcpu_id for MSI-type pirq migration. */ @@ -426,14 +427,24 @@ int pt_irq_create_bind( =20 pirq_dpci->gmsi.posted =3D false; vcpu =3D (dest_vcpu_id >=3D 0) ? d->vcpu[dest_vcpu_id] : NULL; - if ( iommu_intpost ) + if ( hvm_funcs.deliver_posted_intr && delivery_mode =3D=3D dest_Lo= westPrio ) { - if ( delivery_mode =3D=3D dest_LowestPrio ) - vcpu =3D vector_hashing_dest(d, dest, dest_mode, - pirq_dpci->gmsi.gvec); + /* + * NB: when using posted interrupts the vector is signaled + * on the PIRR, and hence Xen needs to force interrupts to be + * delivered to a specific vCPU in order to be able to sync PI= RR + * with IRR when the interrupt binding is destroyed, or else + * pending interrupts in the previous vCPU PIRR field could be + * delivered after the update. + */ + vcpu =3D vector_hashing_dest(d, dest, dest_mode, + pirq_dpci->gmsi.gvec); if ( vcpu ) - pirq_dpci->gmsi.posted =3D true; + pirq_dpci->gmsi.dest_vcpu_id =3D vcpu->vcpu_id; } + if ( iommu_intpost && vcpu ) + pirq_dpci->gmsi.posted =3D true; + if ( vcpu && is_iommu_enabled(d) ) hvm_migrate_pirq(pirq_dpci, vcpu); =20 @@ -442,6 +453,9 @@ int pt_irq_create_bind( pi_update_irte(vcpu ? &vcpu->arch.hvm.vmx.pi_desc : NULL, info, pirq_dpci->gmsi.gvec); =20 + if ( hvm_funcs.deliver_posted_intr && prev_vcpu_id >=3D 0 ) + vlapic_sync_pir_to_irr(d->vcpu[prev_vcpu_id]); + if ( pt_irq_bind->u.msi.gflags & XEN_DOMCTL_VMSI_X86_UNMASKED ) { unsigned long flags; @@ -731,6 +745,9 @@ int pt_irq_destroy_bind( else if ( pirq_dpci && pirq_dpci->gmsi.posted ) pi_update_irte(NULL, pirq, 0); =20 + if ( hvm_funcs.deliver_posted_intr && pirq_dpci->gmsi.dest_vcpu_id >= =3D 0 ) + vlapic_sync_pir_to_irr(d->vcpu[pirq_dpci->gmsi.dest_vcpu_id]); + if ( pirq_dpci && (pirq_dpci->flags & HVM_IRQ_DPCI_MAPPED) && list_empty(&pirq_dpci->digl_list) ) { diff --git a/xen/include/asm-x86/hvm/vlapic.h b/xen/include/asm-x86/hvm/vla= pic.h index dde66b4f0f..b0017d1dae 100644 --- a/xen/include/asm-x86/hvm/vlapic.h +++ b/xen/include/asm-x86/hvm/vlapic.h @@ -150,4 +150,6 @@ bool_t vlapic_match_dest( const struct vlapic *target, const struct vlapic *source, int short_hand, uint32_t dest, bool_t dest_mode); =20 +void vlapic_sync_pir_to_irr(struct vcpu *v); + #endif /* __ASM_X86_HVM_VLAPIC_H__ */ --=20 2.23.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel