From nobody Sat Apr 27 23:58:52 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 151073132382991.9561422006177; Tue, 14 Nov 2017 23:35:23 -0800 (PST) Received: from localhost ([::1]:34990 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEsEA-0005Zb-30 for importer@patchew.org; Wed, 15 Nov 2017 02:35:18 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39628) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEsAu-0003Zu-SH for qemu-devel@nongnu.org; Wed, 15 Nov 2017 02:31:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEsAt-0005i6-1c for qemu-devel@nongnu.org; Wed, 15 Nov 2017 02:31:56 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59750) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEsAs-0005ho-Ru for qemu-devel@nongnu.org; Wed, 15 Nov 2017 02:31:54 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1B5884E33E; Wed, 15 Nov 2017 07:31:54 +0000 (UTC) Received: from dhcp-1-107.brq.redhat.com (ovpn-204-61.brq.redhat.com [10.40.204.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id A3BBD121B84; Wed, 15 Nov 2017 07:31:51 +0000 (UTC) From: Ladi Prosek To: qemu-devel@nongnu.org Date: Wed, 15 Nov 2017 08:31:42 +0100 Message-Id: <20171115073144.14758-2-lprosek@redhat.com> In-Reply-To: <20171115073144.14758-1-lprosek@redhat.com> References: <20171115073144.14758-1-lprosek@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 15 Nov 2017 07:31:54 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 1/3] ivshmem: Don't update non-existent MSI routes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: geoff@hostfission.com, pbonzini@redhat.com, armbru@redhat.com, marcandre.lureau@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" As of commit 660c97eef6f8 ("ivshmem: use kvm irqfd for msi notifications"), QEMU crashes with: kvm_irqchip_commit_routes: Assertion `ret =3D=3D 0' failed. if the ivshmem device is configured with more vectors than what the server supports. This is caused by the ivshmem_vector_unmask() being called on vectors that have not been initialized by ivshmem_add_kvm_msi_virq(). This commit fixes it by adding a simple check to the mask and unmask callbacks. Note that the opposite mismatch, if the server supplies more vectors than what the device is configured for, is already handled and leads to output like: Too many eventfd received, device has 1 vectors To reproduce the assert, run: ivshmem-server -n 0 and QEMU with: -device ivshmem-doorbell,chardev=3Div -chardev socket,path=3D/tmp/ivshmem_socket,id=3Div then load the Windows driver, at the time of writing available at: https://github.com/virtio-win/kvm-guest-drivers-windows/tree/master/ivshmem The issue is believed to have been masked by other guest drivers, notably Linux ones, not enabling MSI-X on the device. Fixes: 660c97eef6f8 ("ivshmem: use kvm irqfd for msi notifications") Signed-off-by: Ladi Prosek Reviewed-by: Marc-Andr=C3=A9 Lureau Reviewed-by: Markus Armbruster --- hw/misc/ivshmem.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c index a5a46827fe..6e46669744 100644 --- a/hw/misc/ivshmem.c +++ b/hw/misc/ivshmem.c @@ -317,6 +317,10 @@ static int ivshmem_vector_unmask(PCIDevice *dev, unsig= ned vector, int ret; =20 IVSHMEM_DPRINTF("vector unmask %p %d\n", dev, vector); + if (!v->pdev) { + error_report("ivshmem: vector %d route does not exist", vector); + return -EINVAL; + } =20 ret =3D kvm_irqchip_update_msi_route(kvm_state, v->virq, msg, dev); if (ret < 0) { @@ -331,12 +335,16 @@ static void ivshmem_vector_mask(PCIDevice *dev, unsig= ned vector) { IVShmemState *s =3D IVSHMEM_COMMON(dev); EventNotifier *n =3D &s->peers[s->vm_id].eventfds[vector]; + MSIVector *v =3D &s->msi_vectors[vector]; int ret; =20 IVSHMEM_DPRINTF("vector mask %p %d\n", dev, vector); + if (!v->pdev) { + error_report("ivshmem: vector %d route does not exist", vector); + return; + } =20 - ret =3D kvm_irqchip_remove_irqfd_notifier_gsi(kvm_state, n, - s->msi_vectors[vector].vir= q); + ret =3D kvm_irqchip_remove_irqfd_notifier_gsi(kvm_state, n, v->virq); if (ret !=3D 0) { error_report("remove_irqfd_notifier_gsi failed"); } --=20 2.13.5 From nobody Sat Apr 27 23:58:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510731236716327.7281043130199; Tue, 14 Nov 2017 23:33:56 -0800 (PST) Received: from localhost ([::1]:34987 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEsCf-0004PK-2S for importer@patchew.org; Wed, 15 Nov 2017 02:33:45 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39644) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEsAv-0003Zw-PY for qemu-devel@nongnu.org; Wed, 15 Nov 2017 02:31:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEsAu-0005iw-Q1 for qemu-devel@nongnu.org; Wed, 15 Nov 2017 02:31:57 -0500 Received: from mx1.redhat.com ([209.132.183.28]:47394) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEsAu-0005iR-Gh for qemu-devel@nongnu.org; Wed, 15 Nov 2017 02:31:56 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AA39FC0587EA; Wed, 15 Nov 2017 07:31:55 +0000 (UTC) Received: from dhcp-1-107.brq.redhat.com (ovpn-204-61.brq.redhat.com [10.40.204.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B36F121E32; Wed, 15 Nov 2017 07:31:54 +0000 (UTC) From: Ladi Prosek To: qemu-devel@nongnu.org Date: Wed, 15 Nov 2017 08:31:43 +0100 Message-Id: <20171115073144.14758-3-lprosek@redhat.com> In-Reply-To: <20171115073144.14758-1-lprosek@redhat.com> References: <20171115073144.14758-1-lprosek@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 15 Nov 2017 07:31:55 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 2/3] ivshmem: Always remove irqfd notifiers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: geoff@hostfission.com, pbonzini@redhat.com, armbru@redhat.com, marcandre.lureau@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" As of commit 660c97eef6f8 ("ivshmem: use kvm irqfd for msi notifications"), QEMU crashes with: ivshmem: msix_set_vector_notifiers failed msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && de= v->msix_vector_release_notifier' failed. if MSI-X is repeatedly enabled and disabled on the ivshmem device, for exam= ple by loading and unloading the Windows ivshmem driver. This is because msix_unset_vector_notifiers() doesn't call any of the release notifier call= backs since MSI-X is already disabled at that point (msix_enabled() returning fal= se is how this transition is detected in the first place). Thus ivshmem_vector= _mask() doesn't run and when MSI-X is subsequently enabled again ivshmem_vector_unm= ask() fails. This is fixed by keeping track of unmasked vectors and making sure that ivshmem_vector_mask() always runs on MSI-X disable. Fixes: 660c97eef6f8 ("ivshmem: use kvm irqfd for msi notifications") Signed-off-by: Ladi Prosek Reviewed-by: Markus Armbruster --- hw/misc/ivshmem.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c index 6e46669744..91364d8364 100644 --- a/hw/misc/ivshmem.c +++ b/hw/misc/ivshmem.c @@ -77,6 +77,7 @@ typedef struct Peer { typedef struct MSIVector { PCIDevice *pdev; int virq; + bool unmasked; } MSIVector; =20 typedef struct IVShmemState { @@ -321,6 +322,7 @@ static int ivshmem_vector_unmask(PCIDevice *dev, unsign= ed vector, error_report("ivshmem: vector %d route does not exist", vector); return -EINVAL; } + assert(!v->unmasked); =20 ret =3D kvm_irqchip_update_msi_route(kvm_state, v->virq, msg, dev); if (ret < 0) { @@ -328,7 +330,13 @@ static int ivshmem_vector_unmask(PCIDevice *dev, unsig= ned vector, } kvm_irqchip_commit_routes(kvm_state); =20 - return kvm_irqchip_add_irqfd_notifier_gsi(kvm_state, n, NULL, v->virq); + ret =3D kvm_irqchip_add_irqfd_notifier_gsi(kvm_state, n, NULL, v->virq= ); + if (ret < 0) { + return ret; + } + v->unmasked =3D true; + + return 0; } =20 static void ivshmem_vector_mask(PCIDevice *dev, unsigned vector) @@ -343,11 +351,14 @@ static void ivshmem_vector_mask(PCIDevice *dev, unsig= ned vector) error_report("ivshmem: vector %d route does not exist", vector); return; } + assert(v->unmasked); =20 ret =3D kvm_irqchip_remove_irqfd_notifier_gsi(kvm_state, n, v->virq); - if (ret !=3D 0) { + if (ret < 0) { error_report("remove_irqfd_notifier_gsi failed"); + return; } + v->unmasked =3D false; } =20 static void ivshmem_vector_poll(PCIDevice *dev, @@ -817,11 +828,20 @@ static void ivshmem_disable_irqfd(IVShmemState *s) PCIDevice *pdev =3D PCI_DEVICE(s); int i; =20 - for (i =3D 0; i < s->peers[s->vm_id].nb_eventfds; i++) { - ivshmem_remove_kvm_msi_virq(s, i); - } - msix_unset_vector_notifiers(pdev); + + for (i =3D 0; i < s->peers[s->vm_id].nb_eventfds; i++) { + /* + * MSI-X is already disabled here so msix_unset_vector_notifiers() + * didn't call our release notifier. Do it now to keep our masks = and + * unmasks balanced. + */ + if (s->msi_vectors[i].unmasked) { + ivshmem_vector_mask(pdev, i); + } + ivshmem_remove_kvm_msi_virq(s, i); + } + } =20 static void ivshmem_write_config(PCIDevice *pdev, uint32_t address, --=20 2.13.5 From nobody Sat Apr 27 23:58:52 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510731238301376.5510245865996; Tue, 14 Nov 2017 23:33:58 -0800 (PST) Received: from localhost ([::1]:34988 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEsCk-0004S6-Eu for importer@patchew.org; Wed, 15 Nov 2017 02:33:50 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39662) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEsAx-0003aR-V7 for qemu-devel@nongnu.org; Wed, 15 Nov 2017 02:32:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEsAw-0005lc-Vy for qemu-devel@nongnu.org; Wed, 15 Nov 2017 02:31:59 -0500 Received: from mx1.redhat.com ([209.132.183.28]:47258) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEsAw-0005kB-NJ for qemu-devel@nongnu.org; Wed, 15 Nov 2017 02:31:58 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C71A283F44; Wed, 15 Nov 2017 07:31:57 +0000 (UTC) Received: from dhcp-1-107.brq.redhat.com (ovpn-204-61.brq.redhat.com [10.40.204.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id 04DC0121E2C; Wed, 15 Nov 2017 07:31:55 +0000 (UTC) From: Ladi Prosek To: qemu-devel@nongnu.org Date: Wed, 15 Nov 2017 08:31:44 +0100 Message-Id: <20171115073144.14758-4-lprosek@redhat.com> In-Reply-To: <20171115073144.14758-1-lprosek@redhat.com> References: <20171115073144.14758-1-lprosek@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 15 Nov 2017 07:31:57 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 3/3] ivshmem: Improve MSI irqfd error handling X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: geoff@hostfission.com, pbonzini@redhat.com, armbru@redhat.com, marcandre.lureau@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Adds a rollback path to ivshmem_enable_irqfd() and fixes ivshmem_disable_irqfd() to bail if irqfd has not been enabled. To reproduce, run: ivshmem-server -n 0 and QEMU with: -device ivshmem-doorbell,chardev=3Div -chardev socket,path=3D/tmp/ivshmem_socket,id=3Div then load, unload, and load again the Windows driver, at the time of writing available at: https://github.com/virtio-win/kvm-guest-drivers-windows/tree/master/ivshmem The issue is believed to have been masked by other guest drivers, notably Linux ones, not enabling MSI-X on the device. Signed-off-by: Ladi Prosek Reviewed-by: Markus Armbruster --- hw/misc/ivshmem.c | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c index 91364d8364..d1bb246d12 100644 --- a/hw/misc/ivshmem.c +++ b/hw/misc/ivshmem.c @@ -786,6 +786,20 @@ static int ivshmem_setup_interrupts(IVShmemState *s, E= rror **errp) return 0; } =20 +static void ivshmem_remove_kvm_msi_virq(IVShmemState *s, int vector) +{ + IVSHMEM_DPRINTF("ivshmem_remove_kvm_msi_virq vector:%d\n", vector); + + if (s->msi_vectors[vector].pdev =3D=3D NULL) { + return; + } + + /* it was cleaned when masked in the frontend. */ + kvm_irqchip_release_virq(kvm_state, s->msi_vectors[vector].virq); + + s->msi_vectors[vector].pdev =3D NULL; +} + static void ivshmem_enable_irqfd(IVShmemState *s) { PCIDevice *pdev =3D PCI_DEVICE(s); @@ -797,7 +811,7 @@ static void ivshmem_enable_irqfd(IVShmemState *s) ivshmem_add_kvm_msi_virq(s, i, &err); if (err) { error_report_err(err); - /* TODO do we need to handle the error? */ + goto undo; } } =20 @@ -806,21 +820,14 @@ static void ivshmem_enable_irqfd(IVShmemState *s) ivshmem_vector_mask, ivshmem_vector_poll)) { error_report("ivshmem: msix_set_vector_notifiers failed"); + goto undo; } -} + return; =20 -static void ivshmem_remove_kvm_msi_virq(IVShmemState *s, int vector) -{ - IVSHMEM_DPRINTF("ivshmem_remove_kvm_msi_virq vector:%d\n", vector); - - if (s->msi_vectors[vector].pdev =3D=3D NULL) { - return; +undo: + while (--i >=3D 0) { + ivshmem_remove_kvm_msi_virq(s, i); } - - /* it was cleaned when masked in the frontend. */ - kvm_irqchip_release_virq(kvm_state, s->msi_vectors[vector].virq); - - s->msi_vectors[vector].pdev =3D NULL; } =20 static void ivshmem_disable_irqfd(IVShmemState *s) @@ -828,6 +835,10 @@ static void ivshmem_disable_irqfd(IVShmemState *s) PCIDevice *pdev =3D PCI_DEVICE(s); int i; =20 + if (!pdev->msix_vector_use_notifier) { + return; + } + msix_unset_vector_notifiers(pdev); =20 for (i =3D 0; i < s->peers[s->vm_id].nb_eventfds; i++) { --=20 2.13.5