From nobody Sat Feb 7 04:00:11 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1571291069; cv=none; d=zoho.com; s=zohoarc; b=OYCaIzZiiDAlYN0uRudZ2rMTSunxn3lfeohaUqzTi51FCq5xqvrNUfii9wp8o/1WkmPm36KnhvfahEFQaJraaD+sApUF5ygGvFsSuZfDTIMbUj1e6c7p5xyBE+DT0Mw8irgd9FGMINPGv2hbNGeMcBuFu49+xuCiDKadAEB1T8k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1571291069; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=o9ubKtyM+kt+UAfXZkVw0bguL6FS42k7Fa9yUeiSmKU=; b=fCk1Col8jxZ9nS/skHZamlteNbubKJNcmx8hNFeGQSFQqgkVTwSMdbED1mRZ7tlwY4Hcu2Mi33XSp0HatQrAjVD3d37u3aH2a95vMgWFjEWZ55D5iRQaetussmW+T3oFyUPY5oghNsJjU/me9gVJTtkniiQx8jkdUEiIRbzre1o= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1571291069313799.8210464020343; Wed, 16 Oct 2019 22:44:29 -0700 (PDT) Received: from localhost ([::1]:35492 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKyaK-0002tO-3Z for importer@patchew.org; Thu, 17 Oct 2019 01:44:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44351) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKyYO-00016J-Ij for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iKyYN-0005Id-AJ for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:28 -0400 Received: from ozlabs.org ([2401:3900:2:1::2]:58097) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iKyYM-0005HW-Uu; Thu, 17 Oct 2019 01:42:27 -0400 Received: by ozlabs.org (Postfix, from userid 1007) id 46tyjT2l0wz9sP3; Thu, 17 Oct 2019 16:42:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1571290941; bh=rWmaGmDQ7uW9xS+FOC3GrRVstZWojEkMY+gYFGPZ/fs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ewjnCdGFcdEy6vO+rc6ctov5yU3cevDOY+tA0u1YT7cEBYjkRXfSnzMUyabwSN7AJ kE3X2fYkGYxESh45Jhh9AynbWS9PSH/+fywTyiRTE4rqPa4ItLq/VUwd7pkGgcJkcM y1MgaNOLiH7548+uux63MDnIImDmGJgx6zFMRAmU= From: David Gibson To: alex.williamson@redhat.com, clg@kaod.org, groug@kaod.org Subject: [RFC 1/5] kvm: Introduce KVM irqchip change notifier Date: Thu, 17 Oct 2019 16:42:14 +1100 Message-Id: <20191017054218.8876-2-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191017054218.8876-1-david@gibson.dropbear.id.au> References: <20191017054218.8876-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2401:3900:2:1::2 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Computed bodyhash is different from the expected one) Content-Type: text/plain; charset="utf-8" Awareness of an in kernel irqchip is usually local to the machine and its top-level interrupt controller. However, in a few cases other things need to know about it. In particular vfio devices need this in order to accelerate interrupt delivery. If interrupt routing is changed, such devices may need to readjust their connection to the KVM irqchip. pci_bus_fire_intx_routing_notifier() exists to do just this. However, for the pseries machine type we have a situation where the routing remains constant but the top-level irq chip itself is changed. This occurs because of PAPR feature negotiation which allows the guest to decide between the older XICS and newer XIVE irq chip models (both of which are paravirtualized). To allow devices like vfio to adjust to this change, introduce a new notifier for the purpose kvm_irqchip_change_notify(). Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson --- accel/kvm/kvm-all.c | 18 ++++++++++++++++++ accel/stubs/kvm-stub.c | 12 ++++++++++++ include/sysemu/kvm.h | 5 +++++ 3 files changed, 35 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index d2d96d73e8..44df1908dd 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -149,6 +149,9 @@ static const KVMCapabilityInfo kvm_required_capabilites= [] =3D { KVM_CAP_LAST_INFO }; =20 +static NotifierList kvm_irqchip_change_notifiers =3D + NOTIFIER_LIST_INITIALIZER(kvm_irqchip_change_notifiers); + #define kvm_slots_lock(kml) qemu_mutex_lock(&(kml)->slots_lock) #define kvm_slots_unlock(kml) qemu_mutex_unlock(&(kml)->slots_lock) =20 @@ -1396,6 +1399,21 @@ void kvm_irqchip_release_virq(KVMState *s, int virq) trace_kvm_irqchip_release_virq(virq); } =20 +void kvm_irqchip_add_change_notifier(Notifier *n) +{ + notifier_list_add(&kvm_irqchip_change_notifiers, n); +} + +void kvm_irqchip_remove_change_notifier(Notifier *n) +{ + notifier_remove(n); +} + +void kvm_irqchip_change_notify(void) +{ + notifier_list_notify(&kvm_irqchip_change_notifiers, NULL); +} + static unsigned int kvm_hash_msi(uint32_t data) { /* This is optimized for IA32 MSI layout. However, no other arch shall diff --git a/accel/stubs/kvm-stub.c b/accel/stubs/kvm-stub.c index 6feb66ed80..82f118d2df 100644 --- a/accel/stubs/kvm-stub.c +++ b/accel/stubs/kvm-stub.c @@ -138,6 +138,18 @@ void kvm_irqchip_commit_routes(KVMState *s) { } =20 +void kvm_irqchip_add_change_notifier(Notifier *n) +{ +} + +void kvm_irqchip_remove_change_notifier(Notifier *n) +{ +} + +void kvm_irqchip_change_notify(void) +{ +} + int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter) { return -ENOSYS; diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index 9d143282bc..9fe233b9bf 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -201,6 +201,7 @@ typedef struct KVMCapabilityInfo { struct KVMState; typedef struct KVMState KVMState; extern KVMState *kvm_state; +typedef struct Notifier Notifier; =20 /* external API */ =20 @@ -401,6 +402,10 @@ int kvm_irqchip_send_msi(KVMState *s, MSIMessage msg); =20 void kvm_irqchip_add_irq_route(KVMState *s, int gsi, int irqchip, int pin); =20 +void kvm_irqchip_add_change_notifier(Notifier *n); +void kvm_irqchip_remove_change_notifier(Notifier *n); +void kvm_irqchip_change_notify(void); + void kvm_get_apic_state(DeviceState *d, struct kvm_lapic_state *kapic); =20 struct kvm_guest_debug; --=20 2.21.0 From nobody Sat Feb 7 04:00:11 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1571291067; cv=none; d=zoho.com; s=zohoarc; b=F8/jn4DrZx65zxD5uBePp3cYtoz9lbUGOwlN41wHxMe+WcTk/2rJBnLqBT1ad//KCxoNPTQf/f3XoRyPXBoswHeb8gnO2EeOXZT29zplhWRfj8d+Q7qdMOp0T+6G4ps2KoiRCgG6MyhEuB9X92WDde6nVqKzuZWZo7jzgdFD6u0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1571291067; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xqaL4QLhwrdR0wZY+sgrj+BgqRSssqHDxNUV4b7pFcQ=; b=HkVYoYM8o2iQGgfLZ0YLYHN/BhFB+fTLEH9syq4/KIVajgLpy/4q8n/AIGQgNZb8T/rZoU8guMXUYrcdpF3U8WuZ/VTzH1ttfgyyG34O5gzYqeZnlQyKQs6qgJo+i7i1I5fRhNn71coTv+U4VpPufqkaoOMhJAGyJVJqB63aHMM= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1571291067818977.2261080009677; Wed, 16 Oct 2019 22:44:27 -0700 (PDT) Received: from localhost ([::1]:35494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKyaI-0002uQ-7O for importer@patchew.org; Thu, 17 Oct 2019 01:44:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44344) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKyYO-00016G-CB for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iKyYN-0005IW-3V for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:28 -0400 Received: from bilbo.ozlabs.org ([203.11.71.1]:54863 helo=ozlabs.org) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iKyYL-0005Ha-RY; Thu, 17 Oct 2019 01:42:27 -0400 Received: by ozlabs.org (Postfix, from userid 1007) id 46tyjT3yNPz9sPJ; Thu, 17 Oct 2019 16:42:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1571290941; bh=QVOoj/fHM/SytkR+SlnuYQR1WMMdt0CVDGRkSbKA0Rw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cEEtCLpUXt9CyZGSqklfXtk3nek7VPdZNHoyv9ehKYcP+e7OR09L1+KA0XD20S9Ie sqX1I2LEHEs0OafzKr0O1gY46ZVQUiVwLbX01pHfh9L5vA2D0R8ujVL7XJXtCDgL1E 6/lyAEyW5HBkC1rp4oS+MNVhreYgbhJd6NXQGWk8= From: David Gibson To: alex.williamson@redhat.com, clg@kaod.org, groug@kaod.org Subject: [RFC 2/5] vfio/pci: Split vfio_intx_update() Date: Thu, 17 Oct 2019 16:42:15 +1100 Message-Id: <20191017054218.8876-3-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191017054218.8876-1-david@gibson.dropbear.id.au> References: <20191017054218.8876-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 203.11.71.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Computed bodyhash is different from the expected one) Content-Type: text/plain; charset="utf-8" This splits the vfio_intx_update() function into one part doing the actual reconnection with the KVM irqchip (vfio_intx_update(), now taking an argument with the new routing) and vfio_intx_routing_notifier() which handles calls to the pci device intx routing notifier and calling vfio_intx_update() when necessary. This will make adding support for the irqchip change notifier easier. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson --- hw/vfio/pci.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 12fac39804..529ad13908 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -215,30 +215,18 @@ static void vfio_intx_disable_kvm(VFIOPCIDevice *vdev) #endif } =20 -static void vfio_intx_update(PCIDevice *pdev) +static void vfio_intx_update(VFIOPCIDevice *vdev, PCIINTxRoute *route) { - VFIOPCIDevice *vdev =3D PCI_VFIO(pdev); - PCIINTxRoute route; Error *err =3D NULL; =20 - if (vdev->interrupt !=3D VFIO_INT_INTx) { - return; - } - - route =3D pci_device_route_intx_to_irq(&vdev->pdev, vdev->intx.pin); - - if (!pci_intx_route_changed(&vdev->intx.route, &route)) { - return; /* Nothing changed */ - } - trace_vfio_intx_update(vdev->vbasedev.name, - vdev->intx.route.irq, route.irq); + vdev->intx.route.irq, route->irq); =20 vfio_intx_disable_kvm(vdev); =20 - vdev->intx.route =3D route; + vdev->intx.route =3D *route; =20 - if (route.mode !=3D PCI_INTX_ENABLED) { + if (route->mode !=3D PCI_INTX_ENABLED) { return; } =20 @@ -251,6 +239,22 @@ static void vfio_intx_update(PCIDevice *pdev) vfio_intx_eoi(&vdev->vbasedev); } =20 +static void vfio_intx_routing_notifier(PCIDevice *pdev) +{ + VFIOPCIDevice *vdev =3D PCI_VFIO(pdev); + PCIINTxRoute route; + + if (vdev->interrupt !=3D VFIO_INT_INTx) { + return; + } + + route =3D pci_device_route_intx_to_irq(&vdev->pdev, vdev->intx.pin); + + if (pci_intx_route_changed(&vdev->intx.route, &route)) { + vfio_intx_update(vdev, &route); + } +} + static int vfio_intx_enable(VFIOPCIDevice *vdev, Error **errp) { uint8_t pin =3D vfio_pci_read_config(&vdev->pdev, PCI_INTERRUPT_PIN, 1= ); @@ -2954,7 +2958,8 @@ static void vfio_realize(PCIDevice *pdev, Error **err= p) if (vfio_pci_read_config(&vdev->pdev, PCI_INTERRUPT_PIN, 1)) { vdev->intx.mmap_timer =3D timer_new_ms(QEMU_CLOCK_VIRTUAL, vfio_intx_mmap_enable, v= dev); - pci_device_set_intx_routing_notifier(&vdev->pdev, vfio_intx_update= ); + pci_device_set_intx_routing_notifier(&vdev->pdev, + vfio_intx_routing_notifier); ret =3D vfio_intx_enable(vdev, errp); if (ret) { goto out_teardown; --=20 2.21.0 From nobody Sat Feb 7 04:00:11 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1571291067; cv=none; d=zoho.com; s=zohoarc; b=CqxlEqozWNoutgtZK0zXIT4mqNlgKPo74ejMMe5XRAionm5oZFXyqDgncp2ptzr2I3xgSl4sxOVh5U9ovCc47mLajk7oBKzWtQtUhEZhUb5dWkbqwEcGC+X7ySlPBHVb0ZPeyHiLlaVzhHWP1DlCtlD1RJ05bv0Oe8r3gL7K0aU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1571291067; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=U8h5LRXJnBn9jjQdWyn5+IbS1rwNBfTkabkZpMRqsP4=; b=hDN86J45s0W10mbRWNaDVaNHNUwfB68hlN2BU+1Tu+W+L+kHTmvQGUafBF8jOeB61IXzCVia0zHTu1gI/wp2fzsoYVIGAgzir9zMcLeiFI6fytxEWgtxrhzlwkI1Cjx/7QJnmIrngKHfgTJ+raDH8tNAPibLLGTaJVdLbuCYXEw= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1571291066162636.297892937737; Wed, 16 Oct 2019 22:44:26 -0700 (PDT) Received: from localhost ([::1]:35490 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKyaG-0002pR-Ff for importer@patchew.org; Thu, 17 Oct 2019 01:44:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44353) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKyYO-00016K-Ok for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iKyYN-0005Iq-Hc for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:28 -0400 Received: from ozlabs.org ([2401:3900:2:1::2]:52041) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iKyYN-0005HZ-61; Thu, 17 Oct 2019 01:42:27 -0400 Received: by ozlabs.org (Postfix, from userid 1007) id 46tyjT5GT8z9sPV; Thu, 17 Oct 2019 16:42:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1571290941; bh=3vh+L/Wvc64AgJedFt6SujDhRBXr/JOzBL3DIiBl6Bs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Xfh9aFlAI/rRN9893ONcEPpEkpB180jIUz9qQGTLhqvEZWNl+sVH8FzkN8bTpGhei Q4sOFXJIP93iavCBis5oGKGEDO7q3DLd2EGt+kuibMW7TkxVfLOMeCAkEtkHBsT/3T 2/zcGTQ3zUvSKCzxX8j3oP72buxJwTK6PeMMV8NE= From: David Gibson To: alex.williamson@redhat.com, clg@kaod.org, groug@kaod.org Subject: [RFC 3/5] vfio/pci: Respond to KVM irqchip change notifier Date: Thu, 17 Oct 2019 16:42:16 +1100 Message-Id: <20191017054218.8876-4-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191017054218.8876-1-david@gibson.dropbear.id.au> References: <20191017054218.8876-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2401:3900:2:1::2 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Computed bodyhash is different from the expected one) Content-Type: text/plain; charset="utf-8" VFIO PCI devices already respond to the pci intx routing notifier, in order to update kernel irqchip mappings when routing is updated. However this won't handle the case where the irqchip itself is replaced by a different model while retaining the same routing. This case can happen on the pseries machine type due to PAPR feature negotiation. To handle that case, add a handler for the irqchip change notifier, which does much the same thing as the routing notifier, but is unconditional, rather than being a no-op when the routing hasn't changed. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson --- hw/vfio/pci.c | 12 ++++++++++++ hw/vfio/pci.h | 2 ++ 2 files changed, 14 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 529ad13908..6aa806baff 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -255,6 +255,14 @@ static void vfio_intx_routing_notifier(PCIDevice *pdev) } } =20 +static void vfio_irqchip_change(Notifier *notify, void *data) +{ + VFIOPCIDevice *vdev =3D container_of(notify, VFIOPCIDevice, + irqchip_change_notifier); + + vfio_intx_update(vdev, &vdev->intx.route); +} + static int vfio_intx_enable(VFIOPCIDevice *vdev, Error **errp) { uint8_t pin =3D vfio_pci_read_config(&vdev->pdev, PCI_INTERRUPT_PIN, 1= ); @@ -2960,6 +2968,8 @@ static void vfio_realize(PCIDevice *pdev, Error **err= p) vfio_intx_mmap_enable, v= dev); pci_device_set_intx_routing_notifier(&vdev->pdev, vfio_intx_routing_notifier); + vdev->irqchip_change_notifier.notify =3D vfio_irqchip_change; + kvm_irqchip_add_change_notifier(&vdev->irqchip_change_notifier); ret =3D vfio_intx_enable(vdev, errp); if (ret) { goto out_teardown; @@ -3009,6 +3019,7 @@ static void vfio_realize(PCIDevice *pdev, Error **err= p) =20 out_teardown: pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); + kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); vfio_teardown_msi(vdev); vfio_bars_exit(vdev); error: @@ -3042,6 +3053,7 @@ static void vfio_exitfn(PCIDevice *pdev) vfio_unregister_req_notifier(vdev); vfio_unregister_err_notifier(vdev); pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); + kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); vfio_disable_interrupts(vdev); if (vdev->intx.mmap_timer) { timer_free(vdev->intx.mmap_timer); diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 834a90d646..11324f28ce 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -168,6 +168,8 @@ typedef struct VFIOPCIDevice { bool no_vfio_ioeventfd; bool enable_ramfb; VFIODisplay *dpy; + + Notifier irqchip_change_notifier; } VFIOPCIDevice; =20 uint32_t vfio_pci_read_config(PCIDevice *pdev, uint32_t addr, int len); --=20 2.21.0 From nobody Sat Feb 7 04:00:11 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1571291221; cv=none; d=zoho.com; s=zohoarc; b=QnZj0/G+Vtgg6PIJtKuH9N6YywFPNM4NBouKMIhi7WUEdFJ1pxD2HyrxGEwOL5+H+P8d8dMdI9n1WX2aDr1cUP2byJK96ANco3LvB7hCLscy2Hy5jRFvGmgMltCtdnYzunKNgWVUTD/z/gB8cUPKATQEN34fNfrdjEfLHSfxPoQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1571291221; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cN3cD/jv5NLMrBFGn0J6jF/Rz8J6wo+GiNWTy3X+sRw=; b=TMcruyUs2Ac7iXP4JgDOPkjCTud8AxNhpwurBif+SOQYYuWbc5USjkFOfZXT1yr0ihP3pM8ab/O7TgraH2wFFsOsHEDsnjz+tgVuk9fTHgtq6A/lnix5cbDvgbgVLnhsRP+vWKNMI7kHGHyADgFD8biKgNBhvx+RGu156+SQrts= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1571291221014143.45406797614396; Wed, 16 Oct 2019 22:47:01 -0700 (PDT) Received: from localhost ([::1]:35584 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKycl-0005zk-Sz for importer@patchew.org; Thu, 17 Oct 2019 01:46:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44387) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKyYQ-00016k-IQ for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iKyYP-0005Jd-JT for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:30 -0400 Received: from bilbo.ozlabs.org ([203.11.71.1]:41887 helo=ozlabs.org) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iKyYP-0005Ik-8L; Thu, 17 Oct 2019 01:42:29 -0400 Received: by ozlabs.org (Postfix, from userid 1007) id 46tyjT6m1bz9sPc; Thu, 17 Oct 2019 16:42:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1571290941; bh=mGl9erpGvqOrf7exLX2jon0zItJbqLirNCGx+VikJ1Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d5nB3dZDlhZ2E1yiltlu72vEEnyFQTnFOMKueOq5If+DHT610AyiIbx0xp6GywPVq Nw0biffJMmE99WBz9ZmvmJocd/aMljYjUcCciVF9ETpHJbZv8xbpOH6NxgSSVoXizq oV/cU57lGpwD5J5yTZYnXbH748sl/lMvtbUATjEo= From: David Gibson To: alex.williamson@redhat.com, clg@kaod.org, groug@kaod.org Subject: [RFC 4/5] spapr: Handle irq backend changes with VFIO PCI devices Date: Thu, 17 Oct 2019 16:42:17 +1100 Message-Id: <20191017054218.8876-5-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191017054218.8876-1-david@gibson.dropbear.id.au> References: <20191017054218.8876-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 203.11.71.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Computed bodyhash is different from the expected one) Content-Type: text/plain; charset="utf-8" pseries machine type can have one of two different interrupt controllers in use depending on feature negotiation with the guest. Usually this is invisible to devices, because they route to a common set of qemu_irqs which in turn dispatch to the correct back end. VFIO passthrough devices, however, wire themselves up directly to the KVM irqchip for performance, which means they are affected by this change in interrupt controller. To get them to adjust correctly for the change in irqchip, we need to fire the kvm irqchip change notifier. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson --- hw/ppc/spapr_irq.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 234d1073e5..45544b8976 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -480,6 +480,12 @@ static void set_active_intc(SpaprMachineState *spapr, } =20 spapr->active_intc =3D new_intc; + + /* + * We've changed the kernel irqchip, let VFIO devices know they + * need to readjust. + */ + kvm_irqchip_change_notify(); } =20 void spapr_irq_update_active_intc(SpaprMachineState *spapr) --=20 2.21.0 From nobody Sat Feb 7 04:00:11 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1571291240; cv=none; d=zoho.com; s=zohoarc; b=I5QUbgsy1OtxTp5j0vSvLTQahNPE4hwfIiazKPV5GhZ85aApKtctt7fnfBv3ZGg8H+h1GQhUQKElvznUDicXphi8viJ0am/GFCFfXjSkRp4FsyoFe1GjrLYY2yY92bw57Mh0x21N57LHQg1ikkqvJ1J5BALEbVvfzHVpY77fzEI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1571291240; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=miTtObZe/k/Y2gV4HdfEBRZhijOfIgfI10ywbSCKQRM=; b=NQVWX/PmwGddcUhRs3bqy5h73c2hdkU8YQ5cz6+y+S6IbQNxa7SmybztpRpPNRUTKx9uslcSsFXwZWdTDIOZtfl7wIicFk8pg2oB/1WWJN/L4DJJ83qfVUg0nEEH+DXvak/T2Myy0IHF2U+dOjv0NLaloEgRPV+cSX2h9wjvA9Q= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1571291240024886.7580918861672; Wed, 16 Oct 2019 22:47:20 -0700 (PDT) Received: from localhost ([::1]:35588 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKyd5-0006Mc-11 for importer@patchew.org; Thu, 17 Oct 2019 01:47:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44343) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iKyYO-00016E-Ha for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iKyYN-0005IO-2q for qemu-devel@nongnu.org; Thu, 17 Oct 2019 01:42:28 -0400 Received: from bilbo.ozlabs.org ([203.11.71.1]:34509 helo=ozlabs.org) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iKyYL-0005HY-SK; Thu, 17 Oct 2019 01:42:27 -0400 Received: by ozlabs.org (Postfix, from userid 1007) id 46tyjT62qhz9sPW; Thu, 17 Oct 2019 16:42:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1571290941; bh=qeV4bTD0CrZcVuH/WEAA1NqQ/0d0617D6DXPN3gPjK4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GzdT5c+wRt8g8RPorfRghqoB7zQ8ShLWhkIKKcwlz/OtHLkKWlGQNE2Q9ETWS3xZ2 hn1cmDG2Fbl/YhsTJt7yaPTDMtSwSTIc0Rnvbj82xNkrH3XzKQHwQHFhmKeqbtcp6r uO55kX84H6U3NYBfbOGc+bl6EFYc9nSHeDAtspBo= From: David Gibson To: alex.williamson@redhat.com, clg@kaod.org, groug@kaod.org Subject: [RFC 5/5] spapr: Work around spurious warnings from vfio INTx initialization Date: Thu, 17 Oct 2019 16:42:18 +1100 Message-Id: <20191017054218.8876-6-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191017054218.8876-1-david@gibson.dropbear.id.au> References: <20191017054218.8876-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 203.11.71.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Computed bodyhash is different from the expected one) Content-Type: text/plain; charset="utf-8" Traditional PCI INTx for vfio devices can only perform well if using an in-kernel irqchip. Therefore, vfio_intx_update() issues a warning if an in kernel irqchip is not available. We usually do have an in-kernel irqchip available for pseries machines on POWER hosts. However, because the platform allows feature negotiation of what interrupt controller model to use, we don't currently initialize it until machine reset. vfio_intx_update() is called (first) from vfio_realize() before that, so it can issue a spurious warning, even if we will have an in kernel irqchip by the time we need it. To workaround this, make a call to spapr_irq_update_active_intc() from spapr_irq_init() which is called at machine realize time, before the vfio realize. This call will be pretty much obsoleted by the later call at reset time, but it serves to suppress the spurious warning from VFIO. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson --- hw/ppc/spapr_irq.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 45544b8976..bb91c61fa0 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -345,6 +345,14 @@ void spapr_irq_init(SpaprMachineState *spapr, Error **= errp) =20 spapr->qirqs =3D qemu_allocate_irqs(spapr_set_irq, spapr, smc->nr_xirqs + SPAPR_XIRQ_BASE); + + /* + * Mostly we don't actually need this until reset, except that not + * having this set up can cause VFIO devices to issue a + * false-positive warning during realize(), because they don't yet + * have an in-kernel irq chip. + */ + spapr_irq_update_active_intc(spapr); } =20 int spapr_irq_claim(SpaprMachineState *spapr, int irq, bool lsi, Error **e= rrp) @@ -500,7 +508,8 @@ void spapr_irq_update_active_intc(SpaprMachineState *sp= apr) * this. */ new_intc =3D SPAPR_INTC(spapr->xive); - } else if (spapr_ovec_test(spapr->ov5_cas, OV5_XIVE_EXPLOIT)) { + } else if (spapr->ov5_cas + && spapr_ovec_test(spapr->ov5_cas, OV5_XIVE_EXPLOIT)) { new_intc =3D SPAPR_INTC(spapr->xive); } else { new_intc =3D SPAPR_INTC(spapr->ics); --=20 2.21.0