From nobody Thu May 9 00:22:16 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1574298222; cv=none; d=zoho.com; s=zohoarc; b=TTG5GXHK7UwtT1wBjhvKnG5ZLihiaFKeYEJjlly3OuPvwElxD+OOl+dcgyXZRdK9l3YW/XC85x17VCNZXFDpV93FYzr5MDzLVkyBH8PSU1NtXhaAfmr5WRPoM0wb+UrTT6yYg7hXUW/GRJlUIzzSST1h2M8LEVJsEJbz5padWO8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1574298222; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=y9hY6JlKZAUtorDTPvfMOWPZpWsH+EzhwANhiushPK8=; b=i6V4MrBK6avShDyLhy4B5jLDfYBcZ5NzIRDrJizvQEpoQGsw2v6Z3RcoXsGD9NlFHJr+Gd5Frzf8ck/jZ6+ZvOFdtNEnrld7fesVQWrJy7oYSpmf5k4vqdeLl5kS3mRnTzUIAc1iVbJ73lNEvgoSq9wy90UdKVWadVOsvlkYEwc= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1574298222416420.78406085660026; Wed, 20 Nov 2019 17:03:42 -0800 (PST) Received: from localhost ([::1]:35512 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXash-0005a5-62 for importer@patchew.org; Wed, 20 Nov 2019 20:03:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33928) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXao5-0001Qc-NI for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iXao4-0001rb-2w for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:49 -0500 Received: from bilbo.ozlabs.org ([203.11.71.1]:39031 helo=ozlabs.org) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iXao3-0001qw-CJ; Wed, 20 Nov 2019 19:58:48 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm41Hqpz9sNx; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=fREJuuqgAEuiDzU2O79CZ0GSodqvDi8dVSOI0miZGuE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IcxKWzfsNp+TDMm/FcmYOF/G6BK8Fwdx7sj3AbbpJBecSmOD3EvejoKZXJ9Hdo5X4 GBJcG9PY7fmZVX3w/91Q8nXcqBSmyoGG35fPguQUP2szAqbFxrua9emPjVKUaoeYJL b8W+cjT9+2k1AAe70PVOMNqijScMEkmCl0xnVIm4= From: David Gibson To: Alex Williamson , clg@kaod.org Subject: [PATCH 1/5] kvm: Introduce KVM irqchip change notifier Date: Thu, 21 Nov 2019 11:56:03 +1100 Message-Id: <20191121005607.274347-2-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 203.11.71.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, qemu-devel@nongnu.org, Jason Wang , Riku Voipio , Laurent Vivier , groug@kaod.org, Alexey Kardashevskiy , qemu-ppc@nongnu.org, =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Paolo Bonzini , philmd@redhat.com, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Awareness of an in kernel irqchip is usually local to the machine and its top-level interrupt controller. However, in a few cases other things need to know about it. In particular vfio devices need this in order to accelerate interrupt delivery. If interrupt routing is changed, such devices may need to readjust their connection to the KVM irqchip. pci_bus_fire_intx_routing_notifier() exists to do just this. However, for the pseries machine type we have a situation where the routing remains constant but the top-level irq chip itself is changed. This occurs because of PAPR feature negotiation which allows the guest to decide between the older XICS and newer XIVE irq chip models (both of which are paravirtualized). To allow devices like vfio to adjust to this change, introduce a new notifier for the purpose kvm_irqchip_change_notify(). Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson Acked-by: Alex Williamson Reviewed-by: Alex Williamson Reviewed-by: Greg Kurz Tested-by: Alex Williamson --- accel/kvm/kvm-all.c | 18 ++++++++++++++++++ accel/stubs/kvm-stub.c | 12 ++++++++++++ include/sysemu/kvm.h | 5 +++++ 3 files changed, 35 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 140b0bd8f6..ca00daa2f5 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -149,6 +149,9 @@ static const KVMCapabilityInfo kvm_required_capabilites= [] =3D { KVM_CAP_LAST_INFO }; =20 +static NotifierList kvm_irqchip_change_notifiers =3D + NOTIFIER_LIST_INITIALIZER(kvm_irqchip_change_notifiers); + #define kvm_slots_lock(kml) qemu_mutex_lock(&(kml)->slots_lock) #define kvm_slots_unlock(kml) qemu_mutex_unlock(&(kml)->slots_lock) =20 @@ -1396,6 +1399,21 @@ void kvm_irqchip_release_virq(KVMState *s, int virq) trace_kvm_irqchip_release_virq(virq); } =20 +void kvm_irqchip_add_change_notifier(Notifier *n) +{ + notifier_list_add(&kvm_irqchip_change_notifiers, n); +} + +void kvm_irqchip_remove_change_notifier(Notifier *n) +{ + notifier_remove(n); +} + +void kvm_irqchip_change_notify(void) +{ + notifier_list_notify(&kvm_irqchip_change_notifiers, NULL); +} + static unsigned int kvm_hash_msi(uint32_t data) { /* This is optimized for IA32 MSI layout. However, no other arch shall diff --git a/accel/stubs/kvm-stub.c b/accel/stubs/kvm-stub.c index 6feb66ed80..82f118d2df 100644 --- a/accel/stubs/kvm-stub.c +++ b/accel/stubs/kvm-stub.c @@ -138,6 +138,18 @@ void kvm_irqchip_commit_routes(KVMState *s) { } =20 +void kvm_irqchip_add_change_notifier(Notifier *n) +{ +} + +void kvm_irqchip_remove_change_notifier(Notifier *n) +{ +} + +void kvm_irqchip_change_notify(void) +{ +} + int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter) { return -ENOSYS; diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index 9d143282bc..9fe233b9bf 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -201,6 +201,7 @@ typedef struct KVMCapabilityInfo { struct KVMState; typedef struct KVMState KVMState; extern KVMState *kvm_state; +typedef struct Notifier Notifier; =20 /* external API */ =20 @@ -401,6 +402,10 @@ int kvm_irqchip_send_msi(KVMState *s, MSIMessage msg); =20 void kvm_irqchip_add_irq_route(KVMState *s, int gsi, int irqchip, int pin); =20 +void kvm_irqchip_add_change_notifier(Notifier *n); +void kvm_irqchip_remove_change_notifier(Notifier *n); +void kvm_irqchip_change_notify(void); + void kvm_get_apic_state(DeviceState *d, struct kvm_lapic_state *kapic); =20 struct kvm_guest_debug; --=20 2.23.0 From nobody Thu May 9 00:22:16 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1574298314; cv=none; d=zoho.com; s=zohoarc; b=TDfUWN0MYJI6tVQvvFnYhLDkBz9vZnv5QQ+vCWBBkl54PF+uwIT3nfFpxBqSXecwUk99RM3Li8i/yUid1VT8pFHo1xQ65NkNSwJlFmFlzHr5K+5+MUBVGiORwkcdf3VaSHvPO+X42K0VBCJ3EwFwZp8VaIpPID+1Do27kPdcRYA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1574298314; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FBZYN/wbpgsVd5oZ7uCfeEZaCY2sFxl1Yc06ruPGE4s=; b=Jtiz3gsdYw3AH+9qD40mV5SALPHgKqh7y/STVE8fOCaFiqHFh1ilBPHEYUgBam6/PDB42/PyrjMo4balnttK+rW+iPwW8o4+HYi4W/Ihn+dJSGY/LOuiRDHCY4HFCeP7yT+sjNgh1AKQrE/OwU57FJlUvX6oo08EvbrDPQzROfY= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1574298314182199.84376303300405; Wed, 20 Nov 2019 17:05:14 -0800 (PST) Received: from localhost ([::1]:35526 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXauH-0007ME-4N for importer@patchew.org; Wed, 20 Nov 2019 20:05:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33932) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXao5-0001Qg-Js for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iXao4-0001s6-HE for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:49 -0500 Received: from bilbo.ozlabs.org ([2401:3900:2:1::2]:35443 helo=ozlabs.org) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iXao4-0001qz-6i; Wed, 20 Nov 2019 19:58:48 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm42L1Gz9sPW; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=M0mcg5/1en/4FPfwraZYSINmLiks1lDcs9ZPZyf4HFY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IrCBuvydoJarKwc/AqXBCCfhRHgtOv2SIHcrg7RMzyu44qkv46jqKR1N3sGuqc4mh I9yv01OBJiKsd+pNQA58bD4EXpoXcEVNT14kxVqV0ZDbKeXx1WZqgw8B5syMI78vvk FKKxC/uAqg99z7l1n69qh3cmU4omIQGoacX/vSQ4= From: David Gibson To: Alex Williamson , clg@kaod.org Subject: [PATCH 2/5] vfio/pci: Split vfio_intx_update() Date: Thu, 21 Nov 2019 11:56:04 +1100 Message-Id: <20191121005607.274347-3-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2401:3900:2:1::2 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, qemu-devel@nongnu.org, Jason Wang , Riku Voipio , Laurent Vivier , groug@kaod.org, Alexey Kardashevskiy , qemu-ppc@nongnu.org, =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Paolo Bonzini , philmd@redhat.com, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This splits the vfio_intx_update() function into one part doing the actual reconnection with the KVM irqchip (vfio_intx_update(), now taking an argument with the new routing) and vfio_intx_routing_notifier() which handles calls to the pci device intx routing notifier and calling vfio_intx_update() when necessary. This will make adding support for the irqchip change notifier easier. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson Acked-by: Alex Williamson Reviewed-by: Alex Williamson Reviewed-by: Greg Kurz Tested-by: Alex Williamson --- hw/vfio/pci.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 0c55883bba..521289aa7d 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -216,30 +216,18 @@ static void vfio_intx_disable_kvm(VFIOPCIDevice *vdev) #endif } =20 -static void vfio_intx_update(PCIDevice *pdev) +static void vfio_intx_update(VFIOPCIDevice *vdev, PCIINTxRoute *route) { - VFIOPCIDevice *vdev =3D PCI_VFIO(pdev); - PCIINTxRoute route; Error *err =3D NULL; =20 - if (vdev->interrupt !=3D VFIO_INT_INTx) { - return; - } - - route =3D pci_device_route_intx_to_irq(&vdev->pdev, vdev->intx.pin); - - if (!pci_intx_route_changed(&vdev->intx.route, &route)) { - return; /* Nothing changed */ - } - trace_vfio_intx_update(vdev->vbasedev.name, - vdev->intx.route.irq, route.irq); + vdev->intx.route.irq, route->irq); =20 vfio_intx_disable_kvm(vdev); =20 - vdev->intx.route =3D route; + vdev->intx.route =3D *route; =20 - if (route.mode !=3D PCI_INTX_ENABLED) { + if (route->mode !=3D PCI_INTX_ENABLED) { return; } =20 @@ -252,6 +240,22 @@ static void vfio_intx_update(PCIDevice *pdev) vfio_intx_eoi(&vdev->vbasedev); } =20 +static void vfio_intx_routing_notifier(PCIDevice *pdev) +{ + VFIOPCIDevice *vdev =3D PCI_VFIO(pdev); + PCIINTxRoute route; + + if (vdev->interrupt !=3D VFIO_INT_INTx) { + return; + } + + route =3D pci_device_route_intx_to_irq(&vdev->pdev, vdev->intx.pin); + + if (pci_intx_route_changed(&vdev->intx.route, &route)) { + vfio_intx_update(vdev, &route); + } +} + static int vfio_intx_enable(VFIOPCIDevice *vdev, Error **errp) { uint8_t pin =3D vfio_pci_read_config(&vdev->pdev, PCI_INTERRUPT_PIN, 1= ); @@ -2967,7 +2971,8 @@ static void vfio_realize(PCIDevice *pdev, Error **err= p) if (vfio_pci_read_config(&vdev->pdev, PCI_INTERRUPT_PIN, 1)) { vdev->intx.mmap_timer =3D timer_new_ms(QEMU_CLOCK_VIRTUAL, vfio_intx_mmap_enable, v= dev); - pci_device_set_intx_routing_notifier(&vdev->pdev, vfio_intx_update= ); + pci_device_set_intx_routing_notifier(&vdev->pdev, + vfio_intx_routing_notifier); ret =3D vfio_intx_enable(vdev, errp); if (ret) { goto out_teardown; --=20 2.23.0 From nobody Thu May 9 00:22:16 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1574298035; cv=none; d=zoho.com; s=zohoarc; b=F4yGLejdt8YXfCUx6VimzPRAujljcUT47OCHm4t3bPer8eJtYDJ4qiBgT5QYi4WXZW1PkDeIUgKQYB2YygSRmJq/F5OlhX81AuASRpvNsWHbbLuJmZzK6H1DfNG7MrdR3c1wmOmPS+ZKslJc+Lr4h0oYxMxx+LzfV8s+pyj00I4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1574298035; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0va4/9SLWSAPirdzYNJR8QcSwL00kJtrXL0kdi3k90Q=; b=g9R1dcdKTRZj04Me1RH21KtTfkLJ972WX8w/cDNNOTCT2rNTOAU6WzEICRTYNtDeRhSCV+xaKpqqKsLHwU17DXeQVduejuIeSq3++jYFWH103B0N1omzDlZFgSmIoXgFd3VERjcnBc6IDP/zwiXrWheXAPkZBG/76AzNUW7aYbs= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1574298035937944.5018828143807; Wed, 20 Nov 2019 17:00:35 -0800 (PST) Received: from localhost ([::1]:35484 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXapm-0002x6-IB for importer@patchew.org; Wed, 20 Nov 2019 20:00:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33929) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXao5-0001Qd-JV for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iXao4-0001rp-95 for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:49 -0500 Received: from bilbo.ozlabs.org ([2401:3900:2:1::2]:43401 helo=ozlabs.org) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iXao3-0001r1-Ex; Wed, 20 Nov 2019 19:58:48 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm434w0z9sPn; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=Q2CHP0Ke6mkaX6t4hswofWQZGdJTg0Yt2bzdTOJZj8s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HTZ8RZOPsHWsPlug7qeJ77TO3NZIzw84Ege2mnZE9FQXPBQNOMFKManr/FpA/JEVX JpUbc+NncdPM4Lln8rWPT5XCKio3EgYMEr4M73X4n8sjX4u6pacwgYbfxPOAc+45r2 /nFqLBv+EQ3+WLKVvQgA3zAbTm5KsbKFNpAU9tfs= From: David Gibson To: Alex Williamson , clg@kaod.org Subject: [PATCH 3/5] vfio/pci: Respond to KVM irqchip change notifier Date: Thu, 21 Nov 2019 11:56:05 +1100 Message-Id: <20191121005607.274347-4-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2401:3900:2:1::2 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, qemu-devel@nongnu.org, Jason Wang , Riku Voipio , Laurent Vivier , groug@kaod.org, Alexey Kardashevskiy , qemu-ppc@nongnu.org, =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Paolo Bonzini , philmd@redhat.com, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" VFIO PCI devices already respond to the pci intx routing notifier, in order to update kernel irqchip mappings when routing is updated. However this won't handle the case where the irqchip itself is replaced by a different model while retaining the same routing. This case can happen on the pseries machine type due to PAPR feature negotiation. To handle that case, add a handler for the irqchip change notifier, which does much the same thing as the routing notifier, but is unconditional, rather than being a no-op when the routing hasn't changed. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson Acked-by: Alex Williamson Reviewed-by: Alex Williamson Reviewed-by: Greg Kurz Tested-by: Alex Williamson --- hw/vfio/pci.c | 23 ++++++++++++++++++----- hw/vfio/pci.h | 1 + 2 files changed, 19 insertions(+), 5 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 521289aa7d..95478c2c55 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -256,6 +256,14 @@ static void vfio_intx_routing_notifier(PCIDevice *pdev) } } =20 +static void vfio_irqchip_change(Notifier *notify, void *data) +{ + VFIOPCIDevice *vdev =3D container_of(notify, VFIOPCIDevice, + irqchip_change_notifier); + + vfio_intx_update(vdev, &vdev->intx.route); +} + static int vfio_intx_enable(VFIOPCIDevice *vdev, Error **errp) { uint8_t pin =3D vfio_pci_read_config(&vdev->pdev, PCI_INTERRUPT_PIN, 1= ); @@ -2973,16 +2981,18 @@ static void vfio_realize(PCIDevice *pdev, Error **e= rrp) vfio_intx_mmap_enable, v= dev); pci_device_set_intx_routing_notifier(&vdev->pdev, vfio_intx_routing_notifier); + vdev->irqchip_change_notifier.notify =3D vfio_irqchip_change; + kvm_irqchip_add_change_notifier(&vdev->irqchip_change_notifier); ret =3D vfio_intx_enable(vdev, errp); if (ret) { - goto out_teardown; + goto out_deregister; } } =20 if (vdev->display !=3D ON_OFF_AUTO_OFF) { ret =3D vfio_display_probe(vdev, errp); if (ret) { - goto out_teardown; + goto out_deregister; } } if (vdev->enable_ramfb && vdev->dpy =3D=3D NULL) { @@ -2992,11 +3002,11 @@ static void vfio_realize(PCIDevice *pdev, Error **e= rrp) if (vdev->display_xres || vdev->display_yres) { if (vdev->dpy =3D=3D NULL) { error_setg(errp, "xres and yres properties require display=3Do= n"); - goto out_teardown; + goto out_deregister; } if (vdev->dpy->edid_regs =3D=3D NULL) { error_setg(errp, "xres and yres properties need edid support"); - goto out_teardown; + goto out_deregister; } } =20 @@ -3020,8 +3030,10 @@ static void vfio_realize(PCIDevice *pdev, Error **er= rp) =20 return; =20 -out_teardown: +out_deregister: pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); + kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); +out_teardown: vfio_teardown_msi(vdev); vfio_bars_exit(vdev); error: @@ -3064,6 +3076,7 @@ static void vfio_exitfn(PCIDevice *pdev) vfio_unregister_req_notifier(vdev); vfio_unregister_err_notifier(vdev); pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); + kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); vfio_disable_interrupts(vdev); if (vdev->intx.mmap_timer) { timer_free(vdev->intx.mmap_timer); diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index b329d50338..35626cd63e 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -169,6 +169,7 @@ typedef struct VFIOPCIDevice { bool enable_ramfb; VFIODisplay *dpy; Error *migration_blocker; + Notifier irqchip_change_notifier; } VFIOPCIDevice; =20 uint32_t vfio_pci_read_config(PCIDevice *pdev, uint32_t addr, int len); --=20 2.23.0 From nobody Thu May 9 00:22:16 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1574298223; cv=none; d=zoho.com; s=zohoarc; b=od/+7CAssMM4mjWSjieK/8PXJ0KqQ1pWjRdNuu5B1hxaeNtrxyKKNiqTCKOkPsk+i2Dd9LzozhHM2IDaBp0rDSnXr1x67EteilzrUDhdIZrOJiSKefr1JlI/Mqm77li08ameAZE6hrk0nae9PamykYV7L4M2MzSoLqe6ZWwP24g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1574298223; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RO7Z+h2z43hgvwBb7quSczqwIsy25yLkz8LevHbSwAk=; b=fOoRCwAffc3XxCKSEXMTBi75GozSTQJH9XaB2P4ww6vR9WQeP3g0UffaG9mDQGLDAHGlImUsQz+H3mjVGanydnow+hm2+P55JvBpX1L2bhs1JzV+UvhBR36/R/lz25IaJHT12hbiLtiPPOG6h/oNVkAX5qVp2sZ2pbJ2Y2lQ5U0= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1574298223292781.0133668670188; Wed, 20 Nov 2019 17:03:43 -0800 (PST) Received: from localhost ([::1]:35514 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXash-0005am-61 for importer@patchew.org; Wed, 20 Nov 2019 20:03:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33969) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXao7-0001R3-9h for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iXao6-0001tO-AP for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:51 -0500 Received: from ozlabs.org ([2401:3900:2:1::2]:45307) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iXao6-0001sA-0Q; Wed, 20 Nov 2019 19:58:50 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm44S9lz9sR2; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=uy06rYtNsupoZ3y3vcdSdik4ZpqVFHNbtPoj6Ji6YFU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GH/ey9I17WPcqhxnfntIjzp89v+GocZFviNoVj5tc/u1wRv9nZRm70g0d6H7E4huG Ko5wf/5H1AmADYpIy9Xxx7C+drHdYhuzBZcJJdM9B/L1KHZaBNVIE79jAeczsA/ns9 js14uu6SHHwCWZDkUy0PdzSRpEXyti5fXNxUYejc= From: David Gibson To: Alex Williamson , clg@kaod.org Subject: [PATCH 4/5] spapr: Handle irq backend changes with VFIO PCI devices Date: Thu, 21 Nov 2019 11:56:06 +1100 Message-Id: <20191121005607.274347-5-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2401:3900:2:1::2 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, qemu-devel@nongnu.org, Jason Wang , Riku Voipio , Laurent Vivier , groug@kaod.org, Alexey Kardashevskiy , qemu-ppc@nongnu.org, =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Paolo Bonzini , philmd@redhat.com, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" pseries machine type can have one of two different interrupt controllers in use depending on feature negotiation with the guest. Usually this is invisible to devices, because they route to a common set of qemu_irqs which in turn dispatch to the correct back end. VFIO passthrough devices, however, wire themselves up directly to the KVM irqchip for performance, which means they are affected by this change in interrupt controller. To get them to adjust correctly for the change in irqchip, we need to fire the kvm irqchip change notifier. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson Acked-by: Alex Williamson Reviewed-by: Alex Williamson Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Greg Kurz Tested-by: Alex Williamson --- hw/ppc/spapr_irq.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 168044be85..1d27034962 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -508,6 +508,12 @@ static void set_active_intc(SpaprMachineState *spapr, } =20 spapr->active_intc =3D new_intc; + + /* + * We've changed the kernel irqchip, let VFIO devices know they + * need to readjust. + */ + kvm_irqchip_change_notify(); } =20 void spapr_irq_update_active_intc(SpaprMachineState *spapr) --=20 2.23.0 From nobody Thu May 9 00:22:16 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1574298030; cv=none; d=zoho.com; s=zohoarc; b=OPialcNqRNXbCg6e6wnntZushYmJNhLWtTdAEMVpZQuXX3tWQpZNW30hjgBm/r7/Yg6wGh9xPS4qmV3piCC2/8s5pQFICNIznv/PGooaKXFmi2OVoSqC/xc1WM8WCnz4H8QuSSNr4I+yEiA6Een8zzX+8pQAiqvfrhHPD+CI9b0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1574298030; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XMh62IW7WYGW4z964YAQKBoagsTNzYB86BTiKAp/wfI=; b=SUI5/3zIHHKhzIzpY5A6rapNAY5E9Q5ynhx0eqoRTRstqm8kGjfeaWT1/SHStQ8YfNIP/L8NOBYWRd1B5COtfi7bYmYaTO7M1EzixT82oRi6Coz4MUODpGlHOxXde0iH8tSkx4CeaIMaKxR5JhpHEUWZmIR1yrc8spxubxDUBEs= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1574298030116237.46690698438465; Wed, 20 Nov 2019 17:00:30 -0800 (PST) Received: from localhost ([::1]:35482 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXapg-0002ug-VP for importer@patchew.org; Wed, 20 Nov 2019 20:00:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33931) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iXao5-0001Qf-Rj for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iXao4-0001s0-Fe for qemu-devel@nongnu.org; Wed, 20 Nov 2019 19:58:49 -0500 Received: from bilbo.ozlabs.org ([2401:3900:2:1::2]:57659 helo=ozlabs.org) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iXao4-0001r2-51; Wed, 20 Nov 2019 19:58:48 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm43p1Sz9sQp; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=kjLCHlKyKt7jP0NFe7tI8vqbJnysCqs9Js2O7NlIAAQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d5Yl8X8jt5f6X3VYPrSZIeC4Sr8ahHtnfT9r4tRoBX95XiR92/bUGI6IPEAELp+p/ gj4PkS/PiYEpGg2AlbgHsnkezWv9MlYEin1RnA0QOdA8ICddY5MvLGDgMhrae1DmfQ tN1PBz0gCD0y8oOnF1ZySK7+BH5GU5e+myP++TYw= From: David Gibson To: Alex Williamson , clg@kaod.org Subject: [PATCH 5/5] spapr: Work around spurious warnings from vfio INTx initialization Date: Thu, 21 Nov 2019 11:56:07 +1100 Message-Id: <20191121005607.274347-6-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2401:3900:2:1::2 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, qemu-devel@nongnu.org, Jason Wang , Riku Voipio , Laurent Vivier , groug@kaod.org, Alexey Kardashevskiy , qemu-ppc@nongnu.org, =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Paolo Bonzini , philmd@redhat.com, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Traditional PCI INTx for vfio devices can only perform well if using an in-kernel irqchip. Therefore, vfio_intx_update() issues a warning if an in kernel irqchip is not available. We usually do have an in-kernel irqchip available for pseries machines on POWER hosts. However, because the platform allows feature negotiation of what interrupt controller model to use, we don't currently initialize it until machine reset. vfio_intx_update() is called (first) from vfio_realize() before that, so it can issue a spurious warning, even if we will have an in kernel irqchip by the time we need it. To workaround this, make a call to spapr_irq_update_active_intc() from spapr_irq_init() which is called at machine realize time, before the vfio realize. This call will be pretty much obsoleted by the later call at reset time, but it serves to suppress the spurious warning from VFIO. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson Acked-by: Alex Williamson Reviewed-by: Alex Williamson Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Greg Kurz Tested-by: Alex Williamson --- hw/ppc/spapr_irq.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 1d27034962..d6bb7fd2d6 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -373,6 +373,14 @@ void spapr_irq_init(SpaprMachineState *spapr, Error **= errp) =20 spapr->qirqs =3D qemu_allocate_irqs(spapr_set_irq, spapr, smc->nr_xirqs + SPAPR_XIRQ_BASE); + + /* + * Mostly we don't actually need this until reset, except that not + * having this set up can cause VFIO devices to issue a + * false-positive warning during realize(), because they don't yet + * have an in-kernel irq chip. + */ + spapr_irq_update_active_intc(spapr); } =20 int spapr_irq_claim(SpaprMachineState *spapr, int irq, bool lsi, Error **e= rrp) @@ -528,7 +536,8 @@ void spapr_irq_update_active_intc(SpaprMachineState *sp= apr) * this. */ new_intc =3D SPAPR_INTC(spapr->xive); - } else if (spapr_ovec_test(spapr->ov5_cas, OV5_XIVE_EXPLOIT)) { + } else if (spapr->ov5_cas + && spapr_ovec_test(spapr->ov5_cas, OV5_XIVE_EXPLOIT)) { new_intc =3D SPAPR_INTC(spapr->xive); } else { new_intc =3D SPAPR_INTC(spapr->ics); --=20 2.23.0