From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557737125; cv=none; d=zoho.com; s=zohoarc; b=WerwTl614T3Gp8p6jNbVfZCl6u7WuobbwtFYc8vUa6XLZL+K6N0KGUoXzBoW6TK3fdp7vGbf5NwvRcaMuvoHBTTmsOtUoC/kMRYeTG2iqOalVzjNWJFf/y8TTyTARvH5D3Q/Gn6lxjnKJxWrld/7wCqCrW9TGZg/fQkICHRYqSg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557737125; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=LZkH8yjbzzSucr3ijvhw6ZtYmRWupnL/f7GBnN2L57w=; b=n9TcMsVotaueTs2M1PyZ7N2GgeUzhXx7/U6Nu0d9Sgu3UwYt5kqKUU83pFZ6t4I6q35oWWBWIhMSnhF4jaKbC5gkWZvymjnIpzRFAQeyRENsp5q0UD5c2Zl/2IjTF3KrPY4Ks4+ii294XwnnQS7jqjU/HhFON5bXryTaQHbResk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557737125985446.53471321982965; Mon, 13 May 2019 01:45:25 -0700 (PDT) Received: from localhost ([127.0.0.1]:53308 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6aB-00039k-94 for importer@patchew.org; Mon, 13 May 2019 04:45:15 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59159) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6YC-00020N-Ic for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6YB-0003wu-Bq for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:12 -0400 Received: from 4.mo173.mail-out.ovh.net ([46.105.34.219]:45720) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6YB-0003tx-29 for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:11 -0400 Received: from player778.ha.ovh.net (unknown [10.108.54.237]) by mo173.mail-out.ovh.net (Postfix) with ESMTP id CFE23FC498 for ; Mon, 13 May 2019 10:43:02 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 075C059C3BE6; Mon, 13 May 2019 08:42:57 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:32 +0200 Message-Id: <20190513084245.25755-2-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3320841777245359078 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.34.219 Subject: [Qemu-devel] [PATCH v4 01/14] linux-headers: update linux headers to kvm-ppc-next-5.2-1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" This updates the kvm headers to commit 0caecf5b0019 (kvm-ppc-next-5.2-1) in branch kvm-ppc-next from : https://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc.git/ These changes provide the interface with the KVM device implementing the XIVE native exploitation interrupt mode. Signed-off-by: C=C3=A9dric Le Goater --- The XIVE capability number will conflict with kvm-arm : =20 555f3d03e7fb ("KVM: arm64: Add a capability to advertise SVE support") a243c16d18be ("KVM: arm64: Add capability to advertise ptrauth for guest") linux-headers/asm-powerpc/kvm.h | 46 +++++++++++++++++++++++++++++++++ linux-headers/linux/kvm.h | 3 +++ 2 files changed, 49 insertions(+) diff --git a/linux-headers/asm-powerpc/kvm.h b/linux-headers/asm-powerpc/kv= m.h index 8c876c166ef2..07daf7bf6fe4 100644 --- a/linux-headers/asm-powerpc/kvm.h +++ b/linux-headers/asm-powerpc/kvm.h @@ -480,6 +480,8 @@ struct kvm_ppc_cpu_char { #define KVM_REG_PPC_ICP_PPRI_SHIFT 16 /* pending irq priority */ #define KVM_REG_PPC_ICP_PPRI_MASK 0xff =20 +#define KVM_REG_PPC_VP_STATE (KVM_REG_PPC | KVM_REG_SIZE_U128 | 0x8d) + /* Device control API: PPC-specific devices */ #define KVM_DEV_MPIC_GRP_MISC 1 #define KVM_DEV_MPIC_BASE_ADDR 0 /* 64-bit */ @@ -675,4 +677,48 @@ struct kvm_ppc_cpu_char { #define KVM_XICS_PRESENTED (1ULL << 43) #define KVM_XICS_QUEUED (1ULL << 44) =20 +/* POWER9 XIVE Native Interrupt Controller */ +#define KVM_DEV_XIVE_GRP_CTRL 1 +#define KVM_DEV_XIVE_RESET 1 +#define KVM_DEV_XIVE_EQ_SYNC 2 +#define KVM_DEV_XIVE_GRP_SOURCE 2 /* 64-bit source identifier */ +#define KVM_DEV_XIVE_GRP_SOURCE_CONFIG 3 /* 64-bit source identifier */ +#define KVM_DEV_XIVE_GRP_EQ_CONFIG 4 /* 64-bit EQ identifier */ +#define KVM_DEV_XIVE_GRP_SOURCE_SYNC 5 /* 64-bit source identifier */ + +/* Layout of 64-bit XIVE source attribute values */ +#define KVM_XIVE_LEVEL_SENSITIVE (1ULL << 0) +#define KVM_XIVE_LEVEL_ASSERTED (1ULL << 1) + +/* Layout of 64-bit XIVE source configuration attribute values */ +#define KVM_XIVE_SOURCE_PRIORITY_SHIFT 0 +#define KVM_XIVE_SOURCE_PRIORITY_MASK 0x7 +#define KVM_XIVE_SOURCE_SERVER_SHIFT 3 +#define KVM_XIVE_SOURCE_SERVER_MASK 0xfffffff8ULL +#define KVM_XIVE_SOURCE_MASKED_SHIFT 32 +#define KVM_XIVE_SOURCE_MASKED_MASK 0x100000000ULL +#define KVM_XIVE_SOURCE_EISN_SHIFT 33 +#define KVM_XIVE_SOURCE_EISN_MASK 0xfffffffe00000000ULL + +/* Layout of 64-bit EQ identifier */ +#define KVM_XIVE_EQ_PRIORITY_SHIFT 0 +#define KVM_XIVE_EQ_PRIORITY_MASK 0x7 +#define KVM_XIVE_EQ_SERVER_SHIFT 3 +#define KVM_XIVE_EQ_SERVER_MASK 0xfffffff8ULL + +/* Layout of EQ configuration values (64 bytes) */ +struct kvm_ppc_xive_eq { + __u32 flags; + __u32 qshift; + __u64 qaddr; + __u32 qtoggle; + __u32 qindex; + __u8 pad[40]; +}; + +#define KVM_XIVE_EQ_ALWAYS_NOTIFY 0x00000001 + +#define KVM_XIVE_TIMA_PAGE_OFFSET 0 +#define KVM_XIVE_ESB_PAGE_OFFSET 4 + #endif /* __LINUX_KVM_POWERPC_H */ diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h index b53ee5974802..f78f9e072f7b 100644 --- a/linux-headers/linux/kvm.h +++ b/linux-headers/linux/kvm.h @@ -988,6 +988,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_ARM_VM_IPA_SIZE 165 #define KVM_CAP_MANUAL_DIRTY_LOG_PROTECT 166 #define KVM_CAP_HYPERV_CPUID 167 +#define KVM_CAP_PPC_IRQ_XIVE 168 =20 #ifdef KVM_CAP_IRQ_ROUTING =20 @@ -1211,6 +1212,8 @@ enum kvm_device_type { #define KVM_DEV_TYPE_ARM_VGIC_V3 KVM_DEV_TYPE_ARM_VGIC_V3 KVM_DEV_TYPE_ARM_VGIC_ITS, #define KVM_DEV_TYPE_ARM_VGIC_ITS KVM_DEV_TYPE_ARM_VGIC_ITS + KVM_DEV_TYPE_XIVE, +#define KVM_DEV_TYPE_XIVE KVM_DEV_TYPE_XIVE KVM_DEV_TYPE_MAX, }; =20 --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557737253; cv=none; d=zoho.com; s=zohoarc; b=iKhYjHcF8ssv/tCjcj6yv5ZAOAY1lKqwdkd19ECm3X70cUBxUF4gpqpwTR4UPkBYF/Nyz04fnYbv1V8e6GNEVXSZtRntvdIllsglwR8mqBfTBXMjO/F+X4EogqaZS/fVpZJqJ0Lrtne4VQe52Geqm0ZRNzsMCENLNE0sQWOXY4w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557737253; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=SNs+Rwvxn39q2zTunIzfCSc+tjCzoPpQcS3RhMC1w4c=; b=UT5P3T8CRqOXHUOUiTfAzThk+MW80af2pzq/mgSNIfPa7xnfvxEOYRwGzvZYOoNoRMHpGUsR7qV4oLqVZnfrRhPaKD+mrSdDV/14J6V4OkeRDCspA8SdfofoTY0ZqQvbnAUmrdFdmD/r2ktvy0hpBzLPm6YlO6e54+DPayOT/d0= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557737253245955.0748148845321; Mon, 13 May 2019 01:47:33 -0700 (PDT) Received: from localhost ([127.0.0.1]:53350 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6cK-000519-1Q for importer@patchew.org; Mon, 13 May 2019 04:47:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59184) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6YF-00022A-27 for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6YC-0003xS-Lp for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:15 -0400 Received: from 3.mo5.mail-out.ovh.net ([46.105.40.108]:34482) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6YC-0003wl-Cz for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:12 -0400 Received: from player778.ha.ovh.net (unknown [10.108.57.245]) by mo5.mail-out.ovh.net (Postfix) with ESMTP id 969322338FA for ; Mon, 13 May 2019 10:43:10 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 9A9B659C3C80; Mon, 13 May 2019 08:43:02 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:33 +0200 Message-Id: <20190513084245.25755-3-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3323093576631552998 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.40.108 Subject: [Qemu-devel] [PATCH v4 02/14] spapr/xive: add KVM support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" This introduces a set of helpers when KVM is in use, which create the KVM XIVE device, initialize the interrupt sources at a KVM level and connect the interrupt presenters to the vCPU. They also handle the initialization of the TIMA and the source ESB memory regions of the controller. These have a different type under KVM. They are 'ram device' memory mappings, similarly to VFIO, exposed to the guest and the associated VMAs on the host are populated dynamically with the appropriate pages using a fault handler. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- include/hw/ppc/spapr_xive.h | 10 ++ include/hw/ppc/xive.h | 13 ++ target/ppc/kvm_ppc.h | 6 + hw/intc/spapr_xive.c | 48 +++++++- hw/intc/spapr_xive_kvm.c | 237 ++++++++++++++++++++++++++++++++++++ hw/intc/xive.c | 21 +++- hw/ppc/spapr_irq.c | 6 +- target/ppc/kvm.c | 7 ++ hw/intc/Makefile.objs | 1 + hw/ppc/Kconfig | 5 + 10 files changed, 344 insertions(+), 10 deletions(-) create mode 100644 hw/intc/spapr_xive_kvm.c diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index fc3e9652f99a..0edcc762dedd 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -38,6 +38,10 @@ typedef struct SpaprXive { /* TIMA mapping address */ hwaddr tm_base; MemoryRegion tm_mmio; + + /* KVM support */ + int fd; + void *tm_mmap; } SpaprXive; =20 bool spapr_xive_irq_claim(SpaprXive *xive, uint32_t lisn, bool lsi); @@ -49,5 +53,11 @@ void spapr_dt_xive(SpaprMachineState *spapr, uint32_t nr= _servers, void *fdt, uint32_t phandle); void spapr_xive_set_tctx_os_cam(XiveTCTX *tctx); void spapr_xive_mmio_set_enabled(SpaprXive *xive, bool enable); +void spapr_xive_map_mmio(SpaprXive *xive); + +/* + * KVM XIVE device helpers + */ +void kvmppc_xive_connect(SpaprXive *xive, Error **errp); =20 #endif /* PPC_SPAPR_XIVE_H */ diff --git a/include/hw/ppc/xive.h b/include/hw/ppc/xive.h index c4f27742ca09..dd115da30ebc 100644 --- a/include/hw/ppc/xive.h +++ b/include/hw/ppc/xive.h @@ -140,6 +140,7 @@ #ifndef PPC_XIVE_H #define PPC_XIVE_H =20 +#include "sysemu/kvm.h" #include "hw/qdev-core.h" #include "hw/sysbus.h" #include "hw/ppc/xive_regs.h" @@ -194,6 +195,9 @@ typedef struct XiveSource { uint32_t esb_shift; MemoryRegion esb_mmio; =20 + /* KVM support */ + void *esb_mmap; + XiveNotifier *xive; } XiveSource; =20 @@ -423,4 +427,13 @@ static inline uint32_t xive_nvt_cam_line(uint8_t nvt_b= lk, uint32_t nvt_idx) return (nvt_blk << 19) | nvt_idx; } =20 +/* + * KVM XIVE device helpers + */ + +void kvmppc_xive_source_reset_one(XiveSource *xsrc, int srcno, Error **err= p); +void kvmppc_xive_source_reset(XiveSource *xsrc, Error **errp); +void kvmppc_xive_source_set_irq(void *opaque, int srcno, int val); +void kvmppc_xive_cpu_connect(XiveTCTX *tctx, Error **errp); + #endif /* PPC_XIVE_H */ diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h index 22385134b44d..45776cad79d9 100644 --- a/target/ppc/kvm_ppc.h +++ b/target/ppc/kvm_ppc.h @@ -60,6 +60,7 @@ bool kvmppc_has_cap_fixup_hcalls(void); bool kvmppc_has_cap_htm(void); bool kvmppc_has_cap_mmu_radix(void); bool kvmppc_has_cap_mmu_hash_v3(void); +bool kvmppc_has_cap_xive(void); int kvmppc_get_cap_safe_cache(void); int kvmppc_get_cap_safe_bounds_check(void); int kvmppc_get_cap_safe_indirect_branch(void); @@ -316,6 +317,11 @@ static inline bool kvmppc_has_cap_mmu_hash_v3(void) return false; } =20 +static inline bool kvmppc_has_cap_xive(void) +{ + return false; +} + static inline int kvmppc_get_cap_safe_cache(void) { return 0; diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index df3c879826b4..1390fc109c80 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -174,7 +174,7 @@ void spapr_xive_pic_print_info(SpaprXive *xive, Monitor= *mon) } } =20 -static void spapr_xive_map_mmio(SpaprXive *xive) +void spapr_xive_map_mmio(SpaprXive *xive) { sysbus_mmio_map(SYS_BUS_DEVICE(xive), 0, xive->vc_base); sysbus_mmio_map(SYS_BUS_DEVICE(xive), 1, xive->end_base); @@ -251,6 +251,9 @@ static void spapr_xive_instance_init(Object *obj) object_initialize_child(obj, "end_source", &xive->end_source, sizeof(xive->end_source), TYPE_XIVE_END_SOURCE, &error_abort, NULL); + + /* Not connected to the KVM XIVE device */ + xive->fd =3D -1; } =20 static void spapr_xive_realize(DeviceState *dev, Error **errp) @@ -259,6 +262,7 @@ static void spapr_xive_realize(DeviceState *dev, Error = **errp) XiveSource *xsrc =3D &xive->source; XiveENDSource *end_xsrc =3D &xive->end_source; Error *local_err =3D NULL; + MachineState *machine =3D MACHINE(qdev_get_machine()); =20 if (!xive->nr_irqs) { error_setg(errp, "Number of interrupt needs to be greater 0"); @@ -305,6 +309,32 @@ static void spapr_xive_realize(DeviceState *dev, Error= **errp) xive->eat =3D g_new0(XiveEAS, xive->nr_irqs); xive->endt =3D g_new0(XiveEND, xive->nr_ends); =20 + xive->nodename =3D g_strdup_printf("interrupt-controller@%" PRIx64, + xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SH= IFT)); + + qemu_register_reset(spapr_xive_reset, dev); + + if (kvm_enabled() && machine_kernel_irqchip_allowed(machine)) { + kvmppc_xive_connect(xive, &local_err); + if (local_err && machine_kernel_irqchip_required(machine)) { + error_prepend(&local_err, + "kernel_irqchip requested but unavailable: "); + error_propagate(errp, local_err); + return; + } + + if (!local_err) { + return; + } + + /* + * We failed to initialize the XIVE KVM device, fallback to + * emulated mode + */ + error_prepend(&local_err, "kernel_irqchip allowed but unavailable:= "); + warn_report_err(local_err); + } + /* TIMA initialization */ memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &xive_tm_ops, xive, "xive.tima", 4ull << TM_SHIFT); @@ -316,11 +346,6 @@ static void spapr_xive_realize(DeviceState *dev, Error= **errp) =20 /* Map all regions */ spapr_xive_map_mmio(xive); - - xive->nodename =3D g_strdup_printf("interrupt-controller@%" PRIx64, - xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SH= IFT)); - - qemu_register_reset(spapr_xive_reset, dev); } =20 static int spapr_xive_get_eas(XiveRouter *xrtr, uint8_t eas_blk, @@ -495,6 +520,17 @@ bool spapr_xive_irq_claim(SpaprXive *xive, uint32_t li= sn, bool lsi) if (lsi) { xive_source_irq_set_lsi(xsrc, lisn); } + + if (kvm_irqchip_in_kernel()) { + Error *local_err =3D NULL; + + kvmppc_xive_source_reset_one(xsrc, lisn, &local_err); + if (local_err) { + error_report_err(local_err); + return false; + } + } + return true; } =20 diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c new file mode 100644 index 000000000000..7d9e771e8a91 --- /dev/null +++ b/hw/intc/spapr_xive_kvm.c @@ -0,0 +1,237 @@ +/* + * QEMU PowerPC sPAPR XIVE interrupt controller model + * + * Copyright (c) 2017-2019, IBM Corporation. + * + * This code is licensed under the GPL version 2 or later. See the + * COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "qemu/error-report.h" +#include "qapi/error.h" +#include "target/ppc/cpu.h" +#include "sysemu/cpus.h" +#include "sysemu/kvm.h" +#include "hw/ppc/spapr.h" +#include "hw/ppc/spapr_xive.h" +#include "hw/ppc/xive.h" +#include "kvm_ppc.h" + +#include + +/* + * Helpers for CPU hotplug + * + * TODO: make a common KVMEnabledCPU layer for XICS and XIVE + */ +typedef struct KVMEnabledCPU { + unsigned long vcpu_id; + QLIST_ENTRY(KVMEnabledCPU) node; +} KVMEnabledCPU; + +static QLIST_HEAD(, KVMEnabledCPU) + kvm_enabled_cpus =3D QLIST_HEAD_INITIALIZER(&kvm_enabled_cpus); + +static bool kvm_cpu_is_enabled(CPUState *cs) +{ + KVMEnabledCPU *enabled_cpu; + unsigned long vcpu_id =3D kvm_arch_vcpu_id(cs); + + QLIST_FOREACH(enabled_cpu, &kvm_enabled_cpus, node) { + if (enabled_cpu->vcpu_id =3D=3D vcpu_id) { + return true; + } + } + return false; +} + +static void kvm_cpu_enable(CPUState *cs) +{ + KVMEnabledCPU *enabled_cpu; + unsigned long vcpu_id =3D kvm_arch_vcpu_id(cs); + + enabled_cpu =3D g_malloc(sizeof(*enabled_cpu)); + enabled_cpu->vcpu_id =3D vcpu_id; + QLIST_INSERT_HEAD(&kvm_enabled_cpus, enabled_cpu, node); +} + +/* + * XIVE Thread Interrupt Management context (KVM) + */ + +void kvmppc_xive_cpu_connect(XiveTCTX *tctx, Error **errp) +{ + SpaprXive *xive =3D SPAPR_MACHINE(qdev_get_machine())->xive; + unsigned long vcpu_id; + int ret; + + /* Check if CPU was hot unplugged and replugged. */ + if (kvm_cpu_is_enabled(tctx->cs)) { + return; + } + + vcpu_id =3D kvm_arch_vcpu_id(tctx->cs); + + ret =3D kvm_vcpu_enable_cap(tctx->cs, KVM_CAP_PPC_IRQ_XIVE, 0, xive->f= d, + vcpu_id, 0); + if (ret < 0) { + error_setg(errp, "XIVE: unable to connect CPU%ld to KVM device: %s= ", + vcpu_id, strerror(errno)); + return; + } + + kvm_cpu_enable(tctx->cs); +} + +/* + * XIVE Interrupt Source (KVM) + */ + +/* + * At reset, the interrupt sources are simply created and MASKED. We + * only need to inform the KVM XIVE device about their type: LSI or + * MSI. + */ +void kvmppc_xive_source_reset_one(XiveSource *xsrc, int srcno, Error **err= p) +{ + SpaprXive *xive =3D SPAPR_XIVE(xsrc->xive); + uint64_t state =3D 0; + + if (xive_source_irq_is_lsi(xsrc, srcno)) { + state |=3D KVM_XIVE_LEVEL_SENSITIVE; + if (xsrc->status[srcno] & XIVE_STATUS_ASSERTED) { + state |=3D KVM_XIVE_LEVEL_ASSERTED; + } + } + + kvm_device_access(xive->fd, KVM_DEV_XIVE_GRP_SOURCE, srcno, &state, + true, errp); +} + +void kvmppc_xive_source_reset(XiveSource *xsrc, Error **errp) +{ + int i; + + for (i =3D 0; i < xsrc->nr_irqs; i++) { + Error *local_err =3D NULL; + + kvmppc_xive_source_reset_one(xsrc, i, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + } +} + +void kvmppc_xive_source_set_irq(void *opaque, int srcno, int val) +{ + XiveSource *xsrc =3D opaque; + struct kvm_irq_level args; + int rc; + + args.irq =3D srcno; + if (!xive_source_irq_is_lsi(xsrc, srcno)) { + if (!val) { + return; + } + args.level =3D KVM_INTERRUPT_SET; + } else { + if (val) { + xsrc->status[srcno] |=3D XIVE_STATUS_ASSERTED; + args.level =3D KVM_INTERRUPT_SET_LEVEL; + } else { + xsrc->status[srcno] &=3D ~XIVE_STATUS_ASSERTED; + args.level =3D KVM_INTERRUPT_UNSET; + } + } + rc =3D kvm_vm_ioctl(kvm_state, KVM_IRQ_LINE, &args); + if (rc < 0) { + error_report("XIVE: kvm_irq_line() failed : %s", strerror(errno)); + } +} + +/* + * sPAPR XIVE interrupt controller (KVM) + */ + +static void *kvmppc_xive_mmap(SpaprXive *xive, int pgoff, size_t len, + Error **errp) +{ + void *addr; + uint32_t page_shift =3D 16; /* TODO: fix page_shift */ + + addr =3D mmap(NULL, len, PROT_WRITE | PROT_READ, MAP_SHARED, xive->fd, + pgoff << page_shift); + if (addr =3D=3D MAP_FAILED) { + error_setg_errno(errp, errno, "XIVE: unable to set memory mapping"= ); + return NULL; + } + + return addr; +} + +/* + * All the XIVE memory regions are now backed by mappings from the KVM + * XIVE device. + */ +void kvmppc_xive_connect(SpaprXive *xive, Error **errp) +{ + XiveSource *xsrc =3D &xive->source; + XiveENDSource *end_xsrc =3D &xive->end_source; + Error *local_err =3D NULL; + size_t esb_len =3D (1ull << xsrc->esb_shift) * xsrc->nr_irqs; + size_t tima_len =3D 4ull << TM_SHIFT; + + if (!kvmppc_has_cap_xive()) { + error_setg(errp, "IRQ_XIVE capability must be present for KVM"); + return; + } + + /* First, create the KVM XIVE device */ + xive->fd =3D kvm_create_device(kvm_state, KVM_DEV_TYPE_XIVE, false); + if (xive->fd < 0) { + error_setg_errno(errp, -xive->fd, "XIVE: error creating KVM device= "); + return; + } + + /* + * 1. Source ESB pages - KVM mapping + */ + xsrc->esb_mmap =3D kvmppc_xive_mmap(xive, KVM_XIVE_ESB_PAGE_OFFSET, es= b_len, + &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + + memory_region_init_ram_device_ptr(&xsrc->esb_mmio, OBJECT(xsrc), + "xive.esb", esb_len, xsrc->esb_mmap); + sysbus_init_mmio(SYS_BUS_DEVICE(xive), &xsrc->esb_mmio); + + /* + * 2. END ESB pages (No KVM support yet) + */ + sysbus_init_mmio(SYS_BUS_DEVICE(xive), &end_xsrc->esb_mmio); + + /* + * 3. TIMA pages - KVM mapping + */ + xive->tm_mmap =3D kvmppc_xive_mmap(xive, KVM_XIVE_TIMA_PAGE_OFFSET, ti= ma_len, + &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + memory_region_init_ram_device_ptr(&xive->tm_mmio, OBJECT(xive), + "xive.tima", tima_len, xive->tm_mmap= ); + sysbus_init_mmio(SYS_BUS_DEVICE(xive), &xive->tm_mmio); + + kvm_kernel_irqchip =3D true; + kvm_msi_via_irqfd_allowed =3D true; + kvm_gsi_direct_mapping =3D true; + + /* Map all regions */ + spapr_xive_map_mmio(xive); +} diff --git a/hw/intc/xive.c b/hw/intc/xive.c index dcf2fcd10893..78047adb11e6 100644 --- a/hw/intc/xive.c +++ b/hw/intc/xive.c @@ -555,6 +555,15 @@ static void xive_tctx_realize(DeviceState *dev, Error = **errp) return; } =20 + /* Connect the presenter to the VCPU (required for CPU hotplug) */ + if (kvm_irqchip_in_kernel()) { + kvmppc_xive_cpu_connect(tctx, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + } + qemu_register_reset(xive_tctx_reset, dev); } =20 @@ -957,6 +966,10 @@ static void xive_source_reset(void *dev) =20 /* PQs are initialized to 0b01 (Q=3D1) which corresponds to "ints off"= */ memset(xsrc->status, XIVE_ESB_OFF, xsrc->nr_irqs); + + if (kvm_irqchip_in_kernel()) { + kvmppc_xive_source_reset(xsrc, &error_fatal); + } } =20 static void xive_source_realize(DeviceState *dev, Error **errp) @@ -990,9 +1003,11 @@ static void xive_source_realize(DeviceState *dev, Err= or **errp) xsrc->status =3D g_malloc0(xsrc->nr_irqs); xsrc->lsi_map =3D bitmap_new(xsrc->nr_irqs); =20 - memory_region_init_io(&xsrc->esb_mmio, OBJECT(xsrc), - &xive_source_esb_ops, xsrc, "xive.esb", - (1ull << xsrc->esb_shift) * xsrc->nr_irqs); + if (!kvm_irqchip_in_kernel()) { + memory_region_init_io(&xsrc->esb_mmio, OBJECT(xsrc), + &xive_source_esb_ops, xsrc, "xive.esb", + (1ull << xsrc->esb_shift) * xsrc->nr_irqs); + } =20 qemu_register_reset(xive_source_reset, dev); } diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index b1f79ea9def6..5c4a44855d0a 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -372,7 +372,11 @@ static void spapr_irq_set_irq_xive(void *opaque, int s= rcno, int val) { SpaprMachineState *spapr =3D opaque; =20 - xive_source_set_irq(&spapr->xive->source, srcno, val); + if (kvm_irqchip_in_kernel()) { + kvmppc_xive_source_set_irq(&spapr->xive->source, srcno, val); + } else { + xive_source_set_irq(&spapr->xive->source, srcno, val); + } } =20 static const char *spapr_irq_get_nodename_xive(SpaprMachineState *spapr) diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c index 1a9caf8f40a7..3bf0a46c3352 100644 --- a/target/ppc/kvm.c +++ b/target/ppc/kvm.c @@ -75,6 +75,7 @@ static int cap_fixup_hcalls; static int cap_htm; /* Hardware transactional memory support */ static int cap_mmu_radix; static int cap_mmu_hash_v3; +static int cap_xive; static int cap_resize_hpt; static int cap_ppc_pvr_compat; static int cap_ppc_safe_cache; @@ -146,6 +147,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_htm =3D kvm_vm_check_extension(s, KVM_CAP_PPC_HTM); cap_mmu_radix =3D kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_RADIX); cap_mmu_hash_v3 =3D kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3); + cap_xive =3D kvm_vm_check_extension(s, KVM_CAP_PPC_IRQ_XIVE); cap_resize_hpt =3D kvm_vm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT); kvmppc_get_cpu_characteristics(s); cap_ppc_nested_kvm_hv =3D kvm_vm_check_extension(s, KVM_CAP_PPC_NESTED= _HV); @@ -2478,6 +2480,11 @@ static int parse_cap_ppc_count_cache_flush_assist(st= ruct kvm_ppc_cpu_char c) return 0; } =20 +bool kvmppc_has_cap_xive(void) +{ + return cap_xive; +} + static void kvmppc_get_cpu_characteristics(KVMState *s) { struct kvm_ppc_cpu_char c; diff --git a/hw/intc/Makefile.objs b/hw/intc/Makefile.objs index df712c3e6c93..03019b9a0343 100644 --- a/hw/intc/Makefile.objs +++ b/hw/intc/Makefile.objs @@ -39,6 +39,7 @@ obj-$(CONFIG_XICS_SPAPR) +=3D xics_spapr.o obj-$(CONFIG_XICS_KVM) +=3D xics_kvm.o obj-$(CONFIG_XIVE) +=3D xive.o obj-$(CONFIG_XIVE_SPAPR) +=3D spapr_xive.o +obj-$(CONFIG_XIVE_KVM) +=3D spapr_xive_kvm.o obj-$(CONFIG_POWERNV) +=3D xics_pnv.o pnv_xive.o obj-$(CONFIG_ALLWINNER_A10_PIC) +=3D allwinner-a10-pic.o obj-$(CONFIG_S390_FLIC) +=3D s390_flic.o diff --git a/hw/ppc/Kconfig b/hw/ppc/Kconfig index a3465155f047..f927ec9c7487 100644 --- a/hw/ppc/Kconfig +++ b/hw/ppc/Kconfig @@ -122,3 +122,8 @@ config XIVE_SPAPR default y depends on PSERIES select XIVE + +config XIVE_KVM + bool + default y + depends on XIVE_SPAPR && KVM --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557737191; cv=none; d=zoho.com; s=zohoarc; b=TK9KkhXIRZj70aT/DC8CXEA+Gk8wJ1/vhE60F3w7NUNknO56S23b3XsmKHFrDb/+f1q97KvDqwBQ1p4Fz33elQwsIQqYQcn6HNRqEqR+mqMIIVG5nwIvfdMQl5qEBn1mdx+1oTTohWL+hUvAAlhSv86s4ymTYlllwNYdy5fUWVc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557737191; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=Nzg2EL8WKaZmsk0sLoW/RMDDnzeOCBII0bxeBCCY69w=; b=nWZukm+jU+FsVV6pCf0WckDVKr9LU/PXaa29Ov10fS7R42UcNDJab/3bMc+9ffzhxhKfSmGXgcw/UUM/PwUxfq2FJ5sK9VmCapEQJHTusGnrLcl9Rkm6/fy1CkcwocEm/wUIKLo9VBn36Z0xjAgvO7022BZvH/tFAYZEz6QIs0s= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557737191587571.9292399047303; Mon, 13 May 2019 01:46:31 -0700 (PDT) Received: from localhost ([127.0.0.1]:53342 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6bG-00049h-DM for importer@patchew.org; Mon, 13 May 2019 04:46:22 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59217) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6YJ-00024c-NM for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6YH-0003zd-ON for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:19 -0400 Received: from 12.mo5.mail-out.ovh.net ([46.105.39.65]:48599) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6YH-0003z7-FI for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:17 -0400 Received: from player778.ha.ovh.net (unknown [10.109.146.163]) by mo5.mail-out.ovh.net (Postfix) with ESMTP id 2320123392A for ; Mon, 13 May 2019 10:43:16 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 6DA5259C3D0B; Mon, 13 May 2019 08:43:10 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:34 +0200 Message-Id: <20190513084245.25755-4-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3324782427746634726 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.39.65 Subject: [Qemu-devel] [PATCH v4 03/14] spapr/xive: add hcall support when under KVM X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" XIVE hcalls are all redirected to QEMU as none are on a fast path. When necessary, QEMU invokes KVM through specific ioctls to perform host operations. QEMU should have done the necessary checks before calling KVM and, in case of failure, H_HARDWARE is simply returned. H_INT_ESB is a special case that could have been handled under KVM but the impact on performance was low when under QEMU. Here are some figures : kernel irqchip OFF ON H_INT_ESB KVM QEMU rtl8139 (LSI ) 1.19 1.24 1.23 Gbits/sec virtio 31.80 42.30 -- Gbits/sec Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- Changes since v3: - added usage of xive_end_qaddr() helper include/hw/ppc/spapr_xive.h | 15 +++ hw/intc/spapr_xive.c | 90 ++++++++++++++-- hw/intc/spapr_xive_kvm.c | 197 ++++++++++++++++++++++++++++++++++++ 3 files changed, 294 insertions(+), 8 deletions(-) diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index 0edcc762dedd..03685910e76b 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -55,9 +55,24 @@ void spapr_xive_set_tctx_os_cam(XiveTCTX *tctx); void spapr_xive_mmio_set_enabled(SpaprXive *xive, bool enable); void spapr_xive_map_mmio(SpaprXive *xive); =20 +int spapr_xive_end_to_target(uint8_t end_blk, uint32_t end_idx, + uint32_t *out_server, uint8_t *out_prio); + /* * KVM XIVE device helpers */ void kvmppc_xive_connect(SpaprXive *xive, Error **errp); +void kvmppc_xive_reset(SpaprXive *xive, Error **errp); +void kvmppc_xive_set_source_config(SpaprXive *xive, uint32_t lisn, XiveEAS= *eas, + Error **errp); +void kvmppc_xive_sync_source(SpaprXive *xive, uint32_t lisn, Error **errp); +uint64_t kvmppc_xive_esb_rw(XiveSource *xsrc, int srcno, uint32_t offset, + uint64_t data, bool write); +void kvmppc_xive_set_queue_config(SpaprXive *xive, uint8_t end_blk, + uint32_t end_idx, XiveEND *end, + Error **errp); +void kvmppc_xive_get_queue_config(SpaprXive *xive, uint8_t end_blk, + uint32_t end_idx, XiveEND *end, + Error **errp); =20 #endif /* PPC_SPAPR_XIVE_H */ diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index 1390fc109c80..6beda26ee440 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -86,6 +86,22 @@ static int spapr_xive_target_to_nvt(uint32_t target, * sPAPR END indexing uses a simple mapping of the CPU vcpu_id, 8 * priorities per CPU */ +int spapr_xive_end_to_target(uint8_t end_blk, uint32_t end_idx, + uint32_t *out_server, uint8_t *out_prio) +{ + + assert(end_blk =3D=3D SPAPR_XIVE_BLOCK_ID); + + if (out_server) { + *out_server =3D end_idx >> 3; + } + + if (out_prio) { + *out_prio =3D end_idx & 0x7; + } + return 0; +} + static void spapr_xive_cpu_to_end(PowerPCCPU *cpu, uint8_t prio, uint8_t *out_end_blk, uint32_t *out_end_= idx) { @@ -792,6 +808,16 @@ static target_ulong h_int_set_source_config(PowerPCCPU= *cpu, new_eas.w =3D xive_set_field64(EAS_END_DATA, new_eas.w, eisn); } =20 + if (kvm_irqchip_in_kernel()) { + Error *local_err =3D NULL; + + kvmppc_xive_set_source_config(xive, lisn, &new_eas, &local_err); + if (local_err) { + error_report_err(local_err); + return H_HARDWARE; + } + } + out: xive->eat[lisn] =3D new_eas; return H_SUCCESS; @@ -1103,6 +1129,16 @@ static target_ulong h_int_set_queue_config(PowerPCCP= U *cpu, */ =20 out: + if (kvm_irqchip_in_kernel()) { + Error *local_err =3D NULL; + + kvmppc_xive_set_queue_config(xive, end_blk, end_idx, &end, &local_= err); + if (local_err) { + error_report_err(local_err); + return H_HARDWARE; + } + } + /* Update END */ memcpy(&xive->endt[end_idx], &end, sizeof(XiveEND)); return H_SUCCESS; @@ -1194,6 +1230,16 @@ static target_ulong h_int_get_queue_config(PowerPCCP= U *cpu, args[2] =3D 0; } =20 + if (kvm_irqchip_in_kernel()) { + Error *local_err =3D NULL; + + kvmppc_xive_get_queue_config(xive, end_blk, end_idx, end, &local_e= rr); + if (local_err) { + error_report_err(local_err); + return H_HARDWARE; + } + } + /* TODO: do we need any locking on the END ? */ if (flags & SPAPR_XIVE_END_DEBUG) { /* Load the event queue generation number into the return flags */ @@ -1346,15 +1392,20 @@ static target_ulong h_int_esb(PowerPCCPU *cpu, return H_P3; } =20 - mmio_addr =3D xive->vc_base + xive_source_esb_mgmt(xsrc, lisn) + offse= t; + if (kvm_irqchip_in_kernel()) { + args[0] =3D kvmppc_xive_esb_rw(xsrc, lisn, offset, data, + flags & SPAPR_XIVE_ESB_STORE); + } else { + mmio_addr =3D xive->vc_base + xive_source_esb_mgmt(xsrc, lisn) + o= ffset; =20 - if (dma_memory_rw(&address_space_memory, mmio_addr, &data, 8, - (flags & SPAPR_XIVE_ESB_STORE))) { - qemu_log_mask(LOG_GUEST_ERROR, "XIVE: failed to access ESB @0x%" - HWADDR_PRIx "\n", mmio_addr); - return H_HARDWARE; + if (dma_memory_rw(&address_space_memory, mmio_addr, &data, 8, + (flags & SPAPR_XIVE_ESB_STORE))) { + qemu_log_mask(LOG_GUEST_ERROR, "XIVE: failed to access ESB @0x= %" + HWADDR_PRIx "\n", mmio_addr); + return H_HARDWARE; + } + args[0] =3D (flags & SPAPR_XIVE_ESB_STORE) ? -1 : data; } - args[0] =3D (flags & SPAPR_XIVE_ESB_STORE) ? -1 : data; return H_SUCCESS; } =20 @@ -1411,7 +1462,20 @@ static target_ulong h_int_sync(PowerPCCPU *cpu, * This is not needed when running the emulation under QEMU */ =20 - /* This is not real hardware. Nothing to be done */ + /* + * This is not real hardware. Nothing to be done unless when + * under KVM + */ + + if (kvm_irqchip_in_kernel()) { + Error *local_err =3D NULL; + + kvmppc_xive_sync_source(xive, lisn, &local_err); + if (local_err) { + error_report_err(local_err); + return H_HARDWARE; + } + } return H_SUCCESS; } =20 @@ -1446,6 +1510,16 @@ static target_ulong h_int_reset(PowerPCCPU *cpu, } =20 device_reset(DEVICE(xive)); + + if (kvm_irqchip_in_kernel()) { + Error *local_err =3D NULL; + + kvmppc_xive_reset(xive, &local_err); + if (local_err) { + error_report_err(local_err); + return H_HARDWARE; + } + } return H_SUCCESS; } =20 diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c index 7d9e771e8a91..964bad0c23fb 100644 --- a/hw/intc/spapr_xive_kvm.c +++ b/hw/intc/spapr_xive_kvm.c @@ -89,6 +89,50 @@ void kvmppc_xive_cpu_connect(XiveTCTX *tctx, Error **err= p) * XIVE Interrupt Source (KVM) */ =20 +void kvmppc_xive_set_source_config(SpaprXive *xive, uint32_t lisn, XiveEAS= *eas, + Error **errp) +{ + uint32_t end_idx; + uint32_t end_blk; + uint8_t priority; + uint32_t server; + bool masked; + uint32_t eisn; + uint64_t kvm_src; + Error *local_err =3D NULL; + + assert(xive_eas_is_valid(eas)); + + end_idx =3D xive_get_field64(EAS_END_INDEX, eas->w); + end_blk =3D xive_get_field64(EAS_END_BLOCK, eas->w); + eisn =3D xive_get_field64(EAS_END_DATA, eas->w); + masked =3D xive_eas_is_masked(eas); + + spapr_xive_end_to_target(end_blk, end_idx, &server, &priority); + + kvm_src =3D priority << KVM_XIVE_SOURCE_PRIORITY_SHIFT & + KVM_XIVE_SOURCE_PRIORITY_MASK; + kvm_src |=3D server << KVM_XIVE_SOURCE_SERVER_SHIFT & + KVM_XIVE_SOURCE_SERVER_MASK; + kvm_src |=3D ((uint64_t) masked << KVM_XIVE_SOURCE_MASKED_SHIFT) & + KVM_XIVE_SOURCE_MASKED_MASK; + kvm_src |=3D ((uint64_t)eisn << KVM_XIVE_SOURCE_EISN_SHIFT) & + KVM_XIVE_SOURCE_EISN_MASK; + + kvm_device_access(xive->fd, KVM_DEV_XIVE_GRP_SOURCE_CONFIG, lisn, + &kvm_src, true, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } +} + +void kvmppc_xive_sync_source(SpaprXive *xive, uint32_t lisn, Error **errp) +{ + kvm_device_access(xive->fd, KVM_DEV_XIVE_GRP_SOURCE_SYNC, lisn, + NULL, true, errp); +} + /* * At reset, the interrupt sources are simply created and MASKED. We * only need to inform the KVM XIVE device about their type: LSI or @@ -125,6 +169,64 @@ void kvmppc_xive_source_reset(XiveSource *xsrc, Error = **errp) } } =20 +/* + * This is used to perform the magic loads on the ESB pages, described + * in xive.h. + * + * Memory barriers should not be needed for loads (no store for now). + */ +static uint64_t xive_esb_rw(XiveSource *xsrc, int srcno, uint32_t offset, + uint64_t data, bool write) +{ + uint64_t *addr =3D xsrc->esb_mmap + xive_source_esb_mgmt(xsrc, srcno) + + offset; + + if (write) { + *addr =3D cpu_to_be64(data); + return -1; + } else { + /* Prevent the compiler from optimizing away the load */ + volatile uint64_t value =3D be64_to_cpu(*addr); + return value; + } +} + +static uint8_t xive_esb_read(XiveSource *xsrc, int srcno, uint32_t offset) +{ + return xive_esb_rw(xsrc, srcno, offset, 0, 0) & 0x3; +} + +static void xive_esb_trigger(XiveSource *xsrc, int srcno) +{ + uint64_t *addr =3D xsrc->esb_mmap + xive_source_esb_page(xsrc, srcno); + + *addr =3D 0x0; +} + +uint64_t kvmppc_xive_esb_rw(XiveSource *xsrc, int srcno, uint32_t offset, + uint64_t data, bool write) +{ + if (write) { + return xive_esb_rw(xsrc, srcno, offset, data, 1); + } + + /* + * Special Load EOI handling for LSI sources. Q bit is never set + * and the interrupt should be re-triggered if the level is still + * asserted. + */ + if (xive_source_irq_is_lsi(xsrc, srcno) && + offset =3D=3D XIVE_ESB_LOAD_EOI) { + xive_esb_read(xsrc, srcno, XIVE_ESB_SET_PQ_00); + if (xsrc->status[srcno] & XIVE_STATUS_ASSERTED) { + xive_esb_trigger(xsrc, srcno); + } + return 0; + } else { + return xive_esb_rw(xsrc, srcno, offset, 0, 0); + } +} + void kvmppc_xive_source_set_irq(void *opaque, int srcno, int val) { XiveSource *xsrc =3D opaque; @@ -155,6 +257,101 @@ void kvmppc_xive_source_set_irq(void *opaque, int src= no, int val) /* * sPAPR XIVE interrupt controller (KVM) */ +void kvmppc_xive_get_queue_config(SpaprXive *xive, uint8_t end_blk, + uint32_t end_idx, XiveEND *end, + Error **errp) +{ + struct kvm_ppc_xive_eq kvm_eq =3D { 0 }; + uint64_t kvm_eq_idx; + uint8_t priority; + uint32_t server; + Error *local_err =3D NULL; + + assert(xive_end_is_valid(end)); + + /* Encode the tuple (server, prio) as a KVM EQ index */ + spapr_xive_end_to_target(end_blk, end_idx, &server, &priority); + + kvm_eq_idx =3D priority << KVM_XIVE_EQ_PRIORITY_SHIFT & + KVM_XIVE_EQ_PRIORITY_MASK; + kvm_eq_idx |=3D server << KVM_XIVE_EQ_SERVER_SHIFT & + KVM_XIVE_EQ_SERVER_MASK; + + kvm_device_access(xive->fd, KVM_DEV_XIVE_GRP_EQ_CONFIG, kvm_eq_idx, + &kvm_eq, false, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + + /* + * The EQ index and toggle bit are updated by HW. These are the + * only fields from KVM we want to update QEMU with. The other END + * fields should already be in the QEMU END table. + */ + end->w1 =3D xive_set_field32(END_W1_GENERATION, 0ul, kvm_eq.qtoggle) | + xive_set_field32(END_W1_PAGE_OFF, 0ul, kvm_eq.qindex); +} + +void kvmppc_xive_set_queue_config(SpaprXive *xive, uint8_t end_blk, + uint32_t end_idx, XiveEND *end, + Error **errp) +{ + struct kvm_ppc_xive_eq kvm_eq =3D { 0 }; + uint64_t kvm_eq_idx; + uint8_t priority; + uint32_t server; + Error *local_err =3D NULL; + + /* + * Build the KVM state from the local END structure. + */ + + kvm_eq.flags =3D 0; + if (xive_get_field32(END_W0_UCOND_NOTIFY, end->w0)) { + kvm_eq.flags |=3D KVM_XIVE_EQ_ALWAYS_NOTIFY; + } + + /* + * If the hcall is disabling the EQ, set the size and page address + * to zero. When migrating, only valid ENDs are taken into + * account. + */ + if (xive_end_is_valid(end)) { + kvm_eq.qshift =3D xive_get_field32(END_W0_QSIZE, end->w0) + 12; + kvm_eq.qaddr =3D xive_end_qaddr(end); + /* + * The EQ toggle bit and index should only be relevant when + * restoring the EQ state + */ + kvm_eq.qtoggle =3D xive_get_field32(END_W1_GENERATION, end->w1); + kvm_eq.qindex =3D xive_get_field32(END_W1_PAGE_OFF, end->w1); + } else { + kvm_eq.qshift =3D 0; + kvm_eq.qaddr =3D 0; + } + + /* Encode the tuple (server, prio) as a KVM EQ index */ + spapr_xive_end_to_target(end_blk, end_idx, &server, &priority); + + kvm_eq_idx =3D priority << KVM_XIVE_EQ_PRIORITY_SHIFT & + KVM_XIVE_EQ_PRIORITY_MASK; + kvm_eq_idx |=3D server << KVM_XIVE_EQ_SERVER_SHIFT & + KVM_XIVE_EQ_SERVER_MASK; + + kvm_device_access(xive->fd, KVM_DEV_XIVE_GRP_EQ_CONFIG, kvm_eq_idx, + &kvm_eq, true, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } +} + +void kvmppc_xive_reset(SpaprXive *xive, Error **errp) +{ + kvm_device_access(xive->fd, KVM_DEV_XIVE_GRP_CTRL, KVM_DEV_XIVE_RESET, + NULL, true, errp); +} =20 static void *kvmppc_xive_mmap(SpaprXive *xive, int pgoff, size_t len, Error **errp) --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738753; cv=none; d=zoho.com; s=zohoarc; b=QQqpTlqOpiw4tWCH/Kbd7nV2Ja3Y1bdih383GjRmt6Kg0KxTGkpf8dO4m2lmO4UpODjs/wU0uKJf9Ybh8xA7HgZ9WLwWH/Jv9oxoQLF3DBtCGGrKJqPGhLNk44SRVCeLts+LyAWxgR9hqwPdD+w8OobydrYN9I/sWgANHGoK118= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738753; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=+9B6wX2N0HFwf07E5Yj5qdIgpUidKPJ0T47tod3ZYPQ=; b=eeCUeGTD4ivNgNY7Chkhw0J8WxQv2+hYumZZW7gTcnE74WD7OuPJk+3fgi0OMApkm2eb6u1kjeSY97KIEESwXeY1dq+jLDI4uzbVVd9hlTGBGVDrnZb0d4bWUuerjYXvU4JzVTYyRM0lxWh5//C4Oa1dKEyYEw1Bri37u+OM0wM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557738753331436.177203237159; Mon, 13 May 2019 02:12:33 -0700 (PDT) Received: from localhost ([127.0.0.1]:53721 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ70a-0007OA-Av for importer@patchew.org; Mon, 13 May 2019 05:12:32 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33622) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6oH-0006kh-4h for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6YQ-000434-1P for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:27 -0400 Received: from 4.mo69.mail-out.ovh.net ([46.105.42.102]:60346) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6YP-00042R-SJ for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:25 -0400 Received: from player778.ha.ovh.net (unknown [10.109.146.50]) by mo69.mail-out.ovh.net (Postfix) with ESMTP id CA06C54A20 for ; Mon, 13 May 2019 10:43:23 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 1B8D959C3D6C; Mon, 13 May 2019 08:43:16 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:35 +0200 Message-Id: <20190513084245.25755-5-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3326752752975973350 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.42.102 Subject: [Qemu-devel] [PATCH v4 04/14] spapr/xive: add state synchronization with KVM X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" This extends the KVM XIVE device backend with 'synchronize_state' methods used to retrieve the state from KVM. The HW state of the sources, the KVM device and the thread interrupt contexts are collected for the monitor usage and also migration. These get operations rely on their KVM counterpart in the host kernel which acts as a proxy for OPAL, the host firmware. The set operations will be added for migration support later. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- include/hw/ppc/spapr_xive.h | 8 ++++ include/hw/ppc/xive.h | 1 + hw/intc/spapr_xive.c | 17 ++++--- hw/intc/spapr_xive_kvm.c | 90 +++++++++++++++++++++++++++++++++++++ hw/intc/xive.c | 10 +++++ 5 files changed, 119 insertions(+), 7 deletions(-) diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index 03685910e76b..7e49badd8c9a 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -44,6 +44,13 @@ typedef struct SpaprXive { void *tm_mmap; } SpaprXive; =20 +/* + * The sPAPR machine has a unique XIVE IC device. Assign a fixed value + * to the controller block id value. It can nevertheless be changed + * for testing purpose. + */ +#define SPAPR_XIVE_BLOCK_ID 0x0 + bool spapr_xive_irq_claim(SpaprXive *xive, uint32_t lisn, bool lsi); bool spapr_xive_irq_free(SpaprXive *xive, uint32_t lisn); void spapr_xive_pic_print_info(SpaprXive *xive, Monitor *mon); @@ -74,5 +81,6 @@ void kvmppc_xive_set_queue_config(SpaprXive *xive, uint8_= t end_blk, void kvmppc_xive_get_queue_config(SpaprXive *xive, uint8_t end_blk, uint32_t end_idx, XiveEND *end, Error **errp); +void kvmppc_xive_synchronize_state(SpaprXive *xive, Error **errp); =20 #endif /* PPC_SPAPR_XIVE_H */ diff --git a/include/hw/ppc/xive.h b/include/hw/ppc/xive.h index dd115da30ebc..78c919c4a590 100644 --- a/include/hw/ppc/xive.h +++ b/include/hw/ppc/xive.h @@ -435,5 +435,6 @@ void kvmppc_xive_source_reset_one(XiveSource *xsrc, int= srcno, Error **errp); void kvmppc_xive_source_reset(XiveSource *xsrc, Error **errp); void kvmppc_xive_source_set_irq(void *opaque, int srcno, int val); void kvmppc_xive_cpu_connect(XiveTCTX *tctx, Error **errp); +void kvmppc_xive_cpu_synchronize_state(XiveTCTX *tctx, Error **errp); =20 #endif /* PPC_XIVE_H */ diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index 6beda26ee440..84cd30e1faff 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -40,13 +40,6 @@ =20 #define SPAPR_XIVE_NVT_BASE 0x400 =20 -/* - * The sPAPR machine has a unique XIVE IC device. Assign a fixed value - * to the controller block id value. It can nevertheless be changed - * for testing purpose. - */ -#define SPAPR_XIVE_BLOCK_ID 0x0 - /* * sPAPR NVT and END indexing helpers */ @@ -157,6 +150,16 @@ void spapr_xive_pic_print_info(SpaprXive *xive, Monito= r *mon) XiveSource *xsrc =3D &xive->source; int i; =20 + if (kvm_irqchip_in_kernel()) { + Error *local_err =3D NULL; + + kvmppc_xive_synchronize_state(xive, &local_err); + if (local_err) { + error_report_err(local_err); + return; + } + } + monitor_printf(mon, " LISN PQ EISN CPU/PRIO EQ\n"); =20 for (i =3D 0; i < xive->nr_irqs; i++) { diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c index 964bad0c23fb..8dd4f96e0b97 100644 --- a/hw/intc/spapr_xive_kvm.c +++ b/hw/intc/spapr_xive_kvm.c @@ -60,6 +60,54 @@ static void kvm_cpu_enable(CPUState *cs) /* * XIVE Thread Interrupt Management context (KVM) */ +static void kvmppc_xive_cpu_get_state(XiveTCTX *tctx, Error **errp) +{ + uint64_t state[2] =3D { 0 }; + int ret; + + ret =3D kvm_get_one_reg(tctx->cs, KVM_REG_PPC_VP_STATE, state); + if (ret !=3D 0) { + error_setg_errno(errp, errno, + "XIVE: could not capture KVM state of CPU %ld", + kvm_arch_vcpu_id(tctx->cs)); + return; + } + + /* word0 and word1 of the OS ring. */ + *((uint64_t *) &tctx->regs[TM_QW1_OS]) =3D state[0]; +} + +typedef struct { + XiveTCTX *tctx; + Error *err; +} XiveCpuGetState; + +static void kvmppc_xive_cpu_do_synchronize_state(CPUState *cpu, + run_on_cpu_data arg) +{ + XiveCpuGetState *s =3D arg.host_ptr; + + kvmppc_xive_cpu_get_state(s->tctx, &s->err); +} + +void kvmppc_xive_cpu_synchronize_state(XiveTCTX *tctx, Error **errp) +{ + XiveCpuGetState s =3D { + .tctx =3D tctx, + .err =3D NULL, + }; + + /* + * Kick the vCPU to make sure they are available for the KVM ioctl. + */ + run_on_cpu(tctx->cs, kvmppc_xive_cpu_do_synchronize_state, + RUN_ON_CPU_HOST_PTR(&s)); + + if (s.err) { + error_propagate(errp, s.err); + return; + } +} =20 void kvmppc_xive_cpu_connect(XiveTCTX *tctx, Error **errp) { @@ -227,6 +275,19 @@ uint64_t kvmppc_xive_esb_rw(XiveSource *xsrc, int srcn= o, uint32_t offset, } } =20 +static void kvmppc_xive_source_get_state(XiveSource *xsrc) +{ + int i; + + for (i =3D 0; i < xsrc->nr_irqs; i++) { + /* Perform a load without side effect to retrieve the PQ bits */ + uint8_t pq =3D xive_esb_read(xsrc, i, XIVE_ESB_GET); + + /* and save PQ locally */ + xive_source_esb_set(xsrc, i, pq); + } +} + void kvmppc_xive_source_set_irq(void *opaque, int srcno, int val) { XiveSource *xsrc =3D opaque; @@ -353,6 +414,35 @@ void kvmppc_xive_reset(SpaprXive *xive, Error **errp) NULL, true, errp); } =20 +static void kvmppc_xive_get_queues(SpaprXive *xive, Error **errp) +{ + Error *local_err =3D NULL; + int i; + + for (i =3D 0; i < xive->nr_ends; i++) { + if (!xive_end_is_valid(&xive->endt[i])) { + continue; + } + + kvmppc_xive_get_queue_config(xive, SPAPR_XIVE_BLOCK_ID, i, + &xive->endt[i], &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + } +} + +void kvmppc_xive_synchronize_state(SpaprXive *xive, Error **errp) +{ + kvmppc_xive_source_get_state(&xive->source); + + /* EAT: there is no extra state to query from KVM */ + + /* ENDT */ + kvmppc_xive_get_queues(xive, errp); +} + static void *kvmppc_xive_mmap(SpaprXive *xive, int pgoff, size_t len, Error **errp) { diff --git a/hw/intc/xive.c b/hw/intc/xive.c index 78047adb11e6..7f1c54a7b581 100644 --- a/hw/intc/xive.c +++ b/hw/intc/xive.c @@ -493,6 +493,16 @@ void xive_tctx_pic_print_info(XiveTCTX *tctx, Monitor = *mon) int cpu_index =3D tctx->cs ? tctx->cs->cpu_index : -1; int i; =20 + if (kvm_irqchip_in_kernel()) { + Error *local_err =3D NULL; + + kvmppc_xive_cpu_synchronize_state(tctx, &local_err); + if (local_err) { + error_report_err(local_err); + return; + } + } + monitor_printf(mon, "CPU[%04x]: QW NSR CPPR IPB LSMFB ACK# INC AGE= PIPR" " W2\n", cpu_index); =20 --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738008; cv=none; d=zoho.com; s=zohoarc; b=VvgF9LwsYzINI7XdAxKk5Q0tVcQbrmVZaX3KT4Vvq6MiCwMCwbgu0f9zqaCQ0cQxwW00IutmqYJQD1DA+PeA8fDlHUG7bkb1gQkYfD+/I53m8vdN4SDaX9rij/nWr8XiIfKyjjDBQooRO5krukgEWhqzE7qyHJdnRDPvqhHJ7GY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738008; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=BWFlsNoeyWE95yWh+xYSBoR6qUhvzGyBW8zRdQUy7j4=; b=Bpeq6rjzWe1iQ/5RDBivfMMWnpZtyjP9iozCjq+7LHWi2gvBlV6Ttk0ZtjkYCtIYmm/LGvK0egC1N8TOlar++KaWNAKLhTsRHMq2F73FbozkEfoEVsj1KwTua9Nx6x/kIVX3MqbQZ5dUayHyrh52yvZc7LxVb9ZekeXHd2WEegU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557738008864635.6910967215719; Mon, 13 May 2019 02:00:08 -0700 (PDT) Received: from localhost ([127.0.0.1]:53469 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6oU-0006sz-HF for importer@patchew.org; Mon, 13 May 2019 05:00:02 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33325) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6nA-0006PK-6H for qemu-devel@nongnu.org; Mon, 13 May 2019 04:58:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6YW-00044d-Nd for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:34 -0400 Received: from 10.mo69.mail-out.ovh.net ([46.105.73.241]:44265) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6YW-000448-Fq for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:32 -0400 Received: from player778.ha.ovh.net (unknown [10.109.143.72]) by mo69.mail-out.ovh.net (Postfix) with ESMTP id 1F3F055227 for ; Mon, 13 May 2019 10:43:31 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 668C859C3DDA; Mon, 13 May 2019 08:43:23 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:36 +0200 Message-Id: <20190513084245.25755-6-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3329004550225431526 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.73.241 Subject: [Qemu-devel] [PATCH v4 05/14] spapr/xive: introduce a VM state change handler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" This handler is in charge of stabilizing the flow of event notifications in the XIVE controller before migrating a guest. This is a requirement before transferring the guest EQ pages to a destination. When the VM is stopped, the handler sets the source PQs to PENDING to stop the flow of events and to possibly catch a triggered interrupt occuring while the VM is stopped. Their previous state is saved. The XIVE controller is then synced through KVM to flush any in-flight event notification and to stabilize the EQs. At this stage, the EQ pages are marked dirty to make sure the EQ pages are transferred if a migration sequence is in progress. The previous configuration of the sources is restored when the VM resumes, after a migration or a stop. If an interrupt was queued while the VM was stopped, the handler simply generates the missing trigger. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- include/hw/ppc/spapr_xive.h | 1 + hw/intc/spapr_xive_kvm.c | 96 ++++++++++++++++++++++++++++++++++++- 2 files changed, 96 insertions(+), 1 deletion(-) diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index 7e49badd8c9a..734662c12a10 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -42,6 +42,7 @@ typedef struct SpaprXive { /* KVM support */ int fd; void *tm_mmap; + VMChangeStateEntry *change; } SpaprXive; =20 /* diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c index 8dd4f96e0b97..735577a6f816 100644 --- a/hw/intc/spapr_xive_kvm.c +++ b/hw/intc/spapr_xive_kvm.c @@ -433,9 +433,100 @@ static void kvmppc_xive_get_queues(SpaprXive *xive, E= rror **errp) } } =20 +/* + * The primary goal of the XIVE VM change handler is to mark the EQ + * pages dirty when all XIVE event notifications have stopped. + * + * Whenever the VM is stopped, the VM change handler sets the source + * PQs to PENDING to stop the flow of events and to possibly catch a + * triggered interrupt occuring while the VM is stopped. The previous + * state is saved in anticipation of a migration. The XIVE controller + * is then synced through KVM to flush any in-flight event + * notification and stabilize the EQs. + * + * At this stage, we can mark the EQ page dirty and let a migration + * sequence transfer the EQ pages to the destination, which is done + * just after the stop state. + * + * The previous configuration of the sources is restored when the VM + * runs again. If an interrupt was queued while the VM was stopped, + * simply generate a trigger. + */ +static void kvmppc_xive_change_state_handler(void *opaque, int running, + RunState state) +{ + SpaprXive *xive =3D opaque; + XiveSource *xsrc =3D &xive->source; + Error *local_err =3D NULL; + int i; + + /* + * Restore the sources to their initial state. This is called when + * the VM resumes after a stop or a migration. + */ + if (running) { + for (i =3D 0; i < xsrc->nr_irqs; i++) { + uint8_t pq =3D xive_source_esb_get(xsrc, i); + uint8_t old_pq; + + old_pq =3D xive_esb_read(xsrc, i, XIVE_ESB_SET_PQ_00 + (pq << = 8)); + + /* + * An interrupt was queued while the VM was stopped, + * generate a trigger. + */ + if (pq =3D=3D XIVE_ESB_RESET && old_pq =3D=3D XIVE_ESB_QUEUED)= { + xive_esb_trigger(xsrc, i); + } + } + + return; + } + + /* + * Mask the sources, to stop the flow of event notifications, and + * save the PQs locally in the XiveSource object. The XiveSource + * state will be collected later on by its vmstate handler if a + * migration is in progress. + */ + for (i =3D 0; i < xsrc->nr_irqs; i++) { + uint8_t pq =3D xive_esb_read(xsrc, i, XIVE_ESB_GET); + + /* + * PQ is set to PENDING to possibly catch a triggered + * interrupt occuring while the VM is stopped (hotplug event + * for instance) . + */ + if (pq !=3D XIVE_ESB_OFF) { + pq =3D xive_esb_read(xsrc, i, XIVE_ESB_SET_PQ_10); + } + xive_source_esb_set(xsrc, i, pq); + } + + /* + * Sync the XIVE controller in KVM, to flush in-flight event + * notification that should be enqueued in the EQs and mark the + * XIVE EQ pages dirty to collect all updates. + */ + kvm_device_access(xive->fd, KVM_DEV_XIVE_GRP_CTRL, + KVM_DEV_XIVE_EQ_SYNC, NULL, true, &local_err); + if (local_err) { + error_report_err(local_err); + return; + } +} + void kvmppc_xive_synchronize_state(SpaprXive *xive, Error **errp) { - kvmppc_xive_source_get_state(&xive->source); + /* + * When the VM is stopped, the sources are masked and the previous + * state is saved in anticipation of a migration. We should not + * synchronize the source state in that case else we will override + * the saved state. + */ + if (runstate_is_running()) { + kvmppc_xive_source_get_state(&xive->source); + } =20 /* EAT: there is no extra state to query from KVM */ =20 @@ -515,6 +606,9 @@ void kvmppc_xive_connect(SpaprXive *xive, Error **errp) "xive.tima", tima_len, xive->tm_mmap= ); sysbus_init_mmio(SYS_BUS_DEVICE(xive), &xive->tm_mmio); =20 + xive->change =3D qemu_add_vm_change_state_handler( + kvmppc_xive_change_state_handler, xive); + kvm_kernel_irqchip =3D true; kvm_msi_via_irqfd_allowed =3D true; kvm_gsi_direct_mapping =3D true; --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738722; cv=none; d=zoho.com; s=zohoarc; b=QCpNUxDORwnWLBr/vlnu2ebw12OYEAWowPc0KHCH9VNTnkFU0qD0+FyX/5/G7fPLdYo3SIsCEYVYrgNXK1pKNXfLeD/ZzIanT97C6SLR+MPcYEUse9OrZjsyL/fGWQoaRqn3jIZ6WjDVPCVjIHns5Mmq3/wbWAvx7rpUpxoyslU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738722; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=jU5G3xoUtfhELkLx4T8h3b/Q1rPCl4k4cobIecjpOic=; b=Oif1ppdcABEr6Kcojt7CrQVmhrPMp627gwYTB5SFTX8xPze8kXyZcJHnC1dBMpIfq7l1XMtoY4lyF/BJSMfraIFu2d0u6GUsggZT7vWmaeLXUfnUOu3ftY03AYIyO5aE4dwuG7LeE/cluFcE28ItVVyYqJSxNO94YUuh1uqPbic= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557738722644696.695801549801; Mon, 13 May 2019 02:12:02 -0700 (PDT) Received: from localhost ([127.0.0.1]:53719 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6zy-0006zY-0w for importer@patchew.org; Mon, 13 May 2019 05:11:54 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33492) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6oF-0006dg-5j for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6Yd-00046P-L6 for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:41 -0400 Received: from 6.mo2.mail-out.ovh.net ([87.98.165.38]:51950) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6Yd-00045x-DF for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:39 -0400 Received: from player778.ha.ovh.net (unknown [10.108.42.170]) by mo2.mail-out.ovh.net (Postfix) with ESMTP id DD87A193FFE for ; Mon, 13 May 2019 10:43:36 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 19B3759C3E98; Mon, 13 May 2019 08:43:31 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:37 +0200 Message-Id: <20190513084245.25755-7-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3330411926312029158 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 87.98.165.38 Subject: [Qemu-devel] [PATCH v4 06/14] spapr/xive: add migration support for KVM X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" When the VM is stopped, the VM state handler stabilizes the XIVE IC and marks the EQ pages dirty. These are then transferred to destination before the transfer of the device vmstates starts. The SpaprXive interrupt controller model captures the XIVE internal tables, EAT and ENDT and the XiveTCTX model does the same for the thread interrupt context registers. At restart, the SpaprXive 'post_load' method restores all the XIVE states. It is called by the sPAPR machine 'post_load' method, when all XIVE states have been transferred and loaded. Finally, the source states are restored in the VM change state handler when the machine reaches the running state. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- include/hw/ppc/spapr_xive.h | 3 ++ include/hw/ppc/xive.h | 1 + hw/intc/spapr_xive.c | 24 ++++++++++ hw/intc/spapr_xive_kvm.c | 95 ++++++++++++++++++++++++++++++++++++- hw/intc/xive.c | 17 +++++++ hw/ppc/spapr_irq.c | 2 +- 6 files changed, 140 insertions(+), 2 deletions(-) diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index 734662c12a10..04294b0ca266 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -55,6 +55,7 @@ typedef struct SpaprXive { bool spapr_xive_irq_claim(SpaprXive *xive, uint32_t lisn, bool lsi); bool spapr_xive_irq_free(SpaprXive *xive, uint32_t lisn); void spapr_xive_pic_print_info(SpaprXive *xive, Monitor *mon); +int spapr_xive_post_load(SpaprXive *xive, int version_id); =20 void spapr_xive_hcall_init(SpaprMachineState *spapr); void spapr_dt_xive(SpaprMachineState *spapr, uint32_t nr_servers, void *fd= t, @@ -83,5 +84,7 @@ void kvmppc_xive_get_queue_config(SpaprXive *xive, uint8_= t end_blk, uint32_t end_idx, XiveEND *end, Error **errp); void kvmppc_xive_synchronize_state(SpaprXive *xive, Error **errp); +int kvmppc_xive_pre_save(SpaprXive *xive); +int kvmppc_xive_post_load(SpaprXive *xive, int version_id); =20 #endif /* PPC_SPAPR_XIVE_H */ diff --git a/include/hw/ppc/xive.h b/include/hw/ppc/xive.h index 78c919c4a590..edb8937f17fb 100644 --- a/include/hw/ppc/xive.h +++ b/include/hw/ppc/xive.h @@ -436,5 +436,6 @@ void kvmppc_xive_source_reset(XiveSource *xsrc, Error *= *errp); void kvmppc_xive_source_set_irq(void *opaque, int srcno, int val); void kvmppc_xive_cpu_connect(XiveTCTX *tctx, Error **errp); void kvmppc_xive_cpu_synchronize_state(XiveTCTX *tctx, Error **errp); +void kvmppc_xive_cpu_get_state(XiveTCTX *tctx, Error **errp); =20 #endif /* PPC_XIVE_H */ diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index 84cd30e1faff..d02993f489df 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -472,10 +472,34 @@ static const VMStateDescription vmstate_spapr_xive_ea= s =3D { }, }; =20 +static int vmstate_spapr_xive_pre_save(void *opaque) +{ + if (kvm_irqchip_in_kernel()) { + return kvmppc_xive_pre_save(SPAPR_XIVE(opaque)); + } + + return 0; +} + +/* + * Called by the sPAPR IRQ backend 'post_load' method at the machine + * level. + */ +int spapr_xive_post_load(SpaprXive *xive, int version_id) +{ + if (kvm_irqchip_in_kernel()) { + return kvmppc_xive_post_load(xive, version_id); + } + + return 0; +} + static const VMStateDescription vmstate_spapr_xive =3D { .name =3D TYPE_SPAPR_XIVE, .version_id =3D 1, .minimum_version_id =3D 1, + .pre_save =3D vmstate_spapr_xive_pre_save, + .post_load =3D NULL, /* handled at the machine level */ .fields =3D (VMStateField[]) { VMSTATE_UINT32_EQUAL(nr_irqs, SpaprXive, NULL), VMSTATE_STRUCT_VARRAY_POINTER_UINT32(eat, SpaprXive, nr_irqs, diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c index 735577a6f816..3999e4b7edfb 100644 --- a/hw/intc/spapr_xive_kvm.c +++ b/hw/intc/spapr_xive_kvm.c @@ -15,6 +15,7 @@ #include "sysemu/cpus.h" #include "sysemu/kvm.h" #include "hw/ppc/spapr.h" +#include "hw/ppc/spapr_cpu_core.h" #include "hw/ppc/spapr_xive.h" #include "hw/ppc/xive.h" #include "kvm_ppc.h" @@ -60,7 +61,24 @@ static void kvm_cpu_enable(CPUState *cs) /* * XIVE Thread Interrupt Management context (KVM) */ -static void kvmppc_xive_cpu_get_state(XiveTCTX *tctx, Error **errp) + +static void kvmppc_xive_cpu_set_state(XiveTCTX *tctx, Error **errp) +{ + uint64_t state[2]; + int ret; + + /* word0 and word1 of the OS ring. */ + state[0] =3D *((uint64_t *) &tctx->regs[TM_QW1_OS]); + + ret =3D kvm_set_one_reg(tctx->cs, KVM_REG_PPC_VP_STATE, state); + if (ret !=3D 0) { + error_setg_errno(errp, errno, + "XIVE: could not restore KVM state of CPU %ld", + kvm_arch_vcpu_id(tctx->cs)); + } +} + +void kvmppc_xive_cpu_get_state(XiveTCTX *tctx, Error **errp) { uint64_t state[2] =3D { 0 }; int ret; @@ -534,6 +552,81 @@ void kvmppc_xive_synchronize_state(SpaprXive *xive, Er= ror **errp) kvmppc_xive_get_queues(xive, errp); } =20 +/* + * The SpaprXive 'pre_save' method is called by the vmstate handler of + * the SpaprXive model, after the XIVE controller is synced in the VM + * change handler. + */ +int kvmppc_xive_pre_save(SpaprXive *xive) +{ + Error *local_err =3D NULL; + + /* EAT: there is no extra state to query from KVM */ + + /* ENDT */ + kvmppc_xive_get_queues(xive, &local_err); + if (local_err) { + error_report_err(local_err); + return -1; + } + + return 0; +} + +/* + * The SpaprXive 'post_load' method is not called by a vmstate + * handler. It is called at the sPAPR machine level at the end of the + * migration sequence by the sPAPR IRQ backend 'post_load' method, + * when all XIVE states have been transferred and loaded. + */ +int kvmppc_xive_post_load(SpaprXive *xive, int version_id) +{ + Error *local_err =3D NULL; + CPUState *cs; + int i; + + /* Restore the ENDT first. The targetting depends on it. */ + for (i =3D 0; i < xive->nr_ends; i++) { + if (!xive_end_is_valid(&xive->endt[i])) { + continue; + } + + kvmppc_xive_set_queue_config(xive, SPAPR_XIVE_BLOCK_ID, i, + &xive->endt[i], &local_err); + if (local_err) { + error_report_err(local_err); + return -1; + } + } + + /* Restore the EAT */ + for (i =3D 0; i < xive->nr_irqs; i++) { + if (!xive_eas_is_valid(&xive->eat[i])) { + continue; + } + + kvmppc_xive_set_source_config(xive, i, &xive->eat[i], &local_err); + if (local_err) { + error_report_err(local_err); + return -1; + } + } + + /* Restore the thread interrupt contexts */ + CPU_FOREACH(cs) { + PowerPCCPU *cpu =3D POWERPC_CPU(cs); + + kvmppc_xive_cpu_set_state(spapr_cpu_state(cpu)->tctx, &local_err); + if (local_err) { + error_report_err(local_err); + return -1; + } + } + + /* The source states will be restored when the machine starts running = */ + return 0; +} + static void *kvmppc_xive_mmap(SpaprXive *xive, int pgoff, size_t len, Error **errp) { diff --git a/hw/intc/xive.c b/hw/intc/xive.c index 7f1c54a7b581..b5ebb33527e7 100644 --- a/hw/intc/xive.c +++ b/hw/intc/xive.c @@ -582,10 +582,27 @@ static void xive_tctx_unrealize(DeviceState *dev, Err= or **errp) qemu_unregister_reset(xive_tctx_reset, dev); } =20 +static int vmstate_xive_tctx_pre_save(void *opaque) +{ + Error *local_err =3D NULL; + + if (kvm_irqchip_in_kernel()) { + kvmppc_xive_cpu_get_state(XIVE_TCTX(opaque), &local_err); + if (local_err) { + error_report_err(local_err); + return -1; + } + } + + return 0; +} + static const VMStateDescription vmstate_xive_tctx =3D { .name =3D TYPE_XIVE_TCTX, .version_id =3D 1, .minimum_version_id =3D 1, + .pre_save =3D vmstate_xive_tctx_pre_save, + .post_load =3D NULL, /* handled by the sPAPRxive model */ .fields =3D (VMStateField[]) { VMSTATE_BUFFER(regs, XiveTCTX), VMSTATE_END_OF_LIST() diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 5c4a44855d0a..8d371523e66e 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -350,7 +350,7 @@ static void spapr_irq_cpu_intc_create_xive(SpaprMachine= State *spapr, =20 static int spapr_irq_post_load_xive(SpaprMachineState *spapr, int version_= id) { - return 0; + return spapr_xive_post_load(spapr->xive, version_id); } =20 static void spapr_irq_reset_xive(SpaprMachineState *spapr, Error **errp) --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738523; cv=none; d=zoho.com; s=zohoarc; b=KsjpchOYBlK+4M/uVwaqPfGgzUTRcuxCp4Rk7e3fQqe3buwrJLycmbuzC+2KeJz3n3ykAlcBqLSyE5m1N5Zks2/3nMoVtfKLoXsWUNcfeB8lp2Wfph1WolU12CBHzGV3xhbqtOvM/JlfW0W9TEge0ku1zmG8c84AZODwQVI//vU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738523; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=oUeXru7mMBeHSYx7zgQdj4y7fAWCyFnLVes7j8cKLQw=; b=nrbpJbJSGqu9uuJoUniYzVYxgXTxWmjhyx2y+EtmqNmfkR5UdMGiUlLdHsq9UdVj4ddSg5RLEmhXXrygS9JaxXaZqKQceBtQnU0lN1wfMaydDT9AwyrmqInpRvtH4za8JBY4d4XNRZiLrQmWV4igf2UmJaOsssxXKl0UPvWlNHA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557738523793722.2099384579801; Mon, 13 May 2019 02:08:43 -0700 (PDT) Received: from localhost ([127.0.0.1]:53663 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6wg-0004ZD-Uw for importer@patchew.org; Mon, 13 May 2019 05:08:30 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33601) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6oF-0006k4-4Z for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6Yj-000481-T3 for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:50 -0400 Received: from 2.mo3.mail-out.ovh.net ([46.105.75.36]:35255) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6Yj-00047Q-KN for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:45 -0400 Received: from player778.ha.ovh.net (unknown [10.109.146.50]) by mo3.mail-out.ovh.net (Postfix) with ESMTP id C06792102F0 for ; Mon, 13 May 2019 10:43:43 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id BE06759C3EDE; Mon, 13 May 2019 08:43:36 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:38 +0200 Message-Id: <20190513084245.25755-8-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3332382250888891366 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.75.36 Subject: [Qemu-devel] [PATCH v4 07/14] spapr/xive: activate KVM support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" All is in place for KVM now. State synchronization and migration will come next. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- hw/ppc/spapr_irq.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 8d371523e66e..e969683f5c49 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -248,19 +248,10 @@ SpaprIrq spapr_irq_xics =3D { static void spapr_irq_init_xive(SpaprMachineState *spapr, int nr_irqs, Error **errp) { - MachineState *machine =3D MACHINE(spapr); uint32_t nr_servers =3D spapr_max_server_number(spapr); DeviceState *dev; int i; =20 - /* KVM XIVE device not yet available */ - if (kvm_enabled()) { - if (machine_kernel_irqchip_required(machine)) { - error_setg(errp, "kernel_irqchip requested. no KVM XIVE suppor= t"); - return; - } - } - dev =3D qdev_create(NULL, TYPE_SPAPR_XIVE); qdev_prop_set_uint32(dev, "nr-irqs", nr_irqs); /* --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738644; cv=none; d=zoho.com; s=zohoarc; b=hsZn0jgpPcBlU3f2jJOOpAxDf0FZdHa8SR/FbQ+TlpXDQdMmCtPBaSq3lOHYT07/uIbtiGpdVdcyqb+/u5y4e38ZjOZkxDG1BQM3wq6I/PPrzyplSvFcZmMWE/yded+DA0NY0N7hZ6HV6VjeATsxG+WcRnlPjNwPsRdmmanYEI0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738644; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=5VkihNQ7guUcMVr7xAOUQnLduJ2oP9J4GbNUAGwR0yc=; b=NmHt2YhmQNpdxdQGSwT5qhY7VWCld9B55SdBjjKYrb3cJzFo5/nDn7nzSYFk0hBCTnp93zIjCx68rEj5U5qUYwA4DcsEquMxNWNRQpV4YUoWDmq3xrdBsxIrX7uRWhWr3sg77mCHCqDKYVczSU/13eBEhwBLg3fIj/1s8bL8zgk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557738644266793.5085525839253; Mon, 13 May 2019 02:10:44 -0700 (PDT) Received: from localhost ([127.0.0.1]:53707 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6yl-0005zJ-9R for importer@patchew.org; Mon, 13 May 2019 05:10:39 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33611) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6oF-0006kQ-3N for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6Yp-00049T-Dz for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:52 -0400 Received: from 7.mo5.mail-out.ovh.net ([178.32.124.100]:43789) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6Yp-00048w-8B for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:51 -0400 Received: from player778.ha.ovh.net (unknown [10.109.143.208]) by mo5.mail-out.ovh.net (Postfix) with ESMTP id 56118233739 for ; Mon, 13 May 2019 10:43:49 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id B61F459C3F50; Mon, 13 May 2019 08:43:43 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:39 +0200 Message-Id: <20190513084245.25755-9-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3334071100790508518 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 178.32.124.100 Subject: [Qemu-devel] [PATCH v4 08/14] sysbus: add a sysbus_mmio_unmap() helper X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" This will be used to remove the MMIO regions of the POWER9 XIVE interrupt controller when the sPAPR machine is reseted. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- include/hw/sysbus.h | 1 + hw/core/sysbus.c | 10 ++++++++++ 2 files changed, 11 insertions(+) diff --git a/include/hw/sysbus.h b/include/hw/sysbus.h index 1aedcf05c92b..4c668fbbdc60 100644 --- a/include/hw/sysbus.h +++ b/include/hw/sysbus.h @@ -89,6 +89,7 @@ qemu_irq sysbus_get_connected_irq(SysBusDevice *dev, int = n); void sysbus_mmio_map(SysBusDevice *dev, int n, hwaddr addr); void sysbus_mmio_map_overlap(SysBusDevice *dev, int n, hwaddr addr, int priority); +void sysbus_mmio_unmap(SysBusDevice *dev, int n); void sysbus_add_io(SysBusDevice *dev, hwaddr addr, MemoryRegion *mem); MemoryRegion *sysbus_address_space(SysBusDevice *dev); diff --git a/hw/core/sysbus.c b/hw/core/sysbus.c index 307cf90a5196..689a867a2274 100644 --- a/hw/core/sysbus.c +++ b/hw/core/sysbus.c @@ -153,6 +153,16 @@ static void sysbus_mmio_map_common(SysBusDevice *dev, = int n, hwaddr addr, } } =20 +void sysbus_mmio_unmap(SysBusDevice *dev, int n) +{ + assert(n >=3D 0 && n < dev->num_mmio); + + if (dev->mmio[n].addr !=3D (hwaddr)-1) { + memory_region_del_subregion(get_system_memory(), dev->mmio[n].memo= ry); + dev->mmio[n].addr =3D (hwaddr)-1; + } +} + void sysbus_mmio_map(SysBusDevice *dev, int n, hwaddr addr) { sysbus_mmio_map_common(dev, n, addr, false, 0); --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738399; cv=none; d=zoho.com; s=zohoarc; b=egGzakAyjU6MWlBuB9D3QpPoUBJi+6uEDan4hcUEhxM3MUoN6ydRdCnEQBMR6e1mFTY0wFGFUlwegfdYWhuDpIeX9o4A6jxDAmZqyNuoeqP1pPQ/xTfgqSk6lu6sH94y1JkxxmAzBbm6o89rBtcWW5pOlj7E9XUjk1VyDwSibXw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738399; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=24ET25+tPJCQvy82mOb2ALvJZ9F7AzLtkOcq31jJSA4=; b=F+cjW/l+b7LsWoN/9yQRqSkrC0iKELtFGYL721id9fS9eEEfShYrWIiFIBxszRWSxcq0iq22KQB+Nt5ZQV8bu8Zm2ck33KAxgJZA4aonX2J7tx93c0huJPmUQhfLAtXVVAb9i07dsGmveSBdEyEvwLXRixxJw5wF19pTwjkTICk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155773839975055.95438904458456; Mon, 13 May 2019 02:06:39 -0700 (PDT) Received: from localhost ([127.0.0.1]:53644 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6um-0003Gw-MW for importer@patchew.org; Mon, 13 May 2019 05:06:32 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33622) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6oC-0006kh-Oc for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6Yw-0004D8-Kw for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:00 -0400 Received: from 6.mo68.mail-out.ovh.net ([46.105.63.100]:35599) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6Yw-0004CS-FQ for qemu-devel@nongnu.org; Mon, 13 May 2019 04:43:58 -0400 Received: from player778.ha.ovh.net (unknown [10.108.57.18]) by mo68.mail-out.ovh.net (Postfix) with ESMTP id B558F129AEB for ; Mon, 13 May 2019 10:43:56 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 50F5E59C3FDB; Mon, 13 May 2019 08:43:49 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:40 +0200 Message-Id: <20190513084245.25755-10-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3336041424952462310 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.63.100 Subject: [Qemu-devel] [PATCH v4 09/14] spapr: introduce routines to delete the KVM IRQ device X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" If a new interrupt mode is chosen by CAS, the machine generates a reset to reconfigure. At this point, the connection with the previous KVM device needs to be closed and a new connection needs to opened with the KVM device operating the chosen interrupt mode. New routines are introduced to destroy the XICS and the XIVE KVM devices. They make use of a new KVM device ioctl which destroys the device and also disconnects the IRQ presenters from the vCPUs. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- Changes since v3: - removed KVM_DESTROY_DEVICE control. KVM device is destroyed when the fd is closed. include/hw/ppc/spapr_xive.h | 1 + include/hw/ppc/xics_spapr.h | 1 + hw/intc/spapr_xive_kvm.c | 56 +++++++++++++++++++++++++++++++++++++ hw/intc/xics_kvm.c | 51 +++++++++++++++++++++++++++++++++ 4 files changed, 109 insertions(+) diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index 04294b0ca266..0b5e972d52c8 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -71,6 +71,7 @@ int spapr_xive_end_to_target(uint8_t end_blk, uint32_t en= d_idx, * KVM XIVE device helpers */ void kvmppc_xive_connect(SpaprXive *xive, Error **errp); +void kvmppc_xive_disconnect(SpaprXive *xive, Error **errp); void kvmppc_xive_reset(SpaprXive *xive, Error **errp); void kvmppc_xive_set_source_config(SpaprXive *xive, uint32_t lisn, XiveEAS= *eas, Error **errp); diff --git a/include/hw/ppc/xics_spapr.h b/include/hw/ppc/xics_spapr.h index 15a8dcff66fc..2476b540edfa 100644 --- a/include/hw/ppc/xics_spapr.h +++ b/include/hw/ppc/xics_spapr.h @@ -34,6 +34,7 @@ void spapr_dt_xics(SpaprMachineState *spapr, uint32_t nr_servers, void *fd= t, uint32_t phandle); int xics_kvm_init(SpaprMachineState *spapr, Error **errp); +void xics_kvm_disconnect(SpaprMachineState *spapr, Error **errp); void xics_spapr_init(SpaprMachineState *spapr); =20 #endif /* XICS_SPAPR_H */ diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c index 3999e4b7edfb..259cd1db9582 100644 --- a/hw/intc/spapr_xive_kvm.c +++ b/hw/intc/spapr_xive_kvm.c @@ -58,6 +58,16 @@ static void kvm_cpu_enable(CPUState *cs) QLIST_INSERT_HEAD(&kvm_enabled_cpus, enabled_cpu, node); } =20 +static void kvm_cpu_disable_all(void) +{ + KVMEnabledCPU *enabled_cpu, *next; + + QLIST_FOREACH_SAFE(enabled_cpu, &kvm_enabled_cpus, node, next) { + QLIST_REMOVE(enabled_cpu, node); + g_free(enabled_cpu); + } +} + /* * XIVE Thread Interrupt Management context (KVM) */ @@ -709,3 +719,49 @@ void kvmppc_xive_connect(SpaprXive *xive, Error **errp) /* Map all regions */ spapr_xive_map_mmio(xive); } + +void kvmppc_xive_disconnect(SpaprXive *xive, Error **errp) +{ + XiveSource *xsrc; + size_t esb_len; + + /* The KVM XIVE device is not in use */ + if (!xive || xive->fd =3D=3D -1) { + return; + } + + if (!kvmppc_has_cap_xive()) { + error_setg(errp, "IRQ_XIVE capability must be present for KVM"); + return; + } + + /* Clear the KVM mapping */ + xsrc =3D &xive->source; + esb_len =3D (1ull << xsrc->esb_shift) * xsrc->nr_irqs; + + sysbus_mmio_unmap(SYS_BUS_DEVICE(xive), 0); + munmap(xsrc->esb_mmap, esb_len); + + sysbus_mmio_unmap(SYS_BUS_DEVICE(xive), 1); + + sysbus_mmio_unmap(SYS_BUS_DEVICE(xive), 2); + munmap(xive->tm_mmap, 4ull << TM_SHIFT); + + /* + * When the KVM device fd is closed, the KVM device is destroyed + * and removed from the list of devices of the VM. The VCPU + * presenters are also detached from the device. + */ + close(xive->fd); + xive->fd =3D -1; + + kvm_kernel_irqchip =3D false; + kvm_msi_via_irqfd_allowed =3D false; + kvm_gsi_direct_mapping =3D false; + + /* Clear the local list of presenter (hotplug) */ + kvm_cpu_disable_all(); + + /* VM Change state handler is not needed anymore */ + qemu_del_vm_change_state_handler(xive->change); +} diff --git a/hw/intc/xics_kvm.c b/hw/intc/xics_kvm.c index 78a252e6dfd4..1185846ff183 100644 --- a/hw/intc/xics_kvm.c +++ b/hw/intc/xics_kvm.c @@ -51,6 +51,16 @@ typedef struct KVMEnabledICP { static QLIST_HEAD(, KVMEnabledICP) kvm_enabled_icps =3D QLIST_HEAD_INITIALIZER(&kvm_enabled_icps); =20 +static void kvm_disable_icps(void) +{ + KVMEnabledICP *enabled_icp, *next; + + QLIST_FOREACH_SAFE(enabled_icp, &kvm_enabled_icps, node, next) { + QLIST_REMOVE(enabled_icp, node); + g_free(enabled_icp); + } +} + /* * ICP-KVM */ @@ -360,3 +370,44 @@ fail: kvmppc_define_rtas_kernel_token(0, "ibm,int-off"); return -1; } + +void xics_kvm_disconnect(SpaprMachineState *spapr, Error **errp) +{ + /* The KVM XICS device is not in use */ + if (kernel_xics_fd =3D=3D -1) { + return; + } + + if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_IRQ_XICS= )) { + error_setg(errp, + "KVM and IRQ_XICS capability must be present for KVM XI= CS device"); + return; + } + + /* + * Only on P9 using the XICS-on XIVE KVM device: + * + * When the KVM device fd is closed, the device is destroyed and + * removed from the list of devices of the VM. The VCPU presenters + * are also detached from the device. + */ + close(kernel_xics_fd); + kernel_xics_fd =3D -1; + + spapr_rtas_unregister(RTAS_IBM_SET_XIVE); + spapr_rtas_unregister(RTAS_IBM_GET_XIVE); + spapr_rtas_unregister(RTAS_IBM_INT_OFF); + spapr_rtas_unregister(RTAS_IBM_INT_ON); + + kvmppc_define_rtas_kernel_token(0, "ibm,set-xive"); + kvmppc_define_rtas_kernel_token(0, "ibm,get-xive"); + kvmppc_define_rtas_kernel_token(0, "ibm,int-on"); + kvmppc_define_rtas_kernel_token(0, "ibm,int-off"); + + kvm_kernel_irqchip =3D false; + kvm_msi_via_irqfd_allowed =3D false; + kvm_gsi_direct_mapping =3D false; + + /* Clear the presenter from the VCPUs */ + kvm_disable_icps(); +} --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738111; cv=none; d=zoho.com; s=zohoarc; b=Mdww2BjuvDXKnfeNLy5AKNO/tsGR/NCtnkhpkNWPC9XL/rzbKnm6RKQiiEFS6UXrDFvV2csGKl8wq+NTXQK8kffkqTiQ26Bpp3JT6PcA8xo/CZZJGszw7y1XZ+oLt7x3tD28tXB+yHXcur7OT1IQcxVSOYXIrze60rP9TRZPQ9A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738111; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ZtkWZshYXT7FutNIUOpyMGixreGcMZoUh8W5RM3PDxs=; b=Ly2aaIcJnSiQM12+N9+yetwUP2qdhHdW3d/uAUjcjebGn/um+xASy0YmrrrV/qTJbBy6cepKGfEurAM4s0zWWJnyrVEb3uK8TuP0iCTaZkm+G/5RWOz9i/foQybG6r/7dk8z9KMpQeYn28cKf16EJwm9hex7RpbFSNR9VJy9XDA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155773811184625.271340196218716; Mon, 13 May 2019 02:01:51 -0700 (PDT) Received: from localhost ([127.0.0.1]:53550 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6qC-0008On-O8 for importer@patchew.org; Mon, 13 May 2019 05:01:48 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33558) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6oA-0006iZ-GD for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6Z2-0004F9-Qg for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:06 -0400 Received: from 6.mo177.mail-out.ovh.net ([46.105.51.249]:47605) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6Z2-0004ER-Ks for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:04 -0400 Received: from player778.ha.ovh.net (unknown [10.109.159.248]) by mo177.mail-out.ovh.net (Postfix) with ESMTP id 1F9B7F6C48 for ; Mon, 13 May 2019 10:44:02 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 9D9FC59C40A6; Mon, 13 May 2019 08:43:56 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:41 +0200 Message-Id: <20190513084245.25755-11-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3337730274498087910 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.51.249 Subject: [Qemu-devel] [PATCH v4 10/14] spapr: check for the activation of the KVM IRQ device X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" The activation of the KVM IRQ device depends on the interrupt mode chosen at CAS time by the machine and some methods used at reset or by the migration need to be protected. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson Signed-off-by: C=C3=A9dric Le Goater --- May be we can improve these checkd by using spapr->ov5_cas in some cases. Changes since v3: - update in kvmppc_xive_source_reset_one() hw/intc/spapr_xive_kvm.c | 33 +++++++++++++++++++++++++++++++++ hw/intc/xics_kvm.c | 31 ++++++++++++++++++++++++++++++- 2 files changed, 63 insertions(+), 1 deletion(-) diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c index 259cd1db9582..078d18d7757f 100644 --- a/hw/intc/spapr_xive_kvm.c +++ b/hw/intc/spapr_xive_kvm.c @@ -90,9 +90,15 @@ static void kvmppc_xive_cpu_set_state(XiveTCTX *tctx, Er= ror **errp) =20 void kvmppc_xive_cpu_get_state(XiveTCTX *tctx, Error **errp) { + SpaprXive *xive =3D SPAPR_MACHINE(qdev_get_machine())->xive; uint64_t state[2] =3D { 0 }; int ret; =20 + /* The KVM XIVE device is not in use */ + if (xive->fd =3D=3D -1) { + return; + } + ret =3D kvm_get_one_reg(tctx->cs, KVM_REG_PPC_VP_STATE, state); if (ret !=3D 0) { error_setg_errno(errp, errno, @@ -143,6 +149,11 @@ void kvmppc_xive_cpu_connect(XiveTCTX *tctx, Error **e= rrp) unsigned long vcpu_id; int ret; =20 + /* The KVM XIVE device is not in use */ + if (xive->fd =3D=3D -1) { + return; + } + /* Check if CPU was hot unplugged and replugged. */ if (kvm_cpu_is_enabled(tctx->cs)) { return; @@ -219,6 +230,11 @@ void kvmppc_xive_source_reset_one(XiveSource *xsrc, in= t srcno, Error **errp) SpaprXive *xive =3D SPAPR_XIVE(xsrc->xive); uint64_t state =3D 0; =20 + /* The KVM XIVE device is not in use */ + if (xive->fd =3D=3D -1) { + return; + } + if (xive_source_irq_is_lsi(xsrc, srcno)) { state |=3D KVM_XIVE_LEVEL_SENSITIVE; if (xsrc->status[srcno] & XIVE_STATUS_ASSERTED) { @@ -319,9 +335,13 @@ static void kvmppc_xive_source_get_state(XiveSource *x= src) void kvmppc_xive_source_set_irq(void *opaque, int srcno, int val) { XiveSource *xsrc =3D opaque; + SpaprXive *xive =3D SPAPR_XIVE(xsrc->xive); struct kvm_irq_level args; int rc; =20 + /* The KVM XIVE device should be in use */ + assert(xive->fd !=3D -1); + args.irq =3D srcno; if (!xive_source_irq_is_lsi(xsrc, srcno)) { if (!val) { @@ -546,6 +566,11 @@ static void kvmppc_xive_change_state_handler(void *opa= que, int running, =20 void kvmppc_xive_synchronize_state(SpaprXive *xive, Error **errp) { + /* The KVM XIVE device is not in use */ + if (xive->fd =3D=3D -1) { + return; + } + /* * When the VM is stopped, the sources are masked and the previous * state is saved in anticipation of a migration. We should not @@ -571,6 +596,11 @@ int kvmppc_xive_pre_save(SpaprXive *xive) { Error *local_err =3D NULL; =20 + /* The KVM XIVE device is not in use */ + if (xive->fd =3D=3D -1) { + return 0; + } + /* EAT: there is no extra state to query from KVM */ =20 /* ENDT */ @@ -595,6 +625,9 @@ int kvmppc_xive_post_load(SpaprXive *xive, int version_= id) CPUState *cs; int i; =20 + /* The KVM XIVE device should be in use */ + assert(xive->fd !=3D -1); + /* Restore the ENDT first. The targetting depends on it. */ for (i =3D 0; i < xive->nr_ends; i++) { if (!xive_end_is_valid(&xive->endt[i])) { diff --git a/hw/intc/xics_kvm.c b/hw/intc/xics_kvm.c index 1185846ff183..12bd5190cfad 100644 --- a/hw/intc/xics_kvm.c +++ b/hw/intc/xics_kvm.c @@ -69,6 +69,11 @@ void icp_get_kvm_state(ICPState *icp) uint64_t state; int ret; =20 + /* The KVM XICS device is not in use */ + if (kernel_xics_fd =3D=3D -1) { + return; + } + /* ICP for this CPU thread is not in use, exiting */ if (!icp->cs) { return; @@ -105,6 +110,11 @@ int icp_set_kvm_state(ICPState *icp) uint64_t state; int ret; =20 + /* The KVM XICS device is not in use */ + if (kernel_xics_fd =3D=3D -1) { + return 0; + } + /* ICP for this CPU thread is not in use, exiting */ if (!icp->cs) { return 0; @@ -133,8 +143,9 @@ void icp_kvm_realize(DeviceState *dev, Error **errp) unsigned long vcpu_id; int ret; =20 + /* The KVM XICS device is not in use */ if (kernel_xics_fd =3D=3D -1) { - abort(); + return; } =20 cs =3D icp->cs; @@ -170,6 +181,11 @@ void ics_get_kvm_state(ICSState *ics) uint64_t state; int i; =20 + /* The KVM XICS device is not in use */ + if (kernel_xics_fd =3D=3D -1) { + return; + } + for (i =3D 0; i < ics->nr_irqs; i++) { ICSIRQState *irq =3D &ics->irqs[i]; =20 @@ -230,6 +246,11 @@ int ics_set_kvm_state_one(ICSState *ics, int srcno) ICSIRQState *irq =3D &ics->irqs[srcno]; int ret; =20 + /* The KVM XICS device is not in use */ + if (kernel_xics_fd =3D=3D -1) { + return 0; + } + state =3D irq->server; state |=3D (uint64_t)(irq->saved_priority & KVM_XICS_PRIORITY_MASK) << KVM_XICS_PRIORITY_SHIFT; @@ -269,6 +290,11 @@ int ics_set_kvm_state(ICSState *ics) { int i; =20 + /* The KVM XICS device is not in use */ + if (kernel_xics_fd =3D=3D -1) { + return 0; + } + for (i =3D 0; i < ics->nr_irqs; i++) { int ret; =20 @@ -286,6 +312,9 @@ void ics_kvm_set_irq(ICSState *ics, int srcno, int val) struct kvm_irq_level args; int rc; =20 + /* The KVM XICS device should be in use */ + assert(kernel_xics_fd !=3D -1); + args.irq =3D srcno + ics->offset; if (ics->irqs[srcno].flags & XICS_FLAGS_IRQ_MSI) { if (!val) { --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738591; cv=none; d=zoho.com; s=zohoarc; b=jYC/4Wbx5pkNenjFOJYPQwfdxFo5AanfvAyqHiBiNAFIbLCdxCWb+PLfG46Oi4WP/krzZSLoZ8UOxlTpCw1aMzmRy87HOLJ25Pq74fMNhE20o0JosPHHTQSnY++8EJlPEM0GzclN873cDGjOpscmL6MZB51A04Voa68Poxw339c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738591; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=hDS9NtBC6jSbnZ9N5c+JtG0jThwLqvCMjmeDyzRDWn8=; b=M2mLhsVNspwaJyOVmDBUfca+ob0U34trpL05pLhj33WqPlM/iWeGREXTg84eprf1Ej5Vo4YaCLdNkf0aMzN2pnz5Uk+Rl210vl0bmnzFcjI6JDKwjC337C82NL8mQtiI62oG677NojlUnfa0oASnoj+XjOACTlkU4Px8CuzSdMU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557738591495549.5068950479034; Mon, 13 May 2019 02:09:51 -0700 (PDT) Received: from localhost ([127.0.0.1]:53675 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6xv-0005Ow-9z for importer@patchew.org; Mon, 13 May 2019 05:09:47 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33611) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6o8-0006kQ-IT for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6Z8-0004Gj-Q2 for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:12 -0400 Received: from 5.mo179.mail-out.ovh.net ([46.105.43.140]:50851) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6Z8-0004G6-KG for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:10 -0400 Received: from player778.ha.ovh.net (unknown [10.108.42.170]) by mo179.mail-out.ovh.net (Postfix) with ESMTP id 2080012D0A0 for ; Mon, 13 May 2019 10:44:09 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 0DEAA59C4118; Mon, 13 May 2019 08:44:02 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:42 +0200 Message-Id: <20190513084245.25755-12-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3339419126787247078 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.43.140 Subject: [Qemu-devel] [PATCH v4 11/14] spapr/irq: introduce a spapr_irq_init_device() helper X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" The way the XICS and the XIVE devices are initialized follows the same pattern. First, try to connect to the KVM device and if not possible fallback on the emulated device, unless a kernel_irqchip is required. The spapr_irq_init_device() routine implements this sequence in generic way using new sPAPR IRQ handlers ->init_emu() and ->init_kvm(). The XIVE init sequence is moved under the associated sPAPR IRQ ->init() handler. This will change again when KVM support is added for the dual interrupt mode. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- include/hw/ppc/spapr_irq.h | 2 + include/hw/ppc/spapr_xive.h | 1 + hw/intc/spapr_xive.c | 26 +++-------- hw/ppc/spapr_irq.c | 89 +++++++++++++++++++++++++++++-------- 4 files changed, 78 insertions(+), 40 deletions(-) diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h index b855f74e4428..14cab73c9c07 100644 --- a/include/hw/ppc/spapr_irq.h +++ b/include/hw/ppc/spapr_irq.h @@ -48,6 +48,8 @@ typedef struct SpaprIrq { void (*reset)(SpaprMachineState *spapr, Error **errp); void (*set_irq)(void *opaque, int srcno, int val); const char *(*get_nodename)(SpaprMachineState *spapr); + void (*init_emu)(SpaprMachineState *spapr, Error **errp); + void (*init_kvm)(SpaprMachineState *spapr, Error **errp); } SpaprIrq; =20 extern SpaprIrq spapr_irq_xics; diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index 0b5e972d52c8..b26befcf6b56 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -66,6 +66,7 @@ void spapr_xive_map_mmio(SpaprXive *xive); =20 int spapr_xive_end_to_target(uint8_t end_blk, uint32_t end_idx, uint32_t *out_server, uint8_t *out_prio); +void spapr_xive_init(SpaprXive *xive, Error **errp); =20 /* * KVM XIVE device helpers diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index d02993f489df..246b21ae7f31 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -281,7 +281,6 @@ static void spapr_xive_realize(DeviceState *dev, Error = **errp) XiveSource *xsrc =3D &xive->source; XiveENDSource *end_xsrc =3D &xive->end_source; Error *local_err =3D NULL; - MachineState *machine =3D MACHINE(qdev_get_machine()); =20 if (!xive->nr_irqs) { error_setg(errp, "Number of interrupt needs to be greater 0"); @@ -332,27 +331,12 @@ static void spapr_xive_realize(DeviceState *dev, Erro= r **errp) xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SH= IFT)); =20 qemu_register_reset(spapr_xive_reset, dev); +} =20 - if (kvm_enabled() && machine_kernel_irqchip_allowed(machine)) { - kvmppc_xive_connect(xive, &local_err); - if (local_err && machine_kernel_irqchip_required(machine)) { - error_prepend(&local_err, - "kernel_irqchip requested but unavailable: "); - error_propagate(errp, local_err); - return; - } - - if (!local_err) { - return; - } - - /* - * We failed to initialize the XIVE KVM device, fallback to - * emulated mode - */ - error_prepend(&local_err, "kernel_irqchip allowed but unavailable:= "); - warn_report_err(local_err); - } +void spapr_xive_init(SpaprXive *xive, Error **errp) +{ + XiveSource *xsrc =3D &xive->source; + XiveENDSource *end_xsrc =3D &xive->end_source; =20 /* TIMA initialization */ memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &xive_tm_ops, xive, diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index e969683f5c49..d1e87577fb0e 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -62,36 +62,50 @@ void spapr_irq_msi_reset(SpaprMachineState *spapr) bitmap_clear(spapr->irq_map, 0, spapr->irq_map_nr); } =20 - -/* - * XICS IRQ backend. - */ - -static void spapr_irq_init_xics(SpaprMachineState *spapr, int nr_irqs, - Error **errp) +static void spapr_irq_init_device(SpaprMachineState *spapr, + SpaprIrq *irq, Error **errp) { MachineState *machine =3D MACHINE(spapr); - Object *obj; Error *local_err =3D NULL; - bool xics_kvm =3D false; =20 - if (kvm_enabled()) { - if (machine_kernel_irqchip_allowed(machine) && - !xics_kvm_init(spapr, &local_err)) { - xics_kvm =3D true; - } - if (machine_kernel_irqchip_required(machine) && !xics_kvm) { + if (kvm_enabled() && machine_kernel_irqchip_allowed(machine)) { + irq->init_kvm(spapr, &local_err); + if (local_err && machine_kernel_irqchip_required(machine)) { error_prepend(&local_err, "kernel_irqchip requested but unavailable: "); error_propagate(errp, local_err); return; } - error_free(local_err); - local_err =3D NULL; + + if (!local_err) { + return; + } + + /* + * We failed to initialize the KVM device, fallback to + * emulated mode + */ + error_prepend(&local_err, "kernel_irqchip allowed but unavailable:= "); + warn_report_err(local_err); } =20 - if (!xics_kvm) { - xics_spapr_init(spapr); + irq->init_emu(spapr, errp); +} + +/* + * XICS IRQ backend. + */ + +static void spapr_irq_init_xics(SpaprMachineState *spapr, int nr_irqs, + Error **errp) +{ + Object *obj; + Error *local_err =3D NULL; + + spapr_irq_init_device(spapr, &spapr_irq_xics, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; } =20 obj =3D object_new(TYPE_ICS_SIMPLE); @@ -220,6 +234,18 @@ static const char *spapr_irq_get_nodename_xics(SpaprMa= chineState *spapr) return XICS_NODENAME; } =20 +static void spapr_irq_init_emu_xics(SpaprMachineState *spapr, Error **errp) +{ + xics_spapr_init(spapr); +} + +static void spapr_irq_init_kvm_xics(SpaprMachineState *spapr, Error **errp) +{ + if (kvm_enabled()) { + xics_kvm_init(spapr, errp); + } +} + #define SPAPR_IRQ_XICS_NR_IRQS 0x1000 #define SPAPR_IRQ_XICS_NR_MSIS \ (XICS_IRQ_BASE + SPAPR_IRQ_XICS_NR_IRQS - SPAPR_IRQ_MSI) @@ -240,6 +266,8 @@ SpaprIrq spapr_irq_xics =3D { .reset =3D spapr_irq_reset_xics, .set_irq =3D spapr_irq_set_irq_xics, .get_nodename =3D spapr_irq_get_nodename_xics, + .init_emu =3D spapr_irq_init_emu_xics, + .init_kvm =3D spapr_irq_init_kvm_xics, }; =20 /* @@ -251,6 +279,7 @@ static void spapr_irq_init_xive(SpaprMachineState *spap= r, int nr_irqs, uint32_t nr_servers =3D spapr_max_server_number(spapr); DeviceState *dev; int i; + Error *local_err =3D NULL; =20 dev =3D qdev_create(NULL, TYPE_SPAPR_XIVE); qdev_prop_set_uint32(dev, "nr-irqs", nr_irqs); @@ -268,6 +297,12 @@ static void spapr_irq_init_xive(SpaprMachineState *spa= pr, int nr_irqs, } =20 spapr_xive_hcall_init(spapr); + + spapr_irq_init_device(spapr, &spapr_irq_xive, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } } =20 static int spapr_irq_claim_xive(SpaprMachineState *spapr, int irq, bool ls= i, @@ -375,6 +410,18 @@ static const char *spapr_irq_get_nodename_xive(SpaprMa= chineState *spapr) return spapr->xive->nodename; } =20 +static void spapr_irq_init_emu_xive(SpaprMachineState *spapr, Error **errp) +{ + spapr_xive_init(spapr->xive, errp); +} + +static void spapr_irq_init_kvm_xive(SpaprMachineState *spapr, Error **errp) +{ + if (kvm_enabled()) { + kvmppc_xive_connect(spapr->xive, errp); + } +} + /* * XIVE uses the full IRQ number space. Set it to 8K to be compatible * with XICS. @@ -399,6 +446,8 @@ SpaprIrq spapr_irq_xive =3D { .reset =3D spapr_irq_reset_xive, .set_irq =3D spapr_irq_set_irq_xive, .get_nodename =3D spapr_irq_get_nodename_xive, + .init_emu =3D spapr_irq_init_emu_xive, + .init_kvm =3D spapr_irq_init_kvm_xive, }; =20 /* @@ -560,6 +609,8 @@ SpaprIrq spapr_irq_dual =3D { .reset =3D spapr_irq_reset_dual, .set_irq =3D spapr_irq_set_irq_dual, .get_nodename =3D spapr_irq_get_nodename_dual, + .init_emu =3D NULL, /* should not be used */ + .init_kvm =3D NULL, /* should not be used */ }; =20 =20 --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738478; cv=none; d=zoho.com; s=zohoarc; b=LKE1vWyNTtoLujqZkwidPc0ilbIQau4fZmg7er7JYFBC81GiR+LfHD5W/5N7Y0G8IzZ4mebdcY7+3grcr58Un6vGKZNMpLhveIm8S/bfhqHFjOcPaWkVbSfm64LXLc12cd5Q+HBN/D19ZujYUuhjKkq7SdtrlYFZFzkn4UxgQKU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738478; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=+QOccqPfY2and8RRTv19u6bIhPmWEt5r6wtMXg2ryWc=; b=Kt8p+Y93uCL9uh8le80IjtaAbXSQLeB4OZKmgVtuuNY6p2GWT03D7OmKL2HtFwAnd8rokvoIdKqr0Bmd0Fwx3R0yunARPcSLTkUhwkqBGEu/GgXiYMIRJNH1WJ3dCmLRBH8RQAmzXQ4MDW0LrnMyGNnsSORZlutJ2DGC3KwQeh8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557738478632717.0786231209298; Mon, 13 May 2019 02:07:58 -0700 (PDT) Received: from localhost ([127.0.0.1]:53661 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6w5-00042N-FR for importer@patchew.org; Mon, 13 May 2019 05:07:53 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33816) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6o8-0006t9-CU for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6ZE-0004JK-R0 for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:17 -0400 Received: from 14.mo3.mail-out.ovh.net ([188.165.43.98]:55390) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6ZE-0004IK-LZ for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:16 -0400 Received: from player778.ha.ovh.net (unknown [10.109.160.46]) by mo3.mail-out.ovh.net (Postfix) with ESMTP id BCBB72102C9 for ; Mon, 13 May 2019 10:44:14 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 0625159C4190; Mon, 13 May 2019 08:44:09 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:43 +0200 Message-Id: <20190513084245.25755-13-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3341107977402747878 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 188.165.43.98 Subject: [Qemu-devel] [PATCH v4 12/14] spapr/irq: initialize the IRQ device only once X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Add a check to make sure that the routine initializing the emulated IRQ device is called once. We don't have much to test on the XICS side, so we introduce a 'init' boolean under ICSState. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- Changes since v3: - introduced a 'init' boolean under ICSState =20 include/hw/ppc/xics.h | 1 + hw/intc/spapr_xive.c | 9 +++++++++ hw/intc/xics_spapr.c | 7 +++++++ 3 files changed, 17 insertions(+) diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h index eb65ad7e43b7..d6f8e4c4c282 100644 --- a/include/hw/ppc/xics.h +++ b/include/hw/ppc/xics.h @@ -119,6 +119,7 @@ struct ICSState { uint32_t offset; ICSIRQState *irqs; XICSFabric *xics; + bool init; /* sPAPR ICS device initialized */ }; =20 #define ICS_PROP_XICS "xics" diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index 246b21ae7f31..1e649d6cdbe1 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -338,6 +338,15 @@ void spapr_xive_init(SpaprXive *xive, Error **errp) XiveSource *xsrc =3D &xive->source; XiveENDSource *end_xsrc =3D &xive->end_source; =20 + /* + * The emulated XIVE device can only be initialized once. If the + * ESB memory region has been already mapped, it means we have been + * through there. + */ + if (memory_region_is_mapped(&xsrc->esb_mmio)) { + return; + } + /* TIMA initialization */ memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &xive_tm_ops, xive, "xive.tima", 4ull << TM_SHIFT); diff --git a/hw/intc/xics_spapr.c b/hw/intc/xics_spapr.c index 9d2b8adef7c5..5a1835e8b1ed 100644 --- a/hw/intc/xics_spapr.c +++ b/hw/intc/xics_spapr.c @@ -239,6 +239,13 @@ static void rtas_int_on(PowerPCCPU *cpu, SpaprMachineS= tate *spapr, =20 void xics_spapr_init(SpaprMachineState *spapr) { + /* Emulated mode can only be initialized once. */ + if (spapr->ics->init) { + return; + } + + spapr->ics->init =3D true; + /* Registration of global state belongs into realize */ spapr_rtas_register(RTAS_IBM_SET_XIVE, "ibm,set-xive", rtas_set_xive); spapr_rtas_register(RTAS_IBM_GET_XIVE, "ibm,get-xive", rtas_get_xive); --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738349; cv=none; d=zoho.com; s=zohoarc; b=Jo8H0FylMBiVoNZbOaHS0Lec/TQf2rXoJ2eQ3X13GNwXAHILR0S7VtaPR9Y4bBkUbn2jSqgqqU3uaLCoMwXNtITXR1QjMtI/0YC+7ycpKOqloAUey3FBOwGS14xmzSOmwHshpdb3p/d8mTL6EECDNG3KCRi2MbQZomB/L5XhV+U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738349; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=v7aZptyUWrPaFG9pRx1pg1CGSw/d7rgG0aABpkfwXgI=; b=jxgp6MTZWoX9+d1ZXWnQpVtO63ckqend3iahyrOuarp2SqRhGiTIR3+WyyLl9cVAFFWqfCE2EPdWozc5eUn8ibJhAwGS32+J5CqK+OiIRT4kjhPgMusuZxJPAt37WdvGUamG+L0WKp0M9ZjzLSQSuqhXHr6eNbl6hZEO1HJIpOs= ARC-Authentication-Results: i=1; mx.zoho.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557738349061869.459795363336; Mon, 13 May 2019 02:05:49 -0700 (PDT) Received: from localhost ([127.0.0.1]:53613 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6tq-0002b5-2E for importer@patchew.org; Mon, 13 May 2019 05:05:34 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33416) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6o4-0006Yd-RI for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6ZM-0004LV-BT for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:25 -0400 Received: from 3.mo178.mail-out.ovh.net ([46.105.44.197]:36645) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6ZM-0004L6-5E for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:24 -0400 Received: from player778.ha.ovh.net (unknown [10.109.146.20]) by mo178.mail-out.ovh.net (Postfix) with ESMTP id 6F9C26010C for ; Mon, 13 May 2019 10:44:22 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id AFE1B59C41EB; Mon, 13 May 2019 08:44:14 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:44 +0200 Message-Id: <20190513084245.25755-14-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3343359773746695142 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 46.105.44.197 Subject: [Qemu-devel] [PATCH v4 13/14] ppc/xics: fix irq priority in ics_set_irq_type() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Recent commits changed the behavior of ics_set_irq_type() to initialize correctly LSIs at the KVM level. ics_set_irq_type() is also called by the realize routine of the different devices of the machine when initial interrupts are claimed, before the ICSState device is reseted. In the case, the ICSIRQState priority is 0x0 and the call to ics_set_irq_type() results in configuring the target of the interrupt. On P9, when using the KVM XICS-on-XIVE device, the target is configured to be server 0, priority 0 and the event queue 0 is created automatically by KVM. With the dual interrupt mode creating the KVM device at reset, it leads to unexpected effects on the guest, mostly blocking IPIs. This is wrong, fix it by reseting the ICSIRQState structure when ics_set_irq_type() is called. Fixes: commit 6cead90c5c9c ("xics: Write source state to KVM at claim time") Signed-off-by: Greg Kurz Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- hw/intc/xics.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/hw/intc/xics.c b/hw/intc/xics.c index af7dc709abab..79f5a8a91665 100644 --- a/hw/intc/xics.c +++ b/hw/intc/xics.c @@ -610,6 +610,12 @@ static const TypeInfo ics_simple_info =3D { .class_size =3D sizeof(ICSStateClass), }; =20 +static void ics_reset_irq(ICSIRQState *irq) +{ + irq->priority =3D 0xff; + irq->saved_priority =3D 0xff; +} + static void ics_base_reset(DeviceState *dev) { ICSState *ics =3D ICS_BASE(dev); @@ -623,8 +629,7 @@ static void ics_base_reset(DeviceState *dev) memset(ics->irqs, 0, sizeof(ICSIRQState) * ics->nr_irqs); =20 for (i =3D 0; i < ics->nr_irqs; i++) { - ics->irqs[i].priority =3D 0xff; - ics->irqs[i].saved_priority =3D 0xff; + ics_reset_irq(ics->irqs + i); ics->irqs[i].flags =3D flags[i]; } } @@ -760,6 +765,7 @@ void ics_set_irq_type(ICSState *ics, int srcno, bool ls= i) lsi ? XICS_FLAGS_IRQ_LSI : XICS_FLAGS_IRQ_MSI; =20 if (kvm_irqchip_in_kernel()) { + ics_reset_irq(ics->irqs + srcno); ics_set_kvm_state_one(ics, srcno); } } --=20 2.20.1 From nobody Fri May 3 09:54:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1557738244; cv=none; d=zoho.com; s=zohoarc; b=HETpIPk8Npr92kH2qNLE7Gi8rs3RDZBy8IGlF8AQWqRj0G+GdBiHnkQi83fhp195+7DmRFpSqz9FNb07nxanDww717yjmQmHs5FTcfLv/XOuEaYtiAVlLG8X5Gg0aAHVrK6Mk9JGLtBtKAr8JmLsQGp7mHgMHsxvPbkilV7XgMQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557738244; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=eG/FKZq+g0b7waj1fliaTmXlJjofbr/vdeQBNNskdRk=; b=dJh9/MGlqHqMh8Osw2JecC3JYFgPHGsg6HLUQrQg+R8UmmzVnAej30hVGicI2+AD30BzUOVJ1cGBhn8Lt8o3xa0gGB14W6Zi72BHU834iX7Z95AhlhkY5hj2Y8bQvwmp3ILJXu1TRo0oFXBKNzaY6yppgYUHCP52wCiCVlQh1UE= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15577382443068.740827057381011; Mon, 13 May 2019 02:04:04 -0700 (PDT) Received: from localhost ([127.0.0.1]:53579 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6sL-0001W9-8x for importer@patchew.org; Mon, 13 May 2019 05:04:01 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33558) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQ6o4-0006iZ-NO for qemu-devel@nongnu.org; Mon, 13 May 2019 04:59:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQ6ZR-0004NA-8F for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:31 -0400 Received: from 10.mo3.mail-out.ovh.net ([87.98.165.232]:55237) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQ6ZR-0004Mm-0s for qemu-devel@nongnu.org; Mon, 13 May 2019 04:44:29 -0400 Received: from player778.ha.ovh.net (unknown [10.109.146.20]) by mo3.mail-out.ovh.net (Postfix) with ESMTP id A77012102FB for ; Mon, 13 May 2019 10:44:27 +0200 (CEST) Received: from kaod.org (lfbn-1-10649-41.w90-89.abo.wanadoo.fr [90.89.235.41]) (Authenticated sender: clg@kaod.org) by player778.ha.ovh.net (Postfix) with ESMTPSA id 26AD959C4255; Mon, 13 May 2019 08:44:22 +0000 (UTC) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: David Gibson Date: Mon, 13 May 2019 10:42:45 +0200 Message-Id: <20190513084245.25755-15-clg@kaod.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190513084245.25755-1-clg@kaod.org> References: <20190513084245.25755-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 3344767151384333286 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduuddrleeggddtiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 87.98.165.232 Subject: [Qemu-devel] [PATCH v4 14/14] spapr/irq: add KVM support to the 'dual' machine X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Greg Kurz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" The interrupt mode is chosen by the CAS negotiation process and activated after a reset to take into account the required changes in the machine. This brings new constraints on how the associated KVM IRQ device is initialized. Currently, each model takes care of the initialization of the KVM device in their realize method but this is not possible anymore as the initialization needs to be done globaly when the interrupt mode is known, i.e. when machine is reseted. It also means that we need a way to delete a KVM device when another mode is chosen. Also, to support migration, the QEMU objects holding the state to transfer should always be available but not necessarily activated. The overall approach of this proposal is to initialize both interrupt mode at the QEMU level to keep the IRQ number space in sync and to allow switching from one mode to another. For the KVM side of things, the whole initialization of the KVM device, sources and presenters, is grouped in a single routine. The XICS and XIVE sPAPR IRQ reset handlers are modified accordingly to handle the init and the delete sequences of the KVM device. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- Changes since v3: - removed extra spapr_irq_init_device() in spapr_irq_init_xive()=20 - reworked slightly the code sequence in xics_kvm_init() creating the presenters and connecting them to the KVM XICS device. =20 include/hw/ppc/xive.h | 1 - hw/intc/spapr_xive_kvm.c | 29 +++++++++++++++++++- hw/intc/xics_kvm.c | 31 +++++++++++++++++++++ hw/intc/xive.c | 4 --- hw/ppc/spapr_irq.c | 58 ++++++++++++++++++++++++++-------------- 5 files changed, 97 insertions(+), 26 deletions(-) diff --git a/include/hw/ppc/xive.h b/include/hw/ppc/xive.h index edb8937f17fb..d872f96d1a1b 100644 --- a/include/hw/ppc/xive.h +++ b/include/hw/ppc/xive.h @@ -432,7 +432,6 @@ static inline uint32_t xive_nvt_cam_line(uint8_t nvt_bl= k, uint32_t nvt_idx) */ =20 void kvmppc_xive_source_reset_one(XiveSource *xsrc, int srcno, Error **err= p); -void kvmppc_xive_source_reset(XiveSource *xsrc, Error **errp); void kvmppc_xive_source_set_irq(void *opaque, int srcno, int val); void kvmppc_xive_cpu_connect(XiveTCTX *tctx, Error **errp); void kvmppc_xive_cpu_synchronize_state(XiveTCTX *tctx, Error **errp); diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c index 078d18d7757f..ec170b304555 100644 --- a/hw/intc/spapr_xive_kvm.c +++ b/hw/intc/spapr_xive_kvm.c @@ -246,7 +246,7 @@ void kvmppc_xive_source_reset_one(XiveSource *xsrc, int= srcno, Error **errp) true, errp); } =20 -void kvmppc_xive_source_reset(XiveSource *xsrc, Error **errp) +static void kvmppc_xive_source_reset(XiveSource *xsrc, Error **errp) { int i; =20 @@ -697,6 +697,15 @@ void kvmppc_xive_connect(SpaprXive *xive, Error **errp) Error *local_err =3D NULL; size_t esb_len =3D (1ull << xsrc->esb_shift) * xsrc->nr_irqs; size_t tima_len =3D 4ull << TM_SHIFT; + CPUState *cs; + + /* + * The KVM XIVE device already in use. This is the case when + * rebooting under the XIVE-only interrupt mode. + */ + if (xive->fd !=3D -1) { + return; + } =20 if (!kvmppc_has_cap_xive()) { error_setg(errp, "IRQ_XIVE capability must be present for KVM"); @@ -745,6 +754,24 @@ void kvmppc_xive_connect(SpaprXive *xive, Error **errp) xive->change =3D qemu_add_vm_change_state_handler( kvmppc_xive_change_state_handler, xive); =20 + /* Connect the presenters to the initial VCPUs of the machine */ + CPU_FOREACH(cs) { + PowerPCCPU *cpu =3D POWERPC_CPU(cs); + + kvmppc_xive_cpu_connect(spapr_cpu_state(cpu)->tctx, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + } + + /* Update the KVM sources */ + kvmppc_xive_source_reset(xsrc, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + kvm_kernel_irqchip =3D true; kvm_msi_via_irqfd_allowed =3D true; kvm_gsi_direct_mapping =3D true; diff --git a/hw/intc/xics_kvm.c b/hw/intc/xics_kvm.c index 12bd5190cfad..5ba5b775615e 100644 --- a/hw/intc/xics_kvm.c +++ b/hw/intc/xics_kvm.c @@ -33,6 +33,7 @@ #include "trace.h" #include "sysemu/kvm.h" #include "hw/ppc/spapr.h" +#include "hw/ppc/spapr_cpu_core.h" #include "hw/ppc/xics.h" #include "hw/ppc/xics_spapr.h" #include "kvm_ppc.h" @@ -342,6 +343,16 @@ static void rtas_dummy(PowerPCCPU *cpu, SpaprMachineSt= ate *spapr, int xics_kvm_init(SpaprMachineState *spapr, Error **errp) { int rc; + CPUState *cs; + Error *local_err =3D NULL; + + /* + * The KVM XICS device already in use. This is the case when + * rebooting under the XICS-only interrupt mode. + */ + if (kernel_xics_fd !=3D -1) { + return 0; + } =20 if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_IRQ_XICS= )) { error_setg(errp, @@ -390,6 +401,26 @@ int xics_kvm_init(SpaprMachineState *spapr, Error **er= rp) kvm_msi_via_irqfd_allowed =3D true; kvm_gsi_direct_mapping =3D true; =20 + /* Create the presenters */ + CPU_FOREACH(cs) { + PowerPCCPU *cpu =3D POWERPC_CPU(cs); + + icp_kvm_realize(DEVICE(spapr_cpu_state(cpu)->icp), &local_err); + if (local_err) { + error_propagate(errp, local_err); + goto fail; + } + } + + /* Update the KVM sources */ + ics_set_kvm_state(spapr->ics); + + /* Connect the presenters to the initial VCPUs of the machine */ + CPU_FOREACH(cs) { + PowerPCCPU *cpu =3D POWERPC_CPU(cs); + icp_set_kvm_state(spapr_cpu_state(cpu)->icp); + } + return 0; =20 fail: diff --git a/hw/intc/xive.c b/hw/intc/xive.c index b5ebb33527e7..0c74e47aa49c 100644 --- a/hw/intc/xive.c +++ b/hw/intc/xive.c @@ -993,10 +993,6 @@ static void xive_source_reset(void *dev) =20 /* PQs are initialized to 0b01 (Q=3D1) which corresponds to "ints off"= */ memset(xsrc->status, XIVE_ESB_OFF, xsrc->nr_irqs); - - if (kvm_irqchip_in_kernel()) { - kvmppc_xive_source_reset(xsrc, &error_fatal); - } } =20 static void xive_source_realize(DeviceState *dev, Error **errp) diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index d1e87577fb0e..3156daf09381 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -102,12 +102,6 @@ static void spapr_irq_init_xics(SpaprMachineState *spa= pr, int nr_irqs, Object *obj; Error *local_err =3D NULL; =20 - spapr_irq_init_device(spapr, &spapr_irq_xics, &local_err); - if (local_err) { - error_propagate(errp, local_err); - return; - } - obj =3D object_new(TYPE_ICS_SIMPLE); object_property_add_child(OBJECT(spapr), "ics", obj, &error_abort); object_property_add_const_link(obj, ICS_PROP_XICS, OBJECT(spapr), @@ -226,7 +220,13 @@ static void spapr_irq_set_irq_xics(void *opaque, int s= rcno, int val) =20 static void spapr_irq_reset_xics(SpaprMachineState *spapr, Error **errp) { - /* TODO: create the KVM XICS device */ + Error *local_err =3D NULL; + + spapr_irq_init_device(spapr, &spapr_irq_xics, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } } =20 static const char *spapr_irq_get_nodename_xics(SpaprMachineState *spapr) @@ -279,7 +279,6 @@ static void spapr_irq_init_xive(SpaprMachineState *spap= r, int nr_irqs, uint32_t nr_servers =3D spapr_max_server_number(spapr); DeviceState *dev; int i; - Error *local_err =3D NULL; =20 dev =3D qdev_create(NULL, TYPE_SPAPR_XIVE); qdev_prop_set_uint32(dev, "nr-irqs", nr_irqs); @@ -297,12 +296,6 @@ static void spapr_irq_init_xive(SpaprMachineState *spa= pr, int nr_irqs, } =20 spapr_xive_hcall_init(spapr); - - spapr_irq_init_device(spapr, &spapr_irq_xive, &local_err); - if (local_err) { - error_propagate(errp, local_err); - return; - } } =20 static int spapr_irq_claim_xive(SpaprMachineState *spapr, int irq, bool ls= i, @@ -382,6 +375,7 @@ static int spapr_irq_post_load_xive(SpaprMachineState *= spapr, int version_id) static void spapr_irq_reset_xive(SpaprMachineState *spapr, Error **errp) { CPUState *cs; + Error *local_err =3D NULL; =20 CPU_FOREACH(cs) { PowerPCCPU *cpu =3D POWERPC_CPU(cs); @@ -390,6 +384,12 @@ static void spapr_irq_reset_xive(SpaprMachineState *sp= apr, Error **errp) spapr_xive_set_tctx_os_cam(spapr_cpu_state(cpu)->tctx); } =20 + spapr_irq_init_device(spapr, &spapr_irq_xive, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + /* Activate the XIVE MMIOs */ spapr_xive_mmio_set_enabled(spapr->xive, true); } @@ -472,14 +472,8 @@ static SpaprIrq *spapr_irq_current(SpaprMachineState *= spapr) static void spapr_irq_init_dual(SpaprMachineState *spapr, int nr_irqs, Error **errp) { - MachineState *machine =3D MACHINE(spapr); Error *local_err =3D NULL; =20 - if (kvm_enabled() && machine_kernel_irqchip_allowed(machine)) { - error_setg(errp, "No KVM support for the 'dual' machine"); - return; - } - spapr_irq_xics.init(spapr, spapr_irq_xics.nr_irqs, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -558,6 +552,9 @@ static int spapr_irq_post_load_dual(SpaprMachineState *= spapr, int version_id) * defaults to XICS at startup. */ if (spapr_ovec_test(spapr->ov5_cas, OV5_XIVE_EXPLOIT)) { + if (kvm_irqchip_in_kernel()) { + xics_kvm_disconnect(spapr, &error_fatal); + } spapr_irq_xive.reset(spapr, &error_fatal); } =20 @@ -566,12 +563,30 @@ static int spapr_irq_post_load_dual(SpaprMachineState= *spapr, int version_id) =20 static void spapr_irq_reset_dual(SpaprMachineState *spapr, Error **errp) { + Error *local_err =3D NULL; + /* * Deactivate the XIVE MMIOs. The XIVE backend will reenable them * if selected. */ spapr_xive_mmio_set_enabled(spapr->xive, false); =20 + /* Destroy all KVM devices */ + if (kvm_irqchip_in_kernel()) { + xics_kvm_disconnect(spapr, &local_err); + if (local_err) { + error_propagate(errp, local_err); + error_prepend(errp, "KVM XICS disconnect failed: "); + return; + } + kvmppc_xive_disconnect(spapr->xive, &local_err); + if (local_err) { + error_propagate(errp, local_err); + error_prepend(errp, "KVM XIVE disconnect failed: "); + return; + } + } + spapr_irq_current(spapr)->reset(spapr, errp); } =20 @@ -809,6 +824,9 @@ SpaprIrq spapr_irq_xics_legacy =3D { .dt_populate =3D spapr_dt_xics, .cpu_intc_create =3D spapr_irq_cpu_intc_create_xics, .post_load =3D spapr_irq_post_load_xics, + .reset =3D spapr_irq_reset_xics, .set_irq =3D spapr_irq_set_irq_xics, .get_nodename =3D spapr_irq_get_nodename_xics, + .init_emu =3D spapr_irq_init_emu_xics, + .init_kvm =3D spapr_irq_init_kvm_xics, }; --=20 2.20.1