From nobody Mon Apr 29 12:15:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1490608226752726.9162856854849; Mon, 27 Mar 2017 02:50:26 -0700 (PDT) Received: from localhost ([::1]:45366 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1csRI9-0007EN-6s for importer@patchew.org; Mon, 27 Mar 2017 05:50:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55130) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1csRGp-0006MB-Sh for qemu-devel@nongnu.org; Mon, 27 Mar 2017 05:49:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1csRGp-0007Z7-04 for qemu-devel@nongnu.org; Mon, 27 Mar 2017 05:49:03 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34536) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1csRGm-0007Y8-Jk; Mon, 27 Mar 2017 05:49:00 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 838E047D2; Mon, 27 Mar 2017 09:48:59 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-27.ams2.redhat.com [10.36.117.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id C829D857B3; Mon, 27 Mar 2017 09:48:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 838E047D2 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eric.auger@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 838E047D2 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, peter.maydell@linaro.org, qemu-arm@nongnu.org, qemu-devel@nongnu.org Date: Mon, 27 Mar 2017 11:48:44 +0200 Message-Id: <1490608126-22187-2-git-send-email-eric.auger@redhat.com> In-Reply-To: <1490608126-22187-1-git-send-email-eric.auger@redhat.com> References: <1490608126-22187-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 27 Mar 2017 09:48:59 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v3 1/3] linux-headers: Partial header update for vITS save/restore X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: drjones@redhat.com, vijay.kilari@gmail.com, quintela@redhat.com, dgilbert@redhat.com, peterx@redhat.com, Vijaya.Kumar@cavium.com, christoffer.dall@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This is a linux header update against 4.11-rc3 plus the non upstreamed ITS migration series. https://github.com/eauger/linux/tree/v4.11-rc3-its-mig-v4 It aims at enhancing the KVM user API with vITS save/restore capability. This consists in two new groups for the ARM_VGIC_ITS KVM device, named: KVM_DEV_ARM_VGIC_GRP_ITS_REGS, KVM_DEV_ARM_VGIC_GRP_ITS_TABLES. Signed-off-by: Eric Auger --- linux-headers/asm-arm/kvm.h | 2 ++ linux-headers/asm-arm64/kvm.h | 2 ++ 2 files changed, 4 insertions(+) diff --git a/linux-headers/asm-arm/kvm.h b/linux-headers/asm-arm/kvm.h index 1101d55..c8b9a68 100644 --- a/linux-headers/asm-arm/kvm.h +++ b/linux-headers/asm-arm/kvm.h @@ -192,6 +192,8 @@ struct kvm_arch_memory_slot { #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5 #define KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS 6 #define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7 +#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8 +#define KVM_DEV_ARM_VGIC_GRP_ITS_TABLES 9 #define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10 #define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \ (0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h index 651ec30..a8d7d67 100644 --- a/linux-headers/asm-arm64/kvm.h +++ b/linux-headers/asm-arm64/kvm.h @@ -212,6 +212,8 @@ struct kvm_arch_memory_slot { #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5 #define KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS 6 #define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7 +#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8 +#define KVM_DEV_ARM_VGIC_GRP_ITS_TABLES 9 #define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10 #define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \ (0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) --=20 2.5.5 From nobody Mon Apr 29 12:15:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1490608324378272.32373926108653; Mon, 27 Mar 2017 02:52:04 -0700 (PDT) Received: from localhost ([::1]:45377 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1csRJi-0008Vz-Rr for importer@patchew.org; Mon, 27 Mar 2017 05:52:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55162) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1csRGz-0006TV-Fa for qemu-devel@nongnu.org; Mon, 27 Mar 2017 05:49:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1csRGy-0007bL-Bs for qemu-devel@nongnu.org; Mon, 27 Mar 2017 05:49:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48604) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1csRGu-0007Zu-2v; Mon, 27 Mar 2017 05:49:08 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F3B514DD60; Mon, 27 Mar 2017 09:49:06 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-27.ams2.redhat.com [10.36.117.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6610189E1; Mon, 27 Mar 2017 09:48:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com F3B514DD60 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eric.auger@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com F3B514DD60 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, peter.maydell@linaro.org, qemu-arm@nongnu.org, qemu-devel@nongnu.org Date: Mon, 27 Mar 2017 11:48:45 +0200 Message-Id: <1490608126-22187-3-git-send-email-eric.auger@redhat.com> In-Reply-To: <1490608126-22187-1-git-send-email-eric.auger@redhat.com> References: <1490608126-22187-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 27 Mar 2017 09:49:07 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v3 2/3] hw/intc/arm_gicv3_its: Implement state save/restore X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: drjones@redhat.com, vijay.kilari@gmail.com, quintela@redhat.com, dgilbert@redhat.com, peterx@redhat.com, Vijaya.Kumar@cavium.com, christoffer.dall@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We need to handle both registers and ITS tables. While register handling is standard, ITS table handling is more challenging since the kernel API is devised so that the tables are flushed into guest RAM and not in vmstate buffers. Flushing the ITS tables on device pre_save() is too late since the guest RAM is already saved at this point. Table flushing needs to happen when we are sure the vcpus are stopped and before the last dirty page saving. The right point is RUN_STATE_FINISH_MIGRATE but sometimes the VM gets stopped before migration launch so let's simply flush the tables each time the VM gets stopped. For regular ITS registers we just can use vmstate pre_save() and post_load() callbacks. Signed-off-by: Eric Auger --- v2 -> v3: - take into account Peter's comments: - add iidr save/restore - reword comments - remove s->ctlr =3D extract64(reg, 0, 32); - rename get()/put() function into pre_save()/post_load() - do not execute put() if iidr =3D=3D 0 --- hw/intc/arm_gicv3_its_common.c | 9 ++++ hw/intc/arm_gicv3_its_kvm.c | 96 ++++++++++++++++++++++++++++++= ++++ include/hw/intc/arm_gicv3_its_common.h | 8 +++ 3 files changed, 113 insertions(+) diff --git a/hw/intc/arm_gicv3_its_common.c b/hw/intc/arm_gicv3_its_common.c index 9d67c5c..efab8c7 100644 --- a/hw/intc/arm_gicv3_its_common.c +++ b/hw/intc/arm_gicv3_its_common.c @@ -49,6 +49,15 @@ static const VMStateDescription vmstate_its =3D { .pre_save =3D gicv3_its_pre_save, .post_load =3D gicv3_its_post_load, .unmigratable =3D true, + .fields =3D (VMStateField[]) { + VMSTATE_UINT32(ctlr, GICv3ITSState), + VMSTATE_UINT32(iidr, GICv3ITSState), + VMSTATE_UINT64(cbaser, GICv3ITSState), + VMSTATE_UINT64(cwriter, GICv3ITSState), + VMSTATE_UINT64(creadr, GICv3ITSState), + VMSTATE_UINT64_ARRAY(baser, GICv3ITSState, 8), + VMSTATE_END_OF_LIST() + }, }; =20 static MemTxResult gicv3_its_trans_read(void *opaque, hwaddr offset, diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c index bd4f3aa..1732831 100644 --- a/hw/intc/arm_gicv3_its_kvm.c +++ b/hw/intc/arm_gicv3_its_kvm.c @@ -53,6 +53,24 @@ static int kvm_its_send_msi(GICv3ITSState *s, uint32_t v= alue, uint16_t devid) return kvm_vm_ioctl(kvm_state, KVM_SIGNAL_MSI, &msi); } =20 +/** + * vm_change_state_handler - VM change state callback aiming at flushing + * ITS tables into guest RAM + * + * The tables get flushed to guest RAM whenever the VM gets stopped. + */ +static void vm_change_state_handler(void *opaque, int running, + RunState state) +{ + GICv3ITSState *s =3D (GICv3ITSState *)opaque; + + if (running) { + return; + } + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_TABLES, + 0, NULL, false); +} + static void kvm_arm_its_realize(DeviceState *dev, Error **errp) { GICv3ITSState *s =3D ARM_GICV3_ITS_COMMON(dev); @@ -89,6 +107,8 @@ static void kvm_arm_its_realize(DeviceState *dev, Error = **errp) kvm_msi_use_devid =3D true; kvm_gsi_direct_mapping =3D false; kvm_msi_via_irqfd_allowed =3D kvm_irqfds_enabled(); + + qemu_add_vm_change_state_handler(vm_change_state_handler, s); } =20 static void kvm_arm_its_init(Object *obj) @@ -102,6 +122,80 @@ static void kvm_arm_its_init(Object *obj) &error_abort); } =20 +/** + * kvm_arm_its_pre_save - handles the saving of ITS registers. + * ITS tables are flushed into guest RAM separately and earlier, + * through the VM change state handler, since at the moment pre_save() + * is called, the guest RAM has already been saved. + */ +static void kvm_arm_its_pre_save(GICv3ITSState *s) +{ + uint64_t reg; + int i; + + for (i =3D 0; i < 8; i++) { + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_BASER + i * 8, &s->baser[i], false); + } + + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_CTLR, &s->ctlr, false); + + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_CBASER, &s->cbaser, false); + + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_CREADR, &s->creadr, false); + + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_CWRITER, &s->cwriter, false); + + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_IIDR, &s->iidr, false); +} + +/** + * kvm_arm_its_post_load - Restore both the ITS registers and tables + */ +static void kvm_arm_its_post_load(GICv3ITSState *s) +{ + uint64_t reg; + int i; + + if (!s->iidr) { + return; + } + + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_IIDR, &s->iidr, true); + + /* + * must be written before GITS_CREADR since GITS_CBASER write + * access resets GITS_CREADR. + */ + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_CBASER, &s->cbaser, true); + + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_CREADR, &s->creadr, true); + + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_CWRITER, &s->cwriter, true); + + + for (i =3D 0; i < 8; i++) { + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_BASER + i * 8, &s->baser[i], true); + } + + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_TABLES, + 0, NULL, true); + + reg =3D s->ctlr; + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_CTLR, ®, true); +} + static void kvm_arm_its_class_init(ObjectClass *klass, void *data) { DeviceClass *dc =3D DEVICE_CLASS(klass); @@ -109,6 +203,8 @@ static void kvm_arm_its_class_init(ObjectClass *klass, = void *data) =20 dc->realize =3D kvm_arm_its_realize; icc->send_msi =3D kvm_its_send_msi; + icc->pre_save =3D kvm_arm_its_pre_save; + icc->post_load =3D kvm_arm_its_post_load; } =20 static const TypeInfo kvm_arm_its_info =3D { diff --git a/include/hw/intc/arm_gicv3_its_common.h b/include/hw/intc/arm_g= icv3_its_common.h index 1ba1894..fd1fe64 100644 --- a/include/hw/intc/arm_gicv3_its_common.h +++ b/include/hw/intc/arm_gicv3_its_common.h @@ -28,6 +28,13 @@ #define ITS_TRANS_SIZE 0x10000 #define ITS_SIZE (ITS_CONTROL_SIZE + ITS_TRANS_SIZE) =20 +#define GITS_CTLR 0x0 +#define GITS_IIDR 0x4 +#define GITS_CBASER 0x80 +#define GITS_CWRITER 0x88 +#define GITS_CREADR 0x90 +#define GITS_BASER 0x100 + struct GICv3ITSState { SysBusDevice parent_obj; =20 @@ -43,6 +50,7 @@ struct GICv3ITSState { =20 /* Registers */ uint32_t ctlr; + uint32_t iidr; uint64_t cbaser; uint64_t cwriter; uint64_t creadr; --=20 2.5.5 From nobody Mon Apr 29 12:15:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1490608247821226.9559563840901; Mon, 27 Mar 2017 02:50:47 -0700 (PDT) Received: from localhost ([::1]:45368 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1csRIU-0007Y6-Lc for importer@patchew.org; Mon, 27 Mar 2017 05:50:46 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55190) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1csRH2-0006W0-R2 for qemu-devel@nongnu.org; Mon, 27 Mar 2017 05:49:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1csRH1-0007cb-RE for qemu-devel@nongnu.org; Mon, 27 Mar 2017 05:49:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44202) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1csRGy-0007bH-SY; Mon, 27 Mar 2017 05:49:13 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C13A0C04B938; Mon, 27 Mar 2017 09:49:11 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-27.ams2.redhat.com [10.36.117.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5637A189E1; Mon, 27 Mar 2017 09:49:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C13A0C04B938 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eric.auger@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C13A0C04B938 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, peter.maydell@linaro.org, qemu-arm@nongnu.org, qemu-devel@nongnu.org Date: Mon, 27 Mar 2017 11:48:46 +0200 Message-Id: <1490608126-22187-4-git-send-email-eric.auger@redhat.com> In-Reply-To: <1490608126-22187-1-git-send-email-eric.auger@redhat.com> References: <1490608126-22187-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 27 Mar 2017 09:49:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v3 3/3] hw/intc/arm_gicv3_its: Allow save/restore X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: drjones@redhat.com, vijay.kilari@gmail.com, quintela@redhat.com, dgilbert@redhat.com, peterx@redhat.com, Vijaya.Kumar@cavium.com, christoffer.dall@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We change the restoration priority of both the GICv3 and ITS. The GICv3 must be restored before the ITS and the ITS needs to be restored before PCIe devices since it translates their MSI transactions. Signed-off-by: Eric Auger --- v2 -> v3: - reword migration blocker message - remove unmigratable setting to false v1 -> v2: - handle case where migrate_add_blocker fails - add comments along with ITS and GICv3 migration priorities --- hw/intc/arm_gicv3_common.c | 1 + hw/intc/arm_gicv3_its_common.c | 2 +- hw/intc/arm_gicv3_its_kvm.c | 24 ++++++++++++------------ include/migration/vmstate.h | 2 ++ 4 files changed, 16 insertions(+), 13 deletions(-) diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c index c6493d6..4228b7c 100644 --- a/hw/intc/arm_gicv3_common.c +++ b/hw/intc/arm_gicv3_common.c @@ -145,6 +145,7 @@ static const VMStateDescription vmstate_gicv3 =3D { .minimum_version_id =3D 1, .pre_save =3D gicv3_pre_save, .post_load =3D gicv3_post_load, + .priority =3D MIG_PRI_GICV3, .fields =3D (VMStateField[]) { VMSTATE_UINT32(gicd_ctlr, GICv3State), VMSTATE_UINT32_ARRAY(gicd_statusr, GICv3State, 2), diff --git a/hw/intc/arm_gicv3_its_common.c b/hw/intc/arm_gicv3_its_common.c index efab8c7..22ce4c4 100644 --- a/hw/intc/arm_gicv3_its_common.c +++ b/hw/intc/arm_gicv3_its_common.c @@ -48,7 +48,7 @@ static const VMStateDescription vmstate_its =3D { .name =3D "arm_gicv3_its", .pre_save =3D gicv3_its_pre_save, .post_load =3D gicv3_its_post_load, - .unmigratable =3D true, + .priority =3D MIG_PRI_GICV3_ITS, .fields =3D (VMStateField[]) { VMSTATE_UINT32(ctlr, GICv3ITSState), VMSTATE_UINT32(iidr, GICv3ITSState), diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c index 1732831..15159f4 100644 --- a/hw/intc/arm_gicv3_its_kvm.c +++ b/hw/intc/arm_gicv3_its_kvm.c @@ -76,18 +76,6 @@ static void kvm_arm_its_realize(DeviceState *dev, Error = **errp) GICv3ITSState *s =3D ARM_GICV3_ITS_COMMON(dev); Error *local_err =3D NULL; =20 - /* - * Block migration of a KVM GICv3 ITS device: the API for saving and - * restoring the state in the kernel is not yet available - */ - error_setg(&s->migration_blocker, "vITS migration is not implemented"); - migrate_add_blocker(s->migration_blocker, &local_err); - if (local_err) { - error_propagate(errp, local_err); - error_free(s->migration_blocker); - return; - } - s->dev_fd =3D kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_ITS, = false); if (s->dev_fd < 0) { error_setg_errno(errp, -s->dev_fd, "error creating in-kernel ITS"); @@ -104,6 +92,18 @@ static void kvm_arm_its_realize(DeviceState *dev, Error= **errp) =20 gicv3_its_init_mmio(s, NULL); =20 + if (!kvm_device_check_attr(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + GITS_CTLR)) { + error_setg(&s->migration_blocker, "This operating system kernel " + "does not support vGICv3 migration"); + migrate_add_blocker(s->migration_blocker, &local_err); + if (local_err) { + error_propagate(errp, local_err); + error_free(s->migration_blocker); + return; + } + } + kvm_msi_use_devid =3D true; kvm_gsi_direct_mapping =3D false; kvm_msi_via_irqfd_allowed =3D kvm_irqfds_enabled(); diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index f2dbf84..8dab9c7 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -198,6 +198,8 @@ enum VMStateFlags { typedef enum { MIG_PRI_DEFAULT =3D 0, MIG_PRI_IOMMU, /* Must happen before PCI devices */ + MIG_PRI_GICV3_ITS, /* Must happen before PCI devices */ + MIG_PRI_GICV3, /* Must happen before the ITS */ MIG_PRI_MAX, } MigrationPriority; =20 --=20 2.5.5