From nobody Mon Feb 9 22:39:16 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1499839105731789.5782437241008; Tue, 11 Jul 2017 22:58:25 -0700 (PDT) Received: from localhost ([::1]:50521 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVAfH-0004zr-8F for importer@patchew.org; Wed, 12 Jul 2017 01:58:23 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49444) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVAb0-0001WY-Rb for qemu-devel@nongnu.org; Wed, 12 Jul 2017 01:54:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dVAaw-0004uV-9h for qemu-devel@nongnu.org; Wed, 12 Jul 2017 01:53:58 -0400 Received: from ozlabs.org ([103.22.144.67]:55219) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dVAav-0004u6-Tg; Wed, 12 Jul 2017 01:53:54 -0400 Received: by ozlabs.org (Postfix, from userid 1007) id 3x6p5s2X7nz9sNx; Wed, 12 Jul 2017 15:53:21 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1499838801; bh=XxwX8dTRFEO7PzrtE5loDGUg7vJsaBH6vGZHfAAfLCM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=namGCCFE2CeY4MA77KqZ6ra9XvRocZl4JmAFgPmOj9f6WrOsqv+KQnvqVuWo5zwP5 JeWxSQ1ilYcs9fduUtgPOJs9lzq27mQlb9+/4NdAPa8j5D2bF1tkaWPlG9huZNbva8 Dz+jNEtuA1xkwzicRmXmcBrXZlA+q2+nLZLKDn3c= From: David Gibson To: mdroth@linux.vnet.ibm.com, groug@kaod.org, danielhb@linux.vnet.ibm.com Date: Wed, 12 Jul 2017 15:53:10 +1000 Message-Id: <20170712055317.26225-2-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170712055317.26225-1-david@gibson.dropbear.id.au> References: <20170712055317.26225-1-david@gibson.dropbear.id.au> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 103.22.144.67 Subject: [Qemu-devel] [PATCHv2 1/8] spapr: Treat devices added before inbound migration as coldplugged X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, qemu-devel@nongnu.org, qemu-ppc@nongnu.org, sjitindarsingh@gmail.com, bharata@linux.vnet.ibm.com, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Laurent Vivier When migrating a guest which has already had devices hotplugged, libvirt typically starts the destination qemu with -incoming defer, adds those hotplugged devices with qmp, then initiates the incoming migration. This causes problems for the management of spapr DRC state. Because the device is treated as hotplugged, it goes into a DRC state for a device immediately after it's plugged, but before the guest has acknowledged its presence. However, chances are the guest on the source machine *has* acknowledged the device's presence and configured it. If the source has fully configured the device, then DRC state won't be sent in the migration stream: for maximum migration compatibility with earlier versions we don't migrate DRCs in coldplug-equivalent state. That means that the DRC effectively changes state over the migrate, causing problems later on. In addition, logging hotplug events for these devices isn't what we want because a) those events should already have been issued on the source host and b) the event queue should get wiped out by the incoming state anyway. In short, what we really want is to treat devices added before an incoming migration as if they were coldplugged. To do this, we first add a spapr_drc_hotplugged() helper which determines if the device is hotplugged in the sense relevant for DRC state management. We only send hotplug events when this is true. Second, when we add a device which isn't hotplugged in this sense, we force a reset of the DRC state - this ensures the DRC is in a coldplug-equivalent state (there isn't usually a system reset between these device adds and the incoming migration). This is based on an earlier patch by Laurent Vivier, cleaned up and extended. Signed-off-by: Laurent Vivier Signed-off-by: David Gibson Reviewed-by: Greg Kurz --- hw/ppc/spapr.c | 24 ++++++++++++++++-------- hw/ppc/spapr_drc.c | 9 ++++++--- hw/ppc/spapr_pci.c | 4 +++- include/hw/ppc/spapr_drc.h | 8 ++++++++ 4 files changed, 33 insertions(+), 12 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 12b3f09..2a059d5 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2636,6 +2636,7 @@ static void spapr_add_lmbs(DeviceState *dev, uint64_t= addr_start, uint64_t size, int i, fdt_offset, fdt_size; void *fdt; uint64_t addr =3D addr_start; + bool hotplugged =3D spapr_drc_hotplugged(dev); Error *local_err =3D NULL; =20 for (i =3D 0; i < nr_lmbs; i++) { @@ -2659,12 +2660,15 @@ static void spapr_add_lmbs(DeviceState *dev, uint64= _t addr_start, uint64_t size, error_propagate(errp, local_err); return; } + if (!hotplugged) { + spapr_drc_reset(drc); + } addr +=3D SPAPR_MEMORY_BLOCK_SIZE; } /* send hotplug notification to the * guest only in case of hotplugged memory */ - if (dev->hotplugged) { + if (hotplugged) { if (dedicated_hp_event_source) { drc =3D spapr_drc_by_id(TYPE_SPAPR_DRC_LMB, addr_start / SPAPR_MEMORY_BLOCK_SIZE); @@ -2998,6 +3002,7 @@ static void spapr_core_plug(HotplugHandler *hotplug_d= ev, DeviceState *dev, int smt =3D kvmppc_smt_threads(); CPUArchId *core_slot; int index; + bool hotplugged =3D spapr_drc_hotplugged(dev); =20 core_slot =3D spapr_find_cpu_slot(MACHINE(hotplug_dev), cc->core_id, &= index); if (!core_slot) { @@ -3018,15 +3023,18 @@ static void spapr_core_plug(HotplugHandler *hotplug= _dev, DeviceState *dev, error_propagate(errp, local_err); return; } - } =20 - if (dev->hotplugged) { - /* - * Send hotplug notification interrupt to the guest only in case - * of hotplugged CPUs. - */ - spapr_hotplug_req_add_by_index(drc); + if (hotplugged) { + /* + * Send hotplug notification interrupt to the guest only + * in case of hotplugged CPUs. + */ + spapr_hotplug_req_add_by_index(drc); + } else { + spapr_drc_reset(drc); + } } + core_slot->cpu =3D OBJECT(dev); =20 if (smc->pre_2_10_has_unused_icps) { diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index f34355d..9b07f80 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -412,10 +412,8 @@ static bool release_pending(sPAPRDRConnector *drc) return drc->awaiting_release; } =20 -static void drc_reset(void *opaque) +void spapr_drc_reset(sPAPRDRConnector *drc) { - sPAPRDRConnector *drc =3D SPAPR_DR_CONNECTOR(opaque); - trace_spapr_drc_reset(spapr_drc_index(drc)); =20 g_free(drc->ccs); @@ -447,6 +445,11 @@ static void drc_reset(void *opaque) } } =20 +static void drc_reset(void *opaque) +{ + spapr_drc_reset(SPAPR_DR_CONNECTOR(opaque)); +} + static bool spapr_drc_needed(void *opaque) { sPAPRDRConnector *drc =3D (sPAPRDRConnector *)opaque; diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index a52dcf8..1e84c55 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -1443,7 +1443,9 @@ static void spapr_pci_plug(HotplugHandler *plug_handl= er, /* If this is function 0, signal hotplug for all the device functions. * Otherwise defer sending the hotplug event. */ - if (plugged_dev->hotplugged && PCI_FUNC(pdev->devfn) =3D=3D 0) { + if (!spapr_drc_hotplugged(plugged_dev)) { + spapr_drc_reset(drc); + } else if (PCI_FUNC(pdev->devfn) =3D=3D 0) { int i; =20 for (i =3D 0; i < 8; i++) { diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h index d15e9eb..715016b 100644 --- a/include/hw/ppc/spapr_drc.h +++ b/include/hw/ppc/spapr_drc.h @@ -15,6 +15,7 @@ =20 #include #include "qom/object.h" +#include "sysemu/sysemu.h" #include "hw/qdev.h" =20 #define TYPE_SPAPR_DR_CONNECTOR "spapr-dr-connector" @@ -223,6 +224,13 @@ typedef struct sPAPRDRConnectorClass { bool (*release_pending)(sPAPRDRConnector *drc); } sPAPRDRConnectorClass; =20 +static inline bool spapr_drc_hotplugged(DeviceState *dev) +{ + return dev->hotplugged && !runstate_check(RUN_STATE_INMIGRATE); +} + +void spapr_drc_reset(sPAPRDRConnector *drc); + uint32_t spapr_drc_index(sPAPRDRConnector *drc); sPAPRDRConnectorType spapr_drc_type(sPAPRDRConnector *drc); =20 --=20 2.9.4