From nobody Sun Apr 28 16:07:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1495481889392937.8286406854259; Mon, 22 May 2017 12:38:09 -0700 (PDT) Received: from localhost ([::1]:44562 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCt9a-0006Y5-SH for importer@patchew.org; Mon, 22 May 2017 15:38:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39526) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCt7q-0005DA-Jz for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dCt7m-0000CK-Qv for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:18 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:35318) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dCt7m-0000Bq-HL for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:14 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v4MJ9DjE113078 for ; Mon, 22 May 2017 15:36:13 -0400 Received: from e24smtp05.br.ibm.com (e24smtp05.br.ibm.com [32.104.18.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2am3rw5pxv-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 22 May 2017 15:36:13 -0400 Received: from localhost by e24smtp05.br.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 22 May 2017 16:36:10 -0300 Received: from d24relay04.br.ibm.com (9.18.232.146) by e24smtp05.br.ibm.com (10.172.0.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 22 May 2017 16:36:09 -0300 Received: from d24av02.br.ibm.com (d24av02.br.ibm.com [9.8.31.93]) by d24relay04.br.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v4MJa8Bx47841500; Mon, 22 May 2017 16:36:09 -0300 Received: from d24av02.br.ibm.com (localhost [127.0.0.1]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v4MJa8iK017091; Mon, 22 May 2017 16:36:08 -0300 Received: from localhost.localdomain ([9.85.129.159]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v4MJa2uU017026; Mon, 22 May 2017 16:36:06 -0300 From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Date: Mon, 22 May 2017 16:35:47 -0300 X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> References: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 17052219-0032-0000-0000-00000560FBA8 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17052219-0033-0000-0000-000011E661DE Message-Id: <20170522193551.826-2-danielhb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-22_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=3 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705220098 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v11 1/5] hw/ppc/spapr.c: adding pending_dimm_unplugs to sPAPRMachineState X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-ppc@nongnu.org, mdroth@linux.vnet.ibm.com, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The LMB DRC release callback, spapr_lmb_release(), uses an opaque parameter, a sPAPRDIMMState struct that stores the current LMBs that are allocated to a DIMM (nr_lmbs). After each call to this callback, the nr_lmbs is decremented by one and, when it reaches zero, the callback proceeds with the qdev calls to hot unplug the LMB. Using drc->detach_cb_opaque is problematic because it can't be migrated in the future DRC migration work. This patch makes the following changes to eliminate the usage of this opaque callback inside spapr_lmb_release: - sPAPRDIMMState was moved from spapr.c and added to spapr.h. A new attribute called 'addr' was added to it. This is used as an unique identifier to associate a sPAPRDIMMState to a PCDIMM element. - sPAPRMachineState now hosts a new QTAILQ called 'pending_dimm_unplugs'. This queue of sPAPRDIMMState elements will store the DIMM state of DIMMs that are currently going under an unplug process. - spapr_lmb_release() will now retrieve the nr_lmbs value by getting the correspondent sPAPRDIMMState. A helper function called spapr_dimm_get_addre= ss was created to fetch the address of a PCDIMM device inside spapr_lmb_releas= e. When nr_lmbs reaches zero and the callback proceeds with the qdev hot unplug calls, the sPAPRDIMMState struct is removed from spapr->pending_dimm_unplug= s. After these changes, the opaque argument for spapr_lmb_release is now unused and is passed as NULL inside spapr_del_lmbs. This and the other opaque arguments can now be safely removed from the code. As an additional cleanup made by this patch, the spapr_del_lmbs function was merged with spapr_memory_unplug_request. The former was being called only by the latter and both were small enough to fit one single function. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 98 +++++++++++++++++++++++++++++++---------------= ---- include/hw/ppc/spapr.h | 6 ++++ 2 files changed, 68 insertions(+), 36 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 75e298b..5743b78 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2059,6 +2059,7 @@ static void ppc_spapr_init(MachineState *machine) msi_nonbroken =3D true; =20 QLIST_INIT(&spapr->phbs); + QTAILQ_INIT(&spapr->pending_dimm_unplugs); =20 /* Allocate RMA if necessary */ rma_alloc_size =3D kvmppc_alloc_rma(&rma); @@ -2612,58 +2613,57 @@ out: error_propagate(errp, local_err); } =20 -typedef struct sPAPRDIMMState { +struct sPAPRDIMMState { + PCDIMMDevice *dimm; uint32_t nr_lmbs; -} sPAPRDIMMState; + QTAILQ_ENTRY(sPAPRDIMMState) next; +}; + +static sPAPRDIMMState *spapr_pending_dimm_unplugs_find(sPAPRMachineState *= s, + PCDIMMDevice *dimm) +{ + sPAPRDIMMState *dimm_state =3D NULL; + QTAILQ_FOREACH(dimm_state, &s->pending_dimm_unplugs, next) { + if (dimm_state->dimm =3D=3D dimm) { + break; + } + } + return dimm_state; +} + +static void spapr_pending_dimm_unplugs_add(sPAPRMachineState *spapr, + sPAPRDIMMState *dimm_state) +{ + g_assert(!spapr_pending_dimm_unplugs_find(spapr, dimm_state->dimm)); + QTAILQ_INSERT_HEAD(&spapr->pending_dimm_unplugs, dimm_state, next); +} + +static void spapr_pending_dimm_unplugs_remove(sPAPRMachineState *spapr, + sPAPRDIMMState *dimm_state) +{ + QTAILQ_REMOVE(&spapr->pending_dimm_unplugs, dimm_state, next); + g_free(dimm_state); +} =20 static void spapr_lmb_release(DeviceState *dev, void *opaque) { - sPAPRDIMMState *ds =3D (sPAPRDIMMState *)opaque; - HotplugHandler *hotplug_ctrl; + HotplugHandler *hotplug_ctrl =3D qdev_get_hotplug_handler(dev); + sPAPRMachineState *spapr =3D SPAPR_MACHINE(hotplug_ctrl); + sPAPRDIMMState *ds =3D spapr_pending_dimm_unplugs_find(spapr, PC_DIMM(= dev)); =20 if (--ds->nr_lmbs) { return; } =20 - g_free(ds); + spapr_pending_dimm_unplugs_remove(spapr, ds); =20 /* * Now that all the LMBs have been removed by the guest, call the * pc-dimm unplug handler to cleanup up the pc-dimm device. */ - hotplug_ctrl =3D qdev_get_hotplug_handler(dev); hotplug_handler_unplug(hotplug_ctrl, dev, &error_abort); } =20 -static void spapr_del_lmbs(DeviceState *dev, uint64_t addr_start, uint64_t= size, - Error **errp) -{ - sPAPRDRConnector *drc; - sPAPRDRConnectorClass *drck; - uint32_t nr_lmbs =3D size / SPAPR_MEMORY_BLOCK_SIZE; - int i; - sPAPRDIMMState *ds =3D g_malloc0(sizeof(sPAPRDIMMState)); - uint64_t addr =3D addr_start; - - ds->nr_lmbs =3D nr_lmbs; - for (i =3D 0; i < nr_lmbs; i++) { - drc =3D spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB, - addr / SPAPR_MEMORY_BLOCK_SIZE); - g_assert(drc); - - drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); - drck->detach(drc, dev, spapr_lmb_release, ds, errp); - addr +=3D SPAPR_MEMORY_BLOCK_SIZE; - } - - drc =3D spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB, - addr_start / SPAPR_MEMORY_BLOCK_SIZE); - drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); - spapr_hotplug_req_remove_by_count_indexed(SPAPR_DR_CONNECTOR_TYPE_LMB, - nr_lmbs, - drck->get_index(drc)); -} - static void spapr_memory_unplug(HotplugHandler *hotplug_dev, DeviceState *= dev, Error **errp) { @@ -2684,14 +2684,40 @@ static void spapr_memory_unplug_request(HotplugHand= ler *hotplug_dev, PCDIMMDeviceClass *ddc =3D PC_DIMM_GET_CLASS(dimm); MemoryRegion *mr =3D ddc->get_memory_region(dimm); uint64_t size =3D memory_region_size(mr); + uint32_t nr_lmbs =3D size / SPAPR_MEMORY_BLOCK_SIZE; uint64_t addr; + sPAPRDRConnector *drc; + sPAPRDRConnectorClass *drck; =20 addr =3D object_property_get_int(OBJECT(dimm), PC_DIMM_ADDR_PROP, &loc= al_err); if (local_err) { goto out; } =20 - spapr_del_lmbs(dev, addr, size, &error_abort); + sPAPRMachineState *spapr =3D SPAPR_MACHINE(hotplug_dev); + sPAPRDIMMState *ds =3D g_malloc0(sizeof(sPAPRDIMMState)); + ds->nr_lmbs =3D nr_lmbs; + ds->dimm =3D dimm; + spapr_pending_dimm_unplugs_add(spapr, ds); + + uint64_t addr_start =3D addr; + int i; + for (i =3D 0; i < nr_lmbs; i++) { + drc =3D spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB, + addr / SPAPR_MEMORY_BLOCK_SIZE); + g_assert(drc); + + drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); + drck->detach(drc, dev, spapr_lmb_release, NULL, errp); + addr +=3D SPAPR_MEMORY_BLOCK_SIZE; + } + + drc =3D spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB, + addr_start / SPAPR_MEMORY_BLOCK_SIZE); + drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); + spapr_hotplug_req_remove_by_count_indexed(SPAPR_DR_CONNECTOR_TYPE_LMB, + nr_lmbs, + drck->get_index(drc)); out: error_propagate(errp, local_err); } diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index be2b3b8..87af2f1 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -32,6 +32,7 @@ struct sPAPRRTCState { int64_t ns_offset; }; =20 +typedef struct sPAPRDIMMState sPAPRDIMMState; typedef struct sPAPRMachineClass sPAPRMachineClass; =20 #define TYPE_SPAPR_MACHINE "spapr-machine" @@ -105,6 +106,11 @@ struct sPAPRMachineState { /* RTAS state */ QTAILQ_HEAD(, sPAPRConfigureConnectorState) ccs_list; =20 + /* Pending DIMM unplug cache. It is populated when a LMB + * unplug starts. It can be regenerated if a migration + * occurs during the unplug process. */ + QTAILQ_HEAD(, sPAPRDIMMState) pending_dimm_unplugs; + /*< public >*/ char *kvm_type; MemoryHotplugState hotplug_memory; --=20 2.9.4 From nobody Sun Apr 28 16:07:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1495481881432277.19976199730354; Mon, 22 May 2017 12:38:01 -0700 (PDT) Received: from localhost ([::1]:44561 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCt9U-0006SP-1y for importer@patchew.org; Mon, 22 May 2017 15:38:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39571) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCt7u-0005FE-9W for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dCt7q-0000DT-Mg for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:22 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:43916) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dCt7q-0000Cu-CH for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:18 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v4MJ9Enj012619 for ; Mon, 22 May 2017 15:36:16 -0400 Received: from e24smtp01.br.ibm.com (e24smtp01.br.ibm.com [32.104.18.85]) by mx0a-001b2d01.pphosted.com with ESMTP id 2am3vpdfje-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 22 May 2017 15:36:16 -0400 Received: from localhost by e24smtp01.br.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 22 May 2017 16:36:13 -0300 Received: from d24relay02.br.ibm.com (9.18.232.42) by e24smtp01.br.ibm.com (10.172.0.143) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 22 May 2017 16:36:12 -0300 Received: from d24av02.br.ibm.com (d24av02.br.ibm.com [9.8.31.93]) by d24relay02.br.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v4MJaBvg17957152; Mon, 22 May 2017 16:36:11 -0300 Received: from d24av02.br.ibm.com (localhost [127.0.0.1]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v4MJaBvS017119; Mon, 22 May 2017 16:36:11 -0300 Received: from localhost.localdomain ([9.85.129.159]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v4MJa2uV017026; Mon, 22 May 2017 16:36:09 -0300 From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Date: Mon, 22 May 2017 16:35:48 -0300 X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> References: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 17052219-1523-0000-0000-000002A1B5F8 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17052219-1524-0000-0000-00002A383605 Message-Id: <20170522193551.826-3-danielhb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-22_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=3 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705220098 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v11 2/5] hw/ppc: removing drc->detach_cb and drc->detach_cb_opaque X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-ppc@nongnu.org, mdroth@linux.vnet.ibm.com, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The pointer drc->detach_cb is being used as a way of informing the detach() function inside spapr_drc.c which cb to execute. This information can also be retrieved simply by checking drc->type and choosing the right callback based on it. In this context, detach_cb is redundant information that must be managed. After the previous spapr_lmb_release change, no detach_cb_opaques are being used by any of the three callbacks functions. This is yet another information that is now unused and, on top of that, can't be migrated either. This patch makes the following changes: - removal of detach_cb_opaque. the 'opaque' argument was removed from the callbacks and from the detach() function of sPAPRConnectorClass. The attribute detach_cb_opaque of sPAPRConnector was removed. - removal of detach_cb from the detach() call. The function pointer detach_cb of sPAPRConnector was removed. detach() now uses a switch(drc->type) to execute the apropriate callback. To achieve this, spapr_core_release, spapr_lmb_release and spapr_phb_remove_pci_device_cb callbacks were made public to be visible inside detach(). Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 10 ++++++---- hw/ppc/spapr_drc.c | 36 ++++++++++++++++++++---------------- hw/ppc/spapr_pci.c | 5 +++-- include/hw/pci-host/spapr.h | 3 +++ include/hw/ppc/spapr.h | 4 ++++ include/hw/ppc/spapr_drc.h | 8 +------- 6 files changed, 37 insertions(+), 29 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 5743b78..463177e 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2645,7 +2645,8 @@ static void spapr_pending_dimm_unplugs_remove(sPAPRMa= chineState *spapr, g_free(dimm_state); } =20 -static void spapr_lmb_release(DeviceState *dev, void *opaque) +/* Callback to be called during DRC release. */ +void spapr_lmb_release(DeviceState *dev) { HotplugHandler *hotplug_ctrl =3D qdev_get_hotplug_handler(dev); sPAPRMachineState *spapr =3D SPAPR_MACHINE(hotplug_ctrl); @@ -2708,7 +2709,7 @@ static void spapr_memory_unplug_request(HotplugHandle= r *hotplug_dev, g_assert(drc); =20 drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); - drck->detach(drc, dev, spapr_lmb_release, NULL, errp); + drck->detach(drc, dev, errp); addr +=3D SPAPR_MEMORY_BLOCK_SIZE; } =20 @@ -2755,7 +2756,8 @@ static void spapr_core_unplug(HotplugHandler *hotplug= _dev, DeviceState *dev, object_unparent(OBJECT(dev)); } =20 -static void spapr_core_release(DeviceState *dev, void *opaque) +/* Callback to be called during DRC release. */ +void spapr_core_release(DeviceState *dev) { HotplugHandler *hotplug_ctrl; =20 @@ -2788,7 +2790,7 @@ void spapr_core_unplug_request(HotplugHandler *hotplu= g_dev, DeviceState *dev, g_assert(drc); =20 drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); - drck->detach(drc, dev, spapr_core_release, NULL, &local_err); + drck->detach(drc, dev, &local_err); if (local_err) { error_propagate(errp, local_err); return; diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index 9fa5545..2851e16 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -20,6 +20,7 @@ #include "qapi/visitor.h" #include "qemu/error-report.h" #include "hw/ppc/spapr.h" /* for RTAS return codes */ +#include "hw/pci-host/spapr.h" /* spapr_phb_remove_pci_device_cb callback = */ #include "trace.h" =20 #define DRC_CONTAINER_PATH "/dr-connector" @@ -99,8 +100,7 @@ static uint32_t set_isolation_state(sPAPRDRConnector *dr= c, if (drc->awaiting_release) { if (drc->configured) { trace_spapr_drc_set_isolation_state_finalizing(get_index(d= rc)); - drck->detach(drc, DEVICE(drc->dev), drc->detach_cb, - drc->detach_cb_opaque, NULL); + drck->detach(drc, DEVICE(drc->dev), NULL); } else { trace_spapr_drc_set_isolation_state_deferring(get_index(dr= c)); } @@ -153,8 +153,7 @@ static uint32_t set_allocation_state(sPAPRDRConnector *= drc, if (drc->awaiting_release && drc->allocation_state =3D=3D SPAPR_DR_ALLOCATION_STATE_UNUSABL= E) { trace_spapr_drc_set_allocation_state_finalizing(get_index(drc)= ); - drck->detach(drc, DEVICE(drc->dev), drc->detach_cb, - drc->detach_cb_opaque, NULL); + drck->detach(drc, DEVICE(drc->dev), NULL); } else if (drc->allocation_state =3D=3D SPAPR_DR_ALLOCATION_STATE_= USABLE) { drc->awaiting_allocation =3D false; } @@ -404,15 +403,10 @@ static void attach(sPAPRDRConnector *drc, DeviceState= *d, void *fdt, NULL, 0, NULL); } =20 -static void detach(sPAPRDRConnector *drc, DeviceState *d, - spapr_drc_detach_cb *detach_cb, - void *detach_cb_opaque, Error **errp) +static void detach(sPAPRDRConnector *drc, DeviceState *d, Error **errp) { trace_spapr_drc_detach(get_index(drc)); =20 - drc->detach_cb =3D detach_cb; - drc->detach_cb_opaque =3D detach_cb_opaque; - /* if we've signalled device presence to the guest, or if the guest * has gone ahead and configured the device (via manually-executed * device add via drmgr in guest, namely), we need to wait @@ -456,8 +450,21 @@ static void detach(sPAPRDRConnector *drc, DeviceState = *d, =20 drc->indicator_state =3D SPAPR_DR_INDICATOR_STATE_INACTIVE; =20 - if (drc->detach_cb) { - drc->detach_cb(drc->dev, drc->detach_cb_opaque); + /* Calling release callbacks based on drc->type. */ + switch (drc->type) { + case SPAPR_DR_CONNECTOR_TYPE_CPU: + spapr_core_release(drc->dev); + break; + case SPAPR_DR_CONNECTOR_TYPE_PCI: + spapr_phb_remove_pci_device_cb(drc->dev); + break; + case SPAPR_DR_CONNECTOR_TYPE_LMB: + spapr_lmb_release(drc->dev); + break; + case SPAPR_DR_CONNECTOR_TYPE_PHB: + case SPAPR_DR_CONNECTOR_TYPE_VIO: + default: + g_assert(false); } =20 drc->awaiting_release =3D false; @@ -467,8 +474,6 @@ static void detach(sPAPRDRConnector *drc, DeviceState *= d, drc->fdt_start_offset =3D 0; object_property_del(OBJECT(drc), "device", NULL); drc->dev =3D NULL; - drc->detach_cb =3D NULL; - drc->detach_cb_opaque =3D NULL; } =20 static bool release_pending(sPAPRDRConnector *drc) @@ -498,8 +503,7 @@ static void reset(DeviceState *d) * force removal if we are */ if (drc->awaiting_release) { - drck->detach(drc, DEVICE(drc->dev), drc->detach_cb, - drc->detach_cb_opaque, NULL); + drck->detach(drc, DEVICE(drc->dev), NULL); } =20 /* non-PCI devices may be awaiting a transition to UNUSABLE */ diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index a7cff32..e4daf8d 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -1369,7 +1369,8 @@ out: } } =20 -static void spapr_phb_remove_pci_device_cb(DeviceState *dev, void *opaque) +/* Callback to be called during DRC release. */ +void spapr_phb_remove_pci_device_cb(DeviceState *dev) { /* some version guests do not wait for completion of a device * cleanup (generally done asynchronously by the kernel) before @@ -1392,7 +1393,7 @@ static void spapr_phb_remove_pci_device(sPAPRDRConnec= tor *drc, { sPAPRDRConnectorClass *drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); =20 - drck->detach(drc, DEVICE(pdev), spapr_phb_remove_pci_device_cb, phb, e= rrp); + drck->detach(drc, DEVICE(pdev), errp); } =20 static sPAPRDRConnector *spapr_phb_get_pci_func_drc(sPAPRPHBState *phb, diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h index 1c2e970..38470b2 100644 --- a/include/hw/pci-host/spapr.h +++ b/include/hw/pci-host/spapr.h @@ -123,6 +123,9 @@ sPAPRPHBState *spapr_pci_find_phb(sPAPRMachineState *sp= apr, uint64_t buid); PCIDevice *spapr_pci_find_dev(sPAPRMachineState *spapr, uint64_t buid, uint32_t config_addr); =20 +/* PCI release callback. */ +void spapr_phb_remove_pci_device_cb(DeviceState *dev); + /* VFIO EEH hooks */ #ifdef CONFIG_LINUX bool spapr_phb_eeh_available(sPAPRPHBState *sphb); diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index 87af2f1..3aca19c 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -643,6 +643,10 @@ void spapr_hotplug_req_remove_by_count_indexed(sPAPRDR= ConnectorType drc_type, void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset, sPAPRMachineState *spapr); =20 +/* CPU and LMB DRC release callbacks. */ +void spapr_core_release(DeviceState *dev); +void spapr_lmb_release(DeviceState *dev); + /* rtas-configure-connector state */ struct sPAPRConfigureConnectorState { uint32_t drc_index; diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h index 5524247..813b9ff 100644 --- a/include/hw/ppc/spapr_drc.h +++ b/include/hw/ppc/spapr_drc.h @@ -130,8 +130,6 @@ typedef enum { SPAPR_DR_CC_RESPONSE_NOT_CONFIGURABLE =3D -9003, } sPAPRDRCCResponse; =20 -typedef void (spapr_drc_detach_cb)(DeviceState *d, void *opaque); - typedef struct sPAPRDRConnector { /*< private >*/ DeviceState parent; @@ -158,8 +156,6 @@ typedef struct sPAPRDRConnector { =20 /* device pointer, via link property */ DeviceState *dev; - spapr_drc_detach_cb *detach_cb; - void *detach_cb_opaque; } sPAPRDRConnector; =20 typedef struct sPAPRDRConnectorClass { @@ -188,9 +184,7 @@ typedef struct sPAPRDRConnectorClass { /* QEMU interfaces for managing hotplug operations */ void (*attach)(sPAPRDRConnector *drc, DeviceState *d, void *fdt, int fdt_start_offset, bool coldplug, Error **errp); - void (*detach)(sPAPRDRConnector *drc, DeviceState *d, - spapr_drc_detach_cb *detach_cb, - void *detach_cb_opaque, Error **errp); + void (*detach)(sPAPRDRConnector *drc, DeviceState *d, Error **errp); bool (*release_pending)(sPAPRDRConnector *drc); void (*set_signalled)(sPAPRDRConnector *drc); } sPAPRDRConnectorClass; --=20 2.9.4 From nobody Sun Apr 28 16:07:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1495481999510469.61988670560766; Mon, 22 May 2017 12:39:59 -0700 (PDT) Received: from localhost ([::1]:44571 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCtBO-0007xF-5W for importer@patchew.org; Mon, 22 May 2017 15:39:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39616) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCt7y-0005I5-BZ for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dCt7v-0000F8-9s for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:26 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:44051) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dCt7v-0000EZ-0N for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:23 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v4MJ9EG0012607 for ; Mon, 22 May 2017 15:36:22 -0400 Received: from e24smtp01.br.ibm.com (e24smtp01.br.ibm.com [32.104.18.85]) by mx0a-001b2d01.pphosted.com with ESMTP id 2am3vpdfmv-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 22 May 2017 15:36:21 -0400 Received: from localhost by e24smtp01.br.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 22 May 2017 16:36:18 -0300 Received: from d24relay02.br.ibm.com (9.18.232.42) by e24smtp01.br.ibm.com (10.172.0.143) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 22 May 2017 16:36:15 -0300 Received: from d24av02.br.ibm.com (d24av02.br.ibm.com [9.8.31.93]) by d24relay02.br.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v4MJaEWs33488902; Mon, 22 May 2017 16:36:14 -0300 Received: from d24av02.br.ibm.com (localhost [127.0.0.1]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v4MJaEAT017145; Mon, 22 May 2017 16:36:14 -0300 Received: from localhost.localdomain ([9.85.129.159]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v4MJa2uW017026; Mon, 22 May 2017 16:36:12 -0300 From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Date: Mon, 22 May 2017 16:35:49 -0300 X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> References: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 17052219-1523-0000-0000-000002A1B5FB X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17052219-1524-0000-0000-00002A383608 Message-Id: <20170522193551.826-4-danielhb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-22_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=3 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705220098 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v11 3/5] hw/ppc: migrating the DRC state of hotplugged devices X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-ppc@nongnu.org, mdroth@linux.vnet.ibm.com, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" In pseries, a firmware abstraction called Dynamic Reconfiguration Connector (DRC) is used to assign a particular dynamic resource to the guest and provide an interface to manage configuration/removal of the resource associated with it. In other words, DRC is the 'plugged state' of a device. Before this patch, DRC wasn't being migrated. This causes post-migration problems due to DRC state mismatch between source and target. The DRC state of a device X in the source might change, while in the target the DRC state of X is still fresh. When migrating the guest, X will not have the same hotplugged state as it did in the source. This means that we can't hot unplug X in the target after migration is completed because its DRC state is not consistent. https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1677552 is one bug that is caused by this DRC state mismatch between source and target. To migrate the DRC state, we defined the VMStateDescription struct for spapr_drc to enable the transmission of spapr_drc state in migration. Not all the elements in the DRC state are migrated - only those that can be modified by guest actions or device add/remove operations: - 'isolation_state', 'allocation_state' and 'indicator_state' are involved in the DR state transition diagram from PAPR+ 2.7, 13.4; - 'configured', 'signalled', 'awaiting_release' and 'awaiting_allocation' are needed in attaching and detaching devices; - 'indicator_state' provides users with hardware state information. These are the DRC elements that are migrated. In this patch the DRC state is migrated for PCI, LMB and CPU connector types. At this moment there is no support to migrate DRC for the PHB (PCI Host Bridge) type. In the 'realize' function the DRC is registered using vmstate_register, similar to what hw/ppc/spapr_iommu.c does in 'spapr_tce_table_realize'. This approach works because DRCs are bus-less and do not sit on a BusClass that implements bc->get_dev_path, so as a fallback the VMSD gets identified via "spapr_drc"/get_index(drc). Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr_drc.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 56 insertions(+) diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index 2851e16..cc2400b 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -519,6 +519,60 @@ static void reset(DeviceState *d) } } =20 +static bool spapr_drc_needed(void *opaque) +{ + sPAPRDRConnector *drc =3D (sPAPRDRConnector *)opaque; + sPAPRDRConnectorClass *drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); + bool rc =3D false; + sPAPRDREntitySense value; + drck->entity_sense(drc, &value); + + /* If no dev is plugged in there is no need to migrate the DRC state */ + if (value !=3D SPAPR_DR_ENTITY_SENSE_PRESENT) { + return false; + } + + /* + * If there is dev plugged in, we need to migrate the DRC state when + * it is different from cold-plugged state + */ + switch (drc->type) { + case SPAPR_DR_CONNECTOR_TYPE_PCI: + rc =3D !((drc->isolation_state =3D=3D SPAPR_DR_ISOLATION_STATE_UNI= SOLATED) && + (drc->allocation_state =3D=3D SPAPR_DR_ALLOCATION_STATE_USA= BLE) && + drc->configured && drc->signalled && !drc->awaiting_release= ); + break; + case SPAPR_DR_CONNECTOR_TYPE_CPU: + case SPAPR_DR_CONNECTOR_TYPE_LMB: + rc =3D !((drc->isolation_state =3D=3D SPAPR_DR_ISOLATION_STATE_ISO= LATED) && + (drc->allocation_state =3D=3D SPAPR_DR_ALLOCATION_STATE_UNU= SABLE) && + drc->configured && drc->signalled && !drc->awaiting_release= ); + break; + case SPAPR_DR_CONNECTOR_TYPE_PHB: + case SPAPR_DR_CONNECTOR_TYPE_VIO: + default: + g_assert(false); + } + return rc; +} + +static const VMStateDescription vmstate_spapr_drc =3D { + .name =3D "spapr_drc", + .version_id =3D 1, + .minimum_version_id =3D 1, + .needed =3D spapr_drc_needed, + .fields =3D (VMStateField []) { + VMSTATE_UINT32(isolation_state, sPAPRDRConnector), + VMSTATE_UINT32(allocation_state, sPAPRDRConnector), + VMSTATE_UINT32(indicator_state, sPAPRDRConnector), + VMSTATE_BOOL(configured, sPAPRDRConnector), + VMSTATE_BOOL(awaiting_release, sPAPRDRConnector), + VMSTATE_BOOL(awaiting_allocation, sPAPRDRConnector), + VMSTATE_BOOL(signalled, sPAPRDRConnector), + VMSTATE_END_OF_LIST() + } +}; + static void realize(DeviceState *d, Error **errp) { sPAPRDRConnector *drc =3D SPAPR_DR_CONNECTOR(d); @@ -547,6 +601,8 @@ static void realize(DeviceState *d, Error **errp) object_unref(OBJECT(drc)); } g_free(child_name); + vmstate_register(DEVICE(drc), drck->get_index(drc), &vmstate_spapr_drc, + drc); trace_spapr_drc_realize_complete(drck->get_index(drc)); } =20 --=20 2.9.4 From nobody Sun Apr 28 16:07:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1495482094575799.0980122887577; Mon, 22 May 2017 12:41:34 -0700 (PDT) Received: from localhost ([::1]:44584 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCtCv-0000dr-8e for importer@patchew.org; Mon, 22 May 2017 15:41:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39620) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCt7y-0005I7-Dr for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dCt7u-0000En-Go for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:26 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:48686 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dCt7u-0000EV-BJ for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:22 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v4MJ9HQS049971 for ; Mon, 22 May 2017 15:36:21 -0400 Received: from e24smtp05.br.ibm.com (e24smtp05.br.ibm.com [32.104.18.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2am3ek6p0q-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 22 May 2017 15:36:21 -0400 Received: from localhost by e24smtp05.br.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 22 May 2017 16:36:19 -0300 Received: from d24relay04.br.ibm.com (9.18.232.146) by e24smtp05.br.ibm.com (10.172.0.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 22 May 2017 16:36:17 -0300 Received: from d24av02.br.ibm.com (d24av02.br.ibm.com [9.8.31.93]) by d24relay04.br.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v4MJaHLx4915614; Mon, 22 May 2017 16:36:17 -0300 Received: from d24av02.br.ibm.com (localhost [127.0.0.1]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v4MJaGSS017173; Mon, 22 May 2017 16:36:17 -0300 Received: from localhost.localdomain ([9.85.129.159]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v4MJa2uX017026; Mon, 22 May 2017 16:36:14 -0300 From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Date: Mon, 22 May 2017 16:35:50 -0300 X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> References: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 17052219-0032-0000-0000-00000560FBAC X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17052219-0033-0000-0000-000011E661E2 Message-Id: <20170522193551.826-5-danielhb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-22_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=3 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705220098 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH v11 4/5] hw/ppc/spapr.c: recover pending LMB unplug info in spapr_lmb_release X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-ppc@nongnu.org, mdroth@linux.vnet.ibm.com, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When a LMB hot unplug starts, the current DRC LMB status is stored at spapr->pending_dimm_unplugs QTAILQ. This queue isn't migrated, thus if a migration occurs in the middle of a LMB unplug the spapr_lmb_release callback will lost track of the LMB unplug progress. This patch implements a new recover function spapr_recover_pending_dimm_sta= te that is used inside spapr_lmb_release to recover this DRC LMB release status that is lost during the migration. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 463177e..c4473b9 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2645,6 +2645,44 @@ static void spapr_pending_dimm_unplugs_remove(sPAPRM= achineState *spapr, g_free(dimm_state); } =20 +static sPAPRDIMMState *spapr_recover_pending_dimm_state(sPAPRMachineState = *ms, + PCDIMMDevice *dimm) +{ + sPAPRDRConnector *drc; + PCDIMMDeviceClass *ddc =3D PC_DIMM_GET_CLASS(dimm); + MemoryRegion *mr =3D ddc->get_memory_region(dimm); + uint64_t size =3D memory_region_size(mr); + uint32_t nr_lmbs =3D size / SPAPR_MEMORY_BLOCK_SIZE; + uint32_t avail_lmbs =3D 0; + + Error *local_err =3D NULL; + uint64_t addr; + addr =3D object_property_get_int(OBJECT(dimm), PC_DIMM_ADDR_PROP, + &local_err); + if (local_err) { + error_propagate(&error_abort, local_err); + return 0; + } + + uint64_t curr_addr =3D addr; + int i; + for (i =3D 0; i < nr_lmbs; i++) { + drc =3D spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB, + curr_addr / SPAPR_MEMORY_BLOCK_SIZE= ); + g_assert(drc); + if (drc->indicator_state !=3D SPAPR_DR_INDICATOR_STATE_INACTIVE) { + avail_lmbs++; + } + curr_addr +=3D SPAPR_MEMORY_BLOCK_SIZE; + } + + sPAPRDIMMState *ds =3D g_malloc0(sizeof(sPAPRDIMMState)); + ds->nr_lmbs =3D avail_lmbs; + ds->dimm =3D dimm; + spapr_pending_dimm_unplugs_add(ms, ds); + return ds; +} + /* Callback to be called during DRC release. */ void spapr_lmb_release(DeviceState *dev) { @@ -2652,7 +2690,14 @@ void spapr_lmb_release(DeviceState *dev) sPAPRMachineState *spapr =3D SPAPR_MACHINE(hotplug_ctrl); sPAPRDIMMState *ds =3D spapr_pending_dimm_unplugs_find(spapr, PC_DIMM(= dev)); =20 - if (--ds->nr_lmbs) { + /* This information will get lost if a migration occurs + * during the unplug process. In this case recover it. */ + if (ds =3D=3D NULL) { + ds =3D spapr_recover_pending_dimm_state(spapr, PC_DIMM(dev)); + if (ds->nr_lmbs) { + return; + } + } else if (--ds->nr_lmbs) { return; } =20 --=20 2.9.4 From nobody Sun Apr 28 16:07:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1495482013200737.3181390133929; Mon, 22 May 2017 12:40:13 -0700 (PDT) Received: from localhost ([::1]:44575 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCtBc-00087g-0F for importer@patchew.org; Mon, 22 May 2017 15:40:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39677) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dCt84-0005MX-8M for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dCt80-0000H3-HG for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:32 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:55204) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dCt80-0000Gl-7e for qemu-devel@nongnu.org; Mon, 22 May 2017 15:36:28 -0400 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v4MJPh18115102 for ; Mon, 22 May 2017 15:36:27 -0400 Received: from e24smtp01.br.ibm.com (e24smtp01.br.ibm.com [32.104.18.85]) by mx0a-001b2d01.pphosted.com with ESMTP id 2am4b3cexn-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 22 May 2017 15:36:26 -0400 Received: from localhost by e24smtp01.br.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 22 May 2017 16:36:24 -0300 Received: from d24relay02.br.ibm.com (9.18.232.42) by e24smtp01.br.ibm.com (10.172.0.143) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 22 May 2017 16:36:21 -0300 Received: from d24av02.br.ibm.com (d24av02.br.ibm.com [9.8.31.93]) by d24relay02.br.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v4MJaKLQ18153870; Mon, 22 May 2017 16:36:20 -0300 Received: from d24av02.br.ibm.com (localhost [127.0.0.1]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v4MJaKED017215; Mon, 22 May 2017 16:36:20 -0300 Received: from localhost.localdomain ([9.85.129.159]) by d24av02.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v4MJa2uY017026; Mon, 22 May 2017 16:36:17 -0300 From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Date: Mon, 22 May 2017 16:35:51 -0300 X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> References: <20170522193551.826-1-danielhb@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 17052219-1523-0000-0000-000002A1B602 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17052219-1524-0000-0000-00002A38360D Message-Id: <20170522193551.826-6-danielhb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-22_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705220098 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v11 5/5] migration: spapr: migrate ccs_list in spapr state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-ppc@nongnu.org, mdroth@linux.vnet.ibm.com, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Jianjun Duan ccs_list in spapr state maintains the device tree related information on the rtas side for hotplugged devices. In racing situations between hotplug events and migration operation, a rtas hotplug event could be migrated from the source guest to target guest, or the source guest could have not yet finished fetching the device tree when migration is started, the target will try to finish fetching the device tree. By migrating ccs_list, the target can fetch the device tree properly. In theory there would be other alternatives besides migrating the css_list to fix this. For example, we could add a flag that indicates whether a device is in the middle of the configure_connector during the migration process, in the post_load we can detect if this flag is active and then return an error informing the guest to restart the hotplug process. However, the DRC state can still be modified outside of hotplug. Using: drmgr -c pci -s -r drmgr -c pci -s -a it is possible to return a device to firmware and then later take it back and reconfigure it. This is not a common case but it's not prohibited, and performing a migration between these 2 operations would fail because the default coldplug state on target assumes a configured state in the source*. Migrating ccs_list is one solution that cover this case as well. ccs_list is put in a subsection in the spapr state VMSD to make sure migration across different versions is not broken. * see http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg01763.html for more information on this discussion. Signed-off-by: Jianjun Duan Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index c4473b9..785a5e9 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1453,6 +1453,37 @@ static bool version_before_3(void *opaque, int versi= on_id) return version_id < 3; } =20 +static bool spapr_ccs_list_needed(void *opaque) +{ + sPAPRMachineState *spapr =3D (sPAPRMachineState *)opaque; + return !QTAILQ_EMPTY(&spapr->ccs_list); +} + +static const VMStateDescription vmstate_spapr_ccs =3D { + .name =3D "spapr_configure_connector_state", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (VMStateField[]) { + VMSTATE_UINT32(drc_index, sPAPRConfigureConnectorState), + VMSTATE_INT32(fdt_offset, sPAPRConfigureConnectorState), + VMSTATE_INT32(fdt_depth, sPAPRConfigureConnectorState), + VMSTATE_END_OF_LIST() + }, +}; + +static const VMStateDescription vmstate_spapr_ccs_list =3D { + .name =3D "spapr_ccs_list", + .version_id =3D 1, + .minimum_version_id =3D 1, + .needed =3D spapr_ccs_list_needed, + .fields =3D (VMStateField[]) { + VMSTATE_QTAILQ_V(ccs_list, sPAPRMachineState, 1, + vmstate_spapr_ccs, sPAPRConfigureConnectorState, + next), + VMSTATE_END_OF_LIST() + }, +}; + static bool spapr_ov5_cas_needed(void *opaque) { sPAPRMachineState *spapr =3D opaque; @@ -1551,6 +1582,7 @@ static const VMStateDescription vmstate_spapr =3D { .subsections =3D (const VMStateDescription*[]) { &vmstate_spapr_ov5_cas, &vmstate_spapr_patb_entry, + &vmstate_spapr_ccs_list, NULL } }; --=20 2.9.4