From nobody Thu Apr 25 21:51:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1581692559149341.9951992707922; Fri, 14 Feb 2020 07:02:39 -0800 (PST) Received: from localhost ([::1]:39960 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2cUH-0007I1-7b for importer@patchew.org; Fri, 14 Feb 2020 10:02:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43327) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2cTN-0005vf-5z for qemu-devel@nongnu.org; Fri, 14 Feb 2020 10:01:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j2cTI-0003mh-1V for qemu-devel@nongnu.org; Fri, 14 Feb 2020 10:01:41 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:61668 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j2cTH-0003lH-Ru for qemu-devel@nongnu.org; Fri, 14 Feb 2020 10:01:35 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 01EExvi9139676 for ; Fri, 14 Feb 2020 10:01:34 -0500 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0b-001b2d01.pphosted.com with ESMTP id 2y5vhpjt41-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Feb 2020 10:01:33 -0500 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 14 Feb 2020 15:01:27 -0000 Received: from b06avi18626390.portsmouth.uk.ibm.com (9.149.26.192) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Fri, 14 Feb 2020 15:01:24 -0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 01EF0ScS37093732 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 14 Feb 2020 15:00:28 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F182C42049; Fri, 14 Feb 2020 15:01:22 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B933042041; Fri, 14 Feb 2020 15:01:22 +0000 (GMT) Received: from bahia.lan (unknown [9.145.71.236]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Fri, 14 Feb 2020 15:01:22 +0000 (GMT) Subject: [PATCH v2 1/2] spapr: Don't use spapr_drc_needed() in CAS code From: Greg Kurz To: David Gibson Date: Fri, 14 Feb 2020 16:01:22 +0100 In-Reply-To: <158169247578.3465937.4013536808417411649.stgit@bahia.lan> References: <158169247578.3465937.4013536808417411649.stgit@bahia.lan> User-Agent: StGit/unknown-version MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 x-cbid: 20021415-0028-0000-0000-000003DA8904 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 20021415-0029-0000-0000-0000249F03EF Message-Id: <158169248180.3465937.9531405453362718771.stgit@bahia.lan> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-02-14_04:2020-02-12, 2020-02-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 bulkscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 spamscore=0 clxscore=1034 adultscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2001150001 definitions=main-2002140119 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Alexey Kardashevskiy , qemu-ppc@nongnu.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" We currently don't support hotplug of devices between boot and CAS. If this happens a CAS reboot is triggered. We detect this during CAS using the spapr_drc_needed() function which is essentially a VMStateDescription .needed callback. Even if the condition for CAS reboot happens to be the same as for DRC migration, it looks wrong to piggyback a migration helper for this. Introduce a helper with slightly more explicit name and use it in both CAS and DRC migration code. Since a subsequent patch will enhance this helper to cover the case of hot unplug, let's go for spapr_drc_transient(). While here convert spapr_hotplugged_dev_before_cas() to the "transient" wording as well. This doesn't change any behaviour. Signed-off-by: Greg Kurz --- v2: - spapr_drc_transient() helper --- hw/ppc/spapr_drc.c | 20 ++++++++++++++------ hw/ppc/spapr_hcall.c | 14 +++++++++----- include/hw/ppc/spapr_drc.h | 4 +++- 3 files changed, 26 insertions(+), 12 deletions(-) diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index fc62e049010f..4c35ce7c5c37 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -456,23 +456,31 @@ void spapr_drc_reset(SpaprDrc *drc) } } =20 -bool spapr_drc_needed(void *opaque) +bool spapr_drc_transient(SpaprDrc *drc) { - SpaprDrc *drc =3D (SpaprDrc *)opaque; SpaprDrcClass *drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); =20 - /* If no dev is plugged in there is no need to migrate the DRC state */ + /* + * If no dev is plugged in there is no need to migrate the DRC state + * nor to reset the DRC at CAS. + */ if (!drc->dev) { return false; } =20 /* - * We need to migrate the state if it's not equal to the expected - * long-term state, which is the same as the coldplugged initial - * state */ + * We need to reset the DRC at CAS or to migrate the DRC state if it's + * not equal to the expected long-term state, which is the same as the + * coldplugged initial state. + */ return (drc->state !=3D drck->ready_state); } =20 +static bool spapr_drc_needed(void *opaque) +{ + return spapr_drc_transient(opaque); +} + static const VMStateDescription vmstate_spapr_drc =3D { .name =3D "spapr_drc", .version_id =3D 1, diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c index b8bb66b5c0d4..6db3dbde9c92 100644 --- a/hw/ppc/spapr_hcall.c +++ b/hw/ppc/spapr_hcall.c @@ -1640,20 +1640,24 @@ static uint32_t cas_check_pvr(SpaprMachineState *sp= apr, PowerPCCPU *cpu, return best_compat; } =20 -static bool spapr_hotplugged_dev_before_cas(void) +static bool spapr_transient_dev_before_cas(void) { - Object *drc_container, *obj; + Object *drc_container; ObjectProperty *prop; ObjectPropertyIterator iter; =20 drc_container =3D container_get(object_get_root(), "/dr-connector"); object_property_iter_init(&iter, drc_container); while ((prop =3D object_property_iter_next(&iter))) { + SpaprDrc *drc; + if (!strstart(prop->type, "link<", NULL)) { continue; } - obj =3D object_property_get_link(drc_container, prop->name, NULL); - if (spapr_drc_needed(obj)) { + drc =3D SPAPR_DR_CONNECTOR(object_property_get_link(drc_container, + prop->name, NULL= )); + + if (spapr_drc_transient(drc)) { return true; } } @@ -1830,7 +1834,7 @@ static target_ulong h_client_architecture_support(Pow= erPCCPU *cpu, =20 spapr_irq_update_active_intc(spapr); =20 - if (spapr_hotplugged_dev_before_cas()) { + if (spapr_transient_dev_before_cas()) { spapr->cas_reboot =3D true; } =20 diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h index df3d958a66a2..21af8deac13f 100644 --- a/include/hw/ppc/spapr_drc.h +++ b/include/hw/ppc/spapr_drc.h @@ -278,7 +278,9 @@ int spapr_dt_drc(void *fdt, int offset, Object *owner, = uint32_t drc_type_mask); =20 void spapr_drc_attach(SpaprDrc *drc, DeviceState *d, Error **errp); void spapr_drc_detach(SpaprDrc *drc); -bool spapr_drc_needed(void *opaque); + +/* Returns true if a hot plug/unplug request is pending */ +bool spapr_drc_transient(SpaprDrc *drc); =20 static inline bool spapr_drc_unplug_requested(SpaprDrc *drc) { From nobody Thu Apr 25 21:51:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 158169274524920.577737123228985; Fri, 14 Feb 2020 07:05:45 -0800 (PST) Received: from localhost ([::1]:40004 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2cXI-00021k-8K for importer@patchew.org; Fri, 14 Feb 2020 10:05:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:44143) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2cWY-0001Xo-8w for qemu-devel@nongnu.org; Fri, 14 Feb 2020 10:04:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j2cWX-0007px-4C for qemu-devel@nongnu.org; Fri, 14 Feb 2020 10:04:58 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:64284) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j2cWW-0007pQ-TE for qemu-devel@nongnu.org; Fri, 14 Feb 2020 10:04:57 -0500 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 01EF0WPH116208 for ; Fri, 14 Feb 2020 10:04:56 -0500 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2y3u53tadf-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Feb 2020 10:02:59 -0500 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 14 Feb 2020 15:01:32 -0000 Received: from b06avi18626390.portsmouth.uk.ibm.com (9.149.26.192) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Fri, 14 Feb 2020 15:01:30 -0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 01EF0Ywv49611196 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 14 Feb 2020 15:00:34 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1E70111C052; Fri, 14 Feb 2020 15:01:29 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DB5AA11C04C; Fri, 14 Feb 2020 15:01:28 +0000 (GMT) Received: from bahia.lan (unknown [9.145.71.236]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Fri, 14 Feb 2020 15:01:28 +0000 (GMT) Subject: [PATCH v2 2/2] spapr: Fix handling of unplugged devices during CAS and migration From: Greg Kurz To: David Gibson Date: Fri, 14 Feb 2020 16:01:28 +0100 In-Reply-To: <158169247578.3465937.4013536808417411649.stgit@bahia.lan> References: <158169247578.3465937.4013536808417411649.stgit@bahia.lan> User-Agent: StGit/unknown-version MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 x-cbid: 20021415-0012-0000-0000-00000386D649 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 20021415-0013-0000-0000-000021C35D55 Message-Id: <158169248798.3465937.1108351365840514270.stgit@bahia.lan> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-02-14_04:2020-02-12, 2020-02-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 lowpriorityscore=0 suspectscore=0 phishscore=0 adultscore=0 malwarescore=0 bulkscore=0 impostorscore=0 mlxscore=0 clxscore=1034 priorityscore=1501 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2001150001 definitions=main-2002140119 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Alexey Kardashevskiy , qemu-ppc@nongnu.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" We already detect if a device is being hot plugged before CAS to trigger a CAS reboot and during migration to migrate the state of the associated DRC. But hot unplugging a device is also an asynchronous operation that requires the guest to take action. This means that if the guest is migrated after the hot unplug event was sent but before it could release the device with RTAS, the destination QEMU doesn't know about the pending unplug operation and doesn't actually remove the device when the guest finally releases it. Similarly, if the unplug request is fired before CAS, the guest isn't notified of the change, just like with hotplug. It ends up booting with the device still present in the DT and configures it, just like it was never removed. Even weirder, since the event is still queued, it will be eventually processed when some other unrelated event is posted to the guest. Enhance spapr_drc_transient() to also return true if an unplug request is pending. This fixes the issue at CAS with a CAS reboot request and causes the DRC state to be migrated. Some extra care is still needed to inform the destination that an unplug request is pending : migrate the unplug_requested field of the DRC in an optional subsection. This might break backwards migration, but this is still better than ending with an inconsistent guest. Signed-off-by: Greg Kurz --- hw/ppc/spapr_drc.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index 4c35ce7c5c37..e373d342eb84 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -456,6 +456,22 @@ void spapr_drc_reset(SpaprDrc *drc) } } =20 +static bool spapr_drc_unplug_requested_needed(void *opaque) +{ + return spapr_drc_unplug_requested(opaque); +} + +static const VMStateDescription vmstate_spapr_drc_unplug_requested =3D { + .name =3D "spapr_drc/unplug_requested", + .version_id =3D 1, + .minimum_version_id =3D 1, + .needed =3D spapr_drc_unplug_requested_needed, + .fields =3D (VMStateField []) { + VMSTATE_BOOL(unplug_requested, SpaprDrc), + VMSTATE_END_OF_LIST() + } +}; + bool spapr_drc_transient(SpaprDrc *drc) { SpaprDrcClass *drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); @@ -471,9 +487,10 @@ bool spapr_drc_transient(SpaprDrc *drc) /* * We need to reset the DRC at CAS or to migrate the DRC state if it's * not equal to the expected long-term state, which is the same as the - * coldplugged initial state. + * coldplugged initial state, or if an unplug request is pending. */ - return (drc->state !=3D drck->ready_state); + return drc->state !=3D drck->ready_state || + spapr_drc_unplug_requested(drc); } =20 static bool spapr_drc_needed(void *opaque) @@ -489,6 +506,10 @@ static const VMStateDescription vmstate_spapr_drc =3D { .fields =3D (VMStateField []) { VMSTATE_UINT32(state, SpaprDrc), VMSTATE_END_OF_LIST() + }, + .subsections =3D (const VMStateDescription * []) { + &vmstate_spapr_drc_unplug_requested, + NULL } }; =20