From nobody Sun Apr 28 00:00:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1504855775509560.7605136301173; Fri, 8 Sep 2017 00:29:35 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B17448008D; Fri, 8 Sep 2017 07:29:32 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8238F5D97B; Fri, 8 Sep 2017 07:29:31 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 886C21806104; Fri, 8 Sep 2017 07:29:26 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v887GuPt000869 for ; Fri, 8 Sep 2017 03:16:56 -0400 Received: by smtp.corp.redhat.com (Postfix) id BF78562930; Fri, 8 Sep 2017 07:16:56 +0000 (UTC) Received: from mx1.redhat.com (ext-mx06.extmail.prod.ext.phx2.redhat.com [10.5.110.30]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 64C2D669BD for ; Fri, 8 Sep 2017 07:16:54 +0000 (UTC) Received: from relay.sw.ru (mailhub.sw.ru [195.214.232.25]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CA142356D4 for ; Fri, 8 Sep 2017 07:16:50 +0000 (UTC) Received: from dim-vz7.qa.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v887GlS4015092 for ; Fri, 8 Sep 2017 10:16:48 +0300 (MSK) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B17448008D Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B17448008D DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com CA142356D4 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=pass (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=nshirokovskiy@virtuozzo.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com CA142356D4 From: Nikolay Shirokovskiy To: libvir-list@redhat.com Date: Fri, 8 Sep 2017 10:16:38 +0300 Message-Id: <1504854998-579788-1-git-send-email-nshirokovskiy@virtuozzo.com> X-Greylist: Delayed for 23:32:11 by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 08 Sep 2017 07:16:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 08 Sep 2017 07:16:53 +0000 (UTC) for IP:'195.214.232.25' DOMAIN:'mailhub.sw.ru' HELO:'relay.sw.ru' FROM:'nshirokovskiy@virtuozzo.com' RCPT:'' X-RedHat-Spam-Score: -0.001 (SPF_PASS) 195.214.232.25 mailhub.sw.ru 195.214.232.25 mailhub.sw.ru X-Scanned-By: MIMEDefang 2.78 on 10.5.110.30 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH] qemu: monitor: fix graceful shutdown corner cases X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 08 Sep 2017 07:29:34 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Patch aeda1b8c needs some enhancement. 1. Shutdown event is delivired on reboot too and we don't want to set willhangup flag is this case. 2. There is a next race condition. - EOF is delivered in event loop thread - qemuMonitorSend is called on client behalf and waits for notification on message response or monitor close - EOF handler tries to accquire job condition and fail on timeout as it is grabbed by the request above Now qemuMonitorSend hangs. Previously if qemuMonitorSend is called after EOF then it failed immediately due to lastError is set. Now we need to check willhangup too. --- Race is easy to trigger with patch [1]. One need to query domain frequently enough in a loop and do a shutdown. [1] Patch to trigger race condition. diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index b782451..4445f88 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -4742,6 +4742,9 @@ static void qemuProcessEventHandler(void *data, void = *opaque) VIR_DEBUG("vm=3D%p, event=3D%d", vm, processEvent->eventType); + if (processEvent->eventType =3D=3D QEMU_PROCESS_EVENT_MONITOR_EOF) + sleep(3); + virObjectLock(vm); switch (processEvent->eventType) { src/qemu/qemu_monitor.c | 16 +++++++++++++++- src/qemu/qemu_monitor.h | 2 ++ src/qemu/qemu_process.c | 1 + 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 19082d8..6270191 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -315,6 +315,15 @@ qemuMonitorEscapeNonPrintable(const char *text) #endif =20 =20 +void +qemuMonitorShutdown(qemuMonitorPtr mon) +{ + virObjectLock(mon); + mon->willhangup =3D 1; + virObjectUnlock(mon); +} + + static void qemuMonitorDispose(void *obj) { @@ -1055,6 +1064,12 @@ qemuMonitorSend(qemuMonitorPtr mon, return -1; } =20 + if (mon->willhangup) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("Domain is shutting down.")); + return -1; + } + mon->msg =3D msg; qemuMonitorUpdateWatch(mon); =20 @@ -1336,7 +1351,6 @@ qemuMonitorEmitShutdown(qemuMonitorPtr mon, virTrista= teBool guest) { int ret =3D -1; VIR_DEBUG("mon=3D%p guest=3D%u", mon, guest); - mon->willhangup =3D 1; =20 QEMU_MONITOR_CALLBACK(mon, ret, domainShutdown, mon->vm, guest); return ret; diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 9805a33..30533ef 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -303,6 +303,8 @@ void qemuMonitorUnregister(qemuMonitorPtr mon) ATTRIBUTE_NONNULL(1); void qemuMonitorClose(qemuMonitorPtr mon); =20 +void qemuMonitorShutdown(qemuMonitorPtr mon); + virErrorPtr qemuMonitorLastError(qemuMonitorPtr mon); =20 int qemuMonitorSetCapabilities(qemuMonitorPtr mon); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index ab81d65..824be86 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -618,6 +618,7 @@ qemuProcessShutdownOrReboot(virQEMUDriverPtr driver, virObjectUnref(vm); } } else { + qemuMonitorShutdown(priv->mon); ignore_value(qemuProcessKill(vm, VIR_QEMU_PROCESS_KILL_NOWAIT)); } } --=20 1.8.3.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list