From nobody Thu Apr 25 00:35:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 153096551915550.4320224607992; Sat, 7 Jul 2018 05:11:59 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0E1938EB4B; Sat, 7 Jul 2018 12:11:58 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BEC8733ECF; Sat, 7 Jul 2018 12:11:57 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 595463FB1D; Sat, 7 Jul 2018 12:11:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w67CBGTr005970 for ; Sat, 7 Jul 2018 08:11:16 -0400 Received: by smtp.corp.redhat.com (Postfix) id F000E6515C; Sat, 7 Jul 2018 12:11:15 +0000 (UTC) Received: from unknown4CEB42C824F4.redhat.com (ovpn-116-46.phx2.redhat.com [10.3.116.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id A99435C301 for ; Sat, 7 Jul 2018 12:11:15 +0000 (UTC) From: John Ferlan To: libvir-list@redhat.com Date: Sat, 7 Jul 2018 08:11:07 -0400 Message-Id: <20180707121107.7629-10-jferlan@redhat.com> In-Reply-To: <20180707121107.7629-1-jferlan@redhat.com> References: <20180707121107.7629-1-jferlan@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2 9/9] APPLY ONLY FOR TESTING PURPOSES X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Sat, 07 Jul 2018 12:11:58 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Modify GetAllStats to generate a long enough pause in order to send a SIGTERM to libvirtd while a client connection is processing. In order to "set things up": 1. In one terminal window from a local branch built using these patches using a root account run libvirtd debug, e.g.: # ./run gdb src/libvirtd once running, type a 'c' (e.g. continue) and 2. Start a domain (or have one running with the current libvirtd) virsh start $domain 3. Prepare a domstats command for that domain (but don't yet hit in order run it): virsh domstats $domain 4. Prepare a kill command for the running libvirtd, e.g.: jferlan 4143 4107 0 09:51 pts/1 00:00:00 vim +1396 src/libvirtd.c root 30054 21195 6 11:17 pts/8 00:00:01 gdb /home/jferlan/git/libvi= rt.work/src/.libs/lt-libvirtd root 30087 30054 7 11:17 pts/8 00:00:01 /home/jferlan/git/libvirt.w= ork/src/.libs/lt-libvirtd root 30385 19861 0 11:17 pts/17 00:00:00 grep --color=3Dauto libvirtd but again don't hit yet. 5. Align your stars perfectly now... a. Hit on your domstats command b. Swap to the kill command window and hit This should cause the libvirtd debug window to stop, but since you already typed 'c' it'll continue at least briefly, for example: ... [Thread 0x7fffc3231700 (LWP 30374) exited] Detaching after fork from child process 30376. Detaching after fork from child process 30377. Detaching after fork from child process 30378. [Thread 0x7fffc4445700 (LWP 30106) exited] c 2018-01-10 16:18:12.962+0000: 30094: info : libvirt version: 4.0.0 2018-01-10 16:18:12.962+0000: 30094: info : hostname: unknown4ceb42c824f4.a= ttlocal.net 2018-01-10 16:18:12.962+0000: 30094: warning : qemuConnectGetAllDomainStats= :20265 : k =3D -5340232226128654848 Thread 1 "lt-libvirtd" received signal SIGTERM, Terminated. 0x00007ffff3ae6d2d in poll () from /lib64/libc.so.6 ... (gdb) c Continuing. [Thread 0x7fffc5c48700 (LWP 30103) exited] [Thread 0x7fffc5447700 (LWP 30104) exited] [Thread 0x7fffc4c46700 (LWP 30105) exited] [Thread 0x7fffc6449700 (LWP 30102) exited] [Thread 0x7fffc6c4a700 (LWP 30101) exited] [Thread 0x7fffe3b57700 (LWP 30097) exited] [Thread 0x7fffe4358700 (LWP 30096) exited] [Thread 0x7fffe2354700 (LWP 30100) exited] [Thread 0x7fffe3356700 (LWP 30098) exited] [Thread 0x7fffe2b55700 (LWP 30099) exited] [Thread 0x7fffe535a700 (LWP 30094) exited] [Thread 0x7fffe5b5b700 (LWP 30093) exited] [Thread 0x7fffe635c700 (LWP 30092) exited] [Thread 0x7fffe6b5d700 (LWP 30091) exited] 2018-01-10 16:18:25.451+0000: 30095: warning : qemuConnectGetAllDomainStats= :20265 : k =3D -5340232226128654848 [Thread 0x7fffe4b59700 (LWP 30095) exited] [Thread 0x7fffc3a32700 (LWP 30187) exited] [Inferior 1 (process 30087) exited normally] (gdb) c The program is not being run. (gdb) quit The virsh domstats window will "close" as follows: error: Disconnected from qemu:///system due to end of file error: End of file while reading data: Input/output error If something's wrong, then the libvirtd window may not exit those final two threads in which case you could interrupt it (^c) and check the threads (thread apply all bt) which will probably show some sort of hang... My testing shows that the hang no longer occurs with all the previous patches applied. The subsequent patch calling virHashRemoveAll from virNetDaemonClose does not seem to be necessary, although I suppose it cannot hurt as the same essential functionality occurs during the Dispose function Signed-off-by: John Ferlan --- src/qemu/qemu_driver.c | 5 +++++ src/rpc/virnetdaemon.c | 3 ++- src/rpc/virnetserver.c | 4 ++-- src/util/virthreadpool.c | 10 +++++++--- 4 files changed, 16 insertions(+), 6 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 9a35e04a85..fa725131a2 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -20454,6 +20454,7 @@ qemuConnectGetAllDomainStats(virConnectPtr conn, bool enforce =3D !!(flags & VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_= STATS); int nstats =3D 0; size_t i; + size_t j, k =3D 0; int ret =3D -1; unsigned int privflags =3D 0; unsigned int domflags =3D 0; @@ -20492,6 +20493,10 @@ qemuConnectGetAllDomainStats(virConnectPtr conn, if (qemuDomainGetStatsNeedMonitor(stats)) privflags |=3D QEMU_DOMAIN_STATS_HAVE_JOB; =20 + for (j =3D 0; j < 10000000000; j++) // Add one more zero for longer... + k =3D j + k; + VIR_WARN("k =3D %zd", k); + for (i =3D 0; i < nvms; i++) { virDomainStatsRecordPtr tmp =3D NULL; domflags =3D 0; diff --git a/src/rpc/virnetdaemon.c b/src/rpc/virnetdaemon.c index c6ed65c8c3..6cce4b7004 100644 --- a/src/rpc/virnetdaemon.c +++ b/src/rpc/virnetdaemon.c @@ -834,7 +834,7 @@ virNetDaemonQuitTimer(int timer ATTRIBUTE_UNUSED, int *quitCount =3D opaque; =20 (*quitCount)++; - VIR_DEBUG("quitCount=3D%d", *quitCount); + VIR_WARN("quitCount=3D%d", *quitCount); } =20 =20 @@ -912,6 +912,7 @@ virNetDaemonRun(virNetDaemonPtr dmn) =20 if (dmn->quitRequested && daemonServerWorkersDone(dmn)) { dmn->quit =3D true; + VIR_WARN ("quitRequested and no workers remain"); } else { /* Firing every 1/2 second and quitTimeout in seconds, force * an exit when there are still worker threads running and we diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 053ef8a5ab..109f369bac 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -833,7 +833,7 @@ virNetServerQuitRequested(virNetServerPtr srv) if (!srv) return; =20 - VIR_DEBUG("Quit server requested '%s'", srv->name); + VIR_WARN ("Quit server requested '%s'", srv->name); =20 for (i =3D 0; i < srv->nservices; i++) virNetServerServiceToggle(srv->services[i], false); @@ -860,7 +860,7 @@ virNetServerWorkerCount(virNetServerPtr srv) virThreadPoolGetPriorityWorkers(srv->workers); =20 if (workerCount > 0) - VIR_DEBUG("server '%s' still has %zd workers", srv->name, workerCo= unt); + VIR_WARN ("server '%s' still has %zd workers", srv->name, workerCo= unt); =20 virObjectUnlock(srv); =20 diff --git a/src/util/virthreadpool.c b/src/util/virthreadpool.c index 137c5d1746..fc7bc64fb5 100644 --- a/src/util/virthreadpool.c +++ b/src/util/virthreadpool.c @@ -136,8 +136,10 @@ static void virThreadPoolWorker(void *opaque) goto out; } =20 - if (pool->quit) + if (pool->quit) { + VIR_WARN("Quit set"); break; + } =20 if (priority) { job =3D pool->jobList.firstPrio; @@ -330,7 +332,7 @@ virThreadPoolQuitRequested(virThreadPoolPtr pool) { virMutexLock(&pool->mutex); =20 - VIR_DEBUG("nWorkers=3D%zd, nPrioWorkers=3D%zd jobQueueDepth=3D%zd", + VIR_WARN ("nWorkers=3D%zd, nPrioWorkers=3D%zd jobQueueDepth=3D%zd", pool->nWorkers, pool->nPrioWorkers, pool->jobQueueDepth); =20 virThreadPoolSetQuit(pool); @@ -415,8 +417,10 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, virThreadPoolJobPtr job; =20 virMutexLock(&pool->mutex); - if (pool->quit) + if (pool->quit) { + VIR_WARN("Quit set"); goto error; + } =20 if (pool->freeWorkers - pool->jobQueueDepth <=3D 0 && pool->nWorkers < pool->maxWorkers && --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list