From nobody Tue Jan 21 07:39:01 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 173636603008596.65348553768024; Wed, 8 Jan 2025 11:53:50 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 0AD8914BC; Wed, 8 Jan 2025 14:53:49 -0500 (EST) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 0926C14D3; Wed, 8 Jan 2025 14:44:17 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id D4DDA141D; Wed, 8 Jan 2025 14:43:58 -0500 (EST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 887291441 for ; Wed, 8 Jan 2025 14:43:30 -0500 (EST) Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-353-1bTpjjsDObStiYImB4kwqA-1; Wed, 08 Jan 2025 14:43:28 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 97FAD19560AE for ; Wed, 8 Jan 2025 19:43:27 +0000 (UTC) Received: from toolbx.redhat.com (unknown [10.42.28.103]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A53363000199; Wed, 8 Jan 2025 19:43:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, RCVD_IN_VALIDITY_RPBL_BLOCKED,RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1736365410; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XIjsi+pt2R/c+xb+tURrlNTgxaI8TSKAEqoQp1CzTjs=; b=OKJMrJo+du/RQIQnIK+WyuUYLq+N/Gh58e4+TBAwyAIfF7lRH/p3V06jrUMhhUOmcA+MrT IG/5Z833b/zQXcm5GqQN3ydLGqkYuXWd1vBYzexdLVSi8Gud74uZUhHCftlE7sObNQ9/Pt I5stpv+jsCRU3yqI1dy14uhc2Pn6Ik4= X-MC-Unique: 1bTpjjsDObStiYImB4kwqA-1 X-Mimecast-MFC-AGG-ID: 1bTpjjsDObStiYImB4kwqA From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: devel@lists.libvirt.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH 21/26] rpc: move state stop into virNetDaemon class Date: Wed, 8 Jan 2025 19:42:54 +0000 Message-ID: <20250108194259.1171990-22-berrange@redhat.com> In-Reply-To: <20250108194259.1171990-1-berrange@redhat.com> References: <20250108194259.1171990-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Uhpsn_vfBFlSE9eMKxy6f4E2Deo6_ZcHxHzXGBOBFLI_1736365407 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Message-ID-Hash: TFOJWLOQM2QWNVWEWFMDXDTQ5OCOWSHJ X-Message-ID-Hash: TFOJWLOQM2QWNVWEWFMDXDTQ5OCOWSHJ X-MailFrom: berrange@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1736366032488116600 Content-Type: text/plain; charset="utf-8" Currently the remote daemon code is responsible for calling virStateStop in a background thread. The virNetDaemon code wants to synchronize with this during shutdown, however, so the virThreadPtr must be passed over. Even the limited synchronization done currently, however, is flawed and to fix this requires the virNetDaemon code to be responsible for calling virStateStop in a thread more directly. Thus the logic is moved over into virStateStop via a further callback to be registered by the remote daemon. Signed-off-by: Daniel P. Berrang=C3=A9 --- src/libvirt_remote.syms | 2 +- src/remote/remote_daemon.c | 40 ++------------------------ src/rpc/virnetdaemon.c | 58 ++++++++++++++++++++++++++++---------- src/rpc/virnetdaemon.h | 20 +++++++++++-- 4 files changed, 64 insertions(+), 56 deletions(-) diff --git a/src/libvirt_remote.syms b/src/libvirt_remote.syms index f0f90815cf..7e87b0bd2a 100644 --- a/src/libvirt_remote.syms +++ b/src/libvirt_remote.syms @@ -90,12 +90,12 @@ virNetDaemonIsPrivileged; virNetDaemonNew; virNetDaemonNewPostExecRestart; virNetDaemonPreExecRestart; +virNetDaemonPreserve; virNetDaemonQuit; virNetDaemonQuitExecRestart; virNetDaemonRemoveShutdownInhibition; virNetDaemonRun; virNetDaemonSetShutdownCallbacks; -virNetDaemonSetStateStopWorkerThread; virNetDaemonUpdateServices; =20 =20 diff --git a/src/remote/remote_daemon.c b/src/remote/remote_daemon.c index c4b930cb70..bf91ee5772 100644 --- a/src/remote/remote_daemon.c +++ b/src/remote/remote_daemon.c @@ -514,41 +514,6 @@ static void daemonInhibitCallback(bool inhibit, void *= opaque) static GDBusConnection *sessionBus; static GDBusConnection *systemBus; =20 -static void daemonStopWorker(void *opaque) -{ - virNetDaemon *dmn =3D opaque; - - VIR_DEBUG("Begin stop dmn=3D%p", dmn); - - ignore_value(virStateStop()); - - VIR_DEBUG("Completed stop dmn=3D%p", dmn); - - /* Exit daemon cleanly */ - virNetDaemonQuit(dmn); -} - - -/* We do this in a thread to not block the main loop */ -static void daemonStop(virNetDaemon *dmn) -{ - virThread *thr; - virObjectRef(dmn); - - thr =3D g_new0(virThread, 1); - - if (virThreadCreateFull(thr, true, - daemonStopWorker, - "daemon-stop", false, dmn) < 0) { - virObjectUnref(dmn); - g_free(thr); - return; - } - - virNetDaemonSetStateStopWorkerThread(dmn, &thr); -} - - static GDBusMessage * handleSessionMessageFunc(GDBusConnection *connection G_GNUC_UNUSED, GDBusMessage *message, @@ -562,7 +527,7 @@ handleSessionMessageFunc(GDBusConnection *connection G_= GNUC_UNUSED, if (virGDBusMessageIsSignal(message, "org.freedesktop.DBus.Local", "Disconnected")) - daemonStop(dmn); + virNetDaemonPreserve(dmn); =20 return message; } @@ -581,7 +546,7 @@ handleSystemMessageFunc(GDBusConnection *connection G_G= NUC_UNUSED, =20 VIR_DEBUG("dmn=3D%p", dmn); =20 - daemonStop(dmn); + virNetDaemonPreserve(dmn); } =20 =20 @@ -625,6 +590,7 @@ static void daemonRunStateInit(void *opaque) g_atomic_int_set(&driversInitialized, 1); =20 virNetDaemonSetShutdownCallbacks(dmn, + virStateStop, virStateShutdownPrepare, virStateShutdownWait); =20 diff --git a/src/rpc/virnetdaemon.c b/src/rpc/virnetdaemon.c index bb7d2ff9a0..19b19ff834 100644 --- a/src/rpc/virnetdaemon.c +++ b/src/rpc/virnetdaemon.c @@ -65,9 +65,10 @@ struct _virNetDaemon { GHashTable *servers; virJSONValue *srvObject; =20 + virNetDaemonShutdownCallback shutdownPreserveCb; virNetDaemonShutdownCallback shutdownPrepareCb; virNetDaemonShutdownCallback shutdownWaitCb; - virThread *stateStopThread; + virThread *shutdownPreserveThread; int finishTimer; bool quit; bool finished; @@ -107,7 +108,7 @@ virNetDaemonDispose(void *obj) virEventRemoveHandle(dmn->sigwatch); #endif /* !WIN32 */ =20 - g_free(dmn->stateStopThread); + g_free(dmn->shutdownPreserveThread); =20 g_clear_pointer(&dmn->servers, g_hash_table_unref); =20 @@ -705,8 +706,8 @@ daemonShutdownWait(void *opaque) =20 virHashForEach(dmn->servers, daemonServerShutdownWait, NULL); if (!dmn->shutdownWaitCb || dmn->shutdownWaitCb() >=3D 0) { - if (dmn->stateStopThread) - virThreadJoin(dmn->stateStopThread); + if (dmn->shutdownPreserveThread) + virThreadJoin(dmn->shutdownPreserveThread); =20 graceful =3D true; } @@ -801,17 +802,6 @@ virNetDaemonRun(virNetDaemon *dmn) } =20 =20 -void -virNetDaemonSetStateStopWorkerThread(virNetDaemon *dmn, - virThread **thr) -{ - VIR_LOCK_GUARD lock =3D virObjectLockGuard(dmn); - - VIR_DEBUG("Setting state stop worker thread on dmn=3D%p to thr=3D%p", = dmn, thr); - dmn->stateStopThread =3D g_steal_pointer(thr); -} - - void virNetDaemonQuit(virNetDaemon *dmn) { @@ -833,6 +823,42 @@ virNetDaemonQuitExecRestart(virNetDaemon *dmn) } =20 =20 +static void virNetDaemonPreserveWorker(void *opaque) +{ + virNetDaemon *dmn =3D opaque; + + VIR_DEBUG("Begin stop dmn=3D%p", dmn); + + dmn->shutdownPreserveCb(); + + VIR_DEBUG("Completed stop dmn=3D%p", dmn); + + virNetDaemonQuit(dmn); + virObjectUnref(dmn); +} + + +void virNetDaemonPreserve(virNetDaemon *dmn) +{ + VIR_LOCK_GUARD lock =3D virObjectLockGuard(dmn); + + if (!dmn->shutdownPreserveCb || + dmn->shutdownPreserveThread) + return; + + virObjectRef(dmn); + dmn->shutdownPreserveThread =3D g_new0(virThread, 1); + + if (virThreadCreateFull(dmn->shutdownPreserveThread, true, + virNetDaemonPreserveWorker, + "daemon-stop", false, dmn) < 0) { + virObjectUnref(dmn); + g_clear_pointer(&dmn->shutdownPreserveThread, g_free); + return; + } +} + + static int daemonServerClose(void *payload, const char *key G_GNUC_UNUSED, @@ -870,11 +896,13 @@ virNetDaemonHasClients(virNetDaemon *dmn) =20 void virNetDaemonSetShutdownCallbacks(virNetDaemon *dmn, + virNetDaemonShutdownCallback preserveCb, virNetDaemonShutdownCallback prepareCb, virNetDaemonShutdownCallback waitCb) { VIR_LOCK_GUARD lock =3D virObjectLockGuard(dmn); =20 + dmn->shutdownPreserveCb =3D preserveCb; dmn->shutdownPrepareCb =3D prepareCb; dmn->shutdownWaitCb =3D waitCb; } diff --git a/src/rpc/virnetdaemon.h b/src/rpc/virnetdaemon.h index 31a355adb4..c64f3bdeb1 100644 --- a/src/rpc/virnetdaemon.h +++ b/src/rpc/virnetdaemon.h @@ -66,13 +66,11 @@ int virNetDaemonAddSignalHandler(virNetDaemon *dmn, void virNetDaemonUpdateServices(virNetDaemon *dmn, bool enabled); =20 -void virNetDaemonSetStateStopWorkerThread(virNetDaemon *dmn, - virThread **thr); - void virNetDaemonRun(virNetDaemon *dmn); =20 void virNetDaemonQuit(virNetDaemon *dmn); void virNetDaemonQuitExecRestart(virNetDaemon *dmn); +void virNetDaemonPreserve(virNetDaemon *dmn); =20 bool virNetDaemonHasClients(virNetDaemon *dmn); =20 @@ -84,6 +82,22 @@ bool virNetDaemonHasServer(virNetDaemon *dmn, =20 typedef int (*virNetDaemonShutdownCallback)(void); =20 +/* + * @preserveCb: preserves any active state + * @prepareCb: start shutting down daemon + * @waitCb: wait for shutdown completion + * + * The sequence of operations during shutdown is as follows + * + * - Listener stops accepting new clients + * - Existing clients are closed + * - Delay until @preserveCb is complete (if running) + * - @prepareCb invoked + * - Server worker pool is drained in background + * - @waitCb is invoked in background + * - Main loop terminates + */ void virNetDaemonSetShutdownCallbacks(virNetDaemon *dmn, + virNetDaemonShutdownCallback preserv= eCb, virNetDaemonShutdownCallback prepare= Cb, virNetDaemonShutdownCallback waitCb); --=20 2.47.1