From nobody Mon Dec 15 09:42:40 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1741800598352346.8237225777144; Wed, 12 Mar 2025 10:29:58 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id AD2691F65; Wed, 12 Mar 2025 13:29:57 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 1C1061AEF; Wed, 12 Mar 2025 13:26:45 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 102C31186; Wed, 12 Mar 2025 13:26:36 -0400 (EDT) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id AD47C1D2C for ; Wed, 12 Mar 2025 13:26:01 -0400 (EDT) Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-436-_ycbdU1IMK2h3Tg3r3nITw-1; Wed, 12 Mar 2025 13:25:58 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 795DD190E8DF for ; Wed, 12 Mar 2025 17:18:32 +0000 (UTC) Received: from toolbx.redhat.com (unknown [10.42.28.57]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 899901800945; Wed, 12 Mar 2025 17:18:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED,SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741800361; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JdDwgdEMrb1Lt6OgNf8XwJIJ3sFBBesk+ozKoxRTlTk=; b=GmaKdChnjyHw45YdLEbywaTPpRpOc4s5k+olRb5hSDCJa1+w5x34/U5s/RyQHo/mcme5pP gZVg4kzK9nfoA3a5cOzlJttmnICj2FB8MCD/Ivxk5LbcLukCHPD+M6FLgbukX/YYAiN+8E 1eyiittNdPnIUfmto+QEz51k6nWqqiA= X-MC-Unique: _ycbdU1IMK2h3Tg3r3nITw-1 X-Mimecast-MFC-AGG-ID: _ycbdU1IMK2h3Tg3r3nITw_1741800357 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: devel@lists.libvirt.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v2 17/22] rpc: move state stop into virNetDaemon class Date: Wed, 12 Mar 2025 17:17:57 +0000 Message-ID: <20250312171802.1854985-18-berrange@redhat.com> In-Reply-To: <20250312171802.1854985-1-berrange@redhat.com> References: <20250312171802.1854985-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: xBYVWdf5C5-Q-3q6Q_8NwoDHLavnqfAcr3XU4qIoLtY_1741800357 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Message-ID-Hash: 2NU4WJYSRM2PIXYVATAHNJ4GIBH4FIEL X-Message-ID-Hash: 2NU4WJYSRM2PIXYVATAHNJ4GIBH4FIEL X-MailFrom: berrange@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1741800600583019000 Content-Type: text/plain; charset="utf-8" Currently the remote daemon code is responsible for calling virStateStop in a background thread. The virNetDaemon code wants to synchronize with this during shutdown, however, so the virThreadPtr must be passed over. Even the limited synchronization done currently, however, is flawed and to fix this requires the virNetDaemon code to be responsible for calling virStateStop in a thread more directly. Thus the logic is moved over into virStateStop via a further callback to be registered by the remote daemon. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Peter Krempa --- src/libvirt_remote.syms | 2 +- src/remote/remote_daemon.c | 40 ++------------------------ src/rpc/virnetdaemon.c | 59 +++++++++++++++++++++++++++++++------- src/rpc/virnetdaemon.h | 27 +++++++++++++++-- 4 files changed, 77 insertions(+), 51 deletions(-) diff --git a/src/libvirt_remote.syms b/src/libvirt_remote.syms index c4053fdcb2..9dcc2792c1 100644 --- a/src/libvirt_remote.syms +++ b/src/libvirt_remote.syms @@ -95,7 +95,7 @@ virNetDaemonQuitExecRestart; virNetDaemonRemoveShutdownInhibition; virNetDaemonRun; virNetDaemonSetLifecycleCallbacks; -virNetDaemonSetStateStopWorkerThread; +virNetDaemonStop; virNetDaemonUpdateServices; =20 =20 diff --git a/src/remote/remote_daemon.c b/src/remote/remote_daemon.c index 9c0025a3bc..16dc1d86f7 100644 --- a/src/remote/remote_daemon.c +++ b/src/remote/remote_daemon.c @@ -514,41 +514,6 @@ static void daemonInhibitCallback(bool inhibit, void *= opaque) static GDBusConnection *sessionBus; static GDBusConnection *systemBus; =20 -static void daemonStopWorker(void *opaque) -{ - virNetDaemon *dmn =3D opaque; - - VIR_DEBUG("Begin stop dmn=3D%p", dmn); - - ignore_value(virStateStop()); - - VIR_DEBUG("Completed stop dmn=3D%p", dmn); - - /* Exit daemon cleanly */ - virNetDaemonQuit(dmn); -} - - -/* We do this in a thread to not block the main loop */ -static void daemonStop(virNetDaemon *dmn) -{ - virThread *thr; - virObjectRef(dmn); - - thr =3D g_new0(virThread, 1); - - if (virThreadCreateFull(thr, true, - daemonStopWorker, - "daemon-stop", false, dmn) < 0) { - virObjectUnref(dmn); - g_free(thr); - return; - } - - virNetDaemonSetStateStopWorkerThread(dmn, &thr); -} - - static GDBusMessage * handleSessionMessageFunc(GDBusConnection *connection G_GNUC_UNUSED, GDBusMessage *message, @@ -562,7 +527,7 @@ handleSessionMessageFunc(GDBusConnection *connection G_= GNUC_UNUSED, if (virGDBusMessageIsSignal(message, "org.freedesktop.DBus.Local", "Disconnected")) - daemonStop(dmn); + virNetDaemonStop(dmn); =20 return message; } @@ -581,7 +546,7 @@ handleSystemMessageFunc(GDBusConnection *connection G_G= NUC_UNUSED, =20 VIR_DEBUG("dmn=3D%p", dmn); =20 - daemonStop(dmn); + virNetDaemonStop(dmn); } =20 =20 @@ -625,6 +590,7 @@ static void daemonRunStateInit(void *opaque) g_atomic_int_set(&driversInitialized, 1); =20 virNetDaemonSetLifecycleCallbacks(dmn, + virStateStop, virStateShutdownPrepare, virStateShutdownWait); =20 diff --git a/src/rpc/virnetdaemon.c b/src/rpc/virnetdaemon.c index 6d20edf28b..43a75b330f 100644 --- a/src/rpc/virnetdaemon.c +++ b/src/rpc/virnetdaemon.c @@ -65,6 +65,7 @@ struct _virNetDaemon { GHashTable *servers; virJSONValue *srvObject; =20 + virNetDaemonLifecycleCallback stopCb; virNetDaemonLifecycleCallback shutdownPrepareCb; virNetDaemonLifecycleCallback shutdownWaitCb; virThread *stateStopThread; @@ -793,8 +794,10 @@ virNetDaemonRun(virNetDaemon *dmn) } } =20 + VIR_DEBUG("Main loop exited"); if (dmn->graceful) { virThreadJoin(&shutdownThread); + VIR_DEBUG("Graceful shutdown complete"); } else { VIR_WARN("Make forcefull daemon shutdown"); exit(EXIT_FAILURE); @@ -806,34 +809,61 @@ virNetDaemonRun(virNetDaemon *dmn) =20 =20 void -virNetDaemonSetStateStopWorkerThread(virNetDaemon *dmn, - virThread **thr) +virNetDaemonQuit(virNetDaemon *dmn) { VIR_LOCK_GUARD lock =3D virObjectLockGuard(dmn); =20 - VIR_DEBUG("Setting state stop worker thread on dmn=3D%p to thr=3D%p", = dmn, thr); - dmn->stateStopThread =3D g_steal_pointer(thr); + VIR_DEBUG("Quit requested %p", dmn); + dmn->quit =3D true; } =20 =20 void -virNetDaemonQuit(virNetDaemon *dmn) +virNetDaemonQuitExecRestart(virNetDaemon *dmn) { VIR_LOCK_GUARD lock =3D virObjectLockGuard(dmn); =20 - VIR_DEBUG("Quit requested %p", dmn); + VIR_DEBUG("Exec-restart requested %p", dmn); dmn->quit =3D true; + dmn->execRestart =3D true; +} + + +static void +virNetDaemonStopWorker(void *opaque) +{ + virNetDaemon *dmn =3D opaque; + + VIR_DEBUG("Begin stop dmn=3D%p", dmn); + + dmn->stopCb(); + + VIR_DEBUG("Completed stop dmn=3D%p", dmn); + + virNetDaemonQuit(dmn); + virObjectUnref(dmn); } =20 =20 void -virNetDaemonQuitExecRestart(virNetDaemon *dmn) +virNetDaemonStop(virNetDaemon *dmn) { VIR_LOCK_GUARD lock =3D virObjectLockGuard(dmn); =20 - VIR_DEBUG("Exec-restart requested %p", dmn); - dmn->quit =3D true; - dmn->execRestart =3D true; + if (!dmn->stopCb || + dmn->stateStopThread) + return; + + virObjectRef(dmn); + dmn->stateStopThread =3D g_new0(virThread, 1); + + if (virThreadCreateFull(dmn->stateStopThread, true, + virNetDaemonStopWorker, + "daemon-stop", false, dmn) < 0) { + virObjectUnref(dmn); + g_clear_pointer(&dmn->stateStopThread, g_free); + return; + } } =20 =20 @@ -874,11 +904,20 @@ virNetDaemonHasClients(virNetDaemon *dmn) =20 void virNetDaemonSetLifecycleCallbacks(virNetDaemon *dmn, + virNetDaemonLifecycleCallback stopCb, virNetDaemonLifecycleCallback prepareCb, virNetDaemonLifecycleCallback waitCb) { VIR_LOCK_GUARD lock =3D virObjectLockGuard(dmn); =20 + VIR_DEBUG("Lifecycle callbacks stop=3D%p prepare=3D%p wait=3D%p", + stopCb, prepareCb, waitCb); + + /* Immutable once set */ + if (dmn->stopCb || dmn->shutdownPrepareCb || dmn->shutdownWaitCb) + return; + + dmn->stopCb =3D stopCb; dmn->shutdownPrepareCb =3D prepareCb; dmn->shutdownWaitCb =3D waitCb; } diff --git a/src/rpc/virnetdaemon.h b/src/rpc/virnetdaemon.h index 0f4ebb6df7..68c95ade4a 100644 --- a/src/rpc/virnetdaemon.h +++ b/src/rpc/virnetdaemon.h @@ -66,13 +66,11 @@ int virNetDaemonAddSignalHandler(virNetDaemon *dmn, void virNetDaemonUpdateServices(virNetDaemon *dmn, bool enabled); =20 -void virNetDaemonSetStateStopWorkerThread(virNetDaemon *dmn, - virThread **thr); - void virNetDaemonRun(virNetDaemon *dmn); =20 void virNetDaemonQuit(virNetDaemon *dmn); void virNetDaemonQuitExecRestart(virNetDaemon *dmn); +void virNetDaemonStop(virNetDaemon *dmn); =20 bool virNetDaemonHasClients(virNetDaemon *dmn); =20 @@ -84,6 +82,29 @@ bool virNetDaemonHasServer(virNetDaemon *dmn, =20 typedef int (*virNetDaemonLifecycleCallback)(void); =20 +/* + * @stopCb: preserves any active state on host shutdown / session exit + * @prepareCb: start shutting down daemon + * @waitCb: wait for shutdown completion + * + * This method may only be invoked once, the callbacks are immutable + * once set. + * + * On host shutdown (privileged) or session exit (unprivileged) + * the @stopCb will be invoked first. + * + * When the daemon shuts down, the sequence of operations is + * as follows + * + * - Listener stops accepting new clients + * - Existing clients are closed + * - Delay until @stopCb is complete (if still running) + * - @prepareCb invoked + * - Server worker pool is drained in background + * - @waitCb is invoked in background + * - Main loop terminates + */ void virNetDaemonSetLifecycleCallbacks(virNetDaemon *dmn, + virNetDaemonLifecycleCallback stopC= b, virNetDaemonLifecycleCallback prepa= reCb, virNetDaemonLifecycleCallback waitC= b); --=20 2.48.1