From nobody Sun Mar 22 15:41:53 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1773920399; cv=none; d=zohomail.com; s=zohoarc; b=nDihe80K2pigmKKVPuey1lir69uhqhQ7wLZD2X8bnAweur7Hhnvt9uQ25Vhgd7dk1lvAMd7DgIXODWFRA9Lrc18H63r6uB+9XT/zsZEYWNeCiVpDSN9JJoRNk+ZE7onFAHuB19Yt+n/QFJrlw4GAEWw5j2+73bMZo6rBqlEbeoo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773920399; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TveFI/sDZqiQ3mrpO0u1dXilS/98mgQUpWeqgtqotqQ=; b=WfvRY5htJYvqI6O42LYTnRp6yHI1u4qvGsyhDgFGUxF8a9LOfTwqKL7j3I4KCFkg+oGbfDA7jBwQ41zlgEIFLVY1y0twi8/0/iJuYFcfcP0VimkHpGfY2vbiLT5yda3h2DomyrUS/MWzWCl5coTSJ8NhYUUC+cwIwfHVCvayaQc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773920399225611.6686287156842; Thu, 19 Mar 2026 04:39:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3Bif-0007K6-7h; Thu, 19 Mar 2026 07:39:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3Bib-0007Js-RO for qemu-devel@nongnu.org; Thu, 19 Mar 2026 07:39:14 -0400 Received: from va-1-112.ptr.blmpb.com ([209.127.230.112]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w3Bia-0004cR-2d for qemu-devel@nongnu.org; Thu, 19 Mar 2026 07:39:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1773920346; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=TveFI/sDZqiQ3mrpO0u1dXilS/98mgQUpWeqgtqotqQ=; b=ENXfe4z0cQx7yKjeFM2ayRbgBztO75YaAJzRVZiUfHZq52JeXuunVfrkoCG6cYC4GAwGIH nufwaOwV3z51LCFshFqkYqmHNeuHTd6SJcgOWui0aRICew23+HhMkC+fl6o996zuF5nxhZ y8fwRIX7kAmjJ32EWMWX/oFlVfsk8u0L3ZU/afUvVgMee94FNbmCejxl6HZuT/EHbsWVY4 XPRk5SyTMMzer2dlRiiFW8MDtFdyX8n4PRgq6equUXL3iYtxJLtPcN6rlcYkrwBItg5xg2 qc0qSXXC/zQ4IJp+wO6wi6hVSJ4403VzKdoNxoex9TMApjanJoc9KTaMW3bngw== Cc: , , , , "hongmianquan" From: "hongmianquan" Subject: [RFC v2 2/2] cpr: use hashtable for cpr fds Mime-Version: 1.0 In-Reply-To: <20260319113830.53867-1-hongmianquan@bytedance.com> To: References: <20260319113830.53867-1-hongmianquan@bytedance.com> X-Mailer: git-send-email 2.32.1 (Apple Git-133) Date: Thu, 19 Mar 2026 19:38:30 +0800 Message-Id: <20260319113830.53867-3-hongmianquan@bytedance.com> X-Lms-Return-Path: X-Original-From: hongmianquan Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=209.127.230.112; envelope-from=hongmianquan@bytedance.com; helo=va-1-112.ptr.blmpb.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1773920403568154100 Content-Type: text/plain; charset="utf-8" Use a hashtable to store cpr fds to reduce the time consumption of `cpr_find_fd` in scenarios with a large number of fds, which relies on the migration support for GHashTable implemented earlier. The time complexity for `cpr_find_fd` is reduced from O(N) to O(1). To demonstrate the performance improvement, we tested the total time consumed by `cpr_find_fd` (called N times for N fds) under our real-world business scenarios with different numbers of file descriptors. The results are measured in nanoseconds: | Number of FDs | Total time with QLIST (ns) | Total time with GHashTable (= ns) | |---------------|----------------------------|-----------------------------= ----| | 540 | 936,753 | 393,358 = | | 2,870 | 24,102,342 | 2,212,113 = | | 7,530 | 152,715,916 | 5,474,310 = | As shown in the data, the lookup time grows exponentially with the QLIST as the number of fds increases. With the GHashTable, the time consumption remains linear (O(1) per lookup), significantly reducing the downtime during the CPR process. Signed-off-by: hongmianquan --- include/migration/cpr.h | 4 +- migration/cpr.c | 129 ++++++++++++++++++++++++++++------------ system/vl.c | 2 + 3 files changed, 95 insertions(+), 40 deletions(-) diff --git a/include/migration/cpr.h b/include/migration/cpr.h index 5850fd1788..858b251bb5 100644 --- a/include/migration/cpr.h +++ b/include/migration/cpr.h @@ -18,16 +18,16 @@ #define QEMU_CPR_FILE_VERSION 0x00000001 #define CPR_STATE "CprState" =20 -typedef QLIST_HEAD(CprFdList, CprFd) CprFdList; typedef QLIST_HEAD(CprVFIODeviceList, CprVFIODevice) CprVFIODeviceList; =20 typedef struct CprState { - CprFdList fds; + GHashTable *fds; CprVFIODeviceList vfio_devices; } CprState; =20 extern CprState cpr_state; =20 +void cpr_state_init(void); void cpr_save_fd(const char *name, int id, int fd); void cpr_delete_fd(const char *name, int id); int cpr_find_fd(const char *name, int id); diff --git a/migration/cpr.c b/migration/cpr.c index a0b37007f5..cfdddb5043 100644 --- a/migration/cpr.c +++ b/migration/cpr.c @@ -27,59 +27,101 @@ CprState cpr_state; =20 /*************************************************************************= ***/ =20 -typedef struct CprFd { +typedef struct CprFdKey { char *name; unsigned int namelen; int id; +} CprFdKey; + +typedef struct CprFdVal { int fd; - QLIST_ENTRY(CprFd) next; -} CprFd; +} CprFdVal; + +#define VMSTATE_FDS_KEY \ +{ \ + .name =3D "cpr-fd-key", \ + .version_id =3D 1, \ + .minimum_version_id =3D 1, \ + .fields =3D (const VMStateField[]) { \ + VMSTATE_UINT32(namelen, CprFdKey), \ + VMSTATE_VBUFFER_ALLOC_UINT32(name, CprFdKey, 0, NULL, namelen), \ + VMSTATE_INT32(id, CprFdKey), \ + VMSTATE_END_OF_LIST() \ + } \ +} =20 -static const VMStateDescription vmstate_cpr_fd =3D { - .name =3D "cpr fd", - .version_id =3D 1, - .minimum_version_id =3D 1, - .fields =3D (VMStateField[]) { - VMSTATE_UINT32(namelen, CprFd), - VMSTATE_VBUFFER_ALLOC_UINT32(name, CprFd, 0, NULL, namelen), - VMSTATE_INT32(id, CprFd), - VMSTATE_FD(fd, CprFd), - VMSTATE_END_OF_LIST() - } +#define VMSTATE_FDS_VAL \ +{ \ + .name =3D "cpr-fd-value", \ + .version_id =3D 1, \ + .minimum_version_id =3D 1, \ + .fields =3D (const VMStateField[]) { \ + VMSTATE_FD(fd, CprFdVal), \ + VMSTATE_END_OF_LIST() \ + } \ +} + +static const VMStateDescription vmstate_fds_hashtable[2] =3D { + VMSTATE_FDS_VAL, /* value */ + VMSTATE_FDS_KEY /* key */ }; =20 +static guint cpr_fd_key_hash(gconstpointer v) +{ + const CprFdKey *key =3D v; + return g_str_hash(key->name) ^ key->id; +} + +static gboolean cpr_fd_key_equal(gconstpointer a, gconstpointer b) +{ + const CprFdKey *key_a =3D a; + const CprFdKey *key_b =3D b; + return !strcmp(key_a->name, key_b->name) && key_a->id =3D=3D key_b->id; +} + +static void cpr_destroy_fd_key(gpointer data) +{ + CprFdKey *k =3D (CprFdKey *) data; + g_free(k->name); + g_free(k); +} + +void cpr_state_init(void) +{ + CprState *s =3D &cpr_state; + + s->fds =3D g_hash_table_new_full(cpr_fd_key_hash, cpr_fd_key_equal, + cpr_destroy_fd_key, g_free); +} + void cpr_save_fd(const char *name, int id, int fd) { - CprFd *elem =3D g_new0(CprFd, 1); + CprFdKey *key =3D g_new0(CprFdKey, 1); + CprFdVal *val =3D g_new0(CprFdVal, 1); =20 trace_cpr_save_fd(name, id, fd); - elem->name =3D g_strdup(name); - elem->namelen =3D strlen(name) + 1; - elem->id =3D id; - elem->fd =3D fd; - QLIST_INSERT_HEAD(&cpr_state.fds, elem, next); + key->name =3D g_strdup(name); + key->namelen =3D strlen(name) + 1; + key->id =3D id; + val->fd =3D fd; + g_hash_table_insert(cpr_state.fds, key, val); } =20 -static CprFd *find_fd(CprFdList *head, const char *name, int id) +static CprFdVal *find_fd(CprFdKey *key) { - CprFd *elem; - - QLIST_FOREACH(elem, head, next) { - if (!strcmp(elem->name, name) && elem->id =3D=3D id) { - return elem; - } - } - return NULL; + return g_hash_table_lookup(cpr_state.fds, key); } =20 void cpr_delete_fd(const char *name, int id) { - CprFd *elem =3D find_fd(&cpr_state.fds, name, id); + CprFdKey key =3D { + .name =3D (char *)name, + .id =3D id, + }; + CprFdVal *elem =3D find_fd(&key); =20 if (elem) { - QLIST_REMOVE(elem, next); - g_free(elem->name); - g_free(elem); + g_hash_table_remove(cpr_state.fds, &key); } =20 trace_cpr_delete_fd(name, id); @@ -87,7 +129,11 @@ void cpr_delete_fd(const char *name, int id) =20 int cpr_find_fd(const char *name, int id) { - CprFd *elem =3D find_fd(&cpr_state.fds, name, id); + CprFdKey key =3D { + .name =3D (char *)name, + .id =3D id, + }; + CprFdVal *elem =3D find_fd(&key); int fd =3D elem ? elem->fd : -1; =20 trace_cpr_find_fd(name, id, fd); @@ -96,7 +142,11 @@ int cpr_find_fd(const char *name, int id) =20 void cpr_resave_fd(const char *name, int id, int fd) { - CprFd *elem =3D find_fd(&cpr_state.fds, name, id); + CprFdKey key =3D { + .name =3D (char *)name, + .id =3D id, + }; + CprFdVal *elem =3D find_fd(&key); int old_fd =3D elem ? elem->fd : -1; =20 if (old_fd < 0) { @@ -125,9 +175,11 @@ int cpr_open_fd(const char *path, int flags, const cha= r *name, int id, =20 bool cpr_walk_fd(cpr_walk_fd_cb cb) { - CprFd *elem; + GHashTableIter iter; + CprFdVal *elem; =20 - QLIST_FOREACH(elem, &cpr_state.fds, next) { + g_hash_table_iter_init(&iter, cpr_state.fds); + while (g_hash_table_iter_next(&iter, NULL, (gpointer *)&elem)) { g_assert(elem->fd >=3D 0); if (!cb(elem->fd)) { return false; @@ -142,7 +194,8 @@ static const VMStateDescription vmstate_cpr_state =3D { .version_id =3D 1, .minimum_version_id =3D 1, .fields =3D (VMStateField[]) { - VMSTATE_QLIST_V(fds, CprState, 1, vmstate_cpr_fd, CprFd, next), + VMSTATE_GHASH_V(fds, CprState, 1, vmstate_fds_hashtable, + CprFdKey, CprFdVal), VMSTATE_END_OF_LIST() }, .subsections =3D (const VMStateDescription * const []) { diff --git a/system/vl.c b/system/vl.c index 3e341142a0..da42de87db 100644 --- a/system/vl.c +++ b/system/vl.c @@ -2894,6 +2894,8 @@ void qemu_init(int argc, char **argv) =20 qemu_init_subsystems(); =20 + cpr_state_init(); + /* first pass of option parsing */ optind =3D 1; while (optind < argc) { --=20 2.32.1 (Apple Git-133)