From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243258; cv=none; d=zohomail.com; s=zohoarc; b=e3zeGmX2zhFqDjRbDZ+4jsSO9Vmzs249XlxO2JkxLQqJLPXY2ujnPMTiwxXO+Z9LJkTZO6IKM/Z+VkV5PUf8nrjyLCsEC9ITGl3Z/3An4OY0ya/dH20QJZWAIs4mZlbblutJ0qocYWeBbm2S7P2oJO9Y31Mg11VtWmTO9e1n09w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243258; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ExL2dc8zYuNhaMMvBjDBzq3S3sLX8sfaz1Y6DiMkFWc=; b=XAlKy696bR4nHmlM9FEZFuiLCzCmlY7HeWliYp3sy7u7HYAVxsPM9xPYHvBIiUmBmX7bsQ2AnjQWViMFN2o+1gLWqsY0RSjGlPpC5eHMj0TzCRt8vtA9bI7wjlt5SXUx6b8hG8MHXp5RFkTMEJCLm2krjqx8tt4fa9x5zv/0U6g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243258189136.61876821318197; Fri, 11 Jul 2025 07:14:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEVh-0006JY-FJ; Fri, 11 Jul 2025 10:14:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaET6-0000tQ-Df for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:18 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaESm-0006Zr-3L for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:10:58 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 3E6181F38C; Fri, 11 Jul 2025 14:10:38 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1B0C813918; Fri, 11 Jul 2025 14:10:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id aM2cMlsbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243039; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ExL2dc8zYuNhaMMvBjDBzq3S3sLX8sfaz1Y6DiMkFWc=; b=sFpRtZARql7guZ6k4VgIvWaJ5AULttD9Y1soLGM91w9n88I6WInexpp15hFbo/Cm/8mcPo uGYxFh3i1IfrECBICv4QWJGa9FRjB/xiwu3srdMD/QXI6kEc8dqYC5lW+mUBHhwiDhoM2A cJjIKtR4JYW0T4uVZjKfTqCNeZD8YB4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243039; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ExL2dc8zYuNhaMMvBjDBzq3S3sLX8sfaz1Y6DiMkFWc=; b=FYuT+4cvF8CypQgco7KbxA0vcDCwD2CiepMvJRI9LTjiVLp1IIvfuu7mrWeMhI9T0lIp73 nSZsc8adqvN7rqCw== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=Ejs8cmDi; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=ji1kMhJJ DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243038; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ExL2dc8zYuNhaMMvBjDBzq3S3sLX8sfaz1Y6DiMkFWc=; b=Ejs8cmDipsYbq4sORTTTJRxd8snhBzVpcYXRbJD9Plye8Hu51O3UBVcFsBmx1J59aSKSye lsCOHCs736c8BFMv09q6Xc5h2hQCrtcWorb3FjhOPUqbbG38KjM6yC75js5vpa6EWrdxAk UBMMo5CWm7zwAZ4bH/JI3E7rx2gGwjU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243038; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ExL2dc8zYuNhaMMvBjDBzq3S3sLX8sfaz1Y6DiMkFWc=; b=ji1kMhJJ6VO6U1mH/TUz4cgsgKxaYgNj0ZdGxZB1XGlz+8VmSrmuivk7CCJ7Klx42k08sp VGusom/zNN56cpAA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Dr . David Alan Gilbert" , Li Zhijian , Juraj Marcin , Mario Casquero Subject: [PULL 01/26] migration/hmp: Reorg "info migrate" once more Date: Fri, 11 Jul 2025 11:10:06 -0300 Message-Id: <20250711141031.423-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-5.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; DWL_DNSWL_MED(-2.00)[suse.de:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,fujitsu.com:email,treblig.org:email,suse.de:mid,suse.de:dkim,suse.de:email]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 3E6181F38C X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -5.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) (identity @suse.de) X-ZM-MESSAGEID: 1752243258915116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Dave suggested the HMP output for "info migrate" can not only leverage the lines but also better grouping: https://lore.kernel.org/r/aC4_-nMc7FwsMf9p@gallifrey I followed Dave's suggestion, and some more modifications on top: - Added all elements into the picture - Use size_to_str() and drop most of the units: benefit is more friendly to most human eyes, bad side effect is lose of details, but that should be corner case per my uses, and one can still leverage the QMP interface when necessary. - Sub-grouping for "Transfers" ("Channels" and "Page Types"). - Better indentations Sample output: (qemu) info migrate Status: postcopy-active Time (ms): total=3D47317, setup=3D5, down=3D8 RAM info: Throughput (Mbps): 1342.83 Sizes: pagesize=3D4 KiB, total=3D4.02 GiB Transfers: transferred=3D1.41 GiB, remain=3D2.46 GiB Channels: precopy=3D15.2 MiB, multifd=3D0 B, postcopy=3D1.39 = GiB Page Types: normal=3D367713, zero=3D41195 Page Rates (pps): transfer=3D40900, dirty=3D4 Others: dirty_syncs=3D2, postcopy_req=3D57503 Suggested-by: Dr. David Alan Gilbert Tested-by: Li Zhijian Reviewed-by: Li Zhijian Acked-by: Dr. David Alan Gilbert Reviewed-by: Juraj Marcin Tested-by: Mario Casquero Link: https://lore.kernel.org/r/20250613140801.474264-2-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration-hmp-cmds.c | 59 ++++++++++++++++++---------------- 1 file changed, 31 insertions(+), 28 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index e8a563c7d8..367ff6037f 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -69,7 +69,7 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) } =20 if (info->has_status) { - monitor_printf(mon, "Status: %s", + monitor_printf(mon, "Status: \t\t%s", MigrationStatus_str(info->status)); if (info->status =3D=3D MIGRATION_STATUS_FAILED && info->error_des= c) { monitor_printf(mon, " (%s)\n", info->error_desc); @@ -78,7 +78,7 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) } =20 if (info->total_time) { - monitor_printf(mon, "Time (ms): total=3D%" PRIu64, + monitor_printf(mon, "Time (ms): \t\ttotal=3D%" PRIu64, info->total_time); if (info->has_setup_time) { monitor_printf(mon, ", setup=3D%" PRIu64, @@ -110,48 +110,51 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdic= t) } =20 if (info->ram) { + g_autofree char *str_psize =3D size_to_str(info->ram->page_size); + g_autofree char *str_total =3D size_to_str(info->ram->total); + g_autofree char *str_transferred =3D size_to_str(info->ram->transf= erred); + g_autofree char *str_remaining =3D size_to_str(info->ram->remainin= g); + g_autofree char *str_precopy =3D size_to_str(info->ram->precopy_by= tes); + g_autofree char *str_multifd =3D size_to_str(info->ram->multifd_by= tes); + g_autofree char *str_postcopy =3D size_to_str(info->ram->postcopy_= bytes); + monitor_printf(mon, "RAM info:\n"); - monitor_printf(mon, " Throughput (Mbps): %0.2f\n", + monitor_printf(mon, " Throughput (Mbps): \t%0.2f\n", info->ram->mbps); - monitor_printf(mon, " Sizes (KiB): pagesize=3D%" PRIu64 - ", total=3D%" PRIu64 ",\n", - info->ram->page_size >> 10, - info->ram->total >> 10); - monitor_printf(mon, " transferred=3D%" PRIu64 - ", remain=3D%" PRIu64 ",\n", - info->ram->transferred >> 10, - info->ram->remaining >> 10); - monitor_printf(mon, " precopy=3D%" PRIu64 - ", multifd=3D%" PRIu64 - ", postcopy=3D%" PRIu64, - info->ram->precopy_bytes >> 10, - info->ram->multifd_bytes >> 10, - info->ram->postcopy_bytes >> 10); + monitor_printf(mon, " Sizes: \t\tpagesize=3D%s, total=3D%s\n", + str_psize, str_total); + monitor_printf(mon, " Transfers: \t\ttransferred=3D%s, remain=3D%= s\n", + str_transferred, str_remaining); + monitor_printf(mon, " Channels: \t\tprecopy=3D%s, " + "multifd=3D%s, postcopy=3D%s", + str_precopy, str_multifd, str_postcopy); =20 if (info->vfio) { - monitor_printf(mon, ", vfio=3D%" PRIu64, - info->vfio->transferred >> 10); + g_autofree char *str_vfio =3D size_to_str(info->vfio->transfer= red); + + monitor_printf(mon, ", vfio=3D%s", str_vfio); } monitor_printf(mon, "\n"); =20 - monitor_printf(mon, " Pages: normal=3D%" PRIu64 ", zero=3D%" PRIu= 64 - ", rate_per_sec=3D%" PRIu64 "\n", - info->ram->normal, - info->ram->duplicate, + monitor_printf(mon, " Page Types: \tnormal=3D%" PRIu64 + ", zero=3D%" PRIu64 "\n", + info->ram->normal, info->ram->duplicate); + monitor_printf(mon, " Page Rates (pps): \ttransfer=3D%" PRIu64, info->ram->pages_per_second); - monitor_printf(mon, " Others: dirty_syncs=3D%" PRIu64, - info->ram->dirty_sync_count); - if (info->ram->dirty_pages_rate) { - monitor_printf(mon, ", dirty_pages_rate=3D%" PRIu64, + monitor_printf(mon, ", dirty=3D%" PRIu64, info->ram->dirty_pages_rate); } + monitor_printf(mon, "\n"); + + monitor_printf(mon, " Others: \t\tdirty_syncs=3D%" PRIu64, + info->ram->dirty_sync_count); if (info->ram->postcopy_requests) { monitor_printf(mon, ", postcopy_req=3D%" PRIu64, info->ram->postcopy_requests); } if (info->ram->downtime_bytes) { - monitor_printf(mon, ", downtime_ram=3D%" PRIu64, + monitor_printf(mon, ", downtime_bytes=3D%" PRIu64, info->ram->downtime_bytes); } if (info->ram->dirty_sync_missed_zero_copy) { --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243196; cv=none; d=zohomail.com; s=zohoarc; b=Vh0FNkSCZ7LS613VmGBupv434h1qMVQx8h7Z+L+sH/LuXoeLMmPv8gkTH4cmyHFnEKZYl2Ugg+a78OCMV+UdYiO3iWMPBaBmDcH1Nbwuy7i7S3D1iFM5LSs+6jt0qI5qGeqqmqX5Yy+rpQqFCzyuBf7S3eEmUf7RmmGUfRiQO3E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243196; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=l0u6vcmpKguZljP7Odh29cS/hGK5fEOpnoNZOACCmqg=; b=NMOfZYsEjPq1pKL4abjYRdE06kNO2ebCh3cFF1CX9VVFfcuPUwsq1hOnCeVBICCgYHjoQ1vG74lXBehepauvuAC+WHSNtAPOfHmLcvVmp4pwj5B9eBbVj/00WtofqU/SJKaPUdCAYfK1hwzU1ws/98OP9NOmu6WSPoFoQeWSYm8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243191459837.7852897277768; Fri, 11 Jul 2025 07:13:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEUs-0005Gt-QV; Fri, 11 Jul 2025 10:13:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaESh-0000pP-7W for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:10:59 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaESc-0006Vt-Ey for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:10:49 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 3061121190; Fri, 11 Jul 2025 14:10:41 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BF99D1388B; Fri, 11 Jul 2025 14:10:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YJ+RHl4bcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:38 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243041; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l0u6vcmpKguZljP7Odh29cS/hGK5fEOpnoNZOACCmqg=; b=BvZwGXZGIjezfKNn4epcUqMB1GUwcwl6IG5hhKFw60MR63IWDulSDvBuRgTXY85RVVVvD8 w2LFXem8Ak23oB/bbmpmPMhSm8M2vy9+1iRL2AQthBCbuXzTxQ341cj3aL/FT6+MeB9/Og 7kcJBnHBCpxMY8ThixDus9vG6HuDLL0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243041; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l0u6vcmpKguZljP7Odh29cS/hGK5fEOpnoNZOACCmqg=; b=CJkf6kYxq9nvfIjDoZxOQmpXpZj/1n660pqRQpdgdtekuPe8wH+7rAizDBr5V5n4XNV+fu oYvXJB38cf9seEBw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=BvZwGXZG; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=CJkf6kYx DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243041; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l0u6vcmpKguZljP7Odh29cS/hGK5fEOpnoNZOACCmqg=; b=BvZwGXZGIjezfKNn4epcUqMB1GUwcwl6IG5hhKFw60MR63IWDulSDvBuRgTXY85RVVVvD8 w2LFXem8Ak23oB/bbmpmPMhSm8M2vy9+1iRL2AQthBCbuXzTxQ341cj3aL/FT6+MeB9/Og 7kcJBnHBCpxMY8ThixDus9vG6HuDLL0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243041; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l0u6vcmpKguZljP7Odh29cS/hGK5fEOpnoNZOACCmqg=; b=CJkf6kYxq9nvfIjDoZxOQmpXpZj/1n660pqRQpdgdtekuPe8wH+7rAizDBr5V5n4XNV+fu oYvXJB38cf9seEBw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Dr . David Alan Gilbert" , Alexey Perevalov , Markus Armbruster , Mario Casquero , Juraj Marcin Subject: [PULL 02/26] migration/hmp: Fix postcopy-blocktime per-vCPU results Date: Fri, 11 Jul 2025 11:10:07 -0300 Message-Id: <20250711141031.423-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-5.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; DWL_DNSWL_MED(-2.00)[suse.de:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:mid,suse.de:dkim,suse.de:email,treblig.org:email,samsung.com:email]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_SEVEN(0.00)[7]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 3061121190 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -5.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243197730116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Unfortunately, it was never correctly shown.. This is only found when I started to look into making the blocktime feature more useful (so as to avoid using bpftrace, even though I'm not sure which one will be harder to use..). So the old dump would look like this: Postcopy vCPU Blocktime: 0-1,4,10,21,33,46,48,59 Even though there're actually 40 vcpus, and the string will merge same elements and also sort them. To fix it, simply loop over the uint32List manually. Now it looks like: Postcopy vCPU Blocktime (ms): [15, 0, 0, 43, 29, 34, 36, 29, 37, 41, 33, 37, 45, 52, 50, 38, 40, 37, 40, 49, 40, 35, 35, 35, 81, 19, 18, 19, 18, 30, 22, 3, 0, 0, 0, 0, 0, 0, 0, 0] Cc: Dr. David Alan Gilbert Cc: Alexey Perevalov Cc: Markus Armbruster Tested-by: Mario Casquero Reviewed-by: Juraj Marcin Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613140801.474264-3-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration-hmp-cmds.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 367ff6037f..867e017b32 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -208,15 +208,19 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdic= t) } =20 if (info->has_postcopy_vcpu_blocktime) { - Visitor *v; - char *str; - v =3D string_output_visitor_new(false, &str); - visit_type_uint32List(v, NULL, &info->postcopy_vcpu_blocktime, - &error_abort); - visit_complete(v, &str); - monitor_printf(mon, "Postcopy vCPU Blocktime: %s\n", str); - g_free(str); - visit_free(v); + uint32List *item =3D info->postcopy_vcpu_blocktime; + const char *sep =3D ""; + int count =3D 0; + + monitor_printf(mon, "Postcopy vCPU Blocktime (ms):\n ["); + + while (item) { + monitor_printf(mon, "%s%"PRIu32, sep, item->value); + item =3D item->next; + /* Each line 10 vcpu results, newline if there's more */ + sep =3D ((++count % 10 =3D=3D 0) && item) ? ",\n " : ", "; + } + monitor_printf(mon, "]\n"); } =20 out: --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243250; cv=none; d=zohomail.com; s=zohoarc; b=ejOvuie2FeveyEDeGjpz9GERUP0L92OTcou4kTI4L/V445oE3latHnnjD6h9N6l4igqwthduRvhHuyHfbgB+6+gEo08LRdwI1FrYgWp427EXC+f4biC5Bby5A5MiYlqnidQnu7RmOb1c0T0hvpvIwOCzYurlxeSZKYDcz1CL2ms= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243250; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Afq2KYMYYL/Thf/uD8cq+0jXlmbxqDVWTWRLkkwZEnM=; b=nYHG+IfJglQzCgM+juOvzyg4HnGR0dMFIihrKFXh65JguYMUqUOonOyl2vwpJPnfbylhBtVzgHdpCMxKz5yb1xOC88gRfR+MU4RA5DxWyHZtfJdkAe6cLyf++Gi6wM8eY8I/BE0qs22h1MWaIcgV9JISpD+Iq1BC5AhSuvSKujE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243250190707.7752323582414; Fri, 11 Jul 2025 07:14:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEVA-0005dM-Af; Fri, 11 Jul 2025 10:13:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaET9-0001Mv-0I for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:19 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaESn-0006Zt-70 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:10:58 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0D67F21192; Fri, 11 Jul 2025 14:10:43 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id AD6241388B; Fri, 11 Jul 2025 14:10:41 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id qJ68GmEbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243043; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Afq2KYMYYL/Thf/uD8cq+0jXlmbxqDVWTWRLkkwZEnM=; b=X1u1Vh/J2gGNhySd3aEJflSbRBprM7qRp4crKBPz/I5dBWXcIxJqmhblJ5XRsFGrN6GVOr V6w07HMWA7nUdFqFkGUucjGNFgAC36yztw3euW9yAAiLvkWzt9zGtLQHmTlNqoi95rftwB T/OVinYRHdLM4rrj16Wb6dqXa5bbJHg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243043; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Afq2KYMYYL/Thf/uD8cq+0jXlmbxqDVWTWRLkkwZEnM=; b=pcVwHQmnIZFugjh8AxQ6kZj6wEEi3Odm5S7o7SbezsjdCqVVIJUfnZAgxe5vaGUfNg2vic A/CNUR0+IawchTAg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243043; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Afq2KYMYYL/Thf/uD8cq+0jXlmbxqDVWTWRLkkwZEnM=; b=X1u1Vh/J2gGNhySd3aEJflSbRBprM7qRp4crKBPz/I5dBWXcIxJqmhblJ5XRsFGrN6GVOr V6w07HMWA7nUdFqFkGUucjGNFgAC36yztw3euW9yAAiLvkWzt9zGtLQHmTlNqoi95rftwB T/OVinYRHdLM4rrj16Wb6dqXa5bbJHg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243043; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Afq2KYMYYL/Thf/uD8cq+0jXlmbxqDVWTWRLkkwZEnM=; b=pcVwHQmnIZFugjh8AxQ6kZj6wEEi3Odm5S7o7SbezsjdCqVVIJUfnZAgxe5vaGUfNg2vic A/CNUR0+IawchTAg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Juraj Marcin Subject: [PULL 03/26] migration/docs: Move docs for postcopy blocktime feature Date: Fri, 11 Jul 2025 11:10:08 -0300 Message-Id: <20250711141031.423-4-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:email,imap1.dmz-prg2.suse.org:helo]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FUZZY_RATELIMITED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_THREE(0.00)[3]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243252834116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Move it out of vanilla postcopy session, but instead a standalone feature. When at it, removing the NOTE because it's incorrect now after introduction of max-postcopy-bandwidth, which can control the throughput even for postcopy phase. Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20250613140801.474264-4-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- docs/devel/migration/postcopy.rst | 36 +++++++++++++++---------------- 1 file changed, 17 insertions(+), 19 deletions(-) diff --git a/docs/devel/migration/postcopy.rst b/docs/devel/migration/postc= opy.rst index 82e7a848c6..e319388d8f 100644 --- a/docs/devel/migration/postcopy.rst +++ b/docs/devel/migration/postcopy.rst @@ -33,25 +33,6 @@ will now cause the transition from precopy to postcopy. It can be issued immediately after migration is started or any time later on. Issuing it after the end of a migration is harmless. =20 -Blocktime is a postcopy live migration metric, intended to show how -long the vCPU was in state of interruptible sleep due to pagefault. -That metric is calculated both for all vCPUs as overlapped value, and -separately for each vCPU. These values are calculated on destination -side. To enable postcopy blocktime calculation, enter following -command on destination monitor: - -``migrate_set_capability postcopy-blocktime on`` - -Postcopy blocktime can be retrieved by query-migrate qmp command. -postcopy-blocktime value of qmp command will show overlapped blocking -time for all vCPU, postcopy-vcpu-blocktime will show list of blocking -time per vCPU. - -.. note:: - During the postcopy phase, the bandwidth limits set using - ``migrate_set_parameter`` is ignored (to avoid delaying requested pages = that - the destination is waiting for). - Postcopy internals =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 @@ -312,3 +293,20 @@ explicitly) to be sent in a separate preempt channel, = rather than queued in the background migration channel. Anyone who cares about latencies of page faults during a postcopy migration should enable this feature. By default, it's not enabled. + +Postcopy blocktime statistics +----------------------------- + +Blocktime is a postcopy live migration metric, intended to show how +long the vCPU was in state of interruptible sleep due to pagefault. +That metric is calculated both for all vCPUs as overlapped value, and +separately for each vCPU. These values are calculated on destination +side. To enable postcopy blocktime calculation, enter following +command on destination monitor: + +``migrate_set_capability postcopy-blocktime on`` + +Postcopy blocktime can be retrieved by query-migrate qmp command. +postcopy-blocktime value of qmp command will show overlapped blocking +time for all vCPU, postcopy-vcpu-blocktime will show list of blocking +time per vCPU. --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243198; cv=none; d=zohomail.com; s=zohoarc; b=TaCAzneyXICLjrmfdyFBHDwqTuQgwrBMhoMqga5Teib+8ozcpHv/A7PgKV1dwOD08NSim76+crAWn93YXHRECEyuMfbrCUcgUBNRjzborUwlElaBNX5qgw85/lZq1mgbvzXRqSWBNsUbmxNZrWLq5/kcXRE2fr+hlYuwfGU+CjU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243198; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9Z704TiT4EebcL/tSRSM977okr+TvK6D7Ri2DJ2tH/A=; b=guDEBSCEeclaioUK+dCbRngt28Vm250tXnOdsNPbc8XCU2qZ+ubedptlYRF9jYJw5C8XvlBuryR5f9zpot5260QOxYNW0Xi7QmhwfDsam3hyyCxS6+wtBdDRmBY8vcng/F/NeqiU6Z4+BighF1k6wOlsJHCFwiOWP6zH28kAh20= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243198125640.7058521902393; Fri, 11 Jul 2025 07:13:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEUt-0005KY-Lf; Fri, 11 Jul 2025 10:13:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaET8-0001Mw-Vn for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:19 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaESs-0006bm-KO for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:04 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E392C2119D; Fri, 11 Jul 2025 14:10:44 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 917721388B; Fri, 11 Jul 2025 14:10:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id +PMuEmMbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243044; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Z704TiT4EebcL/tSRSM977okr+TvK6D7Ri2DJ2tH/A=; b=j7VHw5kPQdj78sRjxmaLh03158oz//lAFoIcx0ZUbcfglCtSJVajnZcBegCR2YrO85Z3xF rKTI+gm3rHYa7Q9a4xgqK5SKOtjIQfERS1/dLyVpkdhxbA450/v8UPQ9RPILpKCzm/dElH rJVhmZ49o/luBoWNWha2T1Sq6ZZzBRg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243044; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Z704TiT4EebcL/tSRSM977okr+TvK6D7Ri2DJ2tH/A=; b=8fvfPdfp2QEhzYW6dh8wd1BcS6zUjcAKsC3T1niYZtPpCc6G+7rlI6ADb7H1HT1Vd3WODC 7iBOc/i9OHo1gBBw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=j7VHw5kP; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=8fvfPdfp DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243044; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Z704TiT4EebcL/tSRSM977okr+TvK6D7Ri2DJ2tH/A=; b=j7VHw5kPQdj78sRjxmaLh03158oz//lAFoIcx0ZUbcfglCtSJVajnZcBegCR2YrO85Z3xF rKTI+gm3rHYa7Q9a4xgqK5SKOtjIQfERS1/dLyVpkdhxbA450/v8UPQ9RPILpKCzm/dElH rJVhmZ49o/luBoWNWha2T1Sq6ZZzBRg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243044; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Z704TiT4EebcL/tSRSM977okr+TvK6D7Ri2DJ2tH/A=; b=8fvfPdfp2QEhzYW6dh8wd1BcS6zUjcAKsC3T1niYZtPpCc6G+7rlI6ADb7H1HT1Vd3WODC 7iBOc/i9OHo1gBBw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Juraj Marcin Subject: [PULL 04/26] migration/bg-snapshot: Do not check for SKIP in iterator Date: Fri, 11 Jul 2025 11:10:09 -0300 Message-Id: <20250711141031.423-5-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: E392C2119D X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo,suse.de:dkim,suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243203719116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu It's not possible to happen in bg-snapshot case. Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20250613140801.474264-5-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 4098870bce..e33e39ac74 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3887,9 +3887,8 @@ static void *bg_migration_thread(void *opaque) =20 while (migration_is_active()) { MigIterateState iter_state =3D bg_migration_iteration_run(s); - if (iter_state =3D=3D MIG_ITERATE_SKIP) { - continue; - } else if (iter_state =3D=3D MIG_ITERATE_BREAK) { + + if (iter_state =3D=3D MIG_ITERATE_BREAK) { break; } =20 --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243279; cv=none; d=zohomail.com; s=zohoarc; b=lysd3eg4XZaQ8qj3b9Q0lEu0j1Ugb/IXvsJ8c0S9Os93AZGPgqRSohCg7Qs2HfeF0OZkW9bRt47/MMPDyjCH+jAK3Dl1uQp10l+srq0E61FB0MhI66Ltp5NlV+EYb/82M57zCRMLV9gS+WwtgYNM8A+4itAJoV6WqkanUe54t0s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243279; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=BVi/ud188k7DZ8kqR4sTOhVMrCU6s+gJeBw/KUDtN5o=; b=INKsiCK0J1lyIixL5tUb8msmL2LK+d+txTYlsPXpJkg8ljWSOtPIy/hDFieCYHbx58Qk93X4gYeNmkV47QmKOpgMfRoliEJt0EyIXHV+3VxJr3yecP8CusjOEflxTuUPX7PSujLv/MLLNeTSM4+ihPNFdSls8SOfjkaBXaoTmtE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243279394108.00461190307601; Fri, 11 Jul 2025 07:14:39 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEVo-0006h5-2u; Fri, 11 Jul 2025 10:14:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaET9-0001N0-0I for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:19 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaESu-0006by-RS for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:06 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id E1FB11F451; Fri, 11 Jul 2025 14:10:46 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6FE6C1388B; Fri, 11 Jul 2025 14:10:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id AKicC2UbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243046; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BVi/ud188k7DZ8kqR4sTOhVMrCU6s+gJeBw/KUDtN5o=; b=WhHAONjqcqTeQ2luBJQjV9UbYFi7ubfWHfr6HPB73XwTkJ59X5wF69bhJjkjD3QGI8Rb1q QGAfBNOjORpiH6CveTC7KYjeXCb6odgnw9sJvyMXc3PNJVsQeEi2v496A9BfskBkOEKd/4 cRf7OVxBg6ntKz84n/kiJuQS16wtcC4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243046; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BVi/ud188k7DZ8kqR4sTOhVMrCU6s+gJeBw/KUDtN5o=; b=HXiWR2L59NiTHLScH2pBTJFldHFdDQleKSqqm6nVTzGjtJzagExkR7jlgjF9M9r54kunLZ XhOYcpsx2EP7kNDg== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=WhHAONjq; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=HXiWR2L5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243046; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BVi/ud188k7DZ8kqR4sTOhVMrCU6s+gJeBw/KUDtN5o=; b=WhHAONjqcqTeQ2luBJQjV9UbYFi7ubfWHfr6HPB73XwTkJ59X5wF69bhJjkjD3QGI8Rb1q QGAfBNOjORpiH6CveTC7KYjeXCb6odgnw9sJvyMXc3PNJVsQeEi2v496A9BfskBkOEKd/4 cRf7OVxBg6ntKz84n/kiJuQS16wtcC4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243046; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BVi/ud188k7DZ8kqR4sTOhVMrCU6s+gJeBw/KUDtN5o=; b=HXiWR2L59NiTHLScH2pBTJFldHFdDQleKSqqm6nVTzGjtJzagExkR7jlgjF9M9r54kunLZ XhOYcpsx2EP7kNDg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Juraj Marcin Subject: [PULL 05/26] migration: Drop save_live_complete_postcopy hook Date: Fri, 11 Jul 2025 11:10:10 -0300 Message-Id: <20250711141031.423-6-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-5.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; DWL_DNSWL_MED(-2.00)[suse.de:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: E1FB11F451 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -5.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243281010116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu The hook is only defined in two vmstate users ("ram" and "block dirty bitmap"), meanwhile both of them define the hook exactly the same as the precopy version. Hence, this postcopy version isn't needed. No functional change intended. Reviewed-by: Juraj Marcin Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613140801.474264-6-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- include/migration/register.h | 24 ++++++++---------------- migration/block-dirty-bitmap.c | 1 - migration/ram.c | 1 - migration/savevm.c | 9 ++++----- 4 files changed, 12 insertions(+), 23 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index b79dc81b8d..e022195785 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -77,26 +77,18 @@ typedef struct SaveVMHandlers { */ void (*save_cleanup)(void *opaque); =20 - /** - * @save_live_complete_postcopy - * - * Called at the end of postcopy for all postcopyable devices. - * - * @f: QEMUFile where to send the data - * @opaque: data pointer passed to register_savevm_live() - * - * Returns zero to indicate success and negative for error - */ - int (*save_live_complete_postcopy)(QEMUFile *f, void *opaque); - /** * @save_live_complete_precopy * * Transmits the last section for the device containing any - * remaining data at the end of a precopy phase. When postcopy is - * enabled, devices that support postcopy will skip this step, - * where the final data will be flushed at the end of postcopy via - * @save_live_complete_postcopy instead. + * remaining data at the end phase of migration. + * + * For precopy, this will be invoked _during_ the switchover phase + * after source VM is stopped. + * + * For postcopy, this will be invoked _after_ the switchover phase + * (except some very unusual cases, like PMEM ramblocks), while + * destination VM can be running. * * @f: QEMUFile where to send the data * @opaque: data pointer passed to register_savevm_live() diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index f2c352d4a7..6ee3c32a76 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -1248,7 +1248,6 @@ static bool dirty_bitmap_has_postcopy(void *opaque) =20 static SaveVMHandlers savevm_dirty_bitmap_handlers =3D { .save_setup =3D dirty_bitmap_save_setup, - .save_live_complete_postcopy =3D dirty_bitmap_save_complete, .save_live_complete_precopy =3D dirty_bitmap_save_complete, .has_postcopy =3D dirty_bitmap_has_postcopy, .state_pending_exact =3D dirty_bitmap_state_pending, diff --git a/migration/ram.c b/migration/ram.c index 2140785a05..6bb4f13031 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4548,7 +4548,6 @@ void postcopy_preempt_shutdown_file(MigrationState *s) static SaveVMHandlers savevm_ram_handlers =3D { .save_setup =3D ram_save_setup, .save_live_iterate =3D ram_save_iterate, - .save_live_complete_postcopy =3D ram_save_complete, .save_live_complete_precopy =3D ram_save_complete, .has_postcopy =3D ram_has_postcopy, .state_pending_exact =3D ram_state_pending_exact, diff --git a/migration/savevm.c b/migration/savevm.c index bb04a4520d..3e20f8608a 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1485,9 +1485,8 @@ bool should_send_vmdesc(void) } =20 /* - * Calls the save_live_complete_postcopy methods - * causing the last few pages to be sent immediately and doing any associa= ted - * cleanup. + * Complete saving any postcopy-able devices. + * * Note postcopy also calls qemu_savevm_state_complete_precopy to complete * all the other devices, but that happens at the point we switch to postc= opy. */ @@ -1497,7 +1496,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f) int ret; =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->save_live_complete_postcopy) { + if (!se->ops || !se->ops->save_live_complete_precopy) { continue; } if (se->ops->is_active) { @@ -1510,7 +1509,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f) qemu_put_byte(f, QEMU_VM_SECTION_END); qemu_put_be32(f, se->section_id); =20 - ret =3D se->ops->save_live_complete_postcopy(f, se->opaque); + ret =3D se->ops->save_live_complete_precopy(f, se->opaque); trace_savevm_section_end(se->idstr, se->section_id, ret); save_section_footer(f, se); if (ret < 0) { --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243509; cv=none; d=zohomail.com; s=zohoarc; b=L0WhzssWkx/PlX8Oce1GxsTkljg72SfLabakwyUhZFsA1Meocs7qzk2c2RWVZh+eCPXZDCwR9wiYPizTACXf5cjN42E/CSGMKrM90Y2On7pEVzqt03uan9Uzoo/pnu7GQc+s+qbWtlw8R7fvGDhiGCrmJTI9IHR+ZZKBH5AO/BI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243509; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9xjV3uDk/KQiqyg6+/FlJeG/5UFVlpXxOHhas+J0JmQ=; b=UuRM1EAxQDiEFl2sSzcJbWTsJ/bzoCl1V5/oMLsyEM6k896MS6bY1lFC7LH4skTuyszDB1zVuLKi+NPvsYSf4Qv6prnD6dDT56begmqgoQmCtMGEMSgDb7kNadtjgEXzg1+LwjlFqUK1bOCpaG2fx6uJUfBcWr+5eCayjNnUgDI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243509854752.7891838541416; Fri, 11 Jul 2025 07:18:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEWb-0000IJ-Ih; Fri, 11 Jul 2025 10:14:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaET9-0001N3-0e for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:19 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaET1-0006e6-Kg for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:13 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id E13061F453; Fri, 11 Jul 2025 14:10:48 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 829A41388B; Fri, 11 Jul 2025 14:10:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id aDYlD2cbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243048; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9xjV3uDk/KQiqyg6+/FlJeG/5UFVlpXxOHhas+J0JmQ=; b=zMYnS1rqtz1mi5I35zQK1Gdfg0ZN/F01bIle7VOR7rUI9qlu2Y6ouTQ4n4XV8yl4+H4I5T 2ocRT4KvfZlZgKeur5LzcLM/pna74jCIFVuHfkuiZwVlCbPnZFvkhSm/Il1DsTuVKlNJxA PRC8flLeouhIRCN9BG1+rDaHwuOPRE8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243048; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9xjV3uDk/KQiqyg6+/FlJeG/5UFVlpXxOHhas+J0JmQ=; b=lmtDCOkcZ7vlFmZbXQKOJDkXONTm+5aKglOOYMEgNPOk0aXHSDTSi0eG3LjLNAu5cWw1tO ZSEtMcojEHfQFGDA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243048; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9xjV3uDk/KQiqyg6+/FlJeG/5UFVlpXxOHhas+J0JmQ=; b=zMYnS1rqtz1mi5I35zQK1Gdfg0ZN/F01bIle7VOR7rUI9qlu2Y6ouTQ4n4XV8yl4+H4I5T 2ocRT4KvfZlZgKeur5LzcLM/pna74jCIFVuHfkuiZwVlCbPnZFvkhSm/Il1DsTuVKlNJxA PRC8flLeouhIRCN9BG1+rDaHwuOPRE8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243048; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9xjV3uDk/KQiqyg6+/FlJeG/5UFVlpXxOHhas+J0JmQ=; b=lmtDCOkcZ7vlFmZbXQKOJDkXONTm+5aKglOOYMEgNPOk0aXHSDTSi0eG3LjLNAu5cWw1tO ZSEtMcojEHfQFGDA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Juraj Marcin Subject: [PULL 06/26] migration: Rename save_live_complete_precopy to save_complete Date: Fri, 11 Jul 2025 11:10:11 -0300 Message-Id: <20250711141031.423-7-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:mid,suse.de:email]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FUZZY_RATELIMITED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_THREE(0.00)[3]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243512172116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Now after merging the precopy and postcopy version of complete() hook, rename the precopy version from save_live_complete_precopy() to save_complete(). Dropping the "live" when at it, because it's in most cases not live when happening (in precopy). No functional change intended. Reviewed-by: Juraj Marcin Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613140801.474264-7-peterx@redhat.com [peterx: squash the fixup that covers a few more doc spots, per Juraj] Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- docs/devel/migration/main.rst | 4 ++-- docs/devel/migration/vfio.rst | 12 ++++++------ hw/ppc/spapr.c | 2 +- hw/s390x/s390-stattrib.c | 2 +- hw/vfio/migration.c | 2 +- include/migration/register.h | 6 +++--- migration/block-dirty-bitmap.c | 2 +- migration/ram.c | 2 +- migration/savevm.c | 8 ++++---- 9 files changed, 20 insertions(+), 20 deletions(-) diff --git a/docs/devel/migration/main.rst b/docs/devel/migration/main.rst index cdd4f4a6d7..6493c1d2bc 100644 --- a/docs/devel/migration/main.rst +++ b/docs/devel/migration/main.rst @@ -508,8 +508,8 @@ An iterative device must provide: the point that stream bandwidth limits tell it to stop. Each call generates one section. =20 - - A ``save_live_complete_precopy`` function that must transmit the - last section for the device containing any remaining data. + - A ``save_complete`` function that must transmit the last section for + the device containing any remaining data. =20 - A ``load_state`` function used to load sections generated by any of the save functions that generate sections. diff --git a/docs/devel/migration/vfio.rst b/docs/devel/migration/vfio.rst index 673e354754..8ff5ab0c74 100644 --- a/docs/devel/migration/vfio.rst +++ b/docs/devel/migration/vfio.rst @@ -75,10 +75,10 @@ VFIO implements the device hooks for the iterative appr= oach as follows: in the non-multifd mode. In the multifd mode it just emits either a dummy EOS marker. =20 -* A ``save_live_complete_precopy`` function that sets the VFIO device in - _STOP_COPY state and iteratively copies the data for the VFIO device unt= il - the vendor driver indicates that no data remains. - In the multifd mode it just emits a dummy EOS marker. +* A ``save_complete`` function that sets the VFIO device in _STOP_COPY + state and iteratively copies the data for the VFIO device until the + vendor driver indicates that no data remains. In the multifd mode it + just emits a dummy EOS marker. =20 * A ``save_live_complete_precopy_thread`` function that in the multifd mode provides thread handler performing multifd device state transfer. @@ -195,9 +195,9 @@ Live migration save path | Then the VFIO device is put in _STOP_COPY state (FINISH_MIGRATE, _ACTIVE, _STOP_COPY) - .save_live_complete_precopy() is called for each active device + .save_complete() is called for each active device For the VFIO device: in the non-multifd mode iterate in - .save_live_complete_precopy() until + .save_complete() until pending data is 0 In the multifd mode this iteration is done in .save_live_complete_precopy_thread() instead. diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 08615f6c90..40f53ad7b3 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2518,7 +2518,7 @@ static void htab_save_cleanup(void *opaque) static SaveVMHandlers savevm_htab_handlers =3D { .save_setup =3D htab_save_setup, .save_live_iterate =3D htab_save_iterate, - .save_live_complete_precopy =3D htab_save_complete, + .save_complete =3D htab_save_complete, .save_cleanup =3D htab_save_cleanup, .load_state =3D htab_load, }; diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index f74cf32636..13a678a803 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -338,7 +338,7 @@ static const TypeInfo qemu_s390_stattrib_info =3D { static SaveVMHandlers savevm_s390_stattrib_handlers =3D { .save_setup =3D cmma_save_setup, .save_live_iterate =3D cmma_save_iterate, - .save_live_complete_precopy =3D cmma_save_complete, + .save_complete =3D cmma_save_complete, .state_pending_exact =3D cmma_state_pending, .state_pending_estimate =3D cmma_state_pending, .save_cleanup =3D cmma_save_cleanup, diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index b76697bd1a..33a71f8999 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -824,7 +824,7 @@ static const SaveVMHandlers savevm_vfio_handlers =3D { .state_pending_exact =3D vfio_state_pending_exact, .is_active_iterate =3D vfio_is_active_iterate, .save_live_iterate =3D vfio_save_iterate, - .save_live_complete_precopy =3D vfio_save_complete_precopy, + .save_complete =3D vfio_save_complete_precopy, .save_state =3D vfio_save_state, .load_setup =3D vfio_load_setup, .load_cleanup =3D vfio_load_cleanup, diff --git a/include/migration/register.h b/include/migration/register.h index e022195785..2a26e76a68 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -78,7 +78,7 @@ typedef struct SaveVMHandlers { void (*save_cleanup)(void *opaque); =20 /** - * @save_live_complete_precopy + * @save_complete * * Transmits the last section for the device containing any * remaining data at the end phase of migration. @@ -95,7 +95,7 @@ typedef struct SaveVMHandlers { * * Returns zero to indicate success and negative for error */ - int (*save_live_complete_precopy)(QEMUFile *f, void *opaque); + int (*save_complete)(QEMUFile *f, void *opaque); =20 /** * @save_live_complete_precopy_thread (invoked in a separate thread) @@ -103,7 +103,7 @@ typedef struct SaveVMHandlers { * Called at the end of a precopy phase from a separate worker thread * in configurations where multifd device state transfer is supported * in order to perform asynchronous transmission of the remaining data= in - * parallel with @save_live_complete_precopy handlers. + * parallel with @save_complete handlers. * When postcopy is enabled, devices that support postcopy will skip t= his * step. * diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 6ee3c32a76..a061aad817 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -1248,7 +1248,7 @@ static bool dirty_bitmap_has_postcopy(void *opaque) =20 static SaveVMHandlers savevm_dirty_bitmap_handlers =3D { .save_setup =3D dirty_bitmap_save_setup, - .save_live_complete_precopy =3D dirty_bitmap_save_complete, + .save_complete =3D dirty_bitmap_save_complete, .has_postcopy =3D dirty_bitmap_has_postcopy, .state_pending_exact =3D dirty_bitmap_state_pending, .state_pending_estimate =3D dirty_bitmap_state_pending, diff --git a/migration/ram.c b/migration/ram.c index 6bb4f13031..ffd839f57c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4548,7 +4548,7 @@ void postcopy_preempt_shutdown_file(MigrationState *s) static SaveVMHandlers savevm_ram_handlers =3D { .save_setup =3D ram_save_setup, .save_live_iterate =3D ram_save_iterate, - .save_live_complete_precopy =3D ram_save_complete, + .save_complete =3D ram_save_complete, .has_postcopy =3D ram_has_postcopy, .state_pending_exact =3D ram_state_pending_exact, .state_pending_estimate =3D ram_state_pending_estimate, diff --git a/migration/savevm.c b/migration/savevm.c index 3e20f8608a..454e914b56 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1496,7 +1496,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f) int ret; =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->save_live_complete_precopy) { + if (!se->ops || !se->ops->save_complete) { continue; } if (se->ops->is_active) { @@ -1509,7 +1509,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f) qemu_put_byte(f, QEMU_VM_SECTION_END); qemu_put_be32(f, se->section_id); =20 - ret =3D se->ops->save_live_complete_precopy(f, se->opaque); + ret =3D se->ops->save_complete(f, se->opaque); trace_savevm_section_end(se->idstr, se->section_id, ret); save_section_footer(f, se); if (ret < 0) { @@ -1583,7 +1583,7 @@ int qemu_savevm_state_complete_precopy_iterable(QEMUF= ile *f, bool in_postcopy) if (!se->ops || (in_postcopy && se->ops->has_postcopy && se->ops->has_postcopy(se->opaque)) || - !se->ops->save_live_complete_precopy) { + !se->ops->save_complete) { continue; } =20 @@ -1598,7 +1598,7 @@ int qemu_savevm_state_complete_precopy_iterable(QEMUF= ile *f, bool in_postcopy) =20 save_section_header(f, se, QEMU_VM_SECTION_END); =20 - ret =3D se->ops->save_live_complete_precopy(f, se->opaque); + ret =3D se->ops->save_complete(f, se->opaque); trace_savevm_section_end(se->idstr, se->section_id, ret); save_section_footer(f, se); if (ret < 0) { --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243191; cv=none; d=zohomail.com; s=zohoarc; b=fb7Jk8G7qp86bshMVseEV0exgwxfdmZStES21CxMPhX4PiRaIT+b5kj8blSv4+sMDpQzAlFeQvUkJS7PX91xC0OmNv8waBvY+FiUtbGSONUyi/IDo/Q7kroKKCwb8s/OYpXTWsIf9QbOYhw7Va0oAw2s7dbyL1MDoIU9b5WTwBM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243191; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=VRaYifWhkJOcWBKaUF8iE1A2dKiwWY17wzieTwSmTFA=; b=PfrGxHAjRqJ4nR0v9AFJUWUXItwjAzVI+/4WDjlxaj1HRHS4fBiXegLeoCwiQVcgxLSr23kuzo3hzs3ARG7woCxjkFtj2NvVyxOv8S6OejolsPv9xdytlNgo9WjqhbVnJr5ooUaDL50a8ifYEdCT4ZOguEvAsxdZ5KpRU9eI/B4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243191904852.9205305183959; Fri, 11 Jul 2025 07:13:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEUr-0005FK-Ss; Fri, 11 Jul 2025 10:13:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaET8-0001N1-WA for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:19 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaET0-0006e3-7B for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:12 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C3EE3211A0; Fri, 11 Jul 2025 14:10:50 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 696B41388B; Fri, 11 Jul 2025 14:10:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YDbiCWkbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:49 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243050; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VRaYifWhkJOcWBKaUF8iE1A2dKiwWY17wzieTwSmTFA=; b=AWXYLsX0ahCWYGNpNRMGrvYsiXeH7Kn2NLogsLc3Tmdlk6RoubCBkGTIV1sYcd/1fVNTpy U2lMHJQvEHd0JiAi7CrERxYprxWlu7C5rJrsjjVno6VdSFH1HKFhsXZ5EQIaNf1D8Aa4PQ QbxotNBSFsu3np6mZQhRQVQTtaMZ5HE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243050; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VRaYifWhkJOcWBKaUF8iE1A2dKiwWY17wzieTwSmTFA=; b=ZblZV7MSJ6ofZt9MEsBx+Yoh41iK/0G/D21gWUjuJthed2XucuQSj/GDhWrP3ksgbIt48i ZyQKHbkC7xB34YCA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=AWXYLsX0; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=ZblZV7MS DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243050; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VRaYifWhkJOcWBKaUF8iE1A2dKiwWY17wzieTwSmTFA=; b=AWXYLsX0ahCWYGNpNRMGrvYsiXeH7Kn2NLogsLc3Tmdlk6RoubCBkGTIV1sYcd/1fVNTpy U2lMHJQvEHd0JiAi7CrERxYprxWlu7C5rJrsjjVno6VdSFH1HKFhsXZ5EQIaNf1D8Aa4PQ QbxotNBSFsu3np6mZQhRQVQTtaMZ5HE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243050; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VRaYifWhkJOcWBKaUF8iE1A2dKiwWY17wzieTwSmTFA=; b=ZblZV7MSJ6ofZt9MEsBx+Yoh41iK/0G/D21gWUjuJthed2XucuQSj/GDhWrP3ksgbIt48i ZyQKHbkC7xB34YCA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Juraj Marcin Subject: [PULL 07/26] migration: qemu_savevm_complete*() helpers Date: Fri, 11 Jul 2025 11:10:12 -0300 Message-Id: <20250711141031.423-8-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-5.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; DWL_DNSWL_MED(-2.00)[suse.de:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:mid,suse.de:dkim,suse.de:email]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: C3EE3211A0 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -5.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243193865116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Since we use the same save_complete() hook for both precopy and postcopy, add a set of helpers to invoke the hook() to dedup the code. Reviewed-by: Juraj Marcin Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613140801.474264-8-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/savevm.c | 78 ++++++++++++++++++++++++++-------------------- 1 file changed, 44 insertions(+), 34 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index 454e914b56..c4fd5f5a5b 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1484,6 +1484,38 @@ bool should_send_vmdesc(void) return !machine->suppress_vmdesc; } =20 +static bool qemu_savevm_complete_exists(SaveStateEntry *se) +{ + return se->ops && se->ops->save_complete; +} + +/* + * Invoke the ->save_complete() if necessary. + * Returns: 0 if skip the current SE or succeeded, <0 if error happened. + */ +static int qemu_savevm_complete(SaveStateEntry *se, QEMUFile *f) +{ + int ret; + + if (se->ops->is_active) { + if (!se->ops->is_active(se->opaque)) { + return 0; + } + } + + trace_savevm_section_start(se->idstr, se->section_id); + save_section_header(f, se, QEMU_VM_SECTION_END); + ret =3D se->ops->save_complete(f, se->opaque); + trace_savevm_section_end(se->idstr, se->section_id, ret); + save_section_footer(f, se); + + if (ret < 0) { + qemu_file_set_error(f, ret); + } + + return ret; +} + /* * Complete saving any postcopy-able devices. * @@ -1493,27 +1525,13 @@ bool should_send_vmdesc(void) void qemu_savevm_state_complete_postcopy(QEMUFile *f) { SaveStateEntry *se; - int ret; =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->save_complete) { + if (!qemu_savevm_complete_exists(se)) { continue; } - if (se->ops->is_active) { - if (!se->ops->is_active(se->opaque)) { - continue; - } - } - trace_savevm_section_start(se->idstr, se->section_id); - /* Section type */ - qemu_put_byte(f, QEMU_VM_SECTION_END); - qemu_put_be32(f, se->section_id); =20 - ret =3D se->ops->save_complete(f, se->opaque); - trace_savevm_section_end(se->idstr, se->section_id, ret); - save_section_footer(f, se); - if (ret < 0) { - qemu_file_set_error(f, ret); + if (qemu_savevm_complete(se, f) < 0) { return; } } @@ -1559,7 +1577,6 @@ int qemu_savevm_state_complete_precopy_iterable(QEMUF= ile *f, bool in_postcopy) { int64_t start_ts_each, end_ts_each; SaveStateEntry *se; - int ret; bool multifd_device_state =3D multifd_device_state_supported(); =20 if (multifd_device_state) { @@ -1580,32 +1597,25 @@ int qemu_savevm_state_complete_precopy_iterable(QEM= UFile *f, bool in_postcopy) } =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || - (in_postcopy && se->ops->has_postcopy && - se->ops->has_postcopy(se->opaque)) || - !se->ops->save_complete) { + if (!qemu_savevm_complete_exists(se)) { continue; } =20 - if (se->ops->is_active) { - if (!se->ops->is_active(se->opaque)) { - continue; - } + if (in_postcopy && se->ops->has_postcopy && + se->ops->has_postcopy(se->opaque)) { + /* + * If postcopy will start soon, and if the SE supports + * postcopy, then we can skip the SE for the postcopy phase. + */ + continue; } =20 start_ts_each =3D qemu_clock_get_us(QEMU_CLOCK_REALTIME); - trace_savevm_section_start(se->idstr, se->section_id); - - save_section_header(f, se, QEMU_VM_SECTION_END); - - ret =3D se->ops->save_complete(f, se->opaque); - trace_savevm_section_end(se->idstr, se->section_id, ret); - save_section_footer(f, se); - if (ret < 0) { - qemu_file_set_error(f, ret); + if (qemu_savevm_complete(se, f) < 0) { goto ret_fail_abort_threads; } end_ts_each =3D qemu_clock_get_us(QEMU_CLOCK_REALTIME); + trace_vmstate_downtime_save("iterable", se->idstr, se->instance_id, end_ts_each - start_ts_each); } --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243513; cv=none; d=zohomail.com; s=zohoarc; b=k7SAc3442jE65btZ68/WbUkUkaSPVYPe3dAcmpRgD27UBAfypx0EvwrptxQe8UvL9/YCnM5Ek/i2/cnzbMwa3YcZJm2WXFi2pP1FcTZ3dmzUUhLKjhfmHuAmKLqMxfykLUdSeDqtqKNvs2WTY0wZoLpd0wzCOBs/hQqyM4AcCLI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243513; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6LEnV05GAdxAxtoxmlpP5GQRyBBCmk1hTN15A8D+Gww=; b=l/jWDN9qOc74onk9/BAfAiYXZYt12woRMVpg7i+hsNwFKUnMToHyXHi3Ia3kkbFrgmHs1e5usD2VgEs+Gi6Rt0yait5mZuLuPguYQri/WQuqc2PEOjrxZ5D1X7hxCC1RK4xYzx3Ev/g0a5IrT8uUbIcvI1o1A8TwB8dnQCmDiH0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243513681159.84566796427282; Fri, 11 Jul 2025 07:18:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEZU-0003i5-98; Fri, 11 Jul 2025 10:17:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETB-0001XR-Dh for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:21 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaET9-0006fZ-5I for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:20 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 9FB571F385; Fri, 11 Jul 2025 14:10:52 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4DE0A1388B; Fri, 11 Jul 2025 14:10:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id aNsTA2sbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:51 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243052; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6LEnV05GAdxAxtoxmlpP5GQRyBBCmk1hTN15A8D+Gww=; b=TxAGTiwCKMxs7XInMYI/5qUiu7F+eI/Yuq2wj7grWoXoA/bCmBcbfCOeFFHzJLng36Olb0 HAVkKIHA8DgLLOLrd+TEU1jARW5WZJldd+SODR7pTKUMAvORTOwxDTcYOozlGDJIyhXdVm H8AhWCAahDxeg9ao6u4Sx+jHrOhXWBg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243052; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6LEnV05GAdxAxtoxmlpP5GQRyBBCmk1hTN15A8D+Gww=; b=MSKg/sTW2GiuhDDYqApINJ5rwd4JXCG9I/lMXaku9qFj5+XrQkyZ1A41bLfwJOflpU+IK6 VG9KJEuUfX0IzuDw== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TxAGTiwC; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="MSKg/sTW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243052; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6LEnV05GAdxAxtoxmlpP5GQRyBBCmk1hTN15A8D+Gww=; b=TxAGTiwCKMxs7XInMYI/5qUiu7F+eI/Yuq2wj7grWoXoA/bCmBcbfCOeFFHzJLng36Olb0 HAVkKIHA8DgLLOLrd+TEU1jARW5WZJldd+SODR7pTKUMAvORTOwxDTcYOozlGDJIyhXdVm H8AhWCAahDxeg9ao6u4Sx+jHrOhXWBg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243052; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6LEnV05GAdxAxtoxmlpP5GQRyBBCmk1hTN15A8D+Gww=; b=MSKg/sTW2GiuhDDYqApINJ5rwd4JXCG9I/lMXaku9qFj5+XrQkyZ1A41bLfwJOflpU+IK6 VG9KJEuUfX0IzuDw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Juraj Marcin Subject: [PULL 08/26] migration/ram: One less indent for ram_find_and_save_block() Date: Fri, 11 Jul 2025 11:10:13 -0300 Message-Id: <20250711141031.423-9-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 9FB571F385 X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo,suse.de:dkim,suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243515783116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu The check over PAGE_DIRTY_FOUND isn't necessary. We could indent one less and assert that instead. Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20250613140801.474264-9-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/ram.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ffd839f57c..aa76f0a53f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2286,16 +2286,18 @@ static int ram_find_and_save_block(RAMState *rs) if (!get_queued_page(rs, pss)) { /* priority queue empty, so just search for something dirty */ int res =3D find_dirty_block(rs, pss); - if (res !=3D PAGE_DIRTY_FOUND) { - if (res =3D=3D PAGE_ALL_CLEAN) { - break; - } else if (res =3D=3D PAGE_TRY_AGAIN) { - continue; - } else if (res < 0) { - pages =3D res; - break; - } + + if (res =3D=3D PAGE_ALL_CLEAN) { + break; + } else if (res =3D=3D PAGE_TRY_AGAIN) { + continue; + } else if (res < 0) { + pages =3D res; + break; } + + /* Otherwise we must have a dirty page to move */ + assert(res =3D=3D PAGE_DIRTY_FOUND); } pages =3D ram_save_host_page(rs, pss); if (pages) { --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243557; cv=none; d=zohomail.com; s=zohoarc; b=Sppb9zh/CQAm0Dfhhcsrl7UcNcARqTtQwG4oGxUryp16+8puIehG9adHmLSj7EBRDmXKwWgj/KfR9x4vxT6EQjYUQdsTHJ9n/rdK3vUKAleEp1ezvzl0pPch+0YgI4QkjuNPImae35mKUo7xdsxHZRCq9wTSNlSu5vgtY5evwCo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243557; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RCqEvWe9GNRh7oYXCD6PB4Slj9Qc/7F7uIcxx9tkTxo=; b=UrAr4lYQYuMwobnTf0FVWmbAZmoUfM0GGAPwxn8z/4s8KvWEpAJEAt2+uTkzHT7ii8vUnRTdOpS6h0YFLUL6iDs9L4HSyP5z4W+7vwtq582NWMPikmpyVqwYb+i5qyCenpBh3JlzhsLRqZHaoTi8OR3KsNNHPwx7uUJVkCDTuuc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243557543640.4979260133077; Fri, 11 Jul 2025 07:19:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEa6-0004Km-Fq; Fri, 11 Jul 2025 10:18:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETB-0001WO-6r for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:21 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaET8-0006fV-NR for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:20 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 7A1BF21190; Fri, 11 Jul 2025 14:10:54 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 262061388B; Fri, 11 Jul 2025 14:10:52 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id CEVQNWwbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243054; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RCqEvWe9GNRh7oYXCD6PB4Slj9Qc/7F7uIcxx9tkTxo=; b=gl6LQsa9V5iopGV5WIbhQJ62IeAiRbJqypzd40RrPutwpeeVk6Sy5yyOvvnIleKO4X7Dy+ Nlwyscrqs+AtgQES22SgD+CR0VDz5wsyGJRXUDzHZUgzi6OYzXb7C0QskYxRTSI/hVKztH Tmmr4vL19+e6+2WgpU/Vq3qQElYb9mE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243054; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RCqEvWe9GNRh7oYXCD6PB4Slj9Qc/7F7uIcxx9tkTxo=; b=4Rz4U+uVexoF9qq5FpX1mf99hQzfqHfL34So0Go5qNc/uQn/1PqDciHFuj/tp0F50Hldsc tIh4chM1eB53hVCA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=gl6LQsa9; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=4Rz4U+uV DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243054; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RCqEvWe9GNRh7oYXCD6PB4Slj9Qc/7F7uIcxx9tkTxo=; b=gl6LQsa9V5iopGV5WIbhQJ62IeAiRbJqypzd40RrPutwpeeVk6Sy5yyOvvnIleKO4X7Dy+ Nlwyscrqs+AtgQES22SgD+CR0VDz5wsyGJRXUDzHZUgzi6OYzXb7C0QskYxRTSI/hVKztH Tmmr4vL19+e6+2WgpU/Vq3qQElYb9mE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243054; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RCqEvWe9GNRh7oYXCD6PB4Slj9Qc/7F7uIcxx9tkTxo=; b=4Rz4U+uVexoF9qq5FpX1mf99hQzfqHfL34So0Go5qNc/uQn/1PqDciHFuj/tp0F50Hldsc tIh4chM1eB53hVCA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Juraj Marcin Subject: [PULL 09/26] migration/ram: Add tracepoints for ram_save_complete() Date: Fri, 11 Jul 2025 11:10:14 -0300 Message-Id: <20250711141031.423-10-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-5.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; DWL_DNSWL_MED(-2.00)[suse.de:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 7A1BF21190 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -5.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243559640116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Take notes on start/end state of dirty pages for the whole system. Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20250613140801.474264-10-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/ram.c | 5 +++++ migration/trace-events | 1 + 2 files changed, 6 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index aa76f0a53f..e4871b4c17 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3290,6 +3290,8 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) RAMState *rs =3D *temp; int ret =3D 0; =20 + trace_ram_save_complete(rs->migration_dirty_pages, 0); + rs->last_stage =3D !migration_in_colo_state(); =20 WITH_RCU_READ_LOCK_GUARD() { @@ -3353,6 +3355,9 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) } =20 qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + + trace_ram_save_complete(rs->migration_dirty_pages, 1); + return qemu_fflush(f); } =20 diff --git a/migration/trace-events b/migration/trace-events index c506e11a2e..dcd8fe9a0c 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -105,6 +105,7 @@ ram_load_postcopy_loop(int channel, uint64_t addr, int = flags) "chan=3D%d addr=3D0x%" ram_postcopy_send_discard_bitmap(void) "" ram_save_page(const char *rbname, uint64_t offset, void *host) "%s: offset= : 0x%" PRIx64 " host: %p" ram_save_queue_pages(const char *rbname, size_t start, size_t len) "%s: st= art: 0x%zx len: 0x%zx" +ram_save_complete(uint64_t dirty_pages, int done) "dirty=3D%" PRIu64 ", do= ne=3D%d" ram_dirty_bitmap_request(char *str) "%s" ram_dirty_bitmap_reload_begin(char *str) "%s" ram_dirty_bitmap_reload_complete(char *str) "%s" --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243309; cv=none; d=zohomail.com; s=zohoarc; b=oEr4j1P5vFFfczOkmmm6QXy6EN2wyVKG+E5PKtZXV8X8u42lhRUX9lKTlvNXwyi08HEHiKzcQcrG2Nov7Nhdhf1/f/8zm8Mlmy6g/LXKt2406DtDltzV2TK/YLKM+bxjMgxp+b4MsUIS7ZEkhE3KPacEBXlSBC4q0bvnWfasN40= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243309; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Bnz20s69+Inf6MGIXR/Cgjh2pMWofI8C1iUr1UrS8nM=; b=VBtL7PBOIzceBBNPLSUnr35FU1fN5+u5FBzTkI+DPujQPtoZdgSueVDzxZ4TBqDwtg5uZx4T9LOK7dAh9TjfVhSCy7q+tfeNG2cZ/Rc0KLfwTekdYCceQTZsZVKjSAc/8qVe3Wf4OxEVK9YDloEaD/ajoZFuDnJZ9pmTHN0hhCk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175224330947212.2232138025106; Fri, 11 Jul 2025 07:15:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEWa-0000FE-LO; Fri, 11 Jul 2025 10:14:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETR-0001sg-Hy for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:44 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETL-0006iO-R1 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:35 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 565DD1F454; Fri, 11 Jul 2025 14:10:56 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0220D1388B; Fri, 11 Jul 2025 14:10:54 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id +HWhLG4bcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243058; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Bnz20s69+Inf6MGIXR/Cgjh2pMWofI8C1iUr1UrS8nM=; b=P93/VpFeWZSJ9qojnmvWKA0tf3OSr7EDls9PHtK3P50tLF/4RQgCsb8yKduBjfHEShk21U 9vYqy4bp9Ofnx/y2wzSIwwtQY35et4kCHhB7vgMDqnAMhGwwQOhGfuVOACXbXwEs4UnN0W miExWSsBdcnHePaaTDWpcjHJ5sHcaDU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243058; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Bnz20s69+Inf6MGIXR/Cgjh2pMWofI8C1iUr1UrS8nM=; b=Muq2n5DUnYc286j4WvUDVgi9nJYV7ACmNvYU5iKNZ3pq7P9fYAOt7ZS64EMmxdh3e2FF36 kmJVFCCyuvb3DEDg== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=m83txFfz; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=073tuCCW DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243056; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Bnz20s69+Inf6MGIXR/Cgjh2pMWofI8C1iUr1UrS8nM=; b=m83txFfzEosbzKm3tbKJJCym+hyjf1gqZeIcH2FTBu353mYfU0qQ+CrEM6hbIExUg0trwj NlwVf5WW2kIWM2mdrPoP/GOaFCDBCuvsXoemX2jqmhX8UJDBh7XGuFIE8qA6hmDuQQcZkr HWx2dfZEF99glNmi1MSTNNn/mBtw9rQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243056; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Bnz20s69+Inf6MGIXR/Cgjh2pMWofI8C1iUr1UrS8nM=; b=073tuCCW5N4TnHnnhvyvTL0vzGkHkv4V11Tl/ruom1ysUdg2CGkvJxAtBZMfLhC1keHLy6 pfY0KEATADC4BaCw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Juraj Marcin Subject: [PULL 10/26] migration: Rewrite the migration complete detect logic Date: Fri, 11 Jul 2025 11:10:15 -0300 Message-Id: <20250711141031.423-11-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 565DD1F454 X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo,suse.de:dkim,suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) (identity @suse.de) X-ZM-MESSAGEID: 1752243311507116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu There're a few things off here in that logic, rewrite it. When at it, add rich comment to explain each of the decisions. Since this is very sensitive path for migration, below are the list of things changed with their reasonings. (1) Exact pending size is only needed for precopy not postcopy Fundamentally it's because "exact" version only does one more deep sync to fetch the pending results, while in postcopy's case it's never going to sync anything more than estimate as the VM on source is stopped. (2) Do _not_ rely on threshold_size anymore to decide whether postcopy should complete threshold_size was calculated from the expected downtime and bandwidth only during precopy as an efficient way to decide when to switchover. It's not sensible to rely on threshold_size in postcopy. For precopy, if switchover is decided, the migration will complete soon. It's not true for postcopy. Logically speaking, postcopy should only complete the migration if all pending data is flushed. Here it used to work because save_complete() used to implicitly contain save_live_iterate() when there's pending size. Even if that looks benign, having RAMs to be migrated in postcopy's save_complete() has other bad side effects: (a) Since save_complete() needs to be run once at a time, it means when moving RAM there's no way moving other things (rather than round-robin iterating the vmstate handlers like what we do with ITERABLE phase). Not an immediate concern, but it may stop working in the future when there're more than one iterables (e.g. vfio postcopy). (b) postcopy recovery, unfortunately, only works during ITERABLE phase. IOW, if src QEMU moves RAM during postcopy's save_complete() and network failed, then it'll crash both QEMUs... OTOH if it failed during iteration it'll still be recoverable. IOW, this change should further reduce the window QEMU split brain and crash in extreme cases. If we enable the ram_save_complete() tracepoints, we'll see this before this patch: 1267959@1748381938.294066:ram_save_complete dirty=3D9627, done=3D0 1267959@1748381938.308884:ram_save_complete dirty=3D0, done=3D1 It means in this migration there're 9627 pages migrated at complete() of postcopy phase. After this change, all the postcopy RAM should be migrated in iterable phase, rather than save_complete(): 1267959@1748381938.294066:ram_save_complete dirty=3D0, done=3D0 1267959@1748381938.308884:ram_save_complete dirty=3D0, done=3D1 (3) Adjust when to decide to switch to postcopy This shouldn't be super important, the movement makes sure there's only one in_postcopy check, then we are clear on what we do with the two completely differnt use cases (precopy v.s. postcopy). (4) Trivial touch up on threshold_size comparision Which changes: "(!pending_size || pending_size < s->threshold_size)" into: "(pending_size <=3D s->threshold_size)" Reviewed-by: Juraj Marcin Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613140801.474264-11-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 57 +++++++++++++++++++++++++++++++------------ 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index e33e39ac74..923400f801 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3436,33 +3436,60 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) Error *local_err =3D NULL; bool in_postcopy =3D s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE; bool can_switchover =3D migration_can_switchover(s); + bool complete_ready; =20 + /* Fast path - get the estimated amount of pending data */ qemu_savevm_state_pending_estimate(&must_precopy, &can_postcopy); pending_size =3D must_precopy + can_postcopy; trace_migrate_pending_estimate(pending_size, must_precopy, can_postcop= y); =20 - if (pending_size < s->threshold_size) { - qemu_savevm_state_pending_exact(&must_precopy, &can_postcopy); - pending_size =3D must_precopy + can_postcopy; - trace_migrate_pending_exact(pending_size, must_precopy, can_postco= py); + if (in_postcopy) { + /* + * Iterate in postcopy until all pending data flushed. Note that + * postcopy completion doesn't rely on can_switchover, because when + * POSTCOPY_ACTIVE it means switchover already happened. + */ + complete_ready =3D !pending_size; + } else { + /* + * Exact pending reporting is only needed for precopy. Taking RAM + * as example, there'll be no extra dirty information after + * postcopy started, so ESTIMATE should always match with EXACT + * during postcopy phase. + */ + if (pending_size < s->threshold_size) { + qemu_savevm_state_pending_exact(&must_precopy, &can_postcopy); + pending_size =3D must_precopy + can_postcopy; + trace_migrate_pending_exact(pending_size, must_precopy, + can_postcopy); + } + + /* Should we switch to postcopy now? */ + if (must_precopy <=3D s->threshold_size && + can_switchover && qatomic_read(&s->start_postcopy)) { + if (postcopy_start(s, &local_err)) { + migrate_set_error(s, local_err); + error_report_err(local_err); + } + return MIG_ITERATE_SKIP; + } + + /* + * For precopy, migration can complete only if: + * + * (1) Switchover is acknowledged by destination + * (2) Pending size is no more than the threshold specified + * (which was calculated from expected downtime) + */ + complete_ready =3D can_switchover && (pending_size <=3D s->thresho= ld_size); } =20 - if ((!pending_size || pending_size < s->threshold_size) && can_switcho= ver) { + if (complete_ready) { trace_migration_thread_low_pending(pending_size); migration_completion(s); return MIG_ITERATE_BREAK; } =20 - /* Still a significant amount to transfer */ - if (!in_postcopy && must_precopy <=3D s->threshold_size && can_switcho= ver && - qatomic_read(&s->start_postcopy)) { - if (postcopy_start(s, &local_err)) { - migrate_set_error(s, local_err); - error_report_err(local_err); - } - return MIG_ITERATE_SKIP; - } - /* Just another iteration step */ qemu_savevm_state_iterate(s->to_dst_file, in_postcopy); return MIG_ITERATE_RESUME; --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243648; cv=none; d=zohomail.com; s=zohoarc; b=DGeojaC+HL3qg3Iq2AYQ53C39yE72ROWCDgQZaaPN6+Pi6T5GzxNlhk8wZD8Iw9cyfXs6zJSbCSgB3MZQd98iNKNrtWZOok5d+BDZbhPhIuV1vr5fPSPVbtH5Ws0jxEuoNJDfHJaA8f0SJN7IKhGMV5KOLh4qbBBu0y/iyGn0gc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243648; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=MtcLz3HoOuMb7dWaVW/7n137qzfJKsmUGSn1VyrRD58=; b=eHKSywadOHjll3LM7r9jzlu/w3oTOVIZoVQRUQ+POcJ46AtZkY+L1I6T3Y29RpoNxJezeYp4XRR9ezy+aSGWV+D2gkgwcCn+YtZM/ai3GBsnMWXOV1F8COPhrW7AtqMTZLnSaqYHbAJcnk5KkyVWV9UIU92OXtrdKFmR4xIDKpI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243648493681.6253956135624; Fri, 11 Jul 2025 07:20:48 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEYI-0002hG-VJ; Fri, 11 Jul 2025 10:16:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETH-0001iI-Po for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:31 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETF-0006hV-5Y for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:26 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 3C9F71F455; Fri, 11 Jul 2025 14:10:58 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D2A7D13918; Fri, 11 Jul 2025 14:10:56 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id CA1yI3AbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243058; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MtcLz3HoOuMb7dWaVW/7n137qzfJKsmUGSn1VyrRD58=; b=SjfxJleEp1JDj4xy2dkfdIf7LPhgK3JA22+/142axIunnsYt+xImBNteYUeQfLo7dv2hRg y/sRVzi1Y1/brnQGamRpkup8CPPjwPpLIeqgwwFFF55+TslEBBJkceAoPQFnQNXrdgkpPO qd2OenIAz4GVsqULZYRrUBzlGJVaHHM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243058; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MtcLz3HoOuMb7dWaVW/7n137qzfJKsmUGSn1VyrRD58=; b=fg/AZulPxsJ4R8duTEe1wKZT3ByQIu4/u2JvKLxuieeFCMbH5bXE2cqeYe1FhRs2OtjJ4h O8FtIRhqwwUYKTDA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243058; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MtcLz3HoOuMb7dWaVW/7n137qzfJKsmUGSn1VyrRD58=; b=SjfxJleEp1JDj4xy2dkfdIf7LPhgK3JA22+/142axIunnsYt+xImBNteYUeQfLo7dv2hRg y/sRVzi1Y1/brnQGamRpkup8CPPjwPpLIeqgwwFFF55+TslEBBJkceAoPQFnQNXrdgkpPO qd2OenIAz4GVsqULZYRrUBzlGJVaHHM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243058; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MtcLz3HoOuMb7dWaVW/7n137qzfJKsmUGSn1VyrRD58=; b=fg/AZulPxsJ4R8duTEe1wKZT3ByQIu4/u2JvKLxuieeFCMbH5bXE2cqeYe1FhRs2OtjJ4h O8FtIRhqwwUYKTDA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Yanfei Xu Subject: [PULL 11/26] migration/postcopy: Avoid clearing dirty bitmap for postcopy too Date: Fri, 11 Jul 2025 11:10:16 -0300 Message-Id: <20250711141031.423-12-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:mid,suse.de:email]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FUZZY_RATELIMITED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_THREE(0.00)[3]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243648884116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu This is a follow up on the other commit "migration/ram: avoid to do log clear in the last round" but for postcopy. https://lore.kernel.org/r/20250514115827.3216082-1-yanfei.xu@bytedance.com I can observe more than 10% reduction of average page fault latency during postcopy phase with this optimization: Before: 268.00us (+-1.87%) After: 232.67us (+-2.01%) The test was done with a 16GB VM with 80 vCPUs, running a workload that busy random writes to 13GB memory. Cc: Yanfei Xu Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613140801.474264-12-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/ram.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index e4871b4c17..7208bc114f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -835,8 +835,10 @@ static inline bool migration_bitmap_clear_dirty(RAMSta= te *rs, * protections isn't needed as we know there will be either (1) no * further writes if migration will complete, or (2) migration fails * at last then tracking isn't needed either. + * + * Do the same for postcopy due to the same reason. */ - if (!rs->last_stage) { + if (!rs->last_stage && !migration_in_postcopy()) { /* * Clear dirty bitmap if needed. This _must_ be called before we * send any of the page in the chunk because we need to make sure --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243751; cv=none; d=zohomail.com; s=zohoarc; b=BPebGWXFPS+By6nACLupjvpbaDIfA3HGe2OEpwivnNQ/wXiVDvRqBRIimMnPKrLmq9JFFuJTNM1k3ZyPRJkYoRxzVL7L1dTgNWlPoDEHNXONfEZetYw/rqFobRLwnvfy7jdHBOeGhs7YoyA4Q0fTdzqUzHmKOJCR9/uibqbN0gE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243751; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dsttLL8hzDtpH4Kw3GDEfOAB0aWQ2GCHEr5G6/o1gVA=; b=UsmZpFufffS27QI/kq3IEdVWqNeVWfyzqewip+/yrAn25REOpo3KsMgvO9qPPJxx6NGzs+eS/V2nMxrlYLD7thPJp/oUyyawgTDBA26cgOkdA5wyejA0/RndpBB1jba0dqUzmWbr1blOAkt7xBGz8hXvjlhEU6vUCcOWOftJXlw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243751784453.63612881924905; Fri, 11 Jul 2025 07:22:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEcJ-000172-Hj; Fri, 11 Jul 2025 10:20:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETa-00020h-OH for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:50 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETW-0006kT-KW for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:44 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BDEF41F38C; Fri, 11 Jul 2025 14:10:59 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id AA8C91388B; Fri, 11 Jul 2025 14:10:58 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id iE0BGnIbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:10:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243059; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dsttLL8hzDtpH4Kw3GDEfOAB0aWQ2GCHEr5G6/o1gVA=; b=sDiot09RFLaATYDrJDgOSGkxLeEygBzQXXKNMisGVYKzEjW2kr7RGQD02keTrtEavYtb0A RX1B44suZBmrRXZkIPtMAyVbJ17RYsgo4k14+099NdfiNtvJ4cWKyunjLViBlcteAiRjR1 i7hlFEBW19tbNhGK8z0rbSI09H2VnZk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243059; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dsttLL8hzDtpH4Kw3GDEfOAB0aWQ2GCHEr5G6/o1gVA=; b=fyg97MyY2926JTlQk/tu6OhmAxQBX6yPFL28G9KrMTwTd1wfGYMc7gW0e0X6v4M7coZw4J Yw2/q22Vet0yTdCw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243059; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dsttLL8hzDtpH4Kw3GDEfOAB0aWQ2GCHEr5G6/o1gVA=; b=sDiot09RFLaATYDrJDgOSGkxLeEygBzQXXKNMisGVYKzEjW2kr7RGQD02keTrtEavYtb0A RX1B44suZBmrRXZkIPtMAyVbJ17RYsgo4k14+099NdfiNtvJ4cWKyunjLViBlcteAiRjR1 i7hlFEBW19tbNhGK8z0rbSI09H2VnZk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243059; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dsttLL8hzDtpH4Kw3GDEfOAB0aWQ2GCHEr5G6/o1gVA=; b=fyg97MyY2926JTlQk/tu6OhmAxQBX6yPFL28G9KrMTwTd1wfGYMc7gW0e0X6v4M7coZw4J Yw2/q22Vet0yTdCw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 12/26] migration: Add option to set postcopy-blocktime Date: Fri, 11 Jul 2025 11:10:17 -0300 Message-Id: <20250711141031.423-13-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:mid,suse.de:email]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243754393116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Add a global property to allow enabling postcopy-blocktime feature. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-2-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/options.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/options.c b/migration/options.c index 162c72cda4..4e923a2e07 100644 --- a/migration/options.c +++ b/migration/options.c @@ -187,6 +187,8 @@ const Property migration_properties[] =3D { DEFINE_PROP_MIG_CAP("x-postcopy-ram", MIGRATION_CAPABILITY_POSTCOPY_RA= M), DEFINE_PROP_MIG_CAP("x-postcopy-preempt", MIGRATION_CAPABILITY_POSTCOPY_PREEMPT), + DEFINE_PROP_MIG_CAP("postcopy-blocktime", + MIGRATION_CAPABILITY_POSTCOPY_BLOCKTIME), DEFINE_PROP_MIG_CAP("x-colo", MIGRATION_CAPABILITY_X_COLO), DEFINE_PROP_MIG_CAP("x-release-ram", MIGRATION_CAPABILITY_RELEASE_RAM), DEFINE_PROP_MIG_CAP("x-return-path", MIGRATION_CAPABILITY_RETURN_PATH), --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243558; cv=none; d=zohomail.com; s=zohoarc; b=hczIlBXCfVmoUULJWaCuyW+OIUmISJ+sqFsH7/EY9n0y6DamexGVbPQfZhfAzagBR3SGgxzPqoTXbRkBHq64fDu4Lg4P6L7YT1eEKTECA8Du5BKzC87lVBFQk4qohnBcGj8e3lKUYwJEgOlLq0EbHAlj02GShx9MIZAj2JHDyxU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243558; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dk32ib5y9aoYEvyT4wmPbKEXZ0gXCpoEwNJ9W1pwe1U=; b=aHIzaYGyRoh9S7xpHqXzCtk59ZrQnG6F8x3HTXOZw8I0n8sEOSE2viBOYTnNnpw/eKo2OaeMuFNRS4Gw3b4Mo/mSMPuaBSAx23QlCKylNfi0S0CAfAdM2Ed7HSUgSvleRmlhdOgLvwb1ASu96C/dPECrmhYFcitjaqdUuLpXY4o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243558296159.7529794591063; Fri, 11 Jul 2025 07:19:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEXI-0001mi-GL; Fri, 11 Jul 2025 10:15:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETH-0001iN-QS for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:31 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETE-0006hL-86 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:26 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5D0A621192; Fri, 11 Jul 2025 14:11:01 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 494C21388B; Fri, 11 Jul 2025 14:11:00 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WJu1AXQbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243061; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dk32ib5y9aoYEvyT4wmPbKEXZ0gXCpoEwNJ9W1pwe1U=; b=aNIRPkPUcRMcAxKBpRu3aUBkTRV8DqMykXliniA6Jbv3keyHNeJmWPdo2/R2o14YxI8IHw kahHDrYTa/EJjreZd/5rj4xzq5/F8NYAG2SUimRBgcMx5tdGt40JYFwo5ybfrFDlEE6GLQ A7PTy8watgmqtL1V8oehWoSreqlMX9g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243061; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dk32ib5y9aoYEvyT4wmPbKEXZ0gXCpoEwNJ9W1pwe1U=; b=Bu8MY1yTv1ze2rXjxqgujoSFXWom/zHHiKNHXfWmxhves9h7V5SULVnU3R7AK2y0rdG0mP 9UGGZH7UrTuWVGAA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243061; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dk32ib5y9aoYEvyT4wmPbKEXZ0gXCpoEwNJ9W1pwe1U=; b=aNIRPkPUcRMcAxKBpRu3aUBkTRV8DqMykXliniA6Jbv3keyHNeJmWPdo2/R2o14YxI8IHw kahHDrYTa/EJjreZd/5rj4xzq5/F8NYAG2SUimRBgcMx5tdGt40JYFwo5ybfrFDlEE6GLQ A7PTy8watgmqtL1V8oehWoSreqlMX9g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243061; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dk32ib5y9aoYEvyT4wmPbKEXZ0gXCpoEwNJ9W1pwe1U=; b=Bu8MY1yTv1ze2rXjxqgujoSFXWom/zHHiKNHXfWmxhves9h7V5SULVnU3R7AK2y0rdG0mP 9UGGZH7UrTuWVGAA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 13/26] migration/postcopy: Push blocktime start/end into page req mutex Date: Fri, 11 Jul 2025 11:10:18 -0300 Message-Id: <20250711141031.423-14-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243559967116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu The postcopy blocktime feature was tricky that it used quite some atomic operations over quite a few arrays and vars, without explaining how that would be thread safe. The thread safety here is about concurrency between the fault thread and the fault resolution threads, possible to access the same chunk of data. All these atomic ops can be expensive too before knowing clearly how it works. OTOH, postcopy has one page_request_mutex used to serialize the received bitmap updates. So far it's ok - we don't yet have a lot of threads contending the lock. It might change after multifd will be supported, but that's a separate story. What is important is, with that mutex, it's pretty lightweight to move all the blocktime maintenance into the mutex critical section. It's because the blocktime layer is lightweighted: almost "remember which vcpu faulted on which address", and "ok we get some fault resolved, calculate how long it takes". It's also an optional feature for now (but I have thought of changing that, maybe in the future). Let's push the blocktime layer into the mutex, so that it's always thread-safe even without any atomic ops. To achieve that, I'll need to add a tid parameter on fault path so that it'll start to pass the faulted thread ID into deeper the stack, but not too deep. When at it, add a comment for the shared fault handler (for example, vhost-user devices running with postcopy), to mention a TODO. One reason it might not be trivial is that vhost-user's userfaultfds should be opened by vhost-user process, so it's pretty hard to control making sure the TID feature will be around. It wasn't supported before, so keep it like that for now. Now we should be as ease when everything is protected by a mutex that we always take anyway. One side effect: we can finally remove one ramblock_recv_bitmap_test() in mark_postcopy_blocktime_begin(), which was pretty weird and which also includes a weird (but maybe necessary.. but maybe not?) operation to inject a blocktime entry then quickly erase it.. When we're with the mutex, and when we make sure it's invoked after checking the receive bitmap, it's not needed anymore. Instead, we assert. As another side effect, this paves way for removing all atomic ops in all the mem accesses in blocktime layer. Note that we need a stub for mark_postcopy_blocktime_begin() for Windows builds. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-3-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 25 +++++++++++------- migration/migration.h | 2 +- migration/postcopy-ram.c | 56 +++++++++++++++++++++------------------- migration/postcopy-ram.h | 2 ++ migration/trace-events | 2 +- 5 files changed, 48 insertions(+), 39 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 923400f801..10c216d25d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -576,22 +576,27 @@ int migrate_send_rp_message_req_pages(MigrationIncomi= ngState *mis, } =20 int migrate_send_rp_req_pages(MigrationIncomingState *mis, - RAMBlock *rb, ram_addr_t start, uint64_t had= dr) + RAMBlock *rb, ram_addr_t start, uint64_t had= dr, + uint32_t tid) { void *aligned =3D (void *)(uintptr_t)ROUND_DOWN(haddr, qemu_ram_pagesi= ze(rb)); bool received =3D false; =20 WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex) { received =3D ramblock_recv_bitmap_test_byte_offset(rb, start); - if (!received && !g_tree_lookup(mis->page_requested, aligned)) { - /* - * The page has not been received, and it's not yet in the page - * request list. Queue it. Set the value of element to 1, so= that - * things like g_tree_lookup() will return TRUE (1) when found. - */ - g_tree_insert(mis->page_requested, aligned, (gpointer)1); - qatomic_inc(&mis->page_requested_count); - trace_postcopy_page_req_add(aligned, mis->page_requested_count= ); + if (!received) { + if (!g_tree_lookup(mis->page_requested, aligned)) { + /* + * The page has not been received, and it's not yet in the + * page request list. Queue it. Set the value of element + * to 1, so that things like g_tree_lookup() will return + * TRUE (1) when found. + */ + g_tree_insert(mis->page_requested, aligned, (gpointer)1); + qatomic_inc(&mis->page_requested_count); + trace_postcopy_page_req_add(aligned, mis->page_requested_c= ount); + } + mark_postcopy_blocktime_begin(haddr, tid, rb); } } =20 diff --git a/migration/migration.h b/migration/migration.h index 739289de93..01329bf824 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -546,7 +546,7 @@ void migrate_send_rp_shut(MigrationIncomingState *mis, void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, - ram_addr_t start, uint64_t haddr); + ram_addr_t start, uint64_t haddr, uint32_t t= id); int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, RAMBlock *rb, ram_addr_t start); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 75fd310fb2..32fa06dabd 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -752,8 +752,12 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd, pagesize); } =20 +/* + * NOTE: @tid is only used when postcopy-blocktime feature is enabled, and + * also optional: when zero is provided, the fault accounting will be igno= red. + */ static int postcopy_request_page(MigrationIncomingState *mis, RAMBlock *rb, - ram_addr_t start, uint64_t haddr) + ram_addr_t start, uint64_t haddr, uint32_= t tid) { void *aligned =3D (void *)(uintptr_t)ROUND_DOWN(haddr, qemu_ram_pagesi= ze(rb)); =20 @@ -772,7 +776,7 @@ static int postcopy_request_page(MigrationIncomingState= *mis, RAMBlock *rb, return received ? 0 : postcopy_place_page_zero(mis, aligned, rb); } =20 - return migrate_send_rp_req_pages(mis, rb, start, haddr); + return migrate_send_rp_req_pages(mis, rb, start, haddr, tid); } =20 /* @@ -793,7 +797,8 @@ int postcopy_request_shared_page(struct PostCopyFD *pcf= d, RAMBlock *rb, qemu_ram_get_idstr(rb), rb_offset); return postcopy_wake_shared(pcfd, client_addr, rb); } - postcopy_request_page(mis, rb, aligned_rbo, client_addr); + /* TODO: support blocktime tracking */ + postcopy_request_page(mis, rb, aligned_rbo, client_addr, 0); return 0; } =20 @@ -819,17 +824,17 @@ static uint32_t get_low_time_offset(PostcopyBlocktime= Context *dc) } =20 /* - * This function is being called when pagefault occurs. It - * tracks down vCPU blocking time. + * This function is being called when pagefault occurs. It tracks down vCPU + * blocking time. It's protected by @page_request_mutex. * * @addr: faulted host virtual address * @ptid: faulted process thread id * @rb: ramblock appropriate to addr */ -static void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid, - RAMBlock *rb) +void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid, + RAMBlock *rb) { - int cpu, already_received; + int cpu; MigrationIncomingState *mis =3D migration_incoming_get_current(); PostcopyBlocktimeContext *dc =3D mis->blocktime_ctx; uint32_t low_time_offset; @@ -852,24 +857,19 @@ static void mark_postcopy_blocktime_begin(uintptr_t a= ddr, uint32_t ptid, qatomic_xchg(&dc->vcpu_addr[cpu], addr); =20 /* - * check it here, not at the beginning of the function, - * due to, check could occur early than bitmap_set in - * qemu_ufd_copy_ioctl + * The caller should only inject a blocktime entry when the page is + * yet missing. */ - already_received =3D ramblock_recv_bitmap_test(rb, (void *)addr); - if (already_received) { - qatomic_xchg(&dc->vcpu_addr[cpu], 0); - qatomic_xchg(&dc->page_fault_vcpu_time[cpu], 0); - qatomic_dec(&dc->smp_cpus_down); - } + assert(!ramblock_recv_bitmap_test(rb, (void *)addr)); + trace_mark_postcopy_blocktime_begin(addr, dc, dc->page_fault_vcpu_time= [cpu], - cpu, already_received); + cpu); } =20 /* - * This function just provide calculated blocktime per cpu and trace it. - * Total blocktime is calculated in mark_postcopy_blocktime_end. - * + * This function just provide calculated blocktime per cpu and trace it. + * Total blocktime is calculated in mark_postcopy_blocktime_end. It's + * protected by @page_request_mutex. * * Assume we have 3 CPU * @@ -1068,17 +1068,14 @@ static void *postcopy_ram_fault_thread(void *opaque) qemu_ram_get_idstr(rb), rb_offset, msg.arg.pagefault.feat.pti= d); - mark_postcopy_blocktime_begin( - (uintptr_t)(msg.arg.pagefault.address), - msg.arg.pagefault.feat.ptid, rb); - retry: /* * Send the request to the source - we want to request one * of our host page sizes (which is >=3D TPS) */ ret =3D postcopy_request_page(mis, rb, rb_offset, - msg.arg.pagefault.address); + msg.arg.pagefault.address, + msg.arg.pagefault.feat.ptid); if (ret) { /* May be network failure, try to wait for recovery */ postcopy_pause_fault_thread(mis); @@ -1299,8 +1296,8 @@ static int qemu_ufd_copy_ioctl(MigrationIncomingState= *mis, void *host_addr, qemu_cond_signal(&mis->page_request_cond); } } - qemu_mutex_unlock(&mis->page_request_mutex); mark_postcopy_blocktime_end((uintptr_t)host_addr); + qemu_mutex_unlock(&mis->page_request_mutex); } return ret; } @@ -1430,6 +1427,11 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd, { g_assert_not_reached(); } + +void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid, + RAMBlock *rb) +{ +} #endif =20 /* -----------------------------------------------------------------------= -- */ diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index a6df1b2811..3852141d7e 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -196,5 +196,7 @@ void postcopy_preempt_new_channel(MigrationIncomingStat= e *mis, QEMUFile *file); void postcopy_preempt_setup(MigrationState *s); int postcopy_preempt_establish_channel(MigrationState *s); bool postcopy_is_paused(MigrationStatus status); +void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid, + RAMBlock *rb); =20 #endif diff --git a/migration/trace-events b/migration/trace-events index dcd8fe9a0c..917f521e88 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -285,7 +285,7 @@ postcopy_nhp_range(const char *ramblock, void *host_add= r, size_t offset, size_t postcopy_place_page(void *host_addr) "host=3D%p" postcopy_place_page_zero(void *host_addr) "host=3D%p" postcopy_ram_enable_notify(void) "" -mark_postcopy_blocktime_begin(uint64_t addr, void *dd, uint32_t time, int = cpu, int received) "addr: 0x%" PRIx64 ", dd: %p, time: %u, cpu: %d, already= _received: %d" +mark_postcopy_blocktime_begin(uint64_t addr, void *dd, uint32_t time, int = cpu) "addr: 0x%" PRIx64 ", dd: %p, time: %u, cpu: %d" mark_postcopy_blocktime_end(uint64_t addr, void *dd, uint32_t time, int af= fected_cpu) "addr: 0x%" PRIx64 ", dd: %p, time: %u, affected_cpu: %d" postcopy_pause_fault_thread(void) "" postcopy_pause_fault_thread_continued(void) "" --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243555; cv=none; d=zohomail.com; s=zohoarc; b=VebEl9aDmMWb/8N1m+Is7siUeFRiH3DYKIknaebDYeHs1BWWLhVh5Xtd0a9jXI8/1ux66tvq26k13gbc/EThsp/V3oB1sPqpbYKDiXtGbOZ1JWvZdGhVQ+lvs8fn6d8ImUND+W+6da/h8fmEdRmG2pTQRVqDndHa6RFCRnDoYr8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243555; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=o9QJlhQ/lxb/HDco23SyqNvtvSX5N8Qjj+ieM9lAfNQ=; b=gjdHgDFSDvDjAf8qygwdDB1CUqYS9LlKFilRNG0pce+SbGaOJkSIX2Sl47lDe+nhgWRAYWXNVbsIW+TnWWD+sNg8cnJ42pKWTSJAE5gGfzzU6EjIp50rSHDiKu2l7ZPNFF6fEqRwA4HQ4MDLmLFobEIPqC/U82Gucty/zPeNfVE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17522435555361018.4679055076725; Fri, 11 Jul 2025 07:19:15 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEWf-0000bY-FS; Fri, 11 Jul 2025 10:14:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETP-0001pv-J5 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:44 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETL-0006iQ-S3 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:34 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E8ABA21197; Fri, 11 Jul 2025 14:11:02 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D84761388B; Fri, 11 Jul 2025 14:11:01 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 8JRQJXUbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243063; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o9QJlhQ/lxb/HDco23SyqNvtvSX5N8Qjj+ieM9lAfNQ=; b=JMkXFVvTiTT2+GaQYHiTIHtugZ2oL5WVut61yTHCUmBeZ0IZimn7T98oxe0U2VURd8GFD7 2av+ptW6ZLDh3IVGEq+nxrT8jBRX80swy9nW280Ydg6iw/8SgLUd9XDk560zJrdNooXOqb DlBceyE+u0I4qak7QOm+4HNCxtue0+s= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243063; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o9QJlhQ/lxb/HDco23SyqNvtvSX5N8Qjj+ieM9lAfNQ=; b=01P3cbhi17LnPL4Vr/3JC7Yy1KtCOLMXAl7C4JamPQFhup4i4BT8nhLpcF+Ra3GJhD/h2M ky8IEAsrPUWAwgDg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243062; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o9QJlhQ/lxb/HDco23SyqNvtvSX5N8Qjj+ieM9lAfNQ=; b=Ah7Kd2g4A5tbXsOqFzPqJ+a0GaIcBm7ObBCuItu5m0anunt9G+hmVMLiRhZKHMAVni6NfD ueYIZXd93jkKFqL0XCMPPDfNCo7Obbq4SWLf9ItbFy0Pd3kuvMjkHzD+UQoZxqieHjwgTg gNatM//vN0E9RXBI2fcapDuq3G2j0HM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243062; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o9QJlhQ/lxb/HDco23SyqNvtvSX5N8Qjj+ieM9lAfNQ=; b=nUNkFGEB8V5/yUOK1qqr+FLLEsxSyLjUINvr/OCC5l8FxZn8hjlqj6OdZ6VSOYci246LT6 +clCpZTgrkmlEfAQ== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 14/26] migration/postcopy: Drop all atomic ops in blocktime feature Date: Fri, 11 Jul 2025 11:10:19 -0300 Message-Id: <20250711141031.423-15-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:email,imap1.dmz-prg2.suse.org:helo]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) (identity @suse.de) X-ZM-MESSAGEID: 1752243557876116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Now with the mutex protection it's not needed anymore. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-4-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/postcopy-ram.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 32fa06dabd..81925532de 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -849,12 +849,12 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, ui= nt32_t ptid, =20 low_time_offset =3D get_low_time_offset(dc); if (dc->vcpu_addr[cpu] =3D=3D 0) { - qatomic_inc(&dc->smp_cpus_down); + dc->smp_cpus_down++; } =20 - qatomic_xchg(&dc->last_begin, low_time_offset); - qatomic_xchg(&dc->page_fault_vcpu_time[cpu], low_time_offset); - qatomic_xchg(&dc->vcpu_addr[cpu], addr); + dc->last_begin =3D low_time_offset; + dc->page_fault_vcpu_time[cpu] =3D low_time_offset; + dc->vcpu_addr[cpu] =3D addr; =20 /* * The caller should only inject a blocktime entry when the page is @@ -915,29 +915,26 @@ static void mark_postcopy_blocktime_end(uintptr_t add= r) for (i =3D 0; i < smp_cpus; i++) { uint32_t vcpu_blocktime =3D 0; =20 - read_vcpu_time =3D qatomic_fetch_add(&dc->page_fault_vcpu_time[i],= 0); - if (qatomic_fetch_add(&dc->vcpu_addr[i], 0) !=3D addr || - read_vcpu_time =3D=3D 0) { + read_vcpu_time =3D dc->page_fault_vcpu_time[i]; + if (dc->vcpu_addr[i] !=3D addr || read_vcpu_time =3D=3D 0) { continue; } - qatomic_xchg(&dc->vcpu_addr[i], 0); + dc->vcpu_addr[i] =3D 0; vcpu_blocktime =3D low_time_offset - read_vcpu_time; affected_cpu +=3D 1; /* we need to know is that mark_postcopy_end was due to * faulted page, another possible case it's prefetched * page and in that case we shouldn't be here */ - if (!vcpu_total_blocktime && - qatomic_fetch_add(&dc->smp_cpus_down, 0) =3D=3D smp_cpus) { + if (!vcpu_total_blocktime && dc->smp_cpus_down =3D=3D smp_cpus) { vcpu_total_blocktime =3D true; } /* continue cycle, due to one page could affect several vCPUs */ dc->vcpu_blocktime[i] +=3D vcpu_blocktime; } =20 - qatomic_sub(&dc->smp_cpus_down, affected_cpu); + dc->smp_cpus_down -=3D affected_cpu; if (vcpu_total_blocktime) { - dc->total_blocktime +=3D low_time_offset - qatomic_fetch_add( - &dc->last_begin, 0); + dc->total_blocktime +=3D low_time_offset - dc->last_begin; } trace_mark_postcopy_blocktime_end(addr, dc, dc->total_blocktime, affected_cpu); --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243310; cv=none; d=zohomail.com; s=zohoarc; b=eLdt2Gqf6Bvocya/IcQM1LyEpMzZlcDBgDzmMHttHyRS2tLg/gQKU7j6ERFuFw45+N4amUBOaYfMsL7XqfQSwd4qwo9Zh5NB8KJ8NaqbN+rhI9hVW8aUik2XWtPBAvEaZe7faVDMKRZ5F77ZsnGUQqNryQdF9AgUDd5ezEvCBmk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243310; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=54fwsoqL/jMCsGFAHTp3/X/pPU81n3Ezh4sNFqmkJqM=; b=DoCIzZ2V1Cm8eH1OT9YYkqmwUAHhf9n9u0Oxl+igR4Zbmb7ywBFRaJzYs8krYFcYfL1jtW47313OGqjD2CLYCmJmJ/WtfDCLXc7x8fO/YcUKu+umKvT/Hfip5s4mVCVBRH2X13/Dfr4F3B5vX9Yk6FDKp0WRGzflbjMhal6FV64= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243310198926.729020928045; Fri, 11 Jul 2025 07:15:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEWg-0000gB-EU; Fri, 11 Jul 2025 10:14:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETi-00029M-Mc for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:58 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETe-0006mw-83 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:52 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 896C81F45B; Fri, 11 Jul 2025 14:11:04 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 757F91388B; Fri, 11 Jul 2025 14:11:03 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id ePKwDHcbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243064; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=54fwsoqL/jMCsGFAHTp3/X/pPU81n3Ezh4sNFqmkJqM=; b=mGbIBY70e/vjS8x4PT+J17kiidAWdyxucnzzIRxU/dnt4kLAeeyISh0/HZ0i4W4XdULswL tL1U+QActGCYG2YwYWsd0KQnQwkyNIsCOLQwsJt+EbddClAIZvpXdMCTgBepDDcBFUmaE+ t4FcSxfeo1vVAPl4P7da3YyugSKp1VM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243064; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=54fwsoqL/jMCsGFAHTp3/X/pPU81n3Ezh4sNFqmkJqM=; b=xbIeizN4qH6O3CdMOJxjzllQLpCexUZzeqW+zsmE1GapWb+2U4kYYNhcTkNH+UrzuaeGVm HFzf+XAbZAdc+uBw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243064; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=54fwsoqL/jMCsGFAHTp3/X/pPU81n3Ezh4sNFqmkJqM=; b=mGbIBY70e/vjS8x4PT+J17kiidAWdyxucnzzIRxU/dnt4kLAeeyISh0/HZ0i4W4XdULswL tL1U+QActGCYG2YwYWsd0KQnQwkyNIsCOLQwsJt+EbddClAIZvpXdMCTgBepDDcBFUmaE+ t4FcSxfeo1vVAPl4P7da3YyugSKp1VM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243064; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=54fwsoqL/jMCsGFAHTp3/X/pPU81n3Ezh4sNFqmkJqM=; b=xbIeizN4qH6O3CdMOJxjzllQLpCexUZzeqW+zsmE1GapWb+2U4kYYNhcTkNH+UrzuaeGVm HFzf+XAbZAdc+uBw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 15/26] migration/postcopy: Make all blocktime vars 64bits Date: Fri, 11 Jul 2025 11:10:20 -0300 Message-Id: <20250711141031.423-16-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:email,imap1.dmz-prg2.suse.org:helo]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243311523116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu I am guessing it was used to be 32bits because of the atomic ops. Now all the atomic ops are gone and we're protected by a mutex instead, it's ok we can switch to 64 bits. Reasons to move over: - Allow further patches to change the unit from ms to us: with postcopy preempt mode, we're really into hundreds of microseconds level on blocktime. We'd better be able to trap those. - This also paves way for some other tricks that the original version used to avoid overflows, e.g., start_time was almost only useful before to make sure the sampled timestamp won't overflow a 32-bit field. - This prepares further reports on top of existing data collected, e.g. average page fault latencies. When average operation is taken into account, milliseconds are simply too coarse grained. When at it: - Rename page_fault_vcpu_time to vcpu_blocktime_start. - Rename vcpu_blocktime to vcpu_blocktime_total. - Touch up the trace-events to not dump blocktime ctx pointer Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-5-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/postcopy-ram.c | 50 ++++++++++++++++++++-------------------- migration/trace-events | 4 ++-- 2 files changed, 27 insertions(+), 27 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 81925532de..ec91821b85 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -112,14 +112,15 @@ void postcopy_thread_create(MigrationIncomingState *m= is, =20 typedef struct PostcopyBlocktimeContext { /* time when page fault initiated per vCPU */ - uint32_t *page_fault_vcpu_time; + uint64_t *vcpu_blocktime_start; + /* blocktime per vCPU */ + uint64_t *vcpu_blocktime_total; /* page address per vCPU */ uintptr_t *vcpu_addr; - uint32_t total_blocktime; - /* blocktime per vCPU */ - uint32_t *vcpu_blocktime; + /* total blocktime when all vCPUs are stopped */ + uint64_t total_blocktime; /* point in time when last page fault was initiated */ - uint32_t last_begin; + uint64_t last_begin; /* number of vCPU are suspended */ int smp_cpus_down; uint64_t start_time; @@ -133,9 +134,9 @@ typedef struct PostcopyBlocktimeContext { =20 static void destroy_blocktime_context(struct PostcopyBlocktimeContext *ctx) { - g_free(ctx->page_fault_vcpu_time); + g_free(ctx->vcpu_blocktime_start); + g_free(ctx->vcpu_blocktime_total); g_free(ctx->vcpu_addr); - g_free(ctx->vcpu_blocktime); g_free(ctx); } =20 @@ -151,13 +152,14 @@ static struct PostcopyBlocktimeContext *blocktime_con= text_new(void) MachineState *ms =3D MACHINE(qdev_get_machine()); unsigned int smp_cpus =3D ms->smp.cpus; PostcopyBlocktimeContext *ctx =3D g_new0(PostcopyBlocktimeContext, 1); - ctx->page_fault_vcpu_time =3D g_new0(uint32_t, smp_cpus); + + ctx->vcpu_blocktime_start =3D g_new0(uint64_t, smp_cpus); + ctx->vcpu_blocktime_total =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_addr =3D g_new0(uintptr_t, smp_cpus); - ctx->vcpu_blocktime =3D g_new0(uint32_t, smp_cpus); - ctx->exit_notifier.notify =3D migration_exit_cb; ctx->start_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); qemu_add_exit_notifier(&ctx->exit_notifier); + return ctx; } =20 @@ -168,7 +170,7 @@ static uint32List *get_vcpu_blocktime_list(PostcopyBloc= ktimeContext *ctx) int i; =20 for (i =3D ms->smp.cpus - 1; i >=3D 0; i--) { - QAPI_LIST_PREPEND(list, ctx->vcpu_blocktime[i]); + QAPI_LIST_PREPEND(list, (uint32_t)ctx->vcpu_blocktime_total[i]); } =20 return list; @@ -191,12 +193,12 @@ void fill_destination_postcopy_migration_info(Migrati= onInfo *info) } =20 info->has_postcopy_blocktime =3D true; - info->postcopy_blocktime =3D bc->total_blocktime; + info->postcopy_blocktime =3D (uint32_t)bc->total_blocktime; info->has_postcopy_vcpu_blocktime =3D true; info->postcopy_vcpu_blocktime =3D get_vcpu_blocktime_list(bc); } =20 -static uint32_t get_postcopy_total_blocktime(void) +static uint64_t get_postcopy_total_blocktime(void) { MigrationIncomingState *mis =3D migration_incoming_get_current(); PostcopyBlocktimeContext *bc =3D mis->blocktime_ctx; @@ -816,11 +818,9 @@ static int get_mem_fault_cpu_index(uint32_t pid) return -1; } =20 -static uint32_t get_low_time_offset(PostcopyBlocktimeContext *dc) +static uint64_t get_low_time_offset(PostcopyBlocktimeContext *dc) { - int64_t start_time_offset =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME) - - dc->start_time; - return start_time_offset < 1 ? 1 : start_time_offset & UINT32_MAX; + return (uint64_t)qemu_clock_get_ms(QEMU_CLOCK_REALTIME) - dc->start_ti= me; } =20 /* @@ -837,7 +837,7 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint= 32_t ptid, int cpu; MigrationIncomingState *mis =3D migration_incoming_get_current(); PostcopyBlocktimeContext *dc =3D mis->blocktime_ctx; - uint32_t low_time_offset; + uint64_t low_time_offset; =20 if (!dc || ptid =3D=3D 0) { return; @@ -853,7 +853,7 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint= 32_t ptid, } =20 dc->last_begin =3D low_time_offset; - dc->page_fault_vcpu_time[cpu] =3D low_time_offset; + dc->vcpu_blocktime_start[cpu] =3D low_time_offset; dc->vcpu_addr[cpu] =3D addr; =20 /* @@ -862,7 +862,7 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint= 32_t ptid, */ assert(!ramblock_recv_bitmap_test(rb, (void *)addr)); =20 - trace_mark_postcopy_blocktime_begin(addr, dc, dc->page_fault_vcpu_time= [cpu], + trace_mark_postcopy_blocktime_begin(addr, dc->vcpu_blocktime_start[cpu= ], cpu); } =20 @@ -901,7 +901,7 @@ static void mark_postcopy_blocktime_end(uintptr_t addr) unsigned int smp_cpus =3D ms->smp.cpus; int i, affected_cpu =3D 0; bool vcpu_total_blocktime =3D false; - uint32_t read_vcpu_time, low_time_offset; + uint64_t read_vcpu_time, low_time_offset; =20 if (!dc) { return; @@ -913,9 +913,9 @@ static void mark_postcopy_blocktime_end(uintptr_t addr) * optimal, more optimal algorithm is keeping tree or hash * where key is address value is a list of */ for (i =3D 0; i < smp_cpus; i++) { - uint32_t vcpu_blocktime =3D 0; + uint64_t vcpu_blocktime =3D 0; =20 - read_vcpu_time =3D dc->page_fault_vcpu_time[i]; + read_vcpu_time =3D dc->vcpu_blocktime_start[i]; if (dc->vcpu_addr[i] !=3D addr || read_vcpu_time =3D=3D 0) { continue; } @@ -929,14 +929,14 @@ static void mark_postcopy_blocktime_end(uintptr_t add= r) vcpu_total_blocktime =3D true; } /* continue cycle, due to one page could affect several vCPUs */ - dc->vcpu_blocktime[i] +=3D vcpu_blocktime; + dc->vcpu_blocktime_total[i] +=3D vcpu_blocktime; } =20 dc->smp_cpus_down -=3D affected_cpu; if (vcpu_total_blocktime) { dc->total_blocktime +=3D low_time_offset - dc->last_begin; } - trace_mark_postcopy_blocktime_end(addr, dc, dc->total_blocktime, + trace_mark_postcopy_blocktime_end(addr, dc->total_blocktime, affected_cpu); } =20 diff --git a/migration/trace-events b/migration/trace-events index 917f521e88..02cdb6e7cc 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -285,8 +285,8 @@ postcopy_nhp_range(const char *ramblock, void *host_add= r, size_t offset, size_t postcopy_place_page(void *host_addr) "host=3D%p" postcopy_place_page_zero(void *host_addr) "host=3D%p" postcopy_ram_enable_notify(void) "" -mark_postcopy_blocktime_begin(uint64_t addr, void *dd, uint32_t time, int = cpu) "addr: 0x%" PRIx64 ", dd: %p, time: %u, cpu: %d" -mark_postcopy_blocktime_end(uint64_t addr, void *dd, uint32_t time, int af= fected_cpu) "addr: 0x%" PRIx64 ", dd: %p, time: %u, affected_cpu: %d" +mark_postcopy_blocktime_begin(uint64_t addr, uint64_t time, int cpu) "addr= : 0x%" PRIx64 ", time: %" PRIu64 ", cpu: %d" +mark_postcopy_blocktime_end(uint64_t addr, uint64_t time, int affected_cpu= ) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", affected_cpus: %d" postcopy_pause_fault_thread(void) "" postcopy_pause_fault_thread_continued(void) "" postcopy_pause_fast_load(void) "" --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243549; cv=none; d=zohomail.com; s=zohoarc; b=Fg8wgC7LSwUwILFyAIYzUwGPd6LJmSYTvNxdJRpe0SgzMA297UBri9ZEuWzwLHOksZ9nisAA9P5G3eFIUlC9/TRAqljlzEut0RtQhIJMlJ6kq9b2troLWHEbi3kJd8+Qrvo49n1IvD3S7MFccJNHcvC77sazRPSU/SJhZyulnB4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243549; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lk/EorubpgbR6NL5e0peICeBcHiwFFvKz1YGb5MUabw=; b=LwbE7j4IBj80nmd9RCKmfGHzKFaPhuDiO3ty4JLud4PL4rzWDWBAq0dXt2zRpgl+EVMF81R29H3SKWv9k968ESsOCijNs4qJEFOX9ODTRfu6nIEVmnd8kaxpV5mr+l2ueFYOfZRISxbyf9qm+EqTpBoT8Vfiwz+4i9kvFk3k/os= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243549593286.0530585381514; Fri, 11 Jul 2025 07:19:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEXD-0001Sb-0c; Fri, 11 Jul 2025 10:15:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETo-0002OH-82 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:00 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETm-0006oA-Aj for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:59 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 30DB41F460; Fri, 11 Jul 2025 14:11:06 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 17E5A1388B; Fri, 11 Jul 2025 14:11:04 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2MIHMHgbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243066; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lk/EorubpgbR6NL5e0peICeBcHiwFFvKz1YGb5MUabw=; b=mE+jbl4pwJWmHGq2uQfLvfHKRDtwc7Ab9hGHJ03zRqpyT1brDOKg+Oyonxi4B/GwJcc1vs VcDaj38mU0pSQjlnyW74x1DDG8+foRvQ6zFCrbxvCl3VgYbHUbIiLmS81G3SqLtRyPysDe Sr6Q4OFlXKqKLHg52uUtD00AdVqJU8A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243066; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lk/EorubpgbR6NL5e0peICeBcHiwFFvKz1YGb5MUabw=; b=lxSmQOfhCJ6Q2TML14NZou4KvCelyh9m6A8drwrl/KW3T3AnUJZ0muT+TkCMMtHgMnNgC1 QeUk00FaG38BEoCw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243066; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lk/EorubpgbR6NL5e0peICeBcHiwFFvKz1YGb5MUabw=; b=mE+jbl4pwJWmHGq2uQfLvfHKRDtwc7Ab9hGHJ03zRqpyT1brDOKg+Oyonxi4B/GwJcc1vs VcDaj38mU0pSQjlnyW74x1DDG8+foRvQ6zFCrbxvCl3VgYbHUbIiLmS81G3SqLtRyPysDe Sr6Q4OFlXKqKLHg52uUtD00AdVqJU8A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243066; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lk/EorubpgbR6NL5e0peICeBcHiwFFvKz1YGb5MUabw=; b=lxSmQOfhCJ6Q2TML14NZou4KvCelyh9m6A8drwrl/KW3T3AnUJZ0muT+TkCMMtHgMnNgC1 QeUk00FaG38BEoCw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 16/26] migration/postcopy: Drop PostcopyBlocktimeContext.start_time Date: Fri, 11 Jul 2025 11:10:21 -0300 Message-Id: <20250711141031.423-17-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; FUZZY_RATELIMITED(0.00)[rspamd.com]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_TWO(0.00)[2]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; TO_DN_SOME(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid, suse.de:email, imap1.dmz-prg2.suse.org:helo] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243551838116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Now with 64bits, the offseting using start_time is not needed anymore, because the array can always remember the whole timestamp. Then drop the unused parameter in get_low_time_offset() altogether. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-6-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/postcopy-ram.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index ec91821b85..e9acb4ef6e 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -123,7 +123,6 @@ typedef struct PostcopyBlocktimeContext { uint64_t last_begin; /* number of vCPU are suspended */ int smp_cpus_down; - uint64_t start_time; =20 /* * Handler for exit event, necessary for @@ -157,7 +156,6 @@ static struct PostcopyBlocktimeContext *blocktime_conte= xt_new(void) ctx->vcpu_blocktime_total =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_addr =3D g_new0(uintptr_t, smp_cpus); ctx->exit_notifier.notify =3D migration_exit_cb; - ctx->start_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); qemu_add_exit_notifier(&ctx->exit_notifier); =20 return ctx; @@ -818,9 +816,9 @@ static int get_mem_fault_cpu_index(uint32_t pid) return -1; } =20 -static uint64_t get_low_time_offset(PostcopyBlocktimeContext *dc) +static uint64_t get_low_time_offset(void) { - return (uint64_t)qemu_clock_get_ms(QEMU_CLOCK_REALTIME) - dc->start_ti= me; + return (uint64_t)qemu_clock_get_ms(QEMU_CLOCK_REALTIME); } =20 /* @@ -847,7 +845,7 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint= 32_t ptid, return; } =20 - low_time_offset =3D get_low_time_offset(dc); + low_time_offset =3D get_low_time_offset(); if (dc->vcpu_addr[cpu] =3D=3D 0) { dc->smp_cpus_down++; } @@ -907,7 +905,7 @@ static void mark_postcopy_blocktime_end(uintptr_t addr) return; } =20 - low_time_offset =3D get_low_time_offset(dc); + low_time_offset =3D get_low_time_offset(); /* lookup cpu, to clear it, * that algorithm looks straightforward, but it's not * optimal, more optimal algorithm is keeping tree or hash --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243320; cv=none; d=zohomail.com; s=zohoarc; b=mPAGf+iqkEnTs8ASffryWHk6a7y38tF5beamzD1KCfDdEHOpqcQ1lRmgTTss0F7hPLrI909IpdP/vOcreeiianEEsgEqs6TtT4SAYAKhzJ+F3kwOEpB1d75nmjN4gfwIZZCFjNH2HBc9rCav2ozKJRajkuceJn3rh3kocoOem6E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243320; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xERf0QzeUJ/kYk97vVnlP2qxGIkQ4LHhvQ4xm0T8gKs=; b=XbppcqD8+2WyyaNKSY1MdlfGFxhbVZWK9qfJTHdy3rE/OyMaAmXYyHlouRAz2tY8280sZgz3wykuYtnD8YoPa+jl24dwADml2HsF8hJA6MQW/IQfAfLFLsavTAMPHrnqKVqVIgv5RGDkfQ+H0o4Nl5WIe7Y+N1IYPJbyb2EnYCo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243320491853.2978116685279; Fri, 11 Jul 2025 07:15:20 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEWe-0000V6-6O; Fri, 11 Jul 2025 10:14:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETa-00020g-OH for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:50 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETW-0006kD-Ca for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:44 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C4D182119B; Fri, 11 Jul 2025 14:11:07 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B0D051388B; Fri, 11 Jul 2025 14:11:06 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id KGvPGnobcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243067; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xERf0QzeUJ/kYk97vVnlP2qxGIkQ4LHhvQ4xm0T8gKs=; b=1MI2bPsFha7OU3TCXmyL7fL5rat6G2Y363XYAEowWzd1W9geRVivnVXHPf8Jqjd2HY+WN3 PsR3xT/2enKIubD0+dOPWPBacep2mP/9Pw3PQOJcJCwTIrfynlnbCn1vJc8M4RZD6eY/tb vtn0UKyJ6ILJ9eTyV19sKishkqcfPb0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243067; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xERf0QzeUJ/kYk97vVnlP2qxGIkQ4LHhvQ4xm0T8gKs=; b=xMdouM+jwsh1pt+XdILmpvDW2q0lPac696bPaDmOJ1v/qbKxhfEuEZORfubzOr7mBk8rVA lmhyxf5hu9rVNRAw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=1MI2bPsF; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=xMdouM+j DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243067; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xERf0QzeUJ/kYk97vVnlP2qxGIkQ4LHhvQ4xm0T8gKs=; b=1MI2bPsFha7OU3TCXmyL7fL5rat6G2Y363XYAEowWzd1W9geRVivnVXHPf8Jqjd2HY+WN3 PsR3xT/2enKIubD0+dOPWPBacep2mP/9Pw3PQOJcJCwTIrfynlnbCn1vJc8M4RZD6eY/tb vtn0UKyJ6ILJ9eTyV19sKishkqcfPb0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243067; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xERf0QzeUJ/kYk97vVnlP2qxGIkQ4LHhvQ4xm0T8gKs=; b=xMdouM+jwsh1pt+XdILmpvDW2q0lPac696bPaDmOJ1v/qbKxhfEuEZORfubzOr7mBk8rVA lmhyxf5hu9rVNRAw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 17/26] migration/postcopy: Bring blocktime layer to ns level Date: Fri, 11 Jul 2025 11:10:22 -0300 Message-Id: <20250711141031.423-18-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCPT_COUNT_TWO(0.00)[2]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: C4D182119B X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243321571116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu With 64-bit fields, it is trivial. The caution is when exposing any values in QMP, it was still declared with milliseconds (ms). Hence it's needed to do the convertion when exporting the values to existing QMP queries. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-7-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/postcopy-ram.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index e9acb4ef6e..9dfa92a62d 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -110,6 +110,7 @@ void postcopy_thread_create(MigrationIncomingState *mis, #include #include =20 +/* All the time records are in unit of nanoseconds */ typedef struct PostcopyBlocktimeContext { /* time when page fault initiated per vCPU */ uint64_t *vcpu_blocktime_start; @@ -168,7 +169,9 @@ static uint32List *get_vcpu_blocktime_list(PostcopyBloc= ktimeContext *ctx) int i; =20 for (i =3D ms->smp.cpus - 1; i >=3D 0; i--) { - QAPI_LIST_PREPEND(list, (uint32_t)ctx->vcpu_blocktime_total[i]); + /* Convert ns -> ms */ + QAPI_LIST_PREPEND( + list, (uint32_t)(ctx->vcpu_blocktime_total[i] / SCALE_MS)); } =20 return list; @@ -191,7 +194,8 @@ void fill_destination_postcopy_migration_info(Migration= Info *info) } =20 info->has_postcopy_blocktime =3D true; - info->postcopy_blocktime =3D (uint32_t)bc->total_blocktime; + /* Convert ns -> ms */ + info->postcopy_blocktime =3D (uint32_t)(bc->total_blocktime / SCALE_MS= ); info->has_postcopy_vcpu_blocktime =3D true; info->postcopy_vcpu_blocktime =3D get_vcpu_blocktime_list(bc); } @@ -816,9 +820,9 @@ static int get_mem_fault_cpu_index(uint32_t pid) return -1; } =20 -static uint64_t get_low_time_offset(void) +static uint64_t get_current_ns(void) { - return (uint64_t)qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + return (uint64_t)qemu_clock_get_ns(QEMU_CLOCK_REALTIME); } =20 /* @@ -835,7 +839,7 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint= 32_t ptid, int cpu; MigrationIncomingState *mis =3D migration_incoming_get_current(); PostcopyBlocktimeContext *dc =3D mis->blocktime_ctx; - uint64_t low_time_offset; + uint64_t current; =20 if (!dc || ptid =3D=3D 0) { return; @@ -845,13 +849,13 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, ui= nt32_t ptid, return; } =20 - low_time_offset =3D get_low_time_offset(); + current =3D get_current_ns(); if (dc->vcpu_addr[cpu] =3D=3D 0) { dc->smp_cpus_down++; } =20 - dc->last_begin =3D low_time_offset; - dc->vcpu_blocktime_start[cpu] =3D low_time_offset; + dc->last_begin =3D current; + dc->vcpu_blocktime_start[cpu] =3D current; dc->vcpu_addr[cpu] =3D addr; =20 /* @@ -899,13 +903,13 @@ static void mark_postcopy_blocktime_end(uintptr_t add= r) unsigned int smp_cpus =3D ms->smp.cpus; int i, affected_cpu =3D 0; bool vcpu_total_blocktime =3D false; - uint64_t read_vcpu_time, low_time_offset; + uint64_t read_vcpu_time, current; =20 if (!dc) { return; } =20 - low_time_offset =3D get_low_time_offset(); + current =3D get_current_ns(); /* lookup cpu, to clear it, * that algorithm looks straightforward, but it's not * optimal, more optimal algorithm is keeping tree or hash @@ -918,7 +922,7 @@ static void mark_postcopy_blocktime_end(uintptr_t addr) continue; } dc->vcpu_addr[i] =3D 0; - vcpu_blocktime =3D low_time_offset - read_vcpu_time; + vcpu_blocktime =3D current - read_vcpu_time; affected_cpu +=3D 1; /* we need to know is that mark_postcopy_end was due to * faulted page, another possible case it's prefetched @@ -932,7 +936,7 @@ static void mark_postcopy_blocktime_end(uintptr_t addr) =20 dc->smp_cpus_down -=3D affected_cpu; if (vcpu_total_blocktime) { - dc->total_blocktime +=3D low_time_offset - dc->last_begin; + dc->total_blocktime +=3D current - dc->last_begin; } trace_mark_postcopy_blocktime_end(addr, dc->total_blocktime, affected_cpu); --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243751; cv=none; d=zohomail.com; s=zohoarc; b=iybnWG7+diqoBnq/nuBKCN39MKxhCFps5DTF5UkP/NU6gTViy8CHVQE4WRAe/clhbS3+/3TkuRmnRLP7pegpkauhRSBMd42pBcL5kxE1JBhTVy6ct+16WKgPUk2ktc0wyw23ldIXH5GOZnY9ZGd1l5u+cIDKS2s57wMVxCV6Zp4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243751; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=J9U1h2MCqxrlE/miqEs5+j53V0GiCpV0gcF61UPWvMQ=; b=eUhn3u1cRNaCU8AQB3qoMKY7R+61t7eyhferSLMUT4jBr7CSSW3eJ8Pwq5X/+BfLhE8Pbuoih/5auGaD6sBnU+rhCpcVtegVgHfzTucVVDqhy1wO4AI0vnhgzJkax+sOd/s/kiIx5Mya2kiiF+x3r0Ic58FvAun75TsMSMwDAzE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175224375190669.07985765026649; Fri, 11 Jul 2025 07:22:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEcA-0000Ps-Jd; Fri, 11 Jul 2025 10:20:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETv-0002sa-PH for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:16 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETt-0006pb-TN for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:07 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 6A70B1F451; Fri, 11 Jul 2025 14:11:09 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4FAF61388B; Fri, 11 Jul 2025 14:11:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id yH4LA3wbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243069; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J9U1h2MCqxrlE/miqEs5+j53V0GiCpV0gcF61UPWvMQ=; b=S17TAfmYuZKHQJpewH9OoQ4YCMLD6X3hIwGyxOTlM8OZ6hqgEp72NBsAA1F0cI3z+Ipjz6 XLRKtgpEfZ3JZC6chE7ngkbSg/Q1QIvScM08GjoSNMT/lYSsy/KpQFkjA/iOKtACU8Cy40 Dr3HGxNpzHe4GxOEgYL+mdwcFyO1vtI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243069; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J9U1h2MCqxrlE/miqEs5+j53V0GiCpV0gcF61UPWvMQ=; b=ipu0Q3tCwYY3RrFV8lW7I5cyGVpgRQ9GvkH6iyef+Zt4tzFbPW3iLqxSEBF6BWzatyORUK FF2Sdv/UdkoqKeDg== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=S17TAfmY; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=ipu0Q3tC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243069; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J9U1h2MCqxrlE/miqEs5+j53V0GiCpV0gcF61UPWvMQ=; b=S17TAfmYuZKHQJpewH9OoQ4YCMLD6X3hIwGyxOTlM8OZ6hqgEp72NBsAA1F0cI3z+Ipjz6 XLRKtgpEfZ3JZC6chE7ngkbSg/Q1QIvScM08GjoSNMT/lYSsy/KpQFkjA/iOKtACU8Cy40 Dr3HGxNpzHe4GxOEgYL+mdwcFyO1vtI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243069; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J9U1h2MCqxrlE/miqEs5+j53V0GiCpV0gcF61UPWvMQ=; b=ipu0Q3tCwYY3RrFV8lW7I5cyGVpgRQ9GvkH6iyef+Zt4tzFbPW3iLqxSEBF6BWzatyORUK FF2Sdv/UdkoqKeDg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 18/26] migration/postcopy: Add blocktime fault counts per-vcpu Date: Fri, 11 Jul 2025 11:10:23 -0300 Message-Id: <20250711141031.423-19-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 6A70B1F451 X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; FUZZY_RATELIMITED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[suse.de:+]; FROM_HAS_DN(0.00)[]; RCVD_TLS_ALL(0.00)[]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns, imap1.dmz-prg2.suse.org:helo, suse.de:dkim, suse.de:mid, suse.de:email] X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243754430116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Add a field to count how many remote faults one vCPU has taken. So far it's still not used, but will be soon. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-8-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/postcopy-ram.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 9dfa92a62d..15ea106910 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -116,6 +116,8 @@ typedef struct PostcopyBlocktimeContext { uint64_t *vcpu_blocktime_start; /* blocktime per vCPU */ uint64_t *vcpu_blocktime_total; + /* count of faults per vCPU */ + uint64_t *vcpu_faults_count; /* page address per vCPU */ uintptr_t *vcpu_addr; /* total blocktime when all vCPUs are stopped */ @@ -136,6 +138,7 @@ static void destroy_blocktime_context(struct PostcopyBl= ocktimeContext *ctx) { g_free(ctx->vcpu_blocktime_start); g_free(ctx->vcpu_blocktime_total); + g_free(ctx->vcpu_faults_count); g_free(ctx->vcpu_addr); g_free(ctx); } @@ -155,6 +158,7 @@ static struct PostcopyBlocktimeContext *blocktime_conte= xt_new(void) =20 ctx->vcpu_blocktime_start =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_blocktime_total =3D g_new0(uint64_t, smp_cpus); + ctx->vcpu_faults_count =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_addr =3D g_new0(uintptr_t, smp_cpus); ctx->exit_notifier.notify =3D migration_exit_cb; qemu_add_exit_notifier(&ctx->exit_notifier); @@ -857,6 +861,7 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint= 32_t ptid, dc->last_begin =3D current; dc->vcpu_blocktime_start[cpu] =3D current; dc->vcpu_addr[cpu] =3D addr; + dc->vcpu_faults_count[cpu]++; =20 /* * The caller should only inject a blocktime entry when the page is --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243405; cv=none; d=zohomail.com; s=zohoarc; b=nQ8Um5O1kegiUhpX75EriYe1oeosvUmXBusJd65Sw4CUG88ocPgeMSFeDvZ779MPss8HQ5YT5TErR1oOK5fCbY++8/LM6uXtqSFpe4HmVqj4AdyF4ZvmxFxW/Bi6b+sT6g8wVDQ72lFp+rbyuWfizcX6OPGQw0Pc8Y3vM//Be8Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243405; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SnDpAQQCKqVOlmyJkam85CqaMlx30YtetQyR2F9+3Q8=; b=S0Ph+vV2nqNGMrQxUT0hmIHkXW48LBZsLUpgIPgPFZE+xy2QvZNlQh8OSg8PsBVHY0jx0SO7gXc7IVMpNfIkx86bfFytEr4qJfwoYLabhxSjbAvVc8rSN+REA1QsSrpvCaywCnZLVs+HiyzKewKCP8uhmC1nMUoBYj6MSZCMqSw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243405460102.47765456061018; Fri, 11 Jul 2025 07:16:45 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEWz-0000nq-8Q; Fri, 11 Jul 2025 10:15:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETk-0002Al-Is for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:58 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETe-0006mz-R7 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:11:54 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C91F92119D; Fri, 11 Jul 2025 14:11:11 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id E93D01388B; Fri, 11 Jul 2025 14:11:09 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id GF7KKH0bcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243073; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SnDpAQQCKqVOlmyJkam85CqaMlx30YtetQyR2F9+3Q8=; b=m74LQLAi6P0acLw2n9JFnWkO7QYBwCr/NQ2KwGZ5awe86xMT2pGyUeup6WoMRETzS3xdk6 k+jW0QzgabbXMlkqDMg/4jb1/5SInwmf9LNNHl1hcz8djJ22ODOPPn79dmcnrsObmzC2zd 9UpRe/1CPvHgVaz6s100njGAto3o/vY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243073; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SnDpAQQCKqVOlmyJkam85CqaMlx30YtetQyR2F9+3Q8=; b=Dr3W1dFTo12HvVeiTNQbzM7pmswm2NxpXbWxA+fd5oCiraQ+KX5heUGWwh6bYNE34kJtRI OU2RaO28N8RzL5DQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=GKnyWkZR; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=y1Ug5Qxf DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243071; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SnDpAQQCKqVOlmyJkam85CqaMlx30YtetQyR2F9+3Q8=; b=GKnyWkZR3cgQhOnp9QI9UmjEXJ/zrVMS3nc6y+kk0orp+wT9SxE0zV9feGA/4dxs/7G7HX 9UUVaxKP5AqnbQSnrIrtt0WP6DmKSVB2J7rXs0fg09mND4G0QJePvAoy2fydmtdzdRPafN oKro+25sgWxUluaSJT82RGZfdWDESjw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243071; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SnDpAQQCKqVOlmyJkam85CqaMlx30YtetQyR2F9+3Q8=; b=y1Ug5QxfECAIETntFKDTxVxhAyN0lE35kjA6ieBLaEtLi3WCILipGU+sN8NgEkjASgUDOP ALU02TDdAXvcmlAA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Markus Armbruster , "Dr . David Alan Gilbert" , Mario Casquero Subject: [PULL 19/26] migration/postcopy: Report fault latencies in blocktime Date: Fri, 11 Jul 2025 11:10:24 -0300 Message-Id: <20250711141031.423-20-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: C91F92119D X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo,treblig.org:email,suse.de:dkim,suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) (identity @suse.de) X-ZM-MESSAGEID: 1752243406751116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Blocktime so far only cares about the time one vcpu (or the whole system) got blocked. It would be also be helpful if it can also report the latency of page requests, which could be very sensitive during postcopy. Blocktime itself is sometimes not very important, especially when one thinks about KVM async PF support, which means vCPUs are literally almost not blocked at all because the guest OS is smart enough to switch to another task when a remote fault is needed. However, latency is still sensitive and important because even if the guest vCPU is running on threads that do not need a remote fault, the workload that accesses some missing page is still affected. Add two entries to the report, showing how long it takes to resolve a remote fault. Mention in the QAPI doc that this is not the real average fault latency, but only the ones that was requested for a remote fault. Unwrap get_vcpu_blocktime_list() so we don't need to walk the list twice, meanwhile add the entry checks in qtests for all postcopy tests. Cc: Markus Armbruster Cc: Dr. David Alan Gilbert Reviewed-by: Fabiano Rosas Tested-by: Mario Casquero Link: https://lore.kernel.org/r/20250613141217.474825-9-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration-hmp-cmds.c | 67 ++++++++++++++++++--------- migration/postcopy-ram.c | 49 +++++++++++++------- qapi/migration.json | 20 ++++++++ tests/qtest/migration/migration-qmp.c | 3 ++ 4 files changed, 102 insertions(+), 37 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 867e017b32..8b3846dab5 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -52,6 +52,51 @@ static void migration_global_dump(Monitor *mon) ms->clear_bitmap_shift); } =20 +static void migration_dump_blocktime(Monitor *mon, MigrationInfo *info) +{ + if (info->has_postcopy_blocktime) { + monitor_printf(mon, "Postcopy Blocktime (ms): %" PRIu32 "\n", + info->postcopy_blocktime); + } + + if (info->has_postcopy_vcpu_blocktime) { + uint32List *item =3D info->postcopy_vcpu_blocktime; + const char *sep =3D ""; + int count =3D 0; + + monitor_printf(mon, "Postcopy vCPU Blocktime (ms):\n ["); + + while (item) { + monitor_printf(mon, "%s%"PRIu32, sep, item->value); + item =3D item->next; + /* Each line 10 vcpu results, newline if there's more */ + sep =3D ((++count % 10 =3D=3D 0) && item) ? ",\n " : ", "; + } + monitor_printf(mon, "]\n"); + } + + if (info->has_postcopy_latency) { + monitor_printf(mon, "Postcopy Latency (ns): %" PRIu64 "\n", + info->postcopy_latency); + } + + if (info->has_postcopy_vcpu_latency) { + uint64List *item =3D info->postcopy_vcpu_latency; + const char *sep =3D ""; + int count =3D 0; + + monitor_printf(mon, "Postcopy vCPU Latencies (ns):\n ["); + + while (item) { + monitor_printf(mon, "%s%"PRIu64, sep, item->value); + item =3D item->next; + /* Each line 10 vcpu results, newline if there's more */ + sep =3D ((++count % 10 =3D=3D 0) && item) ? ",\n " : ", "; + } + monitor_printf(mon, "]\n"); + } +} + void hmp_info_migrate(Monitor *mon, const QDict *qdict) { bool show_all =3D qdict_get_try_bool(qdict, "all", false); @@ -202,27 +247,7 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->dirty_limit_ring_full_time); } =20 - if (info->has_postcopy_blocktime) { - monitor_printf(mon, "Postcopy Blocktime (ms): %" PRIu32 "\n", - info->postcopy_blocktime); - } - - if (info->has_postcopy_vcpu_blocktime) { - uint32List *item =3D info->postcopy_vcpu_blocktime; - const char *sep =3D ""; - int count =3D 0; - - monitor_printf(mon, "Postcopy vCPU Blocktime (ms):\n ["); - - while (item) { - monitor_printf(mon, "%s%"PRIu32, sep, item->value); - item =3D item->next; - /* Each line 10 vcpu results, newline if there's more */ - sep =3D ((++count % 10 =3D=3D 0) && item) ? ",\n " : ", "; - } - monitor_printf(mon, "]\n"); - } - + migration_dump_blocktime(mon, info); out: qapi_free_MigrationInfo(info); } diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 15ea106910..fe940f89b9 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -166,21 +166,6 @@ static struct PostcopyBlocktimeContext *blocktime_cont= ext_new(void) return ctx; } =20 -static uint32List *get_vcpu_blocktime_list(PostcopyBlocktimeContext *ctx) -{ - MachineState *ms =3D MACHINE(qdev_get_machine()); - uint32List *list =3D NULL; - int i; - - for (i =3D ms->smp.cpus - 1; i >=3D 0; i--) { - /* Convert ns -> ms */ - QAPI_LIST_PREPEND( - list, (uint32_t)(ctx->vcpu_blocktime_total[i] / SCALE_MS)); - } - - return list; -} - /* * This function just populates MigrationInfo from postcopy's * blocktime context. It will not populate MigrationInfo, @@ -192,16 +177,48 @@ void fill_destination_postcopy_migration_info(Migrati= onInfo *info) { MigrationIncomingState *mis =3D migration_incoming_get_current(); PostcopyBlocktimeContext *bc =3D mis->blocktime_ctx; + MachineState *ms =3D MACHINE(qdev_get_machine()); + uint64_t latency_total =3D 0, faults =3D 0; + uint32List *list_blocktime =3D NULL; + uint64List *list_latency =3D NULL; + int i; =20 if (!bc) { return; } =20 + for (i =3D ms->smp.cpus - 1; i >=3D 0; i--) { + uint64_t latency, total, count; + + /* Convert ns -> ms */ + QAPI_LIST_PREPEND(list_blocktime, + (uint32_t)(bc->vcpu_blocktime_total[i] / SCALE_M= S)); + + /* The rest in nanoseconds */ + total =3D bc->vcpu_blocktime_total[i]; + latency_total +=3D total; + count =3D bc->vcpu_faults_count[i]; + faults +=3D count; + + if (count) { + latency =3D total / count; + } else { + /* No fault detected */ + latency =3D 0; + } + + QAPI_LIST_PREPEND(list_latency, latency); + } + info->has_postcopy_blocktime =3D true; /* Convert ns -> ms */ info->postcopy_blocktime =3D (uint32_t)(bc->total_blocktime / SCALE_MS= ); info->has_postcopy_vcpu_blocktime =3D true; - info->postcopy_vcpu_blocktime =3D get_vcpu_blocktime_list(bc); + info->postcopy_vcpu_blocktime =3D list_blocktime; + info->has_postcopy_latency =3D true; + info->postcopy_latency =3D faults ? (latency_total / faults) : 0; + info->has_postcopy_vcpu_latency =3D true; + info->postcopy_vcpu_latency =3D list_latency; } =20 static uint64_t get_postcopy_total_blocktime(void) diff --git a/qapi/migration.json b/qapi/migration.json index e8a7d3b2a9..bb41dc0795 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -236,6 +236,17 @@ # This is only present when the postcopy-blocktime migration # capability is enabled. (Since 3.0) # +# @postcopy-latency: average remote page fault latency (in ns). Note that +# this doesn't include all faults, but only the ones that require a +# remote page request. So it should be always bigger than the real +# average page fault latency. This is only present when the +# postcopy-blocktime migration capability is enabled. (Since 10.1) +# +# @postcopy-vcpu-latency: average remote page fault latency per vCPU (in +# ns). It has the same definition of @postcopy-latency, but instead +# this is the per-vCPU statistics. This is only present when the +# postcopy-blocktime migration capability is enabled. (Since 10.1) +# # @socket-address: Only used for tcp, to know what the real port is # (Since 4.0) # @@ -260,6 +271,11 @@ # average memory load of the virtual CPU indirectly. Note that # zero means guest doesn't dirty memory. (Since 8.1) # +# Features: +# +# @unstable: Members @postcopy-latency, @postcopy-vcpu-latency are +# experimental. +# # Since: 0.14 ## { 'struct': 'MigrationInfo', @@ -275,6 +291,10 @@ '*blocked-reasons': ['str'], '*postcopy-blocktime': 'uint32', '*postcopy-vcpu-blocktime': ['uint32'], + '*postcopy-latency': { + 'type': 'uint64', 'features': [ 'unstable' ] }, + '*postcopy-vcpu-latency': { + 'type': ['uint64'], 'features': [ 'unstable' ] }, '*socket-address': ['SocketAddress'], '*dirty-limit-throttle-time-per-round': 'uint64', '*dirty-limit-ring-full-time': 'uint64'} } diff --git a/tests/qtest/migration/migration-qmp.c b/tests/qtest/migration/= migration-qmp.c index fb59741b2c..1a5ab2d229 100644 --- a/tests/qtest/migration/migration-qmp.c +++ b/tests/qtest/migration/migration-qmp.c @@ -358,6 +358,9 @@ void read_blocktime(QTestState *who) =20 rsp_return =3D migrate_query_not_failed(who); g_assert(qdict_haskey(rsp_return, "postcopy-blocktime")); + g_assert(qdict_haskey(rsp_return, "postcopy-vcpu-blocktime")); + g_assert(qdict_haskey(rsp_return, "postcopy-latency")); + g_assert(qdict_haskey(rsp_return, "postcopy-vcpu-latency")); qobject_unref(rsp_return); } =20 --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243760; cv=none; d=zohomail.com; s=zohoarc; b=ZstOIFtC+Nssom5l/k/TztnQ0ONhw45L4GNZHyUZwpUx6pdMESZoHSi1/H1ZBLDe74ybLcVoxedgaG87QUvSZZ7UiuUPLOoL8W7tKlu2DSIYLu/mX7y9gfWkW7ssboCYTDipZldtqch75FyTHL71NCNBHQYMN1WUb1DtdaCt+s8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243760; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZLVxbUl1QbCpvaOs+8U0flUWK625i092YCFo0WiecoQ=; b=YBBuMyAJx7mzjqc6DxC8/Jwefk2bThjk0q+vln8mknSRayYbJTb6+6RWtkOKcrvV2tH9PU694xhTZ9+tvqOv88daFq3rylzy7qDPYdcIhhp/n5Fw9oTFaE2VgM4izjKqeLSu9BBQBHfYSdhCqzQX/TVKNx38+00BZRm6E3+Zrqg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243760683581.8900372086947; Fri, 11 Jul 2025 07:22:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEdo-0003mu-0N; Fri, 11 Jul 2025 10:22:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaEU3-0002u0-Dc for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:16 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETz-0006r4-JV for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:14 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 669901F461; Fri, 11 Jul 2025 14:11:13 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5155113918; Fri, 11 Jul 2025 14:11:12 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 4DtOBIAbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243073; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZLVxbUl1QbCpvaOs+8U0flUWK625i092YCFo0WiecoQ=; b=LzQwW1q1pscp5Ja61+ZMgIzV8aoSTo+oVnjkYi+MEx0xH236M9EaCBnfYQTgHk3BBg11wK bZ6RR+6TVrHAUrgDVw3nBoinZWzyMC+4UvNafjRElqBsX6YX5VX+3vxoaTjGzXfqMUoTx8 wdLZLiiWI9ErtG5UYxNm0dGQT+yV+6o= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243073; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZLVxbUl1QbCpvaOs+8U0flUWK625i092YCFo0WiecoQ=; b=FASXHoFAE2p9UyIYcaBQuEEOc90erJ+G8G2ZV5cqewLNcyMjkVxLhLBVwj71nz5VXWVWyU HO35yFm4/mNAYCCQ== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=LzQwW1q1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=FASXHoFA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243073; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZLVxbUl1QbCpvaOs+8U0flUWK625i092YCFo0WiecoQ=; b=LzQwW1q1pscp5Ja61+ZMgIzV8aoSTo+oVnjkYi+MEx0xH236M9EaCBnfYQTgHk3BBg11wK bZ6RR+6TVrHAUrgDVw3nBoinZWzyMC+4UvNafjRElqBsX6YX5VX+3vxoaTjGzXfqMUoTx8 wdLZLiiWI9ErtG5UYxNm0dGQT+yV+6o= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243073; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZLVxbUl1QbCpvaOs+8U0flUWK625i092YCFo0WiecoQ=; b=FASXHoFAE2p9UyIYcaBQuEEOc90erJ+G8G2ZV5cqewLNcyMjkVxLhLBVwj71nz5VXWVWyU HO35yFm4/mNAYCCQ== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 20/26] migration/postcopy: Initialize blocktime context only until listen Date: Fri, 11 Jul 2025 11:10:25 -0300 Message-Id: <20250711141031.423-21-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RCPT_COUNT_TWO(0.00)[2]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 669901F461 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243762518116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Before this patch, the blocktime context can be created very early, because postcopy_ram_supported_by_host() <- migrate_caps_check() can happen during migration object init. The trick here is the blocktime context needs system vCPU information, which seems to be possible to change after that point. I didn't verify it, but it doesn't sound right. Now move it out and initialize the context only when postcopy listen starts. That is already during a migration so it should be guaranteed the vCPU topology can never change on both sides. While at it, assert that the ctx isn't created instead this time; the old "if" trick isn't needed when we're sure it will only happen once now. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-10-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/postcopy-ram.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index fe940f89b9..dd3615663f 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -325,13 +325,13 @@ static bool ufd_check_and_apply(int ufd, MigrationInc= omingState *mis, } =20 #ifdef UFFD_FEATURE_THREAD_ID + /* + * Postcopy blocktime conditionally needs THREAD_ID feature (introduced + * to Linux in 2017). Always try to enable it when QEMU is compiled + * with such environment. + */ if (UFFD_FEATURE_THREAD_ID & supported_features) { asked_features |=3D UFFD_FEATURE_THREAD_ID; - if (migrate_postcopy_blocktime()) { - if (!mis->blocktime_ctx) { - mis->blocktime_ctx =3D blocktime_context_new(); - } - } } #endif =20 @@ -1239,6 +1239,11 @@ int postcopy_ram_incoming_setup(MigrationIncomingSta= te *mis) return -1; } =20 + if (migrate_postcopy_blocktime()) { + assert(mis->blocktime_ctx =3D=3D NULL); + mis->blocktime_ctx =3D blocktime_context_new(); + } + /* Now an eventfd we use to tell the fault-thread to quit */ mis->userfault_event_fd =3D eventfd(0, EFD_CLOEXEC); if (mis->userfault_event_fd =3D=3D -1) { --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243421; cv=none; d=zohomail.com; s=zohoarc; b=MiA1UsPinOAcKVrx5e542sZv1AQih5M9VoRq8kD1bj1fzV4IyPe4/09TCUMUoqfmZfHuf8+xTwQBFkjWe9FqGWReRG16WxEF+jZQ6XXaYxxpT/FUu0NMYjOVu2OjX32iQMI9PJgry1vVXUoLzyluLmojZ01OC71J2SfpS8XKSV4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243421; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=4mzwXqQN6NKzga9iDDezb4zzzkxloBxPp8r53t2TXtM=; b=PKrmtxEEDy3IAktRQgXsWjDkW4hKDKwFHNdinzbQkVeqbWRh25xPqw+ZMOeE5NJWU0aY4NT4VGG2tme+sadIP0EUVtOCwnBsZzuR5ZcwhLNu8GZMdN+dMfoTa8HT2Xb/sOSDLW+N9RqizEYjfm/CsHx1ZdpNPXQVwwYfZ/6K4Yg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243421153884.7242250010738; Fri, 11 Jul 2025 07:17:01 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEYI-0002bj-VJ; Fri, 11 Jul 2025 10:16:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaEUA-00037c-F9 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:23 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaEU8-0006ru-2p for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:22 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 048E21F453; Fri, 11 Jul 2025 14:11:15 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id E554A1388B; Fri, 11 Jul 2025 14:11:13 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id qCHxJoEbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243075; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4mzwXqQN6NKzga9iDDezb4zzzkxloBxPp8r53t2TXtM=; b=ISVNiFwvvXJNMBnx8upjhmqRwjqaKA2hTah3BpZ67aoH916WyB1Lqi3NGvFY1eG94y71v5 kirK1bi+2ljaQz5t0flE4O8dtB3GNqLZzX/jVVtkGcppm5A19bXZmtblRXk7br/nhIKFEA pVOekjKL7H7HdifXflhWCk4olWcw9+I= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243075; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4mzwXqQN6NKzga9iDDezb4zzzkxloBxPp8r53t2TXtM=; b=aRPYjIcSlYzjesWsNchpnJ2kxnFmMF/oZLEZ4wSX0u2AldTYy/sHTew2fgyTZX6jNeYN3T N89YFOs6Fce/HsDQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243075; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4mzwXqQN6NKzga9iDDezb4zzzkxloBxPp8r53t2TXtM=; b=ISVNiFwvvXJNMBnx8upjhmqRwjqaKA2hTah3BpZ67aoH916WyB1Lqi3NGvFY1eG94y71v5 kirK1bi+2ljaQz5t0flE4O8dtB3GNqLZzX/jVVtkGcppm5A19bXZmtblRXk7br/nhIKFEA pVOekjKL7H7HdifXflhWCk4olWcw9+I= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243075; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4mzwXqQN6NKzga9iDDezb4zzzkxloBxPp8r53t2TXtM=; b=aRPYjIcSlYzjesWsNchpnJ2kxnFmMF/oZLEZ4wSX0u2AldTYy/sHTew2fgyTZX6jNeYN3T N89YFOs6Fce/HsDQ== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 21/26] migration/postcopy: Cache the tid->vcpu mapping for blocktime Date: Fri, 11 Jul 2025 11:10:26 -0300 Message-Id: <20250711141031.423-22-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243422901116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Looking up the vCPU index for each fault can be expensive when there're hundreds of vCPUs. Provide a cache for tid->vcpu instead with a hash table, then lookup from there. When at it, add another counter to record how many non-vCPU faults it gets. For example, the main thread can also access a guest page that was missing. These kind of faults are not accounted by blocktime so far. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-11-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/postcopy-ram.c | 68 ++++++++++++++++++++++++++++++++++------ migration/trace-events | 3 +- 2 files changed, 59 insertions(+), 12 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index dd3615663f..bf65d6035c 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -127,6 +127,17 @@ typedef struct PostcopyBlocktimeContext { /* number of vCPU are suspended */ int smp_cpus_down; =20 + /* + * Fast path for looking up vcpu_index from tid. NOTE: this result + * only reflects the vcpu setup when postcopy is running. It may not + * always match with the current vcpu setup because vcpus can be hot + * attached/detached after migration completes. However this should be + * stable when blocktime is using the structure. + */ + GHashTable *tid_to_vcpu_hash; + /* Count of non-vCPU faults. This is only for debugging purpose. */ + uint64_t non_vcpu_faults; + /* * Handler for exit event, necessary for * releasing whole blocktime_ctx @@ -136,6 +147,7 @@ typedef struct PostcopyBlocktimeContext { =20 static void destroy_blocktime_context(struct PostcopyBlocktimeContext *ctx) { + g_hash_table_destroy(ctx->tid_to_vcpu_hash); g_free(ctx->vcpu_blocktime_start); g_free(ctx->vcpu_blocktime_total); g_free(ctx->vcpu_faults_count); @@ -150,6 +162,36 @@ static void migration_exit_cb(Notifier *n, void *data) destroy_blocktime_context(ctx); } =20 +static GHashTable *blocktime_init_tid_to_vcpu_hash(void) +{ + /* + * TID as an unsigned int can be directly used as the key. However, + * CPU index can NOT be directly used as value, because CPU index can + * be 0, which means NULL. Then when lookup we can never know whether + * it's 0 or "not found". Hence use an indirection for CPU index. + */ + GHashTable *table =3D g_hash_table_new_full(g_direct_hash, g_direct_eq= ual, + NULL, g_free); + CPUState *cpu; + + /* + * Initialize the tid->cpu_id mapping for lookups. The caller needs to + * make sure when reaching here the CPU topology is frozen and will be + * stable for the whole blocktime trapping period. + */ + CPU_FOREACH(cpu) { + int *value =3D g_new(int, 1); + + *value =3D cpu->cpu_index; + g_hash_table_insert(table, + GUINT_TO_POINTER((uint32_t)cpu->thread_id), + value); + trace_postcopy_blocktime_tid_cpu_map(cpu->cpu_index, cpu->thread_i= d); + } + + return table; +} + static struct PostcopyBlocktimeContext *blocktime_context_new(void) { MachineState *ms =3D MACHINE(qdev_get_machine()); @@ -160,6 +202,8 @@ static struct PostcopyBlocktimeContext *blocktime_conte= xt_new(void) ctx->vcpu_blocktime_total =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_faults_count =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_addr =3D g_new0(uintptr_t, smp_cpus); + ctx->tid_to_vcpu_hash =3D blocktime_init_tid_to_vcpu_hash(); + ctx->exit_notifier.notify =3D migration_exit_cb; qemu_add_exit_notifier(&ctx->exit_notifier); =20 @@ -827,18 +871,21 @@ int postcopy_request_shared_page(struct PostCopyFD *p= cfd, RAMBlock *rb, return 0; } =20 -static int get_mem_fault_cpu_index(uint32_t pid) +static int blocktime_get_vcpu(PostcopyBlocktimeContext *ctx, uint32_t tid) { - CPUState *cpu_iter; + int *found; =20 - CPU_FOREACH(cpu_iter) { - if (cpu_iter->thread_id =3D=3D pid) { - trace_get_mem_fault_cpu_index(cpu_iter->cpu_index, pid); - return cpu_iter->cpu_index; - } + found =3D g_hash_table_lookup(ctx->tid_to_vcpu_hash, GUINT_TO_POINTER(= tid)); + if (!found) { + /* + * NOTE: this is possible, because QEMU's non-vCPU threads can + * also access a missing page. Or, when KVM async pf is enabled, a + * fault can even happen from a kworker.. + */ + return -1; } - trace_get_mem_fault_cpu_index(-1, pid); - return -1; + + return *found; } =20 static uint64_t get_current_ns(void) @@ -865,8 +912,9 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint= 32_t ptid, if (!dc || ptid =3D=3D 0) { return; } - cpu =3D get_mem_fault_cpu_index(ptid); + cpu =3D blocktime_get_vcpu(dc, ptid); if (cpu < 0) { + dc->non_vcpu_faults++; return; } =20 diff --git a/migration/trace-events b/migration/trace-events index 02cdb6e7cc..9c1f3b7044 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -310,8 +310,7 @@ postcopy_preempt_tls_handshake(void) "" postcopy_preempt_new_channel(void) "" postcopy_preempt_thread_entry(void) "" postcopy_preempt_thread_exit(void) "" - -get_mem_fault_cpu_index(int cpu, uint32_t pid) "cpu: %d, pid: %u" +postcopy_blocktime_tid_cpu_map(int cpu, uint32_t tid) "cpu: %d, tid: %u" =20 # exec.c migration_exec_outgoing(const char *cmd) "cmd=3D%s" --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243772; cv=none; d=zohomail.com; s=zohoarc; b=H/KSdynt4euumXHov+4NxkyFFxgV1Y7rFCETwQwSnjhNKIcmxmhAmzXMD15yrgZ0QKstcFREbJFf4y1ibAwCp5sKOwGmBzNky1AJrcLwYWNXh6STk6ZebVDZGlukfymTII3zjDxZqO31X7LTFVJSq+uw1LvAX8cTC6EjC7sQF6A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243772; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=UWNsbznBa5U705aIiyQeomhLV9xkwjgD4c9PNhvCW+A=; b=ZuBXodaLYLAGGZuKm9eeKcMi/bF/BDX8mOeZ0J7xGgqPZ95jImU1CnmFrI9+0N+vBtKX0PuSIGJ4Aq6iL9hmoAsDbJo733kOyJTVZPCdWgEMkOEd2qUOMqQdMvLpIR2goblnA7IyhkV72SUL2gIvhouMJb1uIAgwU3x58HcHRr8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243772435672.6920235976913; Fri, 11 Jul 2025 07:22:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEcA-0000R6-Sg; Fri, 11 Jul 2025 10:20:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaEUG-0003Al-SV for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:33 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaEUE-0006so-G3 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:28 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 97F921F46E; Fri, 11 Jul 2025 14:11:16 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 836191388B; Fri, 11 Jul 2025 14:11:15 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id UMK1D4MbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243076; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UWNsbznBa5U705aIiyQeomhLV9xkwjgD4c9PNhvCW+A=; b=q7vBqWO1gicb2ohYijgWxAayORK5P1GUlZWivxUE6xagWUdWp2vfGrEo/zZug0NnFGUrYP KumiCHD5DwNO2OD2CK41D24gK4vK7TP7s/ofihwDxCe3OTvQgsl9Y8zwfzlQOvkR1nv3jR 5XpU+aRzMIC9xsXGuMNQ7Ezv/Ct/3rA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243076; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UWNsbznBa5U705aIiyQeomhLV9xkwjgD4c9PNhvCW+A=; b=O+VRGvt8ANLGGtbBTFrvnetYIXZnYUEu+g1u0jmDlDSKMRFpbDfXlsZ1yhuUy3aK9PBIt/ r5uxPERO6/DhXBAA== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=q7vBqWO1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=O+VRGvt8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243076; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UWNsbznBa5U705aIiyQeomhLV9xkwjgD4c9PNhvCW+A=; b=q7vBqWO1gicb2ohYijgWxAayORK5P1GUlZWivxUE6xagWUdWp2vfGrEo/zZug0NnFGUrYP KumiCHD5DwNO2OD2CK41D24gK4vK7TP7s/ofihwDxCe3OTvQgsl9Y8zwfzlQOvkR1nv3jR 5XpU+aRzMIC9xsXGuMNQ7Ezv/Ct/3rA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243076; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UWNsbznBa5U705aIiyQeomhLV9xkwjgD4c9PNhvCW+A=; b=O+VRGvt8ANLGGtbBTFrvnetYIXZnYUEu+g1u0jmDlDSKMRFpbDfXlsZ1yhuUy3aK9PBIt/ r5uxPERO6/DhXBAA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 22/26] migration/postcopy: Cleanup the total blocktime accounting Date: Fri, 11 Jul 2025 11:10:27 -0300 Message-Id: <20250711141031.423-23-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RCPT_COUNT_TWO(0.00)[2]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 97F921F46E X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243774609116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu The variable vcpu_total_blocktime isn't easy to follow. In reality, it wants to capture the case where all vCPUs are stopped, and now there will be some vCPUs starts running. The name now starts to conflict with vcpu_blocktime_total[], meanwhile it's actually not necessary to have the variable at all: since nobody is touching smp_cpus_down except ourselves, we can safely do the calculation at the end before decrementing smp_cpus_down. Hopefully this makes the logic easier to read, side benefit is we drop one temp var. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-12-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/postcopy-ram.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index bf65d6035c..fd6c0bdb1e 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -972,7 +972,6 @@ static void mark_postcopy_blocktime_end(uintptr_t addr) MachineState *ms =3D MACHINE(qdev_get_machine()); unsigned int smp_cpus =3D ms->smp.cpus; int i, affected_cpu =3D 0; - bool vcpu_total_blocktime =3D false; uint64_t read_vcpu_time, current; =20 if (!dc) { @@ -994,20 +993,19 @@ static void mark_postcopy_blocktime_end(uintptr_t add= r) dc->vcpu_addr[i] =3D 0; vcpu_blocktime =3D current - read_vcpu_time; affected_cpu +=3D 1; - /* we need to know is that mark_postcopy_end was due to - * faulted page, another possible case it's prefetched - * page and in that case we shouldn't be here */ - if (!vcpu_total_blocktime && dc->smp_cpus_down =3D=3D smp_cpus) { - vcpu_total_blocktime =3D true; - } /* continue cycle, due to one page could affect several vCPUs */ dc->vcpu_blocktime_total[i] +=3D vcpu_blocktime; } =20 - dc->smp_cpus_down -=3D affected_cpu; - if (vcpu_total_blocktime) { + /* + * If all vCPUs used to be down, and copying this page would free some + * vCPUs, then the system-level blocktime ends here. + */ + if (dc->smp_cpus_down =3D=3D smp_cpus && affected_cpu) { dc->total_blocktime +=3D current - dc->last_begin; } + dc->smp_cpus_down -=3D affected_cpu; + trace_mark_postcopy_blocktime_end(addr, dc->total_blocktime, affected_cpu); } --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243645; cv=none; d=zohomail.com; s=zohoarc; b=Af7wmSHihSpzCZ/S2k9ZywatZ5L8JNOjbFsIyrdmAtnCKXB45JYdy+N1NF+Eq7Qp/0OBR9ZWv2DbWva7/17fznjTy2e0LsJvnsM4QQocWJRkeupGLNeQWH6McszvnErmzh4SWZordvcQZLap4nRCQOHYj+SZDZ1Mbsp7vqCb3o4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243645; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=yahr4xTDonbuFFFNBGo8xqPBuH7E+nTTPkcHPHdeRHM=; b=ZucO8hIsvXgG36xYLkESjO4wx3puYdmanEzSCOaGImdOiN7wQ/RNZUTGJhIf9l5tlT7mxlREZqIj3gF+L/scT0ucFv93lRYHdEBVKFBYQxb4ur3jCkUtRJjOJMPgw+IEV5wd/2Py0doYDTQtSLrL3l7xrGBSmUZrTLGAM6Foe4k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243645727558.1298205848312; Fri, 11 Jul 2025 07:20:45 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEXI-0001q5-T6; Fri, 11 Jul 2025 10:15:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETs-0002kw-85 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:09 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETo-0006of-ID for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:03 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 31C43211A5; Fri, 11 Jul 2025 14:11:18 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1EFCE1388B; Fri, 11 Jul 2025 14:11:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id yFenM4QbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:16 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243078; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yahr4xTDonbuFFFNBGo8xqPBuH7E+nTTPkcHPHdeRHM=; b=sxqkyAC+85vhlGGdxVXgmQEND0aRP8xyeozCD1Jtis8kxH1U1jbHoWJZ13/k4jkBqgXMMS gEQfuybfvRYb9C7elNYOIaMNM6aTmx60qCrxqfBMITxo4M3oZOCxZpGbW2SKiIwayuAVBO BDQCzHdHfvtKKAYpVzCrTVmuHrQIRtc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243078; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yahr4xTDonbuFFFNBGo8xqPBuH7E+nTTPkcHPHdeRHM=; b=3B+RyH5hPI1OSoz+nKyxTwxPDCxJekmAQMNlLHq+sG8OgJVU1EBZ+sEK3oX6+vRIYsUAK+ o0e0mPuu7FV+QMAw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=sxqkyAC+; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=3B+RyH5h DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243078; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yahr4xTDonbuFFFNBGo8xqPBuH7E+nTTPkcHPHdeRHM=; b=sxqkyAC+85vhlGGdxVXgmQEND0aRP8xyeozCD1Jtis8kxH1U1jbHoWJZ13/k4jkBqgXMMS gEQfuybfvRYb9C7elNYOIaMNM6aTmx60qCrxqfBMITxo4M3oZOCxZpGbW2SKiIwayuAVBO BDQCzHdHfvtKKAYpVzCrTVmuHrQIRtc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243078; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yahr4xTDonbuFFFNBGo8xqPBuH7E+nTTPkcHPHdeRHM=; b=3B+RyH5hPI1OSoz+nKyxTwxPDCxJekmAQMNlLHq+sG8OgJVU1EBZ+sEK3oX6+vRIYsUAK+ o0e0mPuu7FV+QMAw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu Subject: [PULL 23/26] migration/postcopy: Optimize blocktime fault tracking with hashtable Date: Fri, 11 Jul 2025 11:10:28 -0300 Message-Id: <20250711141031.423-24-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RCPT_COUNT_TWO(0.00)[2]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:mid,suse.de:dkim,suse.de:email]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 31C43211A5 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243647107116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Currently, the postcopy blocktime feature maintains vCPU fault information using an array (vcpu_addr[]). It has two issues. Issue 1: Performance Concern =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D The old algorithm was almost OK and fast on inserts, except that the lookup is slow and won't scale if there are a lot of vCPUs: when a page is copied during postcopy, mark_postcopy_blocktime_end() will walk the whole array trying to find which vCPUs are blocked by the address. So it needs constant O(N) walk for each page resolution. Alexey (the author of postcopy blocktime) mentioned the perf issue and how to optimize it in a piece of comment in the page resolution path. The comment was (interestingly..) not complete, but it's relatively clear what he wanted to say about this perf issue. Issue 2: Wrong Accounting on re-entrancies =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D People might think that each vCPU should only and always get one fault at a time, so that when the blocktime layer captured one fault on one vCPU, we should never see another fault message on this vCPU. It's almost correct, except in some extreme rare cases. Case 1: it's possible the fault thread processes the userfaultfd messages too fast so it can see >1 messages on one vCPU before the previous one was resolved. Case 2: it's theoretically also possible one vCPU can get even more than one message on the same fault address if a fault is retried by the kernel (e.g., handle_userfault() got interrupted before page resolution). As this info might be important, instead of using commit message, I put more details into the code as comment, when introducing an array maintaining concurrent faults on one vCPU. Please refer to the comments for details on both cases, especially case 1 which can be tricky. Case 1 sounds rare, but it can be easily reproduced locally for me when we run blocktime together with the migration-test on the vanilla postcopy. New Design =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D This patch should do almost what Alexey mentioned, but slightly differently: instead of having an array to maintain vCPU fault addresses, for each of the fault message we push a message into a hash, indexed by the fault address. With the hash, it can replace the old two structs: both the vcpu_addr[] array, and also the array to store the start time of the fault. However due to above we need one more counter array to account concurrent faults on the same vCPU - that should even be needed in the old code, it's just that the old code was buggy and it will blindly overwrite an existing entry.. now we'll start to really track everything. The hash structure might be more efficient than tree to maintain such addr->(cpu, fault_time) information, so that the insert() and lookup() paths should ideally both be ~O(1). After all, we do not need to sort. Here we need to do one remove() though after the lookup(). It could be slow but only if many vCPUs faulted exactly on the same address (so when the list of cpu entries is long), which should be unlikely. Even with that, it's still a worst case O(N) (consider 400 vCPUs faulted on the same address and how likely is it..) rather than a constant O(N) complexity. When at it, touch up the tracepoints to make them slightly more useful. One tracepoint is added when walking all the fault entries. Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-13-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/postcopy-ram.c | 265 +++++++++++++++++++++++++++++++-------- migration/trace-events | 5 +- 2 files changed, 219 insertions(+), 51 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index fd6c0bdb1e..91c23b446e 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -112,14 +112,69 @@ void postcopy_thread_create(MigrationIncomingState *m= is, =20 /* All the time records are in unit of nanoseconds */ typedef struct PostcopyBlocktimeContext { - /* time when page fault initiated per vCPU */ - uint64_t *vcpu_blocktime_start; /* blocktime per vCPU */ uint64_t *vcpu_blocktime_total; /* count of faults per vCPU */ uint64_t *vcpu_faults_count; - /* page address per vCPU */ - uintptr_t *vcpu_addr; + /* + * count of currently blocked faults per vCPU. + * + * NOTE: Normally there should only be one fault in-progress per vCPU + * thread, so logically it _seems_ vcpu_faults_count[] for any vCPU + * should be either zero or one. However, there can be reasons we see + * >1 faults on the same vCPU thread. + * + * CASE (1): since the process to resolve faults (ioctl(UFFDIO_COPY), + * for example) is done before taking the mutex that protects the + * blocktime context, it can happen that we read more than one faulted + * addresses per vCPU. + * + * One example when we can see >1 faulted addresses for one vCPU: + * + * vcpu1 thread fault thread resolve thread + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * faulted on addr1 + * read uffd msg (addr1) + * MUTEX_LOCK + * add entry (cpu1, addr1) + * MUTEX_UNLOCK + * request remote fault (addr1) + * resolve fault (addr1) + * addr1 resolved, continue.. + * faulted on addr2 + * read uffd msg (addr2) + * MUTEX_LOCK + * add entry (cpu1, addr2) <--------------- [A] + * MUTEX_UNLOCK + * MUTEX_LOCK + * remove entry (cpu1, addr1) + * MUTEX_UNLOCK + * + * In above case, we may see (cpu1, addr1) and (cpu1, addr2) entries to + * appear together at [A], when it gets the lock before the resolve + * thread. Use this counter to maintain such case, and only when it + * reaches zero we know the vCPU is not blocked anymore. + * + * CASE (2): theoretically (the author admit to not have verified + * this..), one vCPU thread can also generate more than one userfaultfd + * message on the same address. It can happen e.g. for whatever reason + * the fault got retried before a resolution arrives. In that extremely + * rare case, we could also see two (cpu1, addr1) entries. + * + * In all cases, be prepared with such re-entrancies with this array. + * + * Using uint8_t should be far enough for now. For example, when + * there're only one resolve thread (postcopy ram listening thread), + * the max (concurrent fault entries) should be two. + */ + uint8_t *vcpu_faults_current; + /* + * The hash that contains addr1->[(cpu1,ts1),(cpu2,ts2) ...] mappings. + * Each of the entry is a tuple of (CPU index, fault timestamp) showing + * that a fault was requested. + */ + GHashTable *vcpu_addr_hash; /* total blocktime when all vCPUs are stopped */ uint64_t total_blocktime; /* point in time when last page fault was initiated */ @@ -145,13 +200,38 @@ typedef struct PostcopyBlocktimeContext { Notifier exit_notifier; } PostcopyBlocktimeContext; =20 +typedef struct { + /* The time the fault was triggered */ + uint64_t fault_time; + /* The vCPU index that was blocked */ + int cpu; +} BlocktimeVCPUEntry; + +/* Alloc an entry to record a vCPU fault */ +static BlocktimeVCPUEntry * +blocktime_vcpu_entry_alloc(int cpu, uint64_t fault_time) +{ + BlocktimeVCPUEntry *entry =3D g_new(BlocktimeVCPUEntry, 1); + + entry->fault_time =3D fault_time; + entry->cpu =3D cpu; + + return entry; +} + +/* Free a @GList of @BlocktimeVCPUEntry */ +static void blocktime_vcpu_list_free(gpointer data) +{ + g_list_free_full(data, g_free); +} + static void destroy_blocktime_context(struct PostcopyBlocktimeContext *ctx) { g_hash_table_destroy(ctx->tid_to_vcpu_hash); - g_free(ctx->vcpu_blocktime_start); + g_hash_table_destroy(ctx->vcpu_addr_hash); g_free(ctx->vcpu_blocktime_total); g_free(ctx->vcpu_faults_count); - g_free(ctx->vcpu_addr); + g_free(ctx->vcpu_faults_current); g_free(ctx); } =20 @@ -198,12 +278,22 @@ static struct PostcopyBlocktimeContext *blocktime_con= text_new(void) unsigned int smp_cpus =3D ms->smp.cpus; PostcopyBlocktimeContext *ctx =3D g_new0(PostcopyBlocktimeContext, 1); =20 - ctx->vcpu_blocktime_start =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_blocktime_total =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_faults_count =3D g_new0(uint64_t, smp_cpus); - ctx->vcpu_addr =3D g_new0(uintptr_t, smp_cpus); + ctx->vcpu_faults_current =3D g_new0(uint8_t, smp_cpus); ctx->tid_to_vcpu_hash =3D blocktime_init_tid_to_vcpu_hash(); =20 + /* + * The key (host virtual addresses) will always be gpointer-sized on + * either 32bits or 64bits systems, so it'll fit as a direct key. + * + * The value will be a list of BlocktimeVCPUEntry entries. + */ + ctx->vcpu_addr_hash =3D g_hash_table_new_full(g_direct_hash, + g_direct_equal, + NULL, + blocktime_vcpu_list_free); + ctx->exit_notifier.notify =3D migration_exit_cb; qemu_add_exit_notifier(&ctx->exit_notifier); =20 @@ -893,6 +983,39 @@ static uint64_t get_current_ns(void) return (uint64_t)qemu_clock_get_ns(QEMU_CLOCK_REALTIME); } =20 +/* Inject an (cpu, fault_time) entry into the database, using addr as key = */ +static void blocktime_fault_inject(PostcopyBlocktimeContext *ctx, + uintptr_t addr, int cpu, uint64_t time) +{ + BlocktimeVCPUEntry *entry =3D blocktime_vcpu_entry_alloc(cpu, time); + GHashTable *table =3D ctx->vcpu_addr_hash; + gpointer key =3D (gpointer)addr; + GList *head, *list; + gboolean result; + + head =3D g_hash_table_lookup(table, key); + if (head) { + /* + * If existed, steal the @head for list operation rather than + * freeing it, making sure steal succeeded. + */ + result =3D g_hash_table_steal(table, key); + assert(result =3D=3D TRUE); + } + + /* + * Now the key is guaranteed to be absent. Two cases: + * + * (1) There's no existing entry, list contains the only one. Insert. + * (2) There're existing entries, after stealing we own it, prepend the + * result and re-insert. + */ + list =3D g_list_prepend(head, entry); + g_hash_table_insert(table, key, list); + + trace_postcopy_blocktime_begin(addr, time, cpu, !!head); +} + /* * This function is being called when pagefault occurs. It tracks down vCPU * blocking time. It's protected by @page_request_mutex. @@ -912,21 +1035,6 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, ui= nt32_t ptid, if (!dc || ptid =3D=3D 0) { return; } - cpu =3D blocktime_get_vcpu(dc, ptid); - if (cpu < 0) { - dc->non_vcpu_faults++; - return; - } - - current =3D get_current_ns(); - if (dc->vcpu_addr[cpu] =3D=3D 0) { - dc->smp_cpus_down++; - } - - dc->last_begin =3D current; - dc->vcpu_blocktime_start[cpu] =3D current; - dc->vcpu_addr[cpu] =3D addr; - dc->vcpu_faults_count[cpu]++; =20 /* * The caller should only inject a blocktime entry when the page is @@ -934,8 +1042,67 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, ui= nt32_t ptid, */ assert(!ramblock_recv_bitmap_test(rb, (void *)addr)); =20 - trace_mark_postcopy_blocktime_begin(addr, dc->vcpu_blocktime_start[cpu= ], - cpu); + current =3D get_current_ns(); + cpu =3D blocktime_get_vcpu(dc, ptid); + + if (cpu >=3D 0) { + /* How many faults on this vCPU in total? */ + dc->vcpu_faults_count[cpu]++; + + /* + * Account how many concurrent faults on this vCPU we trapped. See + * comments above vcpu_faults_current[] on why it can be more than= one. + */ + if (dc->vcpu_faults_current[cpu]++ =3D=3D 0) { + dc->smp_cpus_down++; + /* + * We use last_begin to cover (1) the 1st fault on this specif= ic + * vCPU, but meanwhile (2) the last vCPU that got blocked. It= 's + * only used to calculate system-wide blocktime. + */ + dc->last_begin =3D current; + } + + /* Making sure it won't overflow - it really should never! */ + assert(dc->vcpu_faults_current[cpu] <=3D 255); + } else { + /* We do not support non-vCPU thread tracking yet */ + dc->non_vcpu_faults++; + return; + } + + blocktime_fault_inject(dc, addr, cpu, current); +} + +typedef struct { + PostcopyBlocktimeContext *ctx; + uint64_t current; + int affected_cpus; +} BlockTimeVCPUIter; + +static void blocktime_cpu_list_iter_fn(gpointer data, gpointer user_data) +{ + BlockTimeVCPUIter *iter =3D user_data; + PostcopyBlocktimeContext *ctx =3D iter->ctx; + BlocktimeVCPUEntry *entry =3D data; + int cpu =3D entry->cpu; + + /* + * Time should never go back.. so when the fault is resolved it must be + * later than when it was faulted. + */ + assert(iter->current >=3D entry->fault_time); + + /* + * If we resolved all pending faults on one vCPU due to this page + * resolution, take a note. + */ + if (--ctx->vcpu_faults_current[cpu] =3D=3D 0) { + ctx->vcpu_blocktime_total[cpu] +=3D iter->current - entry->fault_t= ime; + iter->affected_cpus +=3D 1; + } + + trace_postcopy_blocktime_end_one(cpu, ctx->vcpu_faults_current[cpu]); } =20 /* @@ -971,43 +1138,43 @@ static void mark_postcopy_blocktime_end(uintptr_t ad= dr) PostcopyBlocktimeContext *dc =3D mis->blocktime_ctx; MachineState *ms =3D MACHINE(qdev_get_machine()); unsigned int smp_cpus =3D ms->smp.cpus; - int i, affected_cpu =3D 0; - uint64_t read_vcpu_time, current; + BlockTimeVCPUIter iter =3D { + .current =3D get_current_ns(), + .affected_cpus =3D 0, + .ctx =3D dc, + }; + gpointer key =3D (gpointer)addr; + GHashTable *table; + GList *list; =20 if (!dc) { return; } =20 - current =3D get_current_ns(); - /* lookup cpu, to clear it, - * that algorithm looks straightforward, but it's not - * optimal, more optimal algorithm is keeping tree or hash - * where key is address value is a list of */ - for (i =3D 0; i < smp_cpus; i++) { - uint64_t vcpu_blocktime =3D 0; - - read_vcpu_time =3D dc->vcpu_blocktime_start[i]; - if (dc->vcpu_addr[i] !=3D addr || read_vcpu_time =3D=3D 0) { - continue; - } - dc->vcpu_addr[i] =3D 0; - vcpu_blocktime =3D current - read_vcpu_time; - affected_cpu +=3D 1; - /* continue cycle, due to one page could affect several vCPUs */ - dc->vcpu_blocktime_total[i] +=3D vcpu_blocktime; + table =3D dc->vcpu_addr_hash; + /* the address wasn't tracked at all? */ + list =3D g_hash_table_lookup(table, key); + if (!list) { + return; } =20 + /* + * Loop over the set of vCPUs that got blocked on this addr, do the + * blocktime accounting. After that, remove the whole list. + */ + g_list_foreach(list, blocktime_cpu_list_iter_fn, &iter); + g_hash_table_remove(table, key); + /* * If all vCPUs used to be down, and copying this page would free some * vCPUs, then the system-level blocktime ends here. */ - if (dc->smp_cpus_down =3D=3D smp_cpus && affected_cpu) { - dc->total_blocktime +=3D current - dc->last_begin; + if (dc->smp_cpus_down =3D=3D smp_cpus && iter.affected_cpus) { + dc->total_blocktime +=3D iter.current - dc->last_begin; } - dc->smp_cpus_down -=3D affected_cpu; + dc->smp_cpus_down -=3D iter.affected_cpus; =20 - trace_mark_postcopy_blocktime_end(addr, dc->total_blocktime, - affected_cpu); + trace_postcopy_blocktime_end(addr, iter.current, iter.affected_cpus); } =20 static void postcopy_pause_fault_thread(MigrationIncomingState *mis) diff --git a/migration/trace-events b/migration/trace-events index 9c1f3b7044..a36a78f01a 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -285,8 +285,6 @@ postcopy_nhp_range(const char *ramblock, void *host_add= r, size_t offset, size_t postcopy_place_page(void *host_addr) "host=3D%p" postcopy_place_page_zero(void *host_addr) "host=3D%p" postcopy_ram_enable_notify(void) "" -mark_postcopy_blocktime_begin(uint64_t addr, uint64_t time, int cpu) "addr= : 0x%" PRIx64 ", time: %" PRIu64 ", cpu: %d" -mark_postcopy_blocktime_end(uint64_t addr, uint64_t time, int affected_cpu= ) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", affected_cpus: %d" postcopy_pause_fault_thread(void) "" postcopy_pause_fault_thread_continued(void) "" postcopy_pause_fast_load(void) "" @@ -311,6 +309,9 @@ postcopy_preempt_new_channel(void) "" postcopy_preempt_thread_entry(void) "" postcopy_preempt_thread_exit(void) "" postcopy_blocktime_tid_cpu_map(int cpu, uint32_t tid) "cpu: %d, tid: %u" +postcopy_blocktime_begin(uint64_t addr, uint64_t time, int cpu, bool exist= s) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", cpu: %d, exist: %d" +postcopy_blocktime_end(uint64_t addr, uint64_t time, int affected_cpu) "ad= dr: 0x%" PRIx64 ", time: %" PRIu64 ", affected_cpus: %d" +postcopy_blocktime_end_one(int cpu, uint8_t left_faults) "cpu: %d, left_fa= ults: %" PRIu8 =20 # exec.c migration_exec_outgoing(const char *cmd) "cmd=3D%s" --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243813; cv=none; d=zohomail.com; s=zohoarc; b=LjLZKlbUCDXvlrvNHa4g8iYYIaCiSr9p/zYsxUkAPvqvX/+aq3Q947EIa2bVeBrBU+i0xxwK7fY3pdhdkRpuuEGlvR9pG8qmBv/zJsslbtnVCktbuc2lWeXcNLYHywmzTnkFcnEkb66m3nlcrVzLLprPs9TJTe/CkGbtF0Gcvdc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243813; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=FXrUzXE+qpxN513L6w3Tqatg0QrA1uwNuuyVmMVtD84=; b=W6DspMKnjih5tQX0wCjOKq77dBT8ALxRhfXwr3dSDQjpMTGRDUgZIqGkv1VJxWR3f6MwC9FRcLszQGJ+OkjUQ0rKi/EY4SRhdFL72ZMPSxxyoyusSdgI3V/1/G85ldOwM0JQrj5ymFvgvpOE3Z5yB50HW+JanwwOjPw9D8kV6GM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243813452840.8494539980857; Fri, 11 Jul 2025 07:23:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEdt-0004B9-8J; Fri, 11 Jul 2025 10:22:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaETy-0002t1-IQ for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:16 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaETw-0006qG-4L for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:10 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9846E211A1; Fri, 11 Jul 2025 14:11:20 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B386D1388B; Fri, 11 Jul 2025 14:11:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id yF6qGoYbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:18 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243080; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FXrUzXE+qpxN513L6w3Tqatg0QrA1uwNuuyVmMVtD84=; b=Yrq/k7JBhS5yt7ZVfvWJIvA38BfvBuSItusjcTj6c8ofocAMHp2SfrefC/rmJvnekMT8zk QEM7qwnWZpukI2LQrd+S+65rjK+b8QTNSsjCYmZkt3PgmakEKAUsVqti44XRkBt1BlIw1D igGBsgSwiWcxbsjbv1cCSMzdIvOtszs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243080; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FXrUzXE+qpxN513L6w3Tqatg0QrA1uwNuuyVmMVtD84=; b=gnlD6qkz3GJ9qa2sV+IGlV+Ncbb5Bv2WMm3deYOJNXRER2xG8c8KGWsGFhsMHSae/eO2JP B8/W9zfws5ce0aAg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243080; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FXrUzXE+qpxN513L6w3Tqatg0QrA1uwNuuyVmMVtD84=; b=Yrq/k7JBhS5yt7ZVfvWJIvA38BfvBuSItusjcTj6c8ofocAMHp2SfrefC/rmJvnekMT8zk QEM7qwnWZpukI2LQrd+S+65rjK+b8QTNSsjCYmZkt3PgmakEKAUsVqti44XRkBt1BlIw1D igGBsgSwiWcxbsjbv1cCSMzdIvOtszs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243080; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FXrUzXE+qpxN513L6w3Tqatg0QrA1uwNuuyVmMVtD84=; b=gnlD6qkz3GJ9qa2sV+IGlV+Ncbb5Bv2WMm3deYOJNXRER2xG8c8KGWsGFhsMHSae/eO2JP B8/W9zfws5ce0aAg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Markus Armbruster , "Dr . David Alan Gilbert" , Mario Casquero Subject: [PULL 24/26] migration/postcopy: blocktime allows track / report non-vCPU faults Date: Fri, 11 Jul 2025 11:10:29 -0300 Message-Id: <20250711141031.423-25-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo, treblig.org:email, suse.de:email, suse.de:mid] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243815388116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu When used to report page fault latencies, the blocktime feature can be almost useless when KVM async page fault is enabled, because in most cases such remote fault will kickoff async page faults, then it's not trackable from blocktime layer. After all these recent rewrites to blocktime layer, it's finally so easy to also support tracking non-vCPU faults. It'll be even faster if we could always index fault records with TIDs, unfortunately we need to maintain the blocktime API which report things in vCPU indexes. Of course this can work not only for kworkers, but also any guest accesses that may reach a missing page, for example, very likely when in the QEMU main thread too (and all other threads whenever applicable). In this case, we don't care about "how long the threads are blocked", but we only care about "how long the fault will be resolved". Cc: Markus Armbruster Cc: Dr. David Alan Gilbert Reviewed-by: Fabiano Rosas Tested-by: Mario Casquero Link: https://lore.kernel.org/r/20250613141217.474825-14-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration-hmp-cmds.c | 5 +++ migration/postcopy-ram.c | 64 +++++++++++++++++++++------ migration/trace-events | 2 +- qapi/migration.json | 12 ++++- tests/qtest/migration/migration-qmp.c | 1 + 5 files changed, 67 insertions(+), 17 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 8b3846dab5..e1f9530520 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -80,6 +80,11 @@ static void migration_dump_blocktime(Monitor *mon, Migra= tionInfo *info) info->postcopy_latency); } =20 + if (info->has_postcopy_non_vcpu_latency) { + monitor_printf(mon, "Postcopy non-vCPU Latencies (ns): %" PRIu64 "= \n", + info->postcopy_non_vcpu_latency); + } + if (info->has_postcopy_vcpu_latency) { uint64List *item =3D info->postcopy_vcpu_latency; const char *sep =3D ""; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 91c23b446e..f4cb23b3e0 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -192,6 +192,8 @@ typedef struct PostcopyBlocktimeContext { GHashTable *tid_to_vcpu_hash; /* Count of non-vCPU faults. This is only for debugging purpose. */ uint64_t non_vcpu_faults; + /* total blocktime when a non-vCPU thread is stopped */ + uint64_t non_vcpu_blocktime_total; =20 /* * Handler for exit event, necessary for @@ -203,7 +205,10 @@ typedef struct PostcopyBlocktimeContext { typedef struct { /* The time the fault was triggered */ uint64_t fault_time; - /* The vCPU index that was blocked */ + /* + * The vCPU index that was blocked, when cpu=3D=3D-1, it means it's a + * fault from non-vCPU threads. + */ int cpu; } BlocktimeVCPUEntry; =20 @@ -344,6 +349,12 @@ void fill_destination_postcopy_migration_info(Migratio= nInfo *info) QAPI_LIST_PREPEND(list_latency, latency); } =20 + latency_total +=3D bc->non_vcpu_blocktime_total; + faults +=3D bc->non_vcpu_faults; + + info->has_postcopy_non_vcpu_latency =3D true; + info->postcopy_non_vcpu_latency =3D bc->non_vcpu_faults ? + (bc->non_vcpu_blocktime_total / bc->non_vcpu_faults) : 0; info->has_postcopy_blocktime =3D true; /* Convert ns -> ms */ info->postcopy_blocktime =3D (uint32_t)(bc->total_blocktime / SCALE_MS= ); @@ -983,7 +994,10 @@ static uint64_t get_current_ns(void) return (uint64_t)qemu_clock_get_ns(QEMU_CLOCK_REALTIME); } =20 -/* Inject an (cpu, fault_time) entry into the database, using addr as key = */ +/* + * Inject an (cpu, fault_time) entry into the database, using addr as key. + * When cpu=3D=3D-1, it means it's a non-vCPU fault. + */ static void blocktime_fault_inject(PostcopyBlocktimeContext *ctx, uintptr_t addr, int cpu, uint64_t time) { @@ -1066,9 +1080,17 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, u= int32_t ptid, /* Making sure it won't overflow - it really should never! */ assert(dc->vcpu_faults_current[cpu] <=3D 255); } else { - /* We do not support non-vCPU thread tracking yet */ + /* + * For non-vCPU thread faults, we don't care about tid or cpu index + * or time the thread is blocked (e.g., a kworker trying to help + * KVM when async_pf=3Don is OK to be blocked and not affect guest + * responsiveness), but we care about latency. Track it with + * cpu=3D-1. + * + * Note that this will NOT affect blocktime reports on vCPU being + * blocked, but only about system-wide latency reports. + */ dc->non_vcpu_faults++; - return; } =20 blocktime_fault_inject(dc, addr, cpu, current); @@ -1078,6 +1100,7 @@ typedef struct { PostcopyBlocktimeContext *ctx; uint64_t current; int affected_cpus; + int affected_non_cpus; } BlockTimeVCPUIter; =20 static void blocktime_cpu_list_iter_fn(gpointer data, gpointer user_data) @@ -1085,6 +1108,7 @@ static void blocktime_cpu_list_iter_fn(gpointer data,= gpointer user_data) BlockTimeVCPUIter *iter =3D user_data; PostcopyBlocktimeContext *ctx =3D iter->ctx; BlocktimeVCPUEntry *entry =3D data; + uint64_t time_passed; int cpu =3D entry->cpu; =20 /* @@ -1092,17 +1116,27 @@ static void blocktime_cpu_list_iter_fn(gpointer dat= a, gpointer user_data) * later than when it was faulted. */ assert(iter->current >=3D entry->fault_time); + time_passed =3D iter->current - entry->fault_time; =20 - /* - * If we resolved all pending faults on one vCPU due to this page - * resolution, take a note. - */ - if (--ctx->vcpu_faults_current[cpu] =3D=3D 0) { - ctx->vcpu_blocktime_total[cpu] +=3D iter->current - entry->fault_t= ime; - iter->affected_cpus +=3D 1; + if (cpu >=3D 0) { + /* + * If we resolved all pending faults on one vCPU due to this page + * resolution, take a note. + */ + if (--ctx->vcpu_faults_current[cpu] =3D=3D 0) { + ctx->vcpu_blocktime_total[cpu] +=3D time_passed; + iter->affected_cpus +=3D 1; + } + trace_postcopy_blocktime_end_one(cpu, ctx->vcpu_faults_current[cpu= ]); + } else { + iter->affected_non_cpus++; + ctx->non_vcpu_blocktime_total +=3D time_passed; + /* + * We do not maintain how many pending non-vCPU faults because we + * do not care about blocktime, only latency. + */ + trace_postcopy_blocktime_end_one(-1, 0); } - - trace_postcopy_blocktime_end_one(cpu, ctx->vcpu_faults_current[cpu]); } =20 /* @@ -1141,6 +1175,7 @@ static void mark_postcopy_blocktime_end(uintptr_t add= r) BlockTimeVCPUIter iter =3D { .current =3D get_current_ns(), .affected_cpus =3D 0, + .affected_non_cpus =3D 0, .ctx =3D dc, }; gpointer key =3D (gpointer)addr; @@ -1174,7 +1209,8 @@ static void mark_postcopy_blocktime_end(uintptr_t add= r) } dc->smp_cpus_down -=3D iter.affected_cpus; =20 - trace_postcopy_blocktime_end(addr, iter.current, iter.affected_cpus); + trace_postcopy_blocktime_end(addr, iter.current, iter.affected_cpus, + iter.affected_non_cpus); } =20 static void postcopy_pause_fault_thread(MigrationIncomingState *mis) diff --git a/migration/trace-events b/migration/trace-events index a36a78f01a..706db97def 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -310,7 +310,7 @@ postcopy_preempt_thread_entry(void) "" postcopy_preempt_thread_exit(void) "" postcopy_blocktime_tid_cpu_map(int cpu, uint32_t tid) "cpu: %d, tid: %u" postcopy_blocktime_begin(uint64_t addr, uint64_t time, int cpu, bool exist= s) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", cpu: %d, exist: %d" -postcopy_blocktime_end(uint64_t addr, uint64_t time, int affected_cpu) "ad= dr: 0x%" PRIx64 ", time: %" PRIu64 ", affected_cpus: %d" +postcopy_blocktime_end(uint64_t addr, uint64_t time, int affected_cpu, int= affected_non_cpus) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", affected_cpus:= %d, affected_non_cpus: %d" postcopy_blocktime_end_one(int cpu, uint8_t left_faults) "cpu: %d, left_fa= ults: %" PRIu8 =20 # exec.c diff --git a/qapi/migration.json b/qapi/migration.json index bb41dc0795..66fb8ac74d 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -247,6 +247,12 @@ # this is the per-vCPU statistics. This is only present when the # postcopy-blocktime migration capability is enabled. (Since 10.1) # +# @postcopy-non-vcpu-latency: average remote page fault latency for all +# faults happend in non-vCPU threads (in ns). It has the same +# definition of @postcopy-latency but this only provides statistics to +# non-vCPU faults. This is only present when the postcopy-blocktime +# migration capability is enabled. (Since 10.1) +# # @socket-address: Only used for tcp, to know what the real port is # (Since 4.0) # @@ -273,8 +279,8 @@ # # Features: # -# @unstable: Members @postcopy-latency, @postcopy-vcpu-latency are -# experimental. +# @unstable: Members @postcopy-latency, @postcopy-vcpu-latency, +# @postcopy-non-vcpu-latency are experimental. # # Since: 0.14 ## @@ -295,6 +301,8 @@ 'type': 'uint64', 'features': [ 'unstable' ] }, '*postcopy-vcpu-latency': { 'type': ['uint64'], 'features': [ 'unstable' ] }, + '*postcopy-non-vcpu-latency': { + 'type': 'uint64', 'features': [ 'unstable' ] }, '*socket-address': ['SocketAddress'], '*dirty-limit-throttle-time-per-round': 'uint64', '*dirty-limit-ring-full-time': 'uint64'} } diff --git a/tests/qtest/migration/migration-qmp.c b/tests/qtest/migration/= migration-qmp.c index 1a5ab2d229..67a67d4bd6 100644 --- a/tests/qtest/migration/migration-qmp.c +++ b/tests/qtest/migration/migration-qmp.c @@ -361,6 +361,7 @@ void read_blocktime(QTestState *who) g_assert(qdict_haskey(rsp_return, "postcopy-vcpu-blocktime")); g_assert(qdict_haskey(rsp_return, "postcopy-latency")); g_assert(qdict_haskey(rsp_return, "postcopy-vcpu-latency")); + g_assert(qdict_haskey(rsp_return, "postcopy-non-vcpu-latency")); qobject_unref(rsp_return); } =20 --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243658; cv=none; d=zohomail.com; s=zohoarc; b=ij/IJ3tvakr4k/L6xZUktKq3rq2ZE7Zu9w1hswdkIKcgEIE+LZwYTYHMEc2bimHLUcVl3QrrgKEGGyimuHzWmm8MALVxdCfnh64C1zs29/ZMYvZZvZmjXtI3gQtX3YHx0s9fHyaqZ+29W/tEgtnGysw9W7cftOC5tkTSgHNE4rs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243658; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=4c+sKf5zS5Fj2EAwNBg2Y1lHJiq62nREYOaV1obbBy8=; b=aoYq2Ii1GqtNE+OlNaJF71ooU95AvbTP5rnw5Lq+LVP5ZUunaCwWr87JltQYsTxzawPaDHC/KEGeaZ3OCeeWWWp9gMXEKy63uQ56FpHDynFUHAEtRQrgiRK9okhDpHFXd8Wmvkig+FmI5UY82Ut/0F7gOlcJ2OdQUo8gIg/wPVU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243658505562.2679200027035; Fri, 11 Jul 2025 07:20:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEbk-00089p-62; Fri, 11 Jul 2025 10:20:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaEU5-0002uK-I5 for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:20 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaEU3-0006rD-6Y for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:17 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B72D4211A0; Fri, 11 Jul 2025 14:11:22 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 21FF31388B; Fri, 11 Jul 2025 14:11:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id CNYpNIgbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243082; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4c+sKf5zS5Fj2EAwNBg2Y1lHJiq62nREYOaV1obbBy8=; b=Pnt/N4qfqh3sCpGl57i6gNuDc7hgJLBpBJoQmihCsgvZkR/Y4/5589o0c3q0xD5ceutvzE +fvq5z8NeFJp/enVuY3UDmqLEAXS9qfe+HJlKtro0Xyo5xbGTori+AG8FLut1IcvTGGYMZ vH/POsXDWgIXJyuEmBl/+WZqr2XRkaQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243082; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4c+sKf5zS5Fj2EAwNBg2Y1lHJiq62nREYOaV1obbBy8=; b=eQW/mb7wNRM+wQKup+ANbaadQJQ/M/cUW63AvFihehMzahI1wpX3F/Dd78u6y7f12HHmkK ssCak1XZOLxWkoCA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="Pnt/N4qf"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="eQW/mb7w" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243082; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4c+sKf5zS5Fj2EAwNBg2Y1lHJiq62nREYOaV1obbBy8=; b=Pnt/N4qfqh3sCpGl57i6gNuDc7hgJLBpBJoQmihCsgvZkR/Y4/5589o0c3q0xD5ceutvzE +fvq5z8NeFJp/enVuY3UDmqLEAXS9qfe+HJlKtro0Xyo5xbGTori+AG8FLut1IcvTGGYMZ vH/POsXDWgIXJyuEmBl/+WZqr2XRkaQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243082; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4c+sKf5zS5Fj2EAwNBg2Y1lHJiq62nREYOaV1obbBy8=; b=eQW/mb7wNRM+wQKup+ANbaadQJQ/M/cUW63AvFihehMzahI1wpX3F/Dd78u6y7f12HHmkK ssCak1XZOLxWkoCA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Markus Armbruster , "Dr . David Alan Gilbert" Subject: [PULL 25/26] migration/postcopy: Add latency distribution report for blocktime Date: Fri, 11 Jul 2025 11:10:30 -0300 Message-Id: <20250711141031.423-26-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RLi3368pnyb3ujpcs6u1hud8b3)]; TO_DN_SOME(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[treblig.org:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:mid,suse.de:dkim,suse.de:email]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: B72D4211A0 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243659497116600 Content-Type: text/plain; charset="utf-8" From: Peter Xu Add the latency distribution too for blocktime, using order-of-two buckets. It accounts for all the faults, from either vCPU or non-vCPU threads. With prior rework, it's very easy to achieve by adding an array to account for faults in each buckets. Sample output for HMP (while for QMP it's simply an array): Postcopy Latency Distribution: [ 1 us - 2 us ]: 0 [ 2 us - 4 us ]: 0 [ 4 us - 8 us ]: 1 [ 8 us - 16 us ]: 2 [ 16 us - 32 us ]: 2 [ 32 us - 64 us ]: 3 [ 64 us - 128 us ]: 10169 [ 128 us - 256 us ]: 50151 [ 256 us - 512 us ]: 12876 [ 512 us - 1 ms ]: 97 [ 1 ms - 2 ms ]: 42 [ 2 ms - 4 ms ]: 44 [ 4 ms - 8 ms ]: 93 [ 8 ms - 16 ms ]: 138 [ 16 ms - 32 ms ]: 0 [ 32 ms - 65 ms ]: 0 [ 65 ms - 131 ms ]: 0 [ 131 ms - 262 ms ]: 0 [ 262 ms - 524 ms ]: 0 [ 524 ms - 1 sec ]: 0 [ 1 sec - 2 sec ]: 0 [ 2 sec - 4 sec ]: 0 [ 4 sec - 8 sec ]: 0 [ 8 sec - 16 sec ]: 0 Cc: Markus Armbruster Acked-by: Dr. David Alan Gilbert Reviewed-by: Fabiano Rosas Link: https://lore.kernel.org/r/20250613141217.474825-15-peterx@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration-hmp-cmds.c | 32 +++++++++++++++++++ migration/postcopy-ram.c | 46 +++++++++++++++++++++++++++ qapi/migration.json | 12 ++++++- tests/qtest/migration/migration-qmp.c | 1 + 4 files changed, 90 insertions(+), 1 deletion(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index e1f9530520..cef5608210 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -52,6 +52,21 @@ static void migration_global_dump(Monitor *mon) ms->clear_bitmap_shift); } =20 +static const gchar *format_time_str(uint64_t us) +{ + const char *units[] =3D {"us", "ms", "sec"}; + int index =3D 0; + + while (us > 1000) { + us /=3D 1000; + if (++index >=3D (sizeof(units) - 1)) { + break; + } + } + + return g_strdup_printf("%"PRIu64" %s", us, units[index]); +} + static void migration_dump_blocktime(Monitor *mon, MigrationInfo *info) { if (info->has_postcopy_blocktime) { @@ -100,6 +115,23 @@ static void migration_dump_blocktime(Monitor *mon, Mig= rationInfo *info) } monitor_printf(mon, "]\n"); } + + if (info->has_postcopy_latency_dist) { + uint64List *item =3D info->postcopy_latency_dist; + int count =3D 0; + + monitor_printf(mon, "Postcopy Latency Distribution:\n"); + + while (item) { + g_autofree const gchar *from =3D format_time_str(1UL << count); + g_autofree const gchar *to =3D format_time_str(1UL << (count += 1)); + + monitor_printf(mon, " [ %8s - %8s ]: %10"PRIu64"\n", + from, to, item->value); + item =3D item->next; + count++; + } + } } =20 void hmp_info_migrate(Monitor *mon, const QDict *qdict) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index f4cb23b3e0..45af9a361e 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -110,6 +110,15 @@ void postcopy_thread_create(MigrationIncomingState *mi= s, #include #include =20 +/* + * Here we use 24 buckets, which means the last bucket will cover [2^24 us, + * 2^25 us) ~=3D [16, 32) seconds. It should be far enough to record even + * extreme (perf-wise broken) 1G pages moving over, which can sometimes + * take a few seconds due to various reasons. Anything more than that + * might be unsensible to account anymore. + */ +#define BLOCKTIME_LATENCY_BUCKET_N (24) + /* All the time records are in unit of nanoseconds */ typedef struct PostcopyBlocktimeContext { /* blocktime per vCPU */ @@ -175,6 +184,11 @@ typedef struct PostcopyBlocktimeContext { * that a fault was requested. */ GHashTable *vcpu_addr_hash; + /* + * Each bucket stores the count of faults that were resolved within the + * bucket window [2^N us, 2^(N+1) us). + */ + uint64_t latency_buckets[BLOCKTIME_LATENCY_BUCKET_N]; /* total blocktime when all vCPUs are stopped */ uint64_t total_blocktime; /* point in time when last page fault was initiated */ @@ -283,6 +297,9 @@ static struct PostcopyBlocktimeContext *blocktime_conte= xt_new(void) unsigned int smp_cpus =3D ms->smp.cpus; PostcopyBlocktimeContext *ctx =3D g_new0(PostcopyBlocktimeContext, 1); =20 + /* Initialize all counters to be zeros */ + memset(ctx->latency_buckets, 0, sizeof(ctx->latency_buckets)); + ctx->vcpu_blocktime_total =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_faults_count =3D g_new0(uint64_t, smp_cpus); ctx->vcpu_faults_current =3D g_new0(uint8_t, smp_cpus); @@ -320,6 +337,7 @@ void fill_destination_postcopy_migration_info(Migration= Info *info) uint64_t latency_total =3D 0, faults =3D 0; uint32List *list_blocktime =3D NULL; uint64List *list_latency =3D NULL; + uint64List *latency_buckets =3D NULL; int i; =20 if (!bc) { @@ -349,6 +367,10 @@ void fill_destination_postcopy_migration_info(Migratio= nInfo *info) QAPI_LIST_PREPEND(list_latency, latency); } =20 + for (i =3D BLOCKTIME_LATENCY_BUCKET_N - 1; i >=3D 0; i--) { + QAPI_LIST_PREPEND(latency_buckets, bc->latency_buckets[i]); + } + latency_total +=3D bc->non_vcpu_blocktime_total; faults +=3D bc->non_vcpu_faults; =20 @@ -364,6 +386,8 @@ void fill_destination_postcopy_migration_info(Migration= Info *info) info->postcopy_latency =3D faults ? (latency_total / faults) : 0; info->has_postcopy_vcpu_latency =3D true; info->postcopy_vcpu_latency =3D list_latency; + info->has_postcopy_latency_dist =3D true; + info->postcopy_latency_dist =3D latency_buckets; } =20 static uint64_t get_postcopy_total_blocktime(void) @@ -1096,6 +1120,25 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, u= int32_t ptid, blocktime_fault_inject(dc, addr, cpu, current); } =20 +static void blocktime_latency_account(PostcopyBlocktimeContext *ctx, + uint64_t time_us) +{ + /* + * Convert time (in us) to bucket index it belongs. Take extra caution + * of time_us=3D=3D0 even if normally rare - when happens put into buc= ket 0. + */ + int index =3D time_us ? (63 - clz64(time_us)) : 0; + + assert(index >=3D 0); + + /* If it's too large, put into top bucket */ + if (index >=3D BLOCKTIME_LATENCY_BUCKET_N) { + index =3D BLOCKTIME_LATENCY_BUCKET_N - 1; + } + + ctx->latency_buckets[index]++; +} + typedef struct { PostcopyBlocktimeContext *ctx; uint64_t current; @@ -1118,6 +1161,9 @@ static void blocktime_cpu_list_iter_fn(gpointer data,= gpointer user_data) assert(iter->current >=3D entry->fault_time); time_passed =3D iter->current - entry->fault_time; =20 + /* Latency buckets are in microseconds */ + blocktime_latency_account(ctx, time_passed / SCALE_US); + if (cpu >=3D 0) { /* * If we resolved all pending faults on one vCPU due to this page diff --git a/qapi/migration.json b/qapi/migration.json index 66fb8ac74d..2d39a8f748 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -242,6 +242,14 @@ # average page fault latency. This is only present when the # postcopy-blocktime migration capability is enabled. (Since 10.1) # +# @postcopy-latency-dist: remote page fault latency distributions. Each +# element of the array is the number of faults that fall into the +# bucket period. For the N-th bucket (N>=3D0), the latency window is +# [2^Nus, 2^(N+1)us). For example, the 8th element stores how many +# remote faults got resolved within [256us, 512us) window. This is only +# present when the postcopy-blocktime migration capability is enabled. +# (Since 10.1) +# # @postcopy-vcpu-latency: average remote page fault latency per vCPU (in # ns). It has the same definition of @postcopy-latency, but instead # this is the per-vCPU statistics. This is only present when the @@ -280,7 +288,7 @@ # Features: # # @unstable: Members @postcopy-latency, @postcopy-vcpu-latency, -# @postcopy-non-vcpu-latency are experimental. +# @postcopy-latency-dist, @postcopy-non-vcpu-latency are experimental. # # Since: 0.14 ## @@ -299,6 +307,8 @@ '*postcopy-vcpu-blocktime': ['uint32'], '*postcopy-latency': { 'type': 'uint64', 'features': [ 'unstable' ] }, + '*postcopy-latency-dist': { + 'type': ['uint64'], 'features': [ 'unstable' ] }, '*postcopy-vcpu-latency': { 'type': ['uint64'], 'features': [ 'unstable' ] }, '*postcopy-non-vcpu-latency': { diff --git a/tests/qtest/migration/migration-qmp.c b/tests/qtest/migration/= migration-qmp.c index 67a67d4bd6..66dd369ba7 100644 --- a/tests/qtest/migration/migration-qmp.c +++ b/tests/qtest/migration/migration-qmp.c @@ -360,6 +360,7 @@ void read_blocktime(QTestState *who) g_assert(qdict_haskey(rsp_return, "postcopy-blocktime")); g_assert(qdict_haskey(rsp_return, "postcopy-vcpu-blocktime")); g_assert(qdict_haskey(rsp_return, "postcopy-latency")); + g_assert(qdict_haskey(rsp_return, "postcopy-latency-dist")); g_assert(qdict_haskey(rsp_return, "postcopy-vcpu-latency")); g_assert(qdict_haskey(rsp_return, "postcopy-non-vcpu-latency")); qobject_unref(rsp_return); --=20 2.35.3 From nobody Thu Dec 18 04:18:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1752243771; cv=none; d=zohomail.com; s=zohoarc; b=fHONiZAvtIdkVuI3kpFSawawwCic4umODFJXj4zQgt559J1r+mfok1P9vOwX44BKPoiLIPYnuf0EscxQj7Dsj7pCWXwnFKTU0TCge444N11SviJny+/iJQ1Bgt/2xpzXRUkqb3gsnEXiGjwlksNoc0+6xJHKFDpnSaJkmjGyJ44= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752243771; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=aG95BgYcOsHXzWzUs4dX5Sb1S1cl1g5JAcSQb06bXEs=; b=kwCJrs5HCYAX0BPpVoFkvp9yRF7KkOiTBB2FTVWMeeFk0eQLInZmQKScNtQ/oiBOKm4adWKzaaYJXwQ5xrkSBpWQIpFxuQhAkVeOVHOUUSa+YEpsQfHcfjPH6lz3W1bnxWZSOxSVvh0ZUAMXUhtUxpCXcl2ImyzECVI2sKafxH8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752243771515601.3387028089066; Fri, 11 Jul 2025 07:22:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uaEe2-0004li-H7; Fri, 11 Jul 2025 10:22:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uaEUP-0003Rb-Um for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:38 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uaEUK-0006tp-Rl for qemu-devel@nongnu.org; Fri, 11 Jul 2025 10:12:37 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 22F281F745; Fri, 11 Jul 2025 14:11:25 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 42DE61388B; Fri, 11 Jul 2025 14:11:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2PihAIsbcWg7TgAAD6G6ig (envelope-from ); Fri, 11 Jul 2025 14:11:23 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243085; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aG95BgYcOsHXzWzUs4dX5Sb1S1cl1g5JAcSQb06bXEs=; b=xGhuNNFU/9kvNMzKDHGLwPLrmPjdnj6sLqAVAhbA3E96tqbbRgO7APre5o0GXO+1FqwNhf ZUkZvDkNIKR3CdmcNFSzPNzxQ+5y+rEcirxaJ6btBEFwWgkVN8EnuAgqhu3H0YPbzWG2oY sq2unFRzdMAInKEWFYCgCTl8SKUsU48= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243085; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aG95BgYcOsHXzWzUs4dX5Sb1S1cl1g5JAcSQb06bXEs=; b=LEL+EsTYwonbTgV5dVyJFMTZFZqT3p9OjFXg1VvOB3jyix5k24qZNCbFt5MakSeagvpG6a vNEaHTNJnb9/IuDw== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=xGhuNNFU; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=LEL+EsTY DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1752243085; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aG95BgYcOsHXzWzUs4dX5Sb1S1cl1g5JAcSQb06bXEs=; b=xGhuNNFU/9kvNMzKDHGLwPLrmPjdnj6sLqAVAhbA3E96tqbbRgO7APre5o0GXO+1FqwNhf ZUkZvDkNIKR3CdmcNFSzPNzxQ+5y+rEcirxaJ6btBEFwWgkVN8EnuAgqhu3H0YPbzWG2oY sq2unFRzdMAInKEWFYCgCTl8SKUsU48= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1752243085; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aG95BgYcOsHXzWzUs4dX5Sb1S1cl1g5JAcSQb06bXEs=; b=LEL+EsTYwonbTgV5dVyJFMTZFZqT3p9OjFXg1VvOB3jyix5k24qZNCbFt5MakSeagvpG6a vNEaHTNJnb9/IuDw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Juraj Marcin , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 26/26] migration: Rename save_live_complete_precopy_thread to save_complete_precopy_thread Date: Fri, 11 Jul 2025 11:10:31 -0300 Message-Id: <20250711141031.423-27-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250711141031.423-1-farosas@suse.de> References: <20250711141031.423-1-farosas@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; R_RATELIMIT(0.00)[to_ip_from(RLi3368pnyb3ujpcs6u1hud8b3)]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:mid,suse.de:dkim,suse.de:email]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 22F281F745 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.51 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1752243772799116600 From: Juraj Marcin Recent patch [1] renames the save_live_complete_precopy handler to save_complete, as the machine is not live in most cases when this handler is executed. The same is true also for save_live_complete_precopy_thread, therefore this patch removes the "live" keyword from the handler itself and related types to keep the naming unified. In contrast to save_complete, this handler is only executed at the end of precopy, therefore the "precopy" keyword is retained. [1]: https://lore.kernel.org/all/20250613140801.474264-7-peterx@redhat.com/ Cc: Alex Williamson Cc: C=C3=A9dric Le Goater Signed-off-by: Juraj Marcin Link: https://lore.kernel.org/r/20250626085235.294690-1-jmarcin@redhat.com Signed-off-by: Peter Xu Signed-off-by: Fabiano Rosas --- docs/devel/migration/vfio.rst | 4 ++-- hw/vfio/migration-multifd.c | 4 ++-- hw/vfio/migration-multifd.h | 2 +- hw/vfio/migration.c | 2 +- include/migration/misc.h | 8 ++++---- include/migration/register.h | 6 +++--- include/qemu/typedefs.h | 6 +++--- migration/multifd-device-state.c | 10 +++++----- migration/savevm.c | 6 +++--- 9 files changed, 24 insertions(+), 24 deletions(-) diff --git a/docs/devel/migration/vfio.rst b/docs/devel/migration/vfio.rst index 8ff5ab0c74..2d8e5ca9dd 100644 --- a/docs/devel/migration/vfio.rst +++ b/docs/devel/migration/vfio.rst @@ -80,7 +80,7 @@ VFIO implements the device hooks for the iterative approa= ch as follows: vendor driver indicates that no data remains. In the multifd mode it just emits a dummy EOS marker. =20 -* A ``save_live_complete_precopy_thread`` function that in the multifd mode +* A ``save_complete_precopy_thread`` function that in the multifd mode provides thread handler performing multifd device state transfer. It sets the VFIO device to _STOP_COPY state, iteratively reads the data from the VFIO device and queues it for multifd transmission until the ve= ndor @@ -200,7 +200,7 @@ Live migration save path .save_complete() until pending data is 0 In the multifd mode this iteration is done in - .save_live_complete_precopy_thread() instead. + .save_complete_precopy_thread() instead. | (POSTMIGRATE, _COMPLETED, _STOP_COPY) Migraton thread schedules cleanup bottom half and exits diff --git a/hw/vfio/migration-multifd.c b/hw/vfio/migration-multifd.c index 850a319488..55635486c8 100644 --- a/hw/vfio/migration-multifd.c +++ b/hw/vfio/migration-multifd.c @@ -583,7 +583,7 @@ vfio_save_complete_precopy_thread_config_state(VFIODevi= ce *vbasedev, =20 /* * This thread is spawned by the migration core directly via - * .save_live_complete_precopy_thread SaveVMHandler. + * .save_complete_precopy_thread SaveVMHandler. * * It exits after either: * * completing saving the remaining device state and device config, OR: @@ -592,7 +592,7 @@ vfio_save_complete_precopy_thread_config_state(VFIODevi= ce *vbasedev, * multifd_device_state_save_thread_should_exit() returning true. */ bool -vfio_multifd_save_complete_precopy_thread(SaveLiveCompletePrecopyThreadDat= a *d, +vfio_multifd_save_complete_precopy_thread(SaveCompletePrecopyThreadData *d, Error **errp) { VFIODevice *vbasedev =3D d->handler_opaque; diff --git a/hw/vfio/migration-multifd.h b/hw/vfio/migration-multifd.h index 0bab63211d..ebf22a7997 100644 --- a/hw/vfio/migration-multifd.h +++ b/hw/vfio/migration-multifd.h @@ -26,7 +26,7 @@ bool vfio_multifd_load_state_buffer(void *opaque, char *d= ata, size_t data_size, void vfio_multifd_emit_dummy_eos(VFIODevice *vbasedev, QEMUFile *f); =20 bool -vfio_multifd_save_complete_precopy_thread(SaveLiveCompletePrecopyThreadDat= a *d, +vfio_multifd_save_complete_precopy_thread(SaveCompletePrecopyThreadData *d, Error **errp); =20 int vfio_multifd_switchover_start(VFIODevice *vbasedev); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 33a71f8999..c329578eec 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -835,7 +835,7 @@ static const SaveVMHandlers savevm_vfio_handlers =3D { */ .load_state_buffer =3D vfio_multifd_load_state_buffer, .switchover_start =3D vfio_switchover_start, - .save_live_complete_precopy_thread =3D vfio_multifd_save_complete_prec= opy_thread, + .save_complete_precopy_thread =3D vfio_multifd_save_complete_precopy_t= hread, }; =20 /* ---------------------------------------------------------------------- = */ diff --git a/include/migration/misc.h b/include/migration/misc.h index 8fd36eba1d..a261f99d89 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -119,19 +119,19 @@ bool migrate_uri_parse(const char *uri, MigrationChan= nel **channel, Error **errp); =20 /* migration/multifd-device-state.c */ -typedef struct SaveLiveCompletePrecopyThreadData { - SaveLiveCompletePrecopyThreadHandler hdlr; +typedef struct SaveCompletePrecopyThreadData { + SaveCompletePrecopyThreadHandler hdlr; char *idstr; uint32_t instance_id; void *handler_opaque; -} SaveLiveCompletePrecopyThreadData; +} SaveCompletePrecopyThreadData; =20 bool multifd_queue_device_state(char *idstr, uint32_t instance_id, char *data, size_t len); bool multifd_device_state_supported(void); =20 void -multifd_spawn_device_state_save_thread(SaveLiveCompletePrecopyThreadHandle= r hdlr, +multifd_spawn_device_state_save_thread(SaveCompletePrecopyThreadHandler hd= lr, char *idstr, uint32_t instance_id, void *opaque); =20 diff --git a/include/migration/register.h b/include/migration/register.h index 2a26e76a68..ae79794cdd 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -98,7 +98,7 @@ typedef struct SaveVMHandlers { int (*save_complete)(QEMUFile *f, void *opaque); =20 /** - * @save_live_complete_precopy_thread (invoked in a separate thread) + * @save_complete_precopy_thread (invoked in a separate thread) * * Called at the end of a precopy phase from a separate worker thread * in configurations where multifd device state transfer is supported @@ -107,14 +107,14 @@ typedef struct SaveVMHandlers { * When postcopy is enabled, devices that support postcopy will skip t= his * step. * - * @d: a #SaveLiveCompletePrecopyThreadData containing parameters that= the + * @d: a #SaveCompletePrecopyThreadData containing parameters that the * handler may need, including this device section idstr and instance_= id, * and opaque data pointer passed to register_savevm_live(). * @errp: pointer to Error*, to store an error if it happens. * * Returns true to indicate success and false for errors. */ - SaveLiveCompletePrecopyThreadHandler save_live_complete_precopy_thread; + SaveCompletePrecopyThreadHandler save_complete_precopy_thread; =20 /* This runs both outside and inside the BQL. */ =20 diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h index 507f0814d5..4a94af9665 100644 --- a/include/qemu/typedefs.h +++ b/include/qemu/typedefs.h @@ -109,7 +109,7 @@ typedef struct QString QString; typedef struct RAMBlock RAMBlock; typedef struct Range Range; typedef struct ReservedRegion ReservedRegion; -typedef struct SaveLiveCompletePrecopyThreadData SaveLiveCompletePrecopyTh= readData; +typedef struct SaveCompletePrecopyThreadData SaveCompletePrecopyThreadData; typedef struct SHPCDevice SHPCDevice; typedef struct SSIBus SSIBus; typedef struct TCGCPUOps TCGCPUOps; @@ -135,7 +135,7 @@ typedef struct IRQState *qemu_irq; typedef void (*qemu_irq_handler)(void *opaque, int n, int level); typedef bool (*MigrationLoadThread)(void *opaque, bool *should_quit, Error **errp); -typedef bool (*SaveLiveCompletePrecopyThreadHandler)(SaveLiveCompletePreco= pyThreadData *d, - Error **errp); +typedef bool (*SaveCompletePrecopyThreadHandler)(SaveCompletePrecopyThread= Data *d, + Error **errp); =20 #endif /* QEMU_TYPEDEFS_H */ diff --git a/migration/multifd-device-state.c b/migration/multifd-device-st= ate.c index 94222d0eb0..fce64f00b0 100644 --- a/migration/multifd-device-state.c +++ b/migration/multifd-device-state.c @@ -131,7 +131,7 @@ bool multifd_device_state_supported(void) =20 static void multifd_device_state_save_thread_data_free(void *opaque) { - SaveLiveCompletePrecopyThreadData *data =3D opaque; + SaveCompletePrecopyThreadData *data =3D opaque; =20 g_clear_pointer(&data->idstr, g_free); g_free(data); @@ -139,7 +139,7 @@ static void multifd_device_state_save_thread_data_free(= void *opaque) =20 static int multifd_device_state_save_thread(void *opaque) { - SaveLiveCompletePrecopyThreadData *data =3D opaque; + SaveCompletePrecopyThreadData *data =3D opaque; g_autoptr(Error) local_err =3D NULL; =20 if (!data->hdlr(data, &local_err)) { @@ -170,18 +170,18 @@ bool multifd_device_state_save_thread_should_exit(voi= d) } =20 void -multifd_spawn_device_state_save_thread(SaveLiveCompletePrecopyThreadHandle= r hdlr, +multifd_spawn_device_state_save_thread(SaveCompletePrecopyThreadHandler hd= lr, char *idstr, uint32_t instance_id, void *opaque) { - SaveLiveCompletePrecopyThreadData *data; + SaveCompletePrecopyThreadData *data; =20 assert(multifd_device_state_supported()); assert(multifd_send_device_state); =20 assert(!qatomic_read(&multifd_send_device_state->threads_abort)); =20 - data =3D g_new(SaveLiveCompletePrecopyThreadData, 1); + data =3D g_new(SaveCompletePrecopyThreadData, 1); data->hdlr =3D hdlr; data->idstr =3D g_strdup(idstr); data->instance_id =3D instance_id; diff --git a/migration/savevm.c b/migration/savevm.c index c4fd5f5a5b..fabbeb296a 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1581,15 +1581,15 @@ int qemu_savevm_state_complete_precopy_iterable(QEM= UFile *f, bool in_postcopy) =20 if (multifd_device_state) { QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - SaveLiveCompletePrecopyThreadHandler hdlr; + SaveCompletePrecopyThreadHandler hdlr; =20 if (!se->ops || (in_postcopy && se->ops->has_postcopy && se->ops->has_postcopy(se->opaque)) || - !se->ops->save_live_complete_precopy_thread) { + !se->ops->save_complete_precopy_thread) { continue; } =20 - hdlr =3D se->ops->save_live_complete_precopy_thread; + hdlr =3D se->ops->save_complete_precopy_thread; multifd_spawn_device_state_save_thread(hdlr, se->idstr, se->instance= _id, se->opaque); --=20 2.35.3