From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780991997; cv=pass; d=zohomail.com; s=zohoarc; b=Tv/c1hYg3y/a/bgW+GI4eOdoYGPRcGwHOlwnzxG2IW1UuDOUF6GaINQqGWDeE6hcSqabrjxdndojpcRt2ZxcCQWPh4mbKficjjajYzczSQSM4IG0W8ADgJA+9+be0eSrpST/Qdlyfp42if9vHprcNyHOV3SktSK4/swVbJ/OEK0= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780991997; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WhYoOMci1BNzaOqFhLBZwNsiAPyZoDLsa/M5JrnZ10E=; b=UL5CBo9+hBUljzN/OU9IP9k2S7hBXd+5ZZqh1GWCT8J+tUpThhx4yh8hOBBD5iKN+O+loAzWXQp4ZLrY9KfKJTD74BRK8bPL96XZG8mlyvm5dLrMaH6WghTfJPGM3U5O2XN4Q9FaMdmBWt4u1HVi+z1RtOseFRujzL1+tF1wVpM= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 178099199722389.00199693224715; Tue, 9 Jun 2026 00:59:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrN2-0001vi-3p; Tue, 09 Jun 2026 03:59:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrN0-0001vM-Gf for qemu-devel@nongnu.org; Tue, 09 Jun 2026 03:59:34 -0400 Received: from mail-southcentralusazlp170110003.outbound.protection.outlook.com ([2a01:111:f403:c10d::3] helo=SN4PR0501CU005.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrMy-0003Sf-Fe for qemu-devel@nongnu.org; Tue, 09 Jun 2026 03:59:34 -0400 Received: from CYXPR02CA0008.namprd02.prod.outlook.com (2603:10b6:930:cf::14) by PH0PR12MB7862.namprd12.prod.outlook.com (2603:10b6:510:26d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.13; Tue, 9 Jun 2026 07:59:25 +0000 Received: from CY4PEPF0000EE39.namprd03.prod.outlook.com (2603:10b6:930:cf:cafe::22) by CYXPR02CA0008.outlook.office365.com (2603:10b6:930:cf::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 07:59:25 +0000 Received: from mail.nvidia.com (216.228.117.160) by CY4PEPF0000EE39.mail.protection.outlook.com (10.167.242.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 07:59:25 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:06 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:06 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 00:59:00 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bfsGM5HuRApxF3oOXYEdT4f/UOz2H5xa/yGNH2gB+Y8Y90cWbRQtLhvU6mHnVeWu+4kcfszmmG8ca7HlJaKpug6Bs/GJX0ogf9fUBzr6uJHoDLulDI+iHfogpV6cRqm7JFEact/8G5fdEdgzemXRxhv6NILsFlrWo4m8n4z41vnwJeXX/i8Kctq5fkr8rG+Md0i258yGk8gJY5fz18FgXITeizWbLyR23RVq2uJplvcNkXjKPYIhTv4uuGQPgpYs1M01V6HgbmttDWmyYCiRkDrPHtF9+Z7JArMyxSKcrN5Lam2ZlUY5e0ge7ZafPrsVnp18rtRiLSfQcRCAwLzCuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WhYoOMci1BNzaOqFhLBZwNsiAPyZoDLsa/M5JrnZ10E=; b=hYjInA7wndU+8eA1Eif0w/qMzzavL65cJWuFW4/xVWNvisKAUAxnRrB/J5bcIgvyZqU2hpHe2oBt5q77FkCov8X9WxHF47lfWcq5FRq9ZQ8LnsFljbVYx3VZt6IN9Cwr1BJhvc+2lFbIS+4vkfFaD6exKF2zR+EANY/apD4fIn7POSDOef0CERtV9VvnDJqm4P/EkMcmeVw70RGKDrAXavamefj5La4ExBb0DtqelXWyKtEvnu3tASRLM49XyKcVUlM1VBS+tLlHsRNTio+VgSxBGw54VdxeSq8NAdcRBZVUZmIzSTzk6li6eJtTVYBNRJxmYLl8siQkjv6YL9eoMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WhYoOMci1BNzaOqFhLBZwNsiAPyZoDLsa/M5JrnZ10E=; b=ocWAt1mzJAbI+VkCEgNOKBoetOWTBUmFOl7mwG88y6z3oIhBhKGWa4MHUGZv4JkbHRE6p804JFGRMyo0gQEgzuG/yI9m5k8XAXkCRohlPMZHUKE4Ten2Zb+BUkYZeoRUfMZyjVMp8S8OE+Qfm+bsu4SBk82psvbkE5CjuyL0cs3u8BJ07erNDwRqsXISXICXi6KnLtokP6slwyIrQBxAd9KYsrhR96XronMClF6bzRqkCXyXyR+6t3HXmU8jIQGB+WWKMBe5YYXNXM/c+Z6b5o8nVxCaqDbc1IEJe16oKOQnWHaRc+UshGurDwMZtgiXnqxQmU1HEJF4KFtn+F2JZg== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 01/14] migration: Propagate errors in migration_completion_precopy() Date: Tue, 9 Jun 2026 10:57:59 +0300 Message-ID: <20260609075812.32067-2-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE39:EE_|PH0PR12MB7862:EE_ X-MS-Office365-Filtering-Correlation-Id: e494a910-7200-478a-56bb-08dec5fd0633 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700016|7416014|376014|82310400026|22082099003|18002099003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: GwImsUO5/9Ee8OtH1wxvdEoDU7hKidnHQtGoA1rzpe9UNiyeBuwXhqwyNQghknxFcTr16J1l0UteHzRYLwbFIU+Y1IfPxGmvnw1Trl5ZFeYMSTW+FTS1udS694gLfR5Hq3+RVqosQDOIyXht+N9LUZnpLNPjzhK3L2OuMtF+0tsHM08BRbFKt644HxPFM7reVpA6BmuX7j7wNHuVnbHPRiDCqyr6+YNZQ7SNlqWbgHc7chRIstn9EeR6zofSZ4cR5vnsNuion4yGVzqi8CeL9TV5v8CCb0KBVSnbpx007ZK5TPtQiF25ADMDs3vs17f9PRSbzhHQe6YArjl3WtX+vQnPIKVAYxWpdCaWcnX0CA/k7igATEwojpgUefDUrTVXqE86zF+dXdxnMIrmLzMR3CtopzTJxEMxdDnuF/FP0Vrfd0Vi4yF4vvKaJJkYsV1Gl03gYDidrTBHeKQE68rQqxjl+5KYMEF6aSKkY8V5noxLz9kV3K06gYeln1WPawGpPzbxDTBDIssz6DD/u90gX2oT1pNW40ZO9tLyQk5OlJjUDCPUB1Z8LzF/cRXgi+kE8Ky0x3KBZlB7eehVN4rrfZUJd0XYpxiAc5YpJXEdby4VIHpNWinN2SPWOJT6RW9KQEGIAY3A6ZjHEkIuHP2kkSDnrguwz7wlBlGbkiEGdH9kJjGutN7iJJ0B/WeVkfVZujsjQRjAW5/Et3oxlXOi4Zfc3MxcrBUcOD2rIsAZB10= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(36860700016)(7416014)(376014)(82310400026)(22082099003)(18002099003)(11063799006)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 72qZGJWHgTkwvdQ3o7FIFnq49UgUqMjU60MDvLEfvnjAL/mvDDCkE71mzbj65SWgxobl6KDH9r7tCLWo34KP8sE8lwApd130pwxc4pS7apG1XqjQygpm+TOKrvLBsYreJ4Sh376C4HnKBR+vNNaHK+t5Qe8NxEJeCsTzCv5PFId0vZHZ8JtaC030QHVDYXo9BY74AhKEj+zeYpAG5wK6aeZsCKg3QedWWdqPr7Y1+Z0DQaXytJm0qhz3ePBs3YXF0+iEiAwcjhY3ozZVwVzgGwrSnmqZPR+M30m4+IhSJirCUEvFnzPWzIldZ1PzEaMEehTrwkMCgS1tJkqPIX/vbzDP5F/eoMHtYNrDx0juZosOC03sNhSq3nuXgA5kiumCDwptVL+/5eU08OgV9HDh1G7di+a9XhSOsU3tgbYZKrFOxRK7Gt8dbG3UfbrovH9I X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 07:59:25.3765 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e494a910-7200-478a-56bb-08dec5fd0633 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE39.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7862 Received-SPF: permerror client-ip=2a01:111:f403:c10d::3; envelope-from=avihaih@nvidia.com; helo=SN4PR0501CU005.outbound.protection.outlook.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780991999615154100 Content-Type: text/plain; charset="utf-8" migration_completion_precopy() doesn't propagate errors to migration core which leads to error information loss. Fix that. This prepares for a follow-up where migration_switchover_start() can fail on switchover-ack and still report a useful error. Signed-off-by: Avihai Horon Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- migration/savevm.h | 2 +- migration/migration.c | 13 ++++++++----- migration/savevm.c | 29 +++++++++++++++++------------ 3 files changed, 26 insertions(+), 18 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index 96fdf96d4e..b6bb4fa977 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -44,7 +44,7 @@ void qemu_savevm_state_header(QEMUFile *f); int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy); void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); -int qemu_savevm_state_complete_precopy(MigrationState *s); +int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); void qemu_savevm_query_pending(MigPendingData *pending, bool exact); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); diff --git a/migration/migration.c b/migration/migration.c index 074d3f2c69..aad23f3228 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2814,7 +2814,7 @@ static bool migration_switchover_start(MigrationState= *s, Error **errp) return true; } =20 -static int migration_completion_precopy(MigrationState *s) +static int migration_completion_precopy(MigrationState *s, Error **errp) { int ret; =20 @@ -2823,16 +2823,17 @@ static int migration_completion_precopy(MigrationSt= ate *s) if (!migrate_mode_is_cpr()) { ret =3D migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to stop the VM"); goto out_unlock; } } =20 - if (!migration_switchover_start(s, NULL)) { + if (!migration_switchover_start(s, errp)) { ret =3D -EFAULT; goto out_unlock; } =20 - ret =3D qemu_savevm_state_complete_precopy(s); + ret =3D qemu_savevm_state_complete_precopy(s, errp); out_unlock: bql_unlock(); return ret; @@ -2869,7 +2870,7 @@ static void migration_completion(MigrationState *s) Error *local_err =3D NULL; =20 if (s->state =3D=3D MIGRATION_STATUS_ACTIVE) { - ret =3D migration_completion_precopy(s); + ret =3D migration_completion_precopy(s, &local_err); } else if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE) { migration_completion_postcopy(s); } else { @@ -2900,7 +2901,9 @@ static void migration_completion(MigrationState *s) return; =20 fail: - if (qemu_file_get_error_obj(s->to_dst_file, &local_err)) { + if (local_err) { + migrate_error_propagate(s, local_err); + } else if (qemu_file_get_error_obj(s->to_dst_file, &local_err)) { migrate_error_propagate(s, local_err); } else if (ret) { error_setg_errno(&local_err, -ret, "Error in migration completion"= ); diff --git a/migration/savevm.c b/migration/savevm.c index 23adaf9dd9..9d1d58c8f4 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1771,28 +1771,34 @@ int qemu_savevm_state_non_iterable(QEMUFile *f, Err= or **errp) return 0; } =20 -int qemu_savevm_state_complete_precopy(MigrationState *s) +int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp) { + ERRP_GUARD(); QEMUFile *f =3D s->to_dst_file; - Error *local_err =3D NULL; int ret; =20 ret =3D qemu_savevm_state_complete_precopy_iterable(f, false); if (ret) { + qemu_file_get_error_obj(f, errp); + error_prepend(errp, "Failed to save iterable device state: "); return ret; } =20 - /* TODO: pass error upper */ - ret =3D qemu_savevm_state_non_iterable(f, &local_err); + ret =3D qemu_savevm_state_non_iterable(f, errp); if (ret) { - migrate_error_propagate(s, error_copy(local_err)); - error_report_err(local_err); return ret; } =20 qemu_savevm_state_end_precopy(s, f); =20 - return qemu_fflush(f); + ret =3D qemu_fflush(f); + if (ret) { + qemu_file_get_error_obj(f, errp); + error_prepend(errp, "%s: Failed to flush QEMUFile", __func__); + return ret; + } + + return 0; } =20 void qemu_savevm_query_pending(MigPendingData *pending, bool exact) @@ -1874,13 +1880,12 @@ static int qemu_savevm_state(QEMUFile *f, Error **e= rrp) } =20 ret =3D qemu_file_get_error(f); - if (ret =3D=3D 0) { - qemu_savevm_state_complete_precopy(ms); - ret =3D qemu_file_get_error(f); - } - if (ret !=3D 0) { + if (ret) { error_setg_errno(errp, -ret, "Error while writing VM state"); + goto cleanup; } + + ret =3D qemu_savevm_state_complete_precopy(ms, errp); cleanup: qemu_savevm_state_cleanup(); =20 --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992010; cv=pass; d=zohomail.com; s=zohoarc; b=i1+wBha/xmVhJ6p6gV/QhURgPAlJfrQZFdjUqnOVfILe24dg255vpvqi5zbstb17d8RNxF12ZYiZOfeUMVytUS17uD6MdKOhsLHmOOc/LBQVygzYTg02ipho387OApH+mk5tIk0jqmoeyuwRbmNKCV7gKprHv48nU3SN3PYpo5w= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992010; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=cTJ+28Aey7GgxxE7luuxNI/z/lA62dD9O8waK5z3Mnc=; b=dK3IhJstERvGP9GS+Ong2YJSfg1j8pNhjY5N3WIhHScWq7g/MaR/A2VzEiXmRgJ9igiR9J/XStxWTwmdWbaZnRMU9nMdxbdhn9Ezl07mINzeN2sGR3EbJbsBK/fMHv+SAIVoarOMj3PsSlwFRYM2ioAho+x+Ezl1tqc4RF6dB7E= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992010827381.28080366728466; Tue, 9 Jun 2026 01:00:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrND-0001wE-Dh; Tue, 09 Jun 2026 03:59:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNC-0001w6-GF for qemu-devel@nongnu.org; Tue, 09 Jun 2026 03:59:46 -0400 Received: from mail-eastus2azlp170100001.outbound.protection.outlook.com ([2a01:111:f403:c110::1] helo=BN1PR04CU002.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNA-0003Vg-SG for qemu-devel@nongnu.org; Tue, 09 Jun 2026 03:59:46 -0400 Received: from MN2PR14CA0001.namprd14.prod.outlook.com (2603:10b6:208:23e::6) by MW3PR12MB4476.namprd12.prod.outlook.com (2603:10b6:303:2d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.14; Tue, 9 Jun 2026 07:59:38 +0000 Received: from BN1PEPF00005FFD.namprd05.prod.outlook.com (2603:10b6:208:23e:cafe::2f) by MN2PR14CA0001.outlook.office365.com (2603:10b6:208:23e::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.92.15 via Frontend Transport; Tue, 9 Jun 2026 07:59:38 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00005FFD.mail.protection.outlook.com (10.167.243.229) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 07:59:37 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:20 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:19 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 00:59:13 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NeEPXIrHF8YFV5qQK/Uvs+yLlD9reed02eyr3cY2L/nPyMe7a1F8RCj9wFgLtLamf6EAHwe5G26jFdH2/zDM8iqWoowAPw69+NRFMgn20pQeYU1aFPQlliBvylJlZGCD7pAc4hoDmOcbSVVClo9EKqLB2ffTZnjZRJol7LPWm1jEsj5/OFB+bZX5o7PCaPLYsMAmAXwpSLxbrKs3/u3Ul3Bkb9qp1X0fZDSgdQ9CNEosyRbZbFFQ3+R0VRUzFxCa1giN80bep4gA5AsNCOr/uuCpK4UqanWyx6XV6QUBfm2o7Dsab0ltiVLVPo4gSr/E29zKGBpvzYsMdaMjbc3puQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cTJ+28Aey7GgxxE7luuxNI/z/lA62dD9O8waK5z3Mnc=; b=u8p69bSSjwuHKKq9wPhKRtjufPJnXl2ff5U6RCznoUbSaYV2sJFkCLIblDDGbwGrIG/94V2URLHspGLaaerx/R8y6ar4YDSi/GK9h7bfHjORCaWfbZcEryW4sViETlJYyXAXUrz4aSTqGh0FOS94lQBxnPJO3zNoQ0sKfKtK6GgBLw8FFzjE3MbPUcDBFQngrhr+hR4vOIiRRSV7H3DhHKg091gubdfirUcX982/cYowmlGNGWT1ggTRxdZ8QvPM1uPplvWZpaLZv8fDw2D3xKipME8eqsNRKKQ4bbE27IYJx1JW5PEVkp0AcFXUhfCJBKJAYNgD3WhPU+a0lCNvrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cTJ+28Aey7GgxxE7luuxNI/z/lA62dD9O8waK5z3Mnc=; b=T3ClEUqoh0bWNOdHOgzIN7WDMyPDaEou+vtskXpFf+2cxpyjIhu9LnLjNEbEPAd4oExFHrhv/6CVlKg8QZd1dqe+13N/whivm2wTPA2qRRqlQclfckkJqCbVQehfO3PfGA5wy4dBVNoqpTlFEcpQSWoeN/LOPEQ2nzYz/ci6PBwNTUHLXtA9KWclH2+DTvChtYKwDXL4/0OttyPLqsbcvAFjiaUG2TRfeH6LmD4uErezWFCxlmDRrSXWG6LZpGoZ0RfXIf+CAYfZlO+BSql4FC0fdg4QLl7inB4N9fQANXu9NnOR5fiSQVyTcJcEsoOXpmAqz8wSaD409Huelo77gQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon , Gavin Shan , Wei Wang Subject: [PATCH v3 02/14] migration/ram: Use migration_bitmap_sync_precopy() for postcopy discard Date: Tue, 9 Jun 2026 10:58:00 +0300 Message-ID: <20260609075812.32067-3-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00005FFD:EE_|MW3PR12MB4476:EE_ X-MS-Office365-Filtering-Correlation-Id: 192ef29d-d063-444d-1be2-08dec5fd0db1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700016|7416014|376014|1800799024|82310400026|18002099003|22082099003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: zaMzxon7gouxoPh9XdP5mOsNVUW3cB3QvrwM92uROq/iZHI27UKnvfaPd+2y06nTTNatbCBsh0jVbfolxcSfAJEy2APMkjKHeodjMAir+9tK+n45t4+nHCVNCI4jgde4zQsdvl6wQH3q+ZEgdMxwrrpfuuYfrXxZ/nU8tWWMfzETXuSPBZDieAzpkIweJ4eMhLph8rErgxgF0rCJOezXu/BOcZW0NacyL4h/eYPOm5j4cQVHnJFUQfX0kMConhZbvh3sUeQE2DkR9yH7UKCuMULM2Ryb0Dg/KIkiHs8jIfOgkm6eA1QltVzop2yCSI5yJtaoDF2I+1nP57SU/se8XH5gtIJEcnBc6JD0Gh+Uj9p0CO7H20k/Gs3jteflVGnHaO0mdI2f7F7woySkJoc1nZ6esgvITIDK+/1AycidGHpbCvKaI6x8YtqnwjAstfX3t+jXza2HvG4DWyaGfuz0yhdUkETgU+Io+N2mjiU7+UKSrjv6DAEGt8WJep4uyM/7awuJco+c6sdQFilMmcAjl/qJP2lgZm3PTQoVJ97p0yTVXa+AgQ2VM6dU0LeVFyXWLjLoJFHSBGMFyhfbhfrPpzzEsxxheqD89o2wJtybj9WXKEFOxBsxPErfFqEG/Ly41XgJj/vNOtKAnbTE8Dskoq+MdBSFd5hZ75MwG0CEI1ZCGpXsgP6UJqgZXsMOxySw1kuvSDdExLVCxw4zL78tTOXKRknmnphv5LYoU4XIrns= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(36860700016)(7416014)(376014)(1800799024)(82310400026)(18002099003)(22082099003)(11063799006)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: w7fB4ESsJj69SonogTL+1iRJNNLvJbFduBk6dgVO21d7N/eI6+GaJuv/hYeIQkis5m4+2nXn1/wVu9kqi7uiys4MC8Qn5rzgIQQ/l7rwIBDC5JMuNnf9POQ3IqbpZR1RwkyJEHueo3DmKCrfzfwfAWGp23DOCdz+GJ7tB/LSI3sdYDMVpshyjdD0Y5dbTs6Cbl4NUSgVYpKZOr67Gl+mKnDnAIJkwHEwgubeoS6/NEcRYv8BUCp2uacBO+BHiHJURp8pzPKvcTbi+kVLRARAtbarVQBeqSLeybcWBvgC0HrZj2P7D0XoArqxs0zHrwopQCCaxFqBxzfAHncABIe2YmIPWutMcyfvo9UNmZWHBytMHC/TKima/fo5alDalHkoIuLagjffpVxD5Gq+trWnKfqv+dtw/zc8zeZ+sF9rMP2FkyceFCsWFWbbthETbgcJ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 07:59:37.8288 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 192ef29d-d063-444d-1be2-08dec5fd0db1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00005FFD.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4476 Received-SPF: permerror client-ip=2a01:111:f403:c110::1; envelope-from=avihaih@nvidia.com; helo=BN1PR04CU002.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992011723158500 Content-Type: text/plain; charset="utf-8" ram_postcopy_send_discard_bitmap() performs the final RAM dirty bitmap sync at postcopy switchover, before sending discard bitmap to the destination. Unlike the precopy switchover path, it currently calls the raw migration_bitmap_sync() helper and passes last_stage=3Dfalse. Postcopy switchover is also a stopped-VM final sync point, so use migration_bitmap_sync_precopy(true). This keeps RAM final bitmap sync handling consistent across precopy and postcopy switchover, including the precopy bitmap-sync notifier wrapper. Current notifier users are safe in postcopy: virtio-balloon free-page hinting already opts out when postcopy-ram is enabled [1]. This prepares for moving the final sync out of RAM completion/discard paths and into migration_switchover_start(), where the migration core can run a final save_query_pending pass for all modules uniformly. [1] fd51e54fa102 ("virtio-balloon: don't start free page hinting if postcop= y is possible") Signed-off-by: Avihai Horon --- migration/ram.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index fc38ffbf8a..d7b8cc61ea 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2684,7 +2684,7 @@ void ram_postcopy_send_discard_bitmap(MigrationState = *ms) RCU_READ_LOCK_GUARD(); =20 /* This should be our last sync, the src is now paused */ - migration_bitmap_sync(rs, false); + migration_bitmap_sync_precopy(true); =20 /* Easiest way to make sure we don't resume in the middle of a host-pa= ge */ rs->pss[RAM_CHANNEL_PRECOPY].last_sent_block =3D NULL; --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992042; cv=pass; d=zohomail.com; s=zohoarc; b=GF4Blono3LOKDHVOp8yw+CoHgkHY6VOrCuGOqjOje/PPHzQ1wH0Hj3V+QSlw0gwMAIIyRUTxvWW6LOv4Qmmb5l0kwHCr5hCA4z76GIslWsQXX5biy//tsqj/unpHogK01Kyyew7suDNDw74df72HhWczLCuI7aDPCxDPkRg+5eY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992042; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=eAwD98U32mbbZi3g+ttP0Biv1lW/8Yr+PypVnjpbCPI=; b=AX/YzCVhRltJ0Nl6V74hQTQ0SsjqRanYPsttF/1VTVrKmZfIIX4HFHEihxWNd4gIixSyCBavMHMfDsVIik07j3MV44u6Ds1UfRIrmqejeq2w+1QS0Qnwd1nP+dyHCM2BmeBMNu+R+XvYCSS9AV8DgRJLcuTPHBS35yR0lr6dK7Y= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992042122484.9083740325367; Tue, 9 Jun 2026 01:00:42 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrNW-00024L-9M; Tue, 09 Jun 2026 04:00:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNU-00023w-C5 for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:04 -0400 Received: from mail-eastusazlp170120007.outbound.protection.outlook.com ([2a01:111:f403:c101::7] helo=BL0PR03CU003.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNR-0003iD-OI for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:04 -0400 Received: from BN1PR13CA0009.namprd13.prod.outlook.com (2603:10b6:408:e2::14) by CH3PR12MB8755.namprd12.prod.outlook.com (2603:10b6:610:17e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.12; Tue, 9 Jun 2026 07:59:49 +0000 Received: from BN1PEPF00005FFE.namprd05.prod.outlook.com (2603:10b6:408:e2:cafe::75) by BN1PR13CA0009.outlook.office365.com (2603:10b6:408:e2::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 07:59:49 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00005FFE.mail.protection.outlook.com (10.167.243.230) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 07:59:49 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:32 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:32 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 00:59:26 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=IhfZuCevmFFtejKbsy1n7WVSR50D3yedI3hXhYd8ZaF9LqrJ4Z0/eK4mgWiXytChKiIJfnwiOWF82sIVCuveXOWRkOSCOwdAjXhVgnZW7DA05JGk3PmTeVYqinbGJ6CMMW6oULbvp5tf8kn9HWRRZDsfbmotT9HdVXKu0jLFSOsNKKl/+Tg2FpnEoIfE21kczyi/G8bQT617KAOyBcEmO0ruuPJS9gRcLl4gT2x2bBYLHjFRIbf/mxV7gs0eG7mk8E9GFn7I9jsWAZIAXoQojdmr5fPHDTbbQqifTdyvkEZJ6Buui4tYUT2/mg/J4oeCRXAxurvsR7c7EQqqHEKw0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eAwD98U32mbbZi3g+ttP0Biv1lW/8Yr+PypVnjpbCPI=; b=M2OY4E2gnTsF6E28Wq8S0vR1RT/tXPDozsmCKhvF41TzuHiPIJxGrmTBNd1RA/OuzsgfJn8+mU/1ZU9q/0oAtiD71pE3fVSfK1O/yeUSeK9tl1HaMCROcj4g9UBHkW6SlSY72ZHdZ6BHKMSJxbcY1ugpI7FMblrh1WAkMxzbpJRTHuT5UqtqPDjLcDGXj9GEA2j50Y2k/J36oCs/LyEbV2HeNynxqfoIzto7v2GrdS49s3qHhU2vQBePzsKO4+JxrH7wn9ZDPJ+/I8u5A2kjnIp5aPQpOO4xeOEHR+xviKDFDEpp5MZ7WEq1VPPQfoBE+QOXiGwrjvq0XKURY3+Ygw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eAwD98U32mbbZi3g+ttP0Biv1lW/8Yr+PypVnjpbCPI=; b=M6J5fHdjyTZaKqIF1/zHs/FVbJx4zqmsiCzvGI8iHnB9jgIFrFoD4JTvzKv8UdfEa5iIl9lHfr21AP3lxrdRjwYrZbSpwYEKVvjcloGWvZeSIACRfstsX706BoYQhG4yXJuZ3PwuS4cqkaDZRwpRehQMuVtG2gWBILLsnO/XLYVqLOA6Iz0T69igsExJoIPQU5i7g3pTPM/kj1vWDyvKppJWq4ACqcPThnf2oFp+nOmLJ4+jf+OTz+BltSsdJOV08M637OMWIBCWXxPKldiVzM3jwAuLy7Of3s+vU8d7AnmXOwqN1/ojZKgvEjNTAjyKAm5u43kPa0XuhHWMZ6oHdA== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 03/14] migration: Run final save_query_pending at switchover Date: Tue, 9 Jun 2026 10:58:01 +0300 Message-ID: <20260609075812.32067-4-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00005FFE:EE_|CH3PR12MB8755:EE_ X-MS-Office365-Filtering-Correlation-Id: 2a7304b7-a8bc-4b67-e008-08dec5fd1491 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700016|7416014|376014|82310400026|22082099003|18002099003|6133799003|5023799004|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: uimKNfDzRIZqjaho3L7SS59Xjee/DW+FGz7CyIHOOH8N2AlCiDMWV1SfFbmYOyXje6kFNSaPh5csGrnUzR5qENCyWkLD931Bq5WEzrW82j0b52SqQSMCsBDRDWWYozYqqakerFkBeX4eU+K7rLWSV0fxorZ+MjTqFbsfSkFaMSxi0qln0Y8Ks6u5siTd5F94DZfRRKH5zPFg+Jyi7Rjlrr2yy/iSoS1RyUymzpDw2CDdiXnxd+Nn4lWfOVc54BrqseIJAiFoj9bvYaX+cYr9AejIwrYx6oSFE/47BkO2/8QluNY/Zlc0CbsjGC+iRS9YSfd7YkTDuOkXCZayi63cxnQq21BEEil/t00oJU9W0hDMlz2ZvQSWYQGnXUHaH0xOpFivNJ4LjjLbc3q0C6FkywwP86ENMbv2kdrS/EjplTE/VjS4iWUQ0vzqCZJCeop1ZBS93WgaIeuxc57Q3cSc+hTXR7Y5Qng0IV6e3pyRHm+jFTSPpzGXp+S6lSqNLJvTw9RLpfQ1MZk20J5y2KBvsd41IhqgtvBl9wdtsOyb3KXjqig2H/OcT/vdBtPo5Wdl4jR/DzVTdFy6AMNMX1Qw0+LJA2PmeVNZ1qe5CCFpis1F/bJCLD1SbN6FOgv+G8UhdBuGwItRAFBt9ibAo7PN6W9j1t8BXKN5ZPF43LK5b9fWSjpkbPrIhMSryJ1JqY6S0evJ83r9V7/uzo98+kuVTAATQIrGC6yG41OTJDA3hVA= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(36860700016)(7416014)(376014)(82310400026)(22082099003)(18002099003)(6133799003)(5023799004)(11063799006)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fPN1kiR+REvW3ITK+7mZnTOC8DO3Bo40oXK2YnsEknT4vxYMhmHkRdPWo7HK64Xn3EQ6K271OnwN130tMeRNbSFE74DVIb5JlHwHD36PvE64/jsKpK0LET8CKa/Ab4NSP+tvDun5bA1RR9t33WcTUiz69pwdjtZbwpNKe1eFu95yEFEMtZIZkEMSQc+TbdBTXuKoyKn67+t+EhgeyaykicHiD+x7j0Up7siYql9S62zSFbHzP77VGFcyxAEj+NAr2ope7rVtcTYkmko3j9cy/Gz8TxcJ7oLX/J7kw9Kk9CY5kAf0XoYFo6cwVj6s8f+/iK0Aj8tXmHLI7+raJt6u1iq53vE/n54BX8lsXWHSEHaeytMBFv5CBaM6wafgbOlTUDyZphybOop8gtJDBF8G03G4wgxQNwJH3nYd3LtvVPv85ilGLf86h9HPM05x6QH1 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 07:59:49.3896 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2a7304b7-a8bc-4b67-e008-08dec5fd1491 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00005FFE.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8755 Received-SPF: permerror client-ip=2a01:111:f403:c101::7; envelope-from=avihaih@nvidia.com; helo=BL0PR03CU003.outbound.protection.outlook.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992044207154100 Content-Type: text/plain; charset="utf-8" Before switchover, the source needs one last exact pending query so modules can flush dirty state. This is currently done ad hoc in modules handlers. For example, RAM syncs its dirty bitmap in its save_complete handler. This should be a general concept relevant for any module, so extract it to migration core instead by running a final save_query_pending before switchover. The final query requires special handling by modules (e.g., it's called with BQL locked, during VM stop), so extend save_query_pending SaveVMHandlers callback and qemu_savevm_query_pending() with a "final" flag so migration modules can tell the last pending query during switchover from periodic iteration queries. Signed-off-by: Avihai Horon --- include/migration/register.h | 41 ++++++++++++++++++---------------- migration/savevm.h | 3 ++- hw/s390x/s390-stattrib.c | 2 +- hw/vfio/migration.c | 11 ++++++--- migration/block-dirty-bitmap.c | 11 ++++++--- migration/migration.c | 14 ++++++++++-- migration/ram.c | 40 +++++++++++++++++++-------------- migration/savevm.c | 20 +++++++++++++---- hw/vfio/trace-events | 2 +- migration/trace-events | 2 +- 10 files changed, 94 insertions(+), 52 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index 5e5e0ee432..6f632123f1 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -171,6 +171,28 @@ typedef struct SaveVMHandlers { */ bool (*is_active_iterate)(void *opaque); =20 + /** + * @save_query_pending + * + * This estimates the remaining data to transfer on the source side. + * + * When @exact is true, a module must report accurate results. When + * @exact is false, a module may report estimates. + * + * It's highly recommended that modules implement a faster version of + * the query path (for example, by proper caching on the counters) if + * an accurate query will be time-consuming. + * + * @opaque: data pointer passed to register_savevm_live() + * @pending: pointer to a MigPendingData struct + * @exact: set to true for an accurate (slow) query + * @final: set to true for the final query during switchover. When fin= al is + * true, the query is called with BQL locked. Otherwise, it's called w= ith + * BQL unlocked. + */ + void (*save_query_pending)(void *opaque, MigPendingData *pending, + bool exact, bool final); + /* This runs outside the BQL in the migration case, and * within the lock in the savevm case. The callback had better only * use data that is local to the migration thread or protected @@ -210,25 +232,6 @@ typedef struct SaveVMHandlers { */ bool (*save_postcopy_prepare)(QEMUFile *f, void *opaque, Error **errp); =20 - /** - * @save_query_pending - * - * This estimates the remaining data to transfer on the source side. - * - * When @exact is true, a module must report accurate results. When - * @exact is false, a module may report estimates. - * - * It's highly recommended that modules implement a faster version of - * the query path (for example, by proper caching on the counters) if - * an accurate query will be time-consuming. - * - * @opaque: data pointer passed to register_savevm_live() - * @pending: pointer to a MigPendingData struct - * @exact: set to true for an accurate (slow) query - */ - void (*save_query_pending)(void *opaque, MigPendingData *pending, - bool exact); - /** * @load_state * diff --git a/migration/savevm.h b/migration/savevm.h index b6bb4fa977..81abd96dda 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -45,7 +45,8 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy); void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); -void qemu_savevm_query_pending(MigPendingData *pending, bool exact); +void qemu_savevm_query_pending_iter(MigPendingData *pending, bool exact); +void qemu_savevm_query_pending_final(MigPendingData *pending); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); void qemu_savevm_state_end(QEMUFile *f); diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index c334714b31..b0f04eb30c 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -190,7 +190,7 @@ static int cmma_save_setup(QEMUFile *f, void *opaque, E= rror **errp) } =20 static void cmma_state_pending(void *opaque, MigPendingData *pending, - bool exact) + bool exact, bool final) { S390StAttribState *sas =3D S390_STATTRIB(opaque); S390StAttribClass *sac =3D S390_STATTRIB_GET_CLASS(sas); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index fb12b9717f..195498845e 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -622,13 +622,18 @@ static void vfio_state_pending_sync(VFIODevice *vbase= dev) } =20 static void vfio_state_pending(void *opaque, MigPendingData *pending, - bool exact) + bool exact, bool final) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; uint64_t precopy_size, stopcopy_size; =20 - if (exact) { + /* + * The final pending query runs during switchover downtime. VFIO does = not + * need a fresh device pending-data query then to get the latest dirty + * data, so avoid the extra work and report the cached counters below. + */ + if (exact && !final) { vfio_state_pending_sync(vbasedev); } =20 @@ -646,7 +651,7 @@ static void vfio_state_pending(void *opaque, MigPending= Data *pending, =20 trace_vfio_state_pending(vbasedev->name, migration->stopcopy_size, migration->precopy_init_size, - migration->precopy_dirty_size, exact); + migration->precopy_dirty_size, exact, final); } =20 static bool vfio_is_active_iterate(void *opaque) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 7ef3759e53..cba54e25cd 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -767,13 +767,16 @@ static int dirty_bitmap_save_complete(QEMUFile *f, vo= id *opaque) } =20 static void dirty_bitmap_state_pending(void *opaque, MigPendingData *data, - bool exact) + bool exact, bool final) { DBMSaveState *s =3D &((DBMState *)opaque)->save; SaveBitmapState *dbms; uint64_t pending =3D 0; =20 - bql_lock(); + /* Final pending query is called with BQL locked */ + if (!final) { + bql_lock(); + } =20 QSIMPLEQ_FOREACH(dbms, &s->dbms_list, entry) { uint64_t gran =3D bdrv_dirty_bitmap_granularity(dbms->bitmap); @@ -783,7 +786,9 @@ static void dirty_bitmap_state_pending(void *opaque, Mi= gPendingData *data, pending +=3D DIV_ROUND_UP(sectors * BDRV_SECTOR_SIZE, gran); } =20 - bql_unlock(); + if (!final) { + bql_unlock(); + } =20 trace_dirty_bitmap_state_pending(pending); =20 diff --git a/migration/migration.c b/migration/migration.c index aad23f3228..929a6c432c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2787,12 +2787,22 @@ static bool migration_switchover_prepare(MigrationS= tate *s) static bool migration_switchover_start(MigrationState *s, Error **errp) { ERRP_GUARD(); + MigPendingData pending =3D {}; =20 if (!migration_switchover_prepare(s)) { error_setg(errp, "Switchover is interrupted"); return false; } =20 + /* + * The final query to the whole system on dirty data to make sure we + * collect the latest status of the VM. For precopy, source QEMU will + * dump all the dirty data during switchover. For postcopy, this will + * properly update all the dirty bitmaps to finally generate the + * correct discard bitmaps; see ram_postcopy_send_discard_bitmap(). + */ + qemu_savevm_query_pending_final(&pending); + /* Inactivate disks except in COLO */ if (!migrate_colo()) { /* @@ -3285,7 +3295,7 @@ static void migration_iteration_go_next(MigPendingDat= a *pending) /* * Do a slow sync first before boosting the iteration count. */ - qemu_savevm_query_pending(pending, true); + qemu_savevm_query_pending_iter(pending, true); =20 /* * Update the dirty information for the whole system for this @@ -3336,7 +3346,7 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) bool complete_ready; =20 /* Fast path - get the estimated amount of pending data */ - qemu_savevm_query_pending(&pending, false); + qemu_savevm_query_pending_iter(&pending, false); =20 if (in_postcopy) { /* diff --git a/migration/ram.c b/migration/ram.c index d7b8cc61ea..079a9b9275 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2683,9 +2683,6 @@ void ram_postcopy_send_discard_bitmap(MigrationState = *ms) =20 RCU_READ_LOCK_GUARD(); =20 - /* This should be our last sync, the src is now paused */ - migration_bitmap_sync_precopy(true); - /* Easiest way to make sure we don't resume in the middle of a host-pa= ge */ rs->pss[RAM_CHANNEL_PRECOPY].last_sent_block =3D NULL; rs->last_seen_block =3D NULL; @@ -3376,10 +3373,6 @@ static int ram_save_complete(QEMUFile *f, void *opaq= ue) rs->last_stage =3D !migration_in_colo_state(); =20 WITH_RCU_READ_LOCK_GUARD() { - if (!migration_in_postcopy()) { - migration_bitmap_sync_precopy(true); - } - ret =3D rdma_registration_start(f, RAM_CONTROL_FINISH); if (ret < 0) { qemu_file_set_error(f, ret); @@ -3442,25 +3435,38 @@ static int ram_save_complete(QEMUFile *f, void *opa= que) return qemu_fflush(f); } =20 -static void ram_state_pending(void *opaque, MigPendingData *pending, - bool exact) +static void ram_state_pending_sync(bool exact, bool final) { - RAMState **temp =3D opaque; - RAMState *rs =3D *temp; - uint64_t remaining_size; - /* * Sync is not needed either with: (1) a fast query, or (2) after * postcopy has started (no new dirty will generate anymore). */ - if (exact && !migration_in_postcopy()) { + if (!exact || migration_in_postcopy()) { + return; + } + + /* Final pending query is called with BQL locked */ + if (!final) { bql_lock(); - WITH_RCU_READ_LOCK_GUARD() { - migration_bitmap_sync_precopy(false); - } + } + + WITH_RCU_READ_LOCK_GUARD() { + migration_bitmap_sync_precopy(final); + } + + if (!final) { bql_unlock(); } +} + +static void ram_state_pending(void *opaque, MigPendingData *pending, + bool exact, bool final) +{ + RAMState **temp =3D opaque; + RAMState *rs =3D *temp; + uint64_t remaining_size; =20 + ram_state_pending_sync(exact, final); remaining_size =3D rs->migration_dirty_pages * TARGET_PAGE_SIZE; =20 if (migrate_postcopy_ram()) { diff --git a/migration/savevm.c b/migration/savevm.c index 9d1d58c8f4..ed62defadc 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1801,7 +1801,8 @@ int qemu_savevm_state_complete_precopy(MigrationState= *s, Error **errp) return 0; } =20 -void qemu_savevm_query_pending(MigPendingData *pending, bool exact) +static void qemu_savevm_query_pending(MigPendingData *pending, bool exact, + bool final) { SaveStateEntry *se; =20 @@ -1814,7 +1815,7 @@ void qemu_savevm_query_pending(MigPendingData *pendin= g, bool exact) if (!qemu_savevm_state_active(se)) { continue; } - se->ops->save_query_pending(se->opaque, pending, exact); + se->ops->save_query_pending(se->opaque, pending, exact, final); } =20 pending->total_bytes =3D pending->precopy_bytes + @@ -1826,13 +1827,24 @@ void qemu_savevm_query_pending(MigPendingData *pend= ing, bool exact) * close to reality when this got invoked frequently while iterating. */ mig_stats.dirty_bytes_total =3D pending->total_bytes; - - trace_qemu_savevm_query_pending(exact, pending->precopy_bytes, + trace_qemu_savevm_query_pending(exact, final, pending->precopy_bytes, pending->stopcopy_bytes, pending->postcopy_bytes, pending->total_bytes); } =20 +void qemu_savevm_query_pending_iter(MigPendingData *pending, bool exact) +{ + qemu_savevm_query_pending(pending, exact, false); +} + +void qemu_savevm_query_pending_final(MigPendingData *pending) +{ + g_assert(bql_locked()); + + qemu_savevm_query_pending(pending, true, true); +} + void qemu_savevm_state_cleanup(void) { SaveStateEntry *se; diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 2049159015..8f57d0b7d8 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -175,7 +175,7 @@ vfio_save_device_config_state(const char *name) " (%s)" vfio_save_iterate(const char *name, uint64_t precopy_init_size, uint64_t p= recopy_dirty_size) " (%s) precopy initial size %"PRIu64" precopy dirty size= %"PRIu64 vfio_save_iterate_start(const char *name) " (%s)" vfio_save_setup(const char *name, uint64_t data_buffer_size) " (%s) data b= uffer size %"PRIu64 -vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool exact) " (%s) stopcopy siz= e %"PRIu64" precopy initial size %"PRIu64" precopy dirty size %"PRIu64 " ex= act %d" +vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool exact, bool final) " (%s) = stopcopy size %"PRIu64", precopy initial size %"PRIu64", precopy dirty size= %"PRIu64", exact %d, final %d" vfio_vmstate_change(const char *name, int running, const char *reason, con= st char *dev_state) " (%s) running %d reason %s device state %s" vfio_vmstate_change_prepare(const char *name, int running, const char *rea= son, const char *dev_state) " (%s) running %d reason %s device state %s" =20 diff --git a/migration/trace-events b/migration/trace-events index de99d976ab..1c9212d3e2 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -7,7 +7,7 @@ qemu_loadvm_state_section_partend(uint32_t section_id) "%u" qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" -qemu_savevm_query_pending(bool exact, uint64_t precopy, uint64_t stopcopy,= uint64_t postcopy, uint64_t total) "exact=3D%d, precopy=3D%"PRIu64", stopc= opy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PRIu64 +qemu_savevm_query_pending(bool exact, bool final, uint64_t precopy, uint64= _t stopcopy, uint64_t postcopy, uint64_t total) "exact=3D%d, final=3D%d, pr= ecopy=3D%"PRIu64", stopcopy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PR= Iu64 loadvm_state_switchover_ack_needed(unsigned int switchover_ack_pending_num= ) "Switchover ack pending num=3D%u" loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992039; cv=pass; d=zohomail.com; s=zohoarc; b=J1XISCTxsQhQUpSY/9eTXsrOMpZVqRD5w55E+RpXFudP2q+SXn2sYtmxegDHW0xKGOQ6FgEO1qKlhhFt3Rhqzscw+6+3LhxVmuJYTWNcMLnZ29SkDjpHyw932hpeDOn4X7/25qSrxMfJVqKhCpRWOHulkFh8asJ0BfmGG4PsLMY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992039; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Inqtjd09vCQC4e29otc598yk0i6N+toKmv9BnBlyE0U=; b=Vgw9V53hUzvHnGZC3wQIbMs4LvKmKDJbakAkpjzJNJKpINpX7dakYeSMwvc6+d575lfJbhrGKcPqetBYY+9gocPO884rOkqKaPq86XBeC+vorBdb6Shm/nGpP6ttzrFJh2qNd/zDKZlY2jLAN0Lo5/GwSofVVQ3pvHeMfzbvndM= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17809920397458.312628174184965; Tue, 9 Jun 2026 01:00:39 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrNa-00026X-Gx; Tue, 09 Jun 2026 04:00:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNY-00024b-IG for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:08 -0400 Received: from mail-westus2azlp170100005.outbound.protection.outlook.com ([2a01:111:f403:c005::5] helo=CO1PR03CU002.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNW-0003zL-4m for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:08 -0400 Received: from BN1PR13CA0002.namprd13.prod.outlook.com (2603:10b6:408:e2::7) by IA1PR12MB8405.namprd12.prod.outlook.com (2603:10b6:208:3d8::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.14; Tue, 9 Jun 2026 08:00:00 +0000 Received: from BN1PEPF00006000.namprd05.prod.outlook.com (2603:10b6:408:e2:cafe::79) by BN1PR13CA0002.outlook.office365.com (2603:10b6:408:e2::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 08:00:00 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00006000.mail.protection.outlook.com (10.167.243.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 07:59:59 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:43 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:42 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 00:59:36 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=M4KM+RiR7LqfU+uxAhzjj3z9J0Mlk6zpHvP25RRzc9mBTR2e1iwbbz2oEiUuWDEs4ZFcAPxEX5MsWDQSm4Ld99a293p5APWxxqpa2Ht8tYvnqo21Y8ThZNgttEEFrnKuEjIXfSWGzJTMcGEcmB9nQq1B6ps0bAbmPdBK5wleug9n8CkX3yVOV27hkcAH2qgdaPJIvBWX+D3Dc8o7JKPYfhh+eU15rkPAVFXj5z/4jvhep/unX73mMpbeE7/yDlezWpnglCA1BE/cBkC1eH7Bwigt1unWpHecw70HEOmecr+MAv99/NXGbPP4nnJmixPjEJoYzdtcxM+i/JOtBhk/ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Inqtjd09vCQC4e29otc598yk0i6N+toKmv9BnBlyE0U=; b=GyCWPK5l2PbJlO2RyX69aSXQ9dKL20ZOw5HU3be5chYzmV3az6u7xqt/fhb1P+eTXBrBrXruWkgQyd6VHj23+wgx8Q93PClmnx+LcRxbIZbI9wImVRT8fa/rOZ4xhQxg+hz7aw5H2JVIftRRtL8Gaf4brf7be/XVLAynei6NR6FVrDp7OWLeIznKlyh5aVmK4vaGCCGNyJID9fhJQS2uDj9vQgAmmCt5rqdL49rXCP/PiTnp+c7rCqDUjg6iZqOz6ptFor5+5EWP78ja1/29INJiOANMDkfbcqJ7HpBnWuhakiKK0K0fDkO2YLrcRXR4Kgy6eMubQNf7n+XyMsiIxA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Inqtjd09vCQC4e29otc598yk0i6N+toKmv9BnBlyE0U=; b=MoZFcjX9+k34LPBJ9bw72BqjnrOFsdqHlm4wcCJGmf5Vso3w9VVl6Lb5FGu0JPDu5Tgmqm12ygrrB29J60PFxTYGt6p7nyKQwXxgk3OHND3N3i8sQLlHXV7i+lmE7Y48Ty92h+TPf3sf5wFYiU1yH5b7qC4vPjO30Xc+6V+CEhUKb0BPMJshTkZc4HcA9NiyWae/6qvJuWI74WrTaWlqKNr18TWtUI/APoOEq8iCe/uyakxVdO5MNk44Z/73XDoFfzlyAtXSj/627t757AwVtN/SMGFg8PHFn9riYmobulgJAcP/1ZzZmAjDGJiBGsYULgVJCS9eWNaqY0/04G4g8A== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 04/14] migration: Log the approver in qemu_loadvm_approve_switchover() Date: Tue, 9 Jun 2026 10:58:02 +0300 Message-ID: <20260609075812.32067-5-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00006000:EE_|IA1PR12MB8405:EE_ X-MS-Office365-Filtering-Correlation-Id: cf83e228-e083-4a41-5b67-08dec5fd1aaf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700016|7416014|376014|1800799024|82310400026|11063799006|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: R9sJFkvLncrrmr1gIhyPOQiwZw9VdHkXTHNr92jomfYeQ+8Zt5UOj42CICCA9Ryi1d9Trbfd2FDBFdupvPYfugQKHVXeqELmRhVFbUGetX8LZ5Mb9wD1WCAj9qfDUGFfUfNAG6lGntADTk7bezaJ6HKO1SYty5wlWyjB5FF0vgpIALmVKGMp36hwf7WxrIf/+neL1iaiRON0J614cevvzTA+E3kLnUxVnf5ritfUzOV1nQ2hWbsnhbuXoo+WWXSy7NQeCboooHEISBx4iXXSSBf63iMxUEgKSwryOWfMhJ6ZVaoqgDVlo4zUbivaEtPVKyTHPEQjx3VrvJyEbYX/oGoAB3ywT6UtXL8Jlg5SOfxfnDwIa3VRZ3UMucI74HBC8Mhx4WnAuIVEF11m2CcmDnQ3BlL9QztkiuG8T1lKkqOiEDhGRA8zHrxcs8d8MZK7kkf1+taonZCnQ3oRYqzUE0KbCu9xYc8yLdzSZ++08NRyuvTGeMG4FKsocDBNgknTOxLHjq5snujapGr/AlaB7fESsHSPTuB/KMVYT7Bbj+tX3CHQ2+P4+R+4I48HfdoXR0ghXY0+YLy2zcUAvYERyVdzcepscx+qFW9ifPtjHd0JCPA86rTbKTmSrLHlVNh4R2I9GQi4dDJhrl6oNq5yOWswtRRJLfAWW8vbpuNmdZ6PbFTxH7GXYtAbmHnVAZkCS8/vBNdZLUkhHiMOqKraOZiEu1esOmdUDpFH2hvK2nw= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(36860700016)(7416014)(376014)(1800799024)(82310400026)(11063799006)(56012099006)(18002099003)(22082099003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 2fP2dhmEyGwwVBgd32ns4G09AT32DQH86T1axKiA+3WJKAMgkmFoOFFlsLfDdASv1fPx6eHTWfvFr8lxNU8KhSlMMr/O1Cu3SNoShrJ5bt9mO0MeNVSjfsp50m2KRiuPfwAen4I0YbqfYyh2HC+vFdyrQe1k/nKbPOVydoYsZmzNalJLvNvCE2bF9R9LdLX8Z/RrGfBzle4k6UfBHDz194fcJb70vePy87Q5E6m1TagYHBADYJwaaUFEEq+RmpqTFBveuaTseZfZyw9KQaEfjIqgVDSGsux08Svpm2m3V7MewKBAkYoCRZ6l1jWhSRN9nTVlh9ToQE4s1gUzeEiiVqVFqmtqYNFbvaPRGqHe3gH2y0JFzCpqEFcB49xjV3O9T9YJWdQxFvR4JSXi6BwTTcY1biW6ITgTZTY9NQSzQZbfc2Q/CHJBrgUjninYA4vk X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 07:59:59.6373 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cf83e228-e083-4a41-5b67-08dec5fd1aaf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00006000.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8405 Received-SPF: permerror client-ip=2a01:111:f403:c005::5; envelope-from=avihaih@nvidia.com; helo=CO1PR03CU002.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992041850158500 Content-Type: text/plain; charset="utf-8" Pass the device name that approved switchover to qemu_loadvm_approve_switchover() and log it in the trace for debugging purposes. Reviewed-by: Peter Xu Signed-off-by: Avihai Horon --- migration/savevm.h | 2 +- hw/vfio/migration.c | 2 +- migration/savevm.c | 4 ++-- migration/trace-events | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index 81abd96dda..44424be347 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -71,7 +71,7 @@ void qemu_loadvm_state_cleanup(MigrationIncomingState *mi= s); int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis, Error **errp); int qemu_load_device_state(QEMUFile *f, Error **errp); -int qemu_loadvm_approve_switchover(void); +int qemu_loadvm_approve_switchover(const char *approver); int qemu_savevm_state_non_iterable(QEMUFile *f, Error **errp); int qemu_savevm_state_non_iterable_early(QEMUFile *f, JSONWriter *vmdesc, diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 195498845e..180b316bae 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -847,7 +847,7 @@ static int vfio_load_state(QEMUFile *f, void *opaque, i= nt version_id) return -EINVAL; } =20 - ret =3D qemu_loadvm_approve_switchover(); + ret =3D qemu_loadvm_approve_switchover(vbasedev->name); if (ret) { error_report( "%s: qemu_loadvm_approve_switchover failed, err=3D%d (= %s)", diff --git a/migration/savevm.c b/migration/savevm.c index ed62defadc..8ff6c0b17e 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -3174,7 +3174,7 @@ int qemu_load_device_state(QEMUFile *f, Error **errp) return 0; } =20 -int qemu_loadvm_approve_switchover(void) +int qemu_loadvm_approve_switchover(const char *approver) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 @@ -3183,7 +3183,7 @@ int qemu_loadvm_approve_switchover(void) } =20 mis->switchover_ack_pending_num--; - trace_loadvm_approve_switchover(mis->switchover_ack_pending_num); + trace_loadvm_approve_switchover(approver, mis->switchover_ack_pending_= num); =20 if (mis->switchover_ack_pending_num) { return 0; diff --git a/migration/trace-events b/migration/trace-events index 1c9212d3e2..c0c433744c 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -24,7 +24,7 @@ loadvm_postcopy_ram_handle_discard_end(void) "" loadvm_postcopy_ram_handle_discard_header(const char *ramid, uint16_t len)= "%s: %ud" loadvm_process_command(const char *s, uint16_t len) "com=3D%s len=3D%d" loadvm_process_command_ping(uint32_t val) "0x%x" -loadvm_approve_switchover(unsigned int switchover_ack_pending_num) "Switch= over ack pending num=3D%u" +loadvm_approve_switchover(const char *approver, unsigned int switchover_ac= k_pending_num) "Approver %s, switchover_ack_pending_num %u" postcopy_ram_listen_thread_exit(void) "" postcopy_ram_listen_thread_start(void) "" qemu_savevm_send_postcopy_advise(void) "" --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992039; cv=pass; d=zohomail.com; s=zohoarc; b=gmMqJ6jBiWlWy49OowpczizZNpwRSJxDRIUyHhbB52MaL7ymgnSbc2ozW62ocIwu/ulLye5c/22Uc/9ibFXVNzgv/rSmFpDTNytbuoWXo1T7dWDVUMJH2hSchwU5e0ZYzfmm4Rp0yZNm8TeGqYo1kLL8h9ExpiNwN8P30t7q/qw= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992039; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=aqaE1SyekmHJc3gLjGyn6sEPd+um0NqBnn1hwMI6cps=; b=ItZP+Z9gdwn8RLw4JdQtJDUX7E+gO2Vi1oge/F/wIwbSTuEU5wWbc+UVwfgkrDtj/uMQmgtwg6eAkj9YTh1AEg47RJh45peWPeopEKoRbQA7sDppf9MbhLqTKXkGYyOuvQJ9ldLITwKeH861O78UL5VvwKDHEWV280LbNvCgAcw= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992039962467.02686810690966; Tue, 9 Jun 2026 01:00:39 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrNm-0002Bj-Ks; Tue, 09 Jun 2026 04:00:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNl-0002AU-0i for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:21 -0400 Received: from mail-eastus2azlp170100001.outbound.protection.outlook.com ([2a01:111:f403:c110::1] helo=BN1PR04CU002.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNj-0004Cg-16 for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:20 -0400 Received: from DS1PR03CA0009.namprd03.prod.outlook.com (2603:10b6:8:450::15) by MW4PR12MB6897.namprd12.prod.outlook.com (2603:10b6:303:20a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.13; Tue, 9 Jun 2026 08:00:10 +0000 Received: from CY4PEPF0000EE3C.namprd03.prod.outlook.com (2603:10b6:8:450:cafe::26) by DS1PR03CA0009.outlook.office365.com (2603:10b6:8:450::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 08:00:09 +0000 Received: from mail.nvidia.com (216.228.117.160) by CY4PEPF0000EE3C.mail.protection.outlook.com (10.167.242.13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:00:09 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:52 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 00:59:52 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 00:59:46 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uIfgUOAr2IJ9lMAazTzQ/oviFwCdeMkY4KnTkme6zFX4yMMzUE14s0CZ79CBjMVEnqjZ/6J1Z/2EtyC8IgkgHKyI0QJtfVe5+luNOMZFVtip9vmXLIaPWC2PgP2PEQeqk01xg0PpsbsqULZ0WjjzqagYt2RA2xC+RWjqrVpzVT7KprMJ6jOyNNtARbMvSQneMBjpiXdM75BwgEn7Vl8DGhIxFRGRdz7bLsAJas5zhg40CRTgQvow3gitZZwnqzLSNU45JJCkSMVWROTwYYUpczRE9T2kfszUPWpHDSO3NK5/M5ysp2Z6erDIh1+mcikVMqQ/mOQMjcANFpC9/Z8EoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aqaE1SyekmHJc3gLjGyn6sEPd+um0NqBnn1hwMI6cps=; b=MZ21yEJUFwrZrLiL5+wlKofWzbGYJdMr4qQXbkQwfwE3k05jW9GNCuUT50XmLQgRc7XdMM1EyQYxtbdMMSlen8mBTb6e+KdFZvSrkVDHOhJkbwHf7nFNumnJmwD+aXuhFmsS7cERE0nIR/j+o+DFkR3tIyLIYfY2Cv8jnEZiZp2xWW0TMgz24B17li1ob18QOWTtTqY1krBy9SKixa+w0xtQwkw4Yix+YX8+FKXrijSunfFutThFLYhszOUZYbjstqIgftTqtTGhimK24xbkDQgvIHLCgjCd32m8R3Q0SwSjIhrE1kD9x0lqEfB3MHDY7usSAwyXQkC9WZvk5UfSpw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aqaE1SyekmHJc3gLjGyn6sEPd+um0NqBnn1hwMI6cps=; b=TFzQ2R4v7w2yzJ42QitgM7hVzMLoUJ3/CCWnuSFaDJKK/JoQHOX/6/BI8FZkc0LdBKtzhC/4JlqQdhS76aVUmFNYgfabro7HDjsVYX8rh4VkdWRPwdrqHxn5/5vTD0FeNSh70R+2rGXVbnT+j4GJ0/hIUCL7bNpo3GaMMYpNSsFzTpFnyqltLqtEOZ2Ck8AgKJKa2Eo/bqdPA8Pm6x5VBT/IiOBqFbbjxxDdMJp1zwmlYHOkoLCJlTdo8tTtuMcJiMCfsEvj0v1hmErAxVuXVaMRD9gYTZJWAE85xiVgl+okK0/I2k2VNxUjw+GFOLSxnujCdlsSqmwGhE/DPdwx5A== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 05/14] migration: Replace switchover_ack_needed SaveVMHandler Date: Tue, 9 Jun 2026 10:58:03 +0300 Message-ID: <20260609075812.32067-6-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE3C:EE_|MW4PR12MB6897:EE_ X-MS-Office365-Filtering-Correlation-Id: 35ad7029-8575-4f0b-fd15-08dec5fd2070 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700016|7416014|376014|82310400026|22082099003|18002099003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: dYjee91JquxzYwZdBIaoFbE6GLXI+j0rGoF6lnl/GKsk754JNx0EFYM0U1Ls+/NjukNrD6GJOovbGCGaFmoSouDzoR0T1ljS0OZu+xr6ihnwL0WOqam/vXq4oahoL49xUXi8eXy9JvLymWs++G2YraWj1Z/OPmorSl69Wktxua8ml7hCFyfjRHn8pYGwbTHTMIlvGk73Cl/3NprRTvXLGF5qekylq/6JeF5KTqqJ/ayIzln+lkxq3QZPPocAveJSBVkDs9XyGSbvvmhq9e4oZH3fPP8iuSQpD6gdFFaLsf4xUhmMFc1TE5D5L/GOmDcMZc1x5IdFM9tIUXYcsYxg+zqNNNQgcnWVgprmKTUz6muPRW8zxq1fVvUOVA8wJ9yWoniJdKCvg9tdE04tIrfojWIMeD/324IJrGYrDyuFvF3EdL7I7FELRc526ckwXAc8nR905PhXAKUREHvoWPEGVxo8JDPjVrynHLYsAR46CLaHJPIlIoblSh5bbMYHNUkMiYwlHUH/yZL3RpD6BFtMOllWIAk1oXkTBH1uj1SunBiHSWM2+AGcwBAg4Vgz0bZslCgmj6EHF971FSLTRlIv1cIw+BxL2vA54WZ+lkZMnAmEoUZun4XdbFdt/MSVrwTH2WRqHVNohR+WWzSMZYJedxlPCcpPxzCgpEY5q91VUz+DMy3VoAxqAO8yRNEfxOXV6TbuXGDtMScyF6sujNL7mkNeTKfpboX9fH/lC7F7qP4= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(36860700016)(7416014)(376014)(82310400026)(22082099003)(18002099003)(11063799006)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /Fxaz3MSylN/eFvLyxtwIVs+jWmxYlgLRrbJPgwsEb69yYxmkCjNzCVaLyeob2bbC+Aa3XjqhyOaP2YurB4PqIbZ0/m1eJqYrqv5tuMJpowVgaSUVgLHLZ9ELVmtDllGzzyuWqUOzPEhfkGU0hd2qXWvXbfZLjKe+u9bE8FOOTSCPckfwa1ocMKsR3LKcqPXhjueLHzFNG0cMv+chf7UZQ6Ngu1mSdhK/pZMg3udFseXn5TFdbP2PoXUz8LIiKHkeUDSq0xv9QrmRIMDhO09t9l/cqxmz2al/nQGrhK00ZhJgpCctXVsjKG2U/P/imn/ovcwZ+J2OMq2I8sJtuxnHUUOHo9Ln+FhiB6T9eLfG6zVNiat2OW0pp8VuLfobcdXi+GuT8eWPcdanf9+jEs9ahbRmbrROBRK8AFHZbRe9khJ1Ap6AAjd9nhuxvujNmbL X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:00:09.3549 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 35ad7029-8575-4f0b-fd15-08dec5fd2070 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE3C.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6897 Received-SPF: permerror client-ip=2a01:111:f403:c110::1; envelope-from=avihaih@nvidia.com; helo=BN1PR04CU002.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992041793154100 Content-Type: text/plain; charset="utf-8" A new switchover-ack mechanism that will replace the existing one will be added in the following patches. The new mechanism will not use switchover_ack_needed SaveVMHandler, however, the old mechanism must still be kept for backward compatibility. To keep things clear and decrease API surface of old code, replace switchover_ack_needed SaveVMHandler with a regular function migration_request_switchover_ack(). No functional changes intended. Acked-by: Peter Xu Signed-off-by: Avihai Horon --- docs/devel/migration/vfio.rst | 3 --- include/migration/misc.h | 2 ++ include/migration/register.h | 13 ------------- hw/vfio/migration.c | 18 ++++++++++-------- migration/migration.c | 15 +++++++++++++++ migration/savevm.c | 21 --------------------- migration/trace-events | 2 +- 7 files changed, 28 insertions(+), 46 deletions(-) diff --git a/docs/devel/migration/vfio.rst b/docs/devel/migration/vfio.rst index 691061d182..854277b11c 100644 --- a/docs/devel/migration/vfio.rst +++ b/docs/devel/migration/vfio.rst @@ -59,9 +59,6 @@ VFIO implements the device hooks for the iterative approa= ch as follows: * A ``save_live_iterate`` function that reads the VFIO device's data from = the vendor driver during iterative pre-copy phase. =20 -* A ``switchover_ack_needed`` function that checks if the VFIO device uses - "switchover-ack" migration capability when this capability is enabled. - * A ``switchover_start`` function that in the multifd mode starts a thread= that reassembles the multifd received data and loads it in-order into the dev= ice. In the non-multifd mode this function is a NOP. diff --git a/include/migration/misc.h b/include/migration/misc.h index 3159a5e53c..a2219c981b 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -156,4 +156,6 @@ bool multifd_device_state_save_thread_should_exit(void); void multifd_abort_device_state_save_threads(void); bool multifd_join_device_state_save_threads(void); =20 +void migration_request_switchover_ack(const char *requester); + #endif diff --git a/include/migration/register.h b/include/migration/register.h index 6f632123f1..a61c4236d2 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -302,19 +302,6 @@ typedef struct SaveVMHandlers { */ int (*resume_prepare)(MigrationState *s, void *opaque); =20 - /** - * @switchover_ack_needed - * - * Checks if switchover ack should be used. Called only on - * destination. - * - * @opaque: data pointer passed to register_savevm_live() - * - * Returns true if switchover ack should be used and false - * otherwise - */ - bool (*switchover_ack_needed)(void *opaque); - /** * @switchover_start * diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 180b316bae..7055cfbd3e 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -487,6 +487,14 @@ static bool vfio_precopy_supported(VFIODevice *vbasede= v) return migration->mig_flags & VFIO_MIGRATION_PRE_COPY; } =20 +static void vfio_request_switchover_ack(VFIODevice *vbasedev) +{ + if (vfio_precopy_supported(vbasedev)) { + /* Precopy support implies switchover-ack is needed */ + migration_request_switchover_ack(vbasedev->name); + } +} + /* ---------------------------------------------------------------------- = */ =20 static int vfio_save_prepare(void *opaque, Error **errp) @@ -776,6 +784,8 @@ static int vfio_load_setup(QEMUFile *f, void *opaque, E= rror **errp) return ret; } =20 + vfio_request_switchover_ack(vbasedev); + return 0; } =20 @@ -874,13 +884,6 @@ static int vfio_load_state(QEMUFile *f, void *opaque, = int version_id) return ret; } =20 -static bool vfio_switchover_ack_needed(void *opaque) -{ - VFIODevice *vbasedev =3D opaque; - - return vfio_precopy_supported(vbasedev); -} - static int vfio_switchover_start(void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -904,7 +907,6 @@ static const SaveVMHandlers savevm_vfio_handlers =3D { .load_setup =3D vfio_load_setup, .load_cleanup =3D vfio_load_cleanup, .load_state =3D vfio_load_state, - .switchover_ack_needed =3D vfio_switchover_ack_needed, /* * Multifd support */ diff --git a/migration/migration.c b/migration/migration.c index 929a6c432c..d6383b23b9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2196,6 +2196,21 @@ void migration_rp_kick(MigrationState *s) qemu_sem_post(&s->rp_state.rp_sem); } =20 +/* This is called only on destination side */ +void migration_request_switchover_ack(const char *requester) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + if (!migrate_switchover_ack()) { + return; + } + + mis->switchover_ack_pending_num++; + + trace_migration_request_switchover_ack(requester, + mis->switchover_ack_pending_num= ); +} + static struct rp_cmd_args { ssize_t len; /* -1 =3D variable */ const char *name; diff --git a/migration/savevm.c b/migration/savevm.c index 8ff6c0b17e..25afcfd71e 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2800,23 +2800,6 @@ static int qemu_loadvm_state_header(QEMUFile *f, Err= or **errp) return 0; } =20 -static void qemu_loadvm_state_switchover_ack_needed(MigrationIncomingState= *mis) -{ - SaveStateEntry *se; - - QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->switchover_ack_needed) { - continue; - } - - if (se->ops->switchover_ack_needed(se->opaque)) { - mis->switchover_ack_pending_num++; - } - } - - trace_loadvm_state_switchover_ack_needed(mis->switchover_ack_pending_n= um); -} - static int qemu_loadvm_state_setup(QEMUFile *f, Error **errp) { ERRP_GUARD(); @@ -3078,10 +3061,6 @@ int qemu_loadvm_state(QEMUFile *f, Error **errp) return -EINVAL; } =20 - if (migrate_switchover_ack()) { - qemu_loadvm_state_switchover_ack_needed(mis); - } - cpu_synchronize_all_pre_loadvm(); =20 ret =3D qemu_loadvm_state_main(f, mis, errp); diff --git a/migration/trace-events b/migration/trace-events index c0c433744c..5955befcc6 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -8,7 +8,6 @@ qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" qemu_savevm_query_pending(bool exact, bool final, uint64_t precopy, uint64= _t stopcopy, uint64_t postcopy, uint64_t total) "exact=3D%d, final=3D%d, pr= ecopy=3D%"PRIu64", stopcopy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PR= Iu64 -loadvm_state_switchover_ack_needed(unsigned int switchover_ack_pending_num= ) "Switchover ack pending num=3D%u" loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" loadvm_handle_cmd_packaged(unsigned int length) "%u" @@ -199,6 +198,7 @@ process_incoming_migration_co_postcopy_end_main(void) "" postcopy_preempt_enabled(bool value) "%d" migration_precopy_complete(void) "" migration_call_notifiers(int type) "type=3D%d" +migration_request_switchover_ack(const char *requester, unsigned int switc= hover_ack_pending_num) "Requester %s, switchover_ack_pending_num %u" =20 # migration-stats migration_transferred_bytes(uint64_t qemu_file, uint64_t multifd, uint64_t= rdma) "qemu_file %" PRIu64 " multifd %" PRIu64 " RDMA %" PRIu64 --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992044; cv=pass; d=zohomail.com; s=zohoarc; b=JIG5Rf+3P6eL35WVZbw3iZ4CpB0xUqdRiITa/gP2mjhbwN9ZSDUk9FpAunq/2zlWVsh6/3oL4QXhBtZFJtB8R6Z7Ziq6WNuuX550xLCPqG7SsvljRVcFujC8hCRNU55pHRL4yJvh/eSoxmcx4eDI7265yLDjMAtIt7wxT3xp32k= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992044; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=HuAjeF9UOGAH55S8g7CXsr44T4w00et7YZl8H6wcfR8=; b=HJRC69+tuSRnlCt6F2SNF2um0CX6ZYIKeAdJJ4Mpa7dYF+foOY18uQ9zA9wFvHS0u5N2b2CyxmZN/LU83x6M2x7wp41U7kVB63rXsfY3O0ePQqRhXHFauDGtPdtKu5kL7tnKS7QjepQ1hSRcw2WG42sNikn54MBXf6aoscucFLY= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992044780858.5761884999906; Tue, 9 Jun 2026 01:00:44 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrNu-0002pF-56; Tue, 09 Jun 2026 04:00:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNr-0002aW-RM for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:28 -0400 Received: from mail-southcentralusazlp170110003.outbound.protection.outlook.com ([2a01:111:f403:c10d::3] helo=SN4PR0501CU005.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrNp-0004JY-7b for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:27 -0400 Received: from MN2PR14CA0017.namprd14.prod.outlook.com (2603:10b6:208:23e::22) by SA5PPF7F0CA3746.namprd12.prod.outlook.com (2603:10b6:80f:fc04::8d1) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.12; Tue, 9 Jun 2026 08:00:16 +0000 Received: from BN1PEPF00005FFD.namprd05.prod.outlook.com (2603:10b6:208:23e:cafe::42) by MN2PR14CA0017.outlook.office365.com (2603:10b6:208:23e::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 08:00:16 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00005FFD.mail.protection.outlook.com (10.167.243.229) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:00:16 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:02 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:02 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 00:59:56 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uS8rNuP/8TSlzlGao2w3EfGdSegw+Dg4TxMhwKRjfGtotGOnc/AJxRJOgS49twk2maesEhW7NxmZE+Gy7zHofO8F+66e1IdYfwJJqHSUWf6Cn8OZJhdDp7N1PXdkSzlgziiSvIdh9fU1zBLXUTbia1dYozCj9+UjvQZn22wZBxY6b6cTviucffLgWXcwNylGnFMADx1p6bRaswSoQzl+EZq+8qCNen6kfw5SQCclf9e5/3Q0hY8Mn9DTqmGxa5ABSJEZCWewTeVCWw1zvv7PyNZNDdus7p5JLrfxxbIRZaCb5POOtiSdkmJ+8RRYigsy+JzucozBhaM27eDRggP+VA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HuAjeF9UOGAH55S8g7CXsr44T4w00et7YZl8H6wcfR8=; b=Mrfs+r57NA0xXENUPWL/KOH0WVMgoT0ley1AaPZOFxv5E7tXogwolNSOq2RIYubooT5I4L2f2JGdP12CwTDPPNf9iVFGVKlGzy4FU/RQ09ZcJm8bIjAeHthgHcCGioFe7hKHKOfLT2DsCEqxLL++kBFcSHqRLOevziWdm5vWuBb4KfCssrAH2lZy9sGTDjpJXC8tmtZ/2i+bnMCqlgIGAk63uaWBrLqsLX5toSvdtas3SFiWoqHccvvfmbZbQNKNJHflSTCWx239N4PwM6c+ZPWj/UA7yyhLc0zBqzU4IfoPtGOM/J6tQbrWkjAZfjTJJMxVZOtp4jOFe65Fy2KAFw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HuAjeF9UOGAH55S8g7CXsr44T4w00et7YZl8H6wcfR8=; b=FYSWcRbl5Rc1ksfja4PGM3VAzVEophUv9YySHD/uwlm1WNRzo+nmmseokxNuijI6zwN3ViW5DJT6kJ0EyJtrVSIeadVmzyJQIVWt3ErHjG8F5chO4lCr7xd1TjfQW66O1jogB9KmoAJ7JIvg+66FKKzVlA2C5uJnuTUQFt+rLRD4Veg30gLGCdfMH3oUVd4BeCsKGYxa9LT0lWE/gNhd+VCi7jhvsuBYsQNSH/XARKA9E6Tw8jn6R5RjPJnGUl9E0/XEhmbZrJLYGPuxc4Emoh2cZXgYPh+8+OAtBXxjqJjmUV8HRp6fYBvc+WWkybSxk0zTT77F2zLyyMeSTnfk0g== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 06/14] migration: Rename switchover-ack code to legacy Date: Tue, 9 Jun 2026 10:58:04 +0300 Message-ID: <20260609075812.32067-7-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00005FFD:EE_|SA5PPF7F0CA3746:EE_ X-MS-Office365-Filtering-Correlation-Id: 3babc839-4083-465b-20e0-08dec5fd24d3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|1800799024|7416014|376014|36860700016|22082099003|18002099003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: hGYd9uERNDdQ4GCPR878GIMOZn+dfqvsAN8nEzX0fA2hyxN+70VCz9ItB0ScV5eCMNGGHzsOS6cP1gSKaZK+PjwQYdaUeb1PEk6sjYCbOT2e67rM+Sf54SFG+u7W2xXKxFxBJvPbaC8Fwt0ml9MDBN5I46XHF0//YAHhO80JsxuBjLkbqVFjO+EVODVir2n3O1oOhzIs30jtogRgI33FZ85q2GnNu774epYdCQ8Ke9BSM5HNTvBmmsPMnN3m4PTADnTRmayvGXHdAqQGEY666ZWixShzbCOqmIiw3TBI4Pv9ZlTr80hlRrSELcFpyAd1TqnlwkHTOUjvRChm4WC1CU4CnU2qmaeRZkGH2CNeBeX8EzDOFYmof8rubqLxoHiYHDaN5q/PJgzJuYpvIihkjIAAcoQ+fgWpfBo41vFNHabcTYS46818wvXs+CEmHoxTCagVrBL+TZMs06f1id830Rrzy1/kctt6bX2GdZi1mTGa5dd+RhYvjPfmZQlwNPiZBA8uPJhupgRVMFgN9c1zoWjLS4p0k1XyV95vx+6e3DD9KTe58Gd1KN88ojHjpcXUTEXW3T3sUnTUukc8SCoGZnEZ8uUv/GsC2+xyYdSEmApNmIBAd4YDHFxRLhtbRTttL0hqxto/JrI5IilUiTqvgfKge1JwGhX2OcSTnRNFPM3qUGIsmFHiLqJb0Kusnt7EVObe3CZF+LlgCeNvL7H/g9aTw/RI1EPKqI2whInOBuk= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(82310400026)(1800799024)(7416014)(376014)(36860700016)(22082099003)(18002099003)(11063799006)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mPmcGeMLcjotLv9zm03wTjnYRvgJCnzmPVh5xmhri+ezE7JXrIXslNwqTVjCkJYcsjWS6ikTP8ew6W8L2o+w9i/yWa1b98R45Y2LhjD75V6gd7tFF1n6lBGVLNwV9eYsAV0uZ43UGkx3yN+c3aOTO+U992KFxxOmLzBXca8r9HCX5+XYxOjHmq14pITkOJf1R1DAomfE9KxVYBSktwb4zd8V3D9L0+/cAPuY1/X7KR4/KkV+tKQwMwDFLHLymw7VrpwmDzh4FxMOS2C+u1+mBXU24B9VWhdaaKfd+pgeut/E2Zus003Jh8QTXtM26r4XD3Tg0UWNhQEXYkmly464X+k/q6hcz1MqHbZPsLM1mwTV+QhEVPwDokCTy+hd18WbL9gATKyeFSY6Cs7xtMAUZoryEoxaoAWkh5FNAKVLY5qvYV+HM79/FtTdfkUjy5XI X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:00:16.6589 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3babc839-4083-465b-20e0-08dec5fd24d3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00005FFD.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA5PPF7F0CA3746 Received-SPF: permerror client-ip=2a01:111:f403:c10d::3; envelope-from=avihaih@nvidia.com; helo=SN4PR0501CU005.outbound.protection.outlook.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992047699154100 Content-Type: text/plain; charset="utf-8" A new switchover-ack mechanism will be added in the following patches. However, the old mechanism must still be kept for backward compatibility. Rename existing code that will be used only for old switchover-ack mechanism as legacy. This will help to distinguish legacy code from new code and make it more readable and easier for removal later when no longer needed. No functional change intended. Reviewed-by: Peter Xu Signed-off-by: Avihai Horon --- include/migration/misc.h | 2 +- migration/migration.h | 2 +- hw/vfio/migration.c | 6 ++--- migration/migration.c | 8 +++---- migration/savevm.c | 49 +++++++++++++++++++++++++++------------- migration/trace-events | 4 ++-- 6 files changed, 44 insertions(+), 27 deletions(-) diff --git a/include/migration/misc.h b/include/migration/misc.h index a2219c981b..4b43413aee 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -156,6 +156,6 @@ bool multifd_device_state_save_thread_should_exit(void); void multifd_abort_device_state_save_threads(void); bool multifd_join_device_state_save_threads(void); =20 -void migration_request_switchover_ack(const char *requester); +void migration_request_switchover_ack_legacy(const char *requester); =20 #endif diff --git a/migration/migration.h b/migration/migration.h index 841f49b215..da45444f7b 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -246,7 +246,7 @@ struct MigrationIncomingState { * zero an ACK that it's OK to do switchover is sent to the source. No= lock * is needed as this field is updated serially. */ - unsigned int switchover_ack_pending_num; + unsigned int switchover_ack_pending_num_legacy; =20 /* Do exit on incoming migration failure */ bool exit_on_error; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 7055cfbd3e..6b7acb2fa1 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -487,11 +487,11 @@ static bool vfio_precopy_supported(VFIODevice *vbased= ev) return migration->mig_flags & VFIO_MIGRATION_PRE_COPY; } =20 -static void vfio_request_switchover_ack(VFIODevice *vbasedev) +static void vfio_request_switchover_ack_legacy(VFIODevice *vbasedev) { if (vfio_precopy_supported(vbasedev)) { /* Precopy support implies switchover-ack is needed */ - migration_request_switchover_ack(vbasedev->name); + migration_request_switchover_ack_legacy(vbasedev->name); } } =20 @@ -784,7 +784,7 @@ static int vfio_load_setup(QEMUFile *f, void *opaque, E= rror **errp) return ret; } =20 - vfio_request_switchover_ack(vbasedev); + vfio_request_switchover_ack_legacy(vbasedev); =20 return 0; } diff --git a/migration/migration.c b/migration/migration.c index d6383b23b9..8d189fec80 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2197,7 +2197,7 @@ void migration_rp_kick(MigrationState *s) } =20 /* This is called only on destination side */ -void migration_request_switchover_ack(const char *requester) +void migration_request_switchover_ack_legacy(const char *requester) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 @@ -2205,10 +2205,10 @@ void migration_request_switchover_ack(const char *r= equester) return; } =20 - mis->switchover_ack_pending_num++; + mis->switchover_ack_pending_num_legacy++; =20 - trace_migration_request_switchover_ack(requester, - mis->switchover_ack_pending_num= ); + trace_migration_request_switchover_ack_legacy( + requester, mis->switchover_ack_pending_num_legacy); } =20 static struct rp_cmd_args { diff --git a/migration/savevm.c b/migration/savevm.c index 25afcfd71e..8e6c5b7c87 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2478,6 +2478,31 @@ static int loadvm_postcopy_handle_switchover_start(E= rror **errp) return 0; } =20 +/* + * If legacy switchover-ack is enabled but no device uses it, need to send= an + * ACK to source that it's OK to switchover. + */ +static int loadvm_switchover_ack_no_users_legacy(MigrationIncomingState *m= is, + Error **errp) +{ + int ret; + + if (!migrate_switchover_ack()) { + return 0; + } + + if (!mis->switchover_ack_pending_num_legacy) { + ret =3D migrate_send_rp_switchover_ack(mis); + if (ret) { + error_setg_errno(errp, -ret, + "Could not send switchover ack RP MSG"); + return ret; + } + } + + return 0; +} + /* * Process an incoming 'QEMU_VM_COMMAND' * 0 just a normal return @@ -2527,18 +2552,9 @@ static int loadvm_process_command(QEMUFile *f, Error= **errp) } mis->to_src_file =3D qemu_file_get_return_path(f); =20 - /* - * Switchover ack is enabled but no device uses it, so send an ACK= to - * source that it's OK to switchover. Do it here, after return pat= h has - * been created. - */ - if (migrate_switchover_ack() && !mis->switchover_ack_pending_num) { - ret =3D migrate_send_rp_switchover_ack(mis); - if (ret) { - error_setg_errno(errp, -ret, - "Could not send switchover ack RP MSG"); - return ret; - } + ret =3D loadvm_switchover_ack_no_users_legacy(mis, errp); + if (ret) { + return ret; } return 0; =20 @@ -3157,14 +3173,15 @@ int qemu_loadvm_approve_switchover(const char *appr= over) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 - if (!mis->switchover_ack_pending_num) { + if (!mis->switchover_ack_pending_num_legacy) { return -EINVAL; } =20 - mis->switchover_ack_pending_num--; - trace_loadvm_approve_switchover(approver, mis->switchover_ack_pending_= num); + mis->switchover_ack_pending_num_legacy--; + trace_loadvm_approve_switchover_legacy( + approver, mis->switchover_ack_pending_num_legacy); =20 - if (mis->switchover_ack_pending_num) { + if (mis->switchover_ack_pending_num_legacy) { return 0; } =20 diff --git a/migration/trace-events b/migration/trace-events index 5955befcc6..a6b8c31ee1 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -23,7 +23,7 @@ loadvm_postcopy_ram_handle_discard_end(void) "" loadvm_postcopy_ram_handle_discard_header(const char *ramid, uint16_t len)= "%s: %ud" loadvm_process_command(const char *s, uint16_t len) "com=3D%s len=3D%d" loadvm_process_command_ping(uint32_t val) "0x%x" -loadvm_approve_switchover(const char *approver, unsigned int switchover_ac= k_pending_num) "Approver %s, switchover_ack_pending_num %u" +loadvm_approve_switchover_legacy(const char *approver, unsigned int switch= over_ack_pending_num_legacy) "Approver %s, switchover_ack_pending_num_legac= y %u" postcopy_ram_listen_thread_exit(void) "" postcopy_ram_listen_thread_start(void) "" qemu_savevm_send_postcopy_advise(void) "" @@ -198,7 +198,7 @@ process_incoming_migration_co_postcopy_end_main(void) "" postcopy_preempt_enabled(bool value) "%d" migration_precopy_complete(void) "" migration_call_notifiers(int type) "type=3D%d" -migration_request_switchover_ack(const char *requester, unsigned int switc= hover_ack_pending_num) "Requester %s, switchover_ack_pending_num %u" +migration_request_switchover_ack_legacy(const char *requester, unsigned in= t switchover_ack_pending_num_legacy) "Requester %s, switchover_ack_pending_= num_legacy %u" =20 # migration-stats migration_transferred_bytes(uint64_t qemu_file, uint64_t multifd, uint64_t= rdma) "qemu_file %" PRIu64 " multifd %" PRIu64 " RDMA %" PRIu64 --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992054; cv=pass; d=zohomail.com; s=zohoarc; b=T0A9dZIZpu5fgZAMxZOrsItLj9EluU3TxvC3AGI8gLNA64/vtFyTc3mrLokuePCXMlJMdfTDeqA8Z8CSUpm0a008kHKHRs85Vnt3swNdgFgXbIM8IS55eLo0tama6/wEjtswvJ4nAwVP4E2EQDdMS5kl5EUa9ptcxjm8cUzeeus= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992054; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=vOYp16sTFIRxMYxajSp/ozF7C401a1DIjgfcPCtCBD0=; b=KIYd6VLPSMMrCkd+XfNOHd2FilADUgDQqSvkkKtmRoz8LGWeCkxmmrvPNV4VRMhhBSxH7LzR/WEYAcfqinDSJ0uGiWuAFDk4iVf5LhSKHopSjFzkFeNt3k5u99AOPqcXrrsC2j2BJZqSyzW+vHGQf8ziZXeLJ7cdRVlCplvARuE= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 178099205481533.24949596620854; Tue, 9 Jun 2026 01:00:54 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrOE-0004UR-6W; Tue, 09 Jun 2026 04:00:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrOB-0004RX-GF for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:47 -0400 Received: from mail-southcentralusazlp170130001.outbound.protection.outlook.com ([2a01:111:f403:c10c::1] helo=SA9PR02CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrO8-0004dk-Mv for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:47 -0400 Received: from BY1P220CA0023.NAMP220.PROD.OUTLOOK.COM (2603:10b6:a03:5c3::11) by MW4PR12MB7286.namprd12.prod.outlook.com (2603:10b6:303:22f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.13; Tue, 9 Jun 2026 08:00:37 +0000 Received: from CO1PEPF00012E62.namprd05.prod.outlook.com (2603:10b6:a03:5c3:cafe::aa) by BY1P220CA0023.outlook.office365.com (2603:10b6:a03:5c3::11) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 08:00:36 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E62.mail.protection.outlook.com (10.167.249.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:00:36 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:13 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:13 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 01:00:07 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yUUDG8XDb7wsIZWyA31t8QQusd8SeQ0au5Q+Pd46CtAd1zhR/HOPULxk6Pn/SKXjYi176+RVFyK/LHGQLeO+aTHtJGa19vONZrNqnkk6Ha6K8PLxnXrTLf4O139tR7TnEloocohJJP7W+TaDZjzo7OkzHhavvubbBLOnCyW8jdPvuUiFcod9NXnJqJjnc/uuwRJGr0//U1JdX0LH3AGqolG36WuEwZTj39cgJVHVIMjsGazV/2x/fbTHsiCfhPvZZz9XGUzJwgBTzEID4D9yQ1tP0Vm9XKNhqnxgNwpVegsqefgR3U9ACkCAqhRqDHV0EIQ7i9Bsj5YbhQeRvbgVZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vOYp16sTFIRxMYxajSp/ozF7C401a1DIjgfcPCtCBD0=; b=DIz9RY6DWDmlFl+XXu/oUeK46Dv1Q1uxGW2f/e8qUvc3ekyiDvwI1XP80qPoIb3KIKJsUoj5t38l6d50rMHZM6UrrISe+F07VyxV6BzJsMxBogIgejoTyMErQnvl5blR0AFQa6XW+a1AgCsW8dluO0YTGClSCtCPGHEyfIP5b4TjPeXec8CyWQQ4fUjcqR0BEzbZrjAvuXglWCjsihzwVMCd015B30cejtXb/i1v1CNtz711Ub0QYgU8wg8QCv1a6wWce+o9u1QCYI5Bw7J8R/1zrc/VYL96j8kwUfgNoTUM0RFnvP2akofVHBn13zx2F3owMLbUR2mznm3Z0+rF6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vOYp16sTFIRxMYxajSp/ozF7C401a1DIjgfcPCtCBD0=; b=A9YzerFp4lPxaVkFcHNmav12lJslJMzCyNFFvleZWmd9KWUj7flvH5VtmSsVH3hvb3gnqO2fvmNgk+9oNTkoiPVgTlDdA4R8KTVHj5EPnbtjQg/+JRxkty9PEFtoSkMIfCOeZCZ5yhAIuNm2MkHhAWBIzBOfkxlEuNGHvqdFxbK0y0LOE5N0/c6LrsewwslttvQQytGAh+B4n4eAZXKrqWH7J9FQuQFv5UjiOOAVX50pfMOSN66ovcEtx/bzLoJosTO31WPkr/RLzkVoSdh872iaKIAWghWpp1D3ADPEf5zjpGbt29lIQM/Hk1t5rEU21YFaQULCE3GOoOoJ0P6aDA== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 07/14] migration: Make switchover-ack re-usable Date: Tue, 9 Jun 2026 10:58:05 +0300 Message-ID: <20260609075812.32067-8-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E62:EE_|MW4PR12MB7286:EE_ X-MS-Office365-Filtering-Correlation-Id: b27be5ae-4ca3-45b4-4950-08dec5fd3071 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700016|7416014|376014|82310400026|22082099003|18002099003|6133799003|11063799006|5023799004|56012099006|3023799007; X-Microsoft-Antispam-Message-Info: 9HdWB/vKx1/7f5F1TcGYvuQMS2pr50C8HGoMdEFhsHHB9lef12kNCYUZS2HjDFfUItYztkMny9lCPnYCnts4zj9Dg+6kFZZnTk/BcJEaAi7dNlKd0/g5DiTRoZOmsFZqKgkwiBj3X+urMUTvRtFM7QWf0hiGLpYHa7l118Zdz6PpTxxRiMOWEiAUUkTP5SZeFY3jrV0XsL4wSRmP8tieZFPpraDZwDIXiayw4ZdYYK9ebGmTNz+6Wucn4Aq+UX4AWQCLcBslCFPlNUKuKKmimSI0EG5cubYR9XaGudjnoFn7JetB+Gtm2M55r8BToL4gYZA0ba2f+djy5zPAB5EXFNoEijQOKA2QtA5TsJUd7j/lArNBeArfgQvpMa/BlPZJFImV5uhRuUOULD8eTZgoO5etS0FeHWa7pHjZbbeqow+FBdf8Cll93SvoCQRquuiGxGxu/QqlU9mzOEBN9kQR8XW6ViXGSi+9GfPj2QXHOmUhqGaaGBtaGxwQkpb2LX3QNn5geApJB5WARdF8llrCPIsVoqDrIMFJ+/IOW22ciUsVVDty7i3guwP9T4hMgqZrcMMEsLZZxLt1kw53K/FI15r3bUuZZTt2lmJgEbFZct5VRsUZztx0qgnlAUrZ/6cU54qYi1qJ3JKO7tpsexIgu7fmgryCcrZrdEHfJlgAMVuvqcDTLJbEVTn9/n7YHYqwGzxN9COd2upIhejskC8Na7GQzVm43ZEl3MtFb4elPHk= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(36860700016)(7416014)(376014)(82310400026)(22082099003)(18002099003)(6133799003)(11063799006)(5023799004)(56012099006)(3023799007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 5VFwRVAfuO0YzUxsS1XgD+CrPJ1Uwgk18MwLeB3IBYkBoQSzNkP9JBQYCiVS5JC/+oOcyHGY/TBRXmf59VuACxH5vAztk6htZ4WdJG+Nn1VUPOSVHdrDqM/UB7cKB2FQPRC+d+Blw0XuUq488sUlmyNitPCJe8kR7++O9C81CCQylq3Cxdrz7NSp9fc4Kjlxh30tcK84+YKDUQdefykm5eKzekx2fuNOSBDSIaUZNYHb8gjDfIGJG4l2/60aHE/FtT77bbXtlTcfa0aPeWK6qTTKP/F4ttTMfM4PMz2vosl2ILMCDmg6q/wH6+roRI4uSh+PXh8VIA9Rg9BAqqBHexAB8Dud7supiRbtq8hnfyZFfh6XO1vJSlL7/VBuggB+FVbhdvGQkOddC4MQLOhO6I9mjmQOtP3G0ZJhwbPiOCuD/1Vf3mS1oux2XKpw+OLJ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:00:36.2286 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b27be5ae-4ca3-45b4-4950-08dec5fd3071 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E62.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7286 Received-SPF: permerror client-ip=2a01:111:f403:c10c::1; envelope-from=avihaih@nvidia.com; helo=SA9PR02CU001.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992056157158500 Content-Type: text/plain; charset="utf-8" Switchover-ack is a mechanism to synchronize between source and destination QEMU during migration to prevent the source from switching over prematurely. VFIO uses switchover-ack to ensure switchover happens only after destination side has loaded the precopy initial bytes. This is important for VFIO, as otherwise downtime could be impacted and be higher. In its current state, switchover-ack is a one-time mechanism, meaning that switchover is acked only once and past that another ACK cannot be requested again. This was sufficient until now, as VFIO precopy initial bytes was defined to be monotonically decreasing. Thus, when precopy initial bytes reached zero for all VFIO devices, a single ACK would be sent and its validity would hold. However, now the new VFIO_PRECOPY_INFO_REINIT feature allows precopy initial bytes to be re-initialized during precopy. Specifically, it means that initial bytes can grow after reaching zero, which would invalidate a previously sent switchover ACK. To solve this, make switchover-ack reusable and allow devices to request switchover ACKs when needed via the save_query_pending SaveVMHandler. Since now switchover ACK can be requested for a specific device and in different times, make switchover ACK per-device (instead of a single ACK for all devices) and let source side do the pending ACKs accounting. Keep the legacy switchover-ack mechanism for backward compatibility and turn it on by a compatibility property for older machines. Enable the property until VFIO implements the new switchover-ack. Acked-by: Markus Armbruster Signed-off-by: Avihai Horon --- qapi/migration.json | 14 ++++---- include/migration/client-options.h | 1 + include/migration/register.h | 2 ++ migration/migration.h | 32 ++++++++++++++++-- migration/savevm.h | 6 ++-- hw/core/machine.c | 1 + migration/migration.c | 37 ++++++++++++++------- migration/options.c | 10 ++++++ migration/savevm.c | 53 +++++++++++++++++++++++------- migration/trace-events | 5 +-- 10 files changed, 124 insertions(+), 37 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index 27a7970556..9b3070e494 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -508,14 +508,12 @@ # (since 7.1) # # @switchover-ack: If enabled, migration will not stop the source VM -# and complete the migration until an ACK is received from the -# destination that it's OK to do so. Exactly when this ACK is -# sent depends on the migrated devices that use this feature. For -# example, a device can use it to make sure some of its data is -# sent and loaded in the destination before doing switchover. -# This can reduce downtime if devices that support this capability -# are present. 'return-path' capability must be enabled to use -# it. (since 8.1) +# and complete the migration until the destination has +# acknowledged that it is OK to switchover. The acknowledgement +# may depend, for example, on some device's data being loaded in +# the destination before doing switchover. This can reduce +# downtime if devices that support this capability are present. +# Capability @return-path must be enabled to use it. (since 8.1) # # @dirty-limit: If enabled, migration will throttle vCPUs as needed to # keep their dirty page rate within @vcpu-dirty-limit. This can diff --git a/include/migration/client-options.h b/include/migration/client-= options.h index 289c9d7762..78b1daa1a6 100644 --- a/include/migration/client-options.h +++ b/include/migration/client-options.h @@ -13,6 +13,7 @@ =20 /* properties */ bool migrate_send_switchover_start(void); +bool migrate_switchover_ack_legacy(void); =20 /* capabilities */ =20 diff --git a/include/migration/register.h b/include/migration/register.h index a61c4236d2..5825eb30cb 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -23,6 +23,8 @@ typedef struct MigPendingData { uint64_t postcopy_bytes; /* Amount of pending bytes can be transferred only in stopcopy */ uint64_t stopcopy_bytes; + /* Number of new pending switchover ACKs */ + uint32_t switchover_ack_pending; /* * Total pending data, modules do not need to update this field, it * will be automatically calculated by migration core API. diff --git a/migration/migration.h b/migration/migration.h index da45444f7b..086eb9a15d 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -494,6 +494,29 @@ struct MigrationState { */ uint8_t clear_bitmap_shift; =20 + /* + * This decides whether to use legacy switchover-ack or new switchover= -ack. + * The main difference between them is that the former allows acknowle= dging + * switchover only once while the latter multiple times. + * + * In legacy, the destination keeps track of a pending ACKs counter. As + * migration progresses, the devices on the destination acknowledge + * switchover, decreasing the counter. When the counter reaches zero, a + * single ACK message is sent to the source via the return path, indic= ating + * that it's OK to switchover. + * + * In new switchover-ack, the source is the one that keeps track of a + * pending ACKs counter. As migration progresses, the destination send= s ACK + * message per-device via the return path, which decrements the source + * counter. When the counter reaches zero, it's OK to switchover. Duri= ng + * precopy, source-side devices may request additional ACKs, which inc= rement + * the counter again. + * + * In both legacy and new schemes, we rely on per-device protocol to r= equest + * switchover ACK from the destination-side counterpart. + */ + bool switchover_ack_legacy; + /* * This save hostname when out-going migration starts */ @@ -503,10 +526,13 @@ struct MigrationState { JSONWriter *vmdesc; =20 /* - * Indicates whether an ACK from the destination that it's OK to do - * switchover has been received. + * Indicates the number of pending ACKs from the destination. The valu= e may + * increase or decrease during precopy as new ACKs are requested or + * received. When zero is reached, it's OK to switchover. In legacy + * switchover-ack, it's initialized to 1 and decreased to zero upon AC= K. */ - bool switchover_acked; + uint32_t switchover_ack_pending_num; + /* Is this a rdma migration */ bool rdma_migration; =20 diff --git a/migration/savevm.h b/migration/savevm.h index 44424be347..fb92d3bc85 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -45,8 +45,10 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy= ); void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); -void qemu_savevm_query_pending_iter(MigPendingData *pending, bool exact); -void qemu_savevm_query_pending_final(MigPendingData *pending); +void qemu_savevm_query_pending_iter(MigrationState *s, MigPendingData *pen= ding, + bool exact); +void qemu_savevm_query_pending_final(MigrationState *s, + MigPendingData *pending); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); void qemu_savevm_state_end(QEMUFile *f); diff --git a/hw/core/machine.c b/hw/core/machine.c index 4d8b15d99e..8219f13779 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -43,6 +43,7 @@ GlobalProperty hw_compat_11_0[] =3D { { "chardev-vc", "encoding", "cp437" }, { "tpm-crb", "cap-chunk", "off" }, { "tpm-crb", "x-allow-chunk-migration", "off" }, + { "migration", "switchover-ack-legacy", "on" }, }; const size_t hw_compat_11_0_len =3D G_N_ELEMENTS(hw_compat_11_0); =20 diff --git a/migration/migration.c b/migration/migration.c index 8d189fec80..60493e2c10 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1707,7 +1707,9 @@ int migrate_init(MigrationState *s, Error **errp) s->vm_old_state =3D -1; s->iteration_initial_bytes =3D 0; s->threshold_size =3D 0; - s->switchover_acked =3D false; + /* Legacy switchover-ack sends a single ACK for all devices */ + qatomic_set(&s->switchover_ack_pending_num, + migrate_switchover_ack_legacy() ? 1 : 0); s->rdma_migration =3D false; =20 /* @@ -2201,7 +2203,7 @@ void migration_request_switchover_ack_legacy(const ch= ar *requester) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 - if (!migrate_switchover_ack()) { + if (!migrate_switchover_ack() || !migrate_switchover_ack_legacy()) { return; } =20 @@ -2457,9 +2459,18 @@ static void *source_return_path_thread(void *opaque) break; =20 case MIG_RP_MSG_SWITCHOVER_ACK: - ms->switchover_acked =3D true; - trace_source_return_path_thread_switchover_acked(); + { + uint32_t pending_num; + + pending_num =3D qatomic_dec_fetch(&ms->switchover_ack_pending_= num); + trace_source_return_path_thread_switchover_acked(pending_num); + if (pending_num =3D=3D UINT32_MAX) { + error_setg(&err, "Switchover ack pending num underflowed"); + goto out; + } + break; + } =20 default: break; @@ -2816,7 +2827,7 @@ static bool migration_switchover_start(MigrationState= *s, Error **errp) * properly update all the dirty bitmaps to finally generate the * correct discard bitmaps; see ram_postcopy_send_discard_bitmap(). */ - qemu_savevm_query_pending_final(&pending); + qemu_savevm_query_pending_final(s, &pending); =20 /* Inactivate disks except in COLO */ if (!migrate_colo()) { @@ -3266,7 +3277,7 @@ static bool migration_can_switchover(MigrationState *= s) return true; } =20 - return s->switchover_acked; + return qatomic_read(&s->switchover_ack_pending_num) =3D=3D 0; } =20 /* Migration thread iteration status */ @@ -3305,12 +3316,13 @@ static bool migration_iteration_next_ready(Migratio= nState *s, return false; } =20 -static void migration_iteration_go_next(MigPendingData *pending) +static void migration_iteration_go_next(MigrationState *s, + MigPendingData *pending) { /* * Do a slow sync first before boosting the iteration count. */ - qemu_savevm_query_pending_iter(pending, true); + qemu_savevm_query_pending_iter(s, pending, true); =20 /* * Update the dirty information for the whole system for this @@ -3356,12 +3368,12 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) Error *local_err =3D NULL; bool in_postcopy =3D (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE= || s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); - bool can_switchover =3D migration_can_switchover(s); + bool can_switchover; MigPendingData pending =3D { }; bool complete_ready; =20 /* Fast path - get the estimated amount of pending data */ - qemu_savevm_query_pending_iter(&pending, false); + qemu_savevm_query_pending_iter(s, &pending, false); =20 if (in_postcopy) { /* @@ -3402,9 +3414,12 @@ static MigIterateState migration_iteration_run(Migra= tionState *s) * during postcopy phase. */ if (migration_iteration_next_ready(s, &pending)) { - migration_iteration_go_next(&pending); + migration_iteration_go_next(s, &pending); } =20 + /* Check can switchover after qemu_savevm_query_pending() */ + can_switchover =3D migration_can_switchover(s); + /* Should we switch to postcopy now? */ if (can_switchover && postcopy_should_start(s, &pending)) { if (postcopy_start(s, &local_err)) { diff --git a/migration/options.c b/migration/options.c index 5cbfd29099..4c9b25372e 100644 --- a/migration/options.c +++ b/migration/options.c @@ -110,6 +110,9 @@ const Property migration_properties[] =3D { preempt_pre_7_2, false), DEFINE_PROP_BOOL("multifd-clean-tls-termination", MigrationState, multifd_clean_tls_termination, true), + /* Use legacy until VFIO implements new switchover-ack */ + DEFINE_PROP_BOOL("switchover-ack-legacy", MigrationState, + switchover_ack_legacy, true), =20 /* Migration parameters */ DEFINE_PROP_UINT8("x-throttle-trigger-threshold", MigrationState, @@ -467,6 +470,13 @@ bool migrate_rdma(void) return s->rdma_migration; } =20 +bool migrate_switchover_ack_legacy(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->switchover_ack_legacy; +} + typedef enum WriteTrackingSupport { WT_SUPPORT_UNKNOWN =3D 0, WT_SUPPORT_ABSENT, diff --git a/migration/savevm.c b/migration/savevm.c index 8e6c5b7c87..b8c93d86d7 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1801,7 +1801,8 @@ int qemu_savevm_state_complete_precopy(MigrationState= *s, Error **errp) return 0; } =20 -static void qemu_savevm_query_pending(MigPendingData *pending, bool exact, +static void qemu_savevm_query_pending(MigrationState *s, + MigPendingData *pending, bool exact, bool final) { SaveStateEntry *se; @@ -1827,22 +1828,35 @@ static void qemu_savevm_query_pending(MigPendingDat= a *pending, bool exact, * close to reality when this got invoked frequently while iterating. */ mig_stats.dirty_bytes_total =3D pending->total_bytes; - trace_qemu_savevm_query_pending(exact, final, pending->precopy_bytes, - pending->stopcopy_bytes, - pending->postcopy_bytes, - pending->total_bytes); + + if (migrate_switchover_ack() && !migrate_switchover_ack_legacy() && + pending->switchover_ack_pending) { + /* + * NOTE: Currently we rely on per-device protocol to request switc= hover + * ACK from the device on the destination side. + */ + qatomic_add(&s->switchover_ack_pending_num, + pending->switchover_ack_pending); + } + + trace_qemu_savevm_query_pending( + exact, final, pending->precopy_bytes, pending->stopcopy_bytes, + pending->postcopy_bytes, pending->total_bytes, + pending->switchover_ack_pending, + qatomic_read(&s->switchover_ack_pending_num)); } =20 -void qemu_savevm_query_pending_iter(MigPendingData *pending, bool exact) +void qemu_savevm_query_pending_iter(MigrationState *s, MigPendingData *pen= ding, + bool exact) { - qemu_savevm_query_pending(pending, exact, false); + qemu_savevm_query_pending(s, pending, exact, false); } =20 -void qemu_savevm_query_pending_final(MigPendingData *pending) +void qemu_savevm_query_pending_final(MigrationState *s, MigPendingData *pe= nding) { g_assert(bql_locked()); =20 - qemu_savevm_query_pending(pending, true, true); + qemu_savevm_query_pending(s, pending, true, true); } =20 void qemu_savevm_state_cleanup(void) @@ -2487,7 +2501,7 @@ static int loadvm_switchover_ack_no_users_legacy(Migr= ationIncomingState *mis, { int ret; =20 - if (!migrate_switchover_ack()) { + if (!migrate_switchover_ack() || !migrate_switchover_ack_legacy()) { return 0; } =20 @@ -3169,7 +3183,7 @@ int qemu_load_device_state(QEMUFile *f, Error **errp) return 0; } =20 -int qemu_loadvm_approve_switchover(const char *approver) +static int qemu_loadvm_approve_switchover_legacy(const char *approver) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 @@ -3188,6 +3202,23 @@ int qemu_loadvm_approve_switchover(const char *appro= ver) return migrate_send_rp_switchover_ack(mis); } =20 +int qemu_loadvm_approve_switchover(const char *approver) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + if (!migrate_switchover_ack()) { + return 0; + } + + if (migrate_switchover_ack_legacy()) { + return qemu_loadvm_approve_switchover_legacy(approver); + } + + trace_loadvm_approve_switchover(approver); + + return migrate_send_rp_switchover_ack(mis); +} + bool qemu_loadvm_load_state_buffer(const char *idstr, uint32_t instance_id, char *buf, size_t len, Error **errp) { diff --git a/migration/trace-events b/migration/trace-events index a6b8c31ee1..f5339f4193 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -7,7 +7,7 @@ qemu_loadvm_state_section_partend(uint32_t section_id) "%u" qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" -qemu_savevm_query_pending(bool exact, bool final, uint64_t precopy, uint64= _t stopcopy, uint64_t postcopy, uint64_t total) "exact=3D%d, final=3D%d, pr= ecopy=3D%"PRIu64", stopcopy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PR= Iu64 +qemu_savevm_query_pending(bool exact, bool final, uint64_t precopy, uint64= _t stopcopy, uint64_t postcopy, uint64_t total, uint32_t switchover_ack_pen= ding, uint32_t total_switchover_ack_pending) "exact=3D%d, final=3D%d, preco= py=3D%"PRIu64", stopcopy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PRIu6= 4", collected switchover ack pending=3D%"PRIu32", total switchover ack pend= ing=3D%"PRIu32 loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" loadvm_handle_cmd_packaged(unsigned int length) "%u" @@ -24,6 +24,7 @@ loadvm_postcopy_ram_handle_discard_header(const char *ram= id, uint16_t len) "%s: loadvm_process_command(const char *s, uint16_t len) "com=3D%s len=3D%d" loadvm_process_command_ping(uint32_t val) "0x%x" loadvm_approve_switchover_legacy(const char *approver, unsigned int switch= over_ack_pending_num_legacy) "Approver %s, switchover_ack_pending_num_legac= y %u" +loadvm_approve_switchover(const char *approver) "Approver %s" postcopy_ram_listen_thread_exit(void) "" postcopy_ram_listen_thread_start(void) "" qemu_savevm_send_postcopy_advise(void) "" @@ -189,7 +190,7 @@ source_return_path_thread_loop_top(void) "" source_return_path_thread_pong(uint32_t val) "0x%x" source_return_path_thread_shut(uint32_t val) "0x%x" source_return_path_thread_resume_ack(uint32_t v) "%"PRIu32 -source_return_path_thread_switchover_acked(void) "" +source_return_path_thread_switchover_acked(uint32_t pending_num) "switchov= er_ack_pending_num %" PRIu32 source_return_path_thread_postcopy_package_loaded(void) "" migration_thread_low_pending(uint64_t pending) "%" PRIu64 migrate_transferred(uint64_t transferred, uint64_t time_spent, uint64_t ba= ndwidth, uint64_t avail_bw, uint64_t size) "transferred %" PRIu64 " time_sp= ent %" PRIu64 " bandwidth %" PRIu64 " switchover_bw %" PRIu64 " max_size %"= PRId64 --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992064; cv=pass; d=zohomail.com; s=zohoarc; b=imBns5LSncLGPxerqDzZCWIOAE7NlgE3eq4bdQMRRgUajcpNEb79oebDa2OzqMuulYmyS1+dFirfqGsF9Z7OiYch3YGoIe5F6F0kQUxY4VvplNtEWRWMWp2CaMAi6Y4l8eqWzDYwrBF8zfxJ3Chl6qfXmQvvS66SDRvf/k1D+I4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992064; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=N6G36LfBqPWIVOHutzdV8Y5d6qlQPN9Su8FzrZVLpxM=; b=d5vHTHfzh1nO/8R1OQJK+bB04DhsIFwGqsmMTF2j/RChgFOlbVjxtV5O/jorUt8PTWpzXjb/ZqDWIg83s6uOHjNMIj+EZ4iBGs2CG8dvvuG1TrH5abQjGyvxRQACNEOSnBPo6xk5i2hOniMDbmfO5Z8Xskeuyw/NNl8BZuYxKjo= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992064091511.7383965908365; Tue, 9 Jun 2026 01:01:04 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrOM-0004qL-A1; Tue, 09 Jun 2026 04:00:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrOK-0004kN-Lu for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:56 -0400 Received: from mail-westusazlp170100005.outbound.protection.outlook.com ([2a01:111:f403:c001::5] helo=BYAPR05CU005.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrOJ-0004mt-0I for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:00:56 -0400 Received: from MN2PR18CA0012.namprd18.prod.outlook.com (2603:10b6:208:23c::17) by IA0PR12MB8350.namprd12.prod.outlook.com (2603:10b6:208:40d::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.13; Tue, 9 Jun 2026 08:00:48 +0000 Received: from BN1PEPF00006001.namprd05.prod.outlook.com (2603:10b6:208:23c:cafe::77) by MN2PR18CA0012.outlook.office365.com (2603:10b6:208:23c::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 08:00:46 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00006001.mail.protection.outlook.com (10.167.243.233) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:00:46 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:25 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:24 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 01:00:19 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Qsa9f5ySrXE5Jb3E8M2ISeKAKFU50FKHP/8ikhIvnLSQ9WJ0tRzV0zuCkNftArf0FteMKu+IQTvxk9ymU3caa1loWmU7fgnHzK8GtHBqM8Gl/8otLPj5ZfC9KQ442duNY4Qh+HBL4Vcoe9PEtWVC5e4qOc15aNTpoGyt/O8mUEHL4BIWenhg0lqkRxoTZrR3N+VZpeX6ZDS5SjIkQQX3IMxzfmERaDmmryjmmKkyAOdtImCIgB6epVvJlsPFd+AG/3/9N/b53Z31bqzs8iwmawfizzKALfKWSxH6Z4FEKZJpqhxsHozXc0uiisrB2wl1G05XYn1D/Kf6Tq+bpEApYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N6G36LfBqPWIVOHutzdV8Y5d6qlQPN9Su8FzrZVLpxM=; b=R68KW+w7wdejYQ3FFt7WliHiWIVnhpNOATiHwY025uQqsTRXYpYZdFKgepk4D/u47FRggJGQFj4G7ZHoPrRix+Wr12VkYh/mKYrZryENthYa++TyvpChhyxMvwa7KLh2e3aTQdbQbw90vZ5gWwyHLFj39q/KdeW0Z+ommWiEN264D4CzsdLHAaqWs0oQJJ9PzQwUT/HHeYrePmB+/7iUs+ltV9ycsAKcQXny09Dg7a5egnN6H7r0WCoyL2QRVxpi9uJi4xQEybDIZUYTRxYf6vSM6gBSFL+C9D4IoAULFselIqAevMUdUTQCzqQ1T+GTnL+4O52tCzYJIOFXTmOUvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N6G36LfBqPWIVOHutzdV8Y5d6qlQPN9Su8FzrZVLpxM=; b=h10lsapMzEjZ2/FQJrE4sQJirYVCp9abS91nC8ThXN3+MjnNgbs+HavZ1M7L1M6cx6oISdJ5I8ywn21VL5T1a4F5LsF73vO9w+DF7HEs1UDeKXogIBnldpewhZQGAU+41+iEzNZauo9fuFCmvWDsLk8Vabd6IcrzX+6iAot4UHGxLiNJN/Xa7DNn+RjQkOnjJPZwyDG2OYyFGXjLji4uGtMcF06lsMzAC2IgANA3qMDoZlFny8lBpABdWJuQdko/b9KBVVdIU2WnoP/YcENILsS1/5KphUVXGaysYKMjqGlUahNtgj5w8yCAz9BBsXMfuyaT12yH/GaIkKZCCjllVg== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 08/14] migration: Fail migration if switchover-ack is requested after switchover decision Date: Tue, 9 Jun 2026 10:58:06 +0300 Message-ID: <20260609075812.32067-9-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00006001:EE_|IA0PR12MB8350:EE_ X-MS-Office365-Filtering-Correlation-Id: aff56531-8289-47ce-4454-08dec5fd3695 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700016|82310400026|376014|7416014|1800799024|56012099006|11063799006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: keIw2kRL4H8xPgFdJimCUHIA4GRRtm5Pg9MCZyqUYLEQldV+dErPD5ZdH8YrTAHkKKE30Nu8k53GDKdyCCW3i7b+fKFuj5NWadCTBplVA9rChOx5ih6Y3ncTtXHGbeJnSEL/9BvCHJbGAXf/seaq99YSRoN5MjgVN+wb2bXcpA7z5as+EOB8Qzfe0qM06MTSki0192vzfCfHKxbZTWEzEnctZd/QkPw3LqSn2A8S0TViaPKBK5X8lFNRMmXOLP1WdwV/KZlM7MdfjR18FAF05Ks1kbV0RCK3KNzLjBi3Y9ns2+jTg/k9QTA0QZNXWvsr7/lhsWd/QSFk3x1ZgMGnXpbB4nAoKIldwh0LRpep0CqOGnZ6GOLPi762XIylhhGzeVMqolFaI58hI+T17hW1RbfoI61UC4ZOiTGF3uupXWWfruPGQUmlxKeaM75Vs9gajDdOr3DtVXN+0GdZarKf5SJEVxdII//8eqsOqHhglu+nzmYKnrwA8AwYiizS4R5sPiHkyb3zkAf0Jwx9cC55VryqNH5lg5sY0oqK9dCrvAJWhBeAYxAWmPlGYssiBgDg+Dv+aWr8eZvselPDE4sF/1zoB7Ngnmm/+x7vHFgQmEEI7YlZu3uRKStaRWt1GG+UG9h+zoeL/rDWdR3h6MQ/zLWppoWERb5CETOSOexkBh1qZqabTYoKbVc+llRkVvPH3/7/oz8DL/+hsxZykujmb51A0vJezVYXS0RKaRtyvBA= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(36860700016)(82310400026)(376014)(7416014)(1800799024)(56012099006)(11063799006)(22082099003)(18002099003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: neHshY+jkCx/EVYYKFta4gwMTHHfrgtnWh/5u2epIw7XDQuf/PFkRRWqfa9vPgxdNoM1L/HB7d7vv+At+dy3o4Gym0wOA6WyhVxZ7FWbaNqcUNFLIETxANwvqPdmFelmeAFp29Usr7V7Kxcawbau4L7AMMAv5mGiqBdYm4Pl8u4rUOB2ynfeBYNjZKIIsIaMboT3X5N7F7vW7oxpkGAPYMeMqfNOOzUviQh2lXAMneeGDJPDyuw+fk5ZdnnJq45/L497TxweyCjNAvtGXIbWyMUq1trks/RFNcVtld9hFnw5pWw8/qkmhhKhLxrdZVbpafQ7XJyOKTrTDSL7GGVzVr51MwlTewL42yrU2QHghvCW1G8b2iXTuos7QfwzTvhfVaLAHGSQX60HyOCLikUcjW2BEGju8ptL8aHqCxIwQ2oe0XTlQe6BgIDZUEk+CWwC X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:00:46.4282 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: aff56531-8289-47ce-4454-08dec5fd3695 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00006001.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8350 Received-SPF: permerror client-ip=2a01:111:f403:c001::5; envelope-from=avihaih@nvidia.com; helo=BYAPR05CU005.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992065925154100 Content-Type: text/plain; charset="utf-8" Switchover ACK is checked only during precopy while the guest is still running. The last migration_can_switchover() decision and guest stop are not atomic, so a device may want to request another switchover ACK in the gap after switchover decision has been made but before the guest is stopped. Migration would then miss that request, which can increase downtime. Cover this case by failing the migration if a switchover-ack was requested during that time. Ideally, precopy iterations should be resumed in this case, however, VFIO doesn't support going back to precopy after being stopped, so implementing such logic would require non-trivial changes to the guest start/stop flow. Given the above and that this case should be rare, failing the migration seems reasonable. Signed-off-by: Avihai Horon --- migration/savevm.h | 4 ++-- migration/migration.c | 4 +++- migration/savevm.c | 19 ++++++++++++++++++- 3 files changed, 23 insertions(+), 4 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index fb92d3bc85..415198423f 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -47,8 +47,8 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); void qemu_savevm_query_pending_iter(MigrationState *s, MigPendingData *pen= ding, bool exact); -void qemu_savevm_query_pending_final(MigrationState *s, - MigPendingData *pending); +bool qemu_savevm_query_pending_final(MigrationState *s, + MigPendingData *pending, Error **errp= ); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); void qemu_savevm_state_end(QEMUFile *f); diff --git a/migration/migration.c b/migration/migration.c index 60493e2c10..4c20378ed2 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2827,7 +2827,9 @@ static bool migration_switchover_start(MigrationState= *s, Error **errp) * properly update all the dirty bitmaps to finally generate the * correct discard bitmaps; see ram_postcopy_send_discard_bitmap(). */ - qemu_savevm_query_pending_final(s, &pending); + if (!qemu_savevm_query_pending_final(s, &pending, errp)) { + return false; + } =20 /* Inactivate disks except in COLO */ if (!migrate_colo()) { diff --git a/migration/savevm.c b/migration/savevm.c index b8c93d86d7..6737058021 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1852,11 +1852,28 @@ void qemu_savevm_query_pending_iter(MigrationState = *s, MigPendingData *pending, qemu_savevm_query_pending(s, pending, exact, false); } =20 -void qemu_savevm_query_pending_final(MigrationState *s, MigPendingData *pe= nding) +bool qemu_savevm_query_pending_final(MigrationState *s, MigPendingData *pe= nding, + Error **errp) { g_assert(bql_locked()); =20 qemu_savevm_query_pending(s, pending, true, true); + + /* + * Switchover-ack requests done after switchover decision are not allo= wed. + * Fail the migration in this case since we currently don't support go= ing + * back to precopy. + */ + if (migrate_switchover_ack() && !migrate_switchover_ack_legacy() && + pending->switchover_ack_pending > 0) { + error_setg(errp, + "Switchover ACK was requested by %" PRIu32 + " devices during switchover", + pending->switchover_ack_pending); + return false; + } + + return true; } =20 void qemu_savevm_state_cleanup(void) --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992175; cv=pass; d=zohomail.com; s=zohoarc; b=Rk1vodQFMh0d+8MHWcjqMLG3nfpHCJ+7ycL6pKW4yEfTUEVwBWFhrnDX0wCjyJiiG4eX0IC5Ib0l6XNYcHZLKM/3ieW1nRwKQtTU58/u+tEgSVqn9MjJ/AUuA+G1tLrQ7gwXb85+rLAYTGrrZEX5MIXv8kojjxjtyFIh30yM77I= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992175; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uwipct43yTA/LXG4qb+DS18tbfvDcpVvyVNySaLkcnc=; b=MEAGnRuxlanl2mMb7uWGabfVz9NSS5vuo9C/AaudK8E5JjlMAex10HhBhfOxrpJmr8RAN0n/yPemQnhGeUauWcLna2SsMsWS60IcxglHcpjO4ZochqpCztFvAX4z9L5pSQWGBJPvRJR7mjgppPSwYnrETqYxX/XtGRs33Hu/2PE= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992175225673.721949187449; Tue, 9 Jun 2026 01:02:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrQA-0007Lq-56; Tue, 09 Jun 2026 04:02:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrQ8-0007L5-80 for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:02:48 -0400 Received: from mail-westus3azlp170100009.outbound.protection.outlook.com ([2a01:111:f403:c107::9] helo=PH7PR06CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrQ6-000683-OH for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:02:48 -0400 Received: from MW4P221CA0025.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::30) by DM4PR12MB8571.namprd12.prod.outlook.com (2603:10b6:8:187::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.13; Tue, 9 Jun 2026 08:00:57 +0000 Received: from CO1PEPF00012E65.namprd05.prod.outlook.com (2603:10b6:303:8b:cafe::15) by MW4P221CA0025.outlook.office365.com (2603:10b6:303:8b::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 08:00:55 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E65.mail.protection.outlook.com (10.167.249.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:00:55 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:34 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:33 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 01:00:27 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MRBP2uI3G+esDbN3YDkuJMw+AaN8jGGmPZ8JWIXqlAp948XuXkHTiHI1oeAea8coSFLzjh/4DOh2SVloZT/HQxQ4hwptOn6FBRrUhSfFGfLCcOE6o7eqmJaYVN0gIqgA21EARgwoEDRgakjwrAMbauO5ZL6wdpBFTHmoz1rIYZsLt5g03wKQWwzZlilNr1AB13285nMMJ2BHtovjB370ONmRDu9xHpEanLLWTb3tZDRU3Tp+8/TCNbOvb5BoXcPIDB0eUbgUL96v3OkC/5WCZf1wHZZfnfROhp3mBIDLiNRMWRsLiXmRMVd2enmUkHuqWaoDtuwlB26WSbYjwOsEQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uwipct43yTA/LXG4qb+DS18tbfvDcpVvyVNySaLkcnc=; b=J4V2MetUkzIBYDSXHzI52aOUI9BMA5W+Kcj8r4+iN2EtU92cSU9caykorIX00kPuDsYFBXIhmiT+ul2xXM5rJmFRYdC7PJqSS8khWkopttgRb8u4c5Nt067YuH1DaalKQuh5m9AfBsjkw3WWUomuMTMrrTgtmx3bhdxZgfqLGufqgP68adDgHB9FOtupBxXA7fmYJOt2Z5kzwdxa4WoDuANW4rXIeltSuVEBG1KSv+ZCSaypR1SAO5NHnhPPSWRBPWIQU17xoiIjgU+F0siCslD5ZBNdHLcogefGGCQABBUNV6Ft0YMR7n5/jV4uFB5FWTdbOQ1/6a3PunAJfqOAXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uwipct43yTA/LXG4qb+DS18tbfvDcpVvyVNySaLkcnc=; b=STBvSIPqmrmVpy6la9O7nOqdHMnK1yK63/nfRgiZvskHG7R6xwkuM8BXiQkGsQKopxmFqF7HZdS9ExFKcleHFxKxiJIVIYAQHi6tc8/9msEeDVuenN81v29KXpukBXj4RPs35zyNSKf6MHwq2TpH/w5y5U5vwcclaJq/EkKGvLQWmi16gbrtqlX84cz+gKX0BTo0iogaEuxltA9/zYT+2PgSfQvA4lQTutJ96JmiJaAvuurHZArkh/WpKUECVx40vrOYsnioVPZnBMdBk9Rz3E3+jRCAV/tpcaW40Ndgeam9YyX6vZeLg0GYmzkcWmdvRYsdor/SX0nlXdkLDMa9HQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 09/14] vfio/migration: Extract VFIO_MIG_FLAG_DEV_INIT_DATA_SENT sending to helper Date: Tue, 9 Jun 2026 10:58:07 +0300 Message-ID: <20260609075812.32067-10-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E65:EE_|DM4PR12MB8571:EE_ X-MS-Office365-Filtering-Correlation-Id: 83c93739-45c2-4c29-8e9c-08dec5fd3c01 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|82310400026|7416014|376014|36860700016|18002099003|22082099003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: NSuTkZB1UW4ne0Lj8vZV4APUPhfRA8lN3JTU6grt0MZf2H8sfFB+kkQRWimA+zPXOZWduacsF6mzmVyolhXWzKmL6oLVLvKyJFZTFQcdQ4emXosZYqHhkLoSWOndnNMuQKCceDzAxwzAwISwICDnj47fUslqBmGQN5qmJApVaApMywEb5K/asjYblmZWnuCKgkqiDPHFk5wrMtpu1OYCkD7zxrwtwDQpZAwC0MHjYWU9kgFOFfr25DF47Z7/FDrNHfv1DEiJClKuYghBgo7L9IqoNhzL9T+frejfnHDXosbOdqNksujpyKFzx+zzrOnhQk36B2rRPU0JapdCqLnFHimPJhPjPjYAuM9zpE4+Sj37k22CHUGyos9+48Y0zSM4xuPzEcCHgD26YnYDLsobT41kNfHetBnhDgLYak1iqYdZKE4Wz5jRdYkwmwfjN2oM5wsN8mEiAFhevSmiC405uxE7qdIM1Mq6df5tsas7zAta2OM58AFZNYsTPsoTOJOUUgVVStrECS0UHVL2jYykI6woSKuiavEzU3bHz6g/MS771AAHGManAXLTZi3U2e0PqlFpiERzCtVhTIneH2i/qcxiMkMuXy6x/dCfzzbIfDPXd6H6np2L2rtl//4aFR3l4wlsFPBvtgAGoWP6npsrHtN7bGXaguTOAkY5ZK5/4ylP+7njyg6qrps7LCAVR5Fg7wg9BxzyFX2OBJtrXh8FRrnDu/6qJLWiS6YyW6L3QHk= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(82310400026)(7416014)(376014)(36860700016)(18002099003)(22082099003)(11063799006)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 4nCln7/KhJLCJ3KhKD8Apsy3SKrcm4Yp5HsALDMAj5HSifRNy492V2Iaf5LkYQ3lZuUkxVbVrFtugEJGrxdnSanDERhq5CKU4U5l9kzzstZ0s2KY3lNGLqVOpGZJnPdOmyZJtIEKsRur0CU5xMepzTlakuNK4EOwk+174HGUHArC7k1vs8yI5ZL6iE2K3MdkrzoK3e6ZHayY5ir76kOK5utfERcdWvLwbZ9IYOYkwoxo53qZRCu3ABYCYFmCV0sGlvLXrSkGV8UrprYk44lIurPpgORjhTAJWFgpktDpmyTWpi+VWFEMCjrBDNbe75HIXODyfsdlIa0WhK+O87gs4IVdyxXpNMBmFkXVNzl9Q9RqjQofo2kHBJ6ErgMcebYhbuHkPxm5gsRm6udu/KNhY0TbqalqfR6nOVDlJKlDLbR9CigUhkr/TwVq1/Pu1Bz/ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:00:55.6392 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 83c93739-45c2-4c29-8e9c-08dec5fd3c01 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E65.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB8571 Received-SPF: permerror client-ip=2a01:111:f403:c107::9; envelope-from=avihaih@nvidia.com; helo=PH7PR06CU001.outbound.protection.outlook.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992176853158500 Content-Type: text/plain; charset="utf-8" Extract the VFIO_MIG_FLAG_DEV_INIT_DATA_SENT flag sending logic from vfio_save_iterate() into vfio_send_init_data_flag() for clarity. Also add a trace while at it. Reviewed-by: Peter Xu Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 26 +++++++++++++++++++++----- hw/vfio/trace-events | 1 + 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 6b7acb2fa1..45f8e346b4 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -480,6 +480,26 @@ static void vfio_update_estimated_pending_data(VFIOMig= ration *migration, data_size); } =20 +/* Returns true if the init data flag was sent, false otherwise */ +static bool vfio_send_init_data_flag(QEMUFile *f, VFIOMigration *migration) +{ + VFIODevice *vbasedev =3D migration->vbasedev; + + if (!migrate_switchover_ack()) { + return false; + } + + if (migration->precopy_init_size || migration->initial_data_sent) { + return false; + } + + qemu_put_be64(f, VFIO_MIG_FLAG_DEV_INIT_DATA_SENT); + migration->initial_data_sent =3D true; + trace_vfio_send_init_data_flag(vbasedev->name); + + return true; +} + static bool vfio_precopy_supported(VFIODevice *vbasedev) { VFIOMigration *migration =3D vbasedev->migration; @@ -693,11 +713,7 @@ static int vfio_save_iterate(QEMUFile *f, void *opaque) =20 vfio_update_estimated_pending_data(migration, data_size); =20 - if (migrate_switchover_ack() && !migration->precopy_init_size && - !migration->initial_data_sent) { - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_INIT_DATA_SENT); - migration->initial_data_sent =3D true; - } else { + if (!vfio_send_init_data_flag(f, migration)) { qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); } =20 diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 8f57d0b7d8..411a306635 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -176,6 +176,7 @@ vfio_save_iterate(const char *name, uint64_t precopy_in= it_size, uint64_t precopy vfio_save_iterate_start(const char *name) " (%s)" vfio_save_setup(const char *name, uint64_t data_buffer_size) " (%s) data b= uffer size %"PRIu64 vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool exact, bool final) " (%s) = stopcopy size %"PRIu64", precopy initial size %"PRIu64", precopy dirty size= %"PRIu64", exact %d, final %d" +vfio_send_init_data_flag(const char *name) " (%s)" vfio_vmstate_change(const char *name, int running, const char *reason, con= st char *dev_state) " (%s) running %d reason %s device state %s" vfio_vmstate_change_prepare(const char *name, int running, const char *rea= son, const char *dev_state) " (%s) running %d reason %s device state %s" =20 --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992089; cv=pass; d=zohomail.com; s=zohoarc; b=NpBeFLu7y0mNuKyrg1sv3/pAIdecJfYN+udgXmi4fh+38EOyzbLqVFVKRVLwzImSlfpoytWg20c4gpRg78CnWw6x06hvisOMVzFXn5DgvtSzsipqlLJh0skkz897NJkFS6NAUWd/fcVgcMMRLDyuZ7r4ANXuxnoBOxrJ783QYCc= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992089; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Faq4RI6ryoWvOUzJOiuhZ+iNbWjhLG95XX4p9ymduyk=; b=I9C65uqTdOS62IRFtdlZf81AvsF7Z3a9usmBiakwWQ2C1FwM9W+pFLzevYCH6/EbbIevORJz09/1sriyA6IIKO1UXGMQP8ARyuEFOIXK1cfJELEAFh9J2LzfMMpFReCLKKdviYNpv42loFNnzj8rEMuNU2ajbFHq7BL+2WN3R/o= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992089986815.5142954763612; Tue, 9 Jun 2026 01:01:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrOq-0005X2-30; Tue, 09 Jun 2026 04:01:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrOh-0005Un-DQ for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:23 -0400 Received: from mail-westus2azlp170100005.outbound.protection.outlook.com ([2a01:111:f403:c005::5] helo=CO1PR03CU002.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrOf-0004oo-Pa for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:19 -0400 Received: from BY1P220CA0017.NAMP220.PROD.OUTLOOK.COM (2603:10b6:a03:5c3::6) by SA3PR12MB7997.namprd12.prod.outlook.com (2603:10b6:806:307::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.12; Tue, 9 Jun 2026 08:01:07 +0000 Received: from CO1PEPF00012E62.namprd05.prod.outlook.com (2603:10b6:a03:5c3:cafe::a5) by BY1P220CA0017.outlook.office365.com (2603:10b6:a03:5c3::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 08:01:07 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E62.mail.protection.outlook.com (10.167.249.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:01:07 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:46 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:46 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 01:00:40 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=X5l4J8HnLwbJJDHDojKujgqAntvFzoMRaoQJM1lsWuYHB2hMwRk7Fb8/qjUu0oWZEW4K8DN7zFPVoa68Y8Oj/sdMtcmbrh0J61OlYLXsiZ36/P6miPBlotj8cR+WkSY5e2NqeXWSxYlIevAUhI7Z6oXUwSxHWQ87u38akq435fQo1f2Lx2MUn1EkUcCTqXpiPWWHjrqlMDH7h9Rn7qNBFHvxUehom/HwV4gpURZTPh5WEe9xK64cDKQFOhd3MMiPtTPxi6hz74aCqijK+V06Nm+yzEIsKQL3BbK5TH/DQ75+7b20D4zsIn3BpCj+3WdcLtqgTJWLhGqPG8RAPnjAYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Faq4RI6ryoWvOUzJOiuhZ+iNbWjhLG95XX4p9ymduyk=; b=wbjbBOq7Q0DPDaqNvwtv9GCNTv0Gi+lQUoIyyz1JL74lrH3u0zwjN61yCzT8pwk5UkBXiQRDBBVJxJmuPJ5+HpJWbQ/2EnXOUdImi4zM7UMf0GFJLai2rYAlsh4+HmkI2Mk+NhHD+wfVi6r8k+2VtZV8rP+bw4yG6fF+CsFIUV0ySRaAk1t2YqWyG4Ws2buvxo7XFmGv9H12ttzOYs/hVskg4bFVX5h4sP/HebxItOzvP+zfHCdoGTlWForHya9YsusjQgwZM/Y5m57rtHzJn+z7KUHmKIcRxz6XFKZ5KL70VtDcMhUjoqQN+VaeuyiW9ZwtZG9IXr5RrSLLhgyebg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Faq4RI6ryoWvOUzJOiuhZ+iNbWjhLG95XX4p9ymduyk=; b=NjlR7UWzlDxEVnzFtlE6grDhUwimOwbyWPw1M6A8teFj+gSoRVGfIsJCHoLSbLOK5Toi1pGebpsMLk1H8C5gtK11AyqqeDKkBxbCEqOE3uDapIYw/EZqzAY5cazmzzkxgTs1Z/n1KBDZKSjdyAZ8Wui2yDJQcMyTvxO+FEMHzkqkrY+yIIStrfYGn2lF9mPk5uS7rjMziMDcqnqqE2NQEeGK8CGAoKck+Z6YaiDl5xQTga2wy5dcZU0e7PArPmPF2H0xJ/dwSroF4EpLPzIVM+Wfl791tQxzwKBi0IvGW5B2+sGQhuzvpw9mCA9a+sPEu1qPhG9PX/P4Fav18WxJLQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 10/14] vfio/migration: Add Error ** parameter to vfio_migration_init() Date: Tue, 9 Jun 2026 10:58:08 +0300 Message-ID: <20260609075812.32067-11-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E62:EE_|SA3PR12MB7997:EE_ X-MS-Office365-Filtering-Correlation-Id: 54230a35-cd5e-4172-1b2d-08dec5fd4306 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700016|82310400026|1800799024|7416014|376014|18002099003|22082099003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: YIvYFzkafLSGpEgRF2/3L3NJ43EidyxNWQljByncuz7IxmFx/AawTmllQY19UDmZO50AR99RGwGXHqwgcTGpAIMe6OW8Z45vmQvPoccfdPu1tQaaAlvE58e6zd2hu+hMGvKeJAfcH/u9MCHN9tE1RzFo1FE58gmx+iKMy9m5xKBpkDJ6AmxRhvWcS6erOzEuR0sgMW0WaCI12pWKcLjdTqcrwOyWrP9opGg3hLtup0mj9q6ZJfJ8pBQFSfylkRm9Yn/EWdE/Ys85mZdO67NS1y34R9Sqmr0atwwv9fE/c205gLH0j/DzNLc0JeUuHe2MH8g2309Oy/g4awfnGzz1b4siVuMNconiuGnefKcnm8AmwyY4gsI+kniDgUSnX8Vm+sCsxMF1+1CZ6lsz6pbcjPQSsuO/loa2Qjhl6nuILgAzvNy0XUzBK76z64xUhjsp4T2+yUwnIvzTnPaZV1BM6igQla5x8qWqxWdZaKWCNoIJLMN4boG86V4MYBrOZMkUOiZrNvH05yFMKDurjZz0Sz7Nty+sezy1BRrX+rC01Ep4Kx6GEIS6Y3S5vRxWzXw+TeNJ0zGaVvN2eGOoiU2q7XOPK1a71kpaejU3oYMP+GRhba8Mvo7gRDo4RYw+aj289BpMjL/+YNG1y0JvNRo7leE9CMhzX1bZokiBvEMG9DMSDqczSHIdCiD9Nth1rcOnszoXQKExTuGdDpaCEBAAUc1cV776PDfFe32Y3fXFfIk= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(36860700016)(82310400026)(1800799024)(7416014)(376014)(18002099003)(22082099003)(11063799006)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7UpXcOHSHLRrwJnX1EivM5xw75RVGr9p2Nmv7ojFG2zQ0225iB68Q6V0jF3xtB8DnFIjpSrhbxN++8Cd4Renz+ZjKf++Xlhy3JvpYVo58p+MzspLwi3y/H7qgFi6rLPobto4/a5BGlf+d87S7ho7F9UvzK9Osr47XHFvfhiUYbj/ZiGoeF6qXTkpEXL1vL7GN9W2OdODhJ3k6vqyyr0n7/teT8CIAxkJOyIdjetzUUSYtIONtld7n5MggY2Mh7n3qRMeuX7Lu6jtonX9xR1ifRfaYFGbpUCL5in1Nq3fvWjoCBTIY7tGJlp0hmNJSBRQ11oT+CDfTOL9zQ3yBC+KYh3ZdP0R1mrst9xU4xxeIJKil0emR4A50Pf7G7c3yJ+szxuSRHSqWEZ00jJh4l4tb9vslJ3FtstcdKoFzK2nOtKkrK9G6ClWmNEmXNa07Zl6 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:01:07.3964 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 54230a35-cd5e-4172-1b2d-08dec5fd4306 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E62.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB7997 Received-SPF: permerror client-ip=2a01:111:f403:c005::5; envelope-from=avihaih@nvidia.com; helo=CO1PR03CU002.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992090232158500 Content-Type: text/plain; charset="utf-8" vfio_migration_init() already has many failure points and a new one will be added in next patch. Add Error ** parameter to vfio_migration_init() to report a detailed error message through it. Refactor it to return bool as well. Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/migration.c | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 45f8e346b4..3ab6b7248f 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -1056,7 +1056,7 @@ static bool vfio_dma_logging_supported(VFIODevice *vb= asedev) return !ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); } =20 -static int vfio_migration_init(VFIODevice *vbasedev) +static bool vfio_migration_init(VFIODevice *vbasedev, Error **errp) { int ret; Object *obj; @@ -1067,22 +1067,32 @@ static int vfio_migration_init(VFIODevice *vbasedev) VMChangeStateHandler *prepare_cb; =20 if (!vbasedev->ops->vfio_get_object) { - return -EINVAL; + error_setg(errp, "no vfio_get_object handler"); + return false; } =20 obj =3D vbasedev->ops->vfio_get_object(vbasedev); if (!obj) { - return -EINVAL; + error_setg(errp, "failed to get object"); + return false; } =20 ret =3D vfio_migration_query_flags(vbasedev, &mig_flags); if (ret) { - return ret; + if (ret =3D=3D -ENOTTY) { + error_setg_errno(errp, -ret, + "migration is not supported in kernel"); + } else { + error_setg_errno(errp, -ret, "failed to query migration flags"= ); + } + + return false; } =20 /* Basic migration functionality must be supported */ if (!(mig_flags & VFIO_MIGRATION_STOP_COPY)) { - return -EOPNOTSUPP; + error_setg(errp, "VFIO_MIGRATION_STOP_COPY is not supported"); + return false; } =20 vbasedev->migration =3D g_new0(VFIOMigration, 1); @@ -1113,7 +1123,7 @@ static int vfio_migration_init(VFIODevice *vbasedev) migration_add_notifier(&migration->migration_state, vfio_migration_state_notifier); =20 - return 0; + return true; } =20 static Error *multiple_devices_migration_blocker; @@ -1279,18 +1289,8 @@ bool vfio_migration_realize(VFIODevice *vbasedev, Er= ror **errp) return !vfio_block_migration(vbasedev, err, errp); } =20 - ret =3D vfio_migration_init(vbasedev); - if (ret) { - if (ret =3D=3D -ENOTTY) { - error_setg(&err, "%s: VFIO migration is not supported in kerne= l", - vbasedev->name); - } else { - error_setg(&err, - "%s: Migration couldn't be initialized for VFIO dev= ice, " - "err: %d (%s)", - vbasedev->name, ret, strerror(-ret)); - } - + if (!vfio_migration_init(vbasedev, &err)) { + error_prepend(&err, "%s: VFIO migration init failed: ", vbasedev->= name); return !vfio_block_migration(vbasedev, err, errp); } =20 --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992096; cv=pass; d=zohomail.com; s=zohoarc; b=k/9DXDaSO9vuYEDKl68ktDAoer1g9VBky7cznTnWJijvkWYEVqPRpOgV9EUZrVffirb++uR1Ek9V9wLLK3JOqNBkBCHZPtmjagswCKec4vQ9DghcmU9VwZslBzsdr6YnqOEI+EzEFSiLjuaU902ZAGlVTH8H3IifnUGOn8+kvNE= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992096; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uq1hplQ5pK+cE64OCMWgLlH90LFNTj2fkuw/Bad+wOI=; b=bg/4ms/We1ZvnsdqGlbIpu8N1BXEm16SthdLl5mIPX3W3U5UzVrBGLROsLvIvsQ6qd14XGefSXdYE9g4wfN9JEuU3emngEpDHXFZs4rORRVWrFw7sVcQghm/C0FPcLPnM0R8VNxAndFSQLTI2bAMyIaLNc7nyLbUet8QGSN8uDE= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992096715617.9753909514229; Tue, 9 Jun 2026 01:01:36 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrOu-0005Zy-Oc; Tue, 09 Jun 2026 04:01:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrOo-0005X1-Ul for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:27 -0400 Received: from mail-eastusazlp170120007.outbound.protection.outlook.com ([2a01:111:f403:c101::7] helo=BL0PR03CU003.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrOn-0004pG-5E for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:26 -0400 Received: from BY3PR03CA0010.namprd03.prod.outlook.com (2603:10b6:a03:39a::15) by BL4PR12MB9508.namprd12.prod.outlook.com (2603:10b6:208:58e::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.12; Tue, 9 Jun 2026 08:01:19 +0000 Received: from CO1PEPF00012E66.namprd05.prod.outlook.com (2603:10b6:a03:39a:cafe::ad) by BY3PR03CA0010.outlook.office365.com (2603:10b6:a03:39a::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 08:01:19 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E66.mail.protection.outlook.com (10.167.249.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:01:18 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:57 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:00:57 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 01:00:51 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=m+t03MGqB6nbQZLxdZtx+6U11x1pEh2DFnuXwoQRFE+zLi5hQfNjJbQ9B2BNQ+a4TeYwjr5iHKWttFOm62Na6y6TAWEL3IA50hEa4DAmL8jTxTFqVmI7Om+39Xi5YMsa9NpMdgJb5WIYg69WAA20tk+WSBLUxdv/eecdsvVXBE2jhWh0x6HW6IMaSMsrRWRNP/TIjstiz+VybfuOZU4O+i+/LMlUKgkB00buyv8+C1UwuiFyd6aUKSx53eb+TIds6H4/HR5FbUeUbzUze5SXEjNzUXbXCrF7ZSri8vkGbIrDSVqP5xaDVNm+WjsJpNse8gliljZSwBBXgWHrI0LIXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uq1hplQ5pK+cE64OCMWgLlH90LFNTj2fkuw/Bad+wOI=; b=BIPwAlYIOVL1j2ABtH/dYqjmVZ8QVFfuQTcWTeqUo0z9g/lwfsFXGY15f3vjff1Oa+6SiIR5+Ey7CBN/b1WpPn92bJaaiI4kq1T9a3cdBtDeeSfTk3xx46bGh53yG1HQ4B1Ji25Mt0NLccoxJ7BrTz2rWUmrSzQC9ZRR7euTLberPYFdg5bklGf5fmF0iCBP2YNxMhAEF0NmWNKd3Kcx6ZF9OQP9NfiUxBVn+TCuLqwz/sO4DPscI6R+oSQIxez6em0AwPwwHRZHPxx8nuOmS2OHjd2B9PF2KfuW7vyTbl96xNRaAvYcLfqozwTe9pC26lCWXjVRLTvJgzySKz3uOw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uq1hplQ5pK+cE64OCMWgLlH90LFNTj2fkuw/Bad+wOI=; b=XqHsw/YOsg7rbyIw0k7RC+J1sUuE1tCnpfx40s8FcBHvzpzisNyuUoBJ/0smOWfajp5AbkJqJsXBzrcVZ60VbKOI/ossJIA7Z2c8+JEO5X5uTMSdUbITWRG4mOT2gBJN2XwIE/7zbpL31U3Hw+72ZkIWcZyhIN8wnIrAGj18lQuwXVDuPPtZbx/cTpk1tqlfyOrA1/BECAVHrTVpL/Hx03K3Cl2Gt0yJlVRWdec21K8qlVkxXkgF22eljMe4QRIMVXk+qkZZC0AYOQKqa0JuvAX1d0sGKaPTDPFxmec1gQAyflE6lV9s4HF3ksVPMGYZhKoI3Qx9Rbv2IK1AsihHRA== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 11/14] vfio/migration: Add new switchover-ack mechanism Date: Tue, 9 Jun 2026 10:58:09 +0300 Message-ID: <20260609075812.32067-12-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E66:EE_|BL4PR12MB9508:EE_ X-MS-Office365-Filtering-Correlation-Id: 7ebf9f8a-45af-4193-c582-08dec5fd49bd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700016|82310400026|7416014|376014|1800799024|56012099006|11063799006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: pqElLuKGrHnONpyn1jZFscQaySFx0q61txrKjqFYycQ96tTClkuEu8j0jUPw6VqGXY+GPHb8x/ToYC0Tt/A/iJ7iPEgi9XACqCDdMgmAemiQrnE+3ca16FuDDRqW2dgt240RFIHRzaL5esz+66ppbTrGz2SL3i6u8XrtpiyY9rAXxoj8OXv6GV8kL5PJz1om+Y6vp6InCVbWBpeg/OdMQeBWNOX1uDVoEf0LA5HuIBvVvZJ0LXLHuBph7V2IG7yWrU02/dvJDCqquvfj8u6EcMcZnpgOyQ5jh5CmYNFd6l/CH4RBkQQX00QCiZ922WdrvuaSu803UyxmsTiiL2B306dAwD24FcZOLzX2Yv9Cvv640uZg5EBA+dnKBosAMtjApFzhI8itg5figXZroqqgiXGsXDhPgYc85ZyupomMkGRSMoPAGqeGpDUQFs4DHNljeXBEH9IdQWYTBemnBXh92EFZED4F2ImDp5DchB52ahJZ9Y7jpcZh8fKEexAFIU8XTkHFvxF2NaUDc1cLCtGWYk3qqhwHN4YTuNfXnj2Iarl7ySByGABRNANx4oNRid9n8Hinr8bdysqAMczWcQ5IIBAGOXeIULXCWVljuIDDH7+F/aNtx4dSg56wSaBrRp8BRKqwvqGzaNv7dr1w6Tpwaakx48WczPxQeeMKQgR0pPM9g9/uiAAevl9GFXnjScMbyXVMt/6ZOisPSVqoVX3u5LecpAHyNfJUeVCQ+7cW1gA= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(36860700016)(82310400026)(7416014)(376014)(1800799024)(56012099006)(11063799006)(22082099003)(18002099003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: EVzE3wVZ8Ap661CDKFPy7m2LilLYtAxHrCBve5v0S+fDS1EdfPYEigkW38EL4NwaI0txO1km9VFQo6GJzAHO1C6NaDzYFjNH/4zaRtpRyiCnK4EILc7EAQFYjsZ1zbyjjhzd3Xa4OabT7NJ1NM41QpChLYhJPgIfch3dG8Q+soJYjStUMXCC0Y+hrhS2ib2qKAcOKrIwtEkiVEAcy3d3X0clVZyUipG/A54KM7M20N0Nx6kcLDjH7RADwABEMBPFo0rkxBRZ/sR1ot8v/ZcofE988H49TFVLbvmgDRevxqRsAjwJrtsMwOx4wnJs6hCkx+yhBV0xohl5JkE0YnmpXvi1Z4IG5JbxWIb0cLQxhcR6Jk5YDEysWLleH+zOYjx+bonds2r9Za2G6pXsY32I7IV/LpDISaXTGN6sCkYIOnd9//V+8Swvr+bMFE0G5CYt X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:01:18.6635 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7ebf9f8a-45af-4193-c582-08dec5fd49bd X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E66.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL4PR12MB9508 Received-SPF: permerror client-ip=2a01:111:f403:c101::7; envelope-from=avihaih@nvidia.com; helo=BL0PR03CU003.outbound.protection.outlook.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992098441158500 Content-Type: text/plain; charset="utf-8" Add support for the new switchover-ack mechanism. This includes requesting a switchover ACK on the first save_query_pending call (with exact=3Dfalse) if VFIO precopy is supported. This achieves the same functionality of legacy switchover-ack but with the new switchover-ack mechanism. Keep legacy switchover-ack functionality for backward compatibility. Reviewed-by: Peter Xu Signed-off-by: Avihai Horon --- hw/vfio/vfio-migration-internal.h | 1 + hw/vfio/migration.c | 13 ++++++++++++- hw/vfio/trace-events | 2 +- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/hw/vfio/vfio-migration-internal.h b/hw/vfio/vfio-migration-int= ernal.h index a15fc74703..dc741e5142 100644 --- a/hw/vfio/vfio-migration-internal.h +++ b/hw/vfio/vfio-migration-internal.h @@ -58,6 +58,7 @@ typedef struct VFIOMigration { bool multifd_transfer; VFIOMultifd *multifd; bool initial_data_sent; + bool request_switchover_ack; =20 bool event_save_iterate_started; bool event_precopy_empty_hit; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 3ab6b7248f..ebe2eafded 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -582,6 +582,9 @@ static int vfio_save_setup(QEMUFile *f, void *opaque, E= rror **errp) } =20 vfio_query_precopy_size(migration); + if (migrate_switchover_ack() && !migrate_switchover_ack_legacy= ()) { + migration->request_switchover_ack =3D true; + } =20 break; case VFIO_DEVICE_STATE_STOP: @@ -634,6 +637,7 @@ static void vfio_save_cleanup(void *opaque) migration->precopy_init_size =3D 0; migration->precopy_dirty_size =3D 0; migration->initial_data_sent =3D false; + migration->request_switchover_ack =3D false; vfio_migration_cleanup(vbasedev); trace_vfio_save_cleanup(vbasedev->name); } @@ -655,6 +659,7 @@ static void vfio_state_pending(void *opaque, MigPending= Data *pending, VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; uint64_t precopy_size, stopcopy_size; + bool request_switchover_ack =3D false; =20 /* * The final pending query runs during switchover downtime. VFIO does = not @@ -676,10 +681,16 @@ static void vfio_state_pending(void *opaque, MigPendi= ngData *pending, =20 pending->precopy_bytes +=3D precopy_size; pending->stopcopy_bytes +=3D stopcopy_size; + if (migration->request_switchover_ack) { + pending->switchover_ack_pending++; + request_switchover_ack =3D true; + migration->request_switchover_ack =3D false; + } =20 trace_vfio_state_pending(vbasedev->name, migration->stopcopy_size, migration->precopy_init_size, - migration->precopy_dirty_size, exact, final); + migration->precopy_dirty_size, + request_switchover_ack, exact, final); } =20 static bool vfio_is_active_iterate(void *opaque) diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 411a306635..53f0ba357b 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -175,7 +175,7 @@ vfio_save_device_config_state(const char *name) " (%s)" vfio_save_iterate(const char *name, uint64_t precopy_init_size, uint64_t p= recopy_dirty_size) " (%s) precopy initial size %"PRIu64" precopy dirty size= %"PRIu64 vfio_save_iterate_start(const char *name) " (%s)" vfio_save_setup(const char *name, uint64_t data_buffer_size) " (%s) data b= uffer size %"PRIu64 -vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool exact, bool final) " (%s) = stopcopy size %"PRIu64", precopy initial size %"PRIu64", precopy dirty size= %"PRIu64", exact %d, final %d" +vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool request_switchover_ack, bo= ol exact, bool final) " (%s) stopcopy size %"PRIu64", precopy initial size = %"PRIu64", precopy dirty size %"PRIu64", request switchover ack %d, exact %= d, final %d" vfio_send_init_data_flag(const char *name) " (%s)" vfio_vmstate_change(const char *name, int running, const char *reason, con= st char *dev_state) " (%s) running %d reason %s device state %s" vfio_vmstate_change_prepare(const char *name, int running, const char *rea= son, const char *dev_state) " (%s) running %d reason %s device state %s" --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992148; cv=pass; d=zohomail.com; s=zohoarc; b=dXoS75syZgAKOeK78N6rpENi8i2gT+zgcMHxD7RUmuLbcmQYOwPeFmiXCAGDyuyuhKyagQuZ8fDu1zchHpZ1rB2rZ8miIH070tHiddi5OINqXn72rc8YOfpxNf8/x82pmjuZe3aFG+d6DBxfgzIvY9SJqd06BKLh8bCFBrSbGtA= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992148; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=93fh1OwHI68iklAlm8ETJeA+vteug4y5jWG+fFmE/Ss=; b=L8lPo7qSalri3eSMqHwfMuth4CGSgbYmuZxNnROmhfmH2fLr0vtAFiRlT66/sksq3mLTvn5+c4eVHkxUmzmamKVUdy2WDFf3BLdIN4dXJqqa+R7XziI3f1CQbLW0kgGDL9S5DCXLXNGX1YnoRfzSoT9WIhsP6Y1PGMYqrEZyrb4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17809921480001009.9499098312828; Tue, 9 Jun 2026 01:02:28 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrPG-0005wj-Fi; Tue, 09 Jun 2026 04:01:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrP1-0005jb-9L for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:43 -0400 Received: from mail-centralusazlp170100005.outbound.protection.outlook.com ([2a01:111:f403:c111::5] helo=DM1PR04CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrOy-0004xR-Rn for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:39 -0400 Received: from MW4P221CA0016.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::21) by IA1PR12MB6209.namprd12.prod.outlook.com (2603:10b6:208:3e7::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.13; Tue, 9 Jun 2026 08:01:28 +0000 Received: from CO1PEPF00012E65.namprd05.prod.outlook.com (2603:10b6:303:8b:cafe::77) by MW4P221CA0016.outlook.office365.com (2603:10b6:303:8b::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.113.10 via Frontend Transport; Tue, 9 Jun 2026 08:01:28 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E65.mail.protection.outlook.com (10.167.249.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:01:28 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:01:05 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:01:04 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 01:00:59 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rdttqMRKhCNaicHYEfhILmO3f87dMO39g2UfDCJPi7QyduQX6Rm7MSl+2PxZ8V0oiIwfFNYRC+jzsttB7V42eCneOr3j5MbY55wjALREB6qImlIE3NXVhLMvtbfat/CyZiDhtiHSas8YPHtpxrW0n5VaSRjsz5YE2/YrZNQQ5JNxGWBDvfWWauUXARcU0q+HNsz1wGWhm0zfklDPWMt57lo2IRTJ2iGD+uuK7/IoAGumFR46n2oXPYRXL5zHvJs8lZBHa9MNyJ6aIMmCMiJ44kH/ANrbpQasVc1fJshTlRGRSC0McBuXnlYEWx/tfCcwlpQ9cXyGu0Qiuv6Q/Ln6sA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=93fh1OwHI68iklAlm8ETJeA+vteug4y5jWG+fFmE/Ss=; b=jYJrRYruGhCs/9vd++D6ZgvIwFLH5Yrgqm32iGAS/rBhnZbkPyA5UIUMMbpeKb0ijUr9c4/pxdzdNRB44Whfv0CMJ0JyCMg2HYYI6Z9dvUT5gyN3f/ndL6ZLeud14wQ/E3H4IN4zlhNSwvWtzKOh3GvIZ2Zrxy9XSSGxG+iEJkfMIX2CQFQzdl2lS7tst0CQdMll0WtdijXNOrGwFB98/sVsKNkZeTRJZeJRAkQ8PNruiauYsxL5kKDab9w6aBZMSmI9SYD0fhJVKbo3G0f8b1c4Jj4U7bS/Z8s/NRtIMFNT/WiW4ACeVoHeMzI8I8+MYFwy/cHIgycqWplaMWdnyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=93fh1OwHI68iklAlm8ETJeA+vteug4y5jWG+fFmE/Ss=; b=o+kTD3fKXI7QHbJSjTJVQDcQmAg7ndGAk7SFob2sLcMV/gtdV+PgfqsUIXQw+lHLxjGWcyCiy7DW+VtI6W0xg7fBttMVGgXXaxDvCBilh0ZPfzTcEqoNlZv088RJuqvVAHImeJPBeIkbAclKSFs/fRu6Xd+dC8NSbtVecumvI18c3bzYufDD312ZdlqW1JgtcFaGSePFhNPWni1NcE68OpU1BNWz3Hbfq5uUUWKNGHC1SnPoaBjEXKbzb2GLqQSkcxLppPZLlRFeGm2dNW2c5tULWZUTb7nRF90OfTle5Kawxs3YvNDEla9T9o1BauucMQh5vbuQ5hMVKQqo+s18Uw== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 12/14] vfio/migration: Implement VFIO_PRECOPY_INFO_REINIT feature Date: Tue, 9 Jun 2026 10:58:10 +0300 Message-ID: <20260609075812.32067-13-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E65:EE_|IA1PR12MB6209:EE_ X-MS-Office365-Filtering-Correlation-Id: 1b8ff2de-302d-4843-3835-08dec5fd4f50 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700016|7416014|376014|82310400026|22082099003|18002099003|6133799003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: Xm5jFUahkgOUkFm3GGfN9XuSGuIYhhN5M5lA+wS5gZfodKyEuHwRiC/0HrSXR+iXFPRw+7KnYmKOz3dY7SNG7lsziiQF7ZzhzrfPfOwY6n9epctIslRTPuxsQtSrThq/gkB9JAMJtY55ULChvihhYOMqZyx2PC+SxMkD9B2jKSYzRIIyBOWHAIDE6N3SGHbNgW1a3CIwJ37Yb2k5de++FUOSilQ9L2H1Od2TV/+y5rXPISVLlq9SjPGTJoYYGS6VND9qUDjkcwua00M7aRsepT+ucs2EBIRENTDVgWlqmDiLyR0p85UUrDqHDRvaSViOC9KdzeDAQ0E+aYOKGm3V99PATSUtVGOVZnKC8EOdcX+8uQkQjskvMAqV/bmzTjWlyFsWlLucgaFO2cNRasPfdW6BS64QIIC2pQhrrI5VZA7lgzs2sYyuc/a4yJ9rBEk6GbaDviY4pmy/xK/P+KK2o5UnExjIKEajcExyd/XDuhQ50vdbq0PC0GlN5UqkjGBsK6BeSf7UTXjD2tVBSYoBNPX2EwqtWBb/pUCx1ESpRmxUiUt3TUpsPveFtT7UcHDpPVTVoEHlZmjkMlfBzoppBjBhPCSCOYinKVXl6sU/llkug23/mYym1Mk1HrFsNpFskfKTcf1tmfjKcudpG5mu+KJIU9xC5HF+MMaUqQUGYPvS4urxVwcZJGOlw/qXp3tXXUuMb8Dedysyo6sk5EVxD2dA+jAgXEWFpjH60DaueY4= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(36860700016)(7416014)(376014)(82310400026)(22082099003)(18002099003)(6133799003)(11063799006)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Nej0oU1nZVf4Vgf6O9BzTA4CXKhFJ3GcDdB4BKIoW87ZFvV31Vt6TQnColJr6TxfEYCL+lJnljKNm6b+/e5xoUB6YnP9IMCm4o05R+GERvfv8t3UQGqgVG9QbPMAp2VLJtBTUtdnZDM/ENFpAY+tsyPEeGBmLCg0Fac1Q5Wf9SLS/1CPIoIroxEoMGi7rTdfeD8//VUUowAeaSgM/pGXzEDgOQ7CBdTT1YZN0v05upuXNQuxK8yA/jZvAUeKZ+UcgIzt2ZsaBNkLRXtCWGEOfZC8VWKujrg4c59jbaKLwtamF6vXLXDvQOX2p38fWhS5DD3LNauAh2lYnuYT8kfMBBKSLtz/MGoETENQ5WLYON+F2qSW9HMn26E3nbLu9Leh8uFd4fow7zn4ROkX9MSiJ2+OfLI3tdDYqg6BJWRbLj+B3NObe9Usy5mR+f/1hNVX X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:01:28.0237 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1b8ff2de-302d-4843-3835-08dec5fd4f50 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E65.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6209 Received-SPF: permerror client-ip=2a01:111:f403:c111::5; envelope-from=avihaih@nvidia.com; helo=DM1PR04CU001.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992148852158500 Content-Type: text/plain; charset="utf-8" According to VFIO uAPI, precopy initial_bytes is considered as critical data that should be transferred and loaded prior to moving to STOP_COPY state to ensure precopy phase would be effective. As currently defined, initial_bytes can only decrease as it's being read from the data fd. However, there are cases where a new chunk of initial_bytes should be transferred during precopy. The new VFIO_PRECOPY_INFO_REINIT feature addresses this and allows reporting a new value for initial_bytes regardless of any previously reported values. Implement VFIO_PRECOPY_INFO_REINIT feature: 1. Opt-in for VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2 to make VFIO_PRECOPY_INFO_REINIT available. 2. Request a new switchover ACK if initial_bytes increases post of a previous switchover ACK. This ensures the device is not moved to STOP_COPY before initial_bytes has reached zero again. Acked-by: Peter Xu Signed-off-by: Avihai Horon --- docs/devel/migration/vfio.rst | 14 +++++++ hw/vfio/vfio-migration-internal.h | 1 + hw/vfio/migration.c | 68 ++++++++++++++++++++++++++++--- hw/vfio/trace-events | 4 +- 4 files changed, 80 insertions(+), 7 deletions(-) diff --git a/docs/devel/migration/vfio.rst b/docs/devel/migration/vfio.rst index 854277b11c..f235c2d4f9 100644 --- a/docs/devel/migration/vfio.rst +++ b/docs/devel/migration/vfio.rst @@ -23,6 +23,20 @@ and recommends that the initial bytes are sent and loade= d in the destination before stopping the source VM. Enabling this migration capability will guarantee that and thus, can potentially reduce downtime even further. =20 +For example, in mlx5 devices, the initial bytes hold metadata used for time +consuming pre-allocations of resources on the destination. Although init b= ytes +may be small in size and sending them may take little time, loading them i= n the +destination can take a significant amount of time. Switchover-ack guarante= es +that this pre-allocation doesn't happen during downtime. + +Initial bytes was originally defined to be monotonically decreasing, howev= er +there are cases where a new chunk of initial bytes should be transferred d= uring +precopy, e.g., due to a device reconfiguration, etc. The +VFIO_PRECOPY_INFO_REINIT feature addresses this and when supported, allows= to +report a new initial bytes value regardless of any previously reported val= ues. +In this case, a new switchover ACK will be requested to make sure the new +initial bytes are loaded in the destination before switching over. + To support migration of multiple devices that might do P2P transactions be= tween themselves, VFIO migration uAPI defines an intermediate P2P quiescent stat= e. While in the P2P quiescent state, P2P DMA transactions cannot be initiated= by diff --git a/hw/vfio/vfio-migration-internal.h b/hw/vfio/vfio-migration-int= ernal.h index dc741e5142..a1c58b1126 100644 --- a/hw/vfio/vfio-migration-internal.h +++ b/hw/vfio/vfio-migration-internal.h @@ -45,6 +45,7 @@ typedef struct VFIOMigration { void *data_buffer; size_t data_buffer_size; uint64_t mig_flags; + bool precopy_info_v2_used; /* * NOTE: all three sizes cached are reported from VFIO's uAPI, which * are defined as estimate only. QEMU should not trust these values diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index ebe2eafded..1e172dd10b 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -373,9 +373,11 @@ static int vfio_query_stop_copy_size(VFIODevice *vbase= dev) =20 static int vfio_query_precopy_size(VFIOMigration *migration) { + VFIODevice *vbasedev =3D migration->vbasedev; struct vfio_precopy_info precopy =3D { .argsz =3D sizeof(precopy), }; + bool reinit =3D false; int ret =3D 0; =20 if (ioctl(migration->data_fd, VFIO_MIG_GET_PRECOPY_INFO, &precopy)) { @@ -383,25 +385,43 @@ static int vfio_query_precopy_size(VFIOMigration *mig= ration) migration->precopy_dirty_size =3D 0; ret =3D -errno; warn_report_once("VFIO device %s ioctl(VFIO_MIG_GET_PRECOPY_INFO) " - "failed (%d)", migration->vbasedev->name, ret); + "failed (%d)", vbasedev->name, ret); } else { bool overflow; =20 migration->precopy_init_size =3D precopy.initial_bytes; migration->precopy_dirty_size =3D precopy.dirty_bytes; + /* + * struct vfio_precopy_info.flags is valid only if + * VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2 is used. + */ + if (migration->precopy_info_v2_used) { + reinit =3D precopy.flags & VFIO_PRECOPY_INFO_REINIT; + } =20 - overflow =3D vfio_migration_check_overflow(migration->vbasedev, + overflow =3D vfio_migration_check_overflow(vbasedev, migration->precopy_init_size, "precopy init size= "); - overflow |=3D vfio_migration_check_overflow(migration->vbasedev, + overflow |=3D vfio_migration_check_overflow(vbasedev, migration->precopy_dirty_size, "precopy dirty siz= e"); if (overflow) { ret =3D -ERANGE; } } =20 - trace_vfio_query_precopy_size(migration->vbasedev->name, - migration->precopy_init_size, - migration->precopy_dirty_size, ret); + trace_vfio_query_precopy_size(vbasedev->name, migration->precopy_init_= size, + migration->precopy_dirty_size, reinit, r= et); + + /* + * If we got new initial_bytes after previous initial_bytes were + * transferred, request a new switchover ACK. Don't request if legacy + * switchover-ack is used. + */ + if (reinit && migration->initial_data_sent && + !migrate_switchover_ack_legacy()) { + migration->initial_data_sent =3D false; + migration->request_switchover_ack =3D true; + trace_vfio_query_precopy_size_request_switchover_ack(vbasedev->nam= e); + } =20 return ret; } @@ -1054,6 +1074,27 @@ static int vfio_migration_query_flags(VFIODevice *vb= asedev, uint64_t *mig_flags) return 0; } =20 +/* Returns 1 on success, 0 if not supported and negative errno on failure = */ +static int vfio_migration_set_precopy_info_v2(VFIODevice *vbasedev) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + + feature->argsz =3D sizeof(buf); + feature->flags =3D + VFIO_DEVICE_FEATURE_SET | VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + if (errno =3D=3D ENOTTY) { + return 0; + } + + return -errno; + } + + return 1; +} + static bool vfio_dma_logging_supported(VFIODevice *vbasedev) { uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), @@ -1075,6 +1116,7 @@ static bool vfio_migration_init(VFIODevice *vbasedev,= Error **errp) char id[256] =3D ""; g_autofree char *path =3D NULL, *oid =3D NULL; uint64_t mig_flags =3D 0; + bool precopy_info_v2_used =3D false; VMChangeStateHandler *prepare_cb; =20 if (!vbasedev->ops->vfio_get_object) { @@ -1106,12 +1148,22 @@ static bool vfio_migration_init(VFIODevice *vbasede= v, Error **errp) return false; } =20 + if (mig_flags & VFIO_MIGRATION_PRE_COPY) { + ret =3D vfio_migration_set_precopy_info_v2(vbasedev); + if (ret < 0) { + error_setg_errno(errp, -ret, "failed to set precopy info v2"); + return false; + } + precopy_info_v2_used =3D ret; + } + vbasedev->migration =3D g_new0(VFIOMigration, 1); migration =3D vbasedev->migration; migration->vbasedev =3D vbasedev; migration->device_state =3D VFIO_DEVICE_STATE_RUNNING; migration->data_fd =3D -1; migration->mig_flags =3D mig_flags; + migration->precopy_info_v2_used =3D precopy_info_v2_used; =20 vbasedev->dirty_pages_supported =3D vfio_dma_logging_supported(vbasede= v); =20 @@ -1134,6 +1186,10 @@ static bool vfio_migration_init(VFIODevice *vbasedev= , Error **errp) migration_add_notifier(&migration->migration_state, vfio_migration_state_notifier); =20 + trace_vfio_migration_init(vbasedev->name, migration->mig_flags, + migration->precopy_info_v2_used, + vbasedev->dirty_pages_supported); + return true; } =20 diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 53f0ba357b..9f5c30d2f2 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -158,11 +158,13 @@ vfio_load_state_device_buffer_starved(const char *nam= e, uint32_t idx) " (%s) idx vfio_load_state_device_buffer_load_start(const char *name, uint32_t idx) "= (%s) idx %"PRIu32 vfio_load_state_device_buffer_load_end(const char *name, uint32_t idx) " (= %s) idx %"PRIu32 vfio_load_state_device_buffer_end(const char *name) " (%s)" +vfio_migration_init(const char *name, uint64_t mig_flags, bool precopy_inf= o_v2_used, bool dirty_pages_supported) " (%s) mig_flags 0x%"PRIx64", precop= y_info_v2_used %d, dirty_pages_supported %d" vfio_migration_realize(const char *name) " (%s)" vfio_migration_set_device_state(const char *name, const char *state) " (%s= ) state %s" vfio_migration_set_state(const char *name, const char *new_state, const ch= ar *recover_state) " (%s) new state %s, recover state %s" vfio_migration_state_notifier(const char *name, int state) " (%s) state %d" -vfio_query_precopy_size(const char *name, uint64_t init_size, uint64_t dir= ty_size, int ret) " (%s) init %"PRIu64" dirty %"PRIu64" ret %d" +vfio_query_precopy_size(const char *name, uint64_t init_size, uint64_t dir= ty_size, bool reinit, int ret) " (%s) init %"PRIu64", dirty %"PRIu64", rein= it %d, ret %d" +vfio_query_precopy_size_request_switchover_ack(const char *name) " (%s)" vfio_query_stop_copy_size(const char *name, uint64_t size, int ret) " (%s)= stopcopy size %"PRIu64" ret %d" vfio_save_block(const char *name, int data_size) " (%s) data_size %d" vfio_save_block_precopy_empty_hit(const char *name) " (%s)" --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992151; cv=pass; d=zohomail.com; s=zohoarc; b=cEqFB0qJ2yYuVv6XlTnlvrJI4KsMaLMbnkgv7U8zXW7+55wiOJCJrUZtkjpVM+EzxVNAe65YGfNaaDgrdMgWHb/cnqfMoIVhsjgBvcjjDaQJv3PwZUtOFKkls3P/STWFe6N7aP6oboK4FQxD43XYM9hWJPV5/1W/U663iIbXXgo= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992151; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=XOZfBibeYzQV6iqoNNH9pw0B7WARl9iQHYdwfCcgRi0=; b=KyXJKySNnYvS1mRIQ3dIBiARwCwAeI+ohSbztkYkKoWQLyLR+wjlfX6dMmgTLUyASh17UYaTYb8bp1eUgb9tiQyTfTe+j1DWEDiI4pYxutKRO1cmi7l4KeV08mgIn2mxEa9QmElxbDButMQ4Ao2cfw+0GNvFjVGvxBisp+qRwQc= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992151255625.6046292042121; Tue, 9 Jun 2026 01:02:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrPJ-00065L-1g; Tue, 09 Jun 2026 04:01:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrP6-0005oU-MN for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:47 -0400 Received: from mail-westusazlp170120002.outbound.protection.outlook.com ([2a01:111:f403:c001::2] helo=SJ2PR03CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrP5-00054R-25 for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:44 -0400 Received: from MN2PR22CA0009.namprd22.prod.outlook.com (2603:10b6:208:238::14) by DS0PR12MB7850.namprd12.prod.outlook.com (2603:10b6:8:146::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.14; Tue, 9 Jun 2026 08:01:34 +0000 Received: from BN1PEPF00006002.namprd05.prod.outlook.com (2603:10b6:208:238:cafe::6d) by MN2PR22CA0009.outlook.office365.com (2603:10b6:208:238::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.92.13 via Frontend Transport; Tue, 9 Jun 2026 08:01:34 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00006002.mail.protection.outlook.com (10.167.243.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:01:34 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:01:13 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:01:13 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 01:01:07 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UchLnEiTYJBxrxO109zPuOfrPVnDDRxU9TT9+a7j/dThJsujA326/vP51eBG/5qrHwhNDZdDSVF+u+BGXs7OGkJlZxKlbQX/Iu3pWmL+CXn0cQUXyTNYwCWmPd4ENBHVjaxOjGq+Ly7aJ56A5aqP6yNtk7qtid4Y2Owk/UIkTG4qhYwTb3qqqHfuEBGxwEgfo1idPouxxm7fsu4zN7kMKxYjctTzuW968WTyYw3JOtpUCYQ6Vd4i0/0qst+MWhPKWdPitmANVtpMe3ZUcbv+dIOLu70ANye+ciot2+QgnOvYuJcTpwG3Rjp9hVgK1rsVC4ZIoM8Ek2xzJkVNUjbXsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XOZfBibeYzQV6iqoNNH9pw0B7WARl9iQHYdwfCcgRi0=; b=wwAWRWrUuw4JlMpJ5XVfv4tfyKO8ULN9TmjM0z3HA09KONUntg5dUDXF3WE6TQ9XyF+1Aqw129uqF1+zoc4QvyQO3U9AqxhdV4C24V259I6g1DrZmpfVxavNYwLGPVcQVpGgeYCt2aGGxXzyFnMrih9lDCul5u2bsGqjjFYeVou+ubYFFZm6hdP96XuUTOsrk9zEv8afG48lm/CgyFrcs2eVDURwIkXdXdONeaA41sF3IoDegbE61D3M4W86Dm/BMkOiU8hNjNie+uHTt3TKH7ZEr5otKDeefFNWvCZFU2MFMf9HATYuiD8KV0oRAXeUkP6J3aEiIGNGuAoPqgFQrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XOZfBibeYzQV6iqoNNH9pw0B7WARl9iQHYdwfCcgRi0=; b=MJWE/Pe1WpThDPAM5FjFvx3uh65X+U1WwDHpirupMONF/kN3Kztzd6uQu7WkAqAXNfeXwGkSNOxJipqFcn7qUdtIppqET/J4VHEI0nHatgqIoA6Y38g+rYmBtJFy8F9cvz43CuWDOaH7RqA7JYAnLyXha/DX/z2IbomGJRut5FsdYx0uL9ULrGGOai2+tQ24XXtcSllh8SB36yJQy7WPRZst6nXTsSzAyHfU4L8gdE16r3pyAQSX1cF03XEOqfZx61zPkdeGRIsXCJI8rKdz3NsulKZUkEzgLiftdN4+ZgDgVt90lz7Eqj/AtwJWdRYafMgiedjEB+uyDPk1h90bxg== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 13/14] vfio/migration: Check VFIO_PRECOPY_INFO_REINIT during switchover Date: Tue, 9 Jun 2026 10:58:11 +0300 Message-ID: <20260609075812.32067-14-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00006002:EE_|DS0PR12MB7850:EE_ X-MS-Office365-Filtering-Correlation-Id: 6bb3b85f-16a0-4937-06ee-08dec5fd5327 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|7416014|82310400026|36860700016|376014|1800799024|18002099003|22082099003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: poxK4Bh+TXZ+zbyFesq5xkpjvA2H9xCAPC5ViTVyTc4oAyyZRY8JU47o3PDfDiNRDDROftetM7h/EHbY0EhF5tOtrwF0W3drE0HkuOpEk1c4FW1TPeEoVp0ikJea01axCau1Krzw/PppeNU3unjAauITdeT6M2iCBWaZMjMLTSwuhczOmJBx71oqVA0rK6/G/v28IU993hOgLiS/jCbZKiLYzHf0ah+F5ILJYoAGbBMHBjbRSZyN3DMmqsqqCY0zoUD+gauUMfffAh5xN/EhmqU04EzGjM+R0rdV/Mkn8BmPzSXRnGa5IL07wbld/H+pKVhKe4IavxZz+i61EQnaL4HH7AXEFoLgDwkKWjFsrDttC73bwjD2SzDhdQP5kWcJQucP+vKm0Gut2pR70MPU4I/EFa+x/vmcH/+tA4MnMsUJxhl9ar2tgB4rcBwMz1NUfRp1NpusZf8SG2BcQNjmmIDtKzWeaE/ozvNOfYtBiZODylPKqT3igvk1tOtIMBYb+f7FYHOrjOJ5JDpYaZqZicBGGm63FODbskOKf6PKG9djhkxSTyRkAyqSiRxbNUn/nifSr/ye2K9eNSYpDqhVYIT9FURt4YJBwGqqmLXRU0ZP1iWebcprMFCIvnmhgPTvCkH8gO/wSD8oE1kuP+JyQYqomAWuA9N1lO3BmIvnedjmIEHeFZMdPFLIc82fvgFxYZiWDrVF7A6g8wO21lO4bEoBOoy0efDP16H5lU7JNGs= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(7416014)(82310400026)(36860700016)(376014)(1800799024)(18002099003)(22082099003)(11063799006)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6gBo10EQz0veOLcdymk0KmiIQqwr1jwIlT93d5ZmEdULNjxO4orpdX7eekiNaDCXT6SgWS1YPgFYa9Hh6H6b4kNIwlTbSuR0DagbnSOayYuSvnix8jvsz5aap95AEvMIUt2E+gxVRRLhlMVH+J3h0xkNgCH+E7T3bEQkfR2Do6K5e+oJI6erGyhK+FrLx0Ski692lNZaZow9lfIBeAQGBhmnu1ijTqrKqj2ezPqnDMbwl0zzn3Tid7+kN5OrUgJrNuaoeQFzxM22hSvW5LLrMFGAdmfbnNBwiO1Az8eH54QNwdX3DMP1hB57eL6zLY3/LtIG8l4ELlCKZN5amehDn1vWvbUYNOBayYNyGCXoRAKBx/yNPnwRlwX5nMH1AaYL8sKuAAHs1mqfqFcYGGAOOHi2WQsr8wMt0hkJcShMkCDnpdQKC2fwy8Q4fWvwNlGt X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:01:34.3789 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6bb3b85f-16a0-4937-06ee-08dec5fd5327 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00006002.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7850 Received-SPF: permerror client-ip=2a01:111:f403:c001::2; envelope-from=avihaih@nvidia.com; helo=SJ2PR03CU001.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992153310154100 Content-Type: text/plain; charset="utf-8" VFIO_REPCOPY_INFO_REINIT is checked only during precopy, before the switchover decision. However, the switchover decision and guest stop are not atomic, so a VFIO device may want to set VFIO_PRECOPY_INFO_REINIT and request another switchover ACK in the gap after switchover decision has been made but before the guest is stopped. This would be missed and may increase downtime. Solve this by checking if VFIO_PRECOPY_INFO_REINIT was set during that gap, and request a new switchover-ack in the final save_state_pending call. Query precopy info after vCPUs are stopped but before transitioning from PRE_COPY state, when its valid to call the ioctl. Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 1e172dd10b..73c49d8c24 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -685,6 +685,9 @@ static void vfio_state_pending(void *opaque, MigPending= Data *pending, * The final pending query runs during switchover downtime. VFIO does = not * need a fresh device pending-data query then to get the latest dirty * data, so avoid the extra work and report the cached counters below. + * On the other hand, precopy sync is needed to check if switchover AC= K was + * requested, but that's already done during guest stop when device is= in + * PRE_COPY state. */ if (exact && !final) { vfio_state_pending_sync(vbasedev); @@ -964,6 +967,26 @@ static const SaveVMHandlers savevm_vfio_handlers =3D { =20 /* ---------------------------------------------------------------------- = */ =20 +static void vfio_final_precopy_reinit_check(VFIODevice *vbasedev) +{ + VFIOMigration *migration =3D vbasedev->migration; + int ret; + + if (!migration->precopy_info_v2_used || !migrate_switchover_ack() || + migrate_switchover_ack_legacy()) { + return; + } + + ret =3D vfio_query_precopy_size(migration); + if (ret) { + error_report("%s: Final precopy reinit check failed (err: %d)", + vbasedev->name, ret); + /* If query failed, assume reinit and request switchover-ack */ + migration->request_switchover_ack =3D true; + migration->initial_data_sent =3D false; + } +} + static void vfio_vmstate_change_prepare(void *opaque, bool running, RunState state) { @@ -977,6 +1000,15 @@ static void vfio_vmstate_change_prepare(void *opaque,= bool running, VFIO_DEVICE_STATE_PRE_COPY_P2P : VFIO_DEVICE_STATE_RUNNING_P2P; =20 + if (migration->device_state =3D=3D VFIO_DEVICE_STATE_PRE_COPY) { + /* + * Now that vCPUs are stopped, check if new init_bytes are availab= le + * since switchover decision, to be reported in the final + * save_query_pending. + */ + vfio_final_precopy_reinit_check(vbasedev); + } + ret =3D vfio_migration_set_state_or_reset(vbasedev, new_state, &local_= err); if (ret) { /* --=20 2.40.1 From nobody Tue Jun 9 23:09:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1780992136; cv=pass; d=zohomail.com; s=zohoarc; b=ANP4rerrV0WJHuAskRgZijQ+jzCPtTW5Ha5//hywcHybPxOse9ISOKOgT5/k5sERYB5+ZcI2HUPq+IdExEIoXRX6P0bKUQCSCFrX25xFgqO6TT1OOs/C/hVYV0g+hvB201IEIEnKOgk4BUeb4F36MnvTrAFSTZKznxyEJb3Mkps= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780992136; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=IAPf6ypRI91bWzxqxMgh6cVCnam81mpqg/BkfFaWtas=; b=l2L6v6WLG0CUZttUs6YEk+uI44wvek/qNFvzTRcnbW3HeWAOllewvs4iWSf7zETGHTv+teiSkhMPoh/f9M+JCrKCxShDdXo+tUjoYmRvrJeC5771cNJlSRTV0hJmfyqsobfndSvgwmXe8hK6wzLrHe41xf0fy+Nb7xRvGdtbGgo= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1780992136471365.6426091257524; Tue, 9 Jun 2026 01:02:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wWrPL-0006F7-JK; Tue, 09 Jun 2026 04:01:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrPJ-00065W-0n for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:57 -0400 Received: from mail-southcentralusazlp170130001.outbound.protection.outlook.com ([2a01:111:f403:c10c::1] helo=SA9PR02CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wWrPH-0005AB-9l for qemu-devel@nongnu.org; Tue, 09 Jun 2026 04:01:56 -0400 Received: from MN2PR14CA0009.namprd14.prod.outlook.com (2603:10b6:208:23e::14) by CH2PR12MB4086.namprd12.prod.outlook.com (2603:10b6:610:7c::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.48.17; Tue, 9 Jun 2026 08:01:46 +0000 Received: from BN1PEPF00006000.namprd05.prod.outlook.com (2603:10b6:208:23e:cafe::47) by MN2PR14CA0009.outlook.office365.com (2603:10b6:208:23e::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.92.13 via Frontend Transport; Tue, 9 Jun 2026 08:01:45 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00006000.mail.protection.outlook.com (10.167.243.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 08:01:45 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:01:26 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Jun 2026 01:01:26 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 9 Jun 2026 01:01:20 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=niMZAsnhM6ubQ7zMhBCoPQ9cJ+lur0CzIr2fVpSLQa3llROnOBfuqUCnMBaxOlUg9JD4wqlvgMgI29jFm4easJKGAfGAw081W7hk/mMhv7Kb0gkYguADUeWcF5Wo9DZ4MIc4ybZ/9c26jaq6txLE1aUPDz4SlDMoB94MIbiSuVNx5JKlCNNwOPHeaQtb7pl0qk2AOo+mIpssPc3182dqTC+ySDPXBADYe8Qi/ztED8Ft2RyAKxc2jb3slV8Ly4g4JQDjKetJXTqTvJvIE7quYjM4G4TwqurkLxIW023TyZ1NxkgQJ6/NuqYZD/ZueO+1ZlDieE2bgegRQpbVg7nZvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IAPf6ypRI91bWzxqxMgh6cVCnam81mpqg/BkfFaWtas=; b=PXFb2YzzZIYSdpkHKbanhfYGFuweAVl8MG9oq9SRnDg7uVC2BemdSkA2FKOGbZetA4FyHeFMfwq2kC5U9+PQs5Yfv+Bj+93ae6t6KPBf4+PZa2Xbjslmx3KTw66/yyZUm9bJHEruheqmDL11XL/KiY9RvLDpq0h12Fbg37OUNnVC6H0KvobiiuXl3DtBbxJWdkfcmxKdiFHl5ChEm/SlDtXfAplN72fZf4DzH1Bysx/C3f972OegfWBcK2mEm5tmQ6MB7VyQjm4REG7i1B5ULvm9BAW2LIWQ9U7Pnek/R5QlNyKSFNjK7iMU+hKBxbUoIypR8maat1IRI/dAN7nPhQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IAPf6ypRI91bWzxqxMgh6cVCnam81mpqg/BkfFaWtas=; b=tb/Tg0gqj0uuk58ktD6AOO2CRT3+ns2YGkIOUMpJopXrLZYXsAat6E+zYAfpiBSTEam0OGn3VrrSEaInZkNgXiSBZ1ybr1F7FqXHiFvGIZ6xEveALoanUaZ8KXGkl+K5ULc+06sXCQQxpDWSPqIcmixcEhjEsaNVf2VXNCWfffLFZl3qqbVwDX2s+rg20wkVQBL66u4TM0iD5tTtixFi02GV411p+S48UP4WG0ukpm3KUmmWH2Y1ckxcf6dD0GtxVGxDYSWkhzkNzxxonTAlT96LfJWaPV0ufqY0V5VryTeh/Zh+6kGDPdWL+sem8OF2eG51BA7oxTRWY6WbtfR50Q== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Avihai Horon To: CC: Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Xu , Fabiano Rosas , Pierrick Bouvier , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Halil Pasic , "Christian Borntraeger" , Jason Herne , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Eric Farman , Matthew Rosato , "Cornelia Huck" , Eric Blake , "Vladimir Sementsov-Ogievskiy" , John Snow , Markus Armbruster , Maor Gottlieb , Avihai Horon Subject: [PATCH v3 14/14] migration: Enable new switchover-ack Date: Tue, 9 Jun 2026 10:58:12 +0300 Message-ID: <20260609075812.32067-15-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20260609075812.32067-1-avihaih@nvidia.com> References: <20260609075812.32067-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00006000:EE_|CH2PR12MB4086:EE_ X-MS-Office365-Filtering-Correlation-Id: 8c722148-d0b8-4943-05a2-08dec5fd59f1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|7416014|36860700016|82310400026|376014|1800799024|56012099006|22082099003|18002099003|11063799006|6133799003|3023799007; X-Microsoft-Antispam-Message-Info: NkOl9eUCbua+j7klemdWUWhz7W8P9LEIf5ZBbyEeyZvPcHaGL+GDOckk4HYJozjWjuLMFEQ+58V8/nV5yFX0J+scvgN+/fd4DqQaOrGhSMKLomB7KRbvMu5M9YeaaE1h2QLxmEjBX4fnRGBFddguPzPqIUVy4GSOerTvIu/WPxwfqUPQRkkazvoZPbrk6hcWX10GZxeM9Or3bOaDpopMkPoRwiNxwdRjLi/OJ1l4rLpI/BGy8lUX60JLgMJfTuOe8PmLzy/IaVEoFxt290mpRqyTUSinHt/QJebeAKUJatL2HLYt2zK86/TYWrJ9Jqr7/dg+orqvONC2WQT0v8CeOteAyZ0y7xwi5CmexVbxLerBSiWv7L1EnerYWo5MGZina234A+GQ548ULpL3cPYegO9A+g2DcvfqFRz6QfQDjvakC8y5N6iVnCMEWRHv1BkvldjnB4N5JWgustuikKvA3M1xYXFYO/RhOvhoBlEyX7/4Cunki//rhG1o0w+5JX8XhbTyh1aXsZf7TAz9y1M33PuoKY0974hF9hZpO1Mrog7BRLfdhsV/coSJlIgmOTWJOy+DpzMVse0JmwkBFIx0Rge2IHegF/0FZHPBKyxzNE0Vd/aZ3JsRFm0RJov6A7r3TLRAL5Sb9M1A7faocn0n0Ie+kk907CmknQPLhEZ57dqnznDlXkQ61TEcW0F2ceCgUnRFIPa5BVEkdbllqGG3yV+pRJq26JsYMaUvPYB9KPQ= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(7416014)(36860700016)(82310400026)(376014)(1800799024)(56012099006)(22082099003)(18002099003)(11063799006)(6133799003)(3023799007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: CbwuH35zHpgmmOYK9nLWrulQInj49rX+Gy+CFBHmfS8lstko7k8aGr9us2nIQGFg1EuAzTrbYxt7ucZEs5TeUJf5lob+DsM2o5oqlj0lWjOAadSyGMBYt6oxfpB/kN/pph5i8Qk8hN2e958BVG0RDWW05HAP2WzmIzDPHWkwxzP1RV2ROgnxynN8ezhf30o2FvvNqfnd4k1STMxk0tQ77eD8OzkqNHRmQjxMTFw/9UXOjpBMvvRxmzFIrwtcj40uHEbGvaFZ3X9QhRbUnuob0ykumlKPe0Lv2Ux4WeM/1/arirnJrT+Z0W+N/PcsX09UCvkivH42qbYZ4ACWYLeE4dnXnsqWtK3Aj5J3J+Rg7Njf/z7CAS7AVpnkr88Fzyzoonnpa8PW/MKhwqpI5NvjsNIuR6Xd4ZMW3VtOg6VY62+3Z/IIu8EYcy0I4YHY/21F X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 08:01:45.7614 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8c722148-d0b8-4943-05a2-08dec5fd59f1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00006000.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4086 Received-SPF: permerror client-ip=2a01:111:f403:c10c::1; envelope-from=avihaih@nvidia.com; helo=SA9PR02CU001.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1780992139009154100 Content-Type: text/plain; charset="utf-8" Now that VFIO has implemented new switchover-ack, enable it. Signed-off-by: Avihai Horon --- migration/options.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/migration/options.c b/migration/options.c index 4c9b25372e..dfce19405d 100644 --- a/migration/options.c +++ b/migration/options.c @@ -110,9 +110,8 @@ const Property migration_properties[] =3D { preempt_pre_7_2, false), DEFINE_PROP_BOOL("multifd-clean-tls-termination", MigrationState, multifd_clean_tls_termination, true), - /* Use legacy until VFIO implements new switchover-ack */ DEFINE_PROP_BOOL("switchover-ack-legacy", MigrationState, - switchover_ack_legacy, true), + switchover_ack_legacy, false), =20 /* Migration parameters */ DEFINE_PROP_UINT8("x-throttle-trigger-threshold", MigrationState, --=20 2.40.1