From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012945; cv=none; d=zohomail.com; s=zohoarc; b=YVOHZYlZU8etZT6KEHZkNjiZzIBBQ5rh7BfNM1O1h8JlXn1yUPseukx7U4OB5etg5A0NbHs7db2MD93Vea12HNMdRiHGMuen8+aOmYgUjVPslmAMAF0/pjj7rJvr0+8lj388M3xKQVt2NvT+JzpaU5XZELCZiE6XPpIVyJJRwoU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012945; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=cuOw5wi626TK+F1PhtvnlgO8EB6xWCZ2vQsoxWAJJRo=; b=dCk3luZDdJZK/0WbyMrfnEhCJW0ULP6WEGrQkBQ6fWc4rhYjMm2mqpCQk3M+Fl8E9IaCbMeHFu9I2+Pcu5G12QzNpil6AmNJ1+1pL74eaDGpDbCrCj4/7jKu+AYxv40O3ypDwZOhEqk7SeROjYWlqsV1QgwOiw+bRfVzFPccTjc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012945006727.9251186641885; Tue, 5 May 2026 13:29:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMM1-0004Mh-Cn; Tue, 05 May 2026 16:26:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMLz-0004MG-Oz for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMLx-0002ZV-9e for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:51 -0400 Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-607-xlykXANvMiWEnLvXX42xqA-1; Tue, 05 May 2026 16:26:46 -0400 Received: by mail-qv1-f72.google.com with SMTP id 6a1803df08f44-8b552610488so80551246d6.0 for ; Tue, 05 May 2026 13:26:46 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cuOw5wi626TK+F1PhtvnlgO8EB6xWCZ2vQsoxWAJJRo=; b=RY2cdt8G37nzNk4U+kKXf7kRWmpz5eV3026N2D/Ed8kyYbgG5vKdmCLpHQulTXmdfPApNs DOYvpYftSO43WBQOS9UuV5keth+sNoJ+qAxwuHU/izzJhvAK3VA2QVGw8xYelgUEccejEc TtMhA/jWyEut2A+uhEKPcb2GwaNMa2U= X-MC-Unique: xlykXANvMiWEnLvXX42xqA-1 X-Mimecast-MFC-AGG-ID: xlykXANvMiWEnLvXX42xqA_1778012806 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012806; x=1778617606; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cuOw5wi626TK+F1PhtvnlgO8EB6xWCZ2vQsoxWAJJRo=; b=mG2GN8G6WX9raPPUTRsh5YP+Cov9NaUiZoufpWhFXcLN8quN/RM4jwSv58GTv6iCTW WZe929D0852XLWDo7ZC+BuwkMZHHBKrqFkCgdT0zPo9L5ASJuGpc1RC6a7fsoGQG8IQZ 0QVDmL4kPtt+LmK6uUJVtat0YVl0mbKnPP1X+9O+TQ/udqJD/3Ympg6+jWjEwA7mfbt+ U3iB9rwhX3TL2bjGxTTXGMGiNYtx6ZNXsXbt4hGrVcCJXKOJSifmCyLlvx1Lk0rdTtfq 2DhIJFCqnH/yjH81gQZ6sg76gaGDBQ24sSFRaLj5xhIOUeHmp7Iy2YzutibZY8zUxIv5 jH3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012806; x=1778617606; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=cuOw5wi626TK+F1PhtvnlgO8EB6xWCZ2vQsoxWAJJRo=; b=HNTmnXpQWJn3UgiesMqEg8N+sF95ZoNYTaZ3rgZ0OKAZRJfe2gm8YkIX8zK+NMrciQ 1imDLa9a3DoK93Vm4oV+ZnYO0S4iYpmSaIlOSU3SKCxLmzN/CXbrZFpmaPxz2/fex+m4 5xXDKk6jb+/9RM4+g8D5+fRdXhFgItMQiPXJPEnEA9AVgta4wz48qnOO3m67oZ/LDK+/ 2VPMoq0oiedbIzXq6cRxClkc5I9u1RCMORjfWYSloE6jLNezTB4S9kzrIlbRVfkw543T JZEdG79YqusUCa1cwPAnLu5uaymNnQfE+F4++ZNLRJcrk8p3V9Ws5pXDT3OlADn7y6wG MhhQ== X-Gm-Message-State: AOJu0Yw0da5O33hlR9T6mdYcqOckZ/6kdA7sziz7rVq4Z8XTbnv+OBph M2xB3DpHBnkkTiXuJoPZMp8CIiqRyJfi9BPoLKQtcP+jpGMDUxp0Z61jPHs9Sj2EBOAZKTPBOCO eO2XelpjLR5gjgsjxvrVlMDCvrrkQKDcpdpgbqSQo/RRkC+yBbTjzFegDPcoWJBSFzldYf3utfU HlmQugHg8+kD3B2a1PodUbi0YRytv98V6FSjmzPw== X-Gm-Gg: AeBDiesiXnZEmA2w0V0IcRhIl7g5ZDuSTTo24QvIO2THcaIHspqRshdnh58vT/fDBpp pJx356LYnMWi5k2Tct4xbDC7mwE8e7q9H1J4mTR0q0aMvVBbiddQ5tsMiMkoTbJCFNQWETvInFD /lyGU1F0syUu+s7cWXjFFgxobEc5j78mtgiagupVzTP5NIS+kqNJ/cyxDk9uG0AqUkrat8rf0O4 jeQ7JBOpxEthercy+ub4vb0P+tp020KAL3vo2tUOGPxItr4fjh0y/esJBO6mVZQxRif7zGWI7/L GhZEkeb8u1sY4U7ddesY7hVk0PMVALMhXc335bfIQVUabPm0AcK/vNe3YENsIPCwM9MmwnCWXwS 6MTDW6qy27OynJYvYDCgMJ6tRjXgXYY34ewg+5/pPo83Mtw5rmov7gUA= X-Received: by 2002:a05:6214:19c6:b0:8ae:62aa:665a with SMTP id 6a1803df08f44-8bc442dbca6mr4401546d6.28.1778012805746; Tue, 05 May 2026 13:26:45 -0700 (PDT) X-Received: by 2002:a05:6214:19c6:b0:8ae:62aa:665a with SMTP id 6a1803df08f44-8bc442dbca6mr4400786d6.28.1778012805050; Tue, 05 May 2026 13:26:45 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Pranav Tyagi , Juraj Marcin Subject: [PULL 01/23] migration: Fix blocking in POSTCOPY_DEVICE during package load Date: Tue, 5 May 2026 16:26:18 -0400 Message-ID: <20260505202640.1011006-2-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012946605158500 Content-Type: text/plain; charset="utf-8" From: Pranav Tyagi The package_loaded event is not set in case MIG_RP_MSG_PONG does not arrive on the source from the destination in the return path thread. The migration thread would then be blocked waiting for package_loaded event indefinitely in POSTCOPY_DEVICE state. Where as, in such a condition the source VM can safely resume as the destination has not yet started. The pong message can get lost in case of a network failure or destination crash before sending the pong. This patch removes the package_loaded event and uses rp_sem, instead of kicking multiple events. The error is detected in case of network failure or destination crash and rp_sem is set in the out path of the return path thread. This will kick the migration thread out from a condition of indefinitely waiting for rp_sem. The migration thread then fails early and breaks from the migration loop to resume the vm on the source side. Fixes: 7b842fe354c6 ("migration: Introduce POSTCOPY_DEVICE state") Signed-off-by: Pranav Tyagi Reviewed-by: Juraj Marcin Reviewed-by: Peter Xu Link: https://lore.kernel.org/r/20260423094438.43556-1-prtyagi@redhat.com Signed-off-by: Peter Xu --- migration/migration.h | 1 - migration/migration.c | 48 ++++++++++++++++++++++++++++--------------- 2 files changed, 31 insertions(+), 18 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index b6888daced..9081e6a612 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -512,7 +512,6 @@ struct MigrationState { bool rdma_migration; =20 bool postcopy_package_loaded; - QemuEvent postcopy_package_loaded_event; =20 GSource *hup_source; =20 diff --git a/migration/migration.c b/migration/migration.c index 5c9aaa6e58..6e4988a590 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1661,7 +1661,6 @@ int migrate_init(MigrationState *s, Error **errp) migration_reset_vfio_bytes_transferred(); =20 s->postcopy_package_loaded =3D false; - qemu_event_reset(&s->postcopy_package_loaded_event); =20 return 0; } @@ -2317,7 +2316,7 @@ static void *source_return_path_thread(void *opaque) if (tmp32 =3D=3D QEMU_VM_PING_PACKAGED_LOADED) { trace_source_return_path_thread_postcopy_package_loaded(); ms->postcopy_package_loaded =3D true; - qemu_event_set(&ms->postcopy_package_loaded_event); + migration_rp_kick(ms); } break; =20 @@ -2388,16 +2387,21 @@ out: trace_source_return_path_thread_bad_end(); } =20 - if (ms->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { + if (ms->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER || + ms->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE) { /* - * this will be extremely unlikely: that we got yet another network - * issue during recovering of the 1st network failure.. during this - * period the main migration thread can be waiting on rp_sem for - * this thread to sync with the other side. + * The migration thread can get stuck waiting for rp_sem if the + * return path fails to sync with the destination. This handles + * two specific cases: * - * When this happens, explicitly kick the migration thread out of - * RECOVER stage and back to PAUSED, so the admin can try - * everything again. + * POSTCOPY_RECOVER: A failure occurs during a recovery attempt. + * We kick the migration thread back to PAUSED so the admin can + * retry. + * + * POSTCOPY_DEVICE: The MIG_RP_MSG_PONG is lost due to a + * network failure or destination crash. We kick the migration + * thread out of its wait so it can fail the migration and safely + * resume the VM on the source. */ migration_rp_kick(ms); } @@ -3226,12 +3230,24 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE && (s->postcopy_package_loaded || complete_ready)) { /* - * If package has been loaded, the event is set and we will - * immediatelly transition to POSTCOPY_ACTIVE. If we are ready= for - * completion, we need to wait for destination to load the pos= tcopy - * package before actually completing. + * We will immediately transition to POSTCOPY_ACTIVE. + * If we are ready for completion, we need to wait for + * destination to load the postcopy package before actually + * completing. */ - qemu_event_wait(&s->postcopy_package_loaded_event); + while (!s->postcopy_package_loaded) { + if (migration_rp_wait(s)) { + /* + * Error happened. Migration thread was stuck waiting = in + * POSTCOPY_DEVICE for rp_sem which was never set. + */ + migrate_set_state(&s->state, + MIGRATION_STATUS_POSTCOPY_DEVICE, + MIGRATION_STATUS_FAILING); + return MIG_ITERATE_BREAK; + } + } + /* Acknowledgement received from the destination */ migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_DEVICE, MIGRATION_STATUS_POSTCOPY_ACTIVE); } @@ -3863,7 +3879,6 @@ static void migration_instance_finalize(Object *obj) qemu_sem_destroy(&ms->rp_state.rp_pong_acks); qemu_sem_destroy(&ms->postcopy_qemufile_src_sem); error_free(ms->error); - qemu_event_destroy(&ms->postcopy_package_loaded_event); } =20 static void migration_instance_init(Object *obj) @@ -3885,7 +3900,6 @@ static void migration_instance_init(Object *obj) qemu_sem_init(&ms->wait_unplug_sem, 0); qemu_sem_init(&ms->postcopy_qemufile_src_sem, 0); qemu_mutex_init(&ms->qemu_file_lock); - qemu_event_init(&ms->postcopy_package_loaded_event, 0); } =20 /* --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012973; cv=none; d=zohomail.com; s=zohoarc; b=b4UDG72XlMHf/kwdg6xNenz0r+id7Wfw70fhWdAutMlMSdp5p22dGrHV43dbpOpzqEX50vC4IkU4zT0RETj+w4FudCLGsUDPCZJo/DLBT7cEliMa1jDR0Pi5fYxxW948rpCB/C6vpIWoR7IY9H47Pbs8Cj1lV+fCK2wgyn7C8Wc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012973; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZaNTaQWHkBdL/8GhbYMD+GsaQBu6eMR27XHz1q7MUT8=; b=Ve9cTh0m+M4GOJ5NeV+wrFMEAIgYa+ma5zl9yiFOMUc4boRILHdGffTNizfOeXP0j2FPMbxKEzf2q1zYp5k6rvNGlGpzsC8m6kUVmkWpvbMfklTIwjc7T93DK/ebGPZRcM47n++ybR2IksNcFoR9bS1KQzaR3QsQcCkSw1NRzm8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012973862187.79263420838015; Tue, 5 May 2026 13:29:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMM4-0004OX-9I; Tue, 05 May 2026 16:26:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM0-0004MT-VQ for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMLy-0002Zd-E0 for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:52 -0400 Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-606-whxsDk6AM7CuCQYz2ge-jw-1; Tue, 05 May 2026 16:26:47 -0400 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-8ba9baef44cso27199556d6.0 for ; Tue, 05 May 2026 13:26:47 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012809; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZaNTaQWHkBdL/8GhbYMD+GsaQBu6eMR27XHz1q7MUT8=; b=eNwVy2wp8CzgfaIVEOS5BFCp7uEwSIrWTsmFdmapziu32ATipl6JPhIlsAhKEXyqr9qaB0 y8OQdeiIvjKr/tsvVo3rOcW+c+Wy/Eq5GxZihUTQl00aT60LHcqzZ9TZlJsGr3LveatklN E3Z8zXBvBGJQCN4YkbfHo/2tqRxWPig= X-MC-Unique: whxsDk6AM7CuCQYz2ge-jw-1 X-Mimecast-MFC-AGG-ID: whxsDk6AM7CuCQYz2ge-jw_1778012807 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012807; x=1778617607; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZaNTaQWHkBdL/8GhbYMD+GsaQBu6eMR27XHz1q7MUT8=; b=VY+yi6wnetvgT62Ybresc4YpWVqoGr2wLWCInYF8G2BZj6g+uzf5OtNG0DAD9KG7Sm Bcu193XAS3Ot/EIxovnqY2JMBNiBCwmsLNqrRLCK6p9RZ99HYrAuiue2F/QR6J3CvcJG EtC6lb11fBIP7lR4OmmNpjy8vuBFefrxqMS07+Ah0DSHJnXhObLiGM/j7DlCWZmuoc8j IC0bCpDRqPqP95BUJKCg8O2x9HyTd704jbMwhpMoojMgFaGj8Q3VXHWNeOYRZTQ1Zbab xeQYRLnCSDAsB+qbhcgwrG3yxszbW37nKFpPGKxur4lNJ66VVqO75fAcvBvlqoIW2xMX V8fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012807; x=1778617607; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ZaNTaQWHkBdL/8GhbYMD+GsaQBu6eMR27XHz1q7MUT8=; b=mwk9CLUs+Oo5oOmOH5zzzLGX0tZsSUyKHGTyc+paffa23PVvL5PoSoUcX+r44jcpPr C9uVoB5IfbPwyzLsAfOcnpcOxr2CISguHZCJw4uLRPeoqlq37os+Zkm7onpX28YLIg9u v+E3zDCPEYPonnS3IgJJzJ+L7IGUw6salpWzV7VAN0e4UmrljCiM5ApNNIxnrevjoj4v 53HM+VukMKf9+VuojMqj+wZfm7GHTQ/0N2PkbHEo3euOXHeLw8C99NV0Mtx77eLDz6XI CxbwkmEsLU3pQ5DpbLrJfXmPOKPq8X0CuDXRQSToFkjD36xhju37OpWsVcinjsOX3IIw M58Q== X-Gm-Message-State: AOJu0Yza75FkweqLJOSu/KQXj+8v4cW1jpp+kZnVZrg2vj+JeAVB/5ax 8Sm0fiyR8P9YNCFD0gV1IWNzLd/rZYtdSRGiIFnH/R3opk2BDxfNdwsAAlEl7s7G8jbzoY90nDe /yXxm94ZsZ5zmMip/d4uC/Y17nr6JTMkSVZAXylgT5/j7CWM/0t7j54mGnNX0DznbIc9bfRsSle TZVbRWKPpXiRgAMU1JNe0FXKrxS/N8seItdNay7g== X-Gm-Gg: AeBDietbPAq6dixo/Z1q+NUIa15xpCdRERjig9uu2HgECMFZ5HM/s2//uewyMDzqYbG emWZvQ6Nsvvn8Fgd3DemVnGdkQHMA+7Znx+Eu1LcvNYKuoJPXGYSEhxhpOfU6yQ2ybrVmVV3d7J DI8Y8pc16gDzs8z6fSV+tcbXuLzsaymwgfa91B6taM3GZKboRk0GLS395tgrvTkwRwqFxZ3khWt wSrRct3KEHQhK8SJmLdQFTcINspoMMkgOXmk2wvBKruC3GwCfPj6fdLym18AQ75lNTFMsyu18XO bkPalJcmAmDwmXeQ5LLQs87C+1iVa3664WJh4iCd2JN3WNq9LGV/IiQIozOuOW+cAhw8/NHMbEm yQIidIf/8sAK3hUu70fEEXkvIOsUuTFvj4ISnfD9GmSE/RFuKjdu9Kdc= X-Received: by 2002:a05:6214:c4f:b0:8a1:8f74:3eab with SMTP id 6a1803df08f44-8bc4249d7a3mr5203196d6.9.1778012806773; Tue, 05 May 2026 13:26:46 -0700 (PDT) X-Received: by 2002:a05:6214:c4f:b0:8a1:8f74:3eab with SMTP id 6a1803df08f44-8bc4249d7a3mr5202436d6.9.1778012806119; Tue, 05 May 2026 13:26:46 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , qemu-stable , "Maciej S. Szmigiero" , "Maciej S. Szmigiero" Subject: [PULL 02/23] migration: Use QAPI_CLONE_MEMBERS in migrate_params_test_apply Date: Tue, 5 May 2026 16:26:19 -0400 Message-ID: <20260505202640.1011006-3-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012976045154100 Content-Type: text/plain; charset="utf-8" From: Fabiano Rosas Use QAPI_CLONE_MEMBERS instead of making an assignment. The QAPI method makes the handling of the TLS strings more intuitive because it clones them as well. This also fixes a segfault when a NULL TLS option is accessed as part of a validation check for another option (e.g. in the zero-copy + multifd compression case). Details follow: Currently, after copying s->parameters to the temporary MigrationParameters object before migrate_params_check(), the references in temporary object to the TLS options are dropped, either because: a) the user set a new option, in which case that's fine as s->parameters still holds the reference to the old memory or, b) the user did not set a new option, in which case keeping the references in the temporary object would later cause them to be freed along with it, leading to double-free when s->parameters is also freed later on. In this second case, it was overlooked that the TLS options can be accessed already during migrate_params_check() as part of validation of another option. Those pointers should not have been cleared. Using QAPI_CLONE_MEMBERS fixes the issue because the temporary object is not stealing a reference from s->parameters anymore. Cc: qemu-stable Fixes: aed97f0563 ("migration: Normalize tls arguments") Reported-by: Maciej S. Szmigiero Link: https://lore.kernel.org/r/a65a1049-9f19-460a-8e27-a62bb30d2727@maciej= .szmigiero.name Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas Tested-by: Maciej S. Szmigiero Link: https://lore.kernel.org/r/20260414223718.23965-1-farosas@suse.de Signed-off-by: Peter Xu --- migration/options.c | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/migration/options.c b/migration/options.c index 7556fbc06b..68441f0276 100644 --- a/migration/options.c +++ b/migration/options.c @@ -1279,9 +1279,9 @@ bool migrate_params_check(MigrationParameters *params= , Error **errp) static void migrate_params_test_apply(MigrationParameters *params, MigrationParameters *dest) { - *dest =3D migrate_get_current()->parameters; + MigrationState *s =3D migrate_get_current(); =20 - /* TODO use QAPI_CLONE() instead of duplicating it inline */ + QAPI_CLONE_MEMBERS(MigrationParameters, dest, &s->parameters); =20 if (params->has_throttle_trigger_threshold) { dest->throttle_trigger_threshold =3D params->throttle_trigger_thre= shold; @@ -1300,24 +1300,18 @@ static void migrate_params_test_apply(MigrationPara= meters *params, } =20 if (params->tls_creds) { + qapi_free_StrOrNull(dest->tls_creds); dest->tls_creds =3D QAPI_CLONE(StrOrNull, params->tls_creds); - } else { - /* clear the reference, it's owned by s->parameters */ - dest->tls_creds =3D NULL; } =20 if (params->tls_hostname) { + qapi_free_StrOrNull(dest->tls_hostname); dest->tls_hostname =3D QAPI_CLONE(StrOrNull, params->tls_hostname); - } else { - /* clear the reference, it's owned by s->parameters */ - dest->tls_hostname =3D NULL; } =20 if (params->tls_authz) { + qapi_free_StrOrNull(dest->tls_authz); dest->tls_authz =3D QAPI_CLONE(StrOrNull, params->tls_authz); - } else { - /* clear the reference, it's owned by s->parameters */ - dest->tls_authz =3D NULL; } =20 if (params->has_max_bandwidth) { @@ -1374,8 +1368,9 @@ static void migrate_params_test_apply(MigrationParame= ters *params, } =20 if (params->has_block_bitmap_mapping) { - dest->has_block_bitmap_mapping =3D true; - dest->block_bitmap_mapping =3D params->block_bitmap_mapping; + qapi_free_BitmapMigrationNodeAliasList(dest->block_bitmap_mapping); + dest->block_bitmap_mapping =3D QAPI_CLONE(BitmapMigrationNodeAlias= List, + params->block_bitmap_mappi= ng); } =20 if (params->has_x_vcpu_dirty_limit_period) { @@ -1399,7 +1394,8 @@ static void migrate_params_test_apply(MigrationParame= ters *params, } =20 if (params->has_cpr_exec_command) { - dest->cpr_exec_command =3D params->cpr_exec_command; + qapi_free_strList(dest->cpr_exec_command); + dest->cpr_exec_command =3D QAPI_CLONE(strList, params->cpr_exec_co= mmand); } } =20 @@ -1555,4 +1551,6 @@ void qmp_migrate_set_parameters(MigrationParameters *= params, Error **errp) } =20 migrate_tls_opts_free(&tmp); + qapi_free_BitmapMigrationNodeAliasList(tmp.block_bitmap_mapping); + qapi_free_strList(tmp.cpr_exec_command); } --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012913; cv=none; d=zohomail.com; s=zohoarc; b=NnyX6un1kgWv+e+nSnv8Jt9sG5fw3iy0CiWyayRHAbSY7GKLYAEeTTSWM5MAg6hyYmA6TNQbI0PbeTQF/HMumtiBatfUntj0bt9HBKEVd45mLcRLh3B/Pa8mJvBnIg4JywFOQPBeF23powNeYD5cHnQ0tmcgVHvMkp7uFlSx+8M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012913; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ebiyZAuHPg80vY31UF8UVr2ykPbfE9ba9kbGfZ1Nazg=; b=fxgzSHoI1wyXIcV0U0Uhu6IynkbZ0sqayT5T/ed33Ql8LUhDrkiTTM+3yy9HxC4eBceQaemejDYXSch7+6viXOT6QudaafG8zULKSNDJ05stDhg05YnVxgCjCIpfpE7KaDRtSjnlqoc3atyxFehz1Vlisd726Qlb2Fq7cd/FFxA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17780129132285.61878734674508; Tue, 5 May 2026 13:28:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMM4-0004PT-R7; Tue, 05 May 2026 16:26:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM1-0004Mr-OZ for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMLz-0002aC-9m for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:53 -0400 Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-614-8H0zrU3yOfu0u8sEeOm6vg-1; Tue, 05 May 2026 16:26:49 -0400 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-8b597b14a22so10660946d6.0 for ; Tue, 05 May 2026 13:26:49 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012810; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ebiyZAuHPg80vY31UF8UVr2ykPbfE9ba9kbGfZ1Nazg=; b=JWTAGjxS1ToXoQvpui3Dupgaj2hyo1Si2cXTz7OEDv03A3VK0cY7YK3zWTWgtNkDEheUmD Q/A6TYXsMsVhMVjvdoN/u66wlKnKPRBWEE980yGZaMxh1dYmFxmUPLyeh2M19RPsKJJ5ks H4uCztQNhhQSaE2rcLLZivpPfhmbl8g= X-MC-Unique: 8H0zrU3yOfu0u8sEeOm6vg-1 X-Mimecast-MFC-AGG-ID: 8H0zrU3yOfu0u8sEeOm6vg_1778012809 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012808; x=1778617608; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ebiyZAuHPg80vY31UF8UVr2ykPbfE9ba9kbGfZ1Nazg=; b=N81xyfGaCY6GEF5Iazf6zunBof6FYYZ2GOl5Z2JCBDzb/Ge2lcFSMY3OVdSVmzXzTX +EcLFTzgDJ40DYLwUbXLes7jTP/hm+ECfTFiu8/iJHoER9fzMixmr1hBpBiGFOhcyU7+ pEQqBiLaHvkXIA2sYK6UW/gWDnynL5oY/CDoccUSi2w7ahY/VCSjciy5Wt36B4X7UTY1 vpFzJty//3NFcJvibe7Y5FLmkfSazyhRBLEsaab3x33q0Laexo1ZZaPF3FsxSzTTs+3r WdYmhV9NcAAa39ZTTirqJjDwgsynRi7aZfshl+UTjoDl2G79C0XB+L9APtJbmz0DQQuN ZADA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012808; x=1778617608; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ebiyZAuHPg80vY31UF8UVr2ykPbfE9ba9kbGfZ1Nazg=; b=KiB69Y+saiVpLxhaGAELl3LpBsJN3ngnDkYU2xMomZau8+p83MYYtnk74md2fwV9YV sockiZydswhFnmfy1rVUepdAiq4yTA3cWYx3OUCZ5GNM55eXxT1v+G+WIbQFx0+1+JAt HxdMVhXnQzn8z7MQQPx4IIS8oAwB45eo3oD+zG3Q1hQeX5Rk7Oz5Z+WESyrFh4lF9xEE 2Re09L7KB1XPZcRnk4+PeUEi2zbiP+98nwHm8RuVkg0wFDlJw9o0s1Opbq54ga1ETbN9 loTuKYXNcjBcU+B0l+gaSdFnBxZKqfp8y67Iy4jOVFkN9pl7iGEJSgdPz//uSko/O2ys Cp8Q== X-Gm-Message-State: AOJu0YxefgB32wrP+OuE7JupyPhUCyc89xAESYQa3+A0TozH+7F2EGF7 bTyzIdzxTZ8NxHHDoc8Yb3uB5iM7EsMbch8wLYv9z8PGx7MBhVVPB+CE5oeZtZGliXGWdJWyi4k Q0b+v/RHoIXwwSFzyfaG8vbv29gPwOE0964Bzwu6Jtc+nXmq4LxSf5QeVKpKv2oKd4yqb6F1qPJ I34L5gfK9yRYXTSKPWWIK0hS05mO/FzFbq7nFHWw== X-Gm-Gg: AeBDieskUnQpXVrCwEoSKtp2b1NcfnbujzK2w5kBcASGndsm84eAuwfEsZTqqu4gNIJ RmQSx+tiWb9H1by4UrSNcKiezFFznIYn0DNNEc1E6rS2W4uQ/XcR6hEDz7Tphjqx64kszatnQ8G vYfloAa2uhw0lCa+cLRezBb2eJvZNAbWiNdlCcxPlTNstjBxIqsNbPCC1P37r0AlWADxR6Jkczz s+1nPVSiLx4ErYHmupKqU3bU3cc5LJj2adyLtH50WvZwMfyScIzxOhkrMW/0DufH9kLmw0RUxad RjShk/wZbJbVoINcyHYM6IpKZsZi3MVUr82VU20gA0FWYAyKD17QGFyQdUUwONKoO2yPrLqDnEu 373yNFoIjzI5l7TD9BLeewAoIvctsDiMenGc0eV2XdxG7rDplgPaPClQ= X-Received: by 2002:a05:6214:e86:b0:89c:3f06:537a with SMTP id 6a1803df08f44-8baa05bda05mr77133536d6.21.1778012808182; Tue, 05 May 2026 13:26:48 -0700 (PDT) X-Received: by 2002:a05:6214:e86:b0:89c:3f06:537a with SMTP id 6a1803df08f44-8baa05bda05mr77133006d6.21.1778012807498; Tue, 05 May 2026 13:26:47 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Samuel Zhang , Markus Armbruster , Li Zhijian Subject: [PULL 03/23] migration/rdma: add x-rdma-chunk-size parameter Date: Tue, 5 May 2026 16:26:20 -0400 Message-ID: <20260505202640.1011006-4-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012915040154100 Content-Type: text/plain; charset="utf-8" From: Samuel Zhang The default 1MB RDMA chunk size causes slow live migration because each chunk triggers a write_flush (ibv_post_send). For 8GB RAM, 1MB chunk size produces ~15000 flushes vs ~3700 with 1024MB chunk size. Add x-rdma-chunk-size parameter to configure the RDMA chunk size for faster migration. Usage: `migrate_set_parameter x-rdma-chunk-size 1024M` Performance with RDMA live migration of 8GB RAM VM: | x-rdma-chunk-size (B) | time (s) | throughput (MB/s) | |-----------------------|----------|-------------------| | 1M (default) | 37.915 | 1,007 | | 32M | 17.880 | 2,260 | | 1024M | 4.368 | 17,529 | Signed-off-by: Samuel Zhang Acked-by: Markus Armbruster Acked-by: Li Zhijian Tested-by: Li Zhijian Acked-by: Fabiano Rosas Acked-by: Peter Xu Link: https://lore.kernel.org/r/20260427031401.3895523-1-guoqing.zhang@amd.= com Signed-off-by: Peter Xu --- qapi/migration.json | 13 +++++++++++-- migration/options.h | 1 + migration/migration-hmp-cmds.c | 11 +++++++++++ migration/options.c | 33 ++++++++++++++++++++++++++++++++- migration/rdma.c | 30 ++++++++++++++++-------------- 5 files changed, 71 insertions(+), 17 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index 7134d4ce47..0db115ec5e 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -806,7 +806,7 @@ # # Features: # -# @unstable: Members @x-checkpoint-delay and +# @unstable: Members @x-checkpoint-delay, @x-rdma-chunk-size, and # @x-vcpu-dirty-limit-period are experimental. # # Since: 2.4 @@ -831,6 +831,7 @@ 'mode', 'zero-page-detection', 'direct-io', + { 'name': 'x-rdma-chunk-size', 'features': [ 'unstable' ] }, 'cpr-exec-command'] } =20 ## @@ -1007,9 +1008,15 @@ # is @cpr-exec. The first list element is the program's filename, # the remainder its arguments. (Since 10.2) # +# @x-rdma-chunk-size: RDMA memory registration chunk size in bytes. +# Default is 1MiB. Must be a power of 2 in the range +# [1MiB, 1024MiB]. Only applies when migrating via RDMA. +# Must be set to the same value on both source and destination +# before migration starts. (Since 11.1) +# # Features: # -# @unstable: Members @x-checkpoint-delay and +# @unstable: Members @x-checkpoint-delay, @x-rdma-chunk-size, and # @x-vcpu-dirty-limit-period are experimental. # # Since: 2.4 @@ -1046,6 +1053,8 @@ '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', '*direct-io': 'bool', + '*x-rdma-chunk-size': { 'type': 'uint64', + 'features': [ 'unstable' ] }, '*cpr-exec-command': [ 'str' ]} } =20 ## diff --git a/migration/options.h b/migration/options.h index b502871097..b46221998a 100644 --- a/migration/options.h +++ b/migration/options.h @@ -87,6 +87,7 @@ const char *migrate_tls_creds(void); const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); ZeroPageDetection migrate_zero_page_detection(void); +uint64_t migrate_rdma_chunk_size(void); =20 /* parameters helpers */ =20 diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 0a193b8f54..4f6c1dbf89 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -451,6 +451,13 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) params->direct_io ? "on" : "off"); } =20 + if (params->has_x_rdma_chunk_size) { + monitor_printf(mon, "%s: %" PRIu64 " bytes\n", + MigrationParameter_str( + MIGRATION_PARAMETER_X_RDMA_CHUNK_SIZE), + params->x_rdma_chunk_size); + } + assert(params->has_cpr_exec_command); monitor_print_cpr_exec_command(mon, params->cpr_exec_command); } @@ -734,6 +741,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDi= ct *qdict) p->has_direct_io =3D true; visit_type_bool(v, param, &p->direct_io, &err); break; + case MIGRATION_PARAMETER_X_RDMA_CHUNK_SIZE: + p->has_x_rdma_chunk_size =3D true; + visit_type_size(v, param, &p->x_rdma_chunk_size, &err); + break; case MIGRATION_PARAMETER_CPR_EXEC_COMMAND: { /* * NOTE: g_autofree will only auto g_free() the strv array when diff --git a/migration/options.c b/migration/options.c index 68441f0276..5cbfd29099 100644 --- a/migration/options.c +++ b/migration/options.c @@ -13,6 +13,7 @@ =20 #include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/units.h" #include "exec/target_page.h" #include "qapi/clone-visitor.h" #include "qapi/error.h" @@ -90,6 +91,7 @@ const PropertyInfo qdev_prop_StrOrNull; =20 #define DEFAULT_MIGRATE_VCPU_DIRTY_LIMIT_PERIOD 1000 /* millisecond= s */ #define DEFAULT_MIGRATE_VCPU_DIRTY_LIMIT 1 /* MB/s */ +#define DEFAULT_MIGRATE_X_RDMA_CHUNK_SIZE MiB =20 const Property migration_properties[] =3D { DEFINE_PROP_BOOL("store-global-state", MigrationState, @@ -183,6 +185,9 @@ const Property migration_properties[] =3D { DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, parameters.zero_page_detection, ZERO_PAGE_DETECTION_MULTIFD), + DEFINE_PROP_UINT64("x-rdma-chunk-size", MigrationState, + parameters.x_rdma_chunk_size, + DEFAULT_MIGRATE_X_RDMA_CHUNK_SIZE), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -1000,6 +1005,15 @@ ZeroPageDetection migrate_zero_page_detection(void) return s->parameters.zero_page_detection; } =20 +uint64_t migrate_rdma_chunk_size(void) +{ + MigrationState *s =3D migrate_get_current(); + uint64_t size =3D s->parameters.x_rdma_chunk_size; + + assert(MiB <=3D size && size <=3D GiB && is_power_of_2(size)); + return size; +} + /* parameters helpers */ =20 AnnounceParameters *migrate_announce_params(void) @@ -1062,7 +1076,7 @@ static void migrate_mark_all_params_present(Migration= Parameters *p) &p->has_announce_step, &p->has_block_bitmap_mapping, &p->has_x_vcpu_dirty_limit_period, &p->has_vcpu_dirty_limit, &p->has_mode, &p->has_zero_page_detection, &p->has_direct_io, - &p->has_cpr_exec_command, + &p->has_x_rdma_chunk_size, &p->has_cpr_exec_command, }; =20 len =3D ARRAY_SIZE(has_fields); @@ -1273,6 +1287,15 @@ bool migrate_params_check(MigrationParameters *param= s, Error **errp) return false; } =20 + if (params->has_x_rdma_chunk_size && + (params->x_rdma_chunk_size < MiB || + params->x_rdma_chunk_size > GiB || + !is_power_of_2(params->x_rdma_chunk_size))) { + error_setg(errp, "Option x_rdma_chunk_size expects " + "a power of 2 in the range 1MiB to 1024MiB"); + return false; + } + return true; } =20 @@ -1393,6 +1416,10 @@ static void migrate_params_test_apply(MigrationParam= eters *params, dest->direct_io =3D params->direct_io; } =20 + if (params->has_x_rdma_chunk_size) { + dest->x_rdma_chunk_size =3D params->x_rdma_chunk_size; + } + if (params->has_cpr_exec_command) { qapi_free_strList(dest->cpr_exec_command); dest->cpr_exec_command =3D QAPI_CLONE(strList, params->cpr_exec_co= mmand); @@ -1520,6 +1547,10 @@ static void migrate_params_apply(MigrationParameters= *params) s->parameters.direct_io =3D params->direct_io; } =20 + if (params->has_x_rdma_chunk_size) { + s->parameters.x_rdma_chunk_size =3D params->x_rdma_chunk_size; + } + if (params->has_cpr_exec_command) { qapi_free_strList(s->parameters.cpr_exec_command); s->parameters.cpr_exec_command =3D diff --git a/migration/rdma.c b/migration/rdma.c index 55ab85650a..3e37a1d440 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -45,10 +45,12 @@ #define RDMA_RESOLVE_TIMEOUT_MS 10000 =20 /* Do not merge data if larger than this. */ -#define RDMA_MERGE_MAX (2 * 1024 * 1024) -#define RDMA_SIGNALED_SEND_MAX (RDMA_MERGE_MAX / 4096) +static inline uint64_t rdma_merge_max(void) +{ + return migrate_rdma_chunk_size() * 2; +} =20 -#define RDMA_REG_CHUNK_SHIFT 20 /* 1 MB */ +#define RDMA_SIGNALED_SEND_MAX 512 =20 /* * This is only for non-live state being migrated. @@ -527,21 +529,21 @@ static int qemu_rdma_exchange_send(RDMAContext *rdma,= RDMAControlHeader *head, static inline uint64_t ram_chunk_index(const uint8_t *start, const uint8_t *host) { - return ((uintptr_t) host - (uintptr_t) start) >> RDMA_REG_CHUNK_SHIFT; + return ((uintptr_t) host - (uintptr_t) start) / migrate_rdma_chunk_siz= e(); } =20 static inline uint8_t *ram_chunk_start(const RDMALocalBlock *rdma_ram_bloc= k, uint64_t i) { return (uint8_t *)(uintptr_t)(rdma_ram_block->local_host_addr + - (i << RDMA_REG_CHUNK_SHIFT)); + (i * migrate_rdma_chunk_size())); } =20 static inline uint8_t *ram_chunk_end(const RDMALocalBlock *rdma_ram_block, uint64_t i) { uint8_t *result =3D ram_chunk_start(rdma_ram_block, i) + - (1UL << RDMA_REG_CHUNK_SHIFT); + migrate_rdma_chunk_size(); =20 if (result > (rdma_ram_block->local_host_addr + rdma_ram_block->length= )) { result =3D rdma_ram_block->local_host_addr + rdma_ram_block->lengt= h; @@ -1841,6 +1843,7 @@ static int qemu_rdma_write_one(RDMAContext *rdma, struct ibv_send_wr *bad_wr; int reg_result_idx, ret, count =3D 0; uint64_t chunk, chunks; + uint64_t chunk_size =3D migrate_rdma_chunk_size(); uint8_t *chunk_start, *chunk_end; RDMALocalBlock *block =3D &(rdma->local_ram_blocks.block[current_index= ]); RDMARegister reg; @@ -1861,22 +1864,21 @@ retry: chunk_start =3D ram_chunk_start(block, chunk); =20 if (block->is_ram_block) { - chunks =3D length / (1UL << RDMA_REG_CHUNK_SHIFT); + chunks =3D length / chunk_size; =20 - if (chunks && ((length % (1UL << RDMA_REG_CHUNK_SHIFT)) =3D=3D 0))= { + if (chunks && ((length % chunk_size) =3D=3D 0)) { chunks--; } } else { - chunks =3D block->length / (1UL << RDMA_REG_CHUNK_SHIFT); + chunks =3D block->length / chunk_size; =20 - if (chunks && ((block->length % (1UL << RDMA_REG_CHUNK_SHIFT)) =3D= =3D 0)) { + if (chunks && ((block->length % chunk_size) =3D=3D 0)) { chunks--; } } =20 trace_qemu_rdma_write_one_top(chunks + 1, - (chunks + 1) * - (1UL << RDMA_REG_CHUNK_SHIFT) / 1024 / 1= 024); + (chunks + 1) * chunk_size / 1024 / 1024); =20 chunk_end =3D ram_chunk_end(block, chunk + chunks); =20 @@ -2176,7 +2178,7 @@ static int qemu_rdma_write(RDMAContext *rdma, rdma->current_length +=3D len; =20 /* flush it if buffer is too large */ - if (rdma->current_length >=3D RDMA_MERGE_MAX) { + if (rdma->current_length >=3D rdma_merge_max()) { return qemu_rdma_write_flush(rdma, errp); } =20 @@ -3522,7 +3524,7 @@ int rdma_registration_handle(QEMUFile *f) } else { chunk =3D reg->key.chunk; host_addr =3D block->local_host_addr + - (reg->key.chunk * (1UL << RDMA_REG_CHUNK_SHIFT)); + (reg->key.chunk * migrate_rdma_chunk_size()); /* Check for particularly bad chunk value */ if (host_addr < (void *)block->local_host_addr) { error_report("rdma: bad chunk for block %s" --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012863; cv=none; d=zohomail.com; s=zohoarc; b=iRTNYOdinES5G4YTTlJTDpK7Iri/d0YIDRbskTq4cwzcEHO5BCqGGb6ISI/yVM0rbF9h4IIu9k6zSLbOx0gmCfyHDMuIZ5rNj1Nwonju5smhGZmE+NeBi+77WwC80GaR8RaBC/ZQh0Wgo5Yeq1voSLMgjEEaMf++497gsVHuPDQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012863; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=MrH4nxdY2z2cb6i8OQLnpO0ege44PFcIyn690wvAT/8=; b=dnH83P+f3dfhqPCXGDnCWaexgKbI9zKGR+XpEQX7PEG3xkD/pADg13MdFD4DHSTVRSTxqgNFooDuo+Y9Zm4tvHmGo44OEwWxFZkl9KS3SnTD9tZZx3+qYQO+iK7hyValGOySzsKR+GewBI7JwUXex97XISXa+QQGqOSxd4CC23A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012863437107.49254716997734; Tue, 5 May 2026 13:27:43 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMM9-0004Qg-2B; Tue, 05 May 2026 16:27:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM2-0004NS-6V for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM0-0002aa-Fg for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:53 -0400 Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-654-2P-krHfOOAKJzVrno4yVGg-1; Tue, 05 May 2026 16:26:50 -0400 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-8acb26fbbb2so90523556d6.2 for ; Tue, 05 May 2026 13:26:50 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MrH4nxdY2z2cb6i8OQLnpO0ege44PFcIyn690wvAT/8=; b=aZsMaKQ8VwV7xKqJvn4vCCfK38JdDPhN1qsxEixvQj7FG1rvT1KuQWk6PfxY9+qLpkAvXP q3/dASt9O63NabAWbszxokxzgMZ65hnW50erNUcWgirl7ut3zgYQVRHPV6XPguWpsrjoy7 ssl9nhTaQDx2+MejICWpmyv3z1jJ3DI= X-MC-Unique: 2P-krHfOOAKJzVrno4yVGg-1 X-Mimecast-MFC-AGG-ID: 2P-krHfOOAKJzVrno4yVGg_1778012810 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012809; x=1778617609; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MrH4nxdY2z2cb6i8OQLnpO0ege44PFcIyn690wvAT/8=; b=mmED+GT6omG/CJegGd3EmBC7ZSs+4TV5hGeD5aOiqpi9q8I/znhJcXeFIV6DomJN1r Hkr+gXV9+Hy/vK0bKJ0zP5S8sKK4/1R5PYmi+/VgtMc7n238VJU01qSoohNrPUttCamT MlUCfzvBULh8CoPtSn6mke/pZy0JoSsSZs9BK98cVi/gI98wIrJmp0/7oODdFMIB2YT2 EhnOblfJaxR6DBQaO/glJlhPGA5FXExca3ewn+cCYtC6IAR9IScNLR2FvzyuCa+xqdWk nOq1DP//rXlBy30gu9gR033Mdlc2nydMKwyJh/Z/NhYEAmGsp5Mf508+6bS+6ryrow4Q ousw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012809; x=1778617609; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=MrH4nxdY2z2cb6i8OQLnpO0ege44PFcIyn690wvAT/8=; b=P7tlNNlhn1L7NnZihByL6XRo1VLCfYS6dfBuKZawE0VaJPl65D+5A2Y9N2QZeNHnyW xVCWjdBtTVzQ8urmqeQidUuEaHPI/FCDuNGeQkkTFXfFtyBB+HxRcwGrt+OvNZplxv1u AnC7eDtrbDAgMZvHG7/9wbb0BEv1SuuJKIR9bhIbvAwJBGUCBphBqVJ6VmqIg1nEPkJt MAtrpG51cZjMTZ3Oz22SMLqKvs+XSKGFDoDWYOb7WL+ClHuHj/beHEKitb0v5IQ3ztNB 9qOomILu3J+vIg/pe6VShBTT6rlaQnX6noG/U8uk6V8ycbMX97DVnieHG0VfczGSy42n Y6Mg== X-Gm-Message-State: AOJu0YwrnUnR+rX8kWWFC5x5WdA1YsOSGOcN1+MAC+mZ6UDhA6xR2fjq Y8w1U+AAc87e/H3EpmENtNHCa6k17QWsM2apwJHv5fgB2ooQF2aUufwwwD8cELKDR+zGeWDERg1 QcOTVTNvrrc3o84LOEDua0xnuEKOsv/Kqq73t8CVNEJrbfgOvKK30fOknopk0Ei8ebGnVWD07tf ZhbViSBLMvAvSjMRbu3mnfg/yGwJ7vh3a1yH75Dg== X-Gm-Gg: AeBDieuM8XJ+OHELJZLWySf7N7XU4RmD7C/WSzjmpjK5+5uYpTFh75rGYqFHS7tIFJ0 i5WnFzGm14EjKoLeaW+CQ55s5PYCyG6PFTeCqnmVsyRNQsmaGqoVjlq7k2Nj3nacgnd00L0Ya0T /zFC+rAb5aKZq5+KsnLKb/Gjvgdmt+0S46pRWsDh7z3N8fc9ZcOaY9U2JlehflVr+h4glo7Smpa QqNCQTM0JkomSAvPOxQ28m5xncft3duAjxKfDjpA3jYC+r10kFPGYQUlI/dbLkN9xqOLsB39b9S mD69Xy1qft6V7hAnHbU/rMrB+YN2oMe87/ri0uc54gYBighESQc5yX7b+ncQOqWG5LNHeETjXEZ ++v1Pa8Szyl3ElJb6uky8MzNfSQ3KbmTAg9kb+G7bN/edeNZ6q5IYMhQ= X-Received: by 2002:a05:6214:468d:b0:89c:ede4:ea35 with SMTP id 6a1803df08f44-8bc449aff84mr4877976d6.24.1778012809254; Tue, 05 May 2026 13:26:49 -0700 (PDT) X-Received: by 2002:a05:6214:468d:b0:89c:ede4:ea35 with SMTP id 6a1803df08f44-8bc449aff84mr4877256d6.24.1778012808652; Tue, 05 May 2026 13:26:48 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , qemu-stable@nongnu.org, Juraj Marcin Subject: [PULL 04/23] migration: Fix low possibility downtime violation Date: Tue, 5 May 2026 16:26:21 -0400 Message-ID: <20260505202640.1011006-5-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012864156158500 Content-Type: text/plain; charset="utf-8" When QEMU queried the estimated version of pending data and thinks it's ready to converge, it'll send another accurate query to make sure of it. It is needed to make sure we collect the latest reports and that equation still holds true. However we missed one tiny little difference here on "<" v.s. "<=3D" when comparing pending_size (A) to threshold_size (B).. QEMU src only re-query if A Link: https://lore.kernel.org/r/20260421202110.306051-3-peterx@redhat.com Signed-off-by: Peter Xu --- migration/migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 6e4988a590..5f4efb1fe5 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3258,7 +3258,7 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) * postcopy started, so ESTIMATE should always match with EXACT * during postcopy phase. */ - if (pending_size < s->threshold_size) { + if (pending_size <=3D s->threshold_size) { qemu_savevm_state_pending_exact(&must_precopy, &can_postcopy); pending_size =3D must_precopy + can_postcopy; trace_migrate_pending_exact(pending_size, must_precopy, --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 38.145.34.151 as permitted sender) client-ip=38.145.34.151; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 38.145.34.151 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1778012882; cv=none; d=zohomail.com; s=zohoarc; b=LCp3YQJmxbpa6DkcTyuLW/FYnQ/5MtTNNQSS+mUicZd2705gPDkmfZYdDVCy86ZhGzzUa8ZrGgyMDYFHxK0W0Zt4cq8jV9fvZnGmRD/DHB6DwkRxPIJU1u6l97roivWJ3UwusudgxwqGalNj0y4/1M7f4Pdb0TqZ8wfsNRhRbZo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012882; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Owner:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=CmjHBJ8XbN1QfelMZFqFN+lYGBuxnVmoSMYFNc5rg38=; b=HogY6mp+Ydv/g6G0HC45jAti9gcbK7x3I/zA3PANpeUUC4AsLvoIzGicDr43hHGhx9v9pMxD6XnsytlmBc9ozw/Lq0q8GZDlM/hr6283BRphVxFIU705R9qXR3SSOqpzqQ0QuLcbqJ9vQQjHZLg69dnYwgqo5IIj3b9y9jKIPOY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 38.145.34.151 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [38.145.34.151]) by mx.zohomail.com with SMTPS id 1778012882322895.1391324320034; Tue, 5 May 2026 13:28:02 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 993) id 6C26141828; Tue, 5 May 2026 16:28:01 -0400 (EDT) Received: from [172.19.199.9] (unknown [10.16.107.18]) by lists.libvirt.org (Postfix) with ESMTP id F032D41821; Tue, 5 May 2026 16:27:05 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 993) id DD12F3F311; Tue, 5 May 2026 16:26:56 -0400 (EDT) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (3072 bits) server-digest SHA256) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 137CD3F2F2 for ; Tue, 5 May 2026 16:26:54 -0400 (EDT) Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-209-BL0Enlf6NuGuueivIbiyeg-1; Tue, 05 May 2026 16:26:51 -0400 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-8b49424ad88so135190706d6.1 for ; Tue, 05 May 2026 13:26:51 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:49 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012814; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CmjHBJ8XbN1QfelMZFqFN+lYGBuxnVmoSMYFNc5rg38=; b=ejZw3B/A4XLz/1UWzOgE20uK8IBE7mZXH7rMmoF99YK3hfgwd1BBUBDpmayeut36aeoLMo l1DtKAB9wlx9OsCYWzleIkUKvTiuczwxJbxUJknKy0tE7qoSRAv8mt0BcDWywekv1JTL4D s9brCI0NIniz2ZLYHP0+UaRTs5qz8M4= X-MC-Unique: BL0Enlf6NuGuueivIbiyeg-1 X-Mimecast-MFC-AGG-ID: BL0Enlf6NuGuueivIbiyeg_1778012811 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012811; x=1778617611; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CmjHBJ8XbN1QfelMZFqFN+lYGBuxnVmoSMYFNc5rg38=; b=bv5uTE5vxeSDAU/wEZhM25ao8d3BfEeq1RHeAkSxvpMg99iPASqlBoMMJfS9shZFrc vNZik397ajjkIMo/Xx6LF7+AcSYg4mLzzKhrNA2lTZ0sOqWHAeXD3PLYuoFrbw/UcgDf dmwVNmCfKr+5X/VRXBHtb/7btkWCTMUxzbiQ/zcqDq5VZP8PDE3obcqVq6KQr/bIcG+i u2KMyBUb+k4DMBRjtBKSk9jmXP0dKX2TJ9rstzB7mLfbFaH/uxNf5NkLJDSOHucgIhaz BbvDUKsEp4071I6NkUy7jBNx7AePuGuWZPqrew8F9hLmlV+E5MmvGsp4R0lNXUEPWseX mlhQ== X-Forwarded-Encrypted: i=1; AFNElJ/NPpB88ToobUXx84P8kHs//UczvtM50If1wb5wTK1qw8C3Xc67Qxl8KjTqv7wbG+3ntwe1Dg==@lists.libvirt.org X-Gm-Message-State: AOJu0YxY4Pv7QZcg1fzdtfmc+zqhnLtyBltAPSADpeTiHVjGDdTCBKex OU/BtYeHPmt6pkjIGcPYS+AKSfjYtwz0UjZWhAlZ+HmjUE/PQwu85uh4Z1WoD4jc3LXHxjekPRC eLmsSz8ct+Rto/BHd6xu+Wu484DlZm7ipTuw9qYBEiAF7cDg5iS4jxMtMJZo= X-Gm-Gg: AeBDiesMeffRTst6POA0AYwXPlQEZM5SNpKE6JtN7Pz12wKy8KW3hSXKXY9bROMNBTl 5GhxtUyFJQSDnlx/PC2LrhElZZHuJ2v2MTk0yaU9KajZ6hC5fkIE3DCI5DnHQZw6Z2JdGmF3n6x JVxIIgFV3pWhNqR85Ql5hpGoIlMjKPV+fJf+YbY+3Qn7Og/RTvJFOd0AMjHl66RkvQzmPYMabPH ezmcQGS36mg6Pt8BbnU1Kr+Zz8Avjn0GkLG32OYheyQ//Ppr7CwtTSKOLs2Ly850HyfuZ3twXDH UrfEm75WJuNXvAHtcU2Xefn8KNLhI7IPg7OvXSaD18t7rjJhyXYqzMzJFTnOu0P5Ekr/BMOyI+2 3+ODS9+y/KMhfz9idfW6pb/LAzroTqnXaRGXSU3JzXTFCCZ2FkCOOoXs= X-Received: by 2002:ad4:5de4:0:b0:8b8:726a:74d4 with SMTP id 6a1803df08f44-8bc42b6c5d7mr4961846d6.16.1778012810460; Tue, 05 May 2026 13:26:50 -0700 (PDT) X-Received: by 2002:ad4:5de4:0:b0:8b8:726a:74d4 with SMTP id 6a1803df08f44-8bc42b6c5d7mr4961096d6.16.1778012809818; Tue, 05 May 2026 13:26:49 -0700 (PDT) To: qemu-devel@nongnu.org Subject: [PULL 05/23] migration/qapi: Rename MigrationStats to MigrationRAMStats Date: Tue, 5 May 2026 16:26:22 -0400 Message-ID: <20260505202640.1011006-6-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: RP7-VxLmA60Wui-fTJbC8cME-k6Pk3d5JsI1C1F8AWU_1778012811 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-ID-Hash: FH5FPMA3SF5JWPKGK4T2TIT6MAJFJRNA X-Message-ID-Hash: FH5FPMA3SF5JWPKGK4T2TIT6MAJFJRNA X-MailFrom: peterx@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-devel.lists.libvirt.org-0; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Fabiano Rosas , Paolo Bonzini , Peter Xu , devel@lists.libvirt.org, Markus Armbruster , Juraj Marcin , Michal Privoznik X-Mailman-Version: 3.3.10 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Peter Xu via Devel Reply-To: Peter Xu X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1778012888823154100 This stats is only about RAM, make it accurate. This paves way for statistics for all devices. Thanks to Markus, who pointed out that docs/devel/qapi-code-gen.rst has a section "Compatibility considerations" stated: Since type names are not visible in the Client JSON Protocol, types may be freely renamed. Even certain refactorings are invisible, such as splitting members from one type into a common base type. Hence this change is not ABI violation according to the document. While at it, touch up the lines to make it read better, correct the restriction on migration status being 'active' or 'completed': over time we grew too many new status that will also report "ram" section. Cc: Daniel P. Berrang=C3=A9 Cc: devel@lists.libvirt.org Reviewed-by: Markus Armbruster Reviewed-by: Juraj Marcin Reviewed-by: Michal Privoznik Link: https://lore.kernel.org/r/20260421202110.306051-4-peterx@redhat.com Signed-off-by: Peter Xu --- docs/about/removed-features.rst | 2 +- qapi/migration.json | 10 +++++----- migration/migration-stats.h | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.= rst index e75db08410..626162022a 100644 --- a/docs/about/removed-features.rst +++ b/docs/about/removed-features.rst @@ -699,7 +699,7 @@ was superseded by ``sections``. ``query-migrate`` return value member ``skipped`` (removed in 9.1) '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' =20 -Member ``skipped`` of the ``MigrationStats`` struct hasn't been used +Member ``skipped`` of the ``MigrationRAMStats`` struct hasn't been used for more than 10 years. Removed with no replacement. =20 ``migrate`` command option ``inc`` (removed in 9.1) diff --git a/qapi/migration.json b/qapi/migration.json index 0db115ec5e..ed475e4261 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -12,7 +12,7 @@ { 'include': 'sockets.json' } =20 ## -# @MigrationStats: +# @MigrationRAMStats: # # Detailed migration status. # @@ -64,7 +64,7 @@ # # Since: 0.14 ## -{ 'struct': 'MigrationStats', +{ 'struct': 'MigrationRAMStats', 'data': {'transferred': 'int', 'remaining': 'int', 'total': 'int' , 'duplicate': 'int', 'normal': 'int', @@ -209,8 +209,8 @@ # If this field is not returned, no migration process has been # initiated # -# @ram: `MigrationStats` containing detailed migration status, only -# returned if status is 'active' or 'completed'(since 1.2) +# @ram: Detailed migration RAM statistics, only returned if migration +# is in progress or completed (since 1.2) # # @xbzrle-cache: `XBZRLECacheStats` containing detailed XBZRLE # migration statistics, only returned if XBZRLE feature is on and @@ -309,7 +309,7 @@ # Since: 0.14 ## { 'struct': 'MigrationInfo', - 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationStats', + 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationRAMStats', '*vfio': 'VfioStats', '*xbzrle-cache': 'XBZRLECacheStats', '*total-time': 'int', diff --git a/migration/migration-stats.h b/migration/migration-stats.h index c0f50144c9..1153520f7a 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -27,7 +27,7 @@ =20 /* * These are the ram migration statistic counters. It is loosely - * based on MigrationStats. + * based on MigrationRAMStats. */ typedef struct { /* --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012988; cv=none; d=zohomail.com; s=zohoarc; b=SH5ABpMZzYJ3brnQULeLDEWY1EBAJNdyhDzMwmyXiJe9W8Qq+EbKprg7Md1mfxRmCxn4OSd3AWec4thTnuvAj8HguF1htFPUzpficWfivkd5NRO1tdAc+uqcPDiBEoO8rEbru2ER79zRkep49WEndlSxRIvTyWpM2FlwT2cf/7E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012988; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TEOh3+O8AjgnVMvq76LmGzcXnm5dygpEr/gRt4xF6/U=; b=nrv8VSc6uV1x/xqs4AzkTW0qGlWSwIwSFR97/2Acd8gaeii36DaqIyhE5VAaIgShp8Cu00cZ+57No9nYUv5JO0bVek2YJZhfGa10DSEkTaRBC5ikShQyLVQk1PNnNsVFBMkRs2CApzEszYUrgz9uYMCcSiv6eicy5dDJnscAZtQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012988111479.8100152338469; Tue, 5 May 2026 13:29:48 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMM7-0004Pu-II; Tue, 05 May 2026 16:26:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM4-0004Op-Bi for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM2-0002be-EE for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:56 -0400 Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-217-wBguDG--P2myM4yXKMogKQ-1; Tue, 05 May 2026 16:26:52 -0400 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-8b640ede74bso102657056d6.1 for ; Tue, 05 May 2026 13:26:52 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012813; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TEOh3+O8AjgnVMvq76LmGzcXnm5dygpEr/gRt4xF6/U=; b=a7u7/W2EQn29/n9zRRkjiJytCJWwW1MKcGc25kBesV2ALJ2wU01f/WQRDhxdpnyLisTrPN AShy9b2izg6Jgl3ZuWu1DbH3FjY2CsACwdMCsv/JLcyS+/1VvyfxD5f0KXw806CgZo9WZQ lpz4448SuifEYTjZo/bTiJRfRo6cqSM= X-MC-Unique: wBguDG--P2myM4yXKMogKQ-1 X-Mimecast-MFC-AGG-ID: wBguDG--P2myM4yXKMogKQ_1778012812 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012811; x=1778617611; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TEOh3+O8AjgnVMvq76LmGzcXnm5dygpEr/gRt4xF6/U=; b=N/WAZRHj7cWi2uTa48x/mxthBuGhP+bjY10jQYA4C/C2ZLX5n2myWY18U0ZIBVjYHQ sXgkQUAm/s1m71f2EEE+BSNTUtK7BDpDFYyX68sjl1y3jNhCw1mYD2L0JDes/8MYODil YqYjRUprvD1/XCSXDzdvJXV4gRCKxMlhBvCXVBY95KFIoie3oEoQXYOBjBbkjAYppvao d2FVhvmOjYI+x7H+rhyZwaLr2LKh24yR120XFLAd0Tbqiak3vh0hgEVuTn9d6xaxP3QA sFu/3tazTBzjRKihmCo8aCKS+g6MgF1BF3QuwrG0H5Sdy823I1jfkXfQ9jeJ7y8ygeMQ hjNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012811; x=1778617611; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=TEOh3+O8AjgnVMvq76LmGzcXnm5dygpEr/gRt4xF6/U=; b=B7cDPuROJmxD30sSuHPX5B5MNh4J4q/K9mX5pA3zN7MwpB8ZSUnNaRVX0dZu4bvhg4 USKBzcNQQjtWJxwrB04f39p/8JfQvnZJcf+WMWnkxgdwOI5qKaMxv86JcLqEx776LMDT +NuXF4Rs1xoV5GTACdkasURxDYFrQ8Cer31o6XQTJkhYbvdcGOCM/eOpH+40Uad5rl58 qZbZnlTFPDvV9eJCnDmZQnjHvBzFamMJxtjpA/rSKQMaNYFtP5hjYYvLic3lGu8CCmMR rQcivnB5T0sG293i6gEtvs8dSeTMvK6hIRwsXUvJo4NejPTbyhh64Jkk6oSClkR57UVM db8w== X-Gm-Message-State: AOJu0YymWWUF3FTTp+yCdGYJDgHchRqsOBtrqREBkP94MTzD536/mijn cx1McBnNvxVTq3++mTTwr6I4zPQhNiYP6I1GXwHK0Inyy1WVOiBPFaT/yuXQU6Mcx7+E0haxtS2 ko1xOPEPPkq2LtfAk3mWWscykT7j4Sdv7U3QQ0Lhfe+N/Tjm77ODeWI/7WBASiaejzvg8TbK5hU ACUDeBNYxXvJUQu5gKZdJ9/MfSW5PEO8T6KTrWDQ== X-Gm-Gg: AeBDietXUskeCfXWtVgZ6KY+G7wOcMIwfoJPNvNMOPCy8L0U+LWDeYETsRhk4F6Datl R3qhr+RTu/O1LQlQEjpiJ1ywoJD5SAQJeU/KD8a22js0UfwmaqI+wrsiqI0Nmq2DFIXECptAMXl 7ahfq6AYa/Das5elEP09JXrWECuEbVxgVzpGIqEzDLTP6U+sdMWRHI9hG/VYigitWfYyyhBEwN0 cIuYzFLw+ms4bz9L4xcLXlpUEZrUWmn48qE5BXarYQ7O88qV6/+79HtUsyj4w2tjhMN4qikgEo9 8EqBRS/Dpa6ZQ66edH8pjWY8LP1sRUpoPL3/iPJ1Y/umwrwgLzQ/xreSg2RQ7ZnCTaqX22C6apU ZdCw+T6ZZ//GPXGLKg/Hc9StrnCznKANTMAyKjD0XCTohk4at8k2ymKM= X-Received: by 2002:ad4:5c8b:0:b0:8b7:f29b:81ee with SMTP id 6a1803df08f44-8bc45465c5fmr3800846d6.33.1778012811448; Tue, 05 May 2026 13:26:51 -0700 (PDT) X-Received: by 2002:ad4:5c8b:0:b0:8b7:f29b:81ee with SMTP id 6a1803df08f44-8bc45465c5fmr3800016d6.33.1778012810759; Tue, 05 May 2026 13:26:50 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Avihai Horon Subject: [PULL 06/23] vfio/migration: Cache stop size in VFIOMigration Date: Tue, 5 May 2026 16:26:23 -0400 Message-ID: <20260505202640.1011006-7-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012988684158500 Content-Type: text/plain; charset="utf-8" Add a field to cache stop size. Note that there's an initial value change in vfio_save_setup for the stop size default, but it shouldn't matter if it is followed with a math of MIN() against VFIO_MIG_DEFAULT_DATA_BUFFER_SIZE. Document that all the three sizes we read from VFIO's uAPI on dirty or stop sizes are estimates, so QEMU needs to always remember they can be anything. Reviewed-by: Avihai Horon Link: https://lore.kernel.org/r/20260421202110.306051-5-peterx@redhat.com Signed-off-by: Peter Xu --- hw/vfio/vfio-migration-internal.h | 8 +++++ hw/vfio/migration.c | 50 ++++++++++++++++++------------- 2 files changed, 38 insertions(+), 20 deletions(-) diff --git a/hw/vfio/vfio-migration-internal.h b/hw/vfio/vfio-migration-int= ernal.h index 814fbd9eba..a15fc74703 100644 --- a/hw/vfio/vfio-migration-internal.h +++ b/hw/vfio/vfio-migration-internal.h @@ -45,8 +45,16 @@ typedef struct VFIOMigration { void *data_buffer; size_t data_buffer_size; uint64_t mig_flags; + /* + * NOTE: all three sizes cached are reported from VFIO's uAPI, which + * are defined as estimate only. QEMU should not trust these values + * but only use them to do best-effort estimates. Always be prepared + * that these sizes may either grow or even shrink in reality while + * read()ing from the VFIO fds. + */ uint64_t precopy_init_size; uint64_t precopy_dirty_size; + uint64_t stopcopy_size; bool multifd_transfer; VFIOMultifd *multifd; bool initial_data_sent; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 83327b6573..5d5fca09bd 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -41,6 +41,12 @@ */ #define VFIO_MIG_DEFAULT_DATA_BUFFER_SIZE (1 * MiB) =20 +/* + * Migration size of VFIO devices can be as little as a few KBs or as big = as + * many GBs. This value should be big enough to cover the worst case. + */ +#define VFIO_MIG_STOP_COPY_SIZE (100 * GiB) + static unsigned long bytes_transferred; =20 static const char *mig_state_to_str(enum vfio_device_mig_state state) @@ -314,8 +320,7 @@ static void vfio_migration_cleanup(VFIODevice *vbasedev) migration->data_fd =3D -1; } =20 -static int vfio_query_stop_copy_size(VFIODevice *vbasedev, - uint64_t *stop_copy_size) +static int vfio_query_stop_copy_size(VFIODevice *vbasedev) { uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + sizeof(struct vfio_device_feature_mig_data_s= ize), @@ -323,16 +328,22 @@ static int vfio_query_stop_copy_size(VFIODevice *vbas= edev, struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; struct vfio_device_feature_mig_data_size *mig_data_size =3D (struct vfio_device_feature_mig_data_size *)feature->data; + VFIOMigration *migration =3D vbasedev->migration; =20 feature->argsz =3D sizeof(buf); feature->flags =3D VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_MIG_DATA_SIZE; =20 if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + /* + * If getting pending migration size fails, VFIO_MIG_STOP_COPY_SIZE + * is reported so downtime limit won't be violated. + */ + migration->stopcopy_size =3D VFIO_MIG_STOP_COPY_SIZE; return -errno; } =20 - *stop_copy_size =3D mig_data_size->stop_copy_length; + migration->stopcopy_size =3D mig_data_size->stop_copy_length; =20 return 0; } @@ -409,6 +420,16 @@ static void vfio_update_estimated_pending_data(VFIOMig= ration *migration, return; } =20 + /* + * The total size remaining requires separate accounting. Do not trust + * the counter, so what we have read() may be more than what reported. + */ + if (migration->stopcopy_size > data_size) { + migration->stopcopy_size -=3D data_size; + } else { + migration->stopcopy_size =3D 0; + } + if (migration->precopy_init_size) { uint64_t init_size =3D MIN(migration->precopy_init_size, data_size= ); =20 @@ -463,7 +484,6 @@ static int vfio_save_setup(QEMUFile *f, void *opaque, E= rror **errp) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; - uint64_t stop_copy_size =3D VFIO_MIG_DEFAULT_DATA_BUFFER_SIZE; int ret; =20 if (!vfio_multifd_setup(vbasedev, false, errp)) { @@ -472,9 +492,9 @@ static int vfio_save_setup(QEMUFile *f, void *opaque, E= rror **errp) =20 qemu_put_be64(f, VFIO_MIG_FLAG_DEV_SETUP_STATE); =20 - vfio_query_stop_copy_size(vbasedev, &stop_copy_size); + vfio_query_stop_copy_size(vbasedev); migration->data_buffer_size =3D MIN(VFIO_MIG_DEFAULT_DATA_BUFFER_SIZE, - stop_copy_size); + migration->stopcopy_size); migration->data_buffer =3D g_try_malloc0(migration->data_buffer_size); if (!migration->data_buffer) { error_setg(errp, "%s: Failed to allocate migration data buffer", @@ -570,32 +590,22 @@ static void vfio_state_pending_estimate(void *opaque,= uint64_t *must_precopy, migration->precopy_dirty_size); } =20 -/* - * Migration size of VFIO devices can be as little as a few KBs or as big = as - * many GBs. This value should be big enough to cover the worst case. - */ -#define VFIO_MIG_STOP_COPY_SIZE (100 * GiB) - static void vfio_state_pending_exact(void *opaque, uint64_t *must_precopy, uint64_t *can_postcopy) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; - uint64_t stop_copy_size =3D VFIO_MIG_STOP_COPY_SIZE; =20 - /* - * If getting pending migration size fails, VFIO_MIG_STOP_COPY_SIZE is - * reported so downtime limit won't be violated. - */ - vfio_query_stop_copy_size(vbasedev, &stop_copy_size); - *must_precopy +=3D stop_copy_size; + vfio_query_stop_copy_size(vbasedev); + *must_precopy +=3D migration->stopcopy_size; =20 if (vfio_device_state_is_precopy(vbasedev)) { vfio_query_precopy_size(migration); } =20 trace_vfio_state_pending_exact(vbasedev->name, *must_precopy, *can_pos= tcopy, - stop_copy_size, migration->precopy_init= _size, + migration->stopcopy_size, + migration->precopy_init_size, migration->precopy_dirty_size); } =20 --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012885; cv=none; d=zohomail.com; s=zohoarc; b=J+0pG+cw7cRUmTTXHtC7mWbKEIqk1uYH7z/8EGkv1AAmwwPB6uomzeRglfTo6jZISlsDC1P9sxXqwmISsdN+RRhsSj13ZHqlWqTV+VFFfLNb5Duc/JB8DxMhAnI0wYJzpoU4GiajrNVxH+r8pNlGG0W9CUpKlUn/TweOvrnDlN8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012885; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GbxSE6X6ygJUHR7xVHOM2LdntFLpJB3KF+NQvoxXElo=; b=OpSllgsK4dXyXF+wvQ1ePYJO+int+9aneRMBFiDhR8v6dN1cAaeg5OqR/yk/Lad+GMbdn01UauElA3mbftxF/0LQHY/ZauqjuVv6XfQAiMDhiic2veNA7/0gMycUJ6/JyK9n7g8+9pQ8JBiTo5BuqHOPUoxDLeNN+2t33v3etek= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 177801288548687.68948881339963; Tue, 5 May 2026 13:28:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMM-0004a1-0R; Tue, 05 May 2026 16:27:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMB-0004Sb-Me for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM7-0002cX-GG for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:01 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-232-_fbXGpqsOmefzuAdX7mWMQ-1; Tue, 05 May 2026 16:26:54 -0400 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-8b7a1ea06bfso64402976d6.3 for ; Tue, 05 May 2026 13:26:54 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GbxSE6X6ygJUHR7xVHOM2LdntFLpJB3KF+NQvoxXElo=; b=hLBiRIWKMrXnmDbqYm72WURC7Ysw7y9A6WZ1dBAGM4DxTR1ZpEAvSEkM+CG8pFjnyALfqS mCU9CQxbH/y7/MZI1Ij25ltzN4z7u4MBrIhkdkF3HNcXeyadiAtanKt+AvlUL9ZRiyS9Bj pjTGTg5dPSiuCoEzVSrW1ftHbxug9Ys= X-MC-Unique: _fbXGpqsOmefzuAdX7mWMQ-1 X-Mimecast-MFC-AGG-ID: _fbXGpqsOmefzuAdX7mWMQ_1778012814 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012814; x=1778617614; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GbxSE6X6ygJUHR7xVHOM2LdntFLpJB3KF+NQvoxXElo=; b=knZv6Xrl9D8vspdXHQtJ5J12oOxdCsF2W81KZvfTm4iuB0m9NldjGJV2IgYdYLBaSQ Awx9MGDuX74W+xdwE0CVXIxylPVcHTOZq/hmCaKRh0844LuVASfJtP0RT/j20xtu18Hr kpRP8DFNsT1D+Gv3LbQZMiD263QaEKH09+Dr1N/kbC2gHjZ+8JZtjkLJb/m/dSW9U5I8 Eu9rYuRGggfJbdn1PCkuzNen3dz/cQc1QDSzpoH8Ntc6FzL1PCSBaaWiDrXMiavqKJpy 80CnqlNCvHQk69DxZQr2snYTJ9QRj7es60t9rDBuXPaz0zGVwHPdHjQtORFJhN7rte0Z MgvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012814; x=1778617614; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=GbxSE6X6ygJUHR7xVHOM2LdntFLpJB3KF+NQvoxXElo=; b=gjP1ql7A1y5R8V2fNizyOLYTPWSHPaeEei08Wy5cuAcvi+CcHS5ED5EwiWNWhxXpLX +F8rYt+0AE3wePMn3zSJ6tIFXkjmrZjVpSPZ4k8U/l01rV93L2Sa7dX6Rih8QpOLidHS qxSv4oAnO1ipn/EsI2MIoMG/w76Blm4bn8+IWQvksqnINEekVdxcF+IdxDPPgbqAlcsE zUG1erUpECxbXEPqRD2Pz+rxFod8PiGtygyqmMY53rmK7u9VG/C5k1i+oKnTAR1PaOwS y88M6eWx8a7jjnyr+VAj4lXsXRXn2VuNVVCRwHifDP3d28dpvUS+AtbfU8zrRVGDS/Gf z/oQ== X-Gm-Message-State: AOJu0YyxAb/M/V3D7cRxg/wkSp0kMchTVB/goqaLAnURDldYnVGneIy4 fpKLkSho1RYwwUFAPtuOtIIDzRH36i6dxQ8DWmbqouhJaRlWD3UUlERXTJbgvI9aDLMkrEs6ccO +8wpIGjPPuDIP9cgmyKEgLAUr3jeeNbfMzaDcbzDWDr/+ZMEPmFkw9Vc/m0vUMMoigEcgZIHt8E GMs3L/DW7qwGrjiUuSzp75Rv4IJ3dIY8pxkmgkSQ== X-Gm-Gg: AeBDieuPnEdAiineok/fMqTovZ3uTJjbw+LPTC2gNT7kgzyy5T5kLPPuIsbxpNS4fVh /1P09iwSqKvs+sfmGZqrrYx1mHNbcsL2G5f9fBdGXFgMehluW26SXmBZuvacnI5GIsAxGD7gwML 51nzb58ntA16nuM8B+bYt7jviT3TfqWv0n+QGn9M4MBqJcCV1eSH2PjADH8RdAic8a79jden4BB kIoinnUWQ19dzwlPXm8NWSCdQSTgnrffbHBi5+VSUO39XP86W0ByGXpPGsOhLqCzDqfeK0WwMYj TwgQz0z35r3RatqPMGGB4zkhDdR7FYHA+VklZNtkh0vKkb2N20ZA4E0I9gLvhtcHMl8xEoesu2n 4EawKGa5uKHjkUnqsxeCpPSJ7Gb7aQdBiN9HrLu55kR9moobMGNkKt1Y= X-Received: by 2002:ad4:5d63:0:b0:8ac:bb62:fe5a with SMTP id 6a1803df08f44-8bc46014aaamr4056116d6.33.1778012813551; Tue, 05 May 2026 13:26:53 -0700 (PDT) X-Received: by 2002:ad4:5d63:0:b0:8ac:bb62:fe5a with SMTP id 6a1803df08f44-8bc46014aaamr4055126d6.33.1778012812621; Tue, 05 May 2026 13:26:52 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Halil Pasic , Christian Borntraeger , Eric Farman , Matthew Rosato , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Cornelia Huck , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , "Jason J. Herne" , Juraj Marcin , Avihai Horon Subject: [PULL 07/23] migration/treewide: Merge @state_pending_{exact|estimate} APIs Date: Tue, 5 May 2026 16:26:24 -0400 Message-ID: <20260505202640.1011006-8-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012886818158500 Content-Type: text/plain; charset="utf-8" These two APIs are a slight duplication. For example, there're a few users that directly pass in the same function. It might also be error prone to provide two hooks, so that it's easier to happen one module report different things via the two hooks. In reality, they should always report the same thing, only about whether we should use a fast-path when the slow path might be too slow, as QEMU may query these information quite frequently during migration process. Merge it into one API, provide a bool showing if the query is an exact query or not. No functional change intended. Export qemu_savevm_query_pending(). We should use the new API here provided when there're new users to do the query. This will happen very soon. Cc: Halil Pasic Cc: Christian Borntraeger Cc: Eric Farman Cc: Matthew Rosato Cc: Richard Henderson Cc: Ilya Leoshkevich Cc: David Hildenbrand Cc: Cornelia Huck Cc: Eric Blake Cc: Vladimir Sementsov-Ogievskiy Cc: John Snow Reviewed-by: Jason J. Herne Reviewed-by: Juraj Marcin Reviewed-by: Avihai Horon Acked-by: Vladimir Sementsov-Ogievskiy Link: https://lore.kernel.org/r/20260421202110.306051-6-peterx@redhat.com Signed-off-by: Peter Xu --- docs/devel/migration/main.rst | 9 ++---- docs/devel/migration/vfio.rst | 9 ++---- include/migration/register.h | 52 ++++++++++++---------------------- migration/savevm.h | 3 ++ hw/s390x/s390-stattrib.c | 9 +++--- hw/vfio/migration.c | 48 ++++++++++++++----------------- migration/block-dirty-bitmap.c | 10 +++---- migration/ram.c | 33 +++++++-------------- migration/savevm.c | 42 +++++++++++++-------------- hw/vfio/trace-events | 3 +- 10 files changed, 86 insertions(+), 132 deletions(-) diff --git a/docs/devel/migration/main.rst b/docs/devel/migration/main.rst index 2de7050764..430673a499 100644 --- a/docs/devel/migration/main.rst +++ b/docs/devel/migration/main.rst @@ -515,13 +515,8 @@ An iterative device must provide: - A ``load_setup`` function that initialises the data structures on the destination. =20 - - A ``state_pending_exact`` function that indicates how much more - data we must save. The core migration code will use this to - determine when to pause the CPUs and complete the migration. - - - A ``state_pending_estimate`` function that indicates how much more - data we must save. When the estimated amount is smaller than the - threshold, we call ``state_pending_exact``. + - A ``save_query_pending`` function that indicates how much more + data we must save. =20 - A ``save_live_iterate`` function should send a chunk of data until the point that stream bandwidth limits tell it to stop. Each call diff --git a/docs/devel/migration/vfio.rst b/docs/devel/migration/vfio.rst index 0790e5031d..691061d182 100644 --- a/docs/devel/migration/vfio.rst +++ b/docs/devel/migration/vfio.rst @@ -50,13 +50,8 @@ VFIO implements the device hooks for the iterative appro= ach as follows: * A ``load_setup`` function that sets the VFIO device on the destination in _RESUMING state. =20 -* A ``state_pending_estimate`` function that reports an estimate of the - remaining pre-copy data that the vendor driver has yet to save for the V= FIO - device. - -* A ``state_pending_exact`` function that reads pending_bytes from the ven= dor - driver, which indicates the amount of data that the vendor driver has ye= t to - save for the VFIO device. +* A ``save_query_pending`` function that reports the remaining data that + the vendor driver has yet to save for the VFIO device. =20 * An ``is_active_iterate`` function that indicates ``save_live_iterate`` is active only when the VFIO device is in pre-copy states. diff --git a/include/migration/register.h b/include/migration/register.h index d0f37f5f43..e2117e8dd4 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -16,6 +16,13 @@ =20 #include "hw/core/vmstate-if.h" =20 +typedef struct MigPendingData { + /* Amount of pending bytes can be transferred in precopy or stopcopy */ + uint64_t precopy_bytes; + /* Amount of pending bytes can be transferred in postcopy */ + uint64_t postcopy_bytes; +} MigPendingData; + /** * struct SaveVMHandlers: handler structure to finely control * migration of complex subsystems and devices, such as RAM, block and @@ -197,46 +204,23 @@ typedef struct SaveVMHandlers { bool (*save_postcopy_prepare)(QEMUFile *f, void *opaque, Error **errp); =20 /** - * @state_pending_estimate - * - * This estimates the remaining data to transfer + * @save_query_pending * - * Sum of @can_postcopy and @must_postcopy is the whole amount of - * pending data. - * - * @opaque: data pointer passed to register_savevm_live() - * @must_precopy: amount of data that must be migrated in precopy - * or in stopped state, i.e. that must be migrated - * before target start. - * @can_postcopy: amount of data that can be migrated in postcopy - * or in stopped state, i.e. after target start. - * Some can also be migrated during precopy (RAM). - * Some must be migrated after source stops - * (block-dirty-bitmap) - */ - void (*state_pending_estimate)(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy); - - /** - * @state_pending_exact + * This estimates the remaining data to transfer on the source side. * - * This calculates the exact remaining data to transfer + * When @exact is true, a module must report accurate results. When + * @exact is false, a module may report estimates. * - * Sum of @can_postcopy and @must_postcopy is the whole amount of - * pending data. + * It's highly recommended that modules implement a faster version of + * the query path (for example, by proper caching on the counters) if + * an accurate query will be time-consuming. * * @opaque: data pointer passed to register_savevm_live() - * @must_precopy: amount of data that must be migrated in precopy - * or in stopped state, i.e. that must be migrated - * before target start. - * @can_postcopy: amount of data that can be migrated in postcopy - * or in stopped state, i.e. after target start. - * Some can also be migrated during precopy (RAM). - * Some must be migrated after source stops - * (block-dirty-bitmap) + * @pending: pointer to a MigPendingData struct + * @exact: set to true for an accurate (slow) query */ - void (*state_pending_exact)(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy); + void (*save_query_pending)(void *opaque, MigPendingData *pending, + bool exact); =20 /** * @load_state diff --git a/migration/savevm.h b/migration/savevm.h index b3d1e8a13c..e4efd243f3 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -14,6 +14,8 @@ #ifndef MIGRATION_SAVEVM_H #define MIGRATION_SAVEVM_H =20 +#include "migration/register.h" + #define QEMU_VM_FILE_MAGIC 0x5145564d #define QEMU_VM_FILE_VERSION_COMPAT 0x00000002 #define QEMU_VM_FILE_VERSION 0x00000003 @@ -43,6 +45,7 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy); void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s); +void qemu_savevm_query_pending(MigPendingData *pending, bool exact); void qemu_savevm_state_pending_exact(uint64_t *must_precopy, uint64_t *can_postcopy); void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index 2e83aa211c..dfbd452e44 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -187,15 +187,15 @@ static int cmma_save_setup(QEMUFile *f, void *opaque,= Error **errp) return 0; } =20 -static void cmma_state_pending(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy) +static void cmma_state_pending(void *opaque, MigPendingData *pending, + bool exact) { S390StAttribState *sas =3D S390_STATTRIB(opaque); S390StAttribClass *sac =3D S390_STATTRIB_GET_CLASS(sas); long long res =3D sac->get_dirtycount(sas); =20 if (res >=3D 0) { - *must_precopy +=3D res; + pending->precopy_bytes +=3D res; } } =20 @@ -340,8 +340,7 @@ static SaveVMHandlers savevm_s390_stattrib_handlers =3D= { .save_setup =3D cmma_save_setup, .save_live_iterate =3D cmma_save_iterate, .save_complete =3D cmma_save_complete, - .state_pending_exact =3D cmma_state_pending, - .state_pending_estimate =3D cmma_state_pending, + .save_query_pending =3D cmma_state_pending, .save_cleanup =3D cmma_save_cleanup, .load_state =3D cmma_load, .is_active =3D cmma_active, diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 5d5fca09bd..e965ba51fb 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -571,42 +571,39 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } =20 -static void vfio_state_pending_estimate(void *opaque, uint64_t *must_preco= py, - uint64_t *can_postcopy) +static void vfio_state_pending_sync(VFIODevice *vbasedev) { - VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; =20 - if (!vfio_device_state_is_precopy(vbasedev)) { - return; - } - - *must_precopy +=3D - migration->precopy_init_size + migration->precopy_dirty_size; + vfio_query_stop_copy_size(vbasedev); =20 - trace_vfio_state_pending_estimate(vbasedev->name, *must_precopy, - *can_postcopy, - migration->precopy_init_size, - migration->precopy_dirty_size); + if (vfio_device_state_is_precopy(vbasedev)) { + vfio_query_precopy_size(migration); + } } =20 -static void vfio_state_pending_exact(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy) +static void vfio_state_pending(void *opaque, MigPendingData *pending, + bool exact) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; + uint64_t remain; =20 - vfio_query_stop_copy_size(vbasedev); - *must_precopy +=3D migration->stopcopy_size; - - if (vfio_device_state_is_precopy(vbasedev)) { - vfio_query_precopy_size(migration); + if (exact) { + vfio_state_pending_sync(vbasedev); + remain =3D migration->stopcopy_size; + } else { + if (!vfio_device_state_is_precopy(vbasedev)) { + return; + } + remain =3D migration->precopy_init_size + migration->precopy_dirty= _size; } =20 - trace_vfio_state_pending_exact(vbasedev->name, *must_precopy, *can_pos= tcopy, - migration->stopcopy_size, - migration->precopy_init_size, - migration->precopy_dirty_size); + pending->precopy_bytes +=3D remain; + + trace_vfio_state_pending(vbasedev->name, migration->stopcopy_size, + migration->precopy_init_size, + migration->precopy_dirty_size, exact); } =20 static bool vfio_is_active_iterate(void *opaque) @@ -851,8 +848,7 @@ static const SaveVMHandlers savevm_vfio_handlers =3D { .save_prepare =3D vfio_save_prepare, .save_setup =3D vfio_save_setup, .save_cleanup =3D vfio_save_cleanup, - .state_pending_estimate =3D vfio_state_pending_estimate, - .state_pending_exact =3D vfio_state_pending_exact, + .save_query_pending =3D vfio_state_pending, .is_active_iterate =3D vfio_is_active_iterate, .save_live_iterate =3D vfio_save_iterate, .save_complete =3D vfio_save_complete_precopy, diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index a061aad817..15d417013c 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -766,9 +766,8 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void= *opaque) return 0; } =20 -static void dirty_bitmap_state_pending(void *opaque, - uint64_t *must_precopy, - uint64_t *can_postcopy) +static void dirty_bitmap_state_pending(void *opaque, MigPendingData *data, + bool exact) { DBMSaveState *s =3D &((DBMState *)opaque)->save; SaveBitmapState *dbms; @@ -788,7 +787,7 @@ static void dirty_bitmap_state_pending(void *opaque, =20 trace_dirty_bitmap_state_pending(pending); =20 - *can_postcopy +=3D pending; + data->postcopy_bytes +=3D pending; } =20 /* First occurrence of this bitmap. It should be created if doesn't exist = */ @@ -1250,8 +1249,7 @@ static SaveVMHandlers savevm_dirty_bitmap_handlers = =3D { .save_setup =3D dirty_bitmap_save_setup, .save_complete =3D dirty_bitmap_save_complete, .has_postcopy =3D dirty_bitmap_has_postcopy, - .state_pending_exact =3D dirty_bitmap_state_pending, - .state_pending_estimate =3D dirty_bitmap_state_pending, + .save_query_pending =3D dirty_bitmap_state_pending, .save_live_iterate =3D dirty_bitmap_save_iterate, .is_active_iterate =3D dirty_bitmap_is_active_iterate, .load_state =3D dirty_bitmap_load, diff --git a/migration/ram.c b/migration/ram.c index 2046f16caa..44503bf3f7 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3449,30 +3449,18 @@ static int ram_save_complete(QEMUFile *f, void *opa= que) return qemu_fflush(f); } =20 -static void ram_state_pending_estimate(void *opaque, uint64_t *must_precop= y, - uint64_t *can_postcopy) -{ - RAMState **temp =3D opaque; - RAMState *rs =3D *temp; - - uint64_t remaining_size =3D rs->migration_dirty_pages * TARGET_PAGE_SI= ZE; - - if (migrate_postcopy_ram()) { - /* We can do postcopy, and all the data is postcopiable */ - *can_postcopy +=3D remaining_size; - } else { - *must_precopy +=3D remaining_size; - } -} - -static void ram_state_pending_exact(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy) +static void ram_state_pending(void *opaque, MigPendingData *pending, + bool exact) { RAMState **temp =3D opaque; RAMState *rs =3D *temp; uint64_t remaining_size; =20 - if (!migration_in_postcopy()) { + /* + * Sync is not needed either with: (1) a fast query, or (2) after + * postcopy has started (no new dirty will generate anymore). + */ + if (exact && !migration_in_postcopy()) { bql_lock(); WITH_RCU_READ_LOCK_GUARD() { migration_bitmap_sync_precopy(false); @@ -3484,9 +3472,9 @@ static void ram_state_pending_exact(void *opaque, uin= t64_t *must_precopy, =20 if (migrate_postcopy_ram()) { /* We can do postcopy, and all the data is postcopiable */ - *can_postcopy +=3D remaining_size; + pending->postcopy_bytes +=3D remaining_size; } else { - *must_precopy +=3D remaining_size; + pending->precopy_bytes +=3D remaining_size; } } =20 @@ -4709,8 +4697,7 @@ static SaveVMHandlers savevm_ram_handlers =3D { .save_live_iterate =3D ram_save_iterate, .save_complete =3D ram_save_complete, .has_postcopy =3D ram_has_postcopy, - .state_pending_exact =3D ram_state_pending_exact, - .state_pending_estimate =3D ram_state_pending_estimate, + .save_query_pending =3D ram_state_pending, .load_state =3D ram_load, .save_cleanup =3D ram_save_cleanup, .load_setup =3D ram_load_setup, diff --git a/migration/savevm.c b/migration/savevm.c index 765df8ce2d..41f1906598 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1796,46 +1796,44 @@ int qemu_savevm_state_complete_precopy(MigrationSta= te *s) return qemu_fflush(f); } =20 -/* Give an estimate of the amount left to be transferred, - * the result is split into the amount for units that can and - * for units that can't do postcopy. - */ -void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, - uint64_t *can_postcopy) +void qemu_savevm_query_pending(MigPendingData *pending, bool exact) { SaveStateEntry *se; =20 - *must_precopy =3D 0; - *can_postcopy =3D 0; + pending->precopy_bytes =3D 0; + pending->postcopy_bytes =3D 0; =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->state_pending_estimate) { + if (!se->ops || !se->ops->save_query_pending) { continue; } if (!qemu_savevm_state_active(se)) { continue; } - se->ops->state_pending_estimate(se->opaque, must_precopy, can_post= copy); + se->ops->save_query_pending(se->opaque, pending, exact); } } =20 +void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, + uint64_t *can_postcopy) +{ + MigPendingData pending; + + qemu_savevm_query_pending(&pending, false); + + *must_precopy =3D pending.precopy_bytes; + *can_postcopy =3D pending.postcopy_bytes; +} + void qemu_savevm_state_pending_exact(uint64_t *must_precopy, uint64_t *can_postcopy) { - SaveStateEntry *se; + MigPendingData pending; =20 - *must_precopy =3D 0; - *can_postcopy =3D 0; + qemu_savevm_query_pending(&pending, true); =20 - QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->state_pending_exact) { - continue; - } - if (!qemu_savevm_state_active(se)) { - continue; - } - se->ops->state_pending_exact(se->opaque, must_precopy, can_postcop= y); - } + *must_precopy =3D pending.precopy_bytes; + *can_postcopy =3D pending.postcopy_bytes; } =20 void qemu_savevm_state_cleanup(void) diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 846e3625c5..287df0b8cb 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -173,8 +173,7 @@ vfio_save_device_config_state(const char *name) " (%s)" vfio_save_iterate(const char *name, uint64_t precopy_init_size, uint64_t p= recopy_dirty_size) " (%s) precopy initial size %"PRIu64" precopy dirty size= %"PRIu64 vfio_save_iterate_start(const char *name) " (%s)" vfio_save_setup(const char *name, uint64_t data_buffer_size) " (%s) data b= uffer size %"PRIu64 -vfio_state_pending_estimate(const char *name, uint64_t precopy, uint64_t p= ostcopy, uint64_t precopy_init_size, uint64_t precopy_dirty_size) " (%s) pr= ecopy %"PRIu64" postcopy %"PRIu64" precopy initial size %"PRIu64" precopy d= irty size %"PRIu64 -vfio_state_pending_exact(const char *name, uint64_t precopy, uint64_t post= copy, uint64_t stopcopy_size, uint64_t precopy_init_size, uint64_t precopy_= dirty_size) " (%s) precopy %"PRIu64" postcopy %"PRIu64" stopcopy size %"PRI= u64" precopy initial size %"PRIu64" precopy dirty size %"PRIu64 +vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool exact) " (%s) stopcopy siz= e %"PRIu64" precopy initial size %"PRIu64" precopy dirty size %"PRIu64 " ex= act %d" vfio_vmstate_change(const char *name, int running, const char *reason, con= st char *dev_state) " (%s) running %d reason %s device state %s" vfio_vmstate_change_prepare(const char *name, int running, const char *rea= son, const char *dev_state) " (%s) running %d reason %s device state %s" =20 --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012934; cv=none; d=zohomail.com; s=zohoarc; b=QKqXZtUFbl4RRMmSVPnORo9zgUPwVx9vBgI0Midfkx25amnKLxsstNoKRjz4elZedmmUCApkJ4w8fQSZiDaAptcyUDUWHGoFqCy/6Kwmdt+QkU3zdj3qOLfStJ4cOEeBA+l7Si0je7F/5biB4lxaWZ+CuL5/N4GlPgEOyie8kLw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012934; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3dC1qfNQleJy8e4gs+qK727UJ4brhGZIIHFtrIFxT4w=; b=Sb6sl+qc9Nqsh9zkSqRL75UxyLc9pQ7+pPPM94ylFG6dkS1SNCCCTKYcrfSQljKaFkS/lFiQFrEvCfHh+qBwTLEjtzBkFE+zV6G+sSLgtZ2Y72xHzA5LqoQ+0BWNcc/wYp+dkbEYpbL5Tm48em81iyPt+ITHqnjrpiyZ0/06r8Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012934756125.20421634256081; Tue, 5 May 2026 13:28:54 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMA-0004S7-SP; Tue, 05 May 2026 16:27:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM7-0004QT-Nq for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM5-0002cT-PH for qemu-devel@nongnu.org; Tue, 05 May 2026 16:26:59 -0400 Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-681-WrEcZGZaOpucLwz0sWOx0Q-1; Tue, 05 May 2026 16:26:55 -0400 Received: by mail-qv1-f72.google.com with SMTP id 6a1803df08f44-8b5f089a5c3so124372356d6.3 for ; Tue, 05 May 2026 13:26:55 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3dC1qfNQleJy8e4gs+qK727UJ4brhGZIIHFtrIFxT4w=; b=MNRmERA1aHtcusIT+86n56x9HpdOBIS7+jYAVUrNJzG1NtSjM4QWRcSauaRksWd6rGJF5U 6JrYm0Iy4Q6DAbz2b/z4Wh4kq3K/mpjgREKURERoo1DPQ1815GermpulzfpHSxvT0q3Rc8 p27SK/a9d9olwuiVuNnJ5OCgg7GB5jE= X-MC-Unique: WrEcZGZaOpucLwz0sWOx0Q-1 X-Mimecast-MFC-AGG-ID: WrEcZGZaOpucLwz0sWOx0Q_1778012815 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012815; x=1778617615; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3dC1qfNQleJy8e4gs+qK727UJ4brhGZIIHFtrIFxT4w=; b=tHCq57IGFdngi8zGcMAamTyCtQ4YDmEnb3eN56RFaF13R3+SfrymAT+2Zn0auyvTRw H98J5bkAobHn49w5TltKlkC8MrFNFhhTmhON/qHeHjCBT+zoL09YDIm5gEHUSZvPC+uf iHyMSl5FhulbtMEo0boTJUO08or481jneeau+KDPCfKmHDzjC+oNfNoV5MBo47BQa8Fc QYJ8D9eemJlbLv6I5I2uWB9L1vFk5yRs3nsBcKhJIvrDEGlumPvfHbnuHtR4vSLi8+A6 1Ib8iuoV0gb+ZisULhuupA7HUWRCXO9OdOCHXtFd3Z9/ZJGaGHbTW9iHmfIf0V/483fQ 1pkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012815; x=1778617615; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=3dC1qfNQleJy8e4gs+qK727UJ4brhGZIIHFtrIFxT4w=; b=lIQggvhujEeyH1yp6CU5+6hR8myZ70RAYagAZSQRpW9GI4MOYiawgKdaeH0LrAhV2h Omodza1K3XgOrW8CImhgpqzN/WCh8y5cPlMVdS7Ddmehmn9gDqlXJtqNPmRNAWH3ZwkQ hrosXwTuMMVmBxfCR62inUcEUP57Jo8XQqmO39jjlClpkM1qW3c37w4soUy0ZByjUq2i kORG8y0z3dnyZzAYVNq1kL1BsNTy860fjV01jxAhbAf7bROnHCn+WBhhNaHE1/eTD/+X 4PUiyzWihMc8EVC3ccgc1a96fQlk8TcRhV9zEj43MRTEmIhtJm7Rx1JBrpKORe5vt2QF HLYA== X-Gm-Message-State: AOJu0YwQ9NuGnP4Vo6L9lpkpWdsiECgNmI2e0lqTYjiaBnZvm5Z2vJSX AVq3bkCWD0J9RYB8myzgeeXcECV8EseNWqESaUNzegsMUIvIS/pmr5kFkRfVr6dHC36/QBaa71P aH/TPvuFodNL8k4pemFOps6iMwOWTZ+hDWyDQN0jey/MVS2VLwtDFFavbcbV/JhIkRNWw52VyV3 zpbw7+FrXiP/Y0wbAGDZWbw6TYfBaJmhT2Hr01Sw== X-Gm-Gg: AeBDieu9llyLVA6rRSr7ITxfccqTuURlaT/pn4gCCJTXVaFHL6KJtgL7VHUIPdgxrRk kJSlwv97PrmObNgzXhL8dV3T+bUvpW6X6BwnUD2AFsE14G/vFcoTrZSGwkpKErsJzEACinKtjd3 xlfONc1u/CGP0fJEtACfkBzRX70dlLeAZUGv/6wl0DIvzg9n+O7xmW5xDiWllRgmlMWkjE96/az WPegDNcztEhuF0QVl6a99jgxzydSAtSCgTzRr6HclRq4p0DuQsL3byYpeD8VyhCsihryfEDrHPe c1pcPYaIDZEgOq/Fm8goTHFMvOUpWMdEUYKmnHT6xrvYAZPQyTbTZzmFS6nFMMuZUkVcs6Npr/B OYq4sRVZ8GwUUjquhxz4YtCQFzzu5WWotMqNScNtZFohJWPbnYen77Eg= X-Received: by 2002:a05:6214:2245:b0:8b9:428a:e640 with SMTP id 6a1803df08f44-8bc41bcab79mr5615216d6.2.1778012814605; Tue, 05 May 2026 13:26:54 -0700 (PDT) X-Received: by 2002:a05:6214:2245:b0:8b9:428a:e640 with SMTP id 6a1803df08f44-8bc41bcab79mr5614276d6.2.1778012813861; Tue, 05 May 2026 13:26:53 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Juraj Marcin , Avihai Horon Subject: [PULL 08/23] migration: Use the new save_query_pending() API directly Date: Tue, 5 May 2026 16:26:25 -0400 Message-ID: <20260505202640.1011006-9-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012937338154100 Content-Type: text/plain; charset="utf-8" It's easier to use the new API directly in the migration iterations. This also paves way for follow up patches to add new data to report directly to the iterator function. When at it, merge the tracepoints too into one. No functional change intended. Reviewed-by: Juraj Marcin Reviewed-by: Avihai Horon Link: https://lore.kernel.org/r/20260421202110.306051-7-peterx@redhat.com Signed-off-by: Peter Xu --- migration/savevm.h | 4 ---- migration/migration.c | 16 +++++++--------- migration/savevm.c | 23 ++--------------------- migration/trace-events | 3 +-- 4 files changed, 10 insertions(+), 36 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index e4efd243f3..96fdf96d4e 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -46,10 +46,6 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s); void qemu_savevm_query_pending(MigPendingData *pending, bool exact); -void qemu_savevm_state_pending_exact(uint64_t *must_precopy, - uint64_t *can_postcopy); -void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, - uint64_t *can_postcopy); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); void qemu_savevm_state_end(QEMUFile *f); diff --git a/migration/migration.c b/migration/migration.c index 5f4efb1fe5..c75ad01b64 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3208,17 +3208,17 @@ typedef enum { */ static MigIterateState migration_iteration_run(MigrationState *s) { - uint64_t must_precopy, can_postcopy, pending_size; Error *local_err =3D NULL; bool in_postcopy =3D (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE= || s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); bool can_switchover =3D migration_can_switchover(s); + MigPendingData pending =3D { }; + uint64_t pending_size; bool complete_ready; =20 /* Fast path - get the estimated amount of pending data */ - qemu_savevm_state_pending_estimate(&must_precopy, &can_postcopy); - pending_size =3D must_precopy + can_postcopy; - trace_migrate_pending_estimate(pending_size, must_precopy, can_postcop= y); + qemu_savevm_query_pending(&pending, false); + pending_size =3D pending.precopy_bytes + pending.postcopy_bytes; =20 if (in_postcopy) { /* @@ -3259,14 +3259,12 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) * during postcopy phase. */ if (pending_size <=3D s->threshold_size) { - qemu_savevm_state_pending_exact(&must_precopy, &can_postcopy); - pending_size =3D must_precopy + can_postcopy; - trace_migrate_pending_exact(pending_size, must_precopy, - can_postcopy); + qemu_savevm_query_pending(&pending, true); + pending_size =3D pending.precopy_bytes + pending.postcopy_byte= s; } =20 /* Should we switch to postcopy now? */ - if (must_precopy <=3D s->threshold_size && + if (pending.precopy_bytes <=3D s->threshold_size && can_switchover && qatomic_read(&s->start_postcopy)) { if (postcopy_start(s, &local_err)) { migrate_error_propagate(s, error_copy(local_err)); diff --git a/migration/savevm.c b/migration/savevm.c index 41f1906598..72454e15ad 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1812,28 +1812,9 @@ void qemu_savevm_query_pending(MigPendingData *pendi= ng, bool exact) } se->ops->save_query_pending(se->opaque, pending, exact); } -} - -void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, - uint64_t *can_postcopy) -{ - MigPendingData pending; - - qemu_savevm_query_pending(&pending, false); - - *must_precopy =3D pending.precopy_bytes; - *can_postcopy =3D pending.postcopy_bytes; -} - -void qemu_savevm_state_pending_exact(uint64_t *must_precopy, - uint64_t *can_postcopy) -{ - MigPendingData pending; - - qemu_savevm_query_pending(&pending, true); =20 - *must_precopy =3D pending.precopy_bytes; - *can_postcopy =3D pending.postcopy_bytes; + trace_qemu_savevm_query_pending(exact, pending->precopy_bytes, + pending->postcopy_bytes); } =20 void qemu_savevm_state_cleanup(void) diff --git a/migration/trace-events b/migration/trace-events index 34143b14b4..ca7dfd4cb7 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -7,6 +7,7 @@ qemu_loadvm_state_section_partend(uint32_t section_id) "%u" qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" +qemu_savevm_query_pending(bool exact, uint64_t precopy, uint64_t postcopy)= "exact=3D%d, precopy=3D%"PRIu64", postcopy=3D%"PRIu64 loadvm_state_switchover_ack_needed(unsigned int switchover_ack_pending_num= ) "Switchover ack pending num=3D%u" loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" @@ -161,8 +162,6 @@ migration_cleanup(void) "" migrate_error(const char *error_desc) "error=3D%s" migration_cancel(void) "" migrate_handle_rp_req_pages(const char *rbname, size_t start, size_t len) = "in %s at 0x%zx len 0x%zx" -migrate_pending_exact(uint64_t size, uint64_t pre, uint64_t post) "exact p= ending size %" PRIu64 " (pre =3D %" PRIu64 " post=3D%" PRIu64 ")" -migrate_pending_estimate(uint64_t size, uint64_t pre, uint64_t post) "esti= mate pending size %" PRIu64 " (pre =3D %" PRIu64 " post=3D%" PRIu64 ")" migrate_send_rp_message(int msg_type, uint16_t len) "%d: len %d" migrate_send_rp_recv_bitmap(char *name, int64_t size) "block '%s' size 0x%= "PRIi64 migration_completion_file_err(void) "" --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012871; cv=none; d=zohomail.com; s=zohoarc; b=PATPWKVbGF47QThU2JsxhwRJpuVSwPtblnaV1b8kHA9USx1+rRE8XgOGrU7BYzNAhK4i5jiwFnUp1KYfOGCzH8IiT7UfuNppN4yNI6Wal18IU62ximOeH10BVZ5kFDkQpVbEZiNrQmCTIfaskgXI0XKMfALO59eLZFnjZ8jbX2w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012871; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=d1g/dHZLkAZyHMwXIVaTr59DbBEgq11pSkULpZNk7i8=; b=UF5reNuzBD5RDnK7gM1xxLChJ7VvpM0g1xzS361xaEe8jYUrDgHC9AlM96N3PvOlqTpmR5M0NUmk82Z42LFXZPaNa3eBMS+x47fUs4Xx6hyUfBcpiwVGZ1KqUprDLydmKToJ0vyM3OYqJdtEtovaVHtrxs6pcDzk0qJh3B4YJ3c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012871749782.9561164013091; Tue, 5 May 2026 13:27:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMJ-0004VZ-8w; Tue, 05 May 2026 16:27:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM9-0004RX-LZ for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM7-0002cZ-Gn for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:01 -0400 Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-58-3raALB0HMJ-LVSoB2fnW9A-1; Tue, 05 May 2026 16:26:56 -0400 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-8b1f4478ca7so154658396d6.3 for ; Tue, 05 May 2026 13:26:56 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d1g/dHZLkAZyHMwXIVaTr59DbBEgq11pSkULpZNk7i8=; b=fpKLovHGYdv7czmQJGZ7oAVWZilJS16Dt8quR94LjmnkGCwzQMw74BxBlI0iPqpPe4+ZAJ 6p4GvSITHMoye54/4Al5BocDNejJ8s49N9wqpoz6niQ/JSJcdPl+HoaYqN6H/GplKqUeFE yNvJ1NDW3rxQxbOJF7ZapgRsn89hSJ4= X-MC-Unique: 3raALB0HMJ-LVSoB2fnW9A-1 X-Mimecast-MFC-AGG-ID: 3raALB0HMJ-LVSoB2fnW9A_1778012816 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012816; x=1778617616; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d1g/dHZLkAZyHMwXIVaTr59DbBEgq11pSkULpZNk7i8=; b=IQ17ETQ48D2MpgiQlu008B5abfz62KjGhuqtUsyWInWCxIsjPEwPGByQABxS7zm8xt 27h1I29xkZ8h4wOfWCpmo6H0H/UZ1ReiHKhZ6VX8QfjhYjAP/ZMpKQrmzK8/3JKbPHT6 C88Dyp1ANWdxUtiP/TQ1kHPByr946CCAAUO4Phrar8K+lLqkHN9rb//Qlf5wjEFY4J6u y47ishP/q1hZ7oiwgEQFJ8Oe9gpPzm/A06yTbxq5bnsgrYtHJ8W6lmRYEShnqO/L9weW NsBqg3LaObIgxIcgtsLydoNTf7ekU7IKJbbfkzvqLqlTKYnNfM7Q8abzZL+ZB2vQ6OKd 2mRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012816; x=1778617616; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=d1g/dHZLkAZyHMwXIVaTr59DbBEgq11pSkULpZNk7i8=; b=j4waB996sPLSFMt1rAFMhFpcA3iYlB+poNZ4rlbMF6MFvstaQp4B+nLnms4BUKU3jL oRPErsqOtD/lHvLC9sMn0ZDW9qrdr8jEEahBpsa4bWLpH4Hxhqj2dZkYu5ycu1B00IEg rzBz+Xc5jLvNKe/SX3hjMXHgRVXHOfLWS2Ptln3lFn0nYvKivweTuMqCgCaLmZ9HD0py xZqKMSzFUpXWvl2PSgUOELNEx2doFihdSRBbuobyl399VaQtbMlV+tF2cUk9ceMUKueN GsMLc60wsVod+tWUCxX9awyxi767K+BRaupOQqMjVK2iygsT0AgHUq2kkJtdPiPtjkKo vxbQ== X-Gm-Message-State: AOJu0Yy3fhBNqAOlNPoQfZJoKB2RyvVWpSlsIPqX++bxP2gt8LirLdGv kNKidnLHb2FuJ4QtrY8tqOdx5kvRKwg3AW9anZyPUdPJXH+M5iXZs5GCzk5duJBrpgUbtQQtdcH B/5nwQ7TT3udzMQhaQPq8IWwNuNcwYdl7XlZG3cP6WqjpmH/HSB1/6dSsnZ9PUjOZzntNDkVx2T Yqm+MNFGx0FG9X8AjJ5uVG0QCFxfW373GJtbxHjw== X-Gm-Gg: AeBDievZhCzq78OJMvHq5SKEioVpcs3GCwM+doiMz0ZmlFF+FPyBDOY1GjJRhch0EnC wd0Xa1/DDaIqqYQJJPWDqRd5p5m9zI+JBZsYyFg5LDTBvuOpCAp4TeWLzBJCv3tDtdeaGhxMrYR qWuatOy63qxSsq0WNGWi4fV2M1tn8aoAA5gMPOXM0aA+iazGGTNR8InftQ4O8XmDMLOJfI0ukwO wi8HxelQTH8Vwx70S9b81/EFrUtv2mQyp+F7H+TJoxuNgmyKLVfMekfd9y2xjO8Wu9PL/0XGmA5 wzqYx67mRqMNanEN7ZlF2+ZW720rslHUJlABbh9BH+UdX+qnzVUZV8IDatXLRlxIbDO29Zu+Kbr a5vuWIyhBqZF5Q6hHzdj87FfVf161JeB6YbqrZ7LXArzfeBDbBK++DrM= X-Received: by 2002:a05:6214:5a06:b0:8a5:e0a7:d9a7 with SMTP id 6a1803df08f44-8bc42784419mr5048366d6.9.1778012815675; Tue, 05 May 2026 13:26:55 -0700 (PDT) X-Received: by 2002:a05:6214:5a06:b0:8a5:e0a7:d9a7 with SMTP id 6a1803df08f44-8bc42784419mr5047506d6.9.1778012814948; Tue, 05 May 2026 13:26:54 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Avihai Horon , Juraj Marcin Subject: [PULL 09/23] migration: Introduce stopcopy_bytes in save_query_pending() Date: Tue, 5 May 2026 16:26:26 -0400 Message-ID: <20260505202640.1011006-10-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012872312158500 Content-Type: text/plain; charset="utf-8" Allow modules to report data that can only be migrated after VM is stopped. When this concept is introduced, we will need to account stopcopy size to be part of pending_size as before. However, when there're data only can be migrated in stopcopy phase, it means the old "pending_size" may not always be able to reach low enough to kickoff an slow version of query sync. It used to be almost guaranteed to happen as all prior iterative modules doesn't have stopcopy only data. VFIO may change that fact by having some data that must be copied during stop phase. So we need to make sure QEMU will kickoff a synchronized version of query pending when all precopy data is migrated. This might be important to VFIO to keep making progress even if the downtime cannot yet be satisfied. So far, this patch should introduce no functional change, as no module yet report stopcopy size. This paves way for VFIO to properly report its pending data sizes, which will start to include stop-only data. Reviewed-by: Avihai Horon Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20260421202110.306051-8-peterx@redhat.com Signed-off-by: Peter Xu --- include/migration/register.h | 7 ++++ migration/migration.c | 65 ++++++++++++++++++++++++++++++------ migration/savevm.c | 10 ++++-- migration/trace-events | 2 +- 4 files changed, 70 insertions(+), 14 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index e2117e8dd4..5e5e0ee432 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -21,6 +21,13 @@ typedef struct MigPendingData { uint64_t precopy_bytes; /* Amount of pending bytes can be transferred in postcopy */ uint64_t postcopy_bytes; + /* Amount of pending bytes can be transferred only in stopcopy */ + uint64_t stopcopy_bytes; + /* + * Total pending data, modules do not need to update this field, it + * will be automatically calculated by migration core API. + */ + uint64_t total_bytes; } MigPendingData; =20 /** diff --git a/migration/migration.c b/migration/migration.c index c75ad01b64..049b69fbe7 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3202,6 +3202,54 @@ typedef enum { MIG_ITERATE_BREAK, /* Break the loop */ } MigIterateState; =20 +/* Are we ready to move to the next iteration phase? */ +static bool migration_iteration_next_ready(MigrationState *s, + MigPendingData *pending) +{ + /* + * If the estimated values already suggest us to switchover, mark this + * iteration finished, time to do a slow sync. + */ + if (pending->total_bytes <=3D s->threshold_size) { + return true; + } + + /* + * Since we may have modules reporting stop-only data, we also want to + * re-query with slow mode if all precopy data is moved over. This + * will also mark the current iteration done. + * + * This could happen when e.g. a module (like, VFIO) reports stopcopy + * size too large so it will never yet satisfy the downtime with the + * current setup (above check). Here, slow version of re-query helps + * because we keep trying the best to move whatever we have. + */ + if (pending->precopy_bytes =3D=3D 0) { + return true; + } + + return false; +} + +static void migration_iteration_go_next(MigPendingData *pending) +{ + /* + * Do a slow sync will achieve this. TODO: move RAM iteration code + * into the core layer. + */ + qemu_savevm_query_pending(pending, true); +} + +static bool postcopy_should_start(MigrationState *s, MigPendingData *pendi= ng) +{ + /* If postcopy's switchver will violate user specified downtime, stop = */ + if (pending->precopy_bytes + pending->stopcopy_bytes > s->threshold_si= ze) { + return false; + } + + return qatomic_read(&s->start_postcopy); +} + /* * Return true if continue to the next iteration directly, false * otherwise. @@ -3213,12 +3261,10 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); bool can_switchover =3D migration_can_switchover(s); MigPendingData pending =3D { }; - uint64_t pending_size; bool complete_ready; =20 /* Fast path - get the estimated amount of pending data */ qemu_savevm_query_pending(&pending, false); - pending_size =3D pending.precopy_bytes + pending.postcopy_bytes; =20 if (in_postcopy) { /* @@ -3226,7 +3272,7 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) * postcopy completion doesn't rely on can_switchover, because when * POSTCOPY_ACTIVE it means switchover already happened. */ - complete_ready =3D !pending_size; + complete_ready =3D !pending.total_bytes; if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE && (s->postcopy_package_loaded || complete_ready)) { /* @@ -3258,14 +3304,12 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) * postcopy started, so ESTIMATE should always match with EXACT * during postcopy phase. */ - if (pending_size <=3D s->threshold_size) { - qemu_savevm_query_pending(&pending, true); - pending_size =3D pending.precopy_bytes + pending.postcopy_byte= s; + if (migration_iteration_next_ready(s, &pending)) { + migration_iteration_go_next(&pending); } =20 /* Should we switch to postcopy now? */ - if (pending.precopy_bytes <=3D s->threshold_size && - can_switchover && qatomic_read(&s->start_postcopy)) { + if (can_switchover && postcopy_should_start(s, &pending)) { if (postcopy_start(s, &local_err)) { migrate_error_propagate(s, error_copy(local_err)); error_report_err(local_err); @@ -3280,11 +3324,12 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) * (2) Pending size is no more than the threshold specified * (which was calculated from expected downtime) */ - complete_ready =3D can_switchover && (pending_size <=3D s->thresho= ld_size); + complete_ready =3D can_switchover && + (pending.total_bytes <=3D s->threshold_size); } =20 if (complete_ready) { - trace_migration_thread_low_pending(pending_size); + trace_migration_thread_low_pending(pending.total_bytes); migration_completion(s); return MIG_ITERATE_BREAK; } diff --git a/migration/savevm.c b/migration/savevm.c index 72454e15ad..39430470aa 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1800,8 +1800,7 @@ void qemu_savevm_query_pending(MigPendingData *pendin= g, bool exact) { SaveStateEntry *se; =20 - pending->precopy_bytes =3D 0; - pending->postcopy_bytes =3D 0; + memset(pending, 0, sizeof(*pending)); =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { if (!se->ops || !se->ops->save_query_pending) { @@ -1813,8 +1812,13 @@ void qemu_savevm_query_pending(MigPendingData *pendi= ng, bool exact) se->ops->save_query_pending(se->opaque, pending, exact); } =20 + pending->total_bytes =3D pending->precopy_bytes + + pending->stopcopy_bytes + pending->postcopy_bytes; + trace_qemu_savevm_query_pending(exact, pending->precopy_bytes, - pending->postcopy_bytes); + pending->stopcopy_bytes, + pending->postcopy_bytes, + pending->total_bytes); } =20 void qemu_savevm_state_cleanup(void) diff --git a/migration/trace-events b/migration/trace-events index ca7dfd4cb7..de99d976ab 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -7,7 +7,7 @@ qemu_loadvm_state_section_partend(uint32_t section_id) "%u" qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" -qemu_savevm_query_pending(bool exact, uint64_t precopy, uint64_t postcopy)= "exact=3D%d, precopy=3D%"PRIu64", postcopy=3D%"PRIu64 +qemu_savevm_query_pending(bool exact, uint64_t precopy, uint64_t stopcopy,= uint64_t postcopy, uint64_t total) "exact=3D%d, precopy=3D%"PRIu64", stopc= opy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PRIu64 loadvm_state_switchover_ack_needed(unsigned int switchover_ack_pending_num= ) "Switchover ack pending num=3D%u" loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012885; cv=none; d=zohomail.com; s=zohoarc; b=KDgdJx5tfwk6hj/jD/gYwaAtk0gpTX/pR0RUjvGj7496RTpinOgxwuYU04cA0CzQZuTxOtVWhRhPBZgZD13po5LBPPEJogsVKNxOLlBCp7iDHkbpURhXi1x6cpkLwF3PdoHH8mI4HcIkWShpKg/u5ZSYgiPLDhRupFwgyAt22Hg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012885; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=OGmcZTJdrvJChcgQdhUlKR40+p3UPXTZRXwTZoQgmPs=; b=SISFT57zwmp0IW8GlO5H6kr1lT2KGed7AbG8j0Rj1wnYGh4Lj8VPxaRSI77anbQrDY5XRNyxaxfocvkatyilNrj+ljMt3CPUBkb/JK0MkEpk4Aq3Hyk/0r62Va5ljgwWxq4qvf0PuGj9F2jgHooM/UZUTaGFblKRsVJITycs840= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012885690695.658821602799; Tue, 5 May 2026 13:28:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMJ-0004Vb-9D; Tue, 05 May 2026 16:27:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMB-0004SZ-Li for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM7-0002cf-Gq for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:02 -0400 Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-324-kf9f9osJONuADIKTkVENSQ-1; Tue, 05 May 2026 16:26:57 -0400 Received: by mail-qk1-f197.google.com with SMTP id af79cd13be357-902e1e4f8ccso340965185a.0 for ; Tue, 05 May 2026 13:26:57 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OGmcZTJdrvJChcgQdhUlKR40+p3UPXTZRXwTZoQgmPs=; b=cja1Ifg3KvzSIJwQHOyCLzNKTxvSdJjdnLrJsTqlLlGqbKgVeLc8+uUKBgJKKptsAZ2gYL nU9IErTzoJDOGfbu1w7t0dL5yHVLR24gTiDuD5m84T0u2WE/O1DVLVqPmpUNHsTlPRnIWQ OX+QVqsdLEAEAxTMI6pMcv36Cy+QEh4= X-MC-Unique: kf9f9osJONuADIKTkVENSQ-1 X-Mimecast-MFC-AGG-ID: kf9f9osJONuADIKTkVENSQ_1778012817 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012817; x=1778617617; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OGmcZTJdrvJChcgQdhUlKR40+p3UPXTZRXwTZoQgmPs=; b=QWcQMGlGm9tTLfnVrlIQ8gUMgWOEgoF9k2RDTVoLzbyRVYyfiA2nuUp3TK6LJ5P5y7 T9NUcZEi51Ybi/KWvU7j6yJ+e/UHNu8Su/Mj7dYjCVdupH+KERBQs6BPTmINs4x5GdK2 iIOCNE8AGNJa7xsB0VsKk8gJ5SP/hWZrUZaAq/M/RLCReF8sZCIsG9aJJkWK/12ztesv NiUGaulHgPLhrwP2DhWoUuoPJxN+ezps8hpTDqmaCLkg3IqX4U0RTLuOjNNFPemTidZs 0dw3jJrzxXFaNX+cmPgZoaMgvA3ns6ghqkbarWXJyUBDgms6KJBNK05iZj/izq2qfnlt meWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012817; x=1778617617; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=OGmcZTJdrvJChcgQdhUlKR40+p3UPXTZRXwTZoQgmPs=; b=CHG9PheVBHN2DZccpP6xKWuXqCg4NyxChachDozhbPpY0Dfo0dLxM5DHojsQJ1aBza 82B3c0uyNgKzcn/dZ8SVSmmQCjeIEGPiI25dhUZkSo/tmD0Bdh9GvVWFr32oSzstafSR nemGWpbGB+6XrOOnZykoRpdi4qGDpBZivLWvMV4M47+OCFFJzR9JW8WFoA1ZeRbajiIg tREq31IZbOLXdu703G8mZ/k6nwEcrS/83y9n9+oPQLmphwqF3TEpw4BSU1wsq3KVpuoq fOL54aCnolagQGZNxEFB7ZbxHuWc+oLmSF8vWs3CRsguXOQcBdrEshI60qqmBvuBiibw xv+g== X-Gm-Message-State: AOJu0Ywy010U7Yvvmew4i07fZ2PL07YUAztqK0+eCG/aExTf5sFu9TcD vmGsmxwiXjfY13c5xldsMFZu6qkkKPQMwJ+o62akW+GYchXqhxBy58JgC0W6V1Dn80TAZqROPRo xDxTZzGHedA0iWTuZhkuS+hHP1aOeKdFT8G4W7uJgF/YKZdU1sSLWrWkPPobdoFnDvicsa7p8yQ R3dy01XieGgpLuSlqFhIDznHU3Dvrh/AlR19UWSQ== X-Gm-Gg: AeBDiesgzJKRsRYo8hizcr4Mlit9b6CMVRKCKJ7/DUy8mDXiNn7MZM22HBP0i8TdIVO nqzdRX5vh7OkHpSaP8Qm+DaTivdpLB+bC6iZXd6ZX9Gb9NDVV2zxX0O8nUIXHDlMzrqt5IgQhL6 gJzqB6y+3XV/xRktxpccKq/v+i60+7oqMQZWHSyRSrseXCNu3mOYh2cYvdYTtg3n62YDuHzCx4R E9GAujudcXQshjEnMUyPudsGoW9qp69tkltI2tZmaE0FGB359Q4pfpqan+Q37zhhCcnRJl6LVIn jshcGQQwTFB0EQRpS5Wsf1XpkMuXkZZJ3McJyQtgefTwvjDoL4IRLEyL6VzAZMs0BSfLssJy6vq /8D9UGjXwcZYP1MACfnwMWCBj+Mdj2ApNs4zhydfmrgrCyw/CKdB7zBU= X-Received: by 2002:a05:620a:2544:b0:8e5:b6a5:8d0a with SMTP id af79cd13be357-904d74b318cmr90979685a.62.1778012816541; Tue, 05 May 2026 13:26:56 -0700 (PDT) X-Received: by 2002:a05:620a:2544:b0:8e5:b6a5:8d0a with SMTP id af79cd13be357-904d74b318cmr90973285a.62.1778012815837; Tue, 05 May 2026 13:26:55 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Avihai Horon Subject: [PULL 10/23] vfio/migration: Fix incorrect reporting for VFIO pending data Date: Tue, 5 May 2026 16:26:27 -0400 Message-ID: <20260505202640.1011006-11-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012888479154100 Content-Type: text/plain; charset="utf-8" VFIO reports different things in its fast/slow version of query pending results. It was because it wants to make sure precopy data can reach 0, which is needed to make sure sync queries will happen periodically over time. Now with stopcopy size reporting facility it doesn't need this hack anymore. Fix this by reporting the same values in fast/slow versions of query pending request, except that the slow version will do a slow sync with the hardwares. When at it, removing the special casing for vfio_device_state_is_precopy() which may reporting nothing in a fast query. Then ther reporting will be consistent to VFIO devices that do not support precopy phase. Copy stable might be too much; just skip it and skip the Fixes. Reviewed-by: Avihai Horon Tested-by: Avihai Horon Link: https://lore.kernel.org/r/20260421202110.306051-9-peterx@redhat.com Signed-off-by: Peter Xu --- hw/vfio/migration.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index e965ba51fb..e6e6a0d53d 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -587,19 +587,23 @@ static void vfio_state_pending(void *opaque, MigPendi= ngData *pending, { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; - uint64_t remain; + uint64_t precopy_size, stopcopy_size; =20 if (exact) { vfio_state_pending_sync(vbasedev); - remain =3D migration->stopcopy_size; + } + + precopy_size =3D + migration->precopy_init_size + migration->precopy_dirty_size; + + if (migration->stopcopy_size > precopy_size) { + stopcopy_size =3D migration->stopcopy_size - precopy_size; } else { - if (!vfio_device_state_is_precopy(vbasedev)) { - return; - } - remain =3D migration->precopy_init_size + migration->precopy_dirty= _size; + stopcopy_size =3D 0; } =20 - pending->precopy_bytes +=3D remain; + pending->precopy_bytes +=3D precopy_size; + pending->stopcopy_bytes +=3D stopcopy_size; =20 trace_vfio_state_pending(vbasedev->name, migration->stopcopy_size, migration->precopy_init_size, --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012992; cv=none; d=zohomail.com; s=zohoarc; b=MsvmYGTwmPLJ4MYfD8r22FwjIlY7sfhaWfJ7hIEs3PBMG9Mf2Qn96QNaC+t8TkifpqIu9ZIaUwJG743hDZnj0saCPKm84pFvh2RzyRHUII5p9DigQYyBRE2VW7h4SF+eqJRvyLm3HJfyxQpqOl5Thq9nPf1FC/BhOH+k4Wgzstw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012992; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=MloABw5GE3WIOEB46lkCJz/DCq9G15tQHSXMQreAC9I=; b=EA1F2VwS2UxdFt5aPHISMc+o+mmNdTcIWLhEqfxYGbYVROo+ur22tBE2Bx2tjvrjVyhFmB/HMBMYGH7SnzbOWoRrDxoB+mKWsSi6KOAL2tD5VbkMCJsC2XwIkXAS7bBvKk0WUg9SkXfWUsLVrprdQvXVWQDNZM/UI8C3rILOhF8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012992094476.3553812902318; Tue, 5 May 2026 13:29:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMN-0004bC-2G; Tue, 05 May 2026 16:27:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMB-0004Sa-Mb for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMM8-0002co-KO for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:03 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-493-UTfb5-BHNqGE6y2Syc4JWA-1; Tue, 05 May 2026 16:26:58 -0400 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-8acaea1ffe7so142717196d6.2 for ; Tue, 05 May 2026 13:26:58 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012819; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MloABw5GE3WIOEB46lkCJz/DCq9G15tQHSXMQreAC9I=; b=W2OJLdqCd1+8gEaUf3X59eumjz3vK8+ZFgvaravse+lwtQGw32+4K8fKubStQXIWJOfiXr l+gsCizO1GQDbOCcqTiwi3d3VaZQmWfMh+nRiYWwQcftc2gJI91XjWE8oYcTN55Eu/vtiU 0lhpnGYN9pp9zHmaKjebLkOU1xJ2pHc= X-MC-Unique: UTfb5-BHNqGE6y2Syc4JWA-1 X-Mimecast-MFC-AGG-ID: UTfb5-BHNqGE6y2Syc4JWA_1778012818 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012818; x=1778617618; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MloABw5GE3WIOEB46lkCJz/DCq9G15tQHSXMQreAC9I=; b=iFQtbsIv3pLopwKjSYa1ri9MA9S8wfvFDe5Vjd2RGHxLFFtnuJ5KilyRN6td5yvWoT SEhessTL6fRDEZfjVHc1mpLPa2YBPL8FXVMnxFAdTu4ngYX3/MNWkeSsn0dykwc1Kd74 QXnuq11UIQe9vfLe3uzAQnFn/t0q1EIxkjQ7GkScoCh38THOxzQ/BcV3mUZ5zjpxDMaV xMP12P3kotfAb/bmk/t2Sq/zSwwwN2j8iA8z5u6ncNesTNX5gjrmwIJ463G1/d9GqVxE EYjSn75j77y4t6OmNyDQRR5WBxLtBSyvmRjetn4XWyDNp02fl4sm36HUX419AtYWrqUG VIiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012818; x=1778617618; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=MloABw5GE3WIOEB46lkCJz/DCq9G15tQHSXMQreAC9I=; b=CrHMeYbpKU21t6hFAa8W+SzHlCtbnbjJqGbOE2cyCa5zdpvC9pNVCjbIMBTPIVzqUD xstzZFO8XwojiAjM/KNsNrnCeJox5Smo1S6XVeKps4oqBKRQnVOJ4363K1ReDBV/a6la XxklWY2rhnZO5Z2oOfGsOIWAUkc4VMzC18YbK5ydMJCKNIumwWnF7NVe4dnJc0MDlvSz eR9pu7w4ZNxUUim+LjZasN1Qhpyoc3nui1fXU5rzAabpS+PuRsOTbg1J0MFvUyHlxHQb gV6XRdEFJZUD8fhJ+vOoxGuzGtY3eTZg6dsJ2hbkPJOaRQCKXZpkVzgI4P8K7NsaAj09 qIdQ== X-Gm-Message-State: AOJu0YyBtG1bes8j5IFUTWGeA0Y5QSHwJ02W/xWXJXA8PL2qAglemh6A Zi+I6pQEQVvH1pvwPDAsK+EmgJb+gL1aq/06vIDP/rgt7Ikv1Fqj94iU0Yt0sTQGCa62jNv4ryC 1aqUPdWO9/NWRKpea66jPBf/6xCn0XkZ+4W2R17q9QtGAQ+XG5VGRfDV9MVnSy/OMwtGR6HGVX0 cUldua0u9aAejemk/5mh0fBBPk5lIwY6Krs01Z4Q== X-Gm-Gg: AeBDievyq8iAG4oMsT//1pjYNsZ4MZlEmmz4tVn4qokqjGoyD5WvYbaikcesNeenqTR /DIHJawzHzPWmPSdDi9yN5HiOcyskMVK1w20dcZR2FLCsLZJF57ZtWiVxsaJdsPd+ijcatRAxJn Y6VSLcw0buKf8yYOovU61y701YuECO+bz1QY9jbLWTIXSzOmG4vJWhPPy48NiZlt96h2C//vP5E 8eTOI/n/kK2xVn2BAvvS6OMZg3/uCWOb4Fy18cKJCNgW+QGUtZE5OlY8HiJMp9olVURWPChqsz/ 9c2JQ8wc9ZNlwtxhK4h8WLVwT6Ofj9kXAR7Giwq4sn0sMIr/eX6L6ep1tECP1McCTizTi+hgPBT z5tR0IgArUgrHCZwECrcubchAIvyxySB6xQHBW6Agg9yL01L3/U7or3Q= X-Received: by 2002:a0c:e089:0:b0:8ac:acd6:c988 with SMTP id 6a1803df08f44-8bc422a67f3mr5318746d6.1.1778012817573; Tue, 05 May 2026 13:26:57 -0700 (PDT) X-Received: by 2002:a0c:e089:0:b0:8ac:acd6:c988 with SMTP id 6a1803df08f44-8bc422a67f3mr5318116d6.1.1778012816962; Tue, 05 May 2026 13:26:56 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Hyman Huang , Prasad Pandit , Juraj Marcin Subject: [PULL 11/23] migration: Move iteration counter out of RAM Date: Tue, 5 May 2026 16:26:28 -0400 Message-ID: <20260505202640.1011006-12-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012992879158500 Content-Type: text/plain; charset="utf-8" It used to hide in RAM dirty sync path. Now with more modules being able to slow sync on dirty information, keeping it there may not be good anymore because it's not RAM's own concept for iterations: all modules should follow. More importantly, mgmt may try to query dirty info (to make policy decisions like adjusting downtime) by listening to iteration count changes via QMP events. So we must make sure the boost of iterations only happens _after_ the dirty sync operations with whatever form (RAM's dirty bitmap sync, or VFIO's different ioctls to fetch latest dirty info from kernel). Move this to core migration path to manage, together with the event generation, so that it can be well ordered with the sync operations for all modules. This brings a good side effect that we should have an old issue regarding to cpu_throttle_dirty_sync_timer_tick() which can randomly boost iteration counts (because it invokes sync ops). Now it won't, which is actually the right behavior. Said that, we have code (not only QEMU, but likely mgmt too) assuming the 1st iteration will always shows dirty count to 1. Make it initialized with 1 this time, because we'll miss the dirty sync for setup() on boosting this counter now. Reviewed-by: Hyman Huang Reviewed-by: Prasad Pandit Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20260421202110.306051-10-peterx@redhat.com Signed-off-by: Peter Xu --- migration/migration-stats.h | 3 ++- migration/migration.c | 29 ++++++++++++++++++++++++++--- migration/ram.c | 6 ------ 3 files changed, 28 insertions(+), 10 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 1153520f7a..326ddb0088 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -43,7 +43,8 @@ typedef struct { */ uint64_t dirty_pages_rate; /* - * Number of times we have synchronized guest bitmaps. + * Number of times we have synchronized guest bitmaps. This always + * starts from 1 for the 1st iteration. */ uint64_t dirty_sync_count; /* diff --git a/migration/migration.c b/migration/migration.c index 049b69fbe7..8abc7e0327 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1654,10 +1654,15 @@ int migrate_init(MigrationState *s, Error **errp) s->threshold_size =3D 0; s->switchover_acked =3D false; s->rdma_migration =3D false; + /* - * set mig_stats memory to zero for a new migration + * set mig_stats memory to zero for a new migration.. except the + * iteration counter, which we want to make sure it returns 1 for the + * first iteration. */ memset(&mig_stats, 0, sizeof(mig_stats)); + mig_stats.dirty_sync_count =3D 1; + migration_reset_vfio_bytes_transferred(); =20 s->postcopy_package_loaded =3D false; @@ -3234,10 +3239,28 @@ static bool migration_iteration_next_ready(Migratio= nState *s, static void migration_iteration_go_next(MigPendingData *pending) { /* - * Do a slow sync will achieve this. TODO: move RAM iteration code - * into the core layer. + * Do a slow sync first before boosting the iteration count. */ qemu_savevm_query_pending(pending, true); + + /* + * Boost dirty sync count to reflect we finished one iteration. + * + * NOTE: we need to make sure when this happens (together with the + * event sent below) all modules have slow-synced the pending data + * above. That means a write mem barrier, but qatomic_add() should be + * enough. + * + * It's because a mgmt could wait on the iteration event to query again + * on pending data for policy changes (e.g. downtime adjustments). The + * ordering will make sure the query will fetch the latest results from + * all the modules. + */ + qatomic_add(&mig_stats.dirty_sync_count, 1); + + if (migrate_events()) { + qapi_event_send_migration_pass(mig_stats.dirty_sync_count); + } } =20 static bool postcopy_should_start(MigrationState *s, MigPendingData *pendi= ng) diff --git a/migration/ram.c b/migration/ram.c index 44503bf3f7..ecd4b6165c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1136,8 +1136,6 @@ static void migration_bitmap_sync(RAMState *rs, bool = last_stage) RAMBlock *block; int64_t end_time; =20 - qatomic_add(&mig_stats.dirty_sync_count, 1); - if (!rs->time_last_bitmap_sync) { rs->time_last_bitmap_sync =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIM= E); } @@ -1172,10 +1170,6 @@ static void migration_bitmap_sync(RAMState *rs, bool= last_stage) rs->num_dirty_pages_period =3D 0; rs->bytes_xfer_prev =3D migration_transferred_bytes(); } - if (migrate_events()) { - uint64_t generation =3D qatomic_read(&mig_stats.dirty_sync_count); - qapi_event_send_migration_pass(generation); - } } =20 void migration_bitmap_sync_precopy(bool last_stage) --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012991; cv=none; d=zohomail.com; s=zohoarc; b=jMq9wFj4ctNd9U26REzSxP3SFdv1i6StNmexzOT5I0LqUQ0bu28aXEhpiz7TZEUxJFtJICMiwjHq1PYv/VLC1daKjC8EhJLbYEKSRj1Z0a2aoEQ9G0VC7KvcHD6mmfL5XLdVx1iJkEIbzWgsDqHKZNREAlHNQSdXSrgklN50LZM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012991; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=c7J+3/zQgM5oGZ4jYYw3SGbSGgqkUYRSROV8mmjeY+Y=; b=gVIgVl9ToCvovpJQ65woWyyrtcJp+u1p8y9WsdU0XqXjTTHGq+5i0ef3TfevGKu6wwrPMNl8Qb/QjLrqJsyxFvWybFCIFJjXF3V/pu43lk05TtyxVU0C7oXQZXcomG0vJBVyZDcmYZo6VLEHadQyjcvC27AGw8tlPRQ97lujaK4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012991389666.6494118500716; Tue, 5 May 2026 13:29:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMN-0004br-JQ; Tue, 05 May 2026 16:27:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMD-0004Te-Hl for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMB-0002cx-El for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:05 -0400 Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-300-Z3rVoT83MVOe8zBejjIHgg-1; Tue, 05 May 2026 16:27:00 -0400 Received: by mail-qv1-f72.google.com with SMTP id 6a1803df08f44-8b7e68098dcso69803716d6.3 for ; Tue, 05 May 2026 13:27:00 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012821; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c7J+3/zQgM5oGZ4jYYw3SGbSGgqkUYRSROV8mmjeY+Y=; b=IRbRDovxjg16SX11ozbx72hRhrBsAEj48SqXKWCKNhrLSau2JsaNG0kV6pET/7UvsSIFiL rjvozN4cWPEl/4IcnKBFxvyEaTxDyl7a9jce1ocgRD8325eOc6rV5voc3XSxsi2L9cSVm4 MJER4GIVbckz3mcvoXTVtAMrk/T/9D0= X-MC-Unique: Z3rVoT83MVOe8zBejjIHgg-1 X-Mimecast-MFC-AGG-ID: Z3rVoT83MVOe8zBejjIHgg_1778012819 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012819; x=1778617619; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c7J+3/zQgM5oGZ4jYYw3SGbSGgqkUYRSROV8mmjeY+Y=; b=fWZlqEiIPFsfRPJSy0kZZZnNKm+JLtuqhSoXZxEwrpEGmMhZaxvL9zt7ZI7nN1FZow PU4OY6KSSsXIt4DsBE3un7NEu0JqtzM4yCZUKzr978EV3BXn5WPf47prcomsIbu7+dEw lsNwytI6qudamJc4W3AH3608kxWIuuIGW1mSst1WlbOvMD3OP2KhklPI4T4z3WnuZvL7 kab0zhoefA2m7sYE9DGwkIqMyfTHG2k/bI9//0iIBU280TdbeB4agP6WInlM3NGRPugK gDrpgIn0cBrc+6yZMGSu3henpcUQIyQYomFUzRVpuDG99KHwRJ+bYgZOFP3o87Ezp0KN 8U1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012819; x=1778617619; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=c7J+3/zQgM5oGZ4jYYw3SGbSGgqkUYRSROV8mmjeY+Y=; b=gb5sBKwkwMjheVIA85Y45PAtDbqLuTEvxX9uqQTn9IuFD7vWIFck2rSJHLBdwwTSAn 6R2OD5EgWN6VBwbneZzZgPB0SY77FP1sDzw5tfLeTkV3BC6Io4XkmRudUvl019+Q1ie5 aNgXcte45ujsZP2qObmndLKbWvNFZaej/g5mXu0+KW2LnoEqD0VTx5ZYn5GS3nv2EOgd GP/U2DSVouIfQbZRmgKZPPWeW6UME0gC8lfCT6kwfzbmoOhu3ZRXFLOIWO7QoTquHUWU rBlD0DCyF9aOwwR7hejS5JG9FjqdHbl1QkXib7dKRuYUv77j1+vXw7OEnozo3zN43d0c YKvQ== X-Gm-Message-State: AOJu0Yzx+6ko6+Qd82dl+tsIvxPDsbZSgxp1as1aGAR6yIfJaCxyvZeK +aj9a9kL+jQhQJKljv2TjDGspj51mKcSTz4njK6jOPCVsZieKunZOfv6KVvBWBE/eB+eZBqhzjI 5vjUYIunrgvF2/J5qRnJmqGL5jE707uZzFvBaJBvvZ6k3eFxKdjOPdu1XMue2OnyUKK+HncnRBg ZJL7fgH67z6H0sg26fl/niIxj3P+3fNxwf0bz0jw== X-Gm-Gg: AeBDievptcF5OvarxcbtfYk2MY9szAe1ByMJirsEvXlSpouBUwmUPHJVFfJs0g/hh0O /YwZij3TTn152a1WnJed3IUrkEvS0iGHXmod8xs6aNi1VIbQu/2enxJbIOfa5m+NFZd0icle8Au 1vVJfFUOY9fHpsTBgAS9xGFH1JdHaeIR9kHlRAQgx9GPyupB0IlVR31oRWc6zns0NltKfjBzHHT BNniX2PHPmQAbgRSm9HNqq6I5wkR99iUT5IK/Sw0CiVjYgDrkPnC85c4/1Q3alrGJKasroEP07t 7eRje5l7COwokN8EpB+fV6GCRhOLVImLQ79ptTyMkCSD3ukebJkITYnQ6LekWDEZqvFIO5qAs3h vAqwtoc6AmXzTuewdSXk6gwDT7c0QG4Ajx5JLjYqzMZEnfcXBNVAXJpc= X-Received: by 2002:a05:6214:763:b0:8b4:a0be:4b50 with SMTP id 6a1803df08f44-8bc4517a7f1mr4398636d6.33.1778012819059; Tue, 05 May 2026 13:26:59 -0700 (PDT) X-Received: by 2002:a05:6214:763:b0:8b4:a0be:4b50 with SMTP id 6a1803df08f44-8bc4517a7f1mr4397866d6.33.1778012818351; Tue, 05 May 2026 13:26:58 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Juraj Marcin Subject: [PULL 12/23] migration: Introduce a helper to return switchover bw estimate Date: Tue, 5 May 2026 16:26:29 -0400 Message-ID: <20260505202640.1011006-13-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012992773158500 Content-Type: text/plain; charset="utf-8" Add a helper migration_get_switchover_bw() to return an estimate of switchover bandwidth. Use it to simplify the current code. This will be used in later to remove expected_downtime. When at it, remove two qatomic_read() to shrink the lines because atomic ops are not needed when it's always the same thread who does the updates. Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20260421202110.306051-11-peterx@redhat.com Signed-off-by: Peter Xu --- migration/migration.c | 48 +++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 8abc7e0327..4e19fe3409 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -984,6 +984,21 @@ void migrate_send_rp_resume_ack(MigrationIncomingState= *mis, uint32_t value) migrate_send_rp_message(mis, MIG_RP_MSG_RESUME_ACK, sizeof(buf), &buf); } =20 +/* + * Returns the estimated switchover bandwidth (unit: bytes / seconds) + */ +static double migration_get_switchover_bw(MigrationState *s) +{ + uint64_t switchover_bw =3D migrate_avail_switchover_bandwidth(); + + if (switchover_bw) { + /* If user specified, prioritize this value and don't estimate */ + return (double)switchover_bw; + } + + return s->mbps / 8 * 1000 * 1000; +} + bool migration_is_running(void) { MigrationState *s =3D current_migration; @@ -3130,37 +3145,22 @@ static void migration_update_counters(MigrationStat= e *s, { uint64_t transferred, transferred_pages, time_spent; uint64_t current_bytes; /* bytes transferred since the beginning */ - uint64_t switchover_bw; - /* Expected bandwidth when switching over to destination QEMU */ - double expected_bw_per_ms; - double bandwidth; + double switchover_bw_per_ms; =20 if (current_time < s->iteration_start_time + BUFFER_DELAY) { return; } =20 - switchover_bw =3D migrate_avail_switchover_bandwidth(); current_bytes =3D migration_transferred_bytes(); transferred =3D current_bytes - s->iteration_initial_bytes; time_spent =3D current_time - s->iteration_start_time; - bandwidth =3D (double)transferred / time_spent; - - if (switchover_bw) { - /* - * If the user specified a switchover bandwidth, let's trust the - * user so that can be more accurate than what we estimated. - */ - expected_bw_per_ms =3D (double)switchover_bw / 1000; - } else { - /* If the user doesn't specify bandwidth, we use the estimated */ - expected_bw_per_ms =3D bandwidth; - } - - s->threshold_size =3D expected_bw_per_ms * migrate_downtime_limit(); - s->mbps =3D (((double) transferred * 8.0) / ((double) time_spent / 1000.0)) / 1000.0 / 1000.0; =20 + /* NOTE: only update this after bandwidth (s->mbps) updated */ + switchover_bw_per_ms =3D migration_get_switchover_bw(s) / 1000; + s->threshold_size =3D switchover_bw_per_ms * migrate_downtime_limit(); + transferred_pages =3D ram_get_total_transferred_pages() - s->iteration_initial_pages; s->pages_per_second =3D (double) transferred_pages / @@ -3170,10 +3170,9 @@ static void migration_update_counters(MigrationState= *s, * if we haven't sent anything, we don't want to * recalculate. 10000 is a small enough number for our purposes */ - if (qatomic_read(&mig_stats.dirty_pages_rate) && - transferred > 10000) { + if (mig_stats.dirty_pages_rate && transferred > 10000) { s->expected_downtime =3D - qatomic_read(&mig_stats.dirty_bytes_last_sync) / expected_bw_p= er_ms; + mig_stats.dirty_bytes_last_sync / switchover_bw_per_ms; } =20 migration_rate_reset(); @@ -3182,7 +3181,8 @@ static void migration_update_counters(MigrationState = *s, =20 trace_migrate_transferred(transferred, time_spent, /* Both in unit bytes/ms */ - bandwidth, switchover_bw / 1000, + (uint64_t)s->mbps, + (uint64_t)switchover_bw_per_ms, s->threshold_size); } =20 --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012992; cv=none; d=zohomail.com; s=zohoarc; b=mYLSoeVC72ec7sDTT9aMUD1iENGnUnQy/3bb5xuFDcbATfY4VZvsDKymZnQheEc1ftwjdcs1GkoAidQmKZF09ge6L+pogNbmL5tiMmKy7Hfe20u85blvVQ/fOQYd16yexCu99jjjWu2uUYdlACgxI8SD5J7xBXbX2S9sMk0TJD8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012992; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WZf+roECEzrRMn9PvQ2Od9IyuK2k8H7PDFoPmYm9/54=; b=mylbzJm2dGVM1axaMHJNCZGTj8GUA3p3UHOSC8BC/p7iRqwvT7mdfjFR3iz47arkJfPUKPDJzowijPtzt5DST7D+GICHhF4fUv7uyHWrGjEYuP+BbZyzCkuZa/lbY6P3PxnknUl0YhJQuafJv79QboOIq26rFdyXAnH3wDEtOl8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012992425883.7286983621722; Tue, 5 May 2026 13:29:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMML-0004ZZ-R8; Tue, 05 May 2026 16:27:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMD-0004T1-2n for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMB-0002dV-FD for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:04 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-343-Z1ktgMWzMAWAtXN9U5DPBQ-1; Tue, 05 May 2026 16:27:01 -0400 Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-50da529ff48so133026111cf.3 for ; Tue, 05 May 2026 13:27:01 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:26:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012822; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WZf+roECEzrRMn9PvQ2Od9IyuK2k8H7PDFoPmYm9/54=; b=CV+cO9gMx+9N9G1Qg6uCVNC4ooxptbxLMwxeA7HcwN9w7d4pn2TazlbA/XvtaRUSh3D4Mc LcmbpW09hG7DRnmvGv6pk9RG2eNWyX6rU1enP+fFT0HWjVzAKHczKEydUifD8MvJG1TOiB FCgM00wLv9RZyEs7JX2drNKHtUCx4pY= X-MC-Unique: Z1ktgMWzMAWAtXN9U5DPBQ-1 X-Mimecast-MFC-AGG-ID: Z1ktgMWzMAWAtXN9U5DPBQ_1778012821 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012820; x=1778617620; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WZf+roECEzrRMn9PvQ2Od9IyuK2k8H7PDFoPmYm9/54=; b=QAULvYuUlFx4luh36bL8yQPWJAwdaYf1KpIcnCJmxvmjNm3we817OqVLp54TeEJrXo pubzMkSIlojDcNjCxUx+TsGic1ZKdzYIZOq37n3+aGvJrt3/iqB8z5xbj5eR+usCBjzI soQRPpl/0wo9CPRIvI4vv34/CYwhERjcrGAjau5XLWMob1TeLqinEPiHwGq5BtrA5rmm g5/O7mdvydOp3gm7N4WuKyS4k8O2248BCqkKMaPy2oXPfG/AqVRqJ6JhLIXWPjdLJAEs GRE27o2zQWgzvAPkjQnm8hnkELymhxkNKAao1UwqVWP24xGmWaak2o71BVLE2C+FCH/k XelQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012820; x=1778617620; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=WZf+roECEzrRMn9PvQ2Od9IyuK2k8H7PDFoPmYm9/54=; b=EQXe4BuoyFKVUyx/N/gTp028Xt2Wb9/b4Cna0hz/p7douClMPZFT3WZagMqcfb/pgP rJEPtXeztq7oKDytXQLLWr0si0MnrroUTl1G2fjtqnLETZ/6JmZJ/GK+r0jtd8W3iHDn byJIQmEWk1CD5xC3M4E9Lm6MBGeKKwrRIX5Afxwcsut22HZWtuQooAQC5w2xoWlVMiDj Bckg5+3SsBsrlP1ioLvnIBcGx+i3UXAA0wm1xyUn1LC3LnYWGpjD/TIZSayRALk6jmY8 SiZLppF5Bc0snDSa1ut+wFFFvhyKbCWB6S6Eoi+awqzHc8YFSNeitdCUttUqc/srqgmF mYDQ== X-Gm-Message-State: AOJu0YxGUbM5v+NMZtLjy3UirLowNN7KsH6fS38SdWKrdpRzkR1H35TN ggOSIC8LfkAc+4Gck3U4xROLVbE4sntpOa/ocPQcdOKtxMW16QPK7Rz8CheNB2OCw/BG9x3VfhP dKDuqoSSFEKqZViI7KYwj4bW1NDnlVwFwTC/xPpqQg6aDLbvk5DkKfqrrrWLLvZxCAPP0u+Qt2z No3+LJclAiCJFZrP3ILzkBmNaU8E/XbBnrjXA6pA== X-Gm-Gg: AeBDieuyEDGqHzH72xQSkQM8RDOGIWu81pCgds7EZQyQk8rLl+0wGh9p9vVGB00L6YX jvSsXINy98aEefwrfwgJ8bMdNXF/MRAjXyXv7IJlup84RfAV8TmgQ5A+RKRtaDAAurda1jWMi02 3iwG9kvgsIRO95VKUrd4jLM9wT6ep4Rc5KuAzwmqWvtMyby7SwlIjlNN81Ox3M+Ol3vQHxBdeuk mQurNNm2dkgXgWRHXOtiL8HKfFCtN/yByB9vr5dK6qBTdMbCwQoyaVsDSs2PRp6Y9L4ohtvCGuI peVNb2OnpDxwlcV2spVBVbiA4mGVFGLPpFwRp6djB8xKZ1u50HQmzBZSiwsP2UCiFDPiaAk0ZVq t1K9gAUrGNGwuQRKMvPbTWbEVUHyEEmkuVtqIOX/2Vcse95Z4rQUo3qg= X-Received: by 2002:a05:622a:242:b0:50d:a6a0:8f09 with SMTP id d75a77b69052e-51461f7f0c8mr9083911cf.39.1778012820262; Tue, 05 May 2026 13:27:00 -0700 (PDT) X-Received: by 2002:a05:622a:242:b0:50d:a6a0:8f09 with SMTP id d75a77b69052e-51461f7f0c8mr9083191cf.39.1778012819421; Tue, 05 May 2026 13:26:59 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Juraj Marcin Subject: [PULL 13/23] migration: Calculate expected downtime on demand Date: Tue, 5 May 2026 16:26:30 -0400 Message-ID: <20260505202640.1011006-14-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012994173154100 Content-Type: text/plain; charset="utf-8" This value does not need to be calculated as frequent. Only calculate it on demand when query-migrate happened. With that we can remove the variable in MigrationState. This paves way for fixing this value to include all modules (not only RAM but others too). Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20260421202110.306051-12-peterx@redhat.com Signed-off-by: Peter Xu --- migration/migration.h | 2 +- migration/migration.c | 25 ++++++++++++------------- 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 9081e6a612..a5e064a1ac 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -359,7 +359,6 @@ struct MigrationState { /* Timestamp when VM is down (ms) to migrate the last stuff */ int64_t downtime_start; int64_t downtime; - int64_t expected_downtime; bool capabilities[MIGRATION_CAPABILITY__MAX]; int64_t setup_time; =20 @@ -585,6 +584,7 @@ void migration_cancel(void); void migration_populate_vfio_info(MigrationInfo *info); void migration_reset_vfio_bytes_transferred(void); void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page); +int64_t migration_downtime_calc_expected(MigrationState *s); =20 /* * Migration thread waiting for return path thread. Return non-zero if an diff --git a/migration/migration.c b/migration/migration.c index 4e19fe3409..d740d9df85 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1041,6 +1041,17 @@ static bool migrate_show_downtime(MigrationState *s) return (s->state =3D=3D MIGRATION_STATUS_COMPLETED) || migration_in_po= stcopy(); } =20 +/* Return expected downtime (unit: milliseconds) */ +int64_t migration_downtime_calc_expected(MigrationState *s) +{ + if (mig_stats.dirty_sync_count <=3D 1) { + return migrate_downtime_limit(); + } + + return mig_stats.dirty_bytes_last_sync / + migration_get_switchover_bw(s) * 1000; +} + static void populate_time_info(MigrationInfo *info, MigrationState *s) { info->has_status =3D true; @@ -1061,7 +1072,7 @@ static void populate_time_info(MigrationInfo *info, M= igrationState *s) info->downtime =3D s->downtime; } else { info->has_expected_downtime =3D true; - info->expected_downtime =3D s->expected_downtime; + info->expected_downtime =3D migration_downtime_calc_expected(s); } } =20 @@ -1649,7 +1660,6 @@ int migrate_init(MigrationState *s, Error **errp) s->mbps =3D 0.0; s->pages_per_second =3D 0.0; s->downtime =3D 0; - s->expected_downtime =3D 0; s->setup_time =3D 0; s->start_postcopy =3D false; s->migration_thread_running =3D false; @@ -3166,15 +3176,6 @@ static void migration_update_counters(MigrationState= *s, s->pages_per_second =3D (double) transferred_pages / (((double) time_spent / 1000.0)); =20 - /* - * if we haven't sent anything, we don't want to - * recalculate. 10000 is a small enough number for our purposes - */ - if (mig_stats.dirty_pages_rate && transferred > 10000) { - s->expected_downtime =3D - mig_stats.dirty_bytes_last_sync / switchover_bw_per_ms; - } - migration_rate_reset(); =20 update_iteration_initial_status(s); @@ -3841,8 +3842,6 @@ void migration_start_outgoing(MigrationState *s) bool resume =3D (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER_SET= UP); int ret; =20 - s->expected_downtime =3D migrate_downtime_limit(); - if (resume) { /* This is a resumed migration */ rate_limit =3D migrate_max_postcopy_bandwidth(); --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012956; cv=none; d=zohomail.com; s=zohoarc; b=B1TglA/Oj6tPA925t0INBGYun2QyVeaDmYrcgw6sLPt8dTpJaoW7nQKF2RDNuxXlin7f32YWDOgCcKBNLQMO/47r56KgYGEOSeh3WY2l0pt8/ahxQTJPQUxP70vQwJIoYwVS+3baJqRuXUVt2DxL/mWR6bWdmBCdIgbqCf3x3yE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012956; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9WmFUP5dqUbxMM1tZndsShjc1KEJom3N3hcsbb3ZxQQ=; b=WdxemVz8If5TCgPM3VtGayf8BglhXgXiTOJthNyQKcwSpN2Z0PK/Ws/6wriL1lSFCWEWgAh02xO/qSZA1vR+Av6fG1JwbcYOMzz6fln+Xk/wpMZAn4ILr3rOdxx6PdGltVihP1rfztJKAA1zA5amMW3sX37upX32QQkUuRMZKzU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 177801295686664.3104124702678; Tue, 5 May 2026 13:29:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMN-0004b7-14; Tue, 05 May 2026 16:27:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMME-0004Tl-Ct for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMC-0002db-PU for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:06 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-20-2be3P9BEP_CRO5WP5KF-LA-1; Tue, 05 May 2026 16:27:02 -0400 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-8b597b14a22so10669136d6.0 for ; Tue, 05 May 2026 13:27:02 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9WmFUP5dqUbxMM1tZndsShjc1KEJom3N3hcsbb3ZxQQ=; b=MdKK8UXL+/PA/opfipKtkhXQQPQ8Hr4VNcySliNxnIwheDeglvwC7cFsCPgtGTFX4iB5zX amoeMm1cFR8GoyiMdy5iio2bjmpjljqYjN5/wxKt08dMbRHOlrGBo6876y8LefPw1xKixq BxZHuntCSe+f9/9cGYX1GRslSNGBhUY= X-MC-Unique: 2be3P9BEP_CRO5WP5KF-LA-1 X-Mimecast-MFC-AGG-ID: 2be3P9BEP_CRO5WP5KF-LA_1778012822 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012821; x=1778617621; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9WmFUP5dqUbxMM1tZndsShjc1KEJom3N3hcsbb3ZxQQ=; b=DWusmMnTi8JNwRLJ/fzDG33CS+LnDFDdAzMcwPEyhTA8Xy9/77f1J+g5iNbnL5HMvK DHR9Jk08LDUl1snuxMnjN8SguPKWGP4WbQcXCeZSdM7FyPQeF1C089ZcKBtRFfLRm/dl kLzpnGNEuPAkKP4RWC7UZOgK+ruJ7ck9Qigi0F/kGCGvBIvbALdsQVgojz5rOPiv4Bji +ltbdinRtngxPLAFm35LtcE4sSk5qLbpD9qzjKuOKq2NPgOH4k5QdWPKLbn5JjfMg2HD pWdUPTwppqpBGVGiVPur3oBay0XvmLJmCOPGbHLgyTrpywECeIbo9cv2Rg4K6eYGaAoC yEMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012821; x=1778617621; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=9WmFUP5dqUbxMM1tZndsShjc1KEJom3N3hcsbb3ZxQQ=; b=lnR5SZl4TUiBY96bDSGGJPwoWHCwRwiWlrUDTdZPwIsdeYHI7rj9tXDpNlr1w9L4NU OczOYTzw3kYTlU2BBuogYDUfpfPhqv+mOlLTxhkl2o83VxhqNKpalZyyM6MrjWWFNrct Z3eYoLqKQxEafm9O0Sj1RyWjZaMWYnniQZptGMmUjD49iCJ2TKbZoPMitAfgPMPD/8QM SQab1342GXJDa9gIWyskBh8CsMkYHznkvB4+Y9ZabQ54cY+/pVObu2MC8aeTLnr1qJ9F iA167JGSWJ7YSF61IRSUzWoG9t7cOI2Lr3fRx8FaE0mV0PxEpGWYMCeXYDdfkbmqHJf6 QD1g== X-Gm-Message-State: AOJu0YyZdRy3B62QkCL684f5tszc3JZkjL1X0S7nhY99Bk9duN9Ut3G4 H1xRgTh2znari4EaXRFmo28QA4Y6kWkcOhTgA4DCamqshUxKmTMp4w5blEoRbC7T+A6r+bLekRu qwmSPOeCmxqvE4FWKORfXmgD9T8OXyavuHsJghZkt9V9iSRurEfFEBCnXHl0Vc1WMzImIUO24wQ iewFdghFNsoqZV55bnGGiBoJdhsWJh5IzQwxqT7Q== X-Gm-Gg: AeBDiesKtcf01AHrXdxzYwp8zXKZICS8S2NnndSvdEr4Bumr1x0Hx3/MAvpWWYMcZsK 0BATt7gHRFt5eT2XNyMBtful8EIoIOkC6fXPLi/mcgNW6ygraENypTF+vpP5mlhoW1LKu33IiXp qNdTVW46SnyKNn2tM+0i2j8Py68nvXS1KQs/6sZeGfAiQF+5W+X5pRUVD1cxfASJ4rEX0/PtyrR rVh3ADwC1lGdZeCDzAyONC/gPi8iM8czEFiqZPJy0cspuPK1qNrJ9A/UmFggNtCZZLpJgFoBzqu xJXSp0hZ0sezjj6eIv5os8VevqV92SBr7ZLz4xGFjnfxVtioOXTcthBcuQzLFi017Q2wmii8IZg kIs/bkkBfmQH8s3L38QpiLxY0iEnTKy+wjBr+aJ3qE4QfBdo9zklJfl0= X-Received: by 2002:a05:6214:5503:b0:896:fb99:f692 with SMTP id 6a1803df08f44-8bc2a3b6da3mr9953536d6.0.1778012821247; Tue, 05 May 2026 13:27:01 -0700 (PDT) X-Received: by 2002:a05:6214:5503:b0:896:fb99:f692 with SMTP id 6a1803df08f44-8bc2a3b6da3mr9952896d6.0.1778012820570; Tue, 05 May 2026 13:27:00 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Juraj Marcin Subject: [PULL 14/23] migration: Fix calculation of expected_downtime to take VFIO info Date: Tue, 5 May 2026 16:26:31 -0400 Message-ID: <20260505202640.1011006-15-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012958596158500 QEMU will provide an expected downtime for the whole system during migration, by remembering the total dirty RAM that we synced the last time, divides the estimated switchover bandwidth. That was flawed when VFIO is taking into account: consider there is a VFIO GPU device that contains GBs of data to migrate during stop phase. Those will not be accounted in this math. Fix it by updating dirty_bytes_last_sync properly only when we go to the next iteration, rather than hide this update in the RAM code. Meanwhile, fetch the total (rather than RAM-only) portion of dirty bytes, so as to include GPU device states too. Update the comment of the field to reflect its new meaning. Now after this change, the expected-downtime to be read from query-migrate should be very accurate even with VFIO devices involved. Tested-by: C=C3=A9dric Le Goater Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20260421202110.306051-13-peterx@redhat.com Signed-off-by: Peter Xu --- migration/migration-stats.h | 8 +++----- migration/migration.c | 11 ++++++++--- migration/ram.c | 1 - 3 files changed, 11 insertions(+), 9 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 326ddb0088..1775b916df 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -31,11 +31,9 @@ */ typedef struct { /* - * Number of bytes that were dirty last time that we synced with - * the guest memory. We use that to calculate the downtime. As - * the remaining dirty amounts to what we know that is still dirty - * since last iteration, not counting what the guest has dirtied - * since we synchronized bitmaps. + * Number of bytes that were reported dirty after the latest + * system-wise synchronization of dirty information. It is used to do + * best-effort estimation on expected downtime. */ uint64_t dirty_bytes_last_sync; /* diff --git a/migration/migration.c b/migration/migration.c index d740d9df85..ab09dcbcf4 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3244,18 +3244,23 @@ static void migration_iteration_go_next(MigPendingD= ata *pending) */ qemu_savevm_query_pending(pending, true); =20 + /* + * Update the dirty information for the whole system for this + * iteration. This value is used to calculate expected downtime. + */ + qatomic_set(&mig_stats.dirty_bytes_last_sync, pending->total_bytes); + /* * Boost dirty sync count to reflect we finished one iteration. * * NOTE: we need to make sure when this happens (together with the * event sent below) all modules have slow-synced the pending data - * above. That means a write mem barrier, but qatomic_add() should be - * enough. + * above and updated corresponding fields (e.g. dirty_bytes_last_sync). * * It's because a mgmt could wait on the iteration event to query again * on pending data for policy changes (e.g. downtime adjustments). The * ordering will make sure the query will fetch the latest results from - * all the modules. + * all the modules on everything. */ qatomic_add(&mig_stats.dirty_sync_count, 1); =20 diff --git a/migration/ram.c b/migration/ram.c index ecd4b6165c..fc38ffbf8a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1148,7 +1148,6 @@ static void migration_bitmap_sync(RAMState *rs, bool = last_stage) RAMBLOCK_FOREACH_NOT_IGNORED(block) { ramblock_sync_dirty_bitmap(rs, block); } - qatomic_set(&mig_stats.dirty_bytes_last_sync, ram_bytes_remain= ing()); } } =20 --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012997; cv=none; d=zohomail.com; s=zohoarc; b=Yck8phjlZ951jVpThEAkrbV+2Glb4nT0jW5J9VVXIf0xnx0c5P+HuX/AL7Hy8sQA/7y9pz0JfLfAsAvn9XZx5CMuxPlW9qXMINSj1KFLx/yQ1vli55saIVJVHqGy4cWm70Z1o5qSZkKStQma3OZ8wPs4g6IYKaq1WUZOzJqoEjA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012997; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CDXCmWEy4iWodcXndDcYHwui/w6yNztg5xPG4/rtdEc=; b=JmHxXzyYBJiZZhib3exRwspU2voGBS3N1eVQrEFGbAkOOf6sSZvaEYc49EMYZ593iny0KWrLe8W/ZJ0cj63NHw5KaljD77LvwDMobgSeFKeTSfCoe6W0eKgydCWAyyrln/cPMERv7bm5lHjN+CbWAezSWb87LzHNe1tJi0OrhmQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012997703386.9520303780348; Tue, 5 May 2026 13:29:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMO-0004cJ-8T; Tue, 05 May 2026 16:27:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMF-0004Tm-1A for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMD-0002dv-B0 for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:06 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-246--_gS0p7_N6mk1aQL9ndlXA-1; Tue, 05 May 2026 16:27:03 -0400 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-8ba8a1f3dd7so40932406d6.3 for ; Tue, 05 May 2026 13:27:03 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.27.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012824; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CDXCmWEy4iWodcXndDcYHwui/w6yNztg5xPG4/rtdEc=; b=glUcmV+1IkcfukaLnhLRMJUQpZ4X9pz/TKQ0m2oejQRrD6HZBTaohA3KJNoxA8Z4QRuCdE JaolpArOyg41ZpF86NHuEFApgVrvhk4ZexRks4VTCWWndgSPQoLJQIifHoujMoS43WyGG3 dx83eIcHWt52rhmI+t5Lzh1By5PiuYI= X-MC-Unique: -_gS0p7_N6mk1aQL9ndlXA-1 X-Mimecast-MFC-AGG-ID: -_gS0p7_N6mk1aQL9ndlXA_1778012823 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012822; x=1778617622; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CDXCmWEy4iWodcXndDcYHwui/w6yNztg5xPG4/rtdEc=; b=PWr/ixnc+2BMM/9nwYoIAGBWKy9JVsJTH7lvJ9b/n4rK6l6EL9aV0K+T9RyztXRfqZ h6yR4Lohmr8TcV+wC5AnNHEo/reMAw3TQhg36h6CJpI/KmwBom7U57OAnstwSMEpsAQu G+LbKuFXgKBEVAkE0QpMiV5LvllSq3svskVCre/kqXDiQRAdKOgP8km4Rx3IlyTpc53N ovw24bUO8YCni793xx0Kt8LJMWFVffZVPbkIXuUrXfTvIsXrTgDedwwukgLf8XfaK7Kx lKCQDMBB3WXUZm8SGGXN3tvvt+ntW9hrAXgRlsxumpbO08R+nDfSLec1FkcWQzpMhskt NnHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012822; x=1778617622; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CDXCmWEy4iWodcXndDcYHwui/w6yNztg5xPG4/rtdEc=; b=pkyiKdW97IEQd14/6FonzKZjMVVoxbIXPl2no6YIslCaGcQEtawIYdBs9v0JBpR0ds lhu0+eW3wRJz7b1JvwKAmYBUpNlYhWqU7c8gJJ36z8LxcKqrJ6dM540l0pH0RWsD2o5g YQxRiLJknZLJyqGd0y32Xg7zII00qYafpnon1cxScap1QjsaGcuTD4GEaTN+LAHESBHd evuLOWY+7WIB4mhyy+tOo80DMy8QzbSvudl5W8Img/BwW3hko+qL7M3nQQYnjWolpvbD YXn4KFxNzg6Mb98jrZk+ZIsc7eVP2CZqN7szKb9chhOPuS/AI1ywQgKa2Ju0mZdtfdAS xjaw== X-Gm-Message-State: AOJu0YwH2lgQl/+NvaMHF82XjIzYUNnC1OshTuHghNj2olpx9U6Z+zcq KQ/JeCv9w0DwPEtqS0jmHj0QkTPory1T46H5wdPDt+LWGFa3zzzvFJ0kalph483MEtzYer1OjF5 mLtNqDIefSqE0F04AZgqk6AUyZIbnashBa5vWtEbyN0eMCNX/C0WZ1qV1VaW22y28mj0zADQAtk FI1vbCebwRdgtRXHreqpYmU3d/6y9orE4Il5f7sg== X-Gm-Gg: AeBDievf035ibKTiuB/eaX1vmebAiYTyiYYA9hXoVj0yjcdbq9vf/o/iGpEBgCn1SAo eKG7DnVG/GFx09XfO81kFwvV9yCwGmeaK5NRioKLcJqNrFW6mrRtow9qRkYrSFB4Ka1rTH5hYtj lUd5PqAoeOEZwplP2NgZ7Q7y01dU3emhsWYeKcCgUktefiQOlUabHXv0IcTNeRuiSyyEwpLyiSO CJ4ZdC5JTmHji/TBlECEMIlnnvj8nTTKgE68Cvn0+1R9+U/TMxcnyVkQAJkhs7sTawg+aOUvXmt Hwq5ShlOSwrtHloMhxCWLNBUiJN+7zdlWYIJJbBJ8t/zXSbLBkhPTiROddGJk8ZQf3CVqePqoH0 8fKkzTBh41h17vC01kW8LrCZFwh4z93772uHIxTDONrHM0cQ8Yr2GUcU= X-Received: by 2002:a05:6214:dc9:b0:8ac:7d13:d18a with SMTP id 6a1803df08f44-8bc452702b1mr4197916d6.39.1778012822278; Tue, 05 May 2026 13:27:02 -0700 (PDT) X-Received: by 2002:a05:6214:dc9:b0:8ac:7d13:d18a with SMTP id 6a1803df08f44-8bc452702b1mr4197126d6.39.1778012821547; Tue, 05 May 2026 13:27:01 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Juraj Marcin Subject: [PULL 15/23] migration: Remember total dirty bytes in mig_stats Date: Tue, 5 May 2026 16:26:32 -0400 Message-ID: <20260505202640.1011006-16-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012998791158500 Content-Type: text/plain; charset="utf-8" Introduce this new counter to remember the total dirty bytes for the whole system. It will be used for query-migrate command to fetch system-wise remaining data. A prior attempt was made to not use this counter but query directly from all the modules in a QMP handler, but it exposed some complexity not only on migration state machine race conditions (where the query may be invoked anytime of the state machine), or on locking implications (where some of the query hooks may take BQL, which is illegal at least in a QMP handler). For more information, see: https://lore.kernel.org/r/aeZMtxqrKWAMKzdN@x1.local This oneliner will resolve everything, except that it is not as accurate. The hope is it is a worthwhile trade-off solution, after knowing above challenges. Now, there is one more reason we should make each invocation of save_live_iterate() to be lightweight, because this counter will only get updated once for each loop over all save_live_iterate() hooks when present. But that's always the goal. Reviewed-by: Juraj Marcin Link: https://lore.kernel.org/r/20260421202110.306051-14-peterx@redhat.com Signed-off-by: Peter Xu --- migration/migration-stats.h | 7 +++++++ migration/savevm.c | 7 +++++++ 2 files changed, 14 insertions(+) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 1775b916df..9f9a8eb9eb 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -36,6 +36,13 @@ typedef struct { * best-effort estimation on expected downtime. */ uint64_t dirty_bytes_last_sync; + /* + * Number of bytes that were reported dirty now. This is an estimate + * value and will be updated every time migration thread queries from + * modules in an iteration loop. It is used to provide best-effort + * estimation on total remaining data. + */ + uint64_t dirty_bytes_total; /* * Number of pages dirtied per second. */ diff --git a/migration/savevm.c b/migration/savevm.c index 39430470aa..d1dd696c17 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1815,6 +1815,13 @@ void qemu_savevm_query_pending(MigPendingData *pendi= ng, bool exact) pending->total_bytes =3D pending->precopy_bytes + pending->stopcopy_bytes + pending->postcopy_bytes; =20 + /* + * Update system remaining dirty bytes whenever QEMU queries. It will + * make the value to be not as accurate, but should still be pretty + * close to reality when this got invoked frequently while iterating. + */ + mig_stats.dirty_bytes_total =3D pending->total_bytes; + trace_qemu_savevm_query_pending(exact, pending->precopy_bytes, pending->stopcopy_bytes, pending->postcopy_bytes, --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012955; cv=none; d=zohomail.com; s=zohoarc; b=MPiiZKusuhSDI5QjTEW6/hcQcZ2MLWeWzw6W8vyKLO7wJhWRU1yNR3uY84ZbFRwovWwVuQGCSB0SVKuxidt+nPzfNGSS0gmDHcRIwdvUaaKg1eijSEStEc1SEpQ1Ea1pKfmQDLke3wvisaPYp8c/Xt77BQHVUOIHNHyJa0jV0Rc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012955; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=rbz2uMzwfgzI8Mc9mnmGPnBL6d4IDbn2j6z1RXzuo7Q=; b=MFMohl+kZXeZ2S+o6Wvw2CBqIVibnpYjJmna9MBXKBdiHRVxFVsD4m+V5NSSEa+BJxpGjO4YsOr+WRlqbEzeBRe04JK5i5Z2qIhG2JVGXfJhl9s1DfUj6PmSAHgw1kk85ZfK7EYRmq76AnarSR36NFqnOvmyQwb++2ByKWBsWoI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012955962429.88439162380644; Tue, 5 May 2026 13:29:15 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMN-0004bn-F3; Tue, 05 May 2026 16:27:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMH-0004Vf-GP for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMME-0002eD-Hk for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:08 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-501-s3Z_xlT8PTSyp2x2DUzzGA-1; Tue, 05 May 2026 16:27:04 -0400 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-8b597b14a22so10670296d6.0 for ; Tue, 05 May 2026 13:27:04 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.27.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012825; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rbz2uMzwfgzI8Mc9mnmGPnBL6d4IDbn2j6z1RXzuo7Q=; b=M8523RIucVayfY8L8LFWfC2czNOutqfli0WMYArKFhWS920r4andMW4IsPI7+C3Y/i7irt iCE0zMT4Tutk1P+8M/m/sQSFLRbSDc+u9K2WFy4WvvzItV8qAeid76fJ3UnuBZ0tIAmuJ2 ixoMG5uiTcjC7AWYUDbcsF/r8VVxGU0= X-MC-Unique: s3Z_xlT8PTSyp2x2DUzzGA-1 X-Mimecast-MFC-AGG-ID: s3Z_xlT8PTSyp2x2DUzzGA_1778012824 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012823; x=1778617623; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rbz2uMzwfgzI8Mc9mnmGPnBL6d4IDbn2j6z1RXzuo7Q=; b=iJED753ZbH01oaZcY8ecUreEJSfoUQi8LoOF5a3Zsru0LF3eQ+NpLhC++AUnnki+tl aGAqoEnt/izBRxicE4tYYS8q569s/KmhRsug96MLOqbPNLG7+hhxSFbHqpziECEt9CBX bxSn3EEaYeGZKTo2LaLsipptFkvxIj2RnaK6C8QPRMFODnEZJBp8UkntGq1PmaI9AAd+ 5MVTbB9gQ7qqM6WIP3WVMTHdzat0Jdq1ttsEIMxuaY2XAnsWmTKUr4IM8xsZB4QgMHL3 08GqvLhPf1aTM1Cm6yU80x7dyo5CjbHVZkCAZmYI00XTW18CDm35nYei/aiWjoZQz6kf CtNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012823; x=1778617623; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=rbz2uMzwfgzI8Mc9mnmGPnBL6d4IDbn2j6z1RXzuo7Q=; b=Uc8x9tXvx5iV+gIsjPJ87xEdYNPeM5WjmejWkdsPpRkvQaNOgkKGSKVhhCezvcCq9k lwm6JAfTntzYFa15s9dbu/sEkwDuGy2YLA/J87zdE5WHZfeL5CLcxlNKQoZ9O9DCwF4t tssBF+CU1tYbRwD7TZlf0ZJy6/DHDv6ENH3pRtK1mFZDA2ELIBwNxetUREyRMJpGe802 Mm3jOzv5N3PTHGEqW2UP7twF5QpyCmly3vfLIMQWAtW9IWI/tiDsbRoaVUYMCMid34zI rSVNt0G/KutLWehW0tliuUPOaIByC1IA2QWd/F2HWEAF4KAZ0aMmL09gyzHnnlF1bu6d 8hPA== X-Gm-Message-State: AOJu0YycsoaP8QPpSaidHLoGC10inlrxFAMfzQO0P8I42NoJtA6KLLnD uND8BcydWB35NxIWoN5534xvjjTAPFdJWuub7Av5/+NfsnMwQIxJKsIwMZuuMZvuLUpHU/1J5Bg EMYJQ2gZfBPnUsdYh/FT3RwnTSqCouQkuZWCgsHa8UGrj1sEtW1Di0xgzHL9QGBHDi5dwzOBINv tb1s5MRRfFqDlJ76D4xV9/fXHxo9p3rSRz0T8fSg== X-Gm-Gg: AeBDieuzCOiTs1MxctP/FDZdNIhYb025xDVO/cDnqCYaQjpU48rtbYbRaUEtcz9Ltzi YEyP9gWny8bBKgHEo4i7Zf0t1lEa0RuG/Rc/9Qto0BnRA8rnjFjUz2FtPrLoFYeUNvxXvMe6M8Q smWrjp/PTE/9eeir76uKkMtjPi9NfBpuOG8thl9Bf83Jz40yUuV2shfgiWknFHHhMxZE8u4xPOk gYjQIrUId4mTXcIP+ou7ORv7iE2HEsKDaGA2M4PTcp4rLU+NFYj65vr/R09crDoh2iZG0Hxywmk Be7dHgm5CCrXBSO8431h9KyXnVOxgVcxPtV9ZuJW2lvpdOzPueXQjlxELT/ta0m/nzHjlJrtVnd 7tc2R1A0zJ8pI/4h6jmMPA/3PsbUM7e4m5ZCw/46M3QNCnSm7Bf+sCAE= X-Received: by 2002:a05:6214:5e08:b0:8b0:33a2:2520 with SMTP id 6a1803df08f44-8bc2ebdad5dmr7282466d6.10.1778012823313; Tue, 05 May 2026 13:27:03 -0700 (PDT) X-Received: by 2002:a05:6214:5e08:b0:8b0:33a2:2520 with SMTP id 6a1803df08f44-8bc2ebdad5dmr7281756d6.10.1778012822615; Tue, 05 May 2026 13:27:02 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Aseef Imran , Juraj Marcin , "Dr. David Alan Gilbert" , Markus Armbruster Subject: [PULL 16/23] migration/qapi: Introduce system-wide "remaining" reports Date: Tue, 5 May 2026 16:26:33 -0400 Message-ID: <20260505202640.1011006-17-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012956611158500 Content-Type: text/plain; charset="utf-8" Currently, mgmt can only query for remaining RAM using QMP command "query-migrate" and monitor the "ram" section. There's no way to report system-wide remaining data including VFIO devices. It was not a problem before, because for a very long time RAM was the only part that matters. After VFIO migrations landed upstream, it may not be enough. There can be GPU devices that contain GBs of device states. Mgmt may want to know how much remaining for special devices like VFIO, because all of them will be accounted as VM data to migrate and will contribute to downtime in the switchover phase. Add a new "remaining" field in query-migrate results on the top level, reflecting system-wide remaining data, which will include everything like VFIO devices. This information will be useful for mgmt to implement generic way of stall detection that covers all system resources. For example, when system-wide remaining data (especially, if sampled right after each migration iteration) does not decrease anymore for a relatively long period of time, then it may imply there is a challenge of converging, mgmt can react based on how this value changes over time. Before this patch, "expected_downtime" almost played this role. For example, by monitoring "expected_downtime" at the beginning of each iteration can in most cases also reflect the progress of migration system-wide. Said that, "expected_downtime" was always calculated based on a bandwidth value that can fluctuate if avail-switchover-bandwidth is not used. This new "remaining" field will remove that part of uncertainty for mgmt no matter if avail-switchover-bandwidth is used by the mgmt. With the new field, HMP "info migrate" now reports this: (qemu) info migrate Status: active Time (ms): total=3D12080, setup=3D14, exp_down=3D300 Remaining: 1.36 GiB <--- this is the new line RAM info: Throughput (Mbps): 840.50 Sizes: pagesize=3D4 KiB, total=3D4.02 GiB Transfers: transferred=3D1.18 GiB, remain=3D1.36 GiB Channels: precopy=3D1.18 GiB, multifd=3D0 B, postcopy=3D0 B Page Types: normal=3D307923, zero=3D388148 Page Rates (pps): transfer=3D25660 Others: dirty_syncs=3D1 When VFIO is not involved, the value reported in the new field should be approximately the same as reported in the "remaining" field of the RAM section. It is only approximately because the system-wide remaining data is a cached value, which gets frequently updated by migration core. OTOH, the RAM's remaining data is accurate. When VFIO is involved, the new value reported should normally be larger, because it will include the size of VFIO remaining data too. Cc: Aseef Imran Reviewed-by: Juraj Marcin Reviewed-by: Dr. David Alan Gilbert Acked-by: Markus Armbruster # QAPI schema Link: https://lore.kernel.org/r/20260421202110.306051-15-peterx@redhat.com Signed-off-by: Peter Xu --- qapi/migration.json | 4 ++++ migration/migration-hmp-cmds.c | 5 +++++ migration/migration.c | 7 +++++++ 3 files changed, 16 insertions(+) diff --git a/qapi/migration.json b/qapi/migration.json index ed475e4261..9a4817ec73 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -300,6 +300,9 @@ # average memory load of the virtual CPU indirectly. Note that # zero means guest doesn't dirty memory. (Since 8.1) # +# @remaining: amount of bytes remaining to be migrated system-wide, +# includes both RAM and all devices (like VFIO). (Since 11.1) +# # Features: # # @unstable: Members @postcopy-latency, @postcopy-vcpu-latency, @@ -310,6 +313,7 @@ ## { 'struct': 'MigrationInfo', 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationRAMStats', + '*remaining': 'size', '*vfio': 'VfioStats', '*xbzrle-cache': 'XBZRLECacheStats', '*total-time': 'int', diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 4f6c1dbf89..2703448c52 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -178,6 +178,11 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) } } =20 + if (info->has_remaining) { + g_autofree char *remaining =3D size_to_str(info->remaining); + monitor_printf(mon, "Remaining: \t\t%s\n", remaining); + } + if (info->has_socket_address) { SocketAddressList *addr; =20 diff --git a/migration/migration.c b/migration/migration.c index ab09dcbcf4..ecc69dc4d2 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1076,6 +1076,12 @@ static void populate_time_info(MigrationInfo *info, = MigrationState *s) } } =20 +static void populate_global_info(MigrationInfo *info, MigrationState *s) +{ + info->has_remaining =3D true; + info->remaining =3D qatomic_read(&mig_stats.dirty_bytes_total); +} + static void populate_ram_info(MigrationInfo *info, MigrationState *s) { size_t page_size =3D qemu_target_page_size(); @@ -1177,6 +1183,7 @@ static void fill_source_migration_info(MigrationInfo = *info) /* TODO add some postcopy stats */ populate_time_info(info, s); populate_ram_info(info, s); + populate_global_info(info, s); migration_populate_vfio_info(info); break; case MIGRATION_STATUS_COLO: --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012960; cv=none; d=zohomail.com; s=zohoarc; b=UN8EBMKFfhlbD7/k98rymayril9QkUvvKzn+peJEoK6F3z6Bh/bSeL3ONeq0LKx0Lxcbk31o+Z6sFh6TW4f8d8AA42Gv2fa+aXfU9+ajoWwiUS7yn4gIdxkvbU8v1H+m+x45fn0/3eJsxJrRAp46p5dWXuLw1A9HmwlXlCPiX9M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012960; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=yk4IP3x6GBLA9Cfka7+splQgVMlDUHi53tCTj7JWBn4=; b=HTxXIa2EboNs+d74Bp+rEdAq8taAXzFwTjra5m79LbDuFtkh5eEtg+hCUWuwFquDCA8cX7qRiyeuyceFIcE9EenwAWNAxCcwPanXsvxmmua//XpCpUm/9wRbhbqrtUE9gSwsrJMa2AsWEdzVdgDC6kcSmOt2BJUYdeG30l9MsJg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012960933955.0987602308392; Tue, 5 May 2026 13:29:20 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMO-0004cq-QA; Tue, 05 May 2026 16:27:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMH-0004Ve-GD for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMF-0002er-8p for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:08 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-34-Z6N0sNZdMHuojv8MEiyQLw-1; Tue, 05 May 2026 16:27:05 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-50ff0bef198so145539621cf.2 for ; Tue, 05 May 2026 13:27:05 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.27.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012826; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yk4IP3x6GBLA9Cfka7+splQgVMlDUHi53tCTj7JWBn4=; b=aNeWGeNgHRT+bvKNYJvvDNfA/ZP03c41Pt85N3G/uICn+TPJtgc8eXNHQZ0Lv9pgHZPUti QzGsKRDxeAcsMo5HTVK+xtYYEXCkKC83F8QQLzSXprVQ+VjttjahjbYhF5xfBjGvxmSBm+ anD9ESgqqOnmY7e2ig+SOBh3mePgj+Q= X-MC-Unique: Z6N0sNZdMHuojv8MEiyQLw-1 X-Mimecast-MFC-AGG-ID: Z6N0sNZdMHuojv8MEiyQLw_1778012825 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012824; x=1778617624; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yk4IP3x6GBLA9Cfka7+splQgVMlDUHi53tCTj7JWBn4=; b=VSV6jYxlFqB74LfR+vabyflNWUSwboSW+g78NOpSySYSCkJ6dzMc7u+9bkoY5irLXD I7E7LmwzYJEoGuMc7iIsMqPN/0H6gJ8IfEBhylQv/hdtGPziPO2jYu7drtM9Q6f89iUK hl+SvM/Gc1HnvHzZKirg5w5+z50Hk6BTEjnrmaLkOGSr1y3Oc3ymRpnDmMz4EIww3IsP AOj2u28wr7+BVL5UrbEEN30qOibbiyk0Ll6yMZuJqf8S7hb0T5yo6KYhj65qlqO3ZFd8 mTsPhko+4FUU8U5x+Qe6wHCJKx4RKmC6NN02z+yW2CEPohFTs8dcSAlm3ggScoWqQsld 7kVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012824; x=1778617624; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=yk4IP3x6GBLA9Cfka7+splQgVMlDUHi53tCTj7JWBn4=; b=TEjQrqEgTn7i3IppK02Znmy4s5KrOdX96NkR8BRZ6oiqvbxb4Ci8+h5/wScJjfz8y4 13rU6lbllHY/dyxwyDRnpp/8Nz5WdiygDPhj/P7misC5yyLXajA/sqtVSz/NQv5TnoM1 rmkIy3bUmbRkljZRwF7oDW2QPuc84WABQQQAzmmP7LqyQHKkN1bycHjzz+oezoE+aXuC VBrLNuJ7e8pFKfaG2QeLcX6Q1BPRRaONZ8+A5LlXHlv3XR20tEck+GxrruQB8wiW9v4N QkLV8iRLFWzVxWUr9WBP2MUjLq1jtTPKRI+zJdotxImUUBAde56TL2reKZl/VyIIi6gN 9ekw== X-Gm-Message-State: AOJu0YyJU2R4P2yeQ3pEPuraWfz3tAmNOK9AXuH6cDIuNRx6xMwURtBk GSTxXqwXyUkDT/+vlDQv1vMAFeTtEo9UNkM816QkieoYoMeitydTavNoepUI1RB+wuNaNmsUn8c 6jnheWNw8w+LzqptNun5ys4x3FbSCVTHyvlQaeJOXeXtz3HcBIu3WCD/1o3a0d4ZXaqOpVwdk/L yJzBReMjGdfxv+y3B5rKvpwLxAWHVIQUFzeUEgpw== X-Gm-Gg: AeBDietT7ORfA5wlsz4aQfvMxTJw78UlicbmfzAxE1KgWMK7u2OoU4bmliECAsZ1a9u t+KRp2OvN9a3CX3njJygGJSJW1ZZ41HTQB6dRqXb5qASFzFIm1nqWBICtL79S3G5BGIVqUxa/Ud WSx7V+kYXUFfS+Sj4H4TZocqUmqGdGBzUz+0K4xgbQKClxegvT5loEu8yKgVJuM9Z16z/VWRbDE /obDNKPACyf/SNMaH7RsganphQRlQvpzA9o/D9jmxZWMqzrluu94jD5vB9pWAxdJIuYwasWjEFX 4pqVv0D6Eyq5YFP4ysuh/Z3/F/DA2YEnAXqpW8VgfRfj7F9hZy4BdOQSmPOwzG0dDjdeq6t6NPe ZCxUGNkC+4LZXnZ5PaAzW9YqmH0ZtZplZZ738IZTxnZEMcWE7dI/QWtc= X-Received: by 2002:a05:622a:5c0d:b0:50d:a644:699d with SMTP id d75a77b69052e-51461fb988fmr8260881cf.46.1778012824199; Tue, 05 May 2026 13:27:04 -0700 (PDT) X-Received: by 2002:a05:622a:5c0d:b0:50d:a644:699d with SMTP id d75a77b69052e-51461fb988fmr8260251cf.46.1778012823512; Tue, 05 May 2026 13:27:03 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Markus Armbruster , Juraj Marcin Subject: [PULL 17/23] migration/qapi: Update unit for avail-switchover-bandwidth Date: Tue, 5 May 2026 16:26:34 -0400 Message-ID: <20260505202640.1011006-18-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012961630154100 Content-Type: text/plain; charset="utf-8" Add ", in bytes per second". Unfortunately indentations need to be updated completely, but no change on the rest. Cc: Markus Armbruster Suggested-by: Juraj Marcin Reviewed-by: Juraj Marcin Acked-by: Markus Armbruster Link: https://lore.kernel.org/r/20260421202110.306051-16-peterx@redhat.com Signed-off-by: Peter Xu --- qapi/migration.json | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index 9a4817ec73..27a7970556 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -922,15 +922,15 @@ # (Since 2.8) # # @avail-switchover-bandwidth: to set the available bandwidth that -# migration can use during switchover phase. **Note:** this does -# not limit the bandwidth during switchover, but only for -# calculations when making decisions to switchover. By default, -# this value is zero, which means QEMU will estimate the bandwidth -# automatically. This can be set when the estimated value is not -# accurate, while the user is able to guarantee such bandwidth is -# available when switching over. When specified correctly, this -# can make the switchover decision much more accurate. -# (Since 8.2) +# migration can use during switchover phase, in bytes per +# second. **Note:** this does not limit the bandwidth during +# switchover, but only for calculations when making decisions to +# switchover. By default, this value is zero, which means QEMU +# will estimate the bandwidth automatically. This can be set +# when the estimated value is not accurate, while the user is +# able to guarantee such bandwidth is available when switching +# over. When specified correctly, this can make the switchover +# decision much more accurate. (Since 8.2) # # @downtime-limit: set maximum tolerated downtime for migration. # maximum downtime in milliseconds (Since 2.8) --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778013015; cv=none; d=zohomail.com; s=zohoarc; b=JLfG/w9SKFvRAT0UXPVm6I6VC1iPUaxoskHe/fDEt6CXQEv2m7EoPh2OKVNNckqWam+GjS/b20fICaUW3wTBKX5MSsmCzme0DsoWGvx0mnyBNXrX9xvfvFo69xErEsCYZlyccYlU+hSRmjGmp6r0l+auKQcGPs1tqETcxM/h5Ys= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778013015; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZOv4dkPxbigP2amkOsD1OyD2ureHpWxPWa9Eo8OsVWA=; b=hUmjtI+mSHCK8TOT4ujBA2IL0guaw8QJMtwCR4Pz+UUz3iNhhOHB3md1GcC5cJQ4oA5oD3AArPFFaEfvIfMQU3KB/AXbLUpN6iBVVRL5f7sFeL4JJJeVflj2RICdcFrkGvX8ZF2BhPZMPF9isPs+vPbdeVQtYm+FE3UwHSjClCc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778013015311219.15517461648562; Tue, 5 May 2026 13:30:15 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMO-0004c3-2W; Tue, 05 May 2026 16:27:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMK-0004X0-3n for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMH-0002f6-7C for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:11 -0400 Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-457-pjZ11YRgOFGhEb71RgYl_A-1; Tue, 05 May 2026 16:27:06 -0400 Received: by mail-qk1-f198.google.com with SMTP id af79cd13be357-902e1e4f8ccso341009185a.0 for ; Tue, 05 May 2026 13:27:06 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.27.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012827; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZOv4dkPxbigP2amkOsD1OyD2ureHpWxPWa9Eo8OsVWA=; b=c732tGR74BukJZx/YQhdkp/BCRGM60DbD5zLPH0hbWmoKQFtk8lAWNb/j8mn1f7FFgiyZ2 JWuVRpAJH80IKUJrd6eRdePt0Vdgi7kBRBFq1x6c11gEa9yIycUiP9HOeDii9bTPAfPR/1 Kj7l45UZtc5iLGHIQSLdBsmw7jrN5Cg= X-MC-Unique: pjZ11YRgOFGhEb71RgYl_A-1 X-Mimecast-MFC-AGG-ID: pjZ11YRgOFGhEb71RgYl_A_1778012826 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012825; x=1778617625; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZOv4dkPxbigP2amkOsD1OyD2ureHpWxPWa9Eo8OsVWA=; b=BfCcGKoPXrI/ouYOhelUYpBg+Wf+k1EbRIQgJF/gQXRYyRnmQRG0oA6AdwAee/F6By xoExRXSp9Ash5eR/Ww9SlQUNtSUa8Qfoyidoobx10f30zy2LvLRu1WLGcOD6wLLWUobd 6k/lB5EzPKg1x4FRMcPX80ZrcjK8/osUsM0Z7QGX6uuVS45U13ikGaLYvfd+/GbodBcp kwlXt3an085yOxr1jsFdSiRWZGEg/dnTQhVXEl82IJ0kuVGJMscAMlQQp1nK/pwkFI5X 6et0lheNVdwwTsIvxpGnT5KTIKEqoAxDQz/9mw9R4yiNndYzGT3sdigWSs8xRhkyzsN0 e7MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012825; x=1778617625; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ZOv4dkPxbigP2amkOsD1OyD2ureHpWxPWa9Eo8OsVWA=; b=dxQfmEEyS4YLpH3LwbHFZ2y2LJqhFmf5gkFwVQ/pi/HjL60/ZXWz8+kum9imWSouu5 7xcysXZdbchGSFkOrFSoQCE/0iWuxoDi1+22ddh8jwmEyPk0Iemxh/rJSxYPYhqmFHw0 qj+HvFp0Ac7F2kBJ2fSuM/FrTEXM+WbCyeNTVdZJHt6KdWM9RN0MtMVAWfHeMrSJ3LRG Me3Uw0lHp3wG3+ms8ss7EXqtpWIZzKYTPdCiGLk3sgJF+sfXoJVDBjDyU8wxFztQ8EXx Z0oX1ZA1Ke0RCLj/FUyBuChMboJMLtyB0p1/Q7tQ4aAu7oSftLtp7gdeuAnAhPYpI917 xLtg== X-Gm-Message-State: AOJu0Yzs8sTpMPpYYFiVQ2+3nn2vUICBQHYQT10RhTk6FZr4uPY/zgsJ DMsFJLBs0ICh8kg1Q1XGjopsi/t4mvDjPyXGinyfhM22guIj324HVUL6k0Jm/r3TKJ1cjoggO/1 8XUv2r8P8KD3Ah2o5ohO0KrZi3iFbaw1K4bezU0tT8Bs20bYFQqsMd9S9HTNzfot7YRK7iK9h4T Ja0GArs3EykPv+sNMUSn7/bFy6JKWOexy3HqHuSA== X-Gm-Gg: AeBDievwjsBvO17oJlxVMX7OpQEkGP4SbWLUnri3XqzZbVA0zoKrUkJyiTnXreFwrhW IqkEjemZV9qlsZtikrOS1f21Ih+3h4lcNCuQSmO/5k9rzAOQUvRarjxtr+4IKmpHV7n8zumEJya fbQIy0mTg7A+xSZm2H5koE9KaC79B9E1v8gXm++tndjAgULryjxLtCHY/y4JRSrM0OX1W+j7MXn b4d5wqIpY1cLB2KUJFIGb4P6M9kUIEnpBGYlkrfg6uiTnEkOMwNvgLgpaYs4SI9eKzx0JHrdWMw qOPdUiVj6RyYJPZu6sfgd02LduhOTiDQ5UFENOj8GUwL5D1oOPX9IoaJYRj8t/kEa4f3oaQxooj jbm/Lyypl9JcnJy6cnYNflYYaLuR+89hIZLnHsQt/0H5qMWG4bHNXReo= X-Received: by 2002:a05:620a:280e:b0:8f0:10b0:9e4a with SMTP id af79cd13be357-904d6fcc196mr91062185a.57.1778012825154; Tue, 05 May 2026 13:27:05 -0700 (PDT) X-Received: by 2002:a05:620a:280e:b0:8f0:10b0:9e4a with SMTP id af79cd13be357-904d6fcc196mr91057385a.57.1778012824538; Tue, 05 May 2026 13:27:04 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 18/23] vfio/migration: Add tracepoints for precopy/stopcopy query ioctls Date: Tue, 5 May 2026 16:26:35 -0400 Message-ID: <20260505202640.1011006-19-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778013016538154100 Add two tracepoints for both precopy and stopcopy query ioctls. When at it, add one warn_report_once() for each of them when it fails. Reviewed-by: Avihai Horon Tested-by: C=C3=A9dric Le Goater Reviewed-by: C=C3=A9dric Le Goater Link: https://lore.kernel.org/r/20260421202110.306051-17-peterx@redhat.com Signed-off-by: Peter Xu --- hw/vfio/migration.c | 35 +++++++++++++++++++++++++---------- hw/vfio/trace-events | 2 ++ 2 files changed, 27 insertions(+), 10 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index e6e6a0d53d..150e28656e 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -329,6 +329,7 @@ static int vfio_query_stop_copy_size(VFIODevice *vbased= ev) struct vfio_device_feature_mig_data_size *mig_data_size =3D (struct vfio_device_feature_mig_data_size *)feature->data; VFIOMigration *migration =3D vbasedev->migration; + int ret; =20 feature->argsz =3D sizeof(buf); feature->flags =3D @@ -340,12 +341,19 @@ static int vfio_query_stop_copy_size(VFIODevice *vbas= edev) * is reported so downtime limit won't be violated. */ migration->stopcopy_size =3D VFIO_MIG_STOP_COPY_SIZE; - return -errno; + ret =3D -errno; + warn_report_once("VFIO device %s ioctl(VFIO_DEVICE_FEATURE) on " + "VFIO_DEVICE_FEATURE_MIG_DATA_SIZE failed (%d)", + vbasedev->name, ret); + } else { + migration->stopcopy_size =3D mig_data_size->stop_copy_length; + ret =3D 0; } =20 - migration->stopcopy_size =3D mig_data_size->stop_copy_length; + trace_vfio_query_stop_copy_size(vbasedev->name, + migration->stopcopy_size, ret); =20 - return 0; + return ret; } =20 static int vfio_query_precopy_size(VFIOMigration *migration) @@ -353,18 +361,25 @@ static int vfio_query_precopy_size(VFIOMigration *mig= ration) struct vfio_precopy_info precopy =3D { .argsz =3D sizeof(precopy), }; - - migration->precopy_init_size =3D 0; - migration->precopy_dirty_size =3D 0; + int ret; =20 if (ioctl(migration->data_fd, VFIO_MIG_GET_PRECOPY_INFO, &precopy)) { - return -errno; + migration->precopy_init_size =3D 0; + migration->precopy_dirty_size =3D 0; + ret =3D -errno; + warn_report_once("VFIO device %s ioctl(VFIO_MIG_GET_PRECOPY_INFO) " + "failed (%d)", migration->vbasedev->name, ret); + } else { + migration->precopy_init_size =3D precopy.initial_bytes; + migration->precopy_dirty_size =3D precopy.dirty_bytes; + ret =3D 0; } =20 - migration->precopy_init_size =3D precopy.initial_bytes; - migration->precopy_dirty_size =3D precopy.dirty_bytes; + trace_vfio_query_precopy_size(migration->vbasedev->name, + migration->precopy_init_size, + migration->precopy_dirty_size, ret); =20 - return 0; + return ret; } =20 /* Returns the size of saved data on success and -errno on error */ diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 287df0b8cb..2049159015 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -162,6 +162,8 @@ vfio_migration_realize(const char *name) " (%s)" vfio_migration_set_device_state(const char *name, const char *state) " (%s= ) state %s" vfio_migration_set_state(const char *name, const char *new_state, const ch= ar *recover_state) " (%s) new state %s, recover state %s" vfio_migration_state_notifier(const char *name, int state) " (%s) state %d" +vfio_query_precopy_size(const char *name, uint64_t init_size, uint64_t dir= ty_size, int ret) " (%s) init %"PRIu64" dirty %"PRIu64" ret %d" +vfio_query_stop_copy_size(const char *name, uint64_t size, int ret) " (%s)= stopcopy size %"PRIu64" ret %d" vfio_save_block(const char *name, int data_size) " (%s) data_size %d" vfio_save_block_precopy_empty_hit(const char *name) " (%s)" vfio_save_cleanup(const char *name) " (%s)" --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012955; cv=none; d=zohomail.com; s=zohoarc; b=RZ2ElClCYJ51YB2WZ6qjsfok4iw9uJQNr8rsgxMUYEJ4r39Z/hZo3FmoyXtEwTooGP5zVVW1EdiFULVM9E0FTTmvbRnyBhScLB6KjFxNaB2f0STFl8TCqm5UluNnY/PxCWda/rP7AKK/ZGL0HJmsccUsMlvUGcYxysAiKSv5czE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012955; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=UrdJvZlwcHz7aiZ2PAnafLIUZGKxAgtWgdAZUAhTr6E=; b=RORJoNgIyiUiPPtRjPtFCNM+CaPoTwtmRex4Laok13cV68+cAnRD2dPQyBWeKLYy5c4d7yS6Q8zkU863a2Q+QRVWIKbxTVInjwuGKXiStNd+MLRb33DMUEj5FRXYO68axi36uED7HzolVg7zMk4e4Nv/4ceEGL7930c0Ta+oneY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012955212940.7178717028787; Tue, 5 May 2026 13:29:15 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMP-0004dL-5d; Tue, 05 May 2026 16:27:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMML-0004ZD-Hq for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMJ-0002fS-3d for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:13 -0400 Received: from mail-vs1-f71.google.com (mail-vs1-f71.google.com [209.85.217.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-280-ax-VDKlwMP-XHDhM3CqKTg-1; Tue, 05 May 2026 16:27:08 -0400 Received: by mail-vs1-f71.google.com with SMTP id ada2fe7eead31-62f30279e90so2381900137.0 for ; Tue, 05 May 2026 13:27:08 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.27.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012830; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UrdJvZlwcHz7aiZ2PAnafLIUZGKxAgtWgdAZUAhTr6E=; b=RqzPbWI4+yvYqBeV1CCNFh7sAGO3h/Dy5EUljN9JD2GrZ5u6O5kCiRtc5U48xotNWg26Wa BNhrmL5JeKUtm3JXTBV5gSHxo8RZ5V5BAtysQavSGowwC/cWfevkRDgY6EZ8rYo1H348XR 7M5c7mXf6s/05XPRiXzvZxtbWjePf1E= X-MC-Unique: ax-VDKlwMP-XHDhM3CqKTg-1 X-Mimecast-MFC-AGG-ID: ax-VDKlwMP-XHDhM3CqKTg_1778012828 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012828; x=1778617628; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UrdJvZlwcHz7aiZ2PAnafLIUZGKxAgtWgdAZUAhTr6E=; b=MURegrAvqCmj9MQn8VXohb2TiCgXTacphvWGB0r9ADB7UJE7q5PlwYeectXZLrX0cg kh5KoXwz+YWfZU3gcw6QYAKT/JLyiXl18JpzPrJ4Ijua8Epy8ZRZFrXE/gNAejWXT/7a 9IvDP92sgOKwzrVkf17owlkxQC6CfX0Et1EANWw+wUTbsKcNA0aQ8zb0Xx8OitsXYkZL wz8TDerFdN9Y5UCZLGqhgeLrjOaYA7u/gvQ0lv6jGYYFUFB9y1cHNQhjIWterkKnOJUw o7/Fj2HEqYqyy7qxgF6TBpbCOZ6SA7WK2RKGR25gOlXDd0witzYJTVxf7gyOR5VuCjci Xe6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012828; x=1778617628; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=UrdJvZlwcHz7aiZ2PAnafLIUZGKxAgtWgdAZUAhTr6E=; b=UNaxbrRKLkINoPUcAaEWU0rQBlwSWgQrIpzi+107LvpY083n9nKHc1JdjhQfZz9L5A tDCd+ZkT/pLlrGmciRToOSaIVsk+1BVS7hpJtei/j5pn4tfflFFcJh5AAYZVb9IU2tzc BHda/+8w6O/NK5K2ZAORYwk0P3Q04l6aWJNPYcHIwL4EuH0D2LkJZ2til14axeRxSLCz fhcT4L18ajWDwfam3fBBH6hU3vuE0/DDM5YgQy0k0VqeEAZT5eLm27di9EkpxhU8D45U C4JLe4pyQT2IkH+uqhlG8CbKovAP8XZmX6Y5yONDzzeRNkRW2I+0fBmtd9yrvwkSATKd jdaw== X-Gm-Message-State: AOJu0YxhbBcLa9HrluHNxrKHZg2G0Q8nVQzIqFc54k2haHa0bH188nTO zjYT3LNaYdHmXI2gNvf8whJ19UnSGzHvYAV7e/1TbMSJ5apXIju23VI/VDEWPlIVXF20DNb9zS1 N4XKlSygY1n4by68iFpxOw2CGvo2k9ofMKVOhOA0q76/1Cj4JxuDOVxBjicAuG4uOT7L6B5a4BC 8DqT1GyiwEl9DSV4fyUobywAR45tNs7pHTFyXbFA== X-Gm-Gg: AeBDietFUuHHi1EP9VEAA2eoBWRcXRAsvxAX2+MNZpMS9gPmvgYf/RnSHP2sbvEioiZ FIymJicA+1npq/JCFhW0rg/331V6ojYOz2sNv3zP2MOPKuUhUFDSECsoJFujFKVeC+RprbWqxSi FT41iMZUDOBbcmDX11GvSMSUpA92aaoruTsP05OUL8I3ieaUL+k00H2emvFw5vhF3rGrjgJPNUw 9KodJ8O5WR9fdvkeHHYVjM30BmkYs7mEgvdbbTyYbB3dqS7X4D3Ta9auEdF/IwRzGOTz16yzsUT U9KPTNbxQEYp4IJdpgB/Bs2krMGcwseIpmgNvGTMOPQRvunb5eD4g3IBzMnahdYrWpqvXmuI93i TLRjl84itVPm2oBgieyXLmHWduLVQDD66cNWGUwPW2Buf/3sqyIPrpPo= X-Received: by 2002:a05:6102:374d:b0:611:e0c2:1604 with SMTP id ada2fe7eead31-630f8fcf085mr136593137.19.1778012827446; Tue, 05 May 2026 13:27:07 -0700 (PDT) X-Received: by 2002:a05:6102:374d:b0:611:e0c2:1604 with SMTP id ada2fe7eead31-630f8fcf085mr136396137.19.1778012825823; Tue, 05 May 2026 13:27:05 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , CJ Chen , Tomoyuki Hirose , Daniel Henrique Barboza , Peter Maydell , Alistair Francis , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL 19/23] hw/riscv: iommu-trap: remove .impl.unaligned = true Date: Tue, 5 May 2026 16:26:36 -0400 Message-ID: <20260505202640.1011006-20-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012957724154100 From: CJ Chen The riscv_iommu_trap_ops MemoryRegionOps specifies that unaligned accesses are not valid for this device but that it does implement them. This doesn't make much sense, and we want to add an assertion that registered MRs don't specify this invalid combination of settings. Drop .impl.unaligned =3D true, with no behaviour change. Signed-off-by: CJ Chen Acked-by: Tomoyuki Hirose Reported-by: Tomoyuki Hirose Reviewed-by: Daniel Henrique Barboza [PMM: reworded commit message] Reviewed-by: Peter Maydell Signed-off-by: Peter Maydell Reviewed-by: Alistair Francis Reviewed-by: Philippe Mathieu-Daud=C3=A9 Link: https://lore.kernel.org/r/20260428093339.2087081-2-peter.maydell@lina= ro.org Signed-off-by: Peter Xu --- hw/riscv/riscv-iommu.c | 1 - 1 file changed, 1 deletion(-) diff --git a/hw/riscv/riscv-iommu.c b/hw/riscv/riscv-iommu.c index 7ba3240552..2b40ab2ce0 100644 --- a/hw/riscv/riscv-iommu.c +++ b/hw/riscv/riscv-iommu.c @@ -2434,7 +2434,6 @@ static const MemoryRegionOps riscv_iommu_trap_ops =3D= { .impl =3D { .min_access_size =3D 4, .max_access_size =3D 8, - .unaligned =3D true, }, .valid =3D { .min_access_size =3D 4, --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012992; cv=none; d=zohomail.com; s=zohoarc; b=FufWw1iDaOmk0TSMfnAdE44I8CMxgkMjh7SOEWuAfNHMvishOaPuBSMeK4TcLws7gX02pj4JDVAcF/JVj78203HefoSqEQbzPuD7gyWPhoWDs+zGXuHnuNf/IutfWGHn54NseguhPw1aED5hfWitq/HSZ9UJnJK2GmuIHs5qtbc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012992; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=JccatWTm97VV50DRacC5l35//16UROxbyfZ0bobIw3M=; b=kOgwbFVTIdno7m5q9U88QdFukivw+APgxpBE+q11sCV8zdTL48BZDgLpBNjnqYytEVDOc0hxsPhV37nJziI6DNho5Y/as6eUyTPVTdDR7rLajCg51trLWXHtkojE6/58k8N7tSWwr8fHaFJSGwYT4zOz+b3/BZnJCQOVX26Y8Qs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012992399225.23599765494384; Tue, 5 May 2026 13:29:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMO-0004cZ-JP; Tue, 05 May 2026 16:27:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMML-0004Z0-Dd for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMI-0002fP-D2 for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:13 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-507-H_E3Q3HKMW2gMp-KGqmKkA-1; Tue, 05 May 2026 16:27:08 -0400 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-8acb85a973cso83859056d6.3 for ; Tue, 05 May 2026 13:27:08 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.27.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012829; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JccatWTm97VV50DRacC5l35//16UROxbyfZ0bobIw3M=; b=S/JkmYycS5WB+85yZ9qEyaT1dH1dYyqAbAKOjl4IrJMX9ve4GRgunNRWDkfPf7KCWHgfy3 96hTwk3oN/E0rQ7LSvk8lrptoklDoLpOHoyQIwydn47oveNV0lT1LjF9qUw9KDD1sGgKy8 VEFr3XJ0JcXOrbtDymXMVgov913Srfs= X-MC-Unique: H_E3Q3HKMW2gMp-KGqmKkA-1 X-Mimecast-MFC-AGG-ID: H_E3Q3HKMW2gMp-KGqmKkA_1778012828 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012828; x=1778617628; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JccatWTm97VV50DRacC5l35//16UROxbyfZ0bobIw3M=; b=L6OXOoWJ5jx6AYkEQfWGDyf13QtT1VSgF37LVPw36z0jN/35a5KhL3yJa8lQRbt+Fz XMLCeFezdx3Ic5iHT/sGRr8HWWPtY3k2V51O2EQ0patMzOA6rFVG5C4dtWkCVwJNxkt0 QlwHtJffrW0AaTcWNYSVl5QsyfCuRsSc1N6/NmHYNs/4CT7ihR5WjcvOeZiP1YR/dQ2q u+iqNBJqtLX9k3TGlXZOtjbcrqO8Fj4W2hzFnQt90sSeVL6N+BMFMtWWPUlHQS0uyMiW Y05MdIgSEhfI8MV6JRPPQdI/yTDTACZT93ghfQIwqS5NGhuPHZuRjTX388u7HShMUD9m rSkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012828; x=1778617628; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=JccatWTm97VV50DRacC5l35//16UROxbyfZ0bobIw3M=; b=rgeTJEHXmlTzLYkNgr0lh1D4siD1rieLcbGPTTDTTFVaxVEJALrSYnwq0wWjPtc8ub ZFq2V6DvDCJhmsp3cz9/X/J56TQQy1OxMX3wS30f1iEhbujNV73ny27DJIQSJpiN98eN mBqPMgySss0uhkgFP496uaRliJdSLbZQAR3t1SQr390byX+Rx8a+vZGFhQ/lDuJG8bN2 XBN/dYkO4x7uFA9Wz6d7OyfANTb+dGxtOpCp4KhAE371Zf0tnsPMoEhoc1ZmbLA0GyL5 GKE52vb0gCjTw/3cFEHQj1/MvaFEOjpmojd84/L1lX22W6T9yI/7raglK0dflVOGBp0M da8w== X-Gm-Message-State: AOJu0YynuieYRhyX72ED1poiRYgtf8AVgDza+gFh2BmgJJvzFi0yZ30g Wud2n8RYjt1vpj2xeeIh3U3CdAvGNxKm2vTpvI92d5ZeJGhJPybxPYiWydC2CHcmPuk91jw0ylD NbzboS5zGPkBKrY3pMBor7tHYPV1/RX5hmrxOTc5cZihNtozVdpaHb9/aLHc6pnWu1fB8B9LUar 6kSs7PoXOBjOeLozLK+dqSwb3pCysySjmdurcY7Q== X-Gm-Gg: AeBDietBZhg/bo0G/y+Dss1PEIB6g+4gU0u1taf3jV9mN4DqNBgvjNTp9OZvdzmVfnZ 2lDeopOAtJmLKg2EV90qfalKGIg4h1V0dG46JyJxMAvIhpTumiQGLHD2DIk23NEv9xc6Z1yAItK MAISsUFmgrEP99wXW8kQhSkmXMk2X3nVif2TNypE5hkmh8P+vGlTYoYP1Tx5wOZ3iirgB6UC+wE yAmIbEvglm85IGsc38FCT8GopCIjgReZnPoFCmDWp0B6fuDD4DY/AaLFhkxf4Z9SJk7kTP0MG3D niNFRWbefm+4PesApNJehPQVDYwh4cram9F3MTggkN8pXMoIJk+fx5WKFRK0Jpv/kFbD3WV5J4U OLpCkACiNybHJoO8LKuWfMtvsKms55klefJNeLJfPnb/gIdyWKd5tekk= X-Received: by 2002:a05:6214:19ea:b0:8ac:a553:528c with SMTP id 6a1803df08f44-8bc45f1aee3mr3796726d6.42.1778012827587; Tue, 05 May 2026 13:27:07 -0700 (PDT) X-Received: by 2002:a05:6214:19ea:b0:8ac:a553:528c with SMTP id 6a1803df08f44-8bc45f1aee3mr3795886d6.42.1778012826872; Tue, 05 May 2026 13:27:06 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Peter Maydell , CJ Chen Subject: [PULL 20/23] hw/npcm7xx_fiu: Specify .impl for npcm7xx_fiu_flash_ops Date: Tue, 5 May 2026 16:26:37 -0400 Message-ID: <20260505202640.1011006-21-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012994234154100 Content-Type: text/plain; charset="utf-8" From: Peter Maydell Currently npcm7xx_fiu_flash_ops provides no .impl substruct; this means that it gets the default of "implements 1, 2 and 4 byte aligned accesses". This is more constrained than the device permits in its .valid substruct, and also narrower than the functions are written to handle. Add a .impl substruct matching the .valid substruct; this means that all guest accesses are handled directly by the read and write functions, and are never synthesized by the memory subsystem performing multiple accesses to the device (which would not behave correctly, as these read and write fucntions have side effects). Based-on-a-patch-by: CJ Chen Signed-off-by: Peter Maydell Link: https://lore.kernel.org/r/20260428093339.2087081-3-peter.maydell@lina= ro.org Signed-off-by: Peter Xu --- hw/ssi/npcm7xx_fiu.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/ssi/npcm7xx_fiu.c b/hw/ssi/npcm7xx_fiu.c index 02707de350..2d5bed005a 100644 --- a/hw/ssi/npcm7xx_fiu.c +++ b/hw/ssi/npcm7xx_fiu.c @@ -250,6 +250,11 @@ static const MemoryRegionOps npcm7xx_fiu_flash_ops =3D= { .read =3D npcm7xx_fiu_flash_read, .write =3D npcm7xx_fiu_flash_write, .endianness =3D DEVICE_LITTLE_ENDIAN, + .impl =3D { + .min_access_size =3D 1, + .max_access_size =3D 8, + .unaligned =3D true, + }, .valid =3D { .min_access_size =3D 1, .max_access_size =3D 8, --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012971; cv=none; d=zohomail.com; s=zohoarc; b=lTEcI1mpwPzMzmBH1hpR3Tax+rAmNxEl6g5Q+1oMwnLY1cen0u/wfjPo5Jwl0KtYRVjLIr3RffHpi+YF2CO5q94SAsVoypwHYpTKxS3p6LEp2fBw8A3G2IUBiXBQmdAP8whtAzmoN2Rx/il3lC5dGlzDGDXZhg2JsWaptf3HQ5Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012971; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=J6hDMSlX8QSa9powGt5inke+kVsGYViV0WMedvJhWK4=; b=DyngdoQIzxOWbLGkJbK7z82wxmcIj46XNMWjmIUMFNE18OhYErDtKk5HaBryl69hjXTBSaFJwigiSkhJzibBYzBVfOYXNw2GN7Kr8KeQg4fqBlTxKaBOwi7zLmA0HNKqX9WQPch4fYRWIzD8Urscmkk2iDIbTziALNOZ58hONOw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17780129713101002.3982022452344; Tue, 5 May 2026 13:29:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMP-0004di-EZ; Tue, 05 May 2026 16:27:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMML-0004ZQ-MB for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMJ-0002fZ-I7 for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:13 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-675-pYTfOn0vPBS15ph958ChYA-1; Tue, 05 May 2026 16:27:09 -0400 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-8b459fa5f76so77892786d6.1 for ; Tue, 05 May 2026 13:27:09 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.27.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012830; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J6hDMSlX8QSa9powGt5inke+kVsGYViV0WMedvJhWK4=; b=Rn+lBjVF/Gr8El9kFMc401e8T+dxIxIlP941MVAZy38f2zP9dCSVNZJv2EIFRr0dO6+4H/ k6MXk9vJ4FCzBQUncMPv1Va3QupbncLTsJonne2uTQ2knAICjRSszpw6ybk+u1+qzmFGvD p02PsNy28tuEPTMVSQBirbeEtekS/70= X-MC-Unique: pYTfOn0vPBS15ph958ChYA-1 X-Mimecast-MFC-AGG-ID: pYTfOn0vPBS15ph958ChYA_1778012829 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012829; x=1778617629; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=J6hDMSlX8QSa9powGt5inke+kVsGYViV0WMedvJhWK4=; b=PFUhKFcOHtAZ6QgRevYeTULJ7REDN+BtBVe2+yFkjxscxWE6hCwqH3lyJrVCa4UopC n/tzodjXhouMZnR/ohePc6zx5CCzURQDvYVjwvoZIQnaZ9xULdE0C91Tg65GhhZmWK68 N72j/XE5zRZnz/JlpZtWa9jEYZKUKEtudUkHrik2RVVV29DZm87MQJfl7+9EY9swBp2K WGywnbjIi4e1nZb8m7CYXTl/uAiX6Z7Z4H+v6gFaJR9G2Y/+kGRHjBFHR1dKPjaBobdS vLGbX/w9T0CTIl6DR/izcmwoh6QzSN3e3x7c5Zp255G8RnhBoRNY1bHtzUIUBMcIezhZ kNzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012829; x=1778617629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=J6hDMSlX8QSa9powGt5inke+kVsGYViV0WMedvJhWK4=; b=kKuwAyVrp0fdngfRyA6ynGtiXrejSzcvE8ZVEmuFJSmH/Tbx6TyaT6S2c0YIB8hvP8 0az82N8atXureYlqLte2I3/8slPjdUDQLaTyUhabRq+fVbY5hhH99dSKwG22zMqlm1dQ SgxzpKGwcCU6uo68xoJH1Hp/V81vrjs68O47AI4ac3yxQELksCWyj42hKzmq54fqnzp+ xfEXnhKuoDDx1k1GqJKCJXM+V3zWyxyuRGo2YoaRQ2lls+VpGebla/0GKrRa0ocvMf2C tT1UxKvOTDDTbJdpF8aam5A792JiC4JzJSXwmCDMcY/GR8PtT7HG0l9xsUY43Al/ZJL6 aRlQ== X-Gm-Message-State: AOJu0Yzm3zJ6rb3xG7DxBXwG1gadTq5bmRNH2x1LXChFtL/opQmit/uZ sdIfvV26V/LspbcCb3bMVJEg9z38GW9YBTJS0ZWczh3YJJJM4+5+LDXZmfZShq172TiE9FxzYK+ 4f0uTewTQkQSoCUpdv4fqJh+ZqRROd+wUjrq2u/7QtWVcCCP/AHo1PJH7H0g1rCtHlExMivWsSh p8LNZz5tYXJKPjtZ6MiVKS3122seryGMc6yBTVgQ== X-Gm-Gg: AeBDiethUJoRYPdgMYOb42xzi9TbYFqohF/On4tpECoXxU4EwI9BrLCCYrojnBxaIXr VIFLU4MeIdTFxwqvNVqx9uOwPNpa4hglq53N6mEwTbv9vD6uujZAio6LM+aVT1TdrAWaLNNs/th XoxZMtx79PGcrQ38P2r5t1jLiQbzEDrQnVXseQpeTQxWAxdufB8c8ikEXQH83l7QFHr+qT8e/tM 0vDjf6mzlTzXciTjHialk+H5tNMEadhrSHUGW68LPocCCyX8mMguUSobWqSs/7kHk66YbRrUtI8 X+MgVXC6gpqPW3yM0pQf+ZUYUoRQjsbb4Ta83zXJrxT01fOSOTAeJKmHSWsk/f5vNk2HE/+Ic36 vcL83Tkbn/U6jCVS4o9El76koQsFMNNT8J3P18ifzi3nXV+oVADNjp9A= X-Received: by 2002:a05:6214:c4f:b0:8a1:8f74:3eab with SMTP id 6a1803df08f44-8bc4249d7a3mr5224056d6.9.1778012828747; Tue, 05 May 2026 13:27:08 -0700 (PDT) X-Received: by 2002:a05:6214:c4f:b0:8a1:8f74:3eab with SMTP id 6a1803df08f44-8bc4249d7a3mr5223336d6.9.1778012827948; Tue, 05 May 2026 13:27:07 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , Peter Maydell , CJ Chen , Max Filippov Subject: [PULL 21/23] hw/xtensa/mx_pic: Specify xtensa_mx_pic_ops .impl settings Date: Tue, 5 May 2026 16:26:38 -0400 Message-ID: <20260505202640.1011006-22-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012972670158500 Content-Type: text/plain; charset="utf-8" From: Peter Maydell The xtensa mx-pic interrupt controller has a rather odd register setup, where some registers are 32 bits but are decoded at offsets only one apart from each other. The QEMU implementation handles this correctly, but it did not set .impl.unaligned =3D true. This has worked up til now because QEMU has entirely ignored .impl.unaligned, and just allowed through unaligned accesses when .valid.unaligned is set. To allow the possibility of properly implementing synthesis of unaligned accesses by the memory subsystem when they are valid but the device doesn't implement them, and for clarity of intention, state explicitly that this MR's read and write functions directly handle unaligned accesses, by setting .impl.unaligned =3D true. While we are adjusting the MemoryRegionOps, we set also the minimum and maximum allowed access sizes. Since the only way to get at this device is via the CPU's RER and WER instructions, which always operate at 32-bit sizes (see the HELPER(rer) and HELPER(wer) functions in target/xtensa/op_helper.c), we know we will always get 32-bit accesses. Specify explicitly that that is what is valid and implemented for the MR. Add a comment to clarify that the hardware behaviour here is not "true memory-mapped registers", so the odd-looking implementation is correct. Based-on-a-patch-by: CJ Chen Signed-off-by: Peter Maydell Reviewed-by: Max Filippov Link: https://lore.kernel.org/r/20260428093339.2087081-4-peter.maydell@lina= ro.org Signed-off-by: Peter Xu --- hw/xtensa/mx_pic.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/hw/xtensa/mx_pic.c b/hw/xtensa/mx_pic.c index 07c3731aef..098c1aaf85 100644 --- a/hw/xtensa/mx_pic.c +++ b/hw/xtensa/mx_pic.c @@ -69,6 +69,26 @@ struct XtensaMxPic { } cpu[MX_MAX_CPU]; }; =20 +/* + * Note that decode for these registers is rather strange by the usual + * MMIO standards -- the MIROUT and MIPICAUSE areas can be read and + * written at 32-bit length, returning different values for each byte + * offset, because the low bits of the address are treated as selecting + * an IRQ or a processor: + * + * 00nn 0...0p..p Interrupt Routing, route IRQ n to processo= r p + * 01pp 0...0d..d 16 bits (d) 'ored' as single IPI to proces= sor p + * + * This is because (like x86 IO port in/out accesses) the offset is + * not a memory-mapped address but is really a register number, + * accessed via the Xtensa RER/WER "external register" instructions. + * + * We set .valid and .impl to both allow unaligned =3D true to permit + * these byte-offsets. Because this device is not a true memory mapped + * device but is accessible only via the Xtensa RER/WER "external + * register" interface, all accesses are guaranteed 32 bits. + */ + static uint64_t xtensa_mx_pic_ext_reg_read(void *opaque, hwaddr offset, unsigned size) { @@ -267,7 +287,14 @@ static const MemoryRegionOps xtensa_mx_pic_ops =3D { .read =3D xtensa_mx_pic_ext_reg_read, .write =3D xtensa_mx_pic_ext_reg_write, .endianness =3D DEVICE_NATIVE_ENDIAN, + .impl =3D { + .min_access_size =3D 4, + .max_access_size =3D 4, + .unaligned =3D true, + }, .valid =3D { + .min_access_size =3D 4, + .max_access_size =3D 4, .unaligned =3D true, }, }; --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778013004; cv=none; d=zohomail.com; s=zohoarc; b=l0eRCm1dpB72LuHhjdrv+DC8qP1462zj80sZ3zucX9/QU+HmpS8KEaQ87dguHrsaGZpVj4Jm+lhTOLiXGwKd6PrG0MpnLppas8eJpfJJKuFlHW2angIFwnPRApmOa6TvL1L1/CrGvgn7HYOXYYyUGO68/q6HRiRky/+IYBkvhrk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778013004; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Dyb96gHjBVyrb1hhDEwJgJ8h7wgrV0AQCUFZ7NVQicg=; b=ATUqNSfTpvp75iqWVht50TedatHVr8vepISY0EJPosWsNQnNZc8HOexSuSA4SdXGNnuMuRlhd1IlYFqLDsogTA51sgJJfEhX5sdBOxFNLSrqWQFnboLM/W2Wy3ZvKWlmbkVPkyIqKIeLDi0DdG7FLB2YUnI0lXcP+tW7/WUNLPk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778013004397465.20241301315207; Tue, 5 May 2026 13:30:04 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMO-0004c7-5K; Tue, 05 May 2026 16:27:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMM-0004bI-RV for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMML-0002ft-32 for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:14 -0400 Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-402-cEsicOsgNryq82rIoSbxhA-1; Tue, 05 May 2026 16:27:10 -0400 Received: by mail-qk1-f197.google.com with SMTP id af79cd13be357-8eb04352924so815313085a.1 for ; Tue, 05 May 2026 13:27:10 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.27.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012831; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Dyb96gHjBVyrb1hhDEwJgJ8h7wgrV0AQCUFZ7NVQicg=; b=TsEi5XDOGopZC7vkTqBl7NmTsFM6NeRcdKKSoV9M/3YimwwLZ/XKBPrqGr5hlajr3TF6+R 2ahwGRrI/R+nHct1G4yq9q36lWHyjXDCoacKwrbSQxoIruhy6bejlfMr3DYCVlfiNHbip+ yWyEqomA+jLXJdOas+V/+3Jy2jDipwk= X-MC-Unique: cEsicOsgNryq82rIoSbxhA-1 X-Mimecast-MFC-AGG-ID: cEsicOsgNryq82rIoSbxhA_1778012830 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012830; x=1778617630; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Dyb96gHjBVyrb1hhDEwJgJ8h7wgrV0AQCUFZ7NVQicg=; b=WzkNo6eWmEkArALze6qYwMFf/l+CZaA+KOgP6mDS+cJJPU8I/xEB2FIZi82ghLQRdS e9Ei1GpFn/YHb6EWzBnO9ylfCm6xydmnnitoMA4tvH2GpHKCOAon6Sh0AoV3oMLbcwTz k7M0GM/txAt8mDEA3b7Av05EXJW009chSbl5s9FRvnXh30dIUTFhonWoHxD6P8cj732S h4ytZYWqU9jjUOs7qSMo++uiu2M29KhmvVU5Qmi6hdzMdaisdwt8HhEgu5Go4yJuc+O6 X+WZ379JDHW/O+9lh4r2fB2J6b2r9W8yR5WZ/iqy2gXhv3UkeVHW6zuAEksr+gRjvSuk as4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012830; x=1778617630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Dyb96gHjBVyrb1hhDEwJgJ8h7wgrV0AQCUFZ7NVQicg=; b=IDuZWkGF1dNxVoRTmJOqS5U1MjMGGGXp91CLxgcqjCetyy8F9NZhF8QM5IL5f+ENqa FLuEbgY+ZVzV4d/2ZYiBOjF/kiozAHc/8FXjC5yEjB5+iWarRRd2oMGS4W+mioKY3KFp irX+63aj7ojRwWpXugOtMJbMP9er24TewNkNqJe2un4xveFBs0ojo904bCFCby34iJSu jSykx+inhIZgLTpEwrdPTsmQrXA53WXFAwVJat9fLRC3Alea/x2FncwedIk9oFbSh8XS kGzz7yAN976YcE0p8jJMKevvwqKp4D0iBEodJ6wBPHlQZdL2vpAo8aS1pxnXk42UqtYX oRdQ== X-Gm-Message-State: AOJu0YwM3IydTf+UasrAmSD5hDtmH9kTRGaoIC1O6GNX5AtbVumG1QRK sMXYxpyfCOSpuc7s2t2xYD9F6NTu6wbtSklAbtZrGFN2lJXU96kBnklxlCpS45q0FOl21ke5Tho 7Sb8SFS9O5gpBedGMovRyXIQbKZNy8zMoN3PllPI/c8+yF7wKRyC6TMIUZtu+eczymfcql5k6o0 ouGkbPTJql+joBcgxB2sL1e1PMRgSe3bWZNeQ44g== X-Gm-Gg: AeBDiev58QDCyZpWQ474KSrTQkck/ppus2ayHxCWzNgyVfRhRzvQHKyd/dVM49DDrt+ pLzqH2dYO8azw6u3BbPYJKUpevQGnNM4ns5n7xZWBwMV9MG0Wfd3sfTDH/WtdYLncPnQaQ8j24T ukNBhdwLP6ZYVyrhg2Ffi1Ux0lS+BzGi0DN/Q3mus44bwn2CN3A7JClWhzEa2AhjOo8vee25pGi AD7t2ulTCPpZ36A+UbtF75pgu66DhY7hXqXT4IBMl2n+QDrMwzB8+yfc2NlC6T4BgsMhFFqgfEo MWSOQbmVtbbbX/qP3j2RvAQfM9/g5/5SihveBL70sVRIf5tIpvyappVQVFsyHZmT8Z7aTwMO40g XlIdzbvBMaBnHavWNyLEfwisLfmsuN/u6v+0o2ihb7NDk1YWf12pjXmk= X-Received: by 2002:a05:620a:2804:b0:8e8:bedd:14b2 with SMTP id af79cd13be357-904d69d8de7mr101666985a.43.1778012829880; Tue, 05 May 2026 13:27:09 -0700 (PDT) X-Received: by 2002:a05:620a:2804:b0:8e8:bedd:14b2 with SMTP id af79cd13be357-904d69d8de7mr101658685a.43.1778012829095; Tue, 05 May 2026 13:27:09 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu , CJ Chen , Tomoyuki Hirose , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Peter Maydell Subject: [PULL 22/23] system/memory: assert on invalid MemoryRegionOps .unaligned combo Date: Tue, 5 May 2026 16:26:39 -0400 Message-ID: <20260505202640.1011006-23-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778013006316154100 From: CJ Chen When it comes to this pattern: .valid.unaligned =3D false and impl.unaligned =3D true, is effectlvely contradictory. The .valid structure indicates that unaligned access should be rejected at the access validation phase, yet .impl suggests the underlying device implementation can handle unaligned operations. As a result, the upper-layer code will never even reach the .impl logic. Add an assertion that the MemoryRegionOps doesn't specify this invalid combination. Signed-off-by: CJ Chen Tested-by: CJ Chen Suggested-by: Peter Xu Acked-by: Tomoyuki Hirose Reviewed-by: Philippe Mathieu-Daud=C3=A9 [PMM: tweaked commit message] Reviewed-by: Peter Maydell Signed-off-by: Peter Maydell Link: https://lore.kernel.org/r/20260428093339.2087081-5-peter.maydell@lina= ro.org Signed-off-by: Peter Xu --- system/memory.c | 1 + 1 file changed, 1 insertion(+) diff --git a/system/memory.c b/system/memory.c index 225bbe38c3..739ba11da6 100644 --- a/system/memory.c +++ b/system/memory.c @@ -1573,6 +1573,7 @@ void memory_region_init_io(MemoryRegion *mr, Object *= owner, const MemoryRegionOps *ops, void *opaque, const char *name, uint64_t size) { + g_assert(!ops || !(ops->impl.unaligned && !ops->valid.unaligned)); memory_region_init(mr, owner, name, size); memory_region_set_ops(mr, ops, opaque); } --=20 2.53.0 From nobody Wed May 6 14:30:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1778012956; cv=none; d=zohomail.com; s=zohoarc; b=KlvfylBOQupx6iQy5GuZ+4Q3Jb7NK4jz2FrubST0r28U0ZOkqJBUDvxwXxT06upbe3dIveb2ZEkOSDVQe/zG9ZnHOOVbkp8YB7eUzasX8IB2kaAeIPpnLrSJaNW0cy962Dk4GKaoLbCj33Z/nXOpTQ1AY0s+L36NVPI1aDDc0gA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778012956; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=p8kl5RPBsTaXnbl56L2/sgreoYVs8m0Akjsf9qzEtBw=; b=nfHveJ73IVYsBZjo7LOan4l4XMdfFzYHkjnU9CVb4MT1ydiI3Ou72o09tQNQSeOwCgVFcF2/Av7iq2wlzOAvfxdXlMjjpV3+tBlCp+gPd3o62TBEb4VitYInLLJkabOFbz4RDs2z+fZDL8JULzaBHFZE+o4vjjif10Lk6SZsv4E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1778012956649704.8672275897893; Tue, 5 May 2026 13:29:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wKMMW-0004qL-0y; Tue, 05 May 2026 16:27:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMT-0004p9-V2 for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wKMMO-0002h0-O5 for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:21 -0400 Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-235--eEBYiduNYK0Ry66TzLWZA-1; Tue, 05 May 2026 16:27:14 -0400 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-8b513f57611so134089266d6.2 for ; Tue, 05 May 2026 13:27:14 -0700 (PDT) Received: from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.27.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2026 13:27:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778012836; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p8kl5RPBsTaXnbl56L2/sgreoYVs8m0Akjsf9qzEtBw=; b=TtThZKb5kIZ/z5ORNRI8IdT9WJQstWV5Me8g+pRisUmX39W3jSjH2/IE8IIXEytTUUpPRC OZGTAlwtzntheecGopa9JQQPoNYCo/VgAHYXNpVeX4Hpt0vGrvVb/bSNwLFhIUquFulkYc SWuvf7p0RQGEviuwa8QTYT3kfLCjKNc= X-MC-Unique: -eEBYiduNYK0Ry66TzLWZA-1 X-Mimecast-MFC-AGG-ID: -eEBYiduNYK0Ry66TzLWZA_1778012834 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778012833; x=1778617633; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p8kl5RPBsTaXnbl56L2/sgreoYVs8m0Akjsf9qzEtBw=; b=DYHRsPakpfrDepk82tPpOS8VCizm9lMR1VWWzMVLP9Jwu3av3myuBH1PQTxmK0RHhT YVfshLiuqCveYDtLSQKDPM407iHpIFbS57VwdTg0cYU/0fBHc71dCBzmBN9L6UELGS+e jNYZkum3NTlDz7ceTczs+t7rZ+L3W4OUcHWrs8paBBW5oIOtXKJyi0Rd2pDKQ6t5Dy/j 8+iSL/pVfhdsjvz49dEYUrrJeFw4KsHSnNDXICeu63bzf0V0CAZ6NgNX94bRvH9Y8ZZo FdlPE1e22o/ZGFrgQMXqn+/xs1oWxpdRReu2DmPZ0MiUCpzTeYxaLBVoSKFzrUIWTNwI HppQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778012833; x=1778617633; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=p8kl5RPBsTaXnbl56L2/sgreoYVs8m0Akjsf9qzEtBw=; b=bpZmxns6dszl9ETJz53ZdZx/KGSwlSD05i+oR8FPLQpxi5O4w3BjKiO17Z/IVSMo2F t1DqhnFr6RQR3DGaABm9UysXbV3NQzvlAUGnnt/0oWcE9O4j8n2K1624W0BRwStSVD8Y n71PqJbQe0/3kQbIio7qQP0VkQJPRzvEgKVsArhHIu3cdR3Q/Yj7NybniUVNiHzO+6Wp IlX6AWLGU8zM6eCJJo/GVvWGOCT8DcGU5KcAIUU+EcPxF0rkLWQtELd+yS75WKtElMbd VWwEQxArcrOTYOveB1J01b8RbOlPEcCc643lXgBQj6mk3Q2eT/U0xbgyGFIJSF8RUR5e 4KoA== X-Gm-Message-State: AOJu0YydecmN+7OYs7eERQI9LbT0oFN033x5rPFkx3FlM37EUruejks3 S/G0NatcWUkl9qGRO3tZvpgvDiFVLuJCaQ/l0l864NSAzVp+Z7yb5gbN6gTmo7X9chG5fZbvu6i JVU0bRSDU3k4tl7g81XGpkwc46IyAmBoljxFU171FqqnY9n+mmUxxCyeM1SRR6TgLxhrO/HPCIa n9n8/dHr6rP/oou40tvMfpM8kF0sltqd+EIq+EWw== X-Gm-Gg: AeBDiesXg64ec/LOLF1fP8H9KMW78roH0LwuNhvbZ04fFS9tPuig449KMrguFm9Bc/I IPsn/55KJ+9/ljhSDHWrEy7MI3ukbwI4A4dJtt/epwtOcsaT9cYnbTzOfFTcXkLz2yzchTgiYYo qYB7L/wWQo3i6m3BTbCCEjzBkchO/SjUBYn43CvAP35FLjTlPEMlfPirlmjYDuSPQs6G2hpx8IY tV56yS7owRm91rjQ2C0dye23qwisxRZW8U95EsZB/kMhv8wFAsp2ZyfmHXn+hD7trAmZCQVVk+q F4/SbbOVdZubP2VBbyWJArD1iVDun5TpgYpigz7AnQ3fZPgsBtyqwhIj94hy84N6SogM4bnq97Q 8CT/KBxCf+pqQXNPnxaGJW3b5Kdi2j+9GE7ae2/4bVQb+o2weOUBETl4= X-Received: by 2002:a05:6214:4c87:b0:8ac:b2b2:b2f0 with SMTP id 6a1803df08f44-8bc427833famr4681956d6.1.1778012832184; Tue, 05 May 2026 13:27:12 -0700 (PDT) X-Received: by 2002:a05:6214:4c87:b0:8ac:b2b2:b2f0 with SMTP id 6a1803df08f44-8bc427833famr4680386d6.1.1778012830267; Tue, 05 May 2026 13:27:10 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Paolo Bonzini , Peter Xu Subject: [PULL 23/23] tests/qtest/migration: Fix A-B file build Date: Tue, 5 May 2026 16:26:40 -0400 Message-ID: <20260505202640.1011006-24-peterx@redhat.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260505202640.1011006-1-peterx@redhat.com> References: <20260505202640.1011006-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1778012957658154100 Content-Type: text/plain; charset="utf-8" From: Fabiano Rosas The generation of the guest workloads for migration-test is all broken: 1) We moved code around and forgot to update the includes from the guest workloads at qtest/migration//. 2) Since this code doesn't build by default due to the need for cross compilation, we also broke the s390x code entirely by removing the libc implementation from sclp.c. 3) The build adds a blank line at the end of the header files due to how the sed command used to remove extra variable declarations is written. 4) Must be too long since we last built this thing that powerpc compilers would output BE by default. Nowadays we need -mbig-endian to force it. 5) Format of comments needs fixing to please checkpatch. Fix everything. Most changes are trivial, except for s390x where I had to reimplement some of the routines from sclp.c. Compiling that object file in doesn't work because of its various includes that this standalone code can't provide. Testing this change (assuming x86 host): $ cd build $ pushd tests/qtest/migration $ make clean $ make i386 $ make CROSS_PREFIX=3Dyour-favorite-cross-compiler- s390x $ make CROSS_PREFIX=3Dyour-favorite-cross-compiler- aarch64 $ make CROSS_PREFIX=3Dyour-favorite-cross-compiler- ppc64 $ popd $ make -j$(nproc) # must build the tests again $ QTEST_QEMU_BINARY=3D./qemu-system- migration-test Fixes: 9f4278837d ("pc-bios/s390-ccw: Use the libc from SLOF and remove scl= p prints") Fixes: e1803dabdc ("tests/qtest/migration: Move common test code") Signed-off-by: Fabiano Rosas Link: https://lore.kernel.org/r/20260429231025.30818-1-farosas@suse.de Signed-off-by: Peter Xu --- tests/qtest/migration/aarch64/a-b-kernel.h | 7 +- tests/qtest/migration/bootfile.h | 11 + tests/qtest/migration/i386/a-b-bootblock.h | 8 +- tests/qtest/migration/ppc64/a-b-kernel.h | 8 +- tests/qtest/migration/s390x/a-b-bios.h | 611 +++++++++++++-------- tests/qtest/migration/s390x/a-b-bios.c | 66 ++- tests/qtest/migration/Makefile | 11 +- tests/qtest/migration/aarch64/Makefile | 4 +- tests/qtest/migration/aarch64/a-b-kernel.S | 2 +- tests/qtest/migration/i386/Makefile | 4 +- tests/qtest/migration/i386/a-b-bootblock.S | 2 +- tests/qtest/migration/ppc64/Makefile | 6 +- tests/qtest/migration/ppc64/a-b-kernel.S | 2 +- tests/qtest/migration/s390x/Makefile | 8 +- 14 files changed, 492 insertions(+), 258 deletions(-) diff --git a/tests/qtest/migration/aarch64/a-b-kernel.h b/tests/qtest/migra= tion/aarch64/a-b-kernel.h index 34e518d061..c444cdaf73 100644 --- a/tests/qtest/migration/aarch64/a-b-kernel.h +++ b/tests/qtest/migration/aarch64/a-b-kernel.h @@ -1,6 +1,7 @@ -/* This file is automatically generated from the assembly file in - * tests/migration/aarch64. Edit that file and then run "make all" - * inside tests/migration to update, and then remember to send both +/* + * This file is automatically generated from the assembly file in + * tests/qtest/migration/aarch64. Edit that file and then run "make" + * inside tests/qtest/migration to update, and then remember to send both * the header and the assembler differences in your patch submission. */ unsigned char aarch64_kernel[] =3D { diff --git a/tests/qtest/migration/bootfile.h b/tests/qtest/migration/bootf= ile.h index 96e784b163..0ce5b34433 100644 --- a/tests/qtest/migration/bootfile.h +++ b/tests/qtest/migration/bootfile.h @@ -8,6 +8,15 @@ #ifndef BOOTFILE_H #define BOOTFILE_H =20 +/* + * This file is included from qtest, but also from the assembly code + * that generates the header file containing the guest workload. Don't + * expose the function declarations and non-standard types to it. + */ +#ifdef __ASSEMBLER__ +#define MIGRATION_GUEST_CODE +#endif + /* Common */ #define TEST_MEM_PAGE_SIZE 4096 =20 @@ -33,8 +42,10 @@ */ #define ARM_TEST_MAX_KERNEL_SIZE (512 * 1024) =20 +#ifndef MIGRATION_GUEST_CODE void bootfile_delete(void); char *bootfile_create(const char *arch, const char *dir, bool suspend_me); char *bootfile_get(void); +#endif =20 #endif /* BOOTFILE_H */ diff --git a/tests/qtest/migration/i386/a-b-bootblock.h b/tests/qtest/migra= tion/i386/a-b-bootblock.h index c83f8711db..3b5de60161 100644 --- a/tests/qtest/migration/i386/a-b-bootblock.h +++ b/tests/qtest/migration/i386/a-b-bootblock.h @@ -1,6 +1,7 @@ -/* This file is automatically generated from the assembly file in - * tests/migration/i386. Edit that file and then run "make all" - * inside tests/migration to update, and then remember to send both +/* + * This file is automatically generated from the assembly file in + * tests/qtest/migration/i386. Edit that file and then run "make" + * inside tests/qtest/migration to update, and then remember to send both * the header and the assembler differences in your patch submission. */ unsigned char x86_bootsect[] =3D { @@ -48,7 +49,6 @@ unsigned char x86_bootsect[] =3D { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa }; - #define SYM_do_zero 0x00007c3d #define SYM_gdt 0x00007ca0 #define SYM_gdtdesc 0x00007cb8 diff --git a/tests/qtest/migration/ppc64/a-b-kernel.h b/tests/qtest/migrati= on/ppc64/a-b-kernel.h index 673317efdb..8dc2bcd61a 100644 --- a/tests/qtest/migration/ppc64/a-b-kernel.h +++ b/tests/qtest/migration/ppc64/a-b-kernel.h @@ -1,6 +1,7 @@ -/* This file is automatically generated from the assembly file in - * tests/migration/ppc64. Edit that file and then run "make all" - * inside tests/migration to update, and then remember to send both +/* + * This file is automatically generated from the assembly file in + * tests/qtest/migration/ppc64. Edit that file and then run "make" + * inside tests/qtest/migration to update, and then remember to send both * the header and the assembler differences in your patch submission. */ unsigned char ppc64_kernel[] =3D { @@ -39,4 +40,3 @@ unsigned char ppc64_kernel[] =3D { 0x38, 0xa0, 0x00, 0x01, 0x38, 0xc0, 0x00, 0x42, 0x78, 0xc6, 0xc1, 0xc6, 0x44, 0x00, 0x00, 0x22, 0x4b, 0xff, 0xff, 0xcc }; - diff --git a/tests/qtest/migration/s390x/a-b-bios.h b/tests/qtest/migration= /s390x/a-b-bios.h index 96103dadbb..f8e385c75c 100644 --- a/tests/qtest/migration/s390x/a-b-bios.h +++ b/tests/qtest/migration/s390x/a-b-bios.h @@ -1,15 +1,16 @@ -/* This file is automatically generated from the a-b-bios.c file in - * tests/migration/s390x. Edit that file and then run "make all" - * inside tests/migration to update, and then remember to send both +/* + * This file is automatically generated from the assembly file in + * tests/qtest/migration/s390x. Edit that file and then run "make" + * inside tests/qtest/migration to update, and then remember to send both * the header and the assembler differences in your patch submission. */ unsigned char s390x_elf[] =3D { 0x7f, 0x45, 0x4c, 0x46, 0x02, 0x02, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x16, 0x00, 0x00, 0x00, 0x01, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xa8, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x09, 0x80, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x98, 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x00, 0x38, 0x00, 0x07, 0x00, 0x40, - 0x00, 0x0d, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x04, + 0x00, 0x10, 0x00, 0x0f, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x88, 0x00, 0x00, 0x00, 0x00, @@ -21,171 +22,86 @@ unsigned char s390x_elf[] =3D { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x07, 0xac, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x07, 0xac, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x88, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x88, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x06, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x07, 0xb8, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x17, 0xb8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x17, 0xb8, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x38, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x01, 0x18, 0x48, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0e, 0xd8, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x1e, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1e, 0xd8, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x30, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x01, 0x21, 0x28, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x07, 0xb8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x17, 0xb8, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x17, 0xb8, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x01, 0x20, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x20, + 0x00, 0x00, 0x0e, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1e, 0xd8, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1e, 0xd8, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x01, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x64, 0x74, 0xe5, 0x51, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x64, 0x74, 0xe5, 0x52, 0x00, 0x00, 0x00, 0x04, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x07, 0xb8, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x17, 0xb8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x17, 0xb8, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x38, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x01, 0x38, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0e, 0xd8, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x1e, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1e, 0xd8, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x28, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x01, 0x28, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x2f, 0x6c, 0x69, 0x62, 0x2f, 0x6c, 0x64, 0x36, 0x34, 0x2e, 0x73, 0x6f, - 0x2e, 0x31, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, - 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x2e, 0x31, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x02, 0x48, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x03, 0xa8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0c, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0xf0, 0xeb, 0xef, 0xf0, 0x70, - 0x00, 0x24, 0xa7, 0xfb, 0xff, 0x60, 0xc0, 0xe5, 0x00, 0x00, 0x01, 0x5f, - 0xc0, 0x20, 0x00, 0x00, 0x02, 0xa8, 0xc0, 0xe5, 0x00, 0x00, 0x01, 0x75, - 0xa5, 0x2e, 0x00, 0x10, 0xa7, 0x19, 0x63, 0x00, 0x92, 0x00, 0x20, 0x00, - 0xa7, 0x2b, 0x10, 0x00, 0xa7, 0x17, 0xff, 0xfc, 0xa5, 0x1e, 0x00, 0x10, - 0xa7, 0x29, 0x63, 0x00, 0xe3, 0x30, 0x10, 0x00, 0x00, 0x90, 0xa7, 0x3a, - 0x00, 0x01, 0x42, 0x30, 0x10, 0x00, 0xa7, 0x1b, 0x10, 0x00, 0xa7, 0x27, - 0xff, 0xf7, 0xc0, 0x20, 0x00, 0x00, 0x02, 0x8a, 0xc0, 0xe5, 0x00, 0x00, - 0x01, 0x56, 0xa7, 0xf4, 0xff, 0xeb, 0x07, 0x07, 0xc0, 0xf0, 0x00, 0x00, - 0x4e, 0x5c, 0xc0, 0x20, 0x00, 0x00, 0x00, 0x7d, 0xe3, 0x20, 0x20, 0x00, - 0x00, 0x04, 0xc0, 0x30, 0x00, 0x00, 0x96, 0xa3, 0xb9, 0x0b, 0x00, 0x32, - 0xb9, 0x02, 0x00, 0x33, 0xa7, 0x84, 0x00, 0x19, 0xa7, 0x3b, 0xff, 0xff, - 0xeb, 0x43, 0x00, 0x08, 0x00, 0x0c, 0xb9, 0x02, 0x00, 0x44, 0xb9, 0x04, - 0x00, 0x12, 0xa7, 0x84, 0x00, 0x09, 0xd7, 0xff, 0x10, 0x00, 0x10, 0x00, - 0x41, 0x10, 0x11, 0x00, 0xa7, 0x47, 0xff, 0xfb, 0xc0, 0x20, 0x00, 0x00, - 0x00, 0x0d, 0x44, 0x30, 0x20, 0x00, 0xc0, 0x20, 0x00, 0x00, 0x00, 0x5b, - 0xd2, 0x0f, 0x01, 0xd0, 0x20, 0x00, 0xa7, 0xf4, 0xff, 0xa1, 0xd7, 0x00, - 0x10, 0x00, 0x10, 0x00, 0xc0, 0x10, 0x00, 0x00, 0x00, 0x50, 0xb2, 0xb2, - 0x10, 0x00, 0xa7, 0xf4, 0x00, 0x00, 0xeb, 0x00, 0xf0, 0x00, 0x00, 0x25, - 0x96, 0x02, 0xf0, 0x06, 0xeb, 0x00, 0xf0, 0x00, 0x00, 0x2f, 0xc0, 0x10, - 0x00, 0x00, 0x00, 0x2a, 0xe3, 0x10, 0x01, 0xb8, 0x00, 0x24, 0xc0, 0x10, - 0x00, 0x00, 0x00, 0x4b, 0xd2, 0x07, 0x01, 0xb0, 0x10, 0x00, 0xc0, 0x10, - 0x00, 0x00, 0x00, 0x3d, 0xb2, 0xb2, 0x10, 0x00, 0xeb, 0x66, 0xf0, 0x00, - 0x00, 0x25, 0x96, 0xff, 0xf0, 0x04, 0xeb, 0x66, 0xf0, 0x00, 0x00, 0x2f, - 0xc0, 0x10, 0x00, 0x00, 0x00, 0x1a, 0xe3, 0x10, 0x01, 0xf8, 0x00, 0x24, - 0xc0, 0x10, 0x00, 0x00, 0x00, 0x36, 0xd2, 0x07, 0x01, 0xf0, 0x10, 0x00, - 0xc0, 0x10, 0x00, 0x00, 0x00, 0x24, 0xb2, 0xb2, 0x10, 0x00, 0xeb, 0x00, - 0xf0, 0x00, 0x00, 0x25, 0x94, 0xfd, 0xf0, 0x06, 0xeb, 0x00, 0xf0, 0x00, - 0x00, 0x2f, 0x07, 0xfe, 0xeb, 0x66, 0xf0, 0x00, 0x00, 0x25, 0x94, 0x00, - 0xf0, 0x04, 0xeb, 0x66, 0xf0, 0x00, 0x00, 0x2f, 0x07, 0xfe, 0x07, 0x07, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0xf0, 0x00, 0x02, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x07, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x70, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x30, 0x00, + 0xeb, 0xdf, 0xf0, 0x68, 0x00, 0x24, 0xc0, 0xd0, 0x00, 0x00, 0x01, 0x05, + 0xa7, 0xfb, 0xff, 0x60, 0xc0, 0x10, 0x00, 0x00, 0x96, 0xc0, 0xa7, 0x28, + 0x00, 0x1c, 0x40, 0x20, 0x10, 0x00, 0xa5, 0x2c, 0x00, 0x04, 0xe3, 0x20, + 0x10, 0x0a, 0x00, 0x24, 0xa7, 0x28, 0x00, 0x40, 0x40, 0x20, 0x10, 0x12, + 0x58, 0x20, 0xd0, 0x00, 0xb2, 0x20, 0x00, 0x21, 0xb2, 0x22, 0x00, 0x30, + 0x88, 0x30, 0x00, 0x1c, 0xc0, 0xe5, 0x00, 0x00, 0x00, 0x63, 0xa7, 0x29, + 0x00, 0x41, 0xc0, 0xe5, 0x00, 0x00, 0x00, 0xbf, 0xa5, 0x2e, 0x00, 0x10, + 0xa7, 0x19, 0x63, 0x00, 0x92, 0x00, 0x20, 0x00, 0xa7, 0x2b, 0x10, 0x00, + 0xa7, 0x17, 0xff, 0xfc, 0xa5, 0x1e, 0x00, 0x10, 0xa7, 0x29, 0x63, 0x00, + 0xe3, 0x30, 0x10, 0x00, 0x00, 0x90, 0xa7, 0x3a, 0x00, 0x01, 0x42, 0x30, + 0x10, 0x00, 0xa7, 0x1b, 0x10, 0x00, 0xa7, 0x27, 0xff, 0xf7, 0xa7, 0x29, + 0x00, 0x42, 0xc0, 0xe5, 0x00, 0x00, 0x00, 0xa1, 0xa7, 0xf4, 0xff, 0xec, + 0xc0, 0xf0, 0x00, 0x00, 0x56, 0x30, 0xc0, 0x20, 0x00, 0x00, 0x0e, 0x7d, + 0xe3, 0x20, 0x20, 0x00, 0x00, 0x04, 0xc0, 0x30, 0x00, 0x00, 0x9e, 0x77, + 0xb9, 0x0b, 0x00, 0x32, 0xb9, 0x02, 0x00, 0x33, 0xa7, 0x84, 0x00, 0x19, + 0xa7, 0x3b, 0xff, 0xff, 0xeb, 0x43, 0x00, 0x08, 0x00, 0x0c, 0xb9, 0x02, + 0x00, 0x44, 0xb9, 0x04, 0x00, 0x12, 0xa7, 0x84, 0x00, 0x09, 0xd7, 0xff, + 0x10, 0x00, 0x10, 0x00, 0x41, 0x10, 0x11, 0x00, 0xa7, 0x47, 0xff, 0xfb, + 0xc0, 0x20, 0x00, 0x00, 0x00, 0x0d, 0x44, 0x30, 0x20, 0x00, 0xc0, 0x20, + 0x00, 0x00, 0x00, 0x57, 0xd2, 0x0f, 0x01, 0xd0, 0x20, 0x00, 0xa7, 0xf4, + 0xff, 0x89, 0xd7, 0x00, 0x10, 0x00, 0x10, 0x00, 0xc0, 0x10, 0x00, 0x00, + 0x00, 0x4c, 0xb2, 0xb2, 0x10, 0x00, 0xa7, 0xf4, 0x00, 0x00, 0xeb, 0x00, + 0xf0, 0x00, 0x00, 0x25, 0x96, 0x02, 0xf0, 0x06, 0xeb, 0x00, 0xf0, 0x00, + 0x00, 0x2f, 0xc0, 0x10, 0x00, 0x00, 0x00, 0x2a, 0xe3, 0x10, 0x01, 0xb8, + 0x00, 0x24, 0xc0, 0x10, 0x00, 0x00, 0x00, 0x47, 0xd2, 0x07, 0x01, 0xb0, + 0x10, 0x00, 0xc0, 0x10, 0x00, 0x00, 0x00, 0x39, 0xb2, 0xb2, 0x10, 0x00, + 0xeb, 0x66, 0xf0, 0x00, 0x00, 0x25, 0x96, 0xff, 0xf0, 0x04, 0xeb, 0x66, + 0xf0, 0x00, 0x00, 0x2f, 0xc0, 0x10, 0x00, 0x00, 0x00, 0x1a, 0xe3, 0x10, + 0x01, 0xf8, 0x00, 0x24, 0xc0, 0x10, 0x00, 0x00, 0x00, 0x32, 0xd2, 0x07, + 0x01, 0xf0, 0x10, 0x00, 0xc0, 0x10, 0x00, 0x00, 0x00, 0x20, 0xb2, 0xb2, + 0x10, 0x00, 0xeb, 0x00, 0xf0, 0x00, 0x00, 0x25, 0x94, 0xfd, 0xf0, 0x06, + 0xeb, 0x00, 0xf0, 0x00, 0x00, 0x2f, 0x07, 0xfe, 0xeb, 0x66, 0xf0, 0x00, + 0x00, 0x25, 0x94, 0x00, 0xf0, 0x04, 0xeb, 0x66, 0xf0, 0x00, 0x00, 0x2f, + 0x07, 0xfe, 0x07, 0x07, 0x00, 0x02, 0x00, 0x01, 0x80, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x02, 0x00, 0x01, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x03, 0x02, 0x00, 0x01, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x80, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x01, 0x80, 0x00, 0x00, 0x00, 0xeb, 0xbf, 0xf0, 0x58, - 0x00, 0x24, 0xc0, 0x10, 0x00, 0x00, 0x4e, 0x0d, 0xa7, 0xfb, 0xff, 0x60, - 0xb2, 0x20, 0x00, 0x21, 0xb2, 0x22, 0x00, 0xb0, 0x88, 0xb0, 0x00, 0x1c, - 0xc0, 0xe5, 0xff, 0xff, 0xff, 0x91, 0xa7, 0xbe, 0x00, 0x03, 0xa7, 0x84, - 0x00, 0x13, 0xa7, 0xbe, 0x00, 0x02, 0xa7, 0x28, 0x00, 0x00, 0xa7, 0x74, - 0x00, 0x04, 0xa7, 0x28, 0xff, 0xfe, 0xe3, 0x40, 0xf1, 0x10, 0x00, 0x04, - 0xb9, 0x14, 0x00, 0x22, 0xeb, 0xbf, 0xf0, 0xf8, 0x00, 0x04, 0x07, 0xf4, - 0xa7, 0x28, 0xff, 0xff, 0xa7, 0xf4, 0xff, 0xf5, 0x07, 0x07, 0x07, 0x07, - 0xeb, 0xbf, 0xf0, 0x58, 0x00, 0x24, 0xc0, 0xd0, 0x00, 0x00, 0x01, 0x25, - 0xa7, 0xfb, 0xff, 0x60, 0xa7, 0xb9, 0x00, 0x00, 0xa7, 0x19, 0x00, 0x00, - 0xc0, 0x40, 0x00, 0x00, 0x4d, 0xd8, 0xa7, 0x3b, 0x00, 0x01, 0xa7, 0x37, - 0x00, 0x23, 0xc0, 0x20, 0x00, 0x00, 0x4d, 0xd1, 0x18, 0x31, 0xa7, 0x1a, - 0x00, 0x06, 0x40, 0x10, 0x20, 0x08, 0xa7, 0x3a, 0x00, 0x0e, 0xa7, 0x18, - 0x1a, 0x00, 0x40, 0x30, 0x20, 0x00, 0x92, 0x00, 0x20, 0x02, 0x40, 0x10, - 0x20, 0x0a, 0xe3, 0x20, 0xd0, 0x00, 0x00, 0x04, 0xc0, 0xe5, 0xff, 0xff, - 0xff, 0xac, 0xe3, 0x40, 0xf1, 0x10, 0x00, 0x04, 0xb9, 0x04, 0x00, 0x2b, - 0xeb, 0xbf, 0xf0, 0xf8, 0x00, 0x04, 0x07, 0xf4, 0xb9, 0x04, 0x00, 0x51, - 0xa7, 0x5b, 0x00, 0x01, 0xa7, 0x09, 0x0f, 0xf7, 0xb9, 0x21, 0x00, 0x50, - 0xa7, 0x24, 0xff, 0xd7, 0x41, 0xeb, 0x20, 0x00, 0x95, 0x0a, 0xe0, 0x00, - 0xa7, 0x74, 0x00, 0x08, 0x41, 0x11, 0x40, 0x0e, 0x92, 0x0d, 0x10, 0x00, - 0xb9, 0x04, 0x00, 0x15, 0x43, 0x5b, 0x20, 0x00, 0x42, 0x51, 0x40, 0x0e, - 0xa7, 0xbb, 0x00, 0x01, 0x41, 0x10, 0x10, 0x01, 0xa7, 0xf4, 0xff, 0xbf, - 0xc0, 0x50, 0x00, 0x00, 0x00, 0xd8, 0xc0, 0x10, 0x00, 0x00, 0x4d, 0x8d, - 0xa7, 0x48, 0x00, 0x1c, 0x40, 0x40, 0x10, 0x00, 0x50, 0x20, 0x10, 0x0c, - 0xa7, 0x48, 0x00, 0x04, 0xe3, 0x20, 0x50, 0x00, 0x00, 0x04, 0x40, 0x40, - 0x10, 0x0a, 0x50, 0x30, 0x10, 0x10, 0xc0, 0xf4, 0xff, 0xff, 0xff, 0x6b, - 0xa7, 0x39, 0x00, 0x40, 0xa7, 0x29, 0x00, 0x00, 0xc0, 0xf4, 0xff, 0xff, - 0xff, 0xe4, 0x07, 0x07, 0xb9, 0x04, 0x00, 0x13, 0xa7, 0x2a, 0xff, 0xff, - 0xb9, 0x04, 0x00, 0x34, 0xa7, 0x48, 0x00, 0x01, 0x15, 0x24, 0xa7, 0x24, - 0x00, 0x07, 0xb9, 0x04, 0x00, 0x21, 0xc0, 0xf4, 0xff, 0xff, 0xff, 0x7f, - 0xa7, 0x29, 0xff, 0xff, 0x07, 0xfe, 0x07, 0x07, 0xa7, 0x39, 0x00, 0x00, - 0x41, 0x13, 0x20, 0x00, 0x95, 0x00, 0x10, 0x00, 0xa7, 0x74, 0x00, 0x05, - 0xc0, 0xf4, 0xff, 0xff, 0xff, 0x70, 0xa7, 0x3b, 0x00, 0x01, 0xa7, 0xf4, - 0xff, 0xf5, 0x07, 0x07, 0xeb, 0xbf, 0xf0, 0x58, 0x00, 0x24, 0xc0, 0xd0, - 0x00, 0x00, 0x00, 0x95, 0xa7, 0xfb, 0xff, 0x60, 0xb9, 0x04, 0x00, 0xb2, - 0xa7, 0x19, 0x00, 0x20, 0xc0, 0x20, 0x00, 0x00, 0x4d, 0x40, 0x92, 0x00, - 0x20, 0x00, 0xa7, 0x2b, 0x00, 0x01, 0xa7, 0x17, 0xff, 0xfc, 0xc0, 0x10, - 0x00, 0x00, 0x4d, 0x37, 0xa7, 0x28, 0x10, 0x00, 0x40, 0x20, 0x10, 0x00, - 0xe3, 0x20, 0xd0, 0x00, 0x00, 0x04, 0xc0, 0xe5, 0xff, 0xff, 0xff, 0x1d, - 0x12, 0x22, 0xa7, 0x74, 0x00, 0x19, 0xa7, 0x19, 0x00, 0x00, 0xc0, 0x40, - 0x00, 0x00, 0x00, 0x79, 0xa7, 0x39, 0x00, 0x08, 0xc0, 0x20, 0x00, 0x00, - 0x4d, 0x2c, 0x41, 0x21, 0x20, 0x00, 0xe3, 0x20, 0x20, 0x00, 0x00, 0x90, - 0x43, 0x22, 0x40, 0x00, 0x42, 0x21, 0xb0, 0x00, 0xa7, 0x1b, 0x00, 0x01, - 0xa7, 0x37, 0xff, 0xf2, 0xe3, 0x40, 0xf1, 0x10, 0x00, 0x04, 0xeb, 0xbf, - 0xf0, 0xf8, 0x00, 0x04, 0x07, 0xf4, 0x07, 0x07, 0xeb, 0xaf, 0xf0, 0x50, - 0x00, 0x24, 0xc0, 0xd0, 0x00, 0x00, 0x00, 0x55, 0xa7, 0xfb, 0xff, 0x60, - 0xa7, 0x19, 0x0f, 0xf8, 0xb9, 0x21, 0x00, 0x31, 0xb9, 0x04, 0x00, 0xa2, - 0xa7, 0xc4, 0x00, 0x2a, 0xa7, 0xb9, 0x0f, 0xf8, 0xc0, 0x10, 0x00, 0x00, - 0x4c, 0xf6, 0xa7, 0x28, 0x10, 0x00, 0x40, 0x20, 0x10, 0x00, 0x92, 0x00, - 0x10, 0x02, 0xe3, 0x20, 0xd0, 0x00, 0x00, 0x04, 0xc0, 0xe5, 0xff, 0xff, - 0xfe, 0xda, 0xa7, 0xbb, 0x00, 0x01, 0xa7, 0x19, 0x00, 0x00, 0xa7, 0xb7, - 0x00, 0x17, 0xc0, 0x10, 0x00, 0x00, 0x4c, 0xe1, 0xe3, 0x40, 0xf1, 0x10, - 0x00, 0x04, 0xe3, 0x20, 0x10, 0x08, 0x00, 0x91, 0xa7, 0x2a, 0xff, 0xf9, - 0xb9, 0x14, 0x00, 0x22, 0xeb, 0xaf, 0xf0, 0xf0, 0x00, 0x04, 0x07, 0xf4, - 0xb9, 0x04, 0x00, 0xb3, 0xa7, 0xf4, 0xff, 0xd8, 0xc0, 0x20, 0x00, 0x00, - 0x4c, 0xcc, 0x41, 0x31, 0xa0, 0x00, 0x41, 0x21, 0x20, 0x00, 0xa7, 0x1b, - 0x00, 0x01, 0xd2, 0x00, 0x30, 0x00, 0x20, 0x0f, 0xa7, 0xf4, 0xff, 0xdd, - 0x07, 0x07, 0x07, 0x07, 0x00, 0x00, 0x00, 0x00, 0x00, 0x76, 0x00, 0x05, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x78, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x02, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x77, 0x00, 0x05, - 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x20, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x3c, 0x28, 0x2b, 0x7c, 0x26, 0x2e, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x21, 0x24, 0x2a, 0x29, 0x3b, 0x2e, - 0x2d, 0x2f, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2c, - 0x25, 0x5f, 0x3e, 0x3f, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x60, 0x3a, 0x23, 0x40, 0x27, 0x3d, 0x22, 0x2e, 0x61, 0x62, 0x63, - 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x6a, 0x6b, 0x6c, 0x6d, 0x6e, 0x6f, 0x70, 0x71, 0x72, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, - 0x79, 0x7a, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x2e, 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x4a, 0x4b, 0x4c, 0x4d, 0x4e, 0x4f, 0x50, - 0x51, 0x52, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x53, 0x54, - 0x55, 0x56, 0x57, 0x58, 0x59, 0x5a, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, 0x2e, - 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x2e, 0x2e, - 0x2e, 0x2e, 0x2e, 0x2e, 0x41, 0x00, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x6f, 0xff, 0xfe, 0xf5, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xd8, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x02, 0x28, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xf8, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0b, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x15, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x30, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x09, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x16, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1e, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x00, 0x6f, 0xff, 0xff, 0xfb, - 0x00, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, - 0x6f, 0xff, 0xff, 0xf9, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x01, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x80, 0x00, 0x00, 0x00, 0x07, 0xfe, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, + 0xc0, 0x50, 0x00, 0x00, 0x00, 0x20, 0xc0, 0x10, 0x00, 0x00, 0x95, 0xe1, + 0x42, 0x20, 0x10, 0x0e, 0xa7, 0x28, 0x00, 0x0f, 0x40, 0x20, 0x10, 0x00, + 0x58, 0x20, 0x50, 0x04, 0x50, 0x20, 0x10, 0x08, 0x92, 0x00, 0x10, 0x02, + 0x58, 0x20, 0x50, 0x00, 0xb2, 0x20, 0x00, 0x21, 0xb2, 0x22, 0x00, 0x30, + 0x88, 0x30, 0x00, 0x1c, 0xc0, 0xf4, 0xff, 0xff, 0xff, 0x85, 0x07, 0x07, + 0x07, 0x07, 0x07, 0x07, 0x00, 0x76, 0x00, 0x05, 0x00, 0x07, 0x1a, 0x00, + 0x00, 0x78, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, @@ -193,87 +109,340 @@ unsigned char s390x_elf[] =3D { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x17, 0xb8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x47, 0x43, 0x43, 0x3a, - 0x20, 0x28, 0x55, 0x62, 0x75, 0x6e, 0x74, 0x75, 0x20, 0x31, 0x31, 0x2e, - 0x34, 0x2e, 0x30, 0x2d, 0x31, 0x75, 0x62, 0x75, 0x6e, 0x74, 0x75, 0x31, - 0x7e, 0x32, 0x32, 0x2e, 0x30, 0x34, 0x29, 0x20, 0x31, 0x31, 0x2e, 0x34, - 0x2e, 0x30, 0x00, 0x00, 0x2e, 0x73, 0x68, 0x73, 0x74, 0x72, 0x74, 0x61, - 0x62, 0x00, 0x2e, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x70, 0x00, 0x2e, 0x67, - 0x6e, 0x75, 0x2e, 0x68, 0x61, 0x73, 0x68, 0x00, 0x2e, 0x64, 0x79, 0x6e, - 0x73, 0x79, 0x6d, 0x00, 0x2e, 0x64, 0x79, 0x6e, 0x73, 0x74, 0x72, 0x00, - 0x2e, 0x72, 0x65, 0x6c, 0x61, 0x2e, 0x64, 0x79, 0x6e, 0x00, 0x2e, 0x74, - 0x65, 0x78, 0x74, 0x00, 0x2e, 0x72, 0x6f, 0x64, 0x61, 0x74, 0x61, 0x00, - 0x2e, 0x64, 0x79, 0x6e, 0x61, 0x6d, 0x69, 0x63, 0x00, 0x2e, 0x67, 0x6f, - 0x74, 0x00, 0x2e, 0x62, 0x73, 0x73, 0x00, 0x2e, 0x63, 0x6f, 0x6d, 0x6d, - 0x65, 0x6e, 0x74, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x0b, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xc8, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xc8, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x0f, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xd8, + 0x00, 0x00, 0x00, 0x00, 0x6f, 0xff, 0xfe, 0xf5, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x50, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x20, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0b, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x15, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x02, 0x58, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x09, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, + 0x00, 0x00, 0x00, 0x00, 0x6f, 0xff, 0xff, 0xfb, 0x00, 0x00, 0x00, 0x00, + 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x6f, 0xff, 0xff, 0xf9, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x13, 0x6f, 0xff, 0xff, 0xf6, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1e, 0xd8, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x30, 0x00, + 0x47, 0x43, 0x43, 0x3a, 0x20, 0x28, 0x53, 0x55, 0x53, 0x45, 0x20, 0x4c, + 0x69, 0x6e, 0x75, 0x78, 0x29, 0x20, 0x31, 0x32, 0x2e, 0x33, 0x2e, 0x30, + 0x00, 0x00, 0x2e, 0x73, 0x68, 0x73, 0x74, 0x72, 0x74, 0x61, 0x62, 0x00, + 0x2e, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x70, 0x00, 0x2e, 0x67, 0x6e, 0x75, + 0x2e, 0x68, 0x61, 0x73, 0x68, 0x00, 0x2e, 0x64, 0x79, 0x6e, 0x73, 0x79, + 0x6d, 0x00, 0x2e, 0x64, 0x79, 0x6e, 0x73, 0x74, 0x72, 0x00, 0x2e, 0x72, + 0x65, 0x6c, 0x61, 0x2e, 0x64, 0x79, 0x6e, 0x00, 0x2e, 0x74, 0x65, 0x78, + 0x74, 0x00, 0x2e, 0x72, 0x6f, 0x64, 0x61, 0x74, 0x61, 0x00, 0x2e, 0x65, + 0x68, 0x5f, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x00, 0x2e, 0x64, 0x79, 0x6e, + 0x61, 0x6d, 0x69, 0x63, 0x00, 0x2e, 0x67, 0x6f, 0x74, 0x00, 0x2e, 0x64, + 0x61, 0x74, 0x61, 0x00, 0x2e, 0x62, 0x73, 0x73, 0x00, 0x2e, 0x63, 0x6f, + 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0b, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x01, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xd8, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x00, 0x03, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1d, - 0x00, 0x00, 0x00, 0x0b, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xf8, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x01, 0xf8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x30, - 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, - 0x00, 0x00, 0x00, 0x25, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x28, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x28, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x2d, 0x00, 0x00, 0x00, 0x04, + 0x00, 0x00, 0x01, 0xc8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xc8, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0f, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x17, + 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xd8, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x01, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x28, + 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, + 0x00, 0x00, 0x00, 0x13, 0x6f, 0xff, 0xff, 0xf6, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1d, 0x00, 0x00, 0x00, 0x0b, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x02, 0x30, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x30, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x03, + 0x00, 0x00, 0x02, 0x20, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x20, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x30, 0x00, 0x00, 0x00, 0x05, + 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x25, + 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x50, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x02, 0x50, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x2d, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x58, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x58, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x37, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x02, 0x70, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x70, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x37, - 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x48, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x02, 0x48, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x40, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3d, + 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x78, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x04, 0x78, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x88, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x88, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x01, 0x24, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x45, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x88, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x88, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x45, 0x00, 0x00, 0x00, 0x06, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x4f, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x17, 0xb8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x07, 0xb8, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x20, 0x00, 0x00, 0x00, 0x04, + 0x00, 0x00, 0x1e, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0e, 0xd8, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x10, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x4e, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x58, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0xd8, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x08, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1f, 0xe8, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x0f, 0xe8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x53, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, + 0x00, 0x00, 0x00, 0x5d, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0xf0, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x01, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x58, 0x00, 0x00, 0x00, 0x01, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x30, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0xf0, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x2b, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, - 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x63, 0x00, 0x00, 0x00, 0x08, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x30, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x08, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x68, + 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x30, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x10, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x19, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x09, 0x1b, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x61, + 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x21, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x71, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00 }; diff --git a/tests/qtest/migration/s390x/a-b-bios.c b/tests/qtest/migration= /s390x/a-b-bios.c index ff99a3ef57..3999254dde 100644 --- a/tests/qtest/migration/s390x/a-b-bios.c +++ b/tests/qtest/migration/s390x/a-b-bios.c @@ -9,23 +9,75 @@ * option) any later version. */ =20 -#define LOADPARM_LEN 8 /* Needed for sclp.h */ +/* for sclp.h */ +#define LOADPARM_LEN 8 +typedef unsigned int uint32_t; +typedef unsigned short uint16_t; +typedef unsigned char uint8_t; =20 -#include -#include #include =20 -char stack[0x8000] __attribute__((aligned(4096))); - #define START_ADDRESS (1024 * 1024) #define END_ADDRESS (100 * 1024 * 1024) =20 +/* at pc-bios/s390-ccw/start.S */ + +void __attribute__((weak)) consume_sclp_int(void) +{ +} + +static char _sccb[4096] __attribute__((__aligned__(4096))); +char stack[0x8000] __attribute__((aligned(4096))); + +static void sclp_service_call(unsigned int command) +{ + int cc; + + asm volatile( + " .insn rre,0xb2200000,%1,%2\n" /* servc %1,%2 */ + " ipm %0\n" + " srl %0,28" + : "=3D&d" (cc) : "d" (command), "a" (__pa(_sccb)) + : "cc", "memory"); + consume_sclp_int(); +} + +static void sclp_setup(void) +{ + WriteEventMask *sccb =3D (void *)_sccb; + + sccb->h.length =3D sizeof(WriteEventMask); + + sccb->mask_length =3D sizeof(unsigned int); + sccb->cp_receive_mask =3D 0; + sccb->cp_send_mask =3D SCLP_EVENT_MASK_MSG_ASCII; + + sclp_service_call(SCLP_CMD_WRITE_EVENT_MASK); +} + +static void putc(const char c) +{ + WriteEventData *sccb =3D (void *)_sccb; + int len =3D 1; + + sccb->data[0] =3D c; + + sccb->h.length =3D sizeof(WriteEventData) + len; + sccb->h.function_code =3D SCLP_FC_NORMAL_WRITE; + + sccb->ebh.length =3D sizeof(EventBufferHeader) + len; + sccb->ebh.type =3D SCLP_EVENT_ASCII_CONSOLE_DATA; + sccb->ebh.flags =3D 0; + + sclp_service_call(SCLP_CMD_WRITE_EVENT_DATA); +} + void main(void) { unsigned long addr; =20 sclp_setup(); - sclp_print("A"); + putc('A'); =20 /* * Make sure all of the pages have consistent contents before incremen= ting @@ -39,6 +91,6 @@ void main(void) for (addr =3D START_ADDRESS; addr < END_ADDRESS; addr +=3D 4096) { *(volatile char *)addr +=3D 1; /* Change pages */ } - sclp_print("B"); + putc('B'); } } diff --git a/tests/qtest/migration/Makefile b/tests/qtest/migration/Makefile index 2c5ee287ec..c183b69941 100644 --- a/tests/qtest/migration/Makefile +++ b/tests/qtest/migration/Makefile @@ -7,7 +7,7 @@ =20 TARGET_LIST =3D i386 aarch64 s390x ppc64 =20 -SRC_PATH =3D ../.. +SRC_PATH =3D ../../../.. =20 .PHONY: help $(TARGET_LIST) help: @@ -23,16 +23,17 @@ help: @echo " Possible targets are: $(TARGET_LIST)" =20 override define __note -/* This file is automatically generated from the assembly file in - * tests/migration/$@. Edit that file and then run "make all" - * inside tests/migration to update, and then remember to send both +/* + * This file is automatically generated from the assembly file in + * tests/qtest/migration/$@. Edit that file and then run "make" + * inside tests/qtest/migration to update, and then remember to send both * the header and the assembler differences in your patch submission. */ endef export __note =20 $(TARGET_LIST): - $(MAKE) CROSS_PREFIX=3D$(CROSS_PREFIX) -C $@ + $(MAKE) SRC_PATH=3D$(SRC_PATH) CROSS_PREFIX=3D$(CROSS_PREFIX) -C $@ =20 clean: for target in $(TARGET_LIST); do \ diff --git a/tests/qtest/migration/aarch64/Makefile b/tests/qtest/migration= /aarch64/Makefile index 9c4fa18e76..fd102ab776 100644 --- a/tests/qtest/migration/aarch64/Makefile +++ b/tests/qtest/migration/aarch64/Makefile @@ -6,7 +6,7 @@ all: a-b-kernel.h =20 a-b-kernel.h: aarch64.kernel echo "$$__note" > $@ - xxd -i $< | sed -e 's/.*int.*//' >> $@ + xxd -i $< | sed -e '/.*int.*/d' >> $@ =20 aarch64.kernel: aarch64.elf $(CROSS_PREFIX)objcopy -O binary $< $@ @@ -15,4 +15,4 @@ aarch64.elf: a-b-kernel.S $(CROSS_PREFIX)gcc -o $@ -nostdlib -Wl,--build-id=3Dnone $< =20 clean: - $(RM) *.kernel *.elf + $(RM) *.kernel *.elf *.h diff --git a/tests/qtest/migration/aarch64/a-b-kernel.S b/tests/qtest/migra= tion/aarch64/a-b-kernel.S index a4103ecb71..8f29efbb4a 100644 --- a/tests/qtest/migration/aarch64/a-b-kernel.S +++ b/tests/qtest/migration/aarch64/a-b-kernel.S @@ -11,7 +11,7 @@ # pc-relative address. Also the branch instructions should use relative # addresses only. =20 -#include "../migration-test.h" +#include "../bootfile.h" =20 .section .text =20 diff --git a/tests/qtest/migration/i386/Makefile b/tests/qtest/migration/i3= 86/Makefile index 37a72ae353..cbf63103af 100644 --- a/tests/qtest/migration/i386/Makefile +++ b/tests/qtest/migration/i386/Makefile @@ -6,7 +6,7 @@ all: a-b-bootblock.h =20 a-b-bootblock.h: x86.bootsect x86.o echo "$$__note" > header.tmp - xxd -i $< | sed -e 's/.*int.*//' >> header.tmp + xxd -i $< | sed -e '/.*int.*/d' >> header.tmp nm x86.o | awk '{print "#define SYM_"$$3" 0x"$$1}' >> header.tmp mv header.tmp $@ =20 @@ -20,4 +20,4 @@ x86.o: a-b-bootblock.S $(CROSS_PREFIX)gcc -I.. -m32 -march=3Di486 -c $< -o $@ =20 clean: - @rm -rf *.boot *.o *.bootsect + @rm -rf *.boot *.o *.bootsect *.h diff --git a/tests/qtest/migration/i386/a-b-bootblock.S b/tests/qtest/migra= tion/i386/a-b-bootblock.S index 6f39eb6051..99253f5ee5 100644 --- a/tests/qtest/migration/i386/a-b-bootblock.S +++ b/tests/qtest/migration/i386/a-b-bootblock.S @@ -9,7 +9,7 @@ # # Author: dgilbert@redhat.com =20 -#include "migration-test.h" +#include "../bootfile.h" =20 #define ACPI_ENABLE 0xf1 #define ACPI_PORT_SMI_CMD 0xb2 diff --git a/tests/qtest/migration/ppc64/Makefile b/tests/qtest/migration/p= pc64/Makefile index a3a2d98ac8..1bd6d35799 100644 --- a/tests/qtest/migration/ppc64/Makefile +++ b/tests/qtest/migration/ppc64/Makefile @@ -3,13 +3,13 @@ all: a-b-kernel.h =20 a-b-kernel.h: ppc64.kernel echo "$$__note" > $@ - xxd -i $< | sed -e 's/.*int.*//' >> $@ + xxd -i $< | sed -e '/.*int.*/d' >> $@ =20 ppc64.kernel: ppc64.elf $(CROSS_PREFIX)objcopy -O binary -S $< $@ =20 ppc64.elf: a-b-kernel.S - $(CROSS_PREFIX)gcc -static -o $@ -nostdlib -Wl,--build-id=3Dnone $< + $(CROSS_PREFIX)gcc -mbig-endian -static -o $@ -nostdlib -Wl,--build-id=3D= none $< =20 clean: - $(RM) *.kernel *.elf + $(RM) *.kernel *.elf *.h diff --git a/tests/qtest/migration/ppc64/a-b-kernel.S b/tests/qtest/migrati= on/ppc64/a-b-kernel.S index 0613a8d18e..c0009ecd10 100644 --- a/tests/qtest/migration/ppc64/a-b-kernel.S +++ b/tests/qtest/migration/ppc64/a-b-kernel.S @@ -4,7 +4,7 @@ # This work is licensed under the terms of the GNU GPL, version 2 or later. # See the COPYING file in the top-level directory. =20 -#include "../migration-test.h" +#include "../bootfile.h" =20 .section .text =20 diff --git a/tests/qtest/migration/s390x/Makefile b/tests/qtest/migration/s= 390x/Makefile index 6671de2efc..a2399821a7 100644 --- a/tests/qtest/migration/s390x/Makefile +++ b/tests/qtest/migration/s390x/Makefile @@ -3,7 +3,7 @@ =20 .PHONY: all clean all: a-b-bios.h -fwdir=3D../../../pc-bios/s390-ccw +fwdir=3D$(SRC_PATH)/pc-bios/s390-ccw =20 CFLAGS+=3D-ffreestanding -fno-delete-null-pointer-checks -fPIE -Os \ -msoft-float -march=3Dz900 -fno-asynchronous-unwind-tables \ @@ -11,14 +11,14 @@ CFLAGS+=3D-ffreestanding -fno-delete-null-pointer-check= s -fPIE -Os \ =20 a-b-bios.h: s390x.elf echo "$$__note" > header.tmp - xxd -i $< | sed -e 's/.*int.*//' >> header.tmp + xxd -i $< | sed -e '/.*int.*/d' >> header.tmp mv header.tmp $@ =20 # We use common-page-size=3D16 to avoid big padding in the ELF file s390x.elf: a-b-bios.c $(CROSS_PREFIX)gcc $(CFLAGS) -I$(fwdir) $(fwdir)/start.S \ - $(fwdir)/sclp.c -Wl,-zcommon-page-size=3D16 -o $@ $< + -Wl,-zcommon-page-size=3D16 -o $@ $< $(CROSS_PREFIX)strip $@ =20 clean: - @rm -rf *.elf *.o + @rm -rf *.elf *.o *.h --=20 2.53.0